No GCC support if I read that correctly and the effort is expected to take months of full time work. I wonder what that'd break or is Clang significantly more common on OSX now?
Perhaps! It's one of the two candidates that might actually work in the near term. It didn't quite seem to work out of the box and given the newness of the project, we are a bit cautious, but there's some folks looking into it, to see if they can work through the issues.
Not sure if it's changed, but last i checked a few years back, most of the BLAS/LAPACK etc. stuff was f77, which should 'do' with f2c (albeit, perhaps not as fast) and is at least partly why f2c was hosted at netlib..
I had no idea, that explains so much! I was wondering how these open source numerical systems were building fortran code "so well", as in not a huge operational burden. My understanding is that the OSS fortran compilers are used for dev/debug, but when it comes to production, commercial Fortran compilers for your platform vendor offer much better performance.
Would it would work under Rosetta 2 (Apple's x64 emulation/translation layer) until the aarch64/darwin ABI can be ported to GCC? Or is there a reason to assume it might not?
Yes, it works mostly fine under Rosetta. However, it is also precisely the area where you don't really want a translation layer, because you're spending a lot of time doing heavy compute.
There's several! We tried all of them without much luck :(. The most complete seems to be the old flang, which was basically an open sourced version of the PGI compiler hacked up to generate LLVM IR by concating strings to generate textual LLVM IR. However, that project is basically abandoned and also depends on LLVM 7, while you probably want at least LLVM 10 to generate good code for Apple Silicon, so somebody would have to do that porting work, which doesn't seem like a worthwhile investment.
Hopefully apple will do something about it as they did advertise their new system would be great for machine learning and all, which seem to be moot without many popular numeric libraries that directly or indirectly use fortran. Would be great if clang/llvm suddenly gain a great open source fortran compiler just because apple moves to arm.
There was old flang which was basically just the PGI compiler stripped of its backend and with a piece bolted on that converted the compiler's internal IR to LLVM by basically dumping out strings and then parsing them back in. Many people in the LLVM community thought that this was not a maintainable long term strategy for a Fortran frontend, and after much debate, the flang team decided to start over on Greenfield compiler called f18, which was to be architected more like clang. Then Chris released MLIR and said team decided to target that rather than LLVM directly. The only thing that was kept from old flang were some of the runtime libraries. f18 was recently merged into the llvm-project repository whereupon it was renamed flang. Old flang is dead, but the terminology is a bit confusing now. Additionally there are two external LLVM Fortran compiler efforts: lfortran out of the US national labs (which also paid PGI for flang) and fc, which I don't believe is currently open source, but is being developed by as compiler consulting shop on India called CompilerTree (not sure who's footing the bill for that one). Anyway, long story short, none of them seem to work. Old flang generates binaries, but is abandonware at this point, so not the correct foundation to build on. New flang doesn't even generate LLVM IR yet. FC doesn't seem public yet as I said. Lfortran may work, but the authors say it's very alpha and it certainly didn't work out of the box. I know some folks are looking into it to see what would be required.
I wonder why this is allowed to persist? Have there been any studies across benchmark suites relative to compiler and platform? What was the spread? Was this general purpose, it works well enough speed or was this I need to get 1.75% better on task x or my super computer budget is blown?
I find the whole Fortran ecosystem and lineage fascinating for some reason. Like an alternative universe where integers are only used for loop counters.
Apple sucks for scientific computing. It does not let you even load your own generated shared files, because Apple determines it is too dangerous for the health my own purchased computer. I have never had that problem on Windows or Linux. That said, you can get gfortran by as simple as typing brew install gcc-9, or some similar command (requires brew installation).
Sadly, yes. I've clients who still prefer to work on their Mac laptop and desktops. Large runs still go to Linux clusters, but small, debugging runs occur locally.
Per the parent, the lack of Fortran compiler causes an issue because some BLAS/LAPACK libraries align their symbols with whatever a Fortran compiler would do. As such, unless there's been a recent change, things like CMake need the compiler to determine the name for linear algebra routines.
I phrased the above as "sadly, yes" because supporting macos has been a pain and costs a fair amount of labor. In my case, Homebrew is required and there a bunch of specialized quirks in the build and packaging system that I'll contend are more difficult to deal with on macos than on Linux and Windows. Lastly, the requirement that macos be virtualized only on mac hardware complicates automating the build and test system since those processes must be relegated to a macos machine and I'd really rather not do primary development on a mac. Frankly, I'd pay Apple a not small amount of money to virtualize macos legally on non-mac hardware.
BLAS/LAPACK isn't that much of an issue, since Apple probably will need to ship their own implementation of this for Apple Silicon to get reasonable performance, and they can implement this in C, assembly, or whatever they want.
If they don't do that, then you can just use any "generic" C BLAS/LAPACK implementation, performance will be poor, but that's pretty much the only option if your hardware vendor doesn't support BLAS.
I guess people can write their own BLAS as well, but if you want to re-implement BLAS in 2020, then Fortran is probably not the best language to do that.
Apple's copy of LAPACK has not been keeping up with upstream and is missing some of the newer methods and bug fixes. Also while those are certainly the big two dependencies they are by far not the only ones.
I see the opposite from you: Mac laptops, crunching under Linux, very rarely see windows in the wild. I know windows sales are still robust so this is what I see but am not sure how far it can be extrapolated.
Yes. Most people in my lab have a MacBook (Pro) where we do most of our coding, and we ssh into a Linux cluster to run most of the compute. We have desktops but these are all Windows/Linux dual boots. We all practically need to use PowerPoint as well, so Mac is nice to have here since it can’t go on Linux laptops, and nobody wants to constantly reboot all the time.
Walking down the hallway in my physics department (pre-covid lockdown), I see almost all Macs. Some of them run windows, but most are apple hardware and software. Windows is quite rare when it comes to professors. It's more common among the grad students though, about tied with MacOS.
It was like this in my previous department as well.
By Homebrew policy, anything requiring a C/C++ compiler but doesn't compile with clang is out. So gcc not working shouldn't disrupt the overall Homebrew ecosystem too much. The biggest problem is gfortran, which is what the great majority of gcc dependents use it for (just `rg 'depends_on "gcc"'` in homebrew-core and all the "# for gfortran" comments should make this obvious).
Btw, here's the list turned up by a dependent search in core:
Any numpy (which uses gfortran) dependent is on that list. I suspect some could partially work by sacrificing some functionality, or maybe just run slower (disclaimer: I have no idea of numpy internals).
GCC supports ARM targets and the macos platform. It took a couple of hours to add support for the new target triple to Rust, so it should take about the same work to add support to it for each of the GCC frontends (10x couple of hours ~ couple of days, maybe a week).
You don't need to implement neither hardware support nor OS support, just combine the two things that are already implemented, and fix a couple of incompatibilities, but not many (some parts of OS support will be x86 only and these would need fixing, but fixing these should be trivial since you can c&p from the arm+iOS code).
Why would it take months to add support for this to GCC? GCC's architecture would need to be really flawed for this to be true, i.e. , requiring you to re-implement a big chunk of hardware support for each OS, or full OS support for each new hardware. Given how many architectures and OSes GCC supports I just can't imagine this being true.
Unless by this work taking months, you mean "not doing anything the first 99% of the days, and doing all the work in the last couple of days". In that case, sure, this can take as long as you want.
Yeah, I was surprised by that statement too, but then again, I know nothing about how GCC is architected. I'm hoping somebody will come along and just fix it in a few days, but who knows.
