I sort of assume the main benefit of assuming no NaNs is being able to replace `a == a` with `true`?
The handling of isnan is certainly a big question. I can see wanting to respect that, but I can also see littering your code with assertions that isnan is false and compiling with normal optimizations and then hoping that later recompiling with fast-math and all its attendant "fun, safe optimizations" will let you avoid any performance penalty for all those asserts.
There's also several cases where handling NaNs "correctly" may require a fair amount of extra work (C99's behavior for complex multiplication is far more complicated when NaNs are involved).
The handling of isnan is certainly a big question. I can see wanting to respect that, but I can also see littering your code with assertions that isnan is false and compiling with normal optimizations and then hoping that later recompiling with fast-math and all its attendant "fun, safe optimizations" will let you avoid any performance penalty for all those asserts.