I don't think this can be definitively answered without working for one of the hyperscalers. But here are some speculations:
1. Device speeds are intentionally capped to increase device lifetime (but this would only make sense for writes)
2. Networked storage services like EBS are more profitable, and AWS would like to phase out instance-attached storage.
3. Technical limitations/virtualization overhead (See comment above). I don’t have enough insight of how AWS SSDs work under the hood, but high network throughput (600 Gbit/s) is possible even in virtualized instances. Then again, we have certainly seen some weird noisy neighbor effects on cloud SSDs. However, it's worth mentioning that the same throughput limitations also apply to bare metal instances, where users don't benefit from virtualization (https://docs.aws.amazon.com/ec2/latest/instancetypes/so.html...).
4. There’s too little customer demand for fast SSDs, and optimization is not worth the effort.
Speculating: local ssds aren't as valuable in the cloud since they're effectively ephemeral. If the instance restarts, it would lose its storage. Trying to keep a workload affinitized to an SSD or to migrate data to a different SSD when an instance moves increases cost prohibitively.
For a lot of use cases such as caching (e.g., the ephemeral caching layer in Snowflake), ephemeral storage is good enough. If you really want to, you could also achieve persistence by replicating to multiple instances (afaik this is what DynamoDB does)
Thats difficult for most people to implement in their applciations, and it increases latency to be closer to networked SSD anyway. So it remains fairly niche.
I think that number 4 is the big one. AWS only has so much capacity to work on new hardware types, and the number of companies who want to work with on-device NVMe are WAY smaller than then number of companies who just want to slap Kubernetes on some instances with EBS.
