>And yet here we are, able to talk to a computer, that writes Pytorch code that orchestrates the complexity below it.

It writes something that that's almost, but not quite entirely unlike Pytorch. You're putting a little too much value on a simulacrum of a programmer.