This ignores the fact that most people use the same password everywhere, given the opportunity, and you have no idea what website has been breached. I.e. if you don't expire passwords, most people will use the same password everywhere, and you don't know when a compromise has happened, because it happened on some totally other network.
This doesn't need to be solved via time-based expiry though - you can use a breach list (like checking https://haveibeenpwned.com on registration,login, and password change).
Even an aggressive password change policy is typically one month since last change, which would give a long window for access.
Secondary factors, if at no other time then on first use of a machine, are also a good technique to prevent password breaches from spreading into your system.
List-checking can't help every time. Some people use "clever" tactics to use slightly different passwords (hunter2fb for Facebook, hunter2tw for Twitter).
