I tried this approach and the latency was through the roof, easily a second or so off, making it effectively useless for video conferencing. Switched to an HDMI grabber so I could get something that was actually realtime.
You need to dial in the fps. It's a little bit of work to try different parameters and might not work for every camera. But with most cameras I was able to get the latency below 300ms which should work for most use cases.
