AirPlay hand-over rules?

My typical music listening situation is: Music playing via the Music app on my iPhone 13 Pro (iOS 17.4.1), AirPlayed to a pair of original HomePods (also running the latest OS).

Siri, sometimes (being Siri), takes too long to skip a track or change the volume — or maybe I just want to see the album cover or lyrics on my phone — but the phone has become disconnected from what’s playing on the HomePods. It’s displaying a different track, or maybe the same track but its playback-timeline control isn’t moving, and the play/pause/volume controls no longer adjust the HomePods the way they did initially.

I can’t figure out any rules for when this might happen, though. It doesn’t seem to be simply a question of time, or whether the phone’s gone to sleep (it’s usually sitting on a Qi-charging desk lamp).

I’m not even saying I want to change this behaviour. I’d to start by understanding it.