Often such issues are related to DNS config or your local wifi.
That said, ever since Monterey and my new 14" MBP, I’ve been seeing such an issue with Safari. And I know it’s neither DNS nor a specific wifi. I can replicate it on a vanilla Monterey install. I observe it on various wifi networks including both at work (campus wide wifi, non-Apple, and usually very good) as well as at home (latest AirPort Extreme hooked up to Gigabit fiber, excellent performance before Monterey with both Intel and M1-based Macs, incl. two 2020 models). The reason I also know it’s a Safari issue rather than something on the network is that I usually can get Safari to start loading the page, if I launch any other browser on my Mac and have it open the page Safari is struggling with (the other browser so far never struggled loading that page). A Safari restart will also often (but not always) fix the issue. The issue shows sporadicly, it’s hard to pin point exactly when or with which pages it occurs—although there’s one specific work site I use that often suffers from the issue. It must nevertheless have something to do with wifi though because it never seems to show up when I’m connected through Gigabit (when working at the desk, hooked up to a TB4 dock equipped with a USBC-Gigabit adapter), it only shows when connected over wifi exclusively.
I had hopes that 12.1 might fix it, but no mention in the release notes and so far from what I can tell no difference. :(