I have never had the kind of trouble with MA you mention and I have relied on it a lot. Essentially I’ve migrated to every Mac this way for the past 15+ years. And I usually change my main work Mac at least once a year.
That said, I also feel like direct Mac-to-Mac connections in MA have become overly fussy. I would expect from Apple that they take every possible connection you happen to try, from wifi to Gigabit, FW (while it’s still here), USB, and TB, and auto-configure that for MA use such that it either squeezes the maximum possible performance out of it, or it alerts you to potential problems and suggests alternatives. It should also always take the fastest: sure wifi is always there, but why use that when you can instead take the 3 GB/s TB4? More recently, I have found the system to have become less transparent, probably in an attempt at suggesting simplicity, but therefore making it a bit hit and miss.
I’m starting to think I should probably just stick to using MA with the last TM backup of the source system (which will be up to date) rather than try to peer-2-peer the two Macs. A bit of a shame. This shouldn’t be complicated.