iPhone sync copying all photos every single time

I noticed that every once in a while my iPhone sync takes particularly long, regardless of USB vs. wifi. For almost as long as I can remember there’s been the (so far unsolved) issue of syncing huge log files, but this goes even beyond that. Turns out every time I create a new album in Photos (with a few photos imported from my iPhone’s camera roll via USB), the next iPhone sync will lead to all photos in my entire Photos library being copied to my iPhone. Anybody else ever seen this? iOS 13.newest, Catalina Photos, iPhone photo sync preferences set to sync all photos and albums.