You are right that manually-added album covers appear to be stored in the ID3 tags, saved with the rest of the track’s metadata. And artwork from Apple is in a database, apparently shared between all of the album’s tracks.
You can, from the Music app, copy/paste artwork to an external app (like your favorite paint program). Save it and drag/drop that file back to the track’s info in order to replace Apple’s artwork with your own.
But this may not solve your problem. I’ve found that the iOS Music app seems to have many problems in its database. Nearly all of the tracks in my music collection have artwork that I added manually, but some of it doesn’t appear when played on my iPod Touch. But, interestingly enough, the artwork does appear when I play the track over a USB connection to my car.
I see similar things sometimes with track listings. Sometimes, when browsing the library, iOS will show the “grouping” field from the metadata (which I use for tags like “80’s, synth, guitar, volcal”) instead of the track name and “Unknown” instead of the artist name. But if I actually play the track, the currently-playing screen shows the correct title and artist (the correct info also appear on my car’s stereo when played there.)
In other words, if everything looks good in Music on your Mac, but is messed up on your iOS device, it might just be an iOS bug and nothing you can fix on your own.