The main type of metadata for mp3 files is called ID3. The specifications for ID3 are maintained by the ID3 organization and can be found at
http://id3.org. An ID3 tag contains frames which contain the metadata. One of these frames is named "APIC" ("PIC" in version 2) and specifies several types of pictures.
I don't know about Windows, maybe someone can enlighten me, but I do know that MacOS looks at the ID3 tag and extracts the thumbnail, the title and the lead performer from not only mp3 files, but aiff files too. Aiff files are derived from the IFF (Information File Format) format designed by Electronic Arts for the Commodore Amiga. It consists of chunks. One chunk contains the sound data, another contains information such as sample rate, etc. iTunes (as well as other software) can create a chunk which contains an ID3 tag. MacOS will parse this tag just as if it were prepended to an mp3 file.
Here's the thing: wave files are also derived from the IFF format. The first four bytes of every wave file read "RIFF" (Resource Information File Format). Wave files and aiff files are not, in fact, that different from each other. I've written a program which converts from one to the other. Simple. That same program can add and parse an ID3 chunk to a wave file, the same as an aiff.
But iTunes and MacOS refuse to use the ID3 chunk in a wave file, even though they parse the file (they have to in order to play it, which they do). This is the source of the misinformation that wave files can't carry metadata. They can. I repeat: it is misinformation that wave files cannot have ID3 tags.
The problem with storing photos in mp3s, aiffs or, yes, waves is that iTunes and MacOS are terrible at handling ID3 metadata. The specs have no limit on the number of pictures a tag can contain, other than that they have to have unique types and names. Yet iTunes and MacOS will only use the first one encountered. My program can change the order of pictures in an ID3 tag, and I can watch the thumbnail change depending upon which is first. But this means that if you want to store a thumbnail, a head shot, a business card, etc. in the tag you'll need a better metadata editor than iTunes to see them.