Entropy of Music

Entropy can be characterized as unpredictability. Sure, a totally chaotic mess would be more unpredictable, but I would expect a 13 member jazz band to be more unpredictable than many other examples of music.

--
Tim

Yes, he meant its complexity, relative to pop.

Claude Shannon estimated music at 40 bits/sec entropy. What does MP3 achieve?

-- Rich

It's an apples and oranges comparison: Claude there is talking about how many bits of information the musicians need to know what to play, but it's assumed they already know how to proficiently play their instruments as part of an orchestra, which is itself a MASSIVE amount of information. MP3 has nowhere near as much already-known ("dictionary") information. Indeed, about the only thing that MP3 "knows" at all is how the human ear behaves in terms of masking behavior, and hence has a pretty good idea as to what of the "raw audio bits" it takes in can be safely thrown away without anyone noticing too much. (...and some early MP3 encoder didn't even know much about that... :-) )

A better comparison would be with a speech encoder such as Speex: Speech encoders typically model the physics of the human vocal tract, and therefore can often get away with upwards of an order of magnitude less bandwidth (bits/sec) for a given quality of speech relative to something like MP3. ...but they tend to fall apart completely if asked to reproduce something other than human speech, such as music (...just as, e.g., a flute can never sounds like a saxophone...).

---Joel

No, he wasn't. Evidently you don't understand Shannon entropy.

Then the speech playback decoder generates sound through a model of the human vocal tract? Very impressive, if true -

-- Rich

MP3 bitrate is a tunable parameter, though it doesn't go as low as

40 bits/s. In fact MP3 typically has about 40 frames per second, and a frame header requires a lot more than 1 bit.

However, the format encodes a *lot* more than a musical score. Every note from an instrument in a performance is different, and people can hear many of those differences. A performance compressed to 40 bit/s would be much more regular and "mechanical" than any real performance.

Still, it is true that MP3 is poor at compressing very similar sounds that are temporally separated. Frames are mostly independent of one another, so that even most short-range temporal structure is completely ignored.

--
Tim

Shannon did quite a lot of work on entropy of music. Which particular work are you citing with your quote of 40 bits/s?

A mathematical model, yes. Such models do usually have more free parameters than any human can achieve, but close enough to allow very substantial compression ratios.

--
Tim