On 2014-10-14, Max Little wrote:
Hmm .. don't shoot the messenger! I merely said, it's interesting that you don't actually have to specify the distribution of a random variable to compute the Hartley entropy. No idea if that's useful.
Math always has this precise tradeoff: more general but less informative, versus more specific and conducive to a deeper theory. So, true, Hartley's conception is more general and doesn't need as much input fodder.
Yet as Ethan said, the theory then assumes too little to be able to answer most of the interesting questions. You can't analyze anything noisy under that one. Shannon's theory on the other hand does incorporate probabilistic reasoning from the very start, and does it in the most general way possible. So it leads to deeper and more useful results, and surprisingly, doesn't require much more machinery than you'd have to have starting with Hartley's definition. Moreover, it really does fully contain Hartley's definition as one special case.
That is then why we hail Shannon as the progenitor of information theory. He got the formalism and the viewpoint right, plus did the math, under *very* lax assumptions. So that his viewpoint and framework has guided pretty much everything since then. That's what math proper is about, finding fruitful ways of formalizing hard problems in ways which make them seemingly easy to deal with; Shannon did it better, since his required less legwork even for the general case, without messing up even Hartley's special one.
(And of course nobody ever forgot Hartley, either: the one, most fundamental theorem in continuous time analog coding theory is called the Shannon-Hartley theorem. It's just that while Hartley was a formidable, early information theorist, Shannon got his basic framework *just* that little bit better within the discourse. :) )
-- Sampo Syreeni, aka decoy - de...@iki.fi, http://decoy.iki.fi/front +358-40-3255353, 025E D175 ABE5 027C 9494 EEB0 E090 8BA9 0509 85C2 -- dupswapdrop -- the music-dsp mailing list and website: subscription info, FAQ, source code archive, list archive, book reviews, dsp links http://music.columbia.edu/cmc/music-dsp http://music.columbia.edu/mailman/listinfo/music-dsp
