On Sun, 29 Oct 2017 02:31 pm, Gregory Ewing wrote: > Steve D'Aprano wrote: >> I don't think that's right. The entropy of a single message is a >> well-defined quantity, formally called the self-information. >> >> https://en.wikipedia.org/wiki/Self-information > > True, but it still depends on knowing (or assuming) the > probability of getting that particular message out of > the set of all possible messages.
Indeed. > This is *not* what danceswithnumbers did when he > calculated the "entropy" of his example bit sequences. > He didn't define the set they were drawn from or > what their probabilities were. I'm not defending or supporting Danceswithnumbers in his confusion. I'm just pointing out that an entropy-like measure of information for individual messages does exist, and people do often call it "entropy". -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list