On Sun, 29 Oct 2017 01:56 pm, Stefan Ram wrote: > If the entropy of an individual message is not defined, > than it is still available to be defined. I define it > to be log2(1/p), where p is the probability of this > message. I also choose a unit for it, which I call "bit".
That is exactly the definition of self-information: https://en.wikipedia.org/wiki/Self-information See also: https://en.wikipedia.org/wiki/Entropy_(information_theory) which lists several forms of related measures of information: - the self-information of an individual message or symbol taken from a given probability distribution; - the entropy of a given probability distribution of messages or symbols; - the entropy rate of a stochastic process. It also suggests a connection between information entropy and thermodynamic entropy, namely that the information entropy of a system is the amount of additional information needed to determine the microstate of a system (the states of all its particles), given the macrostate (identified by the bulk thermodynamic parameters such as temperature, volume, energy). More here: https://physics.stackexchange.com/questions/263197/is-information-entropy-the-same-as-thermodynamic-entropy -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list