> It’s not just an analogy or a coincidence that the word “entropy” is a word used in both physics and computer science (information theory).
Shannon asked von Neumann what to call it, and von Neumann said "You should call it entropy, for two reasons: In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, nobody knows what entropy really is, so in a debate you will always have the advantage."
Um, only the second part was a joke. The first part was confirmation of your statement that it's not just an analogy or coincidence ... and Shannon followed von Neumann's advice (not joke).
Shannon asked von Neumann what to call it, and von Neumann said "You should call it entropy, for two reasons: In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, nobody knows what entropy really is, so in a debate you will always have the advantage."