Information and Representation

"Ice, ice, baby
Too cold"

It's so cold outside that ice crystals are huddling together for warmth. I should be firmly entrenched under the covers right now, like a good Mid-Westerner, but I was bothered by the sloppiness at the end of my last post.

In struggling to learn the basics mechanics of information theory, I am consistently plagued my pre-existing education in cognition and neuroscience. While the entropy of a fair coin is defined as one bit, I know that there are at least six distinct ways to represent the final state of the coin.

If HEADS, then NOT TAILS.
If HEADS, then NOT (NOT HEADS).
If TAILS, then NOT HEADS.
If TAILS, then NOT (NOT TAILS).

The cognitive representation of each of the six states listed above is different. A different set of neurons is activated in each representation. If I'm arguing that the activation of mental representations contributes to the selection of the state of subsequent observations, then the differences in each of the representations that I just mentioned become critical.

Similarly, the activation of the mental representation 'HEADS' at 5 second intervals for 2 minutes would need to be treated differently than a single activation of 'HEADS', or the activation of 'HEADS' followed by the activation of 'NOT TAILS'.

Additionally, the neurons that are active in a particular representation continuously modify and are modified by the neurons that are subsequently activated and the neurons whose activation immediately preceded their own. (Hint: Pay attention to what you are thinking immediately before and immediately after you try to select an outcome state.) The accessibility of a particular representation is changing with each activation.

My working definition of 'cumulative entropy' tends to take into account such differences in representation, treated them as varying degrees of information. My vocabulary in attempting to communicate these ideas is probably imprecise, as I am trying to reconcile concepts from various disciplines, but I will try to clarify as misunderstandings arise. Apologies for the confusion.