For the benefit of Jorge: Information theory for dummies, take 2

[FreezBee]

In A Mathematical Theory of Communication, Claude Shannon writes these famous words:

The fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point. Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem.

So, as is well-known, 'information' in Shannon's sense does not mean 'meaning'. What then does it mean?

Let's read on:

The significant aspect is that the actual message is one selected from a set of possible messages. The system must be designed to operate for each possible selection, not just the one which will actually be chosen since this is unknown at the time of design.

And that's what it is: information in Shannon's sense deals with selection from a set of possibilities. Information is information about, what was selected.

You receive a message. The first letter is 'T'. The first sentence might then be: "Tea is cheap in China this year", or it might be: "There's a cuckoo in my head, don't wake it up". If these two sentences are all there is to choose from, the second letter will be sufficient to tell you, which sentence was selected by the sender.

Now, assume you know the set of messages the sender may send, and you know the probability with which the sender will send each message. Then you can make an efficient encoding, where the rarest messages are given the longest codes, and the most frequent messages are given the shortest codes. That's how morse code works.

That's the key point of statistical information theory.

Assume information increases on the statistical level. What does that mean?

Back to Shannon:

If the number of messages in the set is finite then this number or any monotonic function of this number can be regarded as a measure of the information produced when one message is chosen from the set, all choices being equally likely. As was pointed out by Hartley the most natural choice is the logarithmic function.

That is, as measure of information is used some monotonic function of the number of possible messages.

Say, you can send one of the following messages:

1. I love bananas
2. I love Lucy
3. I love Carmen

The number of possible messages is 3, but we can use any monotonic function.

This leads to:

Lemma 1: The larger the number of possible messages, the larger is the information in the message actually sent.


In other words: The information in a message that is sent is information about the messages that weren't sent.

This leads to:

Corollary 1: If there is only one message possible, there are no messages that are not sent, and accordingly the information in the message sent is zero.


That is, increasing the information on the statistical level corresponds to an increase in the number of possible messages. This will correspond to an increase on the semantic level, if syntactically didderent messages also are semantically different.

Returning to our example, assume we come up with a fourth message: "It's too hot to wear slippers today", which happens to mean the same thing as "I love bananas". Then we have an increase in information on the statistical level, though not on the semantical level.

HOWEVER, Prof. Dr. Werner Gitt, in the article Information, science and biology, writes:

Theorem 3: Since Shannon’s definition of information relates exclusively to the statistical relationship of chains of symbols and completely ignores their semantic aspect, this concept of information is wholly unsuitable for the evaluation of chains of symbols conveying a meaning.

We are dealing with something slightly different here. Think about a stream of symbols that make up a stream of messages. Assume the frequency of each symbol to be the same in each message as in the total stream. Then there is no difference between the various levels, BECAUSE as you obtain more and more information about the messages, you obtain more and more information about the total stream.

Therefore, while Gitt to some extent has a point, he doesn't have enough of a point.


cheers
- FreezBee

[FreezBee]

Hmmm - I had expected Jorge to post something about the latests results from the Gitt group. Is the lack of response due to there being no results? No, can't be, can it?

- FreezBee

[wattsr1]

Hmmm - I had expected Jorge to post something about the latests results from the Gitt group. Is the lack of response due to there being no results? No, can't be, can it?

- FreezBee

Given Jorge's past displays, I doubt if he understood the post.

[Jorge]

Hmmm - I had expected Jorge to post something about the latests results from the Gitt group. Is the lack of response due to there being no results? No, can't be, can it?

- FreezBee

I was out until now, FreeBee -- why the paranoia?

Also, why do you feel the need to give me some information 'basics'? If you knew my background you'd realize that I'm way, way ahead of what you've provided. In fact, that's part of the problem and explains why I'm generally talking past materialistic Naturalists such as yourself (please try to not take that the wrong way).

As for the latest results from the Gitt group, I'm a bit behind on that. I was invited to participate in a week-long information workshop with Gitt and about a dozen other creationists but, unfortunately, I was tied up elsewhere and couldn't attend. I'll have to get back to you when I do get caught up. My own work in this area has been on hold due to other activities.

Jorge

[Tiggy]

I was out until now, FreeBee -- why the paranoia?

Also, why do you feel the need to give me some information 'basics'? If you knew my background you'd realize that I'm way, way ahead of what you've provided. In fact, that's part of the problem and explains why I'm generally talking past materialistic Naturalists such as yourself (please try to not take that the wrong way).

Gee Jorge, do you ever sprain your arm patting yourself on the back?

AFAIK, all you ever do is run your big mouth and tell us what a genius and expert you are. Funny thing is, when ever you are asked to put up or shut up all you ever do is pass gas and run away.

Here, prove me right by cowardly avoiding these questions

1. Please give us a concise definition of 'Information' as it applies to biological structures

2. Please give us an example of how to calculate the information content in a biological structure.

3. If you can't calculate the information content of a biological structure (which we both know you can't, BTW) , how can you tell if the 'information' increases or decreases?


OK Jorge, have at it! You've been asked these same questions many times before, and cowardly squirmed and avoided them every time.

If you respond with "you just don't understand!!!' , or just some half-witted insult then you admit you have no answers.

If you respond with "already answered!' then link to the answers or you admit you are lying about having previously answered.

- T

[Jorge]

Gee Jorge, do you ever sprain your arm patting yourself on the back?

Factual statements are not patting oneself on the back, Bozo.

AFAIK, all you ever do is run your big mouth and tell us what a genius and expert you are.

Show me one place where I've said that I'm a genius or an expert. Knowing a bit about this particular area is all I've said.
You could be a dim-witted buffoon, Tiggy, except that buffoons are usually amusing and you're not even that.

The rest of your post deserves no attention / reply and so ... no reply.

Jorge

[Tiggy]

Factual statements are not patting oneself on the back, Bozo.

Show me one place where I've said that I'm a genius or an expert. Knowing a bit about this particular area is all I've said.
You could be a dim-witted buffoon, Tiggy, except that buffoons are usually amusing and you're not even that.

The rest of your post deserves no attention / reply and so ... no reply.

Jorge

Thaks Jorge! Just as predicted, you ran your big mouth but when challenged you passed gas and ran away!

You can't define 'information' as it applies to biological structures.

You can't calculate the 'information content' of a biological structure.

So you have no idea if the 'information content' of a biological structure can increase or decrease.

You've convince us Jorge - you know 'a bit' about information theory all right; an INFINITESIMALLY SMALL bit.



- T

[Jorge]

Thaks Jorge! Just as predicted, you ran your big mouth but when challenged you passed gas and ran away!

You can't define 'information' as it applies to biological structures.

You can't calculate the 'information content' of a biological structure.

So you have no idea if the 'information content' of a biological structure can increase or decrease.

You've convince us Jorge - you know 'a bit' about information theory all right; an INFINITESIMALLY SMALL bit.

- T

Yeah, sure ... now go suck a lemon. Hey, Einstein (with my sincere apologies to Albert), who's William J. Bennetta?

Jorge

[Tiggy]

Yeah, sure ... now go suck a lemon.

Better a lemon than the things* you suck on Jorge

Now, can you please answer these question concerning Information Theory?

1. Please give us a concise definition of 'Information' as it applies to biological structures

2. Please give us an example of how to calculate the information content in a biological structure.

3. If you can't calculate the information content of a biological structure , how can you tell if the 'information' increases or decreases?

Thanks for sharing 'a bit' of your knowledge with us

- T

* of course I'm referring to eggs. What else would Jorge like?

[easyboy201]

It seems to me that in these definitions that the number of messages are “finite” seems to be assumed. Does it give you any further explanation to why they assume that there are a finite number of messages and not an infinite number?

[sylas]

It seems to me that in these definitions that the number of messages are “finite” seems to be assumed. Does it give you any further explanation to why they assume that there are a finite number of messages and not an infinite number?

the number of messages of a given length is finite. for example, consider messages using lowercase letters, comma, period, space, and smilie. this is an alphabet of thirty characters, and there are thirty raised to power three hundred possible messages of length three hundred. this is one of them.

[technomage]

It seems to me that in these definitions that the number of messages are “finite” seems to be assumed.

An infinite information set would require an infinite encoding set.

Case in point: the encoding set for written English communication has twenty-six basic members, each with two variations, and perhaps a couple dozen anciliary members (punctuation). The English alphabet can be used to encode an incredible amount of information, but cannot be used to encode an infinite amount of information.

[easyboy201]

Thanks Sylas and Justin!

[FreezBee]

I was out until now, FreeBee -- why the paranoia?

Well, I'm thankful that you manage to find some time in your schedule to respond

Also, why do you feel the need to give me some information 'basics'?

Just in case it was needed. From your prior posts it's been difficult to figure out exactly how much (or how little) you knew about information theory.

If you knew my background you'd realize that I'm way, way ahead of what you've provided.

Your background as an air force mechanic? Or what do you mean?

In fact, that's part of the problem and explains why I'm generally talking past materialistic Naturalists such as yourself (please try to not take that the wrong way).

Oh, I am not a materialistic naturalist, but a structuralist.

As for the latest results from the Gitt group, I'm a bit behind on that. I was invited to participate in a week-long information workshop with Gitt and about a dozen other creationists but, unfortunately, I was tied up elsewhere and couldn't attend.

Hmmm, where have I read that one before?

I'll have to get back to you when I do get caught up. My own work in this area has been on hold due to other activities.

Well, don't worry - I'll look forward to you getting caught up; but in the meanwhile, I suggest, you could explain your definition of information, so we can see, what it would mean that information increases/decreases.

Thanks in advance.


- FreezBee

[Jorge]

Just in case it was needed. From your prior posts it's been difficult to figure out exactly how much (or how little) you knew about information theory.

Believe what you wish.

Your background as an air force mechanic? Or what do you mean?

May I ask where you got the idea that I was an Air Force mechanic? And if I had been, anything wrong with that?
The fact is that I conducted over seven years of hands-on scientific research in the Air Force in various fields including information-related.

Oh, I am not a materialistic naturalist, but a structuralist.

And the difference is ... ???

Hmmm, where have I read that one before?

Wherever you read it you read right.

Well, don't worry - I'll look forward to you getting caught up; but in the meanwhile, I suggest, you could explain your definition of information, so we can see, what it would mean that information increases/decreases.

To the point that I'm able to, I've already answered that. For instance, I hold that statistically information may increase naturally -- even randomly. But that doesn't work if syntax and semantics are introduced (as they must be in a real-world, meaningful definition). Later ...

Jorge