Thanks Bill for this historical overview. I know your definition of information is based on ruling out possibilities, so I think to help the lay person (probably familiar with other ideas of ‘information’) the terms information, knowledge and learning should remain separate. Maybe it is just semantics that I am using, but I do struggle with the idea (my interpretation at least) that, say, to synthesize/analyse ‘information’ does not necessarily provide new information. Wouldn’t new insights from say, synthesizing previous decisions (where possibilities were eliminated), provide further ‘information’, i.e. to possibly rule out/in other/previous possibilities that were not apparent before, or is this just me using hindsight? Whatever the case, I think these types of possible interpretations need to be ‘tightened-up’ moving forward.
I might not understand exactly what you mean, but when I hear the words “analyze” and “synthesize”, additional information immediately comes to mind. In what ways are previous “decisions” being combined or augmented in order to bring about new information? I would say that the very act of synthesizing or processing existing information presupposes additional information, since it is neither self-evident nor inherent to analyze data in only one particular way. That seems to be the case to me anyways!
Thanks for your valued comment Tucker. Bill has used a previous example of … Its raining outside; Its not raining outside; which could convey no information (i.e. possibilities are not eliminated)
However, if ‘time’ is inferred (knowledge/learning) and invoke that storm clouds can produce rain (knowledge), then new information could be conveyed based on the (knowledge) that storm clouds may still be outside.
I would suggest presupposing additional (knowledge), rather than information. Synthesizing options with knowledge could produce new information, such as, in this example, ruling out the option to go outside as the rain may start again (assuming no umbrella/rain-coat that is).
Of course! I really enjoy talking about these niche topics!
Now I think I understand what you mean, but I am skeptical of the knowledge / information distinction when used in this context.
I have no issue in using the word "create" when it comes to new information generated by minds, such as when we write something down, or speak in a specific language. However, in a computational context I think it is problematic to speak this way.
In your example, you grant the following: there are clouds outside, time exists, and clouds can create rain. You also seem to grant that is was previously raining outside. ("may still be")
You call these instances of "knowledge" but they can be understood as pre-existing information. If there are clouds outside that rules out that there aren't clouds outside; if it was raining previously that rules out it wasn't raining previously; if you infer "time" that raises the question of what does one mean by time i.e. how does one specify it; if you grant that clouds can produce rain that rules out that clouds can't produce rain. There are probably even more layers of specification involved in this that I am missing.
(Maybe the description of the clouds would be another, as not all clouds produce rain)
The point is, if you were to try and get a computer program to predict whether you should or shouldn't bring an umbrella with you outside, it would need to compute all of this information to generate a result. In such a context, all of this "knowledge" would have to be re-presented as computable information, under which conservation of information would apply. In other words, a valuable transformation of information would be undertaken, but not a creation ex nihilo of information.
Hopefully you understand what I mean by all of this! As Bill has mentioned, all information can be formulated as "search" and vice versa. This is a very general phenomenon - you could even say that it is the most basic or fundamental phenomenon.
If I am still misunderstanding the point you are trying to make I apologize!
Ok, sure, lets concentrate on the computational context not minds! Would you consider parallel processing in this, or dare I say, potentially quantum computing? Would the conservation of information still apply?
Just using serial processing, I’ll frame this more succinctly to hopefully assist: Decision making (e.g. take umbrella) is based on the probabilities of the remaining choices/options (e.g. chances of raining outside), where ‘information’, via searches, has previously reduced the number of options (but not necessarily based on probability, e.g. the preexisting knowledge that clouds can produce rain). Synthesising, like in neural networks (or is this now called AI?), can be used to learn/train from previous choices (is this new or preexisting information or knowledge?). On the other hand, analysing would be assigning/modifying the probabilities to the choices (this would be new knowledge?).
Thanks so much Tucker as I do understand a lot of what you mean, but maybe we take this off-line as I am appreciative to understand it all! Alternatively, I’ll wait until Bill’s new book is published and hopefully the reviewer’s will have addressed most of the communication issues/concerns on this very interesting topic.
Yeah, I obviously can’t speak for Bill and I’m sure that most of this will be cleared up in his next book. I think it would probably be better to wait for his say on these topics since we are both just laymen trying to wrap our heads around this!
Personally, I just do not see how programs or computation machines CREATE anything. (Again, we come back to semantics perhaps.) I see a fundamental difference between an artist spontaneously painting a picture and a neural network generating one from a massive training set. I am willing to grant “creation” in the former case but not in the latter.
In the future, such as with quantum computing as you alluded to, this might change, but as of now I just can’t see otherwise! I always see more and more layers of information needed for any transformation of pre-existing information to occur. This information seems to me to exist implicitly in the program even if it can be expressed differently depending on the context.
The final probabilities assigned by a neural network might be "new" in the sense they weren't hard coded in before hand, (you wont find them anywhere before they appear as the result) but they are not "new" in the sense of computational continuity with the input information (including the built-in rules of the system itself)
I must confess, I am no expert in parallel processing and it’s relationship to conservation of information, so I will definitely look more into that!
It is possible that I am still missing your point! Either way, it was nice discussing it with you!
(Maybe Bill will see these comments eventually and enlighten us, haha)
Thanks Bill for this historical overview. I know your definition of information is based on ruling out possibilities, so I think to help the lay person (probably familiar with other ideas of ‘information’) the terms information, knowledge and learning should remain separate. Maybe it is just semantics that I am using, but I do struggle with the idea (my interpretation at least) that, say, to synthesize/analyse ‘information’ does not necessarily provide new information. Wouldn’t new insights from say, synthesizing previous decisions (where possibilities were eliminated), provide further ‘information’, i.e. to possibly rule out/in other/previous possibilities that were not apparent before, or is this just me using hindsight? Whatever the case, I think these types of possible interpretations need to be ‘tightened-up’ moving forward.
I might not understand exactly what you mean, but when I hear the words “analyze” and “synthesize”, additional information immediately comes to mind. In what ways are previous “decisions” being combined or augmented in order to bring about new information? I would say that the very act of synthesizing or processing existing information presupposes additional information, since it is neither self-evident nor inherent to analyze data in only one particular way. That seems to be the case to me anyways!
Thanks for your valued comment Tucker. Bill has used a previous example of … Its raining outside; Its not raining outside; which could convey no information (i.e. possibilities are not eliminated)
However, if ‘time’ is inferred (knowledge/learning) and invoke that storm clouds can produce rain (knowledge), then new information could be conveyed based on the (knowledge) that storm clouds may still be outside.
I would suggest presupposing additional (knowledge), rather than information. Synthesizing options with knowledge could produce new information, such as, in this example, ruling out the option to go outside as the rain may start again (assuming no umbrella/rain-coat that is).
Of course! I really enjoy talking about these niche topics!
Now I think I understand what you mean, but I am skeptical of the knowledge / information distinction when used in this context.
I have no issue in using the word "create" when it comes to new information generated by minds, such as when we write something down, or speak in a specific language. However, in a computational context I think it is problematic to speak this way.
In your example, you grant the following: there are clouds outside, time exists, and clouds can create rain. You also seem to grant that is was previously raining outside. ("may still be")
You call these instances of "knowledge" but they can be understood as pre-existing information. If there are clouds outside that rules out that there aren't clouds outside; if it was raining previously that rules out it wasn't raining previously; if you infer "time" that raises the question of what does one mean by time i.e. how does one specify it; if you grant that clouds can produce rain that rules out that clouds can't produce rain. There are probably even more layers of specification involved in this that I am missing.
(Maybe the description of the clouds would be another, as not all clouds produce rain)
The point is, if you were to try and get a computer program to predict whether you should or shouldn't bring an umbrella with you outside, it would need to compute all of this information to generate a result. In such a context, all of this "knowledge" would have to be re-presented as computable information, under which conservation of information would apply. In other words, a valuable transformation of information would be undertaken, but not a creation ex nihilo of information.
Hopefully you understand what I mean by all of this! As Bill has mentioned, all information can be formulated as "search" and vice versa. This is a very general phenomenon - you could even say that it is the most basic or fundamental phenomenon.
If I am still misunderstanding the point you are trying to make I apologize!
Ok, sure, lets concentrate on the computational context not minds! Would you consider parallel processing in this, or dare I say, potentially quantum computing? Would the conservation of information still apply?
Just using serial processing, I’ll frame this more succinctly to hopefully assist: Decision making (e.g. take umbrella) is based on the probabilities of the remaining choices/options (e.g. chances of raining outside), where ‘information’, via searches, has previously reduced the number of options (but not necessarily based on probability, e.g. the preexisting knowledge that clouds can produce rain). Synthesising, like in neural networks (or is this now called AI?), can be used to learn/train from previous choices (is this new or preexisting information or knowledge?). On the other hand, analysing would be assigning/modifying the probabilities to the choices (this would be new knowledge?).
Thanks so much Tucker as I do understand a lot of what you mean, but maybe we take this off-line as I am appreciative to understand it all! Alternatively, I’ll wait until Bill’s new book is published and hopefully the reviewer’s will have addressed most of the communication issues/concerns on this very interesting topic.
Yeah, I obviously can’t speak for Bill and I’m sure that most of this will be cleared up in his next book. I think it would probably be better to wait for his say on these topics since we are both just laymen trying to wrap our heads around this!
Personally, I just do not see how programs or computation machines CREATE anything. (Again, we come back to semantics perhaps.) I see a fundamental difference between an artist spontaneously painting a picture and a neural network generating one from a massive training set. I am willing to grant “creation” in the former case but not in the latter.
In the future, such as with quantum computing as you alluded to, this might change, but as of now I just can’t see otherwise! I always see more and more layers of information needed for any transformation of pre-existing information to occur. This information seems to me to exist implicitly in the program even if it can be expressed differently depending on the context.
The final probabilities assigned by a neural network might be "new" in the sense they weren't hard coded in before hand, (you wont find them anywhere before they appear as the result) but they are not "new" in the sense of computational continuity with the input information (including the built-in rules of the system itself)
I must confess, I am no expert in parallel processing and it’s relationship to conservation of information, so I will definitely look more into that!
It is possible that I am still missing your point! Either way, it was nice discussing it with you!
(Maybe Bill will see these comments eventually and enlighten us, haha)