The efficiency of compression systems is analogous to the efficiency of cleaning, no matter how clean you want to make anything, you have to face that you can only push dirt around, you can't get rid of it...
Regarding applying generative models to message passing, the system efficiency of any message includes the cost of setting up the model.
Of course, by placing the desired message space into a generative model, the amount of data required to activate a message of high complexity can be made very small. But it's not only the cost of invoking a message that matters to efficiency. The costs of maintaining the model must be considered.
For example, consider delivering a complex message by sending a storage device, say an old video tape with various video messages stored on it along with a schedule of its contents. Later, using a different channel than was used to arrange the video player, a complex message is manifest by sending an index to the schedule, and the receiver obtains the message by playing the portion of the tape indicated by the schedule reference.
The particular details of the message storage are important only to operate the mechanism of retrieval. The system could just as well be a book full of messages, and a tabular index. Marshall McLuhan referred to the lightbulb as "pure information" for this reason; the medium of the active bulb creates entirely new information environments that were impossible before its application and does so by simply attenuating the receivers visual field.
The point is that by carefully arranging the contexts of the sender and the receiver, an arbitrarily simple instance of communication can invoke (transmit) an arbitrarily complex message, or even environment.
But when considering efficiency, is the total amount of work to arrange the sending and receiving contexts properly included in the assessment? Without some normalization of setup costs between systems, how can "compression" efficiency be properly analyzed?
When you consider all the amazing complexity of thought that is manifest by the relatively trivial communicative forms of speech, human life might be determined to be the greatest balancing of concerns for efficient communication to be found in the known universe, so much as to reveal a seemingly supernatural gradient of distinction between the energy costs of context setup and open-ended capacity for message passing.
What Afred Korszybski coined "time binding".
More narrowly, it's clear that generative models are an entirely new paradigm for information management, and maybe the greatest promise lies herein, not in a pipe dream of AGI?
_wire_•3w ago
Regarding applying generative models to message passing, the system efficiency of any message includes the cost of setting up the model.
Of course, by placing the desired message space into a generative model, the amount of data required to activate a message of high complexity can be made very small. But it's not only the cost of invoking a message that matters to efficiency. The costs of maintaining the model must be considered.
For example, consider delivering a complex message by sending a storage device, say an old video tape with various video messages stored on it along with a schedule of its contents. Later, using a different channel than was used to arrange the video player, a complex message is manifest by sending an index to the schedule, and the receiver obtains the message by playing the portion of the tape indicated by the schedule reference.
The particular details of the message storage are important only to operate the mechanism of retrieval. The system could just as well be a book full of messages, and a tabular index. Marshall McLuhan referred to the lightbulb as "pure information" for this reason; the medium of the active bulb creates entirely new information environments that were impossible before its application and does so by simply attenuating the receivers visual field.
The point is that by carefully arranging the contexts of the sender and the receiver, an arbitrarily simple instance of communication can invoke (transmit) an arbitrarily complex message, or even environment.
But when considering efficiency, is the total amount of work to arrange the sending and receiving contexts properly included in the assessment? Without some normalization of setup costs between systems, how can "compression" efficiency be properly analyzed?
When you consider all the amazing complexity of thought that is manifest by the relatively trivial communicative forms of speech, human life might be determined to be the greatest balancing of concerns for efficient communication to be found in the known universe, so much as to reveal a seemingly supernatural gradient of distinction between the energy costs of context setup and open-ended capacity for message passing.
What Afred Korszybski coined "time binding".
More narrowly, it's clear that generative models are an entirely new paradigm for information management, and maybe the greatest promise lies herein, not in a pipe dream of AGI?