Blog jamborees are one of the few formats that actually surface writers nobody has heard of yet. The institutional algorithms keep recycling the same top twenty. Glad someone is still doing the curation work.
> Martsinovich argues, via code snippets, that there is no such thing as a “conversation” with a chatbot. The AI is born anew every time it speaks, and it simply reads the dialogue so far and then tries to write the next line.
Actually, this understates things. Every time an LLM adds a new output token to a conversation, it needs to reread every prior word in the conversation. In other words, every word that an LLM generates is produced by an isolated process; it's born anew every time it needs to choose the next word. Every output token that it adds to the conversation could be generated by a completely separate server in a different part of the world, and you would have no idea. (The term for a system that works like this is a Markov process. Also, to get really technical, there is some caching of intermediate calculations done between output token generations, but it's not necessary at all.)
The idea that LLMs are more like industrial processes than people is one of the things I tried to get at in my submission, but I admit that at this point, I have no idea what the state of "common knowledge" around LLMs is, and thus what the starting point for talking about this kind of thing should be. It's definitely one of those topics that's new enough that everyone approaches it with different prior knowledge and also different intuitive understanding and reactions, which makes it both easy to write about (because there's a lot of room for new takes) and hard to write about (because any given reader might have a very different baseline understanding or opinions than you.) (I'm probably going to restructure my submission quite a bit before posting it publicly...)
Can someone explain to me what makes this concept novel to a layperson or at least non-AIML specialist? I don’t mean to be rude here, this is a limitation of my worldview, but can’t you technically represent all communication as equally idempotent processes (assuming randomness can be seeded)? All communication is the sum of context, statements and current person version/state, right?
Definitely. What I (and I believe, the post linked above) are saying is that an LLM has no “current person version/state;” it only has context and statements. Also, the only way that the context or previous statements are used is that the LLM re-reads them every time it needs to generate a new output token. (The reason that it needs to re-read them is that lack of internal version/state.) This makes LLMs different from the human-like AI that a lot of science fiction and futurology has been about: an AI like Data from Star Trek had hidden internal “state” that influenced his behavior, while LLMs only have hidden internal “processes” that operate on the tokens that they’re manually given.
Or I guess I should say, an LLM has no version/state other than the state that a specific model acquires during training, which is the exact same for every conversation and at every point during every conversation
Right, but there is often user personalization involved (from added instructions and prior interactions), and like you’ve mentioned a model’s global frozen state. That said, I don’t see how Data from Star Trek is objectively different from LLMs at this abstract level, and no I don’t believe these things are alive, this is just from a systems modeling point of view.
More and more of humanity's most pressing problems are becoming intertwined as we speak. Climate change continues to change the weather. Economic instability is putting pressure on households and businesses.
Innovation brings about unprecedented opportunities and new risks.
Confidence in institutions is waning in many nations and international tensions remain in the global arena.
These are not separate incidents. They are all part of the same cloth.
Feel the wake-up call. The next wildfire season in California eclipses the season before. A day away, a family in Japan blogs about the upcoming stronger typhoons and more erratic rainfall. And continents away, farmers in the United Kingdom adjust to climate shifts and eat up higher food costs.
Various countries. Various cultures.
The common problem.
Researchers report that global temperatures are at record highs, fueling more frequent and severe heatwaves, floods, droughts and wildfires. Around the world, billions of dollars are being spent on adaptation, but the human toll is felt far beyond the destruction of property-families lose their homes. Businesses lose their income.
Children are deprived of an education.
Entire communities are faced with the dilemma of rebuilding or relocating.
Inevitably, the global economy is also in the middle of a difficult evolution.
In many countries, inflation is less than the recent highs, but millions of people are still feeling the pinch of inflation in the prices of housing, healthcare, food and energy. Will our children be able to enjoy the guaranteed prosperity that our parents have grown used to? Small businesses build their business in a higher interest rate world, because of supply change-readjustments and the quick changing of technologies.
Technology alone has its own extraordinary and cautionary tale to tell:
Artificial intelligence is revolutionizing medicine, education, manufacturing, and scientific research at an astonishing pace: doctors are identifying diseases more rapidly, researchers making new discoveries faster, and companies becoming more efficient.
But those same tools can too magnify lies, power the next level of cybercrimes, and prompt some uncomfortable conversations about privacy, work, and accountability. The task isn't just to create the tech-it is to create society that grows the government, the knowledge, and the morals to shape it wisely.
Believe it or not, trust has become one of the most valuable commodities on the planet.
When the public ceases to trust information, institutions and each other, the challenges we face become significantly more difficult. Heated debates pit side against side; agree on a fact becomes impossible; trust erodes to suspicion;
I acknowledge that it is really hard to quantify this undermining of trust but not so hard to observe it.
It appears when community members are willing to settle for so little. It appears when online discussion condones outrage more than compassion. It appears when individuals cocoon themselves, because bridging the gap seems tougher than ever.
In the meantime, peacemakers, fault lines of geopolitics, are still affecting our everyday life in subtle and hidden ways.
Conflicts and strategic rivalry have an impact on energy rates, availability of food, access to high-technology, shipping worldwide and investments from abroad. Decisions taken in one country impact across big chunks of the world.
Our world is more interconnected than ever before.
That interconnectedness is, at the same time, our greatest strength and one of our greatest weaknesses.
But the story did not end there.
Throughout history moments of great hardship have proven to be a catalyst for innovation and cooperation. The use of cleaner energy technologies keeps growing. Medical research accelerates at a faster pace than ever before.
Communities are building more resilient infrastructure.
Scientists around the world are cooperating across borders to tackle issues no one country can resolve by itself.
Progress doesn't happen in a straight line.
It sometimes grows quietly through the thousands of choices that every worker makes each day: engineers developing better safety systems, teachers getting students ready for a changing economy, doctors and nurses safeguarding their fellow citizens, entrepreneurs developing solutions to mounting sustainability challenges, citizens opting for conversation over conflict.
The questions for the audiences of the United States, the UK, Japan,and the even farther-flung countriesis not whether we are faced with problems-global or otherwise-it is obviously yes.
So the question is-how societies react.
Will we develop resilience proactively? Will technological advancements be accompanied by moral wisdom? Will economic progress embrace sustainability? Will public dialogue focus on data rather than feelings and partnership instead of opposition?
The answers will determine the next ten years-and the generations to follow.
All generations think their problems are the worst. Maybe we really have it worse- not because of the magnitude of the issues, but because of their interconnections.
Climate, economics, technology, security and trust are no longer talked about in isolation.
They have converged into one conversation.
And every person, group, corporation and country has a say in how the next chapter is composed.
Future doesn't have the power to grow brighter or darker by itself. It depends on millions of decisions we make right now-decisions for unity instead of separation, for responsibility instead of passivity, for the future rather than comfort in the now.
Blog jamborees are one of the few formats that actually surface writers nobody has heard of yet. The institutional algorithms keep recycling the same top twenty. Glad someone is still doing the curation work.
> Martsinovich argues, via code snippets, that there is no such thing as a “conversation” with a chatbot. The AI is born anew every time it speaks, and it simply reads the dialogue so far and then tries to write the next line.
Actually, this understates things. Every time an LLM adds a new output token to a conversation, it needs to reread every prior word in the conversation. In other words, every word that an LLM generates is produced by an isolated process; it's born anew every time it needs to choose the next word. Every output token that it adds to the conversation could be generated by a completely separate server in a different part of the world, and you would have no idea. (The term for a system that works like this is a Markov process. Also, to get really technical, there is some caching of intermediate calculations done between output token generations, but it's not necessary at all.)
The idea that LLMs are more like industrial processes than people is one of the things I tried to get at in my submission, but I admit that at this point, I have no idea what the state of "common knowledge" around LLMs is, and thus what the starting point for talking about this kind of thing should be. It's definitely one of those topics that's new enough that everyone approaches it with different prior knowledge and also different intuitive understanding and reactions, which makes it both easy to write about (because there's a lot of room for new takes) and hard to write about (because any given reader might have a very different baseline understanding or opinions than you.) (I'm probably going to restructure my submission quite a bit before posting it publicly...)
Can someone explain to me what makes this concept novel to a layperson or at least non-AIML specialist? I don’t mean to be rude here, this is a limitation of my worldview, but can’t you technically represent all communication as equally idempotent processes (assuming randomness can be seeded)? All communication is the sum of context, statements and current person version/state, right?
Definitely. What I (and I believe, the post linked above) are saying is that an LLM has no “current person version/state;” it only has context and statements. Also, the only way that the context or previous statements are used is that the LLM re-reads them every time it needs to generate a new output token. (The reason that it needs to re-read them is that lack of internal version/state.) This makes LLMs different from the human-like AI that a lot of science fiction and futurology has been about: an AI like Data from Star Trek had hidden internal “state” that influenced his behavior, while LLMs only have hidden internal “processes” that operate on the tokens that they’re manually given.
Or I guess I should say, an LLM has no version/state other than the state that a specific model acquires during training, which is the exact same for every conversation and at every point during every conversation
Right, but there is often user personalization involved (from added instructions and prior interactions), and like you’ve mentioned a model’s global frozen state. That said, I don’t see how Data from Star Trek is objectively different from LLMs at this abstract level, and no I don’t believe these things are alive, this is just from a systems modeling point of view.
More and more of humanity's most pressing problems are becoming intertwined as we speak. Climate change continues to change the weather. Economic instability is putting pressure on households and businesses.
Innovation brings about unprecedented opportunities and new risks.
Confidence in institutions is waning in many nations and international tensions remain in the global arena.
These are not separate incidents. They are all part of the same cloth.
Feel the wake-up call. The next wildfire season in California eclipses the season before. A day away, a family in Japan blogs about the upcoming stronger typhoons and more erratic rainfall. And continents away, farmers in the United Kingdom adjust to climate shifts and eat up higher food costs.
Various countries. Various cultures.
The common problem.
Researchers report that global temperatures are at record highs, fueling more frequent and severe heatwaves, floods, droughts and wildfires. Around the world, billions of dollars are being spent on adaptation, but the human toll is felt far beyond the destruction of property-families lose their homes. Businesses lose their income.
Children are deprived of an education.
Entire communities are faced with the dilemma of rebuilding or relocating.
Inevitably, the global economy is also in the middle of a difficult evolution.
In many countries, inflation is less than the recent highs, but millions of people are still feeling the pinch of inflation in the prices of housing, healthcare, food and energy. Will our children be able to enjoy the guaranteed prosperity that our parents have grown used to? Small businesses build their business in a higher interest rate world, because of supply change-readjustments and the quick changing of technologies.
Technology alone has its own extraordinary and cautionary tale to tell:
Artificial intelligence is revolutionizing medicine, education, manufacturing, and scientific research at an astonishing pace: doctors are identifying diseases more rapidly, researchers making new discoveries faster, and companies becoming more efficient.
But those same tools can too magnify lies, power the next level of cybercrimes, and prompt some uncomfortable conversations about privacy, work, and accountability. The task isn't just to create the tech-it is to create society that grows the government, the knowledge, and the morals to shape it wisely.
Believe it or not, trust has become one of the most valuable commodities on the planet.
When the public ceases to trust information, institutions and each other, the challenges we face become significantly more difficult. Heated debates pit side against side; agree on a fact becomes impossible; trust erodes to suspicion;
I acknowledge that it is really hard to quantify this undermining of trust but not so hard to observe it.
It appears when community members are willing to settle for so little. It appears when online discussion condones outrage more than compassion. It appears when individuals cocoon themselves, because bridging the gap seems tougher than ever.
In the meantime, peacemakers, fault lines of geopolitics, are still affecting our everyday life in subtle and hidden ways.
Conflicts and strategic rivalry have an impact on energy rates, availability of food, access to high-technology, shipping worldwide and investments from abroad. Decisions taken in one country impact across big chunks of the world.
Our world is more interconnected than ever before.
That interconnectedness is, at the same time, our greatest strength and one of our greatest weaknesses.
But the story did not end there.
Throughout history moments of great hardship have proven to be a catalyst for innovation and cooperation. The use of cleaner energy technologies keeps growing. Medical research accelerates at a faster pace than ever before.
Communities are building more resilient infrastructure.
Scientists around the world are cooperating across borders to tackle issues no one country can resolve by itself.
Progress doesn't happen in a straight line.
It sometimes grows quietly through the thousands of choices that every worker makes each day: engineers developing better safety systems, teachers getting students ready for a changing economy, doctors and nurses safeguarding their fellow citizens, entrepreneurs developing solutions to mounting sustainability challenges, citizens opting for conversation over conflict.
The questions for the audiences of the United States, the UK, Japan,and the even farther-flung countriesis not whether we are faced with problems-global or otherwise-it is obviously yes.
So the question is-how societies react.
Will we develop resilience proactively? Will technological advancements be accompanied by moral wisdom? Will economic progress embrace sustainability? Will public dialogue focus on data rather than feelings and partnership instead of opposition?
The answers will determine the next ten years-and the generations to follow.
All generations think their problems are the worst. Maybe we really have it worse- not because of the magnitude of the issues, but because of their interconnections.
Climate, economics, technology, security and trust are no longer talked about in isolation.
They have converged into one conversation.
And every person, group, corporation and country has a say in how the next chapter is composed.
Future doesn't have the power to grow brighter or darker by itself. It depends on millions of decisions we make right now-decisions for unity instead of separation, for responsibility instead of passivity, for the future rather than comfort in the now.
This is the best thing on the internet- thank u for doing this! Looking forward to re-entering next year 🩷
#2 Ricardo Lopez has locked the likes and comments; further, that article is his first post. I simply love that.