hex_m_hell

The “AI” industry would like everyone to believe that we are experiencing a second industrial revolution. Up until recently, this was far from true. LLMs provided a significantly better way to do natural language analysis and transformations than traditional natural language processing. They definitely changed one domain, but produced garbage in others. The hype didn't match the reality, and, in a lot of ways, still doesn't. But things are going to change, and we should soberly assess why and how.

There are patterns that are similar to the Industrial Revolution, specifically around undermining independent skilled labor and destroying or enclosing commons. I think there's also a similar likelihood of revolution, if not even more revolutionary potential. The global wave of revolutions connected to industrialization mostly lead to some variant of liberalism, with greater or lesser degrees of compromise to prevent socialist revolution. The old order also developed an extremely authoritarian and system of crushing revolution that was, to some degree, internationally coordinated.

At the intersection of technology and climate change, we've already seen early waves going back to the Arab Spring. This was probably the first wave of modern revolutions. Back then, revolution in the imperial core seemed impossible with the rapid destruction of Occupy. Today, many of the countries where autocracies were overthrown rapidly returned to some autocratic form of government. But not all of them.

It's important to remember that the monarchies of their day also tried using authoritarianism to stop the future. Learn about Metternich and the Holy Alliance. There was a concerted international effort to crush the wave of liberal revolution that had been topping monarchies since the 1770s. In case you weren't aware, it didn't work.

Some monarchies were able to hold on to power for a while by tightly controlling technological advancement. This did not turn out well for them. While Belgium, the Netherlands, and the UK monarchs survived through massive concessions, the Romanov plan to hold on to everything did not turn out quite so well for, uh, anyone. The proto-fascism of various European powers from the late 1700s through the early 1900s could not ultimately save any of them.

Fascism is not sustainable. It is the fire of a collapsing star, growing even as it frantically consumes itself, a phase that can only be a prelude to explosion. The best it can hope to do is silence civil discourse, gaining some time in exchange for completely losing signal about when the collapse will come and by what means.

Of course, we all know that the liberal democracies that came out of the various Imperial collapses produced the more resilient aristocracy of capital. This aristocracy managed to untether itself from geographically local politics through “free trade,” where capital is free to roam the world while borders keep people in.

We see echoes of Metternich in regional governments supporting for the Junta of Myanmar against popular revolution. Even governments that otherwise (outwardly) disagree with the policies of the dictatorship, would still prefer a “stable trading partner” to a free people. In Syria, too, Turkey would prefer to annihilate Rojava while local powers look for ways to exploit the situation for their own interest. It's common to criticize people comparing Rojava to Anarchist Spain because they are such different situations. But there are parallels in how liberal democracies would prefer fascism to any alternative to liberalism.

There is, like Metternich believed, a natural order. That natural order is, of course, in the form of liberalism, not monarchy, but the underlying structural assumption remains the same even as the systems shift. Power is deeply uncreative.

It is with that lack of creativity that we should pivot back to the current moment. We keep being told that “GenAI” (the set of technologies based on large data sets and transformer models, of which LLMs are one such technology) will destroy art. They will not. Photography allowed anyone to perfectly represent an image. It changed painting, but it didn't destroy it.

Impressionism was a direct reaction to photography. Painters even emulated photographic elements, such as blur and depth of field, in ways that had not ever been imagined before. But the most important reaction was that painting focused more and more on emotion. A photograph with a skilled photographer can capture emotion, but it remains limited in ways that other visual arts are not. It is its own thing. Photography is a specific medium. People predicted that it would destroy painting. It didn't.

GenAI won't destroy visual art. They may change it. But there are feelings that can't be encoded in prompts because they can't be encoded in words. Visual art exists specifically because words categorically can't capture certain things.

But even art that is literally words, literary arts, are not something LLMs can do well. LLM text feels hollow because it is. Writing is, ultimately, a synesthetic activity (even for people without that specific neurodivergence). Words have sounds, they feel ways in your mouth, they're connected to other sensations. An LLM can say “kiki” and “bobo.” It can find statistical associations that make it almost seem as though it understands. But that's all it is, a statistical association. It's a stochastic representation of a hidden process. There's something underneath that, something that's ultimately indecipherable. That thing touches the nature of the universe.

You can't represent it statistically in one big model, because it's not even consistent. Language both discovers and creates those connections, which create and lead to the discovery of more. It's a tangled hierarchy. No LLM will ever be able to do that, because no LLM will have a body shaped by nature.

But not all writing is like this. Some technical writing is simply the dry regurgitation of facts. Bad technical writing anyway, and a lot of it is bad. LLMs can represent bad technical writing pretty well. They're trained on a lot of it. Where it's useful to have such things, LLMs can probably do that just fine.

And yet, a good technical writer is actually not far off from a good creative writer. They will tap in to emotions and images to make concepts obvious. They will be playful to make reading enjoyable (instead the expected horrible slog). LLMs will not do this. LLMs can construct metaphors that are optimized for LLMs, but they can't really optimize for humans because they can't experience the world in a human body. They don't have eyes, ears, or feelings connected to millions of years of evolution.

Even things we are told that LLMs are good at, like text summarization, relies on hidden context. Summarization is necessarily lossy. What you is chosen to keep vs lose depends on the purpose of the summarization. It also depends on the frame within which you're operating. A text can only be summarized well if the summarizer has a model of the reader that includes information about things that they already know (which can be safely dropped), things that they don't know or that may conflict with their existing beliefs (which are essential to highlight).

Today GenAI images and videos have become the ascetics of modern fascism. Fascists use GenAI because no one would actually put in the time and effort to make the horrible images they want to exist. But even more than that, fascists hate art and artists. They hate it because can't control it and they don't understand it. They tell everyone that GenAI will destroy art, in the hopes that it will manifest.

GenAI will not destroy art. It can't. But perhaps in a generation or two, after this wave of fascists succumbs to entropy, GenAI might become another creative tool. In the meantime, I expect art to emphasize what can't be replaced: oil paintings having more texture, writing more focused on emotion, etc.

But there are things that GenAI may well replace. Protein folding is essentially a solved problem now, specifically because of GenAI. It had required humans. There was a huge project where humans would manually fold models of proteins. That created a massive data set, which can now solve for arbitrary proteins. Humans are, of course, still critical to pharmaceutical development, but one boring job is gone.

Web design is “solved.” A human is no longer needed to make something that's “close enough” to customer expectations. If the cost is low enough, then people will compromise. A lot of web design has always been the awful work of just making things look the same, or at least look good, on different platforms. That integration problem should absolutely go away. It's not creative, it's just technical noodling.

However, a good web designer doesn't just “make exactly what the customer wants.” A good web designer makes the right thing for the client. They know that the thing the customer wants will look awful, will not work, will convey things that are not culturally appropriate to their target audience. Anyone skilled in any service role is actually good because they can convince their customer that the thing they want is actually bad. An LLM will never do that.

But if LLMs are “good enough” to do the basic jobs, and they're cheap, then designers never get to learn. They never get to build the skills needed to help people not just implement their vision, but shift it to integrate the knowledge of a skilled artisan.

And that's the shift I expect to see.

I think we're moving into a factory model of technology, where software and digital artifacts can be produced in a way that “good enough” but the market is so flooded with cheap garbage that people can't afford to learn to make things better. The same economics that drive planned obsolescence will drive digital artifacts. This is most obvious in software right now, with people creating mountains of unmaintainable code that does some basic thing then breaks as soon as features start being added.

We already live in a world where technology is always broken. That will only get worse, if we let it.

Technical security has always been arcane and mostly invisible. For decades everything was vulnerable and most people weren't aware. Over time people started to become more and more aware, both as security people started getting better at explaining the risks to media (see FireSheep), and as more stuff got visibly hacked. At this point, nearly everyone has had their government records, financial records, or medical records leaked on the Internet at least once or twice. Passwords get leaked so often that Have I Been Pwned is a whole project, with 17 billion records. And yes. The answer is yes.

There are a bunch of IOT automatic tank gauges (ATG) exposed directly to the Internet. Apparently if you log in and tell them they're rotated a whole bunch of times, they catch fire. I say “log in” as though they all have passwords set, as though those passwords that do exist can't be easily guessed or even derived. That article talks about 900 of them. These have been known about for a long time, and the same article cited 6k during their previous scan.

Your hospital has been owned. MRIs don't get updated because each update has to go through the FDA. Those machines get plugged in to the Internet, they're running some outdated version of Windows, and they get hacked. Hospital security lets them get hacked because, they hope, the hackers will patch to keep other people out and the legit admins can't patch.

That's where we were before all of this LLM stuff. What we have today makes that look like Fort Knox.

People are vibe coding MCP servers using LLMs trained on decades of insecure trash. Then they tell you to install their tools remotely, so there's code you don't control running on your system. Great, it's a chance for a supply chain attack every time you restart the service. But worse still, there's natural language malware. We can finally have an autonomously polymorphic worm. At some point that's gonna intersect with Spiralism, and we'll get a memetic worm that crosses the boundary between humans and machines and back. We already have tons of AI propaganda (which, was always predictable).

With malicious skills and agent files, malicious MCP servers, and people building systems with basically no isolation, it's just a catastrofuck. It's so deeply hopeless that half of the MCP github pages are just like “YOLO, just pipe curl directly into bash disabling all security checks, lol!” LLMs have essentially infinite attack surface since it's all of natural language connected directly to some code exec or another. But what's the proposed solution? Just put an LLM in the middle to review all the LLM generated stuff. Nothing could go wrong. Oh, right, the “AI security” agent is just an LLM, which means it's also more attack surface.

None of this is actually impossible to fix. Humans have been the weakest link in security for a long time. We understand how to design systems that assume compromise. We just aren't doing that because “move fast!!” Fixing these problems requires actual focused engineering from a human. You can't vibe this shit.

Unfortunately, it's not profitable to slow down. We already see that if everyone is always getting owned, then there's no incentive to distinguish yourself as a company with security. Clearly, no one cares. There's obviously no demand. Why waste the money? Just check the boxes and pay the fines. They're cheaper than the security spend anyway. Welcome to the yolo economics of late stage capitalism.

But we can only be forced to accept this state of things if we don't have any alternative. Which is, of course, why “AI” is trashing open source right now. I'm not saying it's intentional or that it's a conspiracy. It's not. But there is a systemic incentive to destroy those commons, to drain them. There have been very intentional efforts in the past, specifically by Microsoft, to “embrace, extend, extinguish” open source projects. LLMs provide an opportunity to hypercharge that.

Basically every company relies on open source at some level, and for most it's almost their whole stack. Most of them don't give back. Everyone freaks out when a tiny bit of investment into security reveals a bunch of kernel bugs. Yeah, “many eyes make bugs shallow” only works if people are actually putting eyes on it. Commons have to be maintained. People have to put resource in, not just take them out. And now we're seeing the extractive collapse of our critical digital commons.

Well, I say people need to put resources in, but it actually matters what goes in. If you take water from a lake and dump sewage in, it's not exactly “managing the commons.” But yeah, that raw sewage into our metaphorical lake is also happening at the same time. I don't know what the digital equivalent of cholera is, but get ready for it.

So there we have it: enclosure/destruction of the commons, attacks on skilled labor, centralization of power, growing authoritarianism. They keep saying it's like the industrial revolution, and, yeah, there are definitely some parallels. Those aren't the only ones, I assure you.

What do we do about it? Yeah, I don't fucking know. I'm just throwing this out here because no single one of us is going to figure it out alone. We're only going to create a better world if we know what we're up against, and we choose to build it together.