Thanks for this really nice piece, Sam. In some ways, the LLM discussion increasingly reminds me of Jurassic Park, when the creator of the park, John Hammond, and the scientist, Ian Malcolm, have a clash over the ability to control and even the desire to. Hammond, seeking to make a massive buck, uses the epic science of pseudo-genetics to create a fantastical place which appears to transcend anything that has occurred before. It is only when Malcolm begins to question Hammond's very assumptions and longing for expansive control and evolution that we see things begin to fall apart. The unanticipated human scenarios of greed, clumsiness, and lack of genetic control create a downstream motion of catastrophe, which eventually leads to the park being abandoned.
I think the hope of expansive abundance is seen as good because we assume that things bring us other things. It almost reminds me of the fraudster Jack Abramoff's argument that God wanted us to be wealthy. Like televangelists who con their parishioners into giving them tithes to support their elaborate lifestyles, we assume that cash will give us access to a partner, new experiences, and ergo a 'better life'.
Reading Knut Hamsun's Hunger, re-reading Crime and Punishment and embarking on The School of Night all remind me of the kind of problem you're teasing out here. Just how far are we willing to go to embrace our desires, and at what point do we recognise that what we believe we want perhaps is not what we truly want? I tend to agree with your last point though that we cannot go backwards. Philosophers such as Rousseau have acknowledged this truth and when it's been attempted (by people such as Saint Just) it has ended in anarchy, bloodshed and tyranny.
On the question of agency, I have no doubt that there are many areas where technology diminishes it. Reading countless essays that are all the same is not the curation of agency but its annihilation at the college level. Sure, they may be able to produce poor-quality work at an unbelievably quick rate, so they can do other things, but this diminishes their ability to fulfil themselves in the future. It also lessens their spirit and denies them the opportunity to push their own boundaries to gain a better understanding of what they truly desire. If you never try, then how can you know if you ever want to?
In this case, I am less of the opinion that AI represents a reach for the great frontier than the belief that all frontiers have now been surpassed. It is the epitome of laziness and boredom that is leading to a reliance upon this technology. We are slumbering into a new age of stupidity where we are at liberty to engorge ourselves without realising the consequences. The risk seems to me less than the ability to do something catastrophic, but not doing anything at all in the medium run.
Interesting thoughts! Jurassic Park and Frankenstein have in common that the technologies in question were being sought deliberately; we aren't shown the build-up of knowledge that might unintentionally have made the scientific quest possible in the first place. This is appropriate as an analogy for LLMs, given that artificial intelligence has indeed been a possible aim ever since Turing et al pioneered the idea of a truly flexible logic machine. The possibility that this might be a very bad idea has, likewise, existed for decades, at least if you take Dune's backstory about a "Butlerian Jihad" as an indication.
By contrast, there's a disanalogy with nuclear weapons. In that case, the work done on radioactivity was mostly about understanding the nature of matter, and the crucial mass-to-energy possibility was a side effect of relativity. Even the possibility of a nuclear chain reaction wasn't conceived of until as late as 1933 (by Leo Szilard), let alone any practical way to create one. Almost all the developmental work was done after WWII had begun. There's plenty to criticize, but it's a fast-moving tragedy in a hot war. Nuclear weapons were neither the calculated strategy of a capitalist nor the ill-advised quest of a lonely scientist driven to succeed.
Given the involvement of capitalism, therefore, Jurassic Park is a pretty good comparison! Mind you, I find Ian Malcolm unconvincing as a scientist. I wasn't allowed to watch the movie as a kid, so by the time I saw it I'm afraid I already knew about chaos theory already, and, as xkcd puts it (https://xkcd.com/1399/), "there is nothing in here about dinosaurs escaping." Malcolm strikes me as a bit of a poseur, as a result. (It probably doesn't help that he spends much of the early scenes literally posing with his shirt off. I'd find him more attractive if he understood the math he's referencing. Maybe that's just me).
Honestly, though, there's a serious point, here, which is that, while science can certainly tell us something about the risks we already understand, truly unanticipated consequences aren't the sort of thing you can usually go to scientists for. Such things call for wisdom, more than knowledge, I think; humility rather than chaos theory. Philosophers are probably better suited to that task, or at least they ought to be.
You remind me of a piece that Alan Jacobs wrote a while back. It has a lot going on in it, but the motif that has stuck with me is the part where he draws on a Daoist utopia in which technologies exist but are not used.
There something challengingly witty about the very idea of a society that has the ability to speed itself up, and yet doesn’t. How hard can it be, to not do a thing? Easy, one might think: you simply don’t do it. But Daoism is a perpetual reminder of the unexpected power of negative space. Not doing things is in fact a difficult art.
It’s also a little mind-boggling to realise that the technologies being renounced are (by our standards) primitive and preindustrial! Returning to knotted cords as an accounting system? Nobody tell this man about computers…
As often happens with these kinds of recommendations, I find more value in seeing the existence of the negative space than in insisting on its use in all situations. The freedom not to do a thing can be hard to see, and correspondingly powerful to have in your toolkit, even if sometimes you’d still choose to act.
I quite like a lot of technology; to take a simple example, I’m unequivocally in favour of kitchen appliances that enable food production as a craft/art, alongside more control of what we eat. Making food for people is also a traditional act of love. What’s not to like? On the other hand, like many, I find my smartphone more ambiguous, and, also like many, I still use it constantly. I can say without regret, however, that I don’t have a car. This is a highly situational choice based on where I live and where I most often need to go, but for me not having a car is a freedom I am glad to be able to have.
Many of the most obviously positive uses of technology are precisely those that enable genuinely creative work. Technologies that help you make things for yourself and people you know can be very nice! They get nicer when paired with technical skill; a sewing machine paired with the expertise (and time) necessary to make your own clothes is a ticket to both personal self-expression and utilitarian quality of clothing that can be hard to achieve in any other way save, I suppose, being even more wealthy and paying a tailor. Musical instruments are nice; again, they require skill. Kitchen appliances have already been mentioned.
Using technology positively is a virtue in many classic ways: dependent on both skill and self-restraint, surprisingly easy to justify on purely selfish grounds, and rather awkwardly aristocratic to recommend, if you’re not careful how you do it. Not that such instincts have always been incompatible with poverty; my mother, as a young woman, aspired to a bicycle and a sewing machine, on grounds that a car might be beyond her means. But industrially made clothes are cheaper than they used to be, and good quality fabric is more expensive, so I suspect her aspirations would no longer carry the sense of economy that they once did, in this day and age.
Personal virtue hardly seems like a complete answer to the social problems of technological acceleration, however. Given enough self-control and personal freedom—including money—a person can arrange a life in which technologies are largely employed to improve life rather than drain it of meaning. But this often involves at least a partial relinquishing of further power: the time taken to live a little more slowly is time not taken to earn even more money and pursue social influence. On a personal level I think it’s probably worth it. On a social level, many people will decide it’s not worth it, or else fail at it, or not even see that they have the choice. Those with the most power and influence and meritocratic “merit” are likely to be among these. And as you note, when it comes to political or economic competition, whether within countries or between them, the threat of being overpowered gives a kind of futility to many kinds of renunciatory wisdom.
Then, too, that money already mentioned: does it come in part from an unequal society in which some simply do not have the power to arrange their lives more comfortably? Quite possibly. The path I am outlining/recommending/following-with-mixed-success is a middle path of middle class restraint. Societies can make it more or less visible/possible, but given that the American middle class isn’t exactly in great shape it’s not obviously plausible to make it dominant there.
Unlike Alan Jacobs’ anarchism, I suspect the nearest political match to the technological attitude that I am describing would be a bread-and-roses democratic socialism, paired with the kind of recommendation of personal virtue that would have been characteristic of, say, the progressive Christian socialists of the early twentieth century. But if LLMs mean that most people just don’t have the power to get the resources they need, I’m not sure any kind of democratic socialism can really get off the ground. Renunciatory anarchism might be slightly more likely, even! Still, I must admit, I do sort of like the idea of bread and roses and self-improvement for everybody.
Great insight here, and good to revisit MLK's insight. I've thought a lot in this vein when it comes to birth control and vasectomies (wildly modern tech!), the largest protestant blind spot I see next to women's ordination.
As for AI. Fortunately the free hand of the market will see it tumble. There's little value in the glorified google we've created for MOST white collar work. Admittedly there's some. But the reason Open AI is doing Ads likely has more to do with the dearth of profits that the company is making and will make for the foreseeable future.
Crux of the matter again, Samuel! We have no choice but to go further in. Creativity is our essence and grounds meaning. Technology is part of nature and AI our aesthetic fate. We are here to produce it.
Thanks for this really nice piece, Sam. In some ways, the LLM discussion increasingly reminds me of Jurassic Park, when the creator of the park, John Hammond, and the scientist, Ian Malcolm, have a clash over the ability to control and even the desire to. Hammond, seeking to make a massive buck, uses the epic science of pseudo-genetics to create a fantastical place which appears to transcend anything that has occurred before. It is only when Malcolm begins to question Hammond's very assumptions and longing for expansive control and evolution that we see things begin to fall apart. The unanticipated human scenarios of greed, clumsiness, and lack of genetic control create a downstream motion of catastrophe, which eventually leads to the park being abandoned.
I think the hope of expansive abundance is seen as good because we assume that things bring us other things. It almost reminds me of the fraudster Jack Abramoff's argument that God wanted us to be wealthy. Like televangelists who con their parishioners into giving them tithes to support their elaborate lifestyles, we assume that cash will give us access to a partner, new experiences, and ergo a 'better life'.
Reading Knut Hamsun's Hunger, re-reading Crime and Punishment and embarking on The School of Night all remind me of the kind of problem you're teasing out here. Just how far are we willing to go to embrace our desires, and at what point do we recognise that what we believe we want perhaps is not what we truly want? I tend to agree with your last point though that we cannot go backwards. Philosophers such as Rousseau have acknowledged this truth and when it's been attempted (by people such as Saint Just) it has ended in anarchy, bloodshed and tyranny.
On the question of agency, I have no doubt that there are many areas where technology diminishes it. Reading countless essays that are all the same is not the curation of agency but its annihilation at the college level. Sure, they may be able to produce poor-quality work at an unbelievably quick rate, so they can do other things, but this diminishes their ability to fulfil themselves in the future. It also lessens their spirit and denies them the opportunity to push their own boundaries to gain a better understanding of what they truly desire. If you never try, then how can you know if you ever want to?
In this case, I am less of the opinion that AI represents a reach for the great frontier than the belief that all frontiers have now been surpassed. It is the epitome of laziness and boredom that is leading to a reliance upon this technology. We are slumbering into a new age of stupidity where we are at liberty to engorge ourselves without realising the consequences. The risk seems to me less than the ability to do something catastrophic, but not doing anything at all in the medium run.
Interesting thoughts! Jurassic Park and Frankenstein have in common that the technologies in question were being sought deliberately; we aren't shown the build-up of knowledge that might unintentionally have made the scientific quest possible in the first place. This is appropriate as an analogy for LLMs, given that artificial intelligence has indeed been a possible aim ever since Turing et al pioneered the idea of a truly flexible logic machine. The possibility that this might be a very bad idea has, likewise, existed for decades, at least if you take Dune's backstory about a "Butlerian Jihad" as an indication.
By contrast, there's a disanalogy with nuclear weapons. In that case, the work done on radioactivity was mostly about understanding the nature of matter, and the crucial mass-to-energy possibility was a side effect of relativity. Even the possibility of a nuclear chain reaction wasn't conceived of until as late as 1933 (by Leo Szilard), let alone any practical way to create one. Almost all the developmental work was done after WWII had begun. There's plenty to criticize, but it's a fast-moving tragedy in a hot war. Nuclear weapons were neither the calculated strategy of a capitalist nor the ill-advised quest of a lonely scientist driven to succeed.
Given the involvement of capitalism, therefore, Jurassic Park is a pretty good comparison! Mind you, I find Ian Malcolm unconvincing as a scientist. I wasn't allowed to watch the movie as a kid, so by the time I saw it I'm afraid I already knew about chaos theory already, and, as xkcd puts it (https://xkcd.com/1399/), "there is nothing in here about dinosaurs escaping." Malcolm strikes me as a bit of a poseur, as a result. (It probably doesn't help that he spends much of the early scenes literally posing with his shirt off. I'd find him more attractive if he understood the math he's referencing. Maybe that's just me).
Honestly, though, there's a serious point, here, which is that, while science can certainly tell us something about the risks we already understand, truly unanticipated consequences aren't the sort of thing you can usually go to scientists for. Such things call for wisdom, more than knowledge, I think; humility rather than chaos theory. Philosophers are probably better suited to that task, or at least they ought to be.
You remind me of a piece that Alan Jacobs wrote a while back. It has a lot going on in it, but the motif that has stuck with me is the part where he draws on a Daoist utopia in which technologies exist but are not used.
https://www.thenewatlantis.com/publications/from-tech-critique-to-ways-of-living
There something challengingly witty about the very idea of a society that has the ability to speed itself up, and yet doesn’t. How hard can it be, to not do a thing? Easy, one might think: you simply don’t do it. But Daoism is a perpetual reminder of the unexpected power of negative space. Not doing things is in fact a difficult art.
It’s also a little mind-boggling to realise that the technologies being renounced are (by our standards) primitive and preindustrial! Returning to knotted cords as an accounting system? Nobody tell this man about computers…
As often happens with these kinds of recommendations, I find more value in seeing the existence of the negative space than in insisting on its use in all situations. The freedom not to do a thing can be hard to see, and correspondingly powerful to have in your toolkit, even if sometimes you’d still choose to act.
I quite like a lot of technology; to take a simple example, I’m unequivocally in favour of kitchen appliances that enable food production as a craft/art, alongside more control of what we eat. Making food for people is also a traditional act of love. What’s not to like? On the other hand, like many, I find my smartphone more ambiguous, and, also like many, I still use it constantly. I can say without regret, however, that I don’t have a car. This is a highly situational choice based on where I live and where I most often need to go, but for me not having a car is a freedom I am glad to be able to have.
Many of the most obviously positive uses of technology are precisely those that enable genuinely creative work. Technologies that help you make things for yourself and people you know can be very nice! They get nicer when paired with technical skill; a sewing machine paired with the expertise (and time) necessary to make your own clothes is a ticket to both personal self-expression and utilitarian quality of clothing that can be hard to achieve in any other way save, I suppose, being even more wealthy and paying a tailor. Musical instruments are nice; again, they require skill. Kitchen appliances have already been mentioned.
Using technology positively is a virtue in many classic ways: dependent on both skill and self-restraint, surprisingly easy to justify on purely selfish grounds, and rather awkwardly aristocratic to recommend, if you’re not careful how you do it. Not that such instincts have always been incompatible with poverty; my mother, as a young woman, aspired to a bicycle and a sewing machine, on grounds that a car might be beyond her means. But industrially made clothes are cheaper than they used to be, and good quality fabric is more expensive, so I suspect her aspirations would no longer carry the sense of economy that they once did, in this day and age.
Personal virtue hardly seems like a complete answer to the social problems of technological acceleration, however. Given enough self-control and personal freedom—including money—a person can arrange a life in which technologies are largely employed to improve life rather than drain it of meaning. But this often involves at least a partial relinquishing of further power: the time taken to live a little more slowly is time not taken to earn even more money and pursue social influence. On a personal level I think it’s probably worth it. On a social level, many people will decide it’s not worth it, or else fail at it, or not even see that they have the choice. Those with the most power and influence and meritocratic “merit” are likely to be among these. And as you note, when it comes to political or economic competition, whether within countries or between them, the threat of being overpowered gives a kind of futility to many kinds of renunciatory wisdom.
Then, too, that money already mentioned: does it come in part from an unequal society in which some simply do not have the power to arrange their lives more comfortably? Quite possibly. The path I am outlining/recommending/following-with-mixed-success is a middle path of middle class restraint. Societies can make it more or less visible/possible, but given that the American middle class isn’t exactly in great shape it’s not obviously plausible to make it dominant there.
Unlike Alan Jacobs’ anarchism, I suspect the nearest political match to the technological attitude that I am describing would be a bread-and-roses democratic socialism, paired with the kind of recommendation of personal virtue that would have been characteristic of, say, the progressive Christian socialists of the early twentieth century. But if LLMs mean that most people just don’t have the power to get the resources they need, I’m not sure any kind of democratic socialism can really get off the ground. Renunciatory anarchism might be slightly more likely, even! Still, I must admit, I do sort of like the idea of bread and roses and self-improvement for everybody.
Great insight here, and good to revisit MLK's insight. I've thought a lot in this vein when it comes to birth control and vasectomies (wildly modern tech!), the largest protestant blind spot I see next to women's ordination.
As for AI. Fortunately the free hand of the market will see it tumble. There's little value in the glorified google we've created for MOST white collar work. Admittedly there's some. But the reason Open AI is doing Ads likely has more to do with the dearth of profits that the company is making and will make for the foreseeable future.
Crux of the matter again, Samuel! We have no choice but to go further in. Creativity is our essence and grounds meaning. Technology is part of nature and AI our aesthetic fate. We are here to produce it.
https://apablog.substack.com/p/cosmic-purpose-creation-and-individual