How America can safeguard human agency and dignity during the AI revolution
For good reason, humans fixate on detecting and preventing possible catastrophes. For good fun, we speculate about unrecognizably transformed worlds. These tendencies have shaped discussion of AI’s promise and peril in recent years, generating all manner of extreme scenarios, positing so-called x-risks, and planning for radical responses in case the emergency calls for breaking the glass. Those conversations can be worthy ones.
Consequently, though, not enough attention focuses on the impacts that AI will certainly have or, in some cases, is already having. Regardless of whether AI is “normal technology” or “this time is different,” it will interact in more predictable ways with human nature, values and priorities, social dynamics, network effects, and information asymmetries. Legible, longstanding forces will shape the development and adoption of this technology and its effects, both good and ill, on relationships, communities, and vocations. Collective decisions, made through political processes, will determine whether nations succeed in choosing the better paths.
What are the principles that hold true and the values worth preserving regardless of technological context? That question is best asked, and answered, behind a veil of ignorance about the specific choices and trade-offs ahead. Enormous efficiency gains will be on offer, and each in isolation will be tempting. But as we have seen with the diffusion of screens and social media into all of life’s nooks and crannies, the cumulative effect of succumbing to so many temptations can be a slide into dependence and lasting harm. AI will be far more challenging, and there may be no recovering if we get it wrong.
The Gattaca Effect
We worry especially about those unchosen impositions of technology that one of us has called the “Gattaca Effect.” In Gattaca, the dystopian 1997 film starring Ethan Hawke and Uma Thurman, a society that has mastered genetic modification organizes itself into castes according to who can afford the most sophisticated customizations of their children. In theory, one might “opt out” and choose natural procreation; in practice, this only condemns one’s family to the lowest rung of the society’s ladder. New technology introduces such effects from time to time, with participation remaining formally voluntary but a prerequisite for full participation in communal, economic, and national life. With AI, it will happen repeatedly.
One channel for the Gattaca Effect’s operation is relational. Human relationships are inherently reciprocal and work only when a person is willing to accept some burdens or give some part of himself to another, and others are willing to do the same. A technology perceived as providing the goods of relationships without requiring the effort will become attractive to many, and, as more make the choice, opportunities for human connection will decline for everyone else. We have already witnessed a relatively mundane example of the effect in multiplayer video gaming, which has steadily shifted norms of friendship among young men from in-person interaction to online game chats. The threat posed by virtual friends who provide unending support and never ask anything in return is orders of magnitude larger.
Another channel is competition. As in Gattaca, a technology that gives an advantage to its users can become irresistible for everyone else in the game, producing a race to the bottom that all might have preferred to avoid. Beneficial uses of technology can prompt races to the top, as when machinery that improves productivity and product quality becomes a must-have for all competitors. But the technology makes no distinction in direction. Offshoring production became a must-have, even as the companies undertaking it recognized in many cases that they were weakening their long-term position, and their nation’s, in the process.
The third channel is intelligibility. As technology becomes incorporated into a process, those without the technology often find themselves excluded. Literacy is a technology that made all manner of institutions more effective, but it also worsened the position of the illiterate. Roads dominated by cars become unsafe for people who might prefer to walk. As we allow the most efficient mode for completing a task to redefine the nature of the task itself, we increasingly leave only one path to participation.
Whatever your views about AI’s future power, these effects across various domains seem certain. As a substitute for human relationships, AI promises to immiserate not only those who initially fall into its clutches, but also everyone else whose social horizons will narrow. As a productivity tool, it is already placing students in an untenable position of having to choose between cheating and falling behind. As it writes more code, more legal briefs, and more résumés that are then processed by AI screeners in a degrading arms race, doing things the old-fashioned way ceases to be an option. The AI’s power necessitates handing the AI more power.
Red Lines and Principles
Who cares? Technological change brings with it social, economic, and political change—some good, some bad; generally, on balance, an improvement in the human condition. But judging from early returns and current trajectory, not to mention the predictions of the technology’s developers and evangelists, we should not naively apply that template to AI. Whether one sees it as an eerily effective imitator of human language or as an alien and conscious intelligence poised to surpass our own, its repeated ability to master increasingly difficult tasks, which were until yesterday the exclusive domain of the human mind, presents the risk of crossing two very serious red lines.
First, we must retain our agency to make our own decisions and shape the world around us. In short, we must remain in control. We can delegate tasks to machines, whether physical or mental, but we must be able to understand what the machines are doing and, if necessary, do it ourselves. One corollary of remaining in control is that we must maintain expertise, which can be a difficult thing to hold in reserve. Like any muscle, it is developed and maintained through rigorous use, which can be difficult to undertake when it is not necessary in the ordinary course of business.
A special case of expertise is literacy, which already appears at risk in large segments of the population. Near-universal literacy is the hard-won accomplishment of centuries, and the sine qua non of representative government. For liberty and self-determination to persist, for people to build decent lives and reach their own full potential, they must have the ability to communicate through and think critically about the written word. As it becomes less necessary for getting by in everyday life, literacy becomes more necessary to teach actively, to employ, and to cherish.
Second, we must assert that human beings are uniquely possessed of inherent dignity, that we need and are entitled to find purpose outside ourselves, and that our civilization’s purpose is the advancement of human flourishing. We can interact in many ways with our machines, and make use of them to do many things for us, but we cannot confuse or treat them as interchangeable with other humans or fool ourselves into believing that we can have real relationships with them. Nor can we allow them to provide for all of our needs, reducing our own role to consuming some share of what they produce for each other.
Agency and dignity come with no conventional economic value. They are not market outcomes, and they cannot be measured. And so, by default, they will be overlooked, traded off, and undermined in deliberations over acceptance of some technological benefit or efficiency that seems immediately more important than a vague sense of something lost. Protecting them must be a red line: an absolute commitment that we will not breach no matter how tempting it will seem.
Directions for Policy
As troubling as these reflections may seem, they have one advantage over many debates in AI policy, which focus on the catastrophic speculations that would render the world unrecognizable. To the extent that AI may deform our human practices and institutions in ways familiar to us from past experience and response, we should be confident that we have the power to choose which course we follow, if we are bold enough to use it. Policymakers can take steps now, using ordinary tools, to block the worst paths AI adoption might take and mark out the best ones. At American Compass, we are at the start of our own work developing these ideas; here are a few initial proposals that we hope will inspire further thinking:
- Prohibiting Human Form Factors. Little if any of AI’s social value, either today or in any forward-looking forecast, requires an interface that presents itself as human to the user. And yet firms continue to direct countless billions of dollars toward not only anthropomorphic chatbots, but also lifelike voice agents indistinguishable from human voices and AI-generated video models that imitate real or imagined human beings. The result is a pervasive blurring of reality and simulation that steadily seduces users toward the latter, which provides a first-order approximation of human interaction without the second-order burdens of considering another’s well-being or serving them in any way. A humanlike AI thinks you’re hilarious, never tires of talking about what you want to talk about, and never has problems of its own, unless you want it to, in which case those problems will exist only until the moment you want them to go away.
Even ordinary adult users can find the experience of interacting with a bot to be uncanny, disorienting, and prone to compromising their judgment, while more vulnerable users regularly fall into parasocial relationships. Many AI boosters seem to assume this is just an awkward transitional phase of the technology, or perhaps even valuable for those whose “demand” for friends exceeds the current “supply.” More likely, though, we will increasingly normalize such relationships and build them into education and health-care systems, leaving us all with fewer opportunities to benefit from real relationships with others and lessening obligation to invest in them.
Chatbots can just as easily perform productive tasks without pretending to have emotions or form relationships, and even provide reminders at regular intervals that they are machines. Voice agents can get the job done while sounding like C-3PO, rather than a sexy secretary. Robots have no need for human faces. Creating and selling products that do try to blur the line between human and machine should be illegal. - Licensing. Some potentially beneficial uses of humanlike interaction do exist, for instance if the AI is playing the role of a therapist. But such a product comes with greatly heightened risk and should be presumptively illegal unless specifically licensed and supervised. More broadly, licensure has an important role to play in both the functions that AI might perform and the tools that users might access.
On the performance side, expert advisors from doctors to lawyers to money managers demonstrate their knowledge of relevant material, compliance with ethical standards, and ability to perform their roles before they are permitted to do so on behalf of clients. Artificial systems capable of playing comparable roles should either be subjected to comparable licensing and fiduciary requirements or else be subordinated to accountable licensed humans. As AI moves further into the physical world, participating on construction sites, installing or using heavy equipment, transporting materials, and so on, products performing those tasks should likewise have to demonstrate the same abilities as any humans who might be involved.
For users, deploying a powerful model can be every bit as dangerous—to them and to others—as driving a dump truck through downtown Manhattan. Users should be required to demonstrate their ability to use advanced models appropriately before gaining access to them. Children should be presumptively prohibited from using them, and in many cases the elderly will need to lose access as their own faculties decline. The AI systems themselves should be responsible for refusing to complete prohibited tasks or assist unqualified users, just as another person would be. As a rule of thumb, any system intelligent enough to perform the kinds of tasks humans perform in the real world should also be intelligent enough to perform the kinds of safety checks that humans routinely perform themselves. - Liability. The English common law, which provides the foundation for the American legal system, operates on the principle that everyone is potentially responsible and someone is always responsible. The same must hold for AI systems and agents. Responsibility for any action taken by an AI system must be attributable to a legally accountable human person—whether that is the model creator, the computing infrastructure provider, the interface developer, or the prompter—and that person must bear liability as if he had taken the actions himself.
In addition to creating better incentives for responsible design and use, a robust framework for liability would have the benefit of creating a space and a need for expertise in overseeing and signing off on whatever AI systems may do. At a minimum, a person will always have to take responsibility for the final product, which will likely require that others be able to vouch for the process steps that led to that product, and others still for the tools that performed those steps. - Legal Reform. There are few areas where human agency and dignity are more at stake than in the domain of self-government, through making and administering the law. While fantasies abound for institutions made democratic, efficient, and unbiased through the infusion of AI, this is a classic case of “Chesterton’s fence”: Those who have not fully grasped the purpose of an institution do not have the right to tear it down.
The engagement by citizens and their representatives in the process of lawmaking is itself vital to the legitimacy of those laws, which must derive from legible and democratic processes if free people are to accept and live under them. And although law ought to be as simple and understandable as possible, it must also be as complex as necessary to mediate the conflicting values and priorities that it governs. AI systems may find apparently easy and downright reductive solutions when they are trained on one set of values deemed best by the trainers—anyone can find such solutions quickly when only their own values count. But the unavoidably complex task of balancing interests and reaching compromise yields a body of law that, while often less “efficient,” stands a chance of helping humans adjudicate their disputes in a pluralistic society.
The frictions of human processing are themselves built into our legal infrastructure and are necessary for guaranteeing rights from participation to due process. Already, AI systems have begun to overload court systems with litigation claims, agencies with information requests, and regulators with comments. Unchecked, the result will be a retreat to AI systems that adjudicate the claims, respond to the requests, and address the comments. As that flywheel accelerates, humans will find themselves increasingly excluded. Algorithmic sentencing promises to marginalize the role of human judges in a quest for objectivity, but as the word “judge” implies, the subjectivity of human judgment is a feature, not a bug, of a justice system in which humans will place any faith.
While there is ample room for productive human-machine partnership in many legal and administrative domains, the human capacities for deliberation and adjudication are fundamental traits of our species and tenets of our civilization; the tasks must remain human ones. That commitment, in turn, requires that advocates are human and that their work is legible to a human reviewer. Rights to due process in court and redress in regulatory proceedings should attach only to humans. As with an insistence on human liability generally, ensuring that humans continue to play the central role in the legal domain will help to ensure that we retain the wisdom and expertise to govern ourselves. This must include support for stepping onto the lowest rung of the legal career ladder, if we are to be left with anyone prepared to climb higher. - Education. While many have despaired at what AI is already doing to schools, we see great hope in the potential for AI’s disruption to refocus education on its core purpose of equipping young people with the skills and values to build decent lives. Those topics or skills for which AI can most easily substitute for a true teacher and mentor, and for which AI will most easily allow students to circumvent the hard work of formation, are likely to be the same ones that are least important in an AI-infused world. Conversely, a greater emphasis on critical thinking, individual expression, agency, competence, and mastery of one’s physical world would be a welcome shift in the orientation of our public schools.
Educators and their communities should seize this opportunity to define anew what it is that students need to learn, beginning from the building blocks of reading, writing, and math that require little technology to teach or learn and then moving on to the habits of mind and understanding of the world that will serve people regardless of vocation, and the practical skills that will ensure they have one. The American culture is slowly shifting away from a “college for all” model to one that takes alternative pathways seriously, but AI should accelerate a more dramatic transition, to a default in which all students benefit from the combined academic and vocational training that the best technical high schools offer today.
For an AI policy community generally obsessed with the existential risks posed by a unique and rapidly advancing technology, this discussion may seem mundane, lacking in imagination, and focused in the wrong place. No question, AI’s potentially catastrophic consequences need attention. But regardless of whether AI proves to be closer to a normal technology or an entirely unprecedented one, it does pose the full set of challenges of a normal technology—and those are the ones most certain to arrive, in many cases already arriving, and most salient in everyday lives.
Alongside efforts to align and govern advanced AI systems at a technical level, we need to align them with the interests of ordinary Americans and maintain human governance of our most critical social and political institutions. Otherwise, extraordinary innovation will inadvertently immiserate. Democratic republics have demonstrated the ability to build legal infrastructure and institutions around new technology in ways that incorporate it beneficially into the lives of citizens. Indeed, the democratic republic itself is an innovation of this kind. But for a revolution to succeed, its principles and effects must operate in accordance with human nature, in a way that advances human flourishing. Revolutions intent upon defying or transcending such concerns are more likely to end with the guillotine.

