The Evil That Is AI Child Porn – Charles Fain Lehman

Child pornography—today usually called “child sexual abuse material,” or CSAM, in recognition of its uniquely violent character—is one of the few things today still subject to near-universal revulsion and taboo. But AI-generated CSAM poses a novel, and important, challenge to our law and culture. As a result of a pivotal Supreme Court decision two decades ago, it is possible, and indeed likely, that the government cannot prohibit the possession or distribution of at least some of it—because, according to the court, it is First Amendment-protected speech.

The rise of increasingly sophisticated video models puts this legal fact on a collision course with reality. But to deal with it, we may need to revise not only our laws, but our concepts of speech itself.

The fight over virtual CSAM is not new. The Supreme Court established that real-life CSAM is not protected by the First Amendment in 1982’s New York v. Ferber. In that case, the court said that while CSAM was speech (like other visual depictions), it could be prohibited because its production is “intrinsically related to the sexual abuse of children,” and because of the state’s compelling interest in “safeguarding” children. Importantly, Ferber permitted proscribing CSAM even when it was not “obscene”—a separate, narrow category of unprotected speech reaching some but not all sexual content. 

Fast forward to 1996. Congress passed the Child Pornography Prevention Act (CPPA), which defined “child pornography” as a visual depiction that “is, or appears to be [emphasis added], of a minor engaging in sexually explicit conduct.” The law was meant to sweep up synthetic content which, Congress noted, was possible with then-new technologies. Apparently, this prohibition went too far for the pornography industry. The Free Speech Coalition, an industry trade group, brought suit, alleging that their members’ free speech rights were being infringed.

Remarkably, the Supreme Court agreed. In 2002’s Ashcroft v. Free Speech Coalition, the majority asserted that virtual child pornography was not covered by Ferber, because no real children were harmed in its production. Nor could it be categorically prohibited as obscene; to establish any given piece of content as obscene, prosecutors would have to show it met the court’s elaborate test for obscenity. 

The government’s other arguments that virtual CSAM should not be protected were all likewise rejected. Consequently, the “appears to be” section of the CPPA was overturned. Congress responded to the ruling with the 2003 PROTECT Act, which prohibits artificial content that is “indistinguishable” from authentic content. 

The PROTECT Act remains on the books today. But the Supreme Court has never had occasion to rule, one way or another, if this provision is good law. And the rise of artificial intelligence raises new, and increasingly pressing, questions about AI-generated CSAM.

It is not the case that computer-generated CSAM is always protected speech. In a helpful review published by Lawfare, Stanford Internet Observatory researcher Riana Pfefferkorn explains that some CSAM is generated either by algorithms that used real CSAM images in their training data, or by “morphing” images of real children for perverted purposes. In both cases, harm to a real child happens, and so the content can be proscribed under Ferber. 

The PROTECT Act separately prohibits both such content and “obscene” content depicting children, a prohibition which would likely survive constitutional scrutiny. In this context, obscenity has the legal definition given to it by the court in 1973’s Miller v. California. For a work to be obscene under Miller, a jury or judge must find that it “appeals to the prurient interest,” “depicts or describes, in a patently offensive way, sexual conduct,” and “lacks serious literary, artistic, political or scientific value.” 

It seems like this standard ought to cover all artificial CSAM. But multiple appeals courts have found instances of virtual child pornography they consider not obscene, arguing that they do not meet the extremely high (and extremely malleable) standard of “patently offensive.” And the “social value” prong may protect depictions of CSAM with some nexus to cultural issues—consider a simulated, uncensored production of Lolita. Mere possession of obscene material, as opposed to its production or distribution, is not prosecutable—unlike possession of CSAM. And obscenity prosecutions are rare, in part because of the challenge of proving material meets the Miller standard.

The conclusion, Pfefferkorn argues, is that there exists some computer-generated CSAM which cannot be legally proscribed. “If a piece of [computer-generated CSAM] was not generated using any actual abuse imagery, does not depict a real child, and is not legally obscene,” she writes, “then it is likely to be deemed protected speech.”

It would seem at first glance that there should be relatively little content that falls into this category. But the advent of AI-generated video means that producing such content in bulk will soon be trivial—and there will be nothing stopping the companies that the Free Speech Coalition fights to protect, like Pornhub parent company Aylo, from selling it. Given their problematic history with distributing CSAM, some may choose to do so (recall that they sued for the right to produce pornography that appears to show minors!). And even if they don’t, someone else will—something the government will have no recourse to prevent, because the content will be protected speech.

Some provocateurs and libertarians, of course, ask: Why is this an issue? If no children are actually harmed in the making of purely synthetic child pornography, then what is wrong with its production or consumption? We may find it unseemly, but is that a valid basis for regulation? Some take this view even further, arguing that synthetic CSAM would act as a substitute for the real stuff—making it almost morally mandatory to make it widely available. 

There are, of course, harm-based rebuttals to this view. Virtual and real CSAM can act as complements as well as substitutes, in much the same way that other addictive products do—people escalate from oxycodone to fentanyl, e.g. Similarly, consumers of CSAM often escalate to actual abuse. Moreover, some societies have treated pederastic conduct as acceptable, and it is primarily through a dense web of legal and social proscriptions that we stigmatize it. Widespread availability of virtual CSAM would almost certainly undermine that stigma.

Such arguments may be true, but they are also inadequate. The two views at play are essentially consequentialist, and turn therefore on which consequences would occur in the event that virtual CSAM was widely available. These arguments are indeterminate, at least and until we actually see the flood of virtual CSAM—at which point it will be too late.

Yet the idea of widely available AI-generated child pornography is horrifying not primarily for its consequences, but because the production, consumption, and mere existence of such content is wrong in and of itself. Our revulsion speaks to this intrinsic wrongness, which goes beyond the wrong of harm. As Leon Kass put it in his classic essay, “The Wisdom of Repugnance”:

Like cannibalism, or incest, or zoophilia, or “just” raping and murdering, the production of CSAM is wrong not merely because it causes harm, but because that harm proceeds from the intrinsic wrongness of the act. CSAM makes a mockery of a fundamental reality: that children are not adults, and that that categorical distinction is produced in large part by the fact that children are not physically or psychologically capable of engaging in sex (and procreation, sex’s natural consequence). “Having sex with a child” is a contradiction in terms, in much the same way that treating a person as food, or treating a family member as a lover, or an animal as a sex object, is a contradiction in terms. It is from this contradiction that the violence of the act springs, and which provokes revulsion even when those acts are presented as somehow less than monstrous.

The problem is that mere revulsion is not, at present, a basis for proscribing speech. Indeed, the court has moved away from such a subjective standard at the exact moment that revolting speech is likely to become significantly more common.

When is speech not protected by the First Amendment? The now-conventional answer, reiterated by the Supreme Court in 2010’s U.S. v. Stevens, is that there are certain “historic and traditional categories” that have been outside the First Amendment’s protection since 1791. These categories—obscenity, defamation, fraud, incitement, “true threats,” “fighting words,” and speech integral to criminal conduct—are the extremely narrow exceptions to the First Amendment’s otherwise broad guarantee. This appeal to history and tradition is consistent with the court’s “originalist” turn under Chief Justice John Roberts, under which the conservative justices—trying to avoid legislating from the bench—attempt to interpret the Constitution using the meanings assigned to its words at the time of their writing. 

Unfortunately, as the University of Chicago’s Genevieve Lakier has observed, the Stevens schema is not actually consistent with the “original public meaning” of the First Amendment. Rather, at the Founding and for a century and a half afterwards, the First Amendment was taken largely to prohibit pre-publication censorship, and to protect, as Stanford’s Jud Campbell explains, “well-intentioned statements of one’s views” on matters of public concern (a result of the Founders’ natural law theory). Government could, and often did, prosecute speech after the fact when it was thought to have a mere “tendency” to incite lawbreaking, create a breach of the peace, or subvert public morals and decency. 

The concept of distinct categories of “low-value” speech, Lakier argues, really dates to 1942’s Chaplinsky v. New Hampshire. In that case, the court simply asserted that there were “certain well defined and narrowly limited classes of speech, the prevention and punishment of which have never been thought to raise any Constitutional problem.” Such classes, the court said, “are no essential part of any exposition of ideas, and are of such slight social value as a step to truth that any benefit that may be derived from them is clearly outweighed by the social interest in order and morality.”

For decades after, Lakier and Stanford’s Evelyn Douek have written, courts took it upon themselves to determine when other kinds of speech were not of “social value as a step to truth,” applying the balancing test articulated in Chaplinsky. In Ferber, for example, the court found that child pornography was outside the protection of the First Amendment despite its inconceivability to the Founders. 

Another such category was at question in Stevens, in which the Obama administration asked the court to find that videos of animal cruelty were speech of “slight social value.” The court’s stark rejection of such judgment, and its appeal to an invented history, was—as Lakier and Douek show—actually the less historically-informed posture. More importantly, it left the court unable to revise the categories of unprotected speech, instead obliging lower courts to shove new exceptions into the pre-existing “traditional” categories, as in the court’s 2023 attempt (in Lakier and Douek’s telling) to redefine stalking by speech as a kind of “true threat.”

Which brings us back to AI-generated CSAM. To allow the state to fully control it, the court would need to repudiate Ashcroft, the case which protected artificial CSAM. To do so, it would need to admit that AI CSAM should be proscribable even when it is neither obscene nor based on real images, thereby creating a new category. It would need to conclude, in other words, that AI CSAM is of such “slight social value as a step to truth that any benefit that may be derived from them is clearly outweighed by the social interest in order and morality.” 

This is obviously true, yet the court has prohibited itself from saying so. This is why we are left struggling to shove virtual CSAM into the preexisting, inadequate categories—because we cannot just come out and say that it is always and everywhere wrong, and should be proscribed as such.

This may all seem like Supreme Court minutiae. But it cuts to a deeper social and cultural problem with how we think about free speech. After all, the consequentialist dispute over the harm of virtual CSAM masks an unwillingness to say that the product is wrong in and of itself. And we are afraid to do so because we do not wish to admit that certain kinds of speech are wrong in and of themselves.

But what the rise of virtual CSAM demonstrates is that refusing to make a judgment is itself also a judgment. And refusing to name actual evil is a poor strategy for protecting our society from that evil.