In the medieval French romantic history of Jean d’Outremeuse, we find Virgil in the full flower of his afterlife as a prophet and magician. In addition to such marvels as building a castle on an ostrich egg and articulating the orthodox Catholic faith as understood in fourteenth-century France, Virgil uses his supernatural arts to build some animated statues that guard his incorrupt corpse after his death. St. Paul shows up in Naples for some chit-chat with the wizard (“Maistre Virgile, beal amis, lais-moy entreir li-dedens por parleir a toy!”) and finds him unresponsive. He neutralizes the mechanical guards and approaches the body. At his touch, it disintegrates into dust, and the apparent poet behind the machines is gone.
Recent arguments about artificial intelligence and the writing business have put me in mind of this strange little episode. The strongest version of these arguments is similar to the claim that there is no ethical consumption under capitalism—that is, everyone in the game is implicated in A.I.’s destruction of human culture. Even if you do not use A.I. to generate your writing, it will become difficult to avoid using it for your research; even if you avoid it for your research, whatever you publish will be sucked into the gaping maw that is everywhere and nowhere.
I like a bit of doom myself, and I certainly agree that at least in the middle term A.I. will touch everything that the internet already touches—which is, as you may have noticed, an awful lot of what constitutes human life. But it is important not to conflate certain things that seem to me qualitatively different. I still think it is possible to be somewhat optimistic about the prospects for writers and writing. What follows is a loose collection of thoughts culled from my own experience as an editor.
It is first important to clarify what we dislike or find suspicious about A.I. writing. The first indesideratum is how bad AI prose tends to be. (Readers can supply their own examples.) The second and more serious issue is the compromise of writerly trust: when we discover that somebody has made use of A.I. in composing a piece, we feel that the person has “cheated.” Such writers have, at the very worst, put words on the page that they have not themselves written; in other cases they have expressed ideas they have not thought or drawn upon research they have not performed. The basic writer-reader compact depends upon the assumption that whoever puts his name on a piece of writing takes responsibility for it. He has thought through the arguments and reconsidered them in the process of writing. The internal dialectic, the testing and proving of thought, has occurred. If we learn that a writer has actually put his name on thoughts he has not thought, we become concerned that he may not actually have performed the rigorous process, that he is selling us something less than what he advertises. Only a human being can be held responsible for an action; you cannot shame, sue, or kill a computer or a network. When people take responsibility for things over which they do not exert control—automobiles, firearms, checking accounts—we tend not to trust them very much.
It is sometimes argued that the first problem, the style, can be developed away as A.I. becomes more powerful and responsive. I very much doubt it on practical grounds; as A.I. use increases, more and more of its training data will itself be generated by A.I., and while weights may be adjusted to correct the potentially vicious cycle, most of the people who are giving the thing feedback are something less than arbitres elegantiae. This is also why I am not too worried about the models gorging themselves on my own writing; the seventy thousand or so words I publish every year are an infinitesimal, almost unmeasurable fraction of the press releases and A.I.-generated news roundups published in the same period. The second problem is metaphysical; it can never be developed away.
Here is another important distinction. I do not think it is reasonable to conflate the use of A.I. with older versions of web-based research (hastily skimming Wikipedia, for example), or even to situate the two somewhere on a continuum. Working through material yourself, engaging in the internal dialectic, even quickly and inexpertly, fulfills the writerly compact. It is fundamentally different from serving the informational equivalent of mystery meat, which is what A.I. use tends toward. (Of course, a responsible writer could use A.I. rigorously with aggressive querying; I think few purists object to such use on strictly writerly grounds, even if they have other issues with it.)
One specifically writerly anxiety is that we editors will soon no longer be able to tell the difference between human and machine-written text. I am not so sure. Not all publications are the same. At a tony literary outfit like this one, the quality of the writing itself is part of the package. If bad writing comes through the editors’ inboxes, it doesn’t matter whether it is a spark thrown off by the A.I. plerouma or a clunker of purely human origins. The editors can and should reject it. The human reader must still be the ultimate judge of good writing; this is unlikely to change. But in many cases what is important is judging bad writing.
Here my own experience with A.I. detectors has been encouraging. I run a newsmagazine, which is a somewhat grimier affair. Newswriters, policy analysts, and the comms shops that produce political op-eds tend toward bad style, often in the same ways that A.I. tends toward bad style. Because of the nature of news media, I also move a great deal of copy—roughly twenty-five thousand words a week, plus thirty thousand-ish words per print issue every other month. Feeding pieces through Pangram, by far the best of the checkers in my experience, is a sort of dead man’s switch for the editor’s attention. While it occasionally throws up false positives, I have yet to encounter a false negative, even after fairly extensive experiments in generating A.I. prose and then rewriting it myself. (It also is fairly effective in detecting human-generated, A.I.-edited prose.)
All of which is to say that, while I won’t insist that victory is assured, I don’t think defeat is either. When my own publication began dealing with slop-slinging in the slush pile, my idea was to post a piece’s Pangram score and, if necessary, a brief editorial note explaining why we believed a less-than-human score was a false positive. In short, everyone hated the idea. Yet I do think as the slop era clarifies in people’s minds what they value about human writing—fundamentally, the issues of responsibility and trust—this sort of practice will become common. And there will be enough embarrassing episodes of shoddy research leading to shoddy arguments that professionals will be encouraged to do their own work rigorously.
The occult machines prowling the grounds of the Castle of Literature are no doubt fearsome. I would submit, however, that they are not omnipotent or invincible. It is still possible for those of us in the scribbling business, with a bit of pluck and maybe some heaven-sent good fortune, to pass through these simulacra to the real writers—or what may remain of them at this late date.