A stark warning from a former AI doomer

A summary of Deep Questions with Cal Newport, “A Stark Warning from a Former AI Doomer” — published 8 October 2026, 1 h 17 m. Guest: Émile P. Torres (they/them), philosopher and historian of extinction, and the co-coiner (with Timnit Gebru) of TESCREAL. Audio: pca.st; video, titled “A Former AI Doomer Reveals a Dark Truth”, at youtu.be/jt508VjX2H8. Timestamps link into the video at that moment.

Structure, from the show’s own chapters: Cal’s framing (0:00), the interview (2:01), Cal’s recap (1:13:57).

This page summarises what the episode claims. See Method and caveats for how it was made, and Verifying the claims for which of the guest’s factual claims hold up against written sources.

The gist in one paragraph

Torres’ thesis is that the race to build artificial superintelligence is not primarily a business story but an ideological one: OpenAI and Anthropic grew out of a specific Bay-Area/Oxford worldview (TESCREAL) whose end state is a posthuman cosmic utopia in which biological Homo sapiens has no part. On that reading the loud public fight between AI “doomers” and “accelerationists” is a family quarrel inside one movement — both wings want superintelligence; they disagree only on timing and control. The words the movement uses in public (“safety”, “human extinction”) carry in-house definitions that invert their plain meaning. And because the doomers built the institutions, conferences and funding pipelines that started the race, Torres argues their warnings are not prescience but a self-fulfilling prophecy they are responsible for. Cal’s own addition: utopian projects with this structure have a historical track record, and it is a body count.

TESCREAL, defined

Coined by Torres with computer scientist Timnit Gebru around late 2022, as a name for “the dominant orthodox worldview within Silicon Valley” plus the intellectual history that produced it (2:31). The letters run roughly in chronological order of emergence (3:19):

Ideology As characterised in the episode
T Transhumanism The substrate — in its modern libertarian form. Re-engineer the human organism into a posthuman species.
E Extropianism The first organised libertarian-transhumanist movement, late 1980s–90s; its mailing list is where Eliezer Yudkowsky and Nick Bostrom met (6:15).
S Singularitarianism Transhumanism focused on an intelligence explosion via recursive self-improvement, then cosmic expansion.
C Cosmism Why stop at the human organism — re-engineer galaxies, possibly spacetime itself (7:41).
R Rationalism Founded by Yudkowsky out of that milieu: to build the technology safely we must first optimise our rationality.
EA Effective altruism Grew out of the rationalist blogs (Overcoming Bias, LessWrong): optimise our morality (9:12).
L Longtermism Cosmism with an explicit ethical base — totalist utilitarianism: maximise total value across cosmic space and time.

Torres’ claim about reach: these views are fringe in American society but orthodox in Silicon Valley — “transhumanism really is the water that people in Silicon Valley swim in” (4:06). Most adherents never claim the label, “for the same reason that I’ve never publicly stated I’m a round earther.”

Two supporting moves on longtermism: Will MacAskill’s What We Owe the Future (2022) yields “a moral case for space settlement,” and Elon Musk has called longtermism “a close match for my philosophy” (10:40).

One map, two teams

Torres’ ideological-map metaphor: stand close and rationalism, EA and longtermism occupy visibly different positions; step five paces back and they are coterminous — while their own anti-transhumanist position, or Gebru’s, is five feet away (12:12).

The pro-extinctionist core

The strongest claim in the episode. Torres argues utopia is inherently exclusionary — always ask what is being left out — and here what is left out is our biological species (14:35):

Cal, drily: “These are the people who watch The Matrix and we’re on the Matrix’s side.”

The vocabulary problem

Torres’ practical warning — the words don’t mean what a listener at home assumes (27:02):

This is the lens Torres applies to the news cycle: the Jacob Coxon resignation at Anthropic and the colleague who endorsed a ~10% chance of human extinction are, they say, being read by the public as ordinary concern for our species’ survival, which is not what the terms mean in-house (26:10).

Why the labs say they can’t stop

Three stated reasons, and Torres’ reduction of all three (23:19):

  1. Market forces. But the race began for ideological reasons — “back in 2010, 2015 … no one was talking about profit,” because utopia outvalues any personal fortune. Profit became a real driver only after ChatGPT’s November 2022 release (24:07).
  2. Game theory between labs — each believes it is the responsible one.
  3. Geopolitics — the US–China race.

All three, they argue, reduce to anxiety about whose utopia gets built: a Chinese victory yields a “communist utopia” and a singleton with Bostrom’s decisive strategic advantage, which is not the TESCREAL utopia (24:51).

Torres’ road in, and out

In (30:37). Raised in a very religious household — convinced in 1992 that Bill Clinton was the antichrist — then apostatised in their early 20s; later worked as a research assistant to Ray Kurzweil on the sequel to The Singularity Is Near. Initially found the view outrageous, then “if you can’t fight them … maybe the best thing is to join them,” and finally found that it “perfectly fit the religion-shaped hole that was left behind” (34:16). The parallels are explicit: resurrection “not from the grave but from the vat” via cryonics; TESCREAL as a theistic new religious movement that inverts the relation — “rather than God creating us, we’re creating God in the form of ASI” (35:02). Published pro- transhumanist work from ~2009; true believer until 2019 (36:31).

The culture (37:58). “The community is very culty”: recommended career paths, in-house jargon, expected-value theory and Bayesianism as shared idiom, heavy intra-dating. Torres recounts spending months in 2019 at Cambridge’s Centre for the Study of Existential Risk and discussing at a party, with fifteen people, whether they were in a cult — one of the factors in defecting (38:47).

The money (40:22). ~$26 bn in pledged EA donations (a pre-FTX-collapse figure); the nicest building on an Oxford campus; EA money in Anthropic. “Longtermism tells billionaires exactly what they want to hear” — you are not merely excused from near-term charity, you are a morally superior person for focusing on the far future (41:02). Torres endorses Molly Lieberman’s description of EA as a bait and switch: recruit on global poverty, then redirect to the expected value of trillions of future digital people (41:48). Anthropic is counted as an EA-longtermist organisation — Amodei was an early Giving What We Can signatory (“#34 on the list”), and its IPO is coming at a figure “over a trillion for sure” (42:37). Also cited: the Effective Ventures purchase of a ~£15 m Oxfordshire estate, justified by the expected value of it producing “the next Sam Bankman-Fried” (44:42); and earn to give as the career path that put SBF on Wall Street and then into crypto — combined with a hardcore utilitarianism in which no act is intrinsically wrong, only consequences count (46:18).

Out — three reasons for the “second apostasy” (47:48):

  1. The poverty of utilitarianism. On that view persons have no intrinsic value; only well-being does, and we are its fungible “containers or vessels.” Which is what licenses maximising the number of (preferably digital) containers (49:23).
  2. The content of the vision is technocapitalism on steroids — Bostrom defines technological maturity as maximised economic productivity and nature subdued; “very Baconian.” And it was produced almost entirely by wealthy white men in Oxford and Silicon Valley, with virtually no reference to feminist, indigenous, disability or queer accounts of a desirable future (50:56). Quoting Monica Bielskyte: “you cannot design for, you must always design with” (52:24).
  3. The transition could only be involuntary. Most of 8.3 billion people will not consent to being superseded, so the handover requires coercion — with Kurzweil’s “you won’t be around for very long to influence the debate” as the alternative (53:14), and Yudkowsky’s “NPCs” as the attitude to the masses (54:49).

Utopia justifies anything

Cal puts the structural comparison: once you drop the premise that a human life has infinite value, you are in the same category as the 20th-century utopian projects — Nazi Germany, Soviet collectivisation — which licensed mass death now against a future worth more (55:27). Torres: “in my opinion, absolutely not” going too far. Utopia is marked by astronomical value, so “there is no act, however extreme, however violent, that can’t be justified” (56:54).

The worked examples given:

Torres then undercuts the premise: they are “very skeptical” the vision is achievable at all (1:02:11) — but historical utopian movements never reached their goals either, and still left a trail of destruction: the thousand-year Reich, the Taiping Heavenly Kingdom (“something like 30 million deaths”), ISIS and its Armageddon at Dabiq (1:02:59). Present-day harms are already on the ledger, in their account: disinformation and deepfakes, work with the US military, a Palantir partnership building “the infrastructure for a totalitarian state,” and mass IP theft — all of which, from inside the worldview, are “mere peccadillos … because utopia forgives everything” (1:04:16).

Not prescience — a self-fulfilling prophecy

Cal’s counter-thesis, which Torres endorses: the common defence of these figures is “sure they’re weird, but they were right — the things they warned about are happening.” Cal argues instead that the predictions are coming true because the predictors are building them, and that in a counterfactual world where the frontier labs were ordinary tech companies with no TESCREAL lineage, there would be no existential-risk conversation and nobody responsible would have run the experiments that have been run (1:05:45).

Torres’ version: AI companies are anomalous among corporations — for almost any other industry the profit motive explains the behaviour, but here the origin and core are ideological (1:07:09). Claiming prescience is like predicting at breakfast that you will shortly have a cup of coffee, then making one. The funding chain they gives as evidence (1:08:35): the Singularity Summit was founded by Yudkowsky, Kurzweil and Peter Thiel; Shane Legg won a $10,000 prize from Yudkowsky’s Singularity Institute for his superintelligence dissertation; Legg and Demis Hassabis spoke at the Summit, followed Thiel back to his mansion and asked for funding — and got it, which is how DeepMind exists. OpenAI was then founded out of fear of Hassabis establishing an “AGI dictatorship.” “It looks like a conspiracy when you actually look at the details … not all conspiracies are conspiracy theories.”

The conclusion (1:10:52): “Nobody has done more than the doomers to launch, sustain, and accelerate the race to build superintelligence.” So point the finger rather than credit the foresight — absent the movement, Torres does not believe the race would exist at all.

Cal’s recap

What was new to Cal (1:14:09): “just how extinctionist all these beliefs are.” Pull the thread and the endpoint is tech leadership having, at some point, to remove billions of biological people who decline to disappear — “you end up killing billions of people on purpose because they’re getting in the way of your dream of a digital utopia” (1:15:02).

Two caveats he and Torres share (1:15:43): neither believes a sufficiently agentic LLM becomes a superintelligent god — “there’s a lot of good computer science reasons to say that’s kind of fanciful” — and the danger is the pursuit, not the arrival. Cal’s closing position (1:16:27): “I am a fan of AI. I think generative AI could produce any number of useful tools” — the objection is that the field is being “hijacked” by a well-funded anti-humanist utopian philosophy driving the two leading labs.

Where to find Torres

Pointers given on air (1:12:14): xriskology.com; the Realtime Techpocalypse newsletter (realtimetechpocalypse.com); the podcast Dystopia Now; and, as a reading entry point, Cal recommends the ~6,000-word Aeon essay Against longtermism (1:13:31), with Torres adding that it predates their work on the pro-extinctionist element and pointing instead at a more recent pro-human manifesto piece.

Method and caveats

Verifying the claims

Added 13 October 2026. This section checks the guest’s factual claims about named people and institutions against written sources outside the episode. It sorts each claim by how well it holds up. It covers factual claims (who said, wrote, founded or funded what), not Torres’ interpretations, such as “pro-extinctionist” or “a self-fulfilling prophecy”. Those are arguments to weigh, not facts to check.

How it was done: web search for primary sources. I read these in full: Torres’ Aeon essay, the Bankless transcript, Altman’s “The Merge” and Bostrom’s 2013 paper. Several items rest on search snippets or secondary reports; those are flagged. Confirmed means a written source says substantially what Torres said. Partly means the core holds but a detail is wrong or the framing is stronger than the source. Not found means no source turned up. It does not mean the claim is false.

Confirmed

Partly right: the detail or framing is off

Wrong or unsupported

Overall

Torres’ documentary claims mostly check out, and often more solidly than you might expect from their tone. The broad definitions of “humanity” and “existential risk” are in Bostrom’s own words. So are Yudkowsky’s “crippled child” remark, his TIME article’s ranking of nuclear risk below AI risk, the “AGI dictatorship” email, the Thiel–DeepMind link and Altman’s merge/Nectome positions. The errors are in the details: a wrong headline figure ($26 bn), a quote attributed to the wrong source (TIME vs Bankless), and several quotes that could not be traced (Kurzweil, Hassabis’s Dyson swarm, the Wytham “next SBF” rationale). There is also a pattern of dropping the source’s own hedges: Häggström’s thought experiment comes across as an endorsement rather than a warning, MacAskill’s “plausible (though uncertain)” disappears, Yudkowsky’s open moral question becomes a stated view, and a fictional character’s “NPCs” becomes the author’s attitude to the masses. None of this settles Torres’ interpretive thesis. It does mean the raw material is real, and that the episode’s strongest-sounding paraphrases are the ones to check before repeating.