A stark warning from a former AI doomer
A summary of Deep Questions with Cal Newport, “A Stark Warning from a Former AI Doomer”
— published 8 October 2026, 1 h 17 m. Guest: Émile P. Torres (they/them), philosopher
and historian of extinction, and the co-coiner (with Timnit Gebru) of TESCREAL. Audio:
pca.st; video, titled “A
Former AI Doomer Reveals a Dark Truth”, at
youtu.be/jt508VjX2H8. Timestamps link into the video at that
moment.
Structure, from the show’s own chapters: Cal’s framing (0:00),
the interview (2:01), Cal’s recap
(1:13:57).
This page summarises what the episode claims. See
Method and caveats for how it was made, and
Verifying the claims for which of the guest’s factual claims
hold up against written sources.
Torres’ thesis is that the race to build artificial superintelligence is not primarily a
business story but an ideological one: OpenAI and Anthropic grew out of a specific
Bay-Area/Oxford worldview (TESCREAL) whose end state is a posthuman cosmic utopia in which
biological Homo sapiens has no part. On that reading the loud public fight between AI
“doomers” and “accelerationists” is a family quarrel inside one movement — both wings want
superintelligence; they disagree only on timing and control. The words the movement uses in
public (“safety”, “human extinction”) carry in-house definitions that invert their plain
meaning. And because the doomers built the institutions, conferences and funding pipelines
that started the race, Torres argues their warnings are not prescience but a self-fulfilling
prophecy they are responsible for. Cal’s own addition: utopian projects with this structure
have a historical track record, and it is a body count.
Coined by Torres with computer scientist Timnit Gebru around late 2022, as a name for
“the dominant orthodox worldview within Silicon Valley” plus the intellectual history that
produced it (2:31). The letters run roughly in
chronological order of emergence (3:19):
|
Ideology |
As characterised in the episode |
| T |
Transhumanism |
The substrate — in its modern libertarian form. Re-engineer the human organism into a posthuman species. |
| E |
Extropianism |
The first organised libertarian-transhumanist movement, late 1980s–90s; its mailing list is where Eliezer Yudkowsky and Nick Bostrom met (6:15). |
| S |
Singularitarianism |
Transhumanism focused on an intelligence explosion via recursive self-improvement, then cosmic expansion. |
| C |
Cosmism |
Why stop at the human organism — re-engineer galaxies, possibly spacetime itself (7:41). |
| R |
Rationalism |
Founded by Yudkowsky out of that milieu: to build the technology safely we must first optimise our rationality. |
| EA |
Effective altruism |
Grew out of the rationalist blogs (Overcoming Bias, LessWrong): optimise our morality (9:12). |
| L |
Longtermism |
Cosmism with an explicit ethical base — totalist utilitarianism: maximise total value across cosmic space and time. |
Torres’ claim about reach: these views are fringe in American society but orthodox in Silicon
Valley — “transhumanism really is the water that people in Silicon Valley swim in”
(4:06). Most adherents never claim the label, “for the
same reason that I’ve never publicly stated I’m a round earther.”
Two supporting moves on longtermism: Will MacAskill’s What We Owe the Future (2022) yields
“a moral case for space settlement,” and Elon Musk has called longtermism “a close match for
my philosophy” (10:40).
Torres’ ideological-map metaphor: stand close and rationalism, EA and longtermism occupy
visibly different positions; step five paces back and they are coterminous — while their own
anti-transhumanist position, or Gebru’s, is five feet away
(12:12).
- Inside the movement, the split is doomers vs accelerationists over the risk of building
ASI (13:47). Everyone, Yudkowsky included, agrees ASI
must eventually be built; Bostrom’s line is that virtually all paths to utopia run through
superintelligence (21:10).
- The split that Torres says should be the public debate is team human vs team posthuman
(19:35). “Within team posthuman, there are family
disputes. And … family disputes can be vicious.” Team posthuman has “hogged the microphone.”
- Hence Cal’s framing of the current fight: huge numbers of EAs and rationalists attacking
OpenAI and Anthropic reads to outsiders as external criticism, but is mostly internal — the
objection is that Altman and Amodei are executing the shared goal badly
(18:14). Torres: “That is correct.”
The strongest claim in the episode. Torres argues utopia is inherently exclusionary — always
ask what is being left out — and here what is left out is our biological species
(14:35):
- In the good case, where ASI is aligned, we still disappear: posthumans will be superior at
everything, so we become technologically, economically and politically obsolete — and worse,
we remain in the way, consuming resources wanted for a Dyson swarm (Hassabis, “early
2030s”) or for covering the planet in data centres (Altman)
(15:15).
- Torres reads Altman’s 2017 “merge with machines” position as pro-extinctionist by
implication: avoiding one kind of extinction by undergoing another
(16:01).
- Yudkowsky, despite If Anyone Builds It, Everyone Dies, is placed on team posthuman: Torres
says he has argued in interviews that once posthumanity arrives it may be unethical to keep
having biological humans, by analogy with choosing to have a disabled child
(19:35).
- MacAskill is cited for the view that most wild animals have lives not worth living — making
destruction of the biosphere potentially good (16:45).
Torres’ summary: “this is a deeply anti-life, antihuman vision”
(17:31).
Cal, drily: “These are the people who watch The Matrix and we’re on the Matrix’s side.”
Torres’ practical warning — the words don’t mean what a listener at home assumes
(27:02):
-
Safety / alignment. Not alignment with human values but with the TESCREAL worldview. A
superintelligence aligned to Torres’ own anti-TESCREAL values would, by the movement’s own
definition, be an existential catastrophe — because “existential risk” is defined as any
event that prevents the posthuman cosmic utopia
(21:10).
-
Humanity. The academic literature uses a broad definition covering our species and
any posthuman successors — so Homo sapiens could die out next year without “human
extinction” having occurred (27:51).
-
Extinction, narrow sense, splits in two (28:35):
- Terminal extinction — our species ends, full stop.
- Final extinction — our species ends and leaves no posthuman successors.
“All the TESCREAL people care about is avoiding final extinction.” Many hold the
pro-extinctionist view that terminal extinction is desirable once successors exist: “we
should survive for long enough to create our posthuman successors. And then … we should dip
out.”
This is the lens Torres applies to the news cycle: the Jacob Coxon resignation at Anthropic
and the colleague who endorsed a ~10% chance of human extinction are, they say, being read by
the public as ordinary concern for our species’ survival, which is not what the terms mean
in-house (26:10).
Three stated reasons, and Torres’ reduction of all three
(23:19):
- Market forces. But the race began for ideological reasons — “back in 2010, 2015 … no
one was talking about profit,” because utopia outvalues any personal fortune. Profit became
a real driver only after ChatGPT’s November 2022 release
(24:07).
- Game theory between labs — each believes it is the responsible one.
- Geopolitics — the US–China race.
All three, they argue, reduce to anxiety about whose utopia gets built: a Chinese victory
yields a “communist utopia” and a singleton with Bostrom’s decisive strategic advantage, which
is not the TESCREAL utopia (24:51).
In (30:37). Raised in a very religious household —
convinced in 1992 that Bill Clinton was the antichrist — then apostatised in their early 20s;
later worked as a research assistant to Ray Kurzweil on the sequel to The Singularity Is
Near. Initially found the view outrageous, then “if you can’t fight them … maybe the best
thing is to join them,” and finally found that it “perfectly fit the religion-shaped hole that
was left behind” (34:16). The parallels are explicit:
resurrection “not from the grave but from the vat” via cryonics; TESCREAL as a theistic new
religious movement that inverts the relation — “rather than God creating us, we’re creating
God in the form of ASI” (35:02). Published pro-
transhumanist work from ~2009; true believer until 2019
(36:31).
The culture (37:58). “The community is very culty”:
recommended career paths, in-house jargon, expected-value theory and Bayesianism as shared
idiom, heavy intra-dating. Torres recounts spending months in 2019 at Cambridge’s Centre for
the Study of Existential Risk and discussing at a party, with fifteen people, whether they
were in a cult — one of the factors in defecting
(38:47).
The money (40:22). ~$26 bn in pledged EA donations
(a pre-FTX-collapse figure); the nicest building on an Oxford campus; EA money in Anthropic.
“Longtermism tells billionaires exactly what they want to hear” — you are not merely excused
from near-term charity, you are a morally superior person for focusing on the far future
(41:02). Torres endorses Molly Lieberman’s description
of EA as a bait and switch: recruit on global poverty, then redirect to the expected value
of trillions of future digital people (41:48). Anthropic
is counted as an EA-longtermist organisation — Amodei was an early Giving What We Can
signatory (“#34 on the list”), and its IPO is coming at a figure “over a trillion for sure”
(42:37). Also cited: the Effective Ventures purchase of
a ~£15 m Oxfordshire estate, justified by the expected value of it producing “the next Sam
Bankman-Fried” (44:42); and earn to give as the
career path that put SBF on Wall Street and then into crypto — combined with a hardcore
utilitarianism in which no act is intrinsically wrong, only consequences count
(46:18).
Out — three reasons for the “second apostasy”
(47:48):
- The poverty of utilitarianism. On that view persons have no intrinsic value; only
well-being does, and we are its fungible “containers or vessels.” Which is what licenses
maximising the number of (preferably digital) containers
(49:23).
- The content of the vision is technocapitalism on steroids — Bostrom defines
technological maturity as maximised economic productivity and nature subdued; “very
Baconian.” And it was produced almost entirely by wealthy white men in Oxford and Silicon
Valley, with virtually no reference to feminist, indigenous, disability or queer accounts of
a desirable future (50:56). Quoting Monica Bielskyte:
“you cannot design for, you must always design with”
(52:24).
- The transition could only be involuntary. Most of 8.3 billion people will not consent to
being superseded, so the handover requires coercion — with Kurzweil’s “you won’t be around
for very long to influence the debate” as the alternative
(53:14), and Yudkowsky’s “NPCs” as the attitude to the
masses (54:49).
Cal puts the structural comparison: once you drop the premise that a human life has infinite
value, you are in the same category as the 20th-century utopian projects — Nazi Germany, Soviet
collectivisation — which licensed mass death now against a future worth more
(55:27). Torres: “in my opinion, absolutely not” going
too far. Utopia is marked by astronomical value, so “there is no act, however extreme, however
violent, that can’t be justified” (56:54).
The worked examples given:
- Olle Häggström’s Here Be Dragons: given Bostrom’s ~10⁵⁸ future digital people, a tiny
reduction in existential risk outweighs billions of actual lives — so a president told there
is a one-in-a-million chance of an existential catastrophe originating in Germany could be
“justified in just nuking all of Germany”
(57:46).
- Yudkowsky’s 2023 Time article: the concern is not that 8.3 billion people die but the
loss of “our glorious transhumanist future” — i.e. final, not terminal, extinction
(58:33).
- The thermonuclear trade, which Cal raises from a circulating tweet and Torres expands: a
global nuclear war would be preferable to misaligned ASI, because survivors could retry
alignment in a few thousand years (59:18). Asked how
many may be “allowed to die” to prevent near-term AGI, the answer is effectively everyone
above the minimum viable population — 40,000, maybe as few as 150 — which rounds to all 8.3
billion (1:00:38).
Torres then undercuts the premise: they are “very skeptical” the vision is achievable at all
(1:02:11) — but historical utopian movements never
reached their goals either, and still left a trail of destruction: the thousand-year Reich, the
Taiping Heavenly Kingdom (“something like 30 million deaths”), ISIS and its Armageddon at Dabiq
(1:02:59). Present-day harms are already on the ledger,
in their account: disinformation and deepfakes, work with the US military, a Palantir partnership
building “the infrastructure for a totalitarian state,” and mass IP theft — all of which, from
inside the worldview, are “mere peccadillos … because utopia forgives everything”
(1:04:16).
Cal’s counter-thesis, which Torres endorses: the common defence of these figures is “sure
they’re weird, but they were right — the things they warned about are happening.” Cal argues
instead that the predictions are coming true because the predictors are building them, and
that in a counterfactual world where the frontier labs were ordinary tech companies with no
TESCREAL lineage, there would be no existential-risk conversation and nobody responsible would
have run the experiments that have been run
(1:05:45).
Torres’ version: AI companies are anomalous among corporations — for almost any other
industry the profit motive explains the behaviour, but here the origin and core are ideological
(1:07:09). Claiming prescience is like predicting at
breakfast that you will shortly have a cup of coffee, then making one. The funding chain they
gives as evidence (1:08:35): the Singularity Summit was
founded by Yudkowsky, Kurzweil and Peter Thiel; Shane Legg won a $10,000 prize from Yudkowsky’s
Singularity Institute for his superintelligence dissertation; Legg and Demis Hassabis spoke at
the Summit, followed Thiel back to his mansion and asked for funding — and got it, which is how
DeepMind exists. OpenAI was then founded out of fear of Hassabis establishing an “AGI
dictatorship.” “It looks like a conspiracy when you actually look at the details … not all
conspiracies are conspiracy theories.”
The conclusion (1:10:52): “Nobody has done more than
the doomers to launch, sustain, and accelerate the race to build superintelligence.” So point
the finger rather than credit the foresight — absent the movement, Torres does not believe the
race would exist at all.
What was new to Cal (1:14:09): “just how extinctionist
all these beliefs are.” Pull the thread and the endpoint is tech leadership having, at some
point, to remove billions of biological people who decline to disappear — “you end up killing
billions of people on purpose because they’re getting in the way of your dream of a digital
utopia” (1:15:02).
Two caveats he and Torres share
(1:15:43): neither believes a sufficiently agentic LLM
becomes a superintelligent god — “there’s a lot of good computer science reasons to say that’s
kind of fanciful” — and the danger is the pursuit, not the arrival. Cal’s closing position
(1:16:27): “I am a fan of AI. I think generative AI
could produce any number of useful tools” — the objection is that the field is being “hijacked”
by a well-funded anti-humanist utopian philosophy driving the two leading labs.
Pointers given on air (1:12:14):
xriskology.com; the Realtime Techpocalypse newsletter
(realtimetechpocalypse.com); the podcast Dystopia
Now; and, as a reading entry point, Cal recommends the ~6,000-word Aeon essay Against
longtermism (1:13:31), with Torres adding that it
predates their work on the pro-extinctionist element and pointing instead at a more recent
pro-human manifesto piece.
- Transcript provenance. No human captions exist for this video. The auto-generated English
track was pulled with
yt-dlp --write-auto-subs --sub-langs en-orig, de-duplicated and
re-flowed into ~50-second blocks (≈14,300 words). Timestamps here are cue times from that
track, so they are accurate to within a block.
- Names are reconstructed. Auto-captions mangle nearly every proper noun in this
conversation. Given here in conventional spelling: Émile P. Torres (captions: “Emil”),
Timnit Gebru (“Timmy/Tim Gabru”), TESCREAL (“Tescril”, “test grill”, “testal”), Eliezer
Yudkowsky (“Yakowski”), Nick Bostrom (“Bostonramm”), Ray Kurzweil (“Kurszwell”), Will
MacAskill (“Mcascal”), Dario Amodei (“Amade”), Sam Altman (“Alman”), Demis Hassabis
(“Hosabibus”), Shane Legg (“Shane Lake/Le”), Ben Goertzel (“Girtzil”), Olle Häggström
(“Oie Hagstrom”), Monica Bielskyte (“Bilskita”), Dabiq (“Deique”), Nectome (“Nectone”).
Where a caption name could not be resolved with confidence — Molly Lieberman, credited
with the “bait and switch” characterisation of EA — it is left as transcribed and should be
treated as unverified.
- Checked against sources outside the episode (four items only):
- Jacob Coxon — the Anthropic resignation Cal and Torres discuss is real: Coxon, who had
worked at OpenAI and then Anthropic from July 2023, resigned in September 2026, writing
that “neither company is acting responsibly … gambling with our lives.” The colleague who
then put the number on it was Evan Hubinger, Anthropic’s alignment science lead, who
said there is a >10% chance AI kills all humans within the decade. Coxon’s name is badly
garbled in the captions (“Jacob Coxin”).
- Anthropic’s IPO. The “over a trillion … I think it was 2 trillion” figure matches
press reporting of a targeted ~$2 tn valuation for an October 2026 listing — but that
number is reported investor speculation, not a company target.
- Torres’ outlets. The newsletter is Realtime Techpocalypse; the podcast is Dystopia
Now, co-hosted with comedian Kate Willett; the site is xriskology.com. Torres uses
they/them, as Cal does throughout the episode.
- Episode identity. The Pocket Casts link resolves to Deep Questions with Cal Newport,
“A Stark Warning from a Former AI Doomer.” The YouTube upload carries a different title but
the same content, and its chapter list matches the episode description.
- Everything else on this page is a summary of what was said on air, including all
attributions of belief to named individuals, all quotations of their writing and tweets, and
all figures ($26 bn pledged to EA, £15 m estate, 10⁵⁸ digital people, 40,000-person minimum
viable population, Amodei at #34 on the Giving What We Can list, Amanda Askell’s
relationship to Will MacAskill, Altman’s Nectome signup, the Yudkowsky nuclear-war and
“allowed to die” quotes, the Palantir and US-military claims). The sections above leave
these as Torres stated them. Many were later traced to sources, and the results are in
Verifying the claims below. Read the summary sections as a
record of the argument the episode makes, and the verification section for how well it holds
up.
Added 13 October 2026. This section checks the guest’s factual claims about named people
and institutions against written sources outside the episode. It sorts each claim by how well
it holds up. It covers factual claims (who said, wrote, founded or funded what), not Torres’
interpretations, such as “pro-extinctionist” or “a self-fulfilling prophecy”. Those are
arguments to weigh, not facts to check.
How it was done: web search for primary sources. I read these in full: Torres’ Aeon essay,
the Bankless transcript, Altman’s “The Merge” and Bostrom’s 2013 paper. Several items rest on
search snippets or secondary reports; those are flagged. Confirmed means a written source
says substantially what Torres said. Partly means the core holds but a detail is wrong or
the framing is stronger than the source. Not found means no source turned up. It does not
mean the claim is false.
- TESCREAL was coined by Gebru and Torres. It appeared in print as Gebru & Torres, “The
TESCREAL bundle: Eugenics and the promise of utopia through artificial general
intelligence”, First
Monday 29(4), April 2024. The acronym had circulated since about 2022–23.
- Existential risk means lost potential, not just extinction. Bostrom (2002): a risk
“where an adverse outcome would either annihilate Earth-originating intelligent life or
permanently and drastically curtail its potential”
(nickbostrom.com).
- “Humanity” is defined broadly. Bostrom’s 2013 “Existential Risk Prevention as Global
Priority” defines “humanity” as “Earth-originating
intelligent life rather than as the particular biologically defined species Homo
sapiens.” He adds that a change in species identity “need not be in any sense a
catastrophe.” This is the textual basis for Torres’ point that Homo sapiens could die out
without “human extinction” occurring.
- Bostrom defines technological maturity in economic terms. The same 2013 paper: “the
attainment of capabilities affording a level of economic productivity and control over
nature close to the maximum that could feasibly be achieved.” Torres’ “very Baconian” gloss
is fair.
- Bostrom’s 10⁵⁸ figure. In Superintelligence (2014), Bostrom estimates at least 10⁵⁸
emulated human lives if the cosmic endowment is used for computation. It is an upper-bound
estimate of what is possible, not a forecast. The 2013 paper’s figure is 10⁵².
- Yudkowsky on biological children. This is the strongest claim about any individual,
and it checks out. On the
Bankless podcast (February 2023):
“I have basic moral questions about whether it’s ethical for humans to have human children,
if having transhuman children is an option instead,” which he likens to “deliberately
having the equivalent of a very crippled child when you could just as easily have a not
crippled child.” His word was “crippled”, not “disabled”. He framed it as an open moral
question that he says friends strongly disagree with, not as a policy.
- Yudkowsky’s TIME article prefers nuclear risk to AI risk. His March 2023
TIME piece calls
for treaties under which “preventing AI extinction scenarios is considered a priority above
preventing a full nuclear exchange,” and for being “willing to destroy a rogue datacenter
by airstrike.” Context: this is about enforcing an international treaty. He did not say a
nuclear war is desirable.
- Yudkowsky favours eventually building ASI. A 2024 MIRI statement says “we want
humanity to build those systems, but only once we know how to do so safely,” and that
“failing to build smarter-than-human systems someday would be tragic.” Torres quotes it in
Truthdig.
Torres’ claim that everyone in the movement wants ASI eventually holds for Yudkowsky.
- Musk on longtermism. On 2 August 2022 he tweeted about What We Owe the Future: “Worth
reading. This is a close match for my philosophy.” The tweet was about the book.
- The “AGI dictatorship” fear behind OpenAI. A 2017 email from Ilya Sutskever and Greg
Brockman to Musk, released in litigation, reads: “The goal of OpenAI is to make the future
good and to avoid an AGI dictatorship. You are concerned that Demis could create an AGI
dictatorship. So do we”
(Transformer,
Futurism).
- Shane Legg’s $10,000 prize. Legg’s 2008 thesis, Machine Super Intelligence, won the
Singularity Institute for AI’s $10,000 prize
(Wikipedia; Oxford Martin bio). Only
biographies were found, not a primary announcement.
- Altman’s “merge”. In “The Merge” (2017): “I
believe the merge has already started,” and “we will be the first species ever to design our
own descendants.”
- Altman and Nectome. In 2018 Altman put down a $10,000 refundable deposit on Nectome’s
waitlist. Nectome offers brain preservation that its co-founder called “100% fatal.” Altman
told the magazine: “I assume my brain will be uploaded to the cloud”
(MIT Technology Review).
- Anthropic and Palantir. On 7 November 2024, Anthropic, Palantir and AWS announced that
Claude would be available to US intelligence and defence agencies
(Business Wire).
The partnership is real. “Infrastructure for a totalitarian state” is Torres’ opinion.
- MacAskill and SBF. MacAskill pitched earning to give to Bankman-Fried over lunch near
Harvard in 2012–13, and SBF then went to Jane Street
(80,000 Hours,
Fortune).
- Askell and MacAskill. They married in 2013 and took a shared married name, MacAskill.
After their 2015 divorce she reworked it to Askell
(Wikipedia).
- The “bait and switch” scholar is Mollie Gleiberman, not “Molly Lieberman”. The caption
name resolves to Mollie Gleiberman, “Effective Altruism and the strategic ambiguity of
‘doing good’” (University of
Antwerp IOB discussion paper, 2023). The paper argues that EA recruits on global poverty and
then channels people towards longtermism and AI. I did not verify the exact phrase “bait and
switch” in the PDF.
- Torres’ own history. Torres was a research assistant to Kurzweil and spent several
months of 2019 as a visiting scholar at Cambridge’s CSER, both per interview bios
(Singularity Weblog,
Wikipedia). The pro-longtermist
period is confirmed by Torres’ book Morality, Foresight, and Human Flourishing (2017).
- Taiping death toll. Usually given as 20–30 million
(Wikipedia). Estimates range much wider,
and much of the toll was famine and disease.
- The Aeon essay. Its headline is Against longtermism, at a slug titled “Why
longtermism is the world’s most dangerous secular credo.” It is about 6,100 words, dated 19
October 2021 (Aeon).
Cal’s “~6,000 words” is right.
- “Glorious transhumanist future” is in Bankless, not TIME. Torres attributes the phrase
to the TIME article. It actually comes from the Bankless interview: “if there’s a glorious
transhumanist future (as it is sometimes conventionally known) at the end of this.” The TIME
article itself puts the stress on the fact that “literally everyone on Earth will die.”
That cuts against Torres’ reading that the TIME article cares about final rather than
terminal extinction. The Bankless phrase does support the broader point that Yudkowsky’s
hoped-for endpoint is transhumanist.
- The “allowed to die” exchange. Yudkowsky’s reply was: “There should be enough survivors
on Earth in close contact to form a viable reproductive population, with room to spare, and
they should have a sustainable food supply. So long as that’s true, there’s still a chance of
reaching the stars someday.” It is quoted in
Pharyngula,
among others. This matches Torres’ account, but the original tweet was not retrieved. The
“40,000, maybe 150” population figures are Torres’ gloss; they are not in the quoted reply.
- The Häggström Germany passage. The passage is real, in Here Be Dragons (2016) as
quoted in Torres’ Aeon essay. But Häggström raises it as a warning. He opens: “I feel
extremely uneasy about the prospect that [these calculations] might become recognised among
politicians.” His scenario is a lunatic in Germany building a doomsday weapon with a
one-in-a-million chance of success. Torres’ retelling (“a one-in-a-million chance of an
existential catastrophe originating in Germany”) is a fair compression. Still, a listener
could miss that Häggström is a critic of the reasoning, not an advocate.
- MacAskill on wild animals. What We Owe the Future says that if wild animals’ lives are
“worse than nothing on average, which I think is plausible (though uncertain),” then human
expansion has been “a good thing” from the animals’ perspective
(quoted on the EA Forum).
The substance is there. Torres drops MacAskill’s hedge and states the implication more
strongly than MacAskill does.
- Yudkowsky and Bostrom on the Extropians list. Yudkowsky’s 1990s posts to the list are
archived, and Bostrom was active in the same scene. No source was found saying they met
there.
- Yudkowsky and “NPCs”. What Torres said (54:58):
“as Yudkowsky would say, they’re NPCs, you know, non-player characters, so they just don’t
really matter.” The term does appear in his fiction in that dismissive sense. In Harry
Potter and the Methods of Rationality,
ch. 88 (2013), an unhelpful librarian prompts this from
Yudkowsky’s protagonist: “Harry’s brain flagged this as I’m talking to NPCs again”. The
word “again” implies it is a habit. That is one of only three uses of “NPC” in the novel’s
122 chapters. In ch. 8 Harry jokes “PC or NPC, that is the question?” about Hermione. In
ch. 75 he tells her that relying on rules and teachers “puts your PC self to sleep and turns
you into an NPC again.” In both, the word means lack of agency. That matches how the
rationalist community uses the PC/NPC split, for example the LessWrong thread
“To what degree do you model people as agents?”
(2013), which equates it with “agents versus complex systems.” It is a judgment about who
takes initiative, not about who has moral worth. Yudkowsky’s own 2003 short story
“Non-Player Character” (first published in
Transhumanity, revised 2005) argues almost the opposite. A gamer dismisses a game
character (“Don’t worry, babe, they were only NPCs”) and is then left unsure whether NPCs
have inner lives. It ends: “are the personalities of the NPCs real … The mindsmiths who try
for real AI say they’re damn sure it isn’t so. Maybe they know. But I don’t.” HPMOR also
presents Harry’s failure to see other people as capable as a character flaw
(ch. 31). No
non-fiction use by Yudkowsky turned up; his tweets were not searched exhaustively. Verdict:
Torres was pointing at something real, but “they just don’t really matter” is the reading
that his own story most directly argues against. A protagonist’s thought is also not the
author speaking. (Thanks to the reader who pointed to the story and to a
r/HPMOR thread
on the HPMOR usage.)
- The Singularity Summit. It is usually described as founded in 2006 by the Singularity
Institute (Yudkowsky’s organisation), Kurzweil and Thiel
(Wikipedia). The first event was
organised with Stanford. “Founded by Yudkowsky, Kurzweil and Thiel” is a fair shorthand.
- DeepMind and Thiel. Hassabis pitched Thiel at his California home around the August
2010 Summit, and Founders Fund backed DeepMind that year
(CNBC).
The core story holds. Reports differ on the round size, and that both founders spoke at the
Summit was not separately confirmed.
- Amodei and Giving What We Can. Amodei, Askell and Jack Clark have all signed the pledge.
One secondary source calls Amodei the 43rd signatory, not the 34th; Torres may have
transposed the digits. EA money in Anthropic is real: FTX/Alameda put about $500 m into the
2022 round (The Block),
and Moskovitz and Jaan Tallinn invested earlier.
- Wytham Abbey. Effective Ventures bought it in April 2022 for £14.9 m, mostly with Open
Philanthropy money. The stated rationale was a venue for workshops and retreats
(EA Forum).
The purchase is accurate. The justification Torres gives, producing “the next Sam
Bankman-Fried”, was not found in any source.
- Altman on data centres. On Theo Von’s podcast (2025), Altman mused that much of the
world gets covered in data centres and floated a Dyson sphere. That supports Torres’ point,
loosely.
- “$26 bn pledged to EA.” The widely cited figure is Benjamin Todd’s 2021 estimate of about
$46 bn committed, mostly Moskovitz and SBF money. $26 bn was SBF’s peak personal net
worth. Torres appears to have conflated the two.
- Kurzweil’s “you won’t be around for very long to influence the debate.” No source was
found in Kurzweil’s writing or in Wikiquote. It may come from another writer or be a
paraphrase.
- Hassabis predicting a Dyson swarm by the early 2030s. No primary source was found. The
only hit was an essay on Torres’ own newsletter, without a citation.
- Monika Bielskyte, “you cannot design for, you must always design with.” Not found
as a quote. It fits her published “Protopia Futures” framework, which centres the
leadership of the most affected communities.
- “No act is intrinsically wrong.” Offered as the utilitarianism behind earning to give,
this is unsupported as a description of EA leaders. SBF did tell Tyler Cowen in 2022 he
would take a 51/49 bet on doubling or destroying the world. MacAskill, though, publicly
rejected “ends justify the means” reasoning after FTX collapsed. Torres’ line describes
textbook act-utilitarianism, and it is not a position the movement’s leaders state.
Torres’ documentary claims mostly check out, and often more solidly than you might expect
from their tone. The broad definitions of “humanity” and “existential risk” are in Bostrom’s
own words. So are Yudkowsky’s “crippled child” remark, his TIME article’s ranking of nuclear
risk below AI risk, the “AGI dictatorship” email, the Thiel–DeepMind link and Altman’s
merge/Nectome positions. The errors are in the details: a wrong headline figure ($26 bn),
a quote attributed to the wrong source (TIME vs Bankless), and several quotes that could not
be traced (Kurzweil, Hassabis’s Dyson swarm, the Wytham “next SBF” rationale). There
is also a pattern of dropping the source’s own hedges: Häggström’s thought experiment
comes across as an endorsement rather than a warning, MacAskill’s “plausible (though
uncertain)” disappears, Yudkowsky’s open moral question becomes a stated view, and a fictional
character’s “NPCs” becomes the author’s attitude to the masses. None of
this settles Torres’ interpretive thesis. It does mean the raw material is real, and that
the episode’s strongest-sounding paraphrases are the ones to check before repeating.