The Loop That Closes Itself: Richard Socher, Recursive Self-Improvement and the Politics of Verifiable Intelligence

An optimist among accelerationists

A founder of a self-improving-AI company ends up as the cautious voice on the panel.

The Moonshots panel comes to most AI questions with its conclusions settled. Diamandis calls himself a “data-driven optimist.” Wissner-Gross, introduced as the show’s “in-house ASI,” says mathematics, science and engineering are “cooked.” Blundin invests in the companies he talks about. So the episode’s most revealing moment came when the guest, whose company was founded to build “recursively self-improving super intelligence,” noticed he had become the panel’s skeptic. “I usually on all podcasts I’m the one who is the optimist,” Socher said. “We’re just disagreeing on timelines and it makes me kind of feel like I’m the pessimist.”

Socher has unusual standing in this debate. He is among the most cited researchers in natural language processing. He founded MetaMind, which Salesforce acquired, served as Salesforce’s chief scientist, founded You.com and the venture firm AIX Ventures, and is now co-founder and CEO of Recursive. According to the lead investor’s announcement and trade coverage, Recursive raised $650 million at a $4.65 billion valuation in May 2026, led by GV and Greycroft with participation from AMD Ventures and NVIDIA. The founding team includes researchers best known for open-endedness and self-improvement research, among them Tim Rocktäschel and Jeff Clune.^[3][4] In July, Recursive signed a $410 million multi-year compute agreement with AWS. Socher told TechCrunch it was “likely going to be one of the smallest compute deals” the company would sign.^[5][6] On air, Diamandis put the raise at $670 million and Blundin put the compute deal at $450 million. When Blundin asked about it, Socher repeated the TechCrunch line: “that may have been one of the smallest compute deals we’ve done.”

The money explains why the episode matters. Labs and governments have usually treated RSI as a hypothetical capability to be monitored. Recursive treats it as a product roadmap. In the same week, the panel discussed a congressional bill that would ban it. The two events show a field in which the central technical idea, and the main political target, is an AI that writes, trains and evaluates its own successors.

The Eureka Machine thesis

Fragmentation, abstraction and the verifiability principle.

Diamandis opened with Socher’s new book, The Eureka Machine, which he summarized as predicting “a century of scientific breakthroughs in the next decade.” Socher’s diagnosis of slow scientific progress is sociological before it is technical. Progress has slowed, he said, “because we have so many different subdisciplines and disciplines… there’s so much separation that it’s hard to weave it all together.” In his account, AI is the tool that can “weave back together all these separate pieces.”

The diagnosis has a substantial economic literature behind it. Benjamin Jones’s “burden of knowledge” model shows that as the frontier grows, researchers spend longer in training, specialize more narrowly and work in larger teams.^[7] Bloom, Jones, Van Reenen and Webb estimate that research productivity has fallen sharply across many fields, so that holding growth steady takes ever more researchers.^[8] Park, Leahey and Funk find that papers and patents have become steadily less “disruptive” since the mid-twentieth century.^[9] Socher’s claim that fragmentation, not funding, is the binding constraint is a strong reading of this evidence. The literature is consistent with it but does not establish causation. Still, the mechanism he proposes is specific. A single model trained across the literature can hold knowledge that no human specialist can, and therefore can recombine it.

His recipe has four ingredients, which he calls pillars. The first is world knowledge, now available as large language models. The second is growing stores of digitized scientific data. The third is better simulation. The fourth is laboratory automation, run by “an agent swarm.” Socher reduced the scientific method to three verbs, “the ideation, implementation and validation of ideas,” and argued that “the faster we can close that loop and then put an open-ended process on top of it,” the faster discovery will go. The same three verbs later became his definition of RSI. That is the book’s main conceptual move: automated science and self-improving AI are one loop applied to different objects.

The second major idea is what this article calls the verifiability principle. Diamandis paraphrased the book as saying AI becomes superhuman “in any domain you can simulate or verify and nowhere else.” Socher put it this way: “anything you can simulate, anything you can verify, AI will then be able to solve things in that domain.” The principle builds on Rich Sutton’s “Bitter Lesson,” which Socher summarized at length. General methods that scale with computation have repeatedly beaten clever, hand-built human knowledge.^[11] If the Bitter Lesson says scale wins, the verifiability principle says where scale can be applied: wherever a cheap, reliable error signal exists.

Socher used the principle to make a counterintuitive prediction. On air, the panel described physics as the next field to be “incinerated” after mathematics. Socher argued instead that biology is the better fit for AI, because physics data “require often like large hadron colliders and just like billions of dollars.” His analogy is worth quoting: “what calculus did for physics like understanding really well micro individualized separated phenomena, neural nets are great in combining all these little things.” Biology, in this view, is a domain where the parts are known and their interactions are not. That is where function approximation adds most to theory. The constraint is data, which is why he highlighted large perturbation studies such as Tahoe Therapeutics’ and the turn to “virtual cell” models.

Socher also extended the critique to economics, which he called “even more lacking than biology,” with models built around “a one-step economy that’s provably correctly taxed and subsidized.” That is a caricature of modern macroeconomics, which uses heterogeneous-agent models and dynamic general equilibrium models. His underlying point stands, though. Economics is a field where controlled experiments are rare, so the verifiability principle predicts it will benefit from AI later than molecular biology. The same logic underlies his warning on drug timelines. ASI might in principle “cure all diseases,” he argued, “but just to know whether that happened will take more than 3 years.” Validation is a physical process with its own clock. Diamandis’s reply, that cell simulators will make human trials “incinerated,” assumes the validation step can itself be simulated. That is exactly the question the principle leaves open.

What counts as recursion?

From I.J. Good’s “intelligence explosion” to a five-rung ladder and five learnable axes.

The idea is older than the field’s current vocabulary. In 1965, I.J. Good argued that an “ultraintelligent machine” could design even better machines, setting off an “intelligence explosion.” In his formulation, the first such machine would be “the last invention that man need ever make.”^[13] Jürgen Schmidhuber’s Gödel machine (2003) made the idea formal: a self-referential problem solver that rewrites its own code once it can prove the rewrite improves expected utility.^[14] The proof requirement made the design elegant and also impractical. Jeff Clune’s 2019 “AI-generating algorithms” agenda, now represented on Recursive’s founding team, swapped proof for empirical search. It proposed three pillars: meta-learning architectures, meta-learning the learning algorithms themselves, and automatically generating learning environments.^[15] The Darwin Gödel Machine (2025) took the same approach with coding agents. It keeps an archive of agent variants that rewrite their own scaffolding and checks each on benchmarks, raising SWE-bench performance from 20% to 50% without changing the underlying foundation model.^[16]

That history explains why Socher, asked “where are we at the moment with RSI,” answered with a qualified yes: “in various weak forms, we already have RSI.” What changed in 2026, he said, is that “AI can now code… AI is code and AI can code, right? So you have a loop that you can close there.” He described the labs’ own use of Codex and Claude to write internal code as “a weak form of recursive self-improvement because you still have like deeply embedded humans in that loop.” Recursive’s goal is to limit humans to “setting up the rewards and the environment and the goals,” and to give the AI “the entirety of the process of ideation, implementation, and validation.”

Salim Ismail pushed back with the most useful question of the segment. He proposed a five-step continuum and asked where recursion begins. Socher answered with his definition rather than a rung number. “True recursive self-improvement in its strongest sense” requires all three stages done by AI, each as an inner loop, plus “an outer process that is more open-ended… similar to biological, cultural, and technological evolution.” He also cited his “friend Jason Weston” at Meta. Weston and Jakob Foerster list five “learnable axes” of self-improvement: parameters, training data, objective, architecture and overall code. Socher noted that “no one has really cracked the nut of doing all of these five.”^[10] One detail he did not mention is that Weston and Foerster argue for human–AI “co-improvement” and against fully autonomous RSI, on safety grounds. The framework he cited comes from a paper whose normative conclusion is the opposite of Recursive’s mission.

The ladder makes clear something the broadcast only hinted at. “RSI” covers at least five different technical regimes, and they carry different risks. Rung 1 is ordinary software engineering with better tools. Rung 5 is the scenario that safety research has worried about for half a century. The trouble for policy is that the rungs are continuous. As Socher said in the Washington segment, a “very small form of recursive self-improvement” can be “hack[ed] up in like 20 minutes with prompt engineering.” No natural legal line separates an agent that tunes its own harness from one that rewrites its objective.

AI is code and AI can code, right? So you have a loop that you can close there.Richard Socher, on why 2026 is different

Wissner-Gross then raised a question that is arguably more interesting than the timeline debate: what is the fixed point of RSI? He pointed to the nanoGPT speedrun, where the time to train a GPT-2-class model has collapsed through algorithmic changes, and to a dispute over whether recent records counted because they “factor[ed] out world knowledge.” He asked whether the end-state model would separate a small “perfect reasoning kernel,” perhaps “a megabyte in size,” from knowledge kept in a database. Socher said no: “just like humans benefit from memorizing things… in order to be able to creatively think through concepts. So does an AI.” Retrieval will complement the weights, he said, but “the main model will have a lot of that mixed in for sure.” This is a live research question. Retrieval-augmented and memory-augmented architectures have shown that a lot of factual knowledge can live outside the parameters. Whether abstraction and analogy can be separated from a broad base of world knowledge is an open empirical question. Socher’s answer amounts to a bet against extreme modularity.

Spiky, general, and physically bottlenecked

Why an RSI founder can believe superintelligence is decades away.

Asked to define artificial superintelligence, Socher offered a layered answer. At minimum, ASI “has to spike… across several different capabilities.” “In the grandest definition,” it must “supersede not just arbitrary humans like a Turing test but all of humanity to solve arbitrarily hard tasks.” He added two conditions that are unusual in capability-centered definitions. A superintelligence must be able to “choose to some degree what it works on” and have “some metacognition about its thought.” He measures it across ten “spaces of intelligence” described in his book. On air he named perceptual, communicative, social, creative, speed, metacognitive, knowledge and mathematical-reasoning spaces. By that standard, being better than all of humanity combined “will take us probably several decades.”

Ismail objected that “smart” is underspecified (“what the hell does smart mean?”), echoing long-standing arguments about multiple intelligences. Diamandis cited Elon Musk’s forecast of 2029–2030. Wissner-Gross was more direct: “Completely don’t buy the 20-year timeline.” His argument was that connecting a capable model to actuators is “the easy part.”

Socher’s reply contained the episode’s most important technical distinction. He expects spiky ASI soon: AI “better at programming than all of humanity,” “better at math than all of humanity… in a few years,” and better at any perfect-information game. What takes decades, he argues, is physical superiority. “Humanity can build a large hadron collider. Humanity can create a gold atom.” To improve its own substrate, an AI needs new supply chains, materials and fabrication tools: “it will take more than 2 years to build such a machine [an ASML lithography system] even if you had the perfect blueprint for it.” This is effectively a Moravec-style claim about recursion. The loop can close quickly in software and only slowly in atoms.^[19] Blundin’s counter, that a free-electron-laser alternative to ASML “might take 3 years,” does not refute the point. A three-year build cycle is a bottleneck.

Seen this way, the founder’s apparent contradiction (an RSI company with a decades-long ASI timeline) is internally consistent. Recursive’s commercial thesis needs only spiky superintelligence in the domains where Socher thinks verification is cheap: algorithm design, programming and the AI research loop itself. His long timeline applies to the parts of the world that can be checked only by building things. The disagreement with Wissner-Gross is therefore partly empirical, about how fast embodied AI and fabrication will scale. It is also partly definitional, because Socher’s strongest-sense ASI includes autonomy and goal selection, which most benchmark-driven definitions leave out.

Banning the loop

The Human Control Over AI Act, the enforcement argument, and a misattributed poll.

On Sept. 28, 2026, Rep. Ro Khanna (D-Calif.) told CNBC he would introduce the Human Control Over AI Act, which he called the most comprehensive AI legislation to date. According to CNBC and follow-up coverage, the bill would bar models that recursively self-improve or autonomously modify their own core objectives, containment or shutdown controls until federal guardrails exist and a regulator approves. It would create a licensing agency, place independent auditors on site at frontier labs, add oversight of advanced chips, impose criminal penalties for disabling safeguards or deploying unlicensed systems, and require liability insurance.^{[18][20][21][22]} These details come from press accounts. We could not find bill text at the time of writing, and any analysis should be revised once it is published.

Socher’s response rested on enforceability. “To really truly enforce no recursive self-improvement,” he argued, “you would need a totalitarian surveillance state the likes of which humanity has never seen, because anyone can have a GPU on their little laptop and ask that AI to just like improve its harness.” Wissner-Gross summed it up as “an AI Stasi.” Ismail called the ban “clearly nonworkable,” and Blundin called such proposals “childish.” Wissner-Gross pushed back on Ismail’s claim that there is “no mechanism” to enforce it, recalling 1990s export controls on cryptography-adjacent number theory: “They could start arresting people.”

To enforce that kind of legislation would require you to literally have a thought police that thinks about and hears everything you say to your private LLM on your own laptop.Richard Socher

The enforcement argument is strong against a literal ban on every rung of the ladder. It is much weaker against the regime that press accounts describe. Khanna’s framework, like the EU AI Act, is aimed at frontier developers, not laptops. The EU law presumes “systemic risk” for general-purpose models trained with more than 10²⁵ FLOP and regulates them through obligations on providers.^[24] Licensing, embedded auditors and chip oversight are the tools used for nuclear materials and financial systems. They do not require monitoring private thought. They work by regulating the small number of actors who can afford the scarce input, which here is frontier-scale compute. Socher’s own company illustrates the chokepoint: a $410 million cloud contract is not something a hobbyist can sign. A rung-1 harness tweak on a laptop and a rung-5 self-succession run on a hyperscale cluster are different activities. A well-drafted statute could treat them differently, and the real drafting challenge is how to define that line.

The panel’s more persuasive point came from Blundin, who argued that “the real risk is not so much getting arrested, it’s corporate liability.” If labs become liable for everything users do with their models, he said, “Anthropic gives its best models to everybody in America to build incredible things. That will stop in a heartbeat.” This is the real regulatory tradeoff. Later, Socher presented liability as a solution. He cited Hippocratic AI, which is liable for its healthcare calls and therefore “figured it out and… solved it.” Both statements can be true. Liability is a powerful incentive for reliability, and broad, uninsurable liability can push a technology toward closed, in-house deployment. Where to set it is a policy choice, and neither “P(doom)=0” nor “ban RSI” answers it.

Socher’s historical analogies deserve the same scrutiny. He quoted (via the Pessimists Archive) Pope Alexander VI’s 1501 call to “maintain full control” over printing.^[25][26] He also noted that the wheel “killed so many people.” Such analogies show that technology panics are common. They cannot show that a particular panic is mistaken. David Collingridge’s dilemma states the problem precisely: early in a technology’s life, control is easy but its effects are unclear; once the effects are clear, control is expensive or impossible.^[27] RSI governance sits right in that dilemma. That is why the episode’s strongest claim, “P(doom) is zero,” carries so much weight. Socher’s argument is that, after hours of debate, experts reduce extinction scenarios to bounded harms (“maybe a 100 million people would get somehow hurt or killed”). Bounded harms can be addressed with cyber defense, existing gain-of-function rules and media literacy. Even read generously, that is an argument that the probability is low and the harms are manageable. It is not a demonstration that the probability is zero. A risk estimate that excludes, by assumption, the scenarios its critics put forward is a modeling choice, not a measurement.

Constitutions and hard constraints

Socher’s charge that Anthropic’s constitution is “fake,” checked against the text.

Asked about Anthropic’s constitutional approach, Socher was blunt: “Mostly because it’s fake.” He pointed to the constitution’s list of hard constraints, which he read on air from anthropic.com/constitution. One of them says Claude should never “create cyberweapons or malicious code that could cause significant damage if deployed.”^[28] He contrasted that with Project Glasswing. Announced in April 2026, Glasswing gave a restricted group of partners access to the unreleased Claude Mythos Preview, which Anthropic said “can surpass all but the most skilled humans at finding and exploiting software vulnerabilities.” The aim was to fix those vulnerabilities defensively before the capability spread.^[29] “They created a cyber weapon,” Socher said; “the proof is in the pudding that it didn’t work.”

The constitution’s own wording supports a narrower reading. It describes hard constraints as “restrictions on the actions Claude itself actively performs; they are not broader goals.” It says they apply centrally to cases where Claude is “knowingly and intentionally engaging in an action of the relevant restricted category.”^[28] On that reading, a model that finds vulnerabilities for authorized defenders is not “creating cyberweapons,” even though the same skill can be used for offense. This is the dual-use problem that runs through cybersecurity. Socher’s critique is strongest when narrowed to two empirical questions. Do constitutionally trained models reliably refuse offensive use when deployed? And do a lab’s product decisions follow the spirit of its stated constraints? He asserted that misuse happened (“people used it to hack”), but on air he gave no specifics we could verify, and we treat the claim as unverified.

Under questioning from Wissner-Gross, who offered Asimov’s laws and Anthropic’s later “soul documents” as precedents, Socher made his position clearer. He is “not against post training… RL training… supervised fine-tuning.” He agreed that “the goal is a good one.” His real concern is reward hacking, where AIs “find a solution to get to what you said you wanted, but maybe not what you meant.” He expects markets to fix it because “no one wants to pay a ton of money for an AI that doesn’t actually solve the problems that you give it.” Market pressure works well for failures that customers can see. It works less well for failures that appear only under adversarial or rare conditions, which are the failures alignment research focuses on.

Benchmarks as contested evidence

Sonnet 5.5, Gemini 4 Argon and TypeSafe’s Jev, and why vendor numbers and independent numbers diverge.

The model-news segment worked as a lesson in evidence. Diamandis reported that Anthropic’s Sonnet 5.5 “jumped from 10% to 70%” on Terminal-Bench 4.0, beating Opus 5.5 “for half the price.” Anthropic’s launch page supports the headline: 70.6% at maximum effort against 10.3% for Sonnet 5 and 66.4% for Opus 5.5, with per-token pricing of $2/$10 against Opus’s $4/$20. Sonnet 5.5 is also the first Sonnet launched with cyber safeguards.^[30] Anthropic itself says Opus 5.5 “remains clearly stronger at complex, open-ended work.” Wissner-Gross argued that Sonnet’s cost–performance curve on the launch chart lies below Opus’s on a cost-per-attempt basis, which breaks the usual pattern in which a distilled model sits “up and to the left.” His conclusion was “I do not plan to use Sonnet 5.5.”

The independent numbers support both sides in part. Artificial Analysis scored Sonnet 5.5 at 64 and Opus 5.5 at 60, so the vendor’s ranking holds while the absolute gap narrows.^[31] Blundin offered a strategic explanation for the release. Orchestration vendors pair a frontier model with cheaper open-weight sub-models such as Kimi K3 or Qwen. A cheaper Anthropic model lets the company argue, “go with Anthropic top to bottom,” before enterprises move to open-weight models from China, which he estimated is “about three months behind.” Ismail called model advantage short-lived: “It’s how do you metabolize that into some decent capability?”

Google’s Gemini 4 Argon shows a different problem: a single headline index can hide very different error profiles. Artificial Analysis placed Argon at 53 on its Intelligence Index, tied with OpenAI’s GPT-6 Astra and ahead of GPT-6.1 Sol at 52, at a discounted cost per task of $1.99 against Astra’s $3.26. The more striking result was on the AA-Omniscience test. Argon’s hallucination rate was 15%, against 51% for Astra and 54% for Sol, while its accuracy was lower (50 vs. 63 for Astra).^[31][32] A model can lead on calibration (knowing when it does not know) and trail on recall. Socher’s comment matched the data. In search, “you don’t usually want any hallucinations,” while in protein design you want the model to “hallucinate novel ideas.” He also observed that Google’s business model does not require frontier intelligence for most queries.

The third story, TypeSafe’s Jev, is a reminder that progress is not only about bigger reasoning models. Jev is a “System One Model”: a closed model that returns typed outputs (a choice, a score or a boolean) with calibrated probabilities. The company quotes latencies of 70–500 ms and input pricing of $0.042 per million tokens. It also claims Jev is 193.6× faster and 444.6× cheaper on its own workflow evaluations, which are vendor benchmarks that have not been independently replicated.^[33] The name refers to Kahneman’s fast “System 1” and to W.S. Jevons, whose 1865 Coal Question argued that efficiency gains raise total consumption.^[34][35] Wissner-Gross placed it in architectural history: the encoder half of the original transformer “has basically gone missing in action,” and decision models are “a reinvention of the encoder transformer type wheel.” Socher agreed (“Classifiers are back”). Ismail spelled out the organizational stakes. Most corporate decisions are “micro decisions,” and using a frontier LLM for them “is like trying to bring the Supreme Court together to decide… which checkout line of the supermarket you should go to.” If that is right, the coordination costs that Coasean theory says define firm boundaries could fall sharply, which supports Ismail’s “organizational singularity” thesis.

Perception and personhood

Reading images from the brain, passing a video Turing test, and why the test has flipped.

Two stories tested the verifiability principle in the domain of human experience. The first was Brain-IT, from Michal Irani’s group at the Weizmann Institute, reported by MIT Technology Review on Oct. 1. Brain-IT reconstructs images seen by people in an fMRI scanner by sending brain signals through separate “structure” and “semantic” branches into a diffusion model. It was trained on eight people from the Natural Scenes Dataset, about 9,000 images each.^[36][37][38] Diamandis highlighted that the model also runs in reverse. An encoder predicts brain responses to images no subject has seen, producing synthetic training pairs. The paper reports that about 70% of training images had no real fMRI pairing, and that a new subject can be fitted with about an hour of scanning instead of roughly 40 hours.^[36][37]

Socher, asked whether the brain is “a simulatable domain,” answered “It’s not yet.” He listed per-subject calibration as the main obstacle, since “you have to train for each brain because they’re all slightly different.” The one-hour adaptation result speaks directly to his concern, and the encoder that generates synthetic data shows the principle in action. Once a domain has even a partial simulator, it can generate its own training signal. Ismail warned against overreading the result: “We’re decoding the perception here, not the actual thought.” Wissner-Gross replied that imagery and perception share neural substrates. Both are right in part. Visual imagery does recruit early visual cortex, but decoding imagined content is much less accurate than decoding perceived content. The line of work from Takagi and Nishimoto’s latent-diffusion reconstructions to Brain-IT remains a perception result.^[39] Wissner-Gross also credited early cat visual-cortex decoding to the Gallant lab at Berkeley. The best-known cat reconstruction is Stanley, Li and Dan (1999), also at Berkeley but from Yang Dan’s lab. Jack Gallant’s group is known for later human fMRI decoding.^[40]

The second story was Tavus’s Griffin model, which the company describes as the first to pass a “video Turing test.” In a company-run study, 26 of 54 participants (48%) who talked with Griffin on a live video call judged it human, compared with 1 of 41 (2.4%) for its predecessor, Phoenix-4.5. Participants were not told in advance that they might be talking to an AI. Griffin also led NVIDIA’s VideoFDB full-duplex benchmark.^[41][42] Diamandis rounded the baseline to “3%,” which is accurate. Two caveats matter. The study was run by the vendor with a small sample. And the 48% figure is close to chance in a two-way judgment, which is the usual threshold for “passing.” It is comparable in structure to Jones and Bergen’s 2025 finding that GPT-4.5, given a persona prompt, was judged human 73% of the time in a three-party text Turing test.^[43]

Socher’s response is a real conceptual contribution. In his view the Turing test has “flipped.” To tell whether you are talking to a human, you now “ask it questions that are so hard no human could ever answer it.” For example, an agent that returns “50,000 lines of code” in ten seconds reveals itself. Blundin objected that a system can throttle itself, and that is the deeper point. Once machines exceed human performance, a test of humanlike behavior measures only a system’s willingness to hide what it can do. Diamandis also asked whether Griffin satisfies the verifiability principle. Socher said no, because human judges do not scale: “You can’t quite automate it completely.” Humanlikeness can only be verified by humans, so it should advance more slowly than skills that can be checked automatically. Griffin’s jump from 2.4% to 48% in one model generation suggests that human judgment can be modeled well enough to optimize against, which puts some strain on that claim.

On governance, Socher proposed moving from KYC to “KYUC, know your use case.” He cited the deepfake video-conference fraud in which a Hong Kong employee was persuaded to transfer funds. Socher put the loss at “$20 million.” CNN reported HK$200 million (about US$25.6 million) lost by the engineering firm Arup in early 2024.^[44] Use-case regulation is consistent with the position he takes throughout: regulate applications, not capabilities. It also requires platforms (he named Zoom and Google) to build countermeasures for identifying people, which is itself a form of the infrastructure he warned against in the RSI debate.

Biology as engineering policy

Executive Order 14433 and the return of genetic pest control.

Diamandis presented the third science story as “biology as engineering arriving as federal policy.” Executive Order 14433, signed Sept. 29, 2026, directs a campaign to cut Asian tiger, Aedes aegypti and other invasive mosquito populations in Washington, D.C., by 90% and ticks by 50% by 2028. It prioritizes sterile insect techniques and “safe biotechnologies.” The EPA framework is to consider “sterilization techniques, safe genetic modification, and use of safe, beneficial bacteria.”^[45][46] Two details from the broadcast need adjusting. The order’s listed diseases are West Nile, Zika, chikungunya and dengue, not yellow fever. And the order does not mention gene drives by name. Diamandis linked it to “the exact kind of gene drive work that Colossal… is doing.” That is his interpretation, not the order’s text.

Wissner-Gross explained the gene drive concept by reference to his MIT classmate Kevin Esvelt, whose 2014 eLife paper with colleagues proposed RNA-guided (CRISPR) gene drives for wild populations and stressed governance and reversibility.^[47] Gene drives deserve separate treatment because they invert the verifiability principle. Laboratory efficacy is easy to measure. Ecological effects (on predators, pollinators and ecosystems across borders) are slow, expensive and partly irreversible to verify, which is why Socher added, “keep the bees like alive.” The diseases are serious. The WHO estimates 610,000 malaria deaths in 2024, higher than Diamandis’s “half a million.”^[48] The governance question is how to stage releases so each step can be checked before the next. That is a good template for RSI as well.

Wissner-Gross and Socher then extended the argument into a general critique of precaution. They cited nuclear power, where Wissner-Gross said “we arguably lost 50 plus years of progress,” Germany’s energy policy and overpopulation scares. These are serious cases, but they do not prove that every precautionary argument fails. The episode’s optimism about EO 14433 and its hostility to Khanna’s bill both depend on the same unstated premise: that society can judge, early on, which risks are overstated.

Meridian and machine warfare

A 120-day commission, a new combatant command, and where human judgment sits in lethal decisions.

On Sept. 30, 2026, at Marine Corps Base Quantico, Secretary Pete Hegseth announced Project Meridian. It is a 120-day review of future warfare, co-directed by Elon Musk, Palmer Luckey and Newt Gingrich, commissioned through MITRE by Under Secretary Emil Michael. It is due Jan. 28, 2027, with a public report and a classified annex. Its scope runs “from subterranean depths to the cislunar frontier.”^[49][50][51] In the same speech he announced Project Agincourt, a plan to stand up a four-star Autonomous Warfare Command (AUTOWARCOM), with a target date of Oct. 1, 2027, subject to congressional action.^[52][53] Diamandis noted the obvious conflicts. SpaceX and Anduril hold multibillion-dollar defense contracts, and Anduril builds autonomous weapons.

Socher stood by his essay’s position that AI should not control lethal decisions without human oversight: “The more we get to deciding not just to impact human lives but to end human lives, the more we should have human oversight.” Blundin said he is “not a fan of autonomous weapons that make decisions in the field” but acknowledged Luckey’s “very rational arguments.” He reported Luckey’s point that undersea systems cannot rely on communications and so already operate autonomously once in “hunt mode.” Wissner-Gross called AUTOWARCOM “a transformative moment” and highlighted the seabed and cislunar space as “wildly underserved domains.” Ismail framed the problem as institutional: “can a procurement organization built for 20-year weapons programs operate on like a 90-day technology cycle.” Diamandis replied, “you can’t fund exponential technology with linear procurement.”

U.S. policy already rejects a ban on autonomous weapons and also rejects unconstrained autonomy. DoD Directive 3000.09 requires that autonomous weapon systems allow “appropriate levels of human judgment over the use of force” and undergo senior review before development and fielding.^[54] AUTOWARCOM will test whether that standard holds as autonomy becomes an organizing principle rather than one feature among many. Ismail’s claim that Ukraine and Russia will each make “10 million drones” this year is partly supported. President Zelenskyy said in July that Ukraine now produces 10 million drones a year.^[55] Independent estimates are lower, and we found no reliable source for the Russian figure. His statement that this escalation happens “without humans in the loop” is not supported. Most drones in that war are still piloted by people or keep a human in the loop.

Coda: elasticity and attention

The listener Q&A shows the economic theory behind the episode.

The listener questions drew some of the episode’s clearest economic reasoning. Asked where the value goes if AI makes firms ten times more productive, Socher answered with price elasticity. Illustration demand did not rise enough when prices fell “from 200 bucks to like 2 cents,” so “we just didn’t need as many illustrators.” Software, by contrast, can be personalized for billions of people, so “Jevons paradox is going to be alive in that world.” This matches James Bessen’s research. Automation raises employment in an industry when demand is elastic, and reduces it once demand is saturated.^[56] The theory predicts different labor-market outcomes for each occupation, not one aggregate result. Socher added a distributional point: AI will be “a huge force for more entrepreneurship” in the best case and “a force of more inequality” in the worst.

His answer on the attention economy was more original. “The one thing that you cannot scale exponentially is the hours in the day that people can pay attention to you,” so in an abundant society “fame and brand and network effects… will become more and more of a currency.” This updates Herbert Simon’s observation that information abundance creates a scarcity of attention. Socher also asserted that bots and agents now outnumber people online, a prediction he says came true “a few months ago.” We did not verify it. Wissner-Gross applied Moravec’s paradox to the question of the lowest-status job. In a fully AI civilization, the cheapest jobs to automate, perhaps those of the “great writers, business people, the creative actors,” might end up the least valuable.^[19] Blundin answered that it would be a political job that is “completely irrelevant already.”

Claims and evidence

Factual claims made on air, checked against primary or high-quality sources.

Conclusion: what to watch

The empirical questions behind the disagreements.

The episode makes clearest that the RSI debate is really several debates. There is a definitional dispute over where on Ismail’s ladder recursion begins. There is an empirical dispute over how quickly self-improvement escapes software into atoms. And there is a governance dispute over whether frontier compute can be regulated without monitoring individual users. Socher’s verifiability principle is the best tool the conversation offers for all three. It predicts fast recursion where evaluation is cheap, slow recursion where it is physical, social or regulatory, and it implies that the most important safety infrastructure is better verification: of models, of biology and of who is on the other end of a video call.

It also shows the limit of his optimism. If capability follows verifiability, the most dangerous capabilities may be the easiest to verify. Code exploits pass or fail, and so do jailbreaks. Recursive’s fast lane and Glasswing’s threat model are the same lane. “P(doom) is zero” cannot settle that. Measurement can, and the coming year will provide some.

1. Statutory language. Whether Khanna’s text defines RSI by capability, compute or autonomy, and whether it distinguishes rung-1 tooling from rung-5 succession.
2. Independent replication. Third-party evaluations of Griffin’s Turing-style results and of Jev’s speed and cost claims.
3. Recursive’s first public artifact. Any paper or result showing closed-loop ideation–implementation–validation on all five Weston–Foerster axes would be an important milestone for both the science and the politics.
4. Liability doctrine. Insurance requirements and court decisions may shape deployment more than bans or licenses do, as Blundin argued.
5. Ecological staging under EO 14433. Whether the EPA framework includes staged, reversible field trials, which would be a working model of verify-then-scale governance.

Glossary

Tap a term to expand it.

Recursive self-improvement (RSI)

An AI system improving the process that produces its successors. Socher’s strong definition requires AI-performed ideation, implementation and validation inside an open-ended outer loop. Weaker forms include AI-assisted coding at AI labs.

Learnable axes

Weston & Foerster’s five dimensions along which a system could improve itself: parameters, training data, objective, architecture and overall code.^[10]

Open-endedness

A research tradition (Stanley, Clune, Rocktäschel and others) that seeks algorithms that keep generating novel, increasingly complex artifacts instead of converging on a fixed objective.^[15]

Verifiability principle

This article’s name for Socher’s claim that AI becomes superhuman wherever outcomes can be simulated or verified cheaply.

The Bitter Lesson

Rich Sutton’s 2019 essay arguing that general methods that scale with computation (search and learning) eventually beat approaches built on hand-crafted human knowledge.^[11]

Spiky vs. general ASI

Socher’s distinction between superhuman performance in particular domains (math, code, games) and superiority to all of humanity across all ten of his “spaces of intelligence,” including physical innovation.

Hard constraints

In Anthropic’s constitution, bright-line restrictions on actions Claude itself performs, such as never creating cyberweapons, which apply regardless of operator or user instructions.^[28]

Reward hacking

When an optimizing system satisfies the literal objective in ways its designers did not intend. Socher calls it “one of the biggest issues.”

Video Turing test

A test of whether people on a live video call can tell an AI avatar from a human. Tavus reports 48% for Griffin.^[41]

Anti-Turing test

Socher’s inversion: identify machines by asking for feats no human could perform, such as producing 50,000 lines of code in ten seconds.

KYUC

“Know your use case,” Socher’s proposed extension of know-your-customer rules to governing what an AI system is used for.

Decision model / System One model

A model that returns typed, bounded outputs (choice, score, boolean) with calibrated probabilities at low latency, instead of generating free text.^[33]

Jevons paradox

The observation (Jevons, 1865) that efficiency gains can increase total consumption of a resource by lowering its effective price.^[35]

Gene drive

A genetic construct, often CRISPR-based, that is inherited at higher than Mendelian rates, allowing a trait to spread through a wild population.^[47]

Sterile insect technique

Releasing large numbers of sterilized males to suppress a pest population. EO 14433 prioritizes this approach.^[45]

Moravec’s paradox

High-level reasoning is computationally cheap and sensorimotor skill is expensive, so AI automates “hard” cognitive tasks before “easy” physical ones.^[19]

Collingridge dilemma

A technology’s effects are hard to predict while it is still easy to control, and hard to control once they are clear.^[27]

Sources

Numbered in order of first citation. Accessed Oct. 6, 2026.

1. Moonshots with Peter Diamandis, EP #299, “Recursive’s $670M Bet on Self-Improving AI, Sonnet 5.5 Hits 70%, Elon Co-Leads Pentagon Push,” YouTube, Oct. 3, 2026. https://www.youtube.com/watch?v=Blyb1D927pM
2. Apple Podcasts episode listing, Moonshots EP #299. https://podcasts.apple.com/us/podcast/recursives-%24670m-bet-on-self-improving-ai-sonnet-5/id1648228034?i=1000793011855
3. GV, “Recursive Superintelligence: Why Self-Improving AI is the Next Frontier,” May 2026. https://www.gv.com/news/recursive-superintelligence-self-improving-ai
4. Tech Funding News, “UK AI startup Recursive hits $4.65B valuation with $650M raise from Nvidia and GV,” May 2026. https://techfundingnews.com/uk-ai-startup-recursive-hits-4-65b-valuation-with-650m-raise-from-nvidia-and-gv/
5. Amazon, “Recursive Signs $410 Million, Multi-Year Collaboration with AWS to Scale Self-Improving AI,” Jul. 28, 2026. https://press.aboutamazon.com/aws/2026/7/recursive-signs-410-million-multi-year-collaboration-with-aws-to-scale-self-improving-ai
6. TechCrunch, “Recursive Superintelligence signs $410M compute deal with Amazon,” Jul. 28, 2026. https://techcrunch.com/2026/07/28/recursive-superintelligence-signs-400-compute-deal-with-amazon/
7. Benjamin F. Jones, “The Burden of Knowledge and the ‘Death of the Renaissance Man’,” Review of Economic Studies 76(1), 2009. https://doi.org/10.1111/j.1467-937X.2008.00531.x
8. Nicholas Bloom, Charles I. Jones, John Van Reenen & Michael Webb, “Are Ideas Getting Harder to Find?” American Economic Review 110(4), 2020. https://doi.org/10.1257/aer.20180338
9. Michael Park, Erin Leahey & Russell J. Funk, “Papers and patents are becoming less disruptive over time,” Nature 613, 2023. https://doi.org/10.1038/s41586-022-05543-x
10. Jason Weston & Jakob Foerster, “AI & Human Co-Improvement for Safer Co-Superintelligence,” arXiv:2512.05356, Dec. 2025. https://arxiv.org/abs/2512.05356
11. Rich Sutton, “The Bitter Lesson,” Mar. 13, 2019. http://www.incompleteideas.net/IncIdeas/BitterLesson.html
12. Business Wire, “Parallel Bio Secures $21 Million in Series A to Advance Human-First Drug Discovery,” Jun. 12, 2025. https://www.businesswire.com/news/home/20250612697548/en/Parallel-Bio-Secures-%2421-Million-in-Series-A-to-Advance-Human-First-Drug-Discovery
13. I.J. Good, “Speculations Concerning the First Ultraintelligent Machine,” Advances in Computers 6, 1965. https://doi.org/10.1016/S0065-2458(08)60418-0
14. Jürgen Schmidhuber, “Gödel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements,” arXiv:cs/0309048, 2003. https://arxiv.org/abs/cs/0309048
15. Jeff Clune, “AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence,” arXiv:1905.10985, 2019. https://arxiv.org/abs/1905.10985
16. Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange & Jeff Clune, “Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents,” arXiv:2505.22954, 2025. https://arxiv.org/abs/2505.22954
17. Rishi Bommasani et al., “On the Opportunities and Risks of Foundation Models,” arXiv:2108.07258, 2021. https://arxiv.org/abs/2108.07258
18. CNBC, “Khanna to introduce AI safety bill with ban on ‘recursive’ technology until safeguards exist,” Sept. 28, 2026. https://www.cnbc.com/2026/09/28/khanna-ai-safety-bill.html
19. “Moravec’s paradox” (after Hans Moravec, Mind Children, 1988), Wikipedia. https://en.wikipedia.org/wiki/Moravec%27s_paradox
20. The Next Web, “Ro Khanna plans a bill to ban self-improving AI, CNBC reports,” Sept. 2026. https://thenextweb.com/news/human-control-over-ai-act-khanna-self-improving-ai
21. Quartz, “Ro Khanna introduces bill to ban self-improving AI,” Sept. 29, 2026. https://qz.com/ro-khanna-human-control-over-ai-act-self-improving-ban-092926
22. Common Dreams, “Citing ‘Civilizational Extinction Risk,’ Khanna to Intro Bill to Ban Recursive AI,” Sept. 2026. https://www.commondreams.org/news/ro-khanna-recursive-ai
23. Common Dreams, “68% of US Voters Back Sanders/Casar Bill for AI Pause, Ban on Superintelligence: Poll,” Sept. 10, 2026. https://www.commondreams.org/news/sanders-casar-ai-bill-poll
24. EU Artificial Intelligence Act, Article 51: Classification of general-purpose AI models as models with systemic risk. https://artificialintelligenceact.eu/article/51/
25. HistoryofInformation.com, “Pope Alexander VI issues the bull Inter multiplices” (1501). https://historyofinformation.com/detail.php?id=2414
26. Pessimists Archive. https://www.pessimistsarchive.org/
27. “Collingridge dilemma” (after David Collingridge, The Social Control of Technology, 1980), Wikipedia. https://en.wikipedia.org/wiki/Collingridge_dilemma
28. Anthropic, “Claude’s Constitution” (Hard constraints). https://www.anthropic.com/constitution
29. Anthropic, “Project Glasswing: Securing critical software for the AI era,” Apr. 7, 2026. https://www.anthropic.com/glasswing
30. Anthropic, “Claude Sonnet 5.5,” Sept. 28, 2026. https://www.anthropic.com/claude-sonnet-5-5
31. Artificial Analysis, “Gemini 4 Argon: Google is back as one of the top three labs in intelligence achieved,” Sept. 30, 2026. https://artificialanalysis.ai/articles/gemini-4-argon-google-top-three-labs
32. Google, “Introducing Gemini 4 Argon,” The Keyword, Sept. 30, 2026. https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-4-argon/
33. TypeSafe AI, “Introducing System One Models & Jev,” Sept. 15, 2026. https://typesafe.ai/blog/introducing-system-one-models-and-jev
34. Daniel Kahneman, Thinking, Fast and Slow (2011), overview. https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow
35. W. Stanley Jevons, The Coal Question (1865), Library of Economics and Liberty. https://www.econlib.org/library/YPDBooks/Jevons/jvnCQ.html
36. Jessica Hamzelou, “An AI ‘mind-reading’ tool can reconstruct what you’re looking at based on a brain scan,” MIT Technology Review, Oct. 1, 2026. https://www.technologyreview.com/2026/10/01/1145588/ai-mind-reading-reconstructs-what-youre-looking-at/
37. Weizmann Institute (Irani group), “Brain-IT: Image Reconstruction from fMRI via Brain-Interaction Transformer,” arXiv:2510.25976. https://arxiv.org/abs/2510.25976
38. Emily J. Allen et al., “A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence” (Natural Scenes Dataset), Nature Neuroscience 25, 2022. https://doi.org/10.1038/s41593-021-00962-x
39. Yu Takagi & Shinji Nishimoto, “High-resolution image reconstruction with latent diffusion models from human brain activity,” bioRxiv / CVPR 2023. https://doi.org/10.1101/2022.11.18.517004
40. Garrett B. Stanley, Fei F. Li & Yang Dan, “Reconstruction of Natural Scenes from Ensemble Responses in the Lateral Geniculate Nucleus,” Journal of Neuroscience 19(18), 1999. https://doi.org/10.1523/JNEUROSCI.19-18-08036.1999
41. Tavus, “Griffin: The First Human Interaction Model.” https://www.tavus.io/griffin
42. Decrypt, “This AI Is Already Fooling People on Video Calls Into Thinking It’s Human, Company Says,” Oct. 2026. https://decrypt.co/379975/tavus-griffin-ai-fools-people-video-thinking-human
43. Cameron R. Jones & Benjamin K. Bergen, “Large Language Models Pass the Turing Test,” arXiv:2503.23674, 2025. https://arxiv.org/abs/2503.23674
44. CNN, “Arup revealed as victim of $25 million deepfake scam involving Hong Kong employee,” May 16, 2024. https://www.cnn.com/2024/05/16/tech/arup-deepfake-scam-loss-hong-kong-intl-hnk
45. The White House, Executive Order “Eliminating Disease-Carrying Pests and Restoring Enjoyment of the Great Outdoors,” Sept. 29, 2026. https://www.whitehouse.gov/presidential-actions/2026/09/eliminating-disease-carrying-pests-and-restoring-enjoyment-of-the-great-outdoors/
46. Federal Register, Executive Order 14433, “Eliminating Disease-Carrying Pests and Restoring Enjoyment of the Great Outdoors,” Oct. 2, 2026. https://www.federalregister.gov/documents/2026/10/02/2026-20320/eliminating-disease-carrying-pests-and-restoring-enjoyment-of-the-great-outdoors
47. Kevin M. Esvelt, Andrea L. Smidler, Flaminia Catteruccia & George M. Church, “Concerning RNA-guided gene drives for the control of wild populations,” eLife 3:e03401, 2014. https://doi.org/10.7554/eLife.03401
48. World Health Organization, World Malaria Report 2025. https://www.who.int/teams/global-malaria-programme/reports/world-malaria-report-2025
49. The Hill, report on Hegseth’s Project Meridian announcement, Sept. 30, 2026. https://thehill.com/policy/defense/6121108-pete-hegseth-pentagon-project-meridian-warfare-future/
50. GovConWire, “Hegseth Unveils Project Meridian & 5 Other DOW Initiatives,” Sept. 2026. https://www.govconwire.com/articles/hegseth-project-meridian-dow-initiatives
51. NPR, “Pentagon taps Elon Musk and Palmer Luckey for strategy study,” Oct. 6, 2026. https://www.npr.org/2026/10/06/nx-s1-5991899/elon-musk-palmer-luckey-pentagon-drones-ai
52. Defense One, “Hegseth plans ‘Autonomous Warfare Command’,” Sept. 2026. https://www.defenseone.com/policy/2026/09/hegseth-plans-autonomous-warfare-command/416358/
53. Air & Space Forces Magazine, “Hegseth Unveils New Autonomous Warfare Command,” Sept. 2026. https://www.airandspaceforces.com/hegseth-drones-autonomous-warfare-command-state-of-the-force/
54. U.S. Department of Defense, Directive 3000.09, “Autonomy in Weapon Systems,” Jan. 25, 2023. https://www.esd.whs.mil/portals/54/documents/dd/issuances/dodd/300009p.pdf
55. RBC-Ukraine, “Zelenskyy says Ukraine now makes 10 million drones a year, aims for 20 million,” Jul. 2026. https://newsukraine.rbc.ua/news/zelenskyy-says-ukraine-now-makes-10-million-1784114976.html
56. James Bessen, “Automation and jobs: when technology boosts employment,” Economic Policy 34(100), 2019. https://doi.org/10.1093/epolic/eiaa001
57. Tal Shomrat & Michael Levin, “An automated training paradigm reveals long-term memory in planarians and its persistence through head regeneration,” Journal of Experimental Biology 216(20), 2013. https://journals.biologists.com/jeb/article/216/20/3799/11702/
58. U.S. FDA, “FDA Announces Plan to Phase Out Animal Testing Requirement for Monoclonal Antibodies and Other Drugs,” Apr. 10, 2025. https://www.fda.gov/news-events/press-announcements/fda-announces-plan-phase-out-animal-testing-requirement-monoclonal-antibodies-and-other-drugs
59. Neuralink, “Pretraining on 50,000 Hours of Unlabeled Brain Data,” Oct. 1, 2026. https://neuralink.com/updates/pretraining-on-50000-hours/
60. Alec Radford, Rafal Jozefowicz & Ilya Sutskever, “Learning to Generate Reviews and Discovering Sentiment,” arXiv:1704.01444, 2017. https://arxiv.org/abs/1704.01444