A warehouse robot that worked perfectly yesterday is given a new returns workflow today. The boxes are slightly larger. The packing material is different. The lighting in this aisle is dimmer. No new training run is scheduled.

In robotics, the hard problem is no longer solving a single manipulation task under ideal conditions. It is handling small, relentless variation without brittle retraining. Modern Vision-Language-Action (VLA) systems can perform impressively in controlled setups, but they remain reactive at their core. They observe the scene and select the next action given a goal. For short-horizon tasks, that works.

The trouble begins when actions interact across time. A grasp that technically succeeds may create a collision two steps later. A slightly unstable placement can trigger a recovery loop that costs thirty seconds. Reactive policies often repeat near-miss actions because nothing in their internal state marks a trajectory as fundamentally flawed rather than temporarily unlucky.

In our previous essay on Cortex 1.6, we described how robots can improve by learning from execution itself, drawing signal from how tasks unfold rather than simply whether they succeed. That made systems more robust after mistakes occurred.

With Cortex 2.0, Sereact pushes the idea further upstream. Instead of learning only from what happened, the system evaluates possible futures before committing motion, generating and scoring imagined continuations of the scene to avoid unstable trajectories in the first place.

Plan, imagine, then execute

Instead of immediately choosing an action, Cortex 2.0 generates several imagined continuations of the scene in visual latent space. These are short predicted futures, each corresponding to a different plausible sequence of movements.

Those imagined futures are scored using the reward logic introduced in Cortex 1.6. The model estimates progress toward the goal, risk of instability or collision, and expected smoothness. Only the highest-scoring candidate shapes the execution policy.

The reactive controller remains, running at high frequency to make fine motor corrections. But it is no longer acting blindly. Its movements are influenced by trajectories that already look stable. In the video below, we depict the Cortex 2.0 architecture, which now includes a world model generating candidate futures, PRO scoring, and execution conditioning. The shift is subtle but material. The system no longer treats every next action as equally plausible. It favors futures that look coherent before instability appears.

Why visual prediction?

Planning in visual latent space is pragmatic. Images encode geometry, contact, and occlusion in a representation that transfers across hardware platforms. A box and its contents look similar regardless of the arm that manipulates them. Planning in joint space would bind reasoning to specific kinematics.

But let’s not forget that latent predictions are imperfect. A world model can generate futures that look plausible but diverge from real dynamics. Cortex 2.0 mitigates this by training on real production data rather than simulation, grounding predictions in warehouse interactions. Rollouts are short and continuously corrected by the reactive controller, limiting error accumulation. Each deployment produces prediction-versus-reality comparisons, tightening alignment over time.

Subscribe now

How much thinking is enough?

Cortex 2.0 does not fix the amount of planning. The parameter K controls how many candidate futures are generated per decision. As K increases, success rates rise, as does latency.

In evaluations, K is set to 2 to balance foresight and throughput. For high-stakes manipulations, one can use more planning, while for low-risk tasks we can afford to use less planning. In this way, the robot has an explicit task-specific thinking budget.

Benchmarking Cortex 2.0

In the multi-step shoebox manipulation task, Cortex 2.0 completes the full sequence - opening the box, removing packing material, and extracting both shoes - with a 95% success rate, dramatically higher success than open-source baselines, completing the task reliably end-to-end without human intervention while other methods frequently stall or fail entirely.

In fine-grained screw sorting, Cortex achieves near-perfect per-operation success (~95%+) and the shortest average completion times, while baselines often hit unrecoverable states and fail to finish.

In cluttered sorting of items and trash, Cortex again outperforms its peers, reaching per-operation success rates above 90% and completing all rollouts autonomously, whereas all baseline policies require human intervention or time out before finishing.

Across tasks, Cortex’s combination of high success, low intervention, and reduced duration under identical runtime constraints highlights planning’s concrete impact on real-world manipulation performance. These results come from controlled evaluations, but the system's longer-term value depends on what happens when it operates continuously across a growing fleet of 100+ robots.

Subscribe now

Learning in production

With Cortex 2.0, deployment generates more than execution traces. Each imagined trajectory can be compared with what actually happened so that the model learns where its forecasts were accurate and where they drifted.

As predictive alignment improves, planning becomes more reliable. As reliability increases, deployment expands. As deployment expands, the distribution of physical interactions broadens. The flywheel compounds forecasting accuracy as well as execution quality.

Today, Cortex 2.0 is being validated most heavily in returns handling, where sequence-dependent failures are common and intervention costs are high. Whether similar gains hold across the long tail of industrial manipulation remains an empirical question.

Reactive systems can perform well in structured tasks. The test for world-model planning is whether it consistently turns fragile autonomy into durable autonomy across varied environments.

The architecture is in place. The next phase is scale.

Read more about Cortex 2.0

The bottleneck today is less about perception, planning, or control in isolation. Those capabilities largely exist. The challenge lies in the brittle interfaces that bind them together. Traditional robotics stacks rely on hand-engineered pipelines where perception feeds symbolic state into planners, which then dispatch actions to controllers. Each stage encodes assumptions that break under real-world variation. When they do, failures are physical, costly, and difficult to recover from.

A growing class of robotics efforts aims to replace these brittle interfaces with learned systems trained directly on real interaction data. One of the more compelling examples is Sereact, a Stuttgart-based robotics company deploying learning-based manipulation systems in live production environments. This essay examines Sereact’s Cortex, and in particular Cortex 1.6, as evidence that robotics may be entering a new phase of learning-driven progress.

Cortex treats manipulation as a learning problem end to end because learning should determine how sensory inputs are translated into actions, especially under real-world variation. Cortex 1.6 strengthens this claim by changing how learning signal itself is extracted.

Why robotics has resisted scaling

Robotics is uniquely unforgiving. Language models can hallucinate. Coding agents can stumble and try again inside a virtual machine. Robots drop objects, collide with equipment, or endanger nearby people. Small errors often cascade into failure.

Historically, the field narrowed the problem to achieve robustness. Robots were deployed in highly structured environments and tuned for fixed tasks. When variation crept in, engineers patched systems with heuristics, additional sensors, or narrowly targeted data collection. Over time, these stacks grew complex, fragile, and expensive to maintain.

By contrast, frontier models in other domains improved by absorbing variation through scale. Instead of encoding rules for every edge case, they learned directly from large and diverse data distributions. Robotics largely missed this shift because large-scale interaction data was difficult to collect, expensive to label, and hard to reuse across deployments.

Subscribe now

Sereact Cortex: a vision-language-action (VLA) model

Cortex is a vision-language-action model trained to map sensory inputs directly to robot actions, bypassing brittle intermediate abstractions. Instead of separating perception, planning, and control into independently engineered modules, Cortex learns the full loop as a single system.

Crucially, this learning takes place on real robot interaction data collected across tasks, objects, and environments within customer facilities. The underlying hypothesis is that generalization emerges from exposure to sufficient diversity, and that failures should be incorporated into learning rather than handled as special cases downstream.

This stands in contrast to how most robotic systems learn today. In many production systems, learning is driven by sparse terminal outcomes. A task either completes or it does not, and learning happens after the fact. In physical systems, this abstraction is limiting.

Learning from execution, not just outcomes

Cortex 1.6 changes how learning signal is obtained. Instead of relying solely on hand-engineered rewards or terminal success labels, it introduces a learned Process-Reward Operator that evaluates execution as it unfolds.

Rather than asking only how a task ends, the system continuously assesses how it is progressing. Signals related to stability, completion likelihood, and risk are inferred from raw operational telemetry such as motion dynamics, force profiles, retries, and recovery behavior. This allows reinforcement learning to operate on dense, process-level feedback grounded in real execution rather than sparse post hoc outcomes.

Importantly, this reward model is trained entirely from real deployment data. That reflects a core advantage of Sereact’s approach: a fleet of robots operating continuously across third-party logistics, warehousing, e-commerce, and manufacturing environments provides a steady stream of high-fidelity interaction data.

Outcome-based learning collapses rich execution dynamics into a single label. Smooth executions and fragile recoveries are treated as equivalent if both succeed. By contrast, execution-aware learning can reinforce behavior that completes tasks with margin and suppress behavior that relies on late or unstable corrections, even when both technically succeed.

Because execution itself provides learning signal, improvement can continue during deployment. Optimization shifts away from raw completion rates toward reliability.

Empirical evaluation in production workflows

To evaluate this learning regime, Cortex 1.6 was tested on three live production workflows: pick-and-place, shoebox opening, and returns handling. All data was collected from real deployments. Performance was compared across three systems: a baseline vision-language-action policy trained via imitation learning, Cortex 1.5, which relies on binary success signals and human-triggered policy patching, and Cortex 1.6, which incorporates dense execution-level rewards via the Process-Reward Operator.

Across all tasks, Cortex 1.6 achieves the highest overall success rates, outperforming both the imitation baseline and Cortex 1.5.

More notably, the introduction of execution-level rewards substantially accelerates learning. Time to convergence is reduced by roughly a factor of two relative to Cortex 1.5, and by more than a factor of three relative to the baseline system.

Improvements are not limited to success rates or learning speed. Recovery behavior improves materially. After an initial failure, recovery success increases from approximately 45% in the baseline system, to around 65% with Cortex 1.5, and to roughly 80% with Cortex 1.6. Average task retries per episode fall by 30-50% after training with execution-aware rewards.

What the numbers mean

Taken together, the results support three empirical conclusions.

First, Cortex 1.6 demonstrates robust generalization under real operational conditions. Performance gains persist across different workflows and under distribution shifts that commonly break deployed systems, including novel objects, clutter, and execution noise.

Second, learning becomes markedly more efficient when reward is derived from execution itself. Replacing sparse terminal feedback with dense, process-level signal reduces the amount of interaction time required to reach high performance. Learning progresses through incremental improvements rather than episodic jumps, even while robots are already deployed.

Third, the gains extend beyond headline success rates. Recovery behavior improves and retries decrease, indicating that the system is learning how to act well throughout a task, not merely how to reach a successful endpoint.

These findings highlight a broader lesson. In traditional robotics learning, sparse or delayed feedback obscures where instability begins and where robustness is earned. By exposing learning algorithms to execution-level signal, Cortex 1.6 changes both what is learned and how quickly it is learned. Reinforcement learning becomes grounded in real operational behavior rather than post hoc outcomes.

If frontier models are to work reliably in the physical world, they must be trained on more than success and failure. Cortex 1.6 offers early evidence that learning directly from execution is a viable path toward robots that are not only capable, but consistently reliable.

Subscribe now

Over the past few weeks, three essays and a podcast have been circling my mind. One essay, Why AGI Will Not Happen, by Tim Dettmers, argued that hardware, energy, and economic constraints impose hard limits that will increasingly bind AI progress. A subsequent response, Yes, AGI Can Happen - A Computational Perspective, by Dan Fu, took a more bullish view, arguing that efficiency gains and system-level optimization continue to compound despite those constraints. A third, by Andrej Karpathy, former Director of AI at Tesla and co-founder of OpenAI, reviewed what large language models actually delivered in 2025. Finally, I listened to a 1hr conversation with Sebastian Borgeaud, who leads pre-training for Gemini 3 at Google DeepMind, and heard something that resonates with me: “We’re not really building a model anymore. We’re building a system.”

These position statements arrived at an interesting moment. As 2025 draws to a close, markets are oscillating between excitement and anxiety. AI is discussed in the language of bubbles, frothy capital cycles, and circular deals, while hyperscale capex commitments, datacenter build-outs, and power infrastructure investments are predicated on revenue projections that assume growing usage and continued model improvement. Skepticism is understandable.

On Air Street Press, we prefer to step away from market sentiment and back toward the technical and empirical record. Regardless of how one feels about valuations or deal structures, AI systems made a genuine leap forward this year. Not a single breakthrough per se, but a broad shift in capability, usability, and integration that surprised even the people building them.

This essay is an attempt to reconcile those signals. We won’t speculate about distant futures, but we’ll take stock of what 2025 actually delivered, why progress looked the way it did under real constraints, and what kind of AI progress now compounds as we head into 2026.

2025 was not incremental

The clearest way to ground this claim is to focus on what changed in practice.

Both Andrej Karpathy’s review of the year and our Air Street Capital State of AI Report 2025 point to the same inflection: 2025 was the year AI crossed a genuine usability threshold. Although reasoning, planning, and tool use did not become flawless, they became dependable enough to deploy without constant supervision. Models started showing up as working components inside real systems.

That shift showed up across capability, deployment, and adoption at once. Frontier models extended their reach on reasoning and longer-horizon tasks, while agentic systems moved out of demos and into tightly scoped roles in coding, customer support, and conversational interfaces. Usage broadened rapidly as AI tools became part of daily professional routines, particularly in software engineering rather than optional experiments, narrowing the gap between what models could do and what organizations were willing to rely on. It wasn’t long ago that engineers argued that early coding tools were only good for people who couldn’t code well. Today, even the very best engineers use coding tools in their daily work - far beyond autocomplete and copy/pasting code blocks into ChatGPT or Claude for Q&A.

But there’s more that contributed to compressing the distance between capability and use. Distillation and inference-side optimization lowered the cost of competence, allowing these systems to spread beyond frontier users. As a result, models increasingly appeared more as infrastructure - embedded, assumed, and quietly doing work.

This pattern is visible across benchmarks and model behavior. Our Air Street Capital State of AI survey of over 1,400 practitioners shows that the vast majority of respondents now use AI tools weekly or daily across both technical and non-technical roles, with a significant share paying out of pocket and integrating these systems directly into how work gets done. Even holding model scale constant, that level of deployment would have made 2025 exceptional. But scale did not stand still. Capabilities advanced as well, perhaps unevenly, but unmistakably.

That combination updates a key prior: progress didn’t stall under visible constraints.

Constraints shaped progress

By 2025, the constraints shaping AI progress were well understood and widely felt. Tim Dettmers articulated them most clearly: compute costs dominated economics, power availability shaped deployment decisions, and inference workloads mattered more than training runs. Memory bandwidth, latency, and reliability featured prominently in how practitioners reasoned about what was feasible. These constraints framed much of the year’s discussion about where AI progress could realistically go.

And yet, under constraint, the field adapted in ways that shifted where progress came from. One of the clearest signals of this adaptation shows up in the economics of deployed models. As documented in the Air Street Capital State of AI Report 2025, the amount of model capability available per dollar has improved at an exceptional pace. Using both benchmark-based measures and real-world pricing, intelligence-per-dollar for leading models has been doubling every few months rather than every few years, roughly every 3-4 months for Google’s flagship models, and every 6-8 months for OpenAI’s. Between early 2023 and late 2025, the cost-adjusted performance of frontier language models increased by more than an order of magnitude, even as absolute model capability continued to rise. Prices fell sharply while benchmarks climbed, producing a sustained and measurable improvement in what users could afford to deploy.

This improvement came from working with constraints. Inference-time compute began to scale faster than training as teams learned to spend compute selectively rather than uniformly. Sparse architectures, routing mechanisms, and distillation techniques moved into production systems, reshaping the cost-capability trade-off. At the same time, evaluation, scheduling, and reliability emerged as first-order concerns, reflecting the reality that these models were no longer curiosities but infrastructure expected to work predictably at scale.

As a result, progress became more disciplined, more engineered, and more tightly coupled to economics. Capability gains increasingly emerged from how systems were composed and operated, signalling a clear shift from pure research to scaled-up engineering. That shift is the central through-line of what follows.

AI is now a global optimization effort

This is the deeper reason 2025 should update expectations.

For years, it was common to say that only a small number of highly specialised teams were capable of building AI. I’d argue this is no longer the right framing. As AI has expanded from a model-centric endeavour into a system, it now spans energy and power infrastructure, datacenters, silicon and interconnects. Above that sit systems software, compilers and runtimes, data pipelines, model training and inference, evaluation, and deployment. Taken together, this has turned AI into the highest-leverage optimization project in the global economy. The result is a widening aperture for contribution, pulling in talent from across signal processing, compilers, kernels, networking, distributed systems, web-scale infrastructure, hardware-aware performance engineering, and many other corners of modern software and systems engineering.

This matters because the bottlenecks I described above are not static. Every constraint exposed by scale becomes a new surface for optimization. And those surfaces are precisely where this talent excels.

In fact, Sebastian Borgeaud’s account from inside Google DeepMind makes this observation concrete. Gemini 3 did not improve because of a single architectural leap. It improved because hundreds of people worked across each of data, models, infrastructure, evaluation, and post-training, integrating thousands of incremental improvements into a coherent whole.

“We’re not really building a model anymore,” he said. “We’re building a system.”

Scale still matters

Evidence from recent large-scale pre-training efforts suggests there remains meaningful headroom in scale-driven progress. Essential AI’s RNJ-1 results show that carefully designed pre-training regimes continue to unlock improvements in reasoning and generalization, even without dramatic increases in raw parameter count. Similarly, the Gemini 3 release reflects what Oriol Vinyals described as progress driven by “better pre-training and better post-training” rather than a single architectural break - a signal that optimization within pre-training itself remains far from exhausted.

This echoes a conversation I had in late 2024 with Eiso Kant at poolside, at a moment when talk of an imminent “scaling wall” was reaching peak volume. The point then was not that scale alone would solve everything, but that deep learning has repeatedly absorbed apparent limits by changing how scale is expressed - through data, architecture, objectives, and systems design. A year on, that pattern looks intact, even accounting for the market jitters in between.

As Borgeaud notes, scale now operates as one component in a broader optimization loop. Architecture, data quality, evaluation design, post-training, and inference efficiency increasingly dominate marginal returns. The shift from an effectively unlimited data regime to a finite one has altered how research proceeds, reintroducing discipline around data use while expanding the importance of architectural and algorithmic efficiency. Pre-training, post-training, and inference-time optimization now compound rather than substitute for one another.

Scaling laws are not broken. They have been absorbed into system-level optimization.

From models to systems - and into 2026

The most profound implication of this shift is not commercial, but epistemic - about how knowledge is created, tested, and advanced.

In my recent essay on AI for science, I argued that the real transition occurs when AI moves beyond prediction and into discovery loops - generating hypotheses, designing experiments, analysing results, and suggesting the next iteration. For those of us deep in the field, starting a new research project or line of investigation without kicking off a conversation with our favorite AI seems wild. A year or so ago, models just weren’t good enough to provide this kind of nuanced feedback.

Seen in this light, the move from models to systems is the mechanism by which progress now compounds. Once AI is embedded in iterative workflows across research, engineering, and scientific discovery, improvement becomes endogenous. The system can accelerate its own development by shortening the loop between hypothesis, execution, and evaluation.

This framing also clarifies what to expect next. If 2025 was the year AI became reliably useful at scale, 2026 will be defined by whether system-level optimization continues to compound. We could still see dramatic progress leaps, but it’s fair to expect that iterative improvements across an increasingly large surface of AI development will continue to move the field forward.

Against a backdrop of market skepticism and capital-cycle anxiety, the technical record of 2025 offers a useful anchor. AI progress did not slow down: it moved further, and in more consequential ways, than many expected.

Every industry has its high‑stakes systems: software that cannot fail because the cost of failure is borne by real people. In clinical research, that system is the infrastructure that manages participant journeys through clinical trials. These trials are the gateway through which every new therapy must pass, and their performance determines how quickly medicines reach patients.

Yet this critical layer has long been held together by inboxes, spreadsheets, and heroic vigilance. Trials generate a constant stream of emails, phone calls, documents, reschedules, protocol changes, and participant questions. In fact, trials now generate >250% more data points compared to 10 years ago, leading teams to compensate with manual effort and constant oversight. The outcome is predictable: delays, deviations, and participants waiting longer than they should for treatments that could change or save their lives.

This pressure is only increasing. Over the past decade, AI-first biotech companies have expanded the number of viable drug candidates reaching the clinic. Indeed, the second generation (founded ‘18-’21) of AI biotechs alone is expected to bring more drugs into the clinic in 2026 than the first generation (founded ‘11-’14) managed across multiple prior years. The bottleneck has shifted from discovering molecules to running trials fast enough, cleanly enough, and at sufficient scale to keep up.

That’s why earlier this year we invested in Delfa. The company is rebuilding one of the most critical pieces of clinical trial infrastructure from first principles. With its Participant Relationship Management (PRM) system, Delfa is taking the next step toward rewriting the operating system for clinical trials.

Managing participant relationships at scale

The Delfa PRM is built on modern AI systems that can interpret unstructured communication, understand documents, coordinate tasks, and maintain context across an entire participant journey. Instead of acting as a passive system of record, it operates continuously in the background. The system does the work, rather than asking humans to do it on its behalf.

With Delfa PRM, clinical trial sites get:

• A unified inbox where AI triages communication and extracts structured data.

• Protocol-aware workflows that generate and manage tasks automatically.

• Scheduling that adapts to visit windows, constraints, and participant availability.

• Real-time document understanding across forms, labs, and reports.

• A complete operational memory of every interaction, without manual upkeep.

• And automated, multi-channel text or audio-based recruitment sequences.

Taken together the impact is immediate for research sites: faster recruitment, fewer deviations, higher‑quality data, and staff freed from the coordination burden that has historically limited scale.

For participants, the impact is simpler and more important. Communication is consistent. Follow‑ups happen. Appointments stay on track. The experience feels human, not bureaucratic.

High-stakes software, AI-first

Delfa’s PRM also points to a broader shift underway across high‑stakes software categories. For years, enterprise software focused on digitising workflows and making work visible. As AI systems become capable of executing multi‑step, high‑variance tasks, the challenge has changed. The new constraint is no longer recording work, but orchestrating it reliably under real‑world conditions.

Delfa is doing exactly that for clinical research. PRM adds the operational layer that allows trials to run with the speed and precision the work demands. When trials run better, medicines move faster, development costs fall, and patients see the benefits sooner.

This is what rebuilding high‑stakes software AI‑first makes possible.

Subscribe now

A central question has shaped the AI-for-science debate: does AI merely reproduce the knowledge it is trained on, or can it generate new knowledge altogether? Over the past year, this question has shifted from theoretical to empirical. Advances in reasoning models, agentic systems, and autonomous research pipelines mean AI is beginning to function as an accelerator of scientific discovery.

At the national level, the United States has just launched the Genesis Mission, an attempt to build a scientific infrastructure in which AI helps drive simulation, data analysis, and experimental workflows. At the system level, platforms such as FutureHouse Kosmos and AI Scientist-v2 automate increasingly more of the research workflow. And at the level of active scientific projects, frontier models such as GPT-5 are already contributing concrete, verifiable steps across mathematics, physics, astronomy, biology, and materials science, as documented in OpenAI’s Early Science Acceleration Experiments.

In the State of AI Report 2025, we predicted that “open-ended agents will make a meaningful scientific discovery end-to-end.” Whether this happens this year or next matters less than the trajectory now clearly forming. Let’s dive in.

Reasoning models enter the scientific workflow

GPT-5 provides the clearest evidence that frontier models are beginning to contribute genuinely useful scientific support. In mathematics, GPT-5 contributed to four new results on previously unsolved problems, including a novel inequality in high-dimensional geometry, a new approach to a combinatorial question, and two further propositions derived from GPT-5’s candidate lemmas and proof sketches. All were verified by human experts. In one case, the model suggested a structural transformation that unlocked a proof direction that researchers had struggled to identify.

Outside mathematics, GPT-5’s contributions are smaller, but concrete nonetheless. In plasma physics, the model identified a symmetry in a simulation that the researchers had overlooked, leading to a corrected interpretation. In quantum systems, it traced a subtle boundary-condition error through a codebase. In astronomy, it proposed a new re-weighting method for exoplanet transit data that outperformed the team’s heuristic baseline. In computational biology, it redesigned an RNA modelling pipeline by replacing a Monte Carlo routine with an analytic approximation retrieved from literature. And in materials science, its alternative density-functional formulation reduced runtime by more than an order of magnitude.

To be clear, these contributions are not considered independent breakthroughs. But they do demonstrate that GPT-5 can produce intermediate reasoning steps that specialists accept as correct and sometimes useful. In this way, the role of the model is increasingly to explore large hypothesis spaces and propose candidate structures. Humans continue to supply intuition, constraints and verdicts. And that’s genuinely useful.

Subscribe now

System-level “AI scientists”

Where GPT-5 demonstrates the power of a single reasoning engine, system-level architectures such as Kosmos show how these models behave inside orchestrated research workflows.

A typical Kosmos run lasts around 12 hours, ingests ~1,500 scientific papers, and generates ~42,000 lines of code across data analysis, simulation, and visualisation modules. The result is a structured scientific artefact linking claims directly to the evidence that supports them.

Crucially, Kosmos has now undergone structured external evaluation. Independent PhD-level reviewers examined 850 claims from Kosmos outputs. They judged 79.4% to be supported by the underlying evidence. Support rates were 85.5% for data-derived claims and 82.1% for literature-derived ones, falling to around 60% for cross-domain hypotheses, the hardest category and the one most associated with novelty.

The case studies reveal both the system’s promise and its current limitations. In one run, Kosmos assembled a multi-step hypothesis connecting SOD2 enzymatic activity to oxidative-stress compensation in tumor microenvironments. Reviewers agreed that many sub-claims were coherent and literature-supported, but disagreed on whether the integrated mechanism was genuinely new. In materials science, Kosmos proposed a plausible relationship among defect energetics in perovskites, again judged likely correct but not obviously novel.

The evidence suggests that Kosmos can generate structured, evidence-linked scientific reasoning at scale, but that its ability to consistently produce new insight remains to be proven.

By contrast, AI Scientist-v2 is entirely in silico. It conducts machine-learning research end-to-end on standard benchmarks, designing experiments, running code, analysing output, and drafting manuscripts. In Sakana’s evaluation, one of three fully AI-generated papers passed the reviewer acceptance threshold at an ICLR workshop. This demonstrates autonomous research capability in computational domains, not physical scientific discovery.

A national AI stack for scientific discovery

The US Government’s recently announced Genesis Mission represents the most ambitious federal effort to date to build a national AI-enabled scientific capability. As laid out in the Presidential Action, Genesis aims to create a unified platform that integrates the Department of Energy’s scientific user facilities, national laboratories, high‑performance computing centers, and decades of federally funded datasets into a single AI‑accelerated research ecosystem.

The mission directs federal agencies to develop scientific foundation models, deploy AI agents capable of generating and testing hypotheses, and expand autonomous laboratory systems across priority domains including fusion energy, advanced nuclear technologies, climate and Earth system modelling, biomedicine, drug discovery, materials science, and grid resilience. It also establishes a new governance framework to manage safety, provenance, and access to these AI systems. Lastly, the project reflects a geopolitical shift in which scientific competitiveness - and national resilience - are tied to the strategic integration of AI across the full stack of discovery.

Subscribe now

How scientific practice is beginning to change

These developments are already reshaping aspects of the scientific method.

One shift is toward executable research. Kosmos and AI Scientist-v2 generate artifacts where claims are linked to code and data, creating a computational form of reproducibility. Another shift is the emergence of abundant machine attention. Systems like GPT-5 and Kosmos can sweep literature and simulation spaces at scales impossible for humans, shifting the bottleneck from idea generation to physical validation.

Research groups may also evolve into hybrids of human and digital labor. Instead of relying solely on students and postdocs, labs may soon supervise persistent fleets of agentic researchers. Finally, verification becomes the central constraint. While some fields provide clean verifiers, biology and medicine do not, yet these remain the domains with the greatest pressure to adopt AI.

So, how close are we to AI creating new knowledge?

Taken together, GPT-5, Kosmos, and AI Scientist-v2 provide a picture of meaningful but incomplete progress. GPT-5 has already contributed new results in mathematics and useful reasoning steps in multiple sciences. Kosmos produces large volumes of mostly correct scientific reasoning, with occasional glimmers of novelty. And AI Scientist-v2 shows that autonomous research is possible in constrained computational domains.

But none of these systems constitute a general-purpose scientific discoverer just yet. The frontier remains uneven across domains, and the line between new insight and recombination remains difficult to draw. The most defensible position today is that AI is beginning to accelerate scientific discovery and, in some aspects, originating it. And that is meaningful.

The path ahead

Whether open-ended agents achieve the State of AI Report 2025 prediction this year is almost beside the point. It is a matter of time. As Demis Hassabis has argued, the point of building increasingly general AI systems is not to automate what scientists already do, but to expand the scientific frontier itself. AlphaFold was the first demonstration that AI can deliver solutions to problems that resisted decades of human effort. The systems emerging today extend that logic: they explore hypothesis spaces humans cannot hold in mind, recognise structures we overlook, and recombine knowledge at scales that make new questions thinkable.

The trajectory is clear. For the first time, we are witnessing the early phases of a scientific ecosystem in which ideas, experiments, and interpretations emerge from a hybrid of human and machine intelligence. The systems deployed today are imperfect, uneven across domains, and still fundamentally dependent on expert oversight. Yet they mark the beginning of a shift in the practice of science that is unlikely to reverse.

Share

Last year, in our Embodied AI outtake from the State of AI Report, we argued that robotics had moved from being the “unloved cousin” of AI to an area undergoing a genuine renaissance driven by progress in foundation models. Since then, the field has advanced rapidly. On the research front, what was previously a collection of isolated systems is now beginning to reveal clearer architectural patterns from groups like AI2, Google DeepMind and NVIDIA, and on the commercial side, we’re seeing large-scale deployments of Embodied AI systems by robotics companies like Sereact and Wayve.

This essay revisits and updates our previous analysis with substantive progress in the field and potential future directions.

World models offer richer virtual playgrounds

In the last year, we’ve seen the release of Odyssey 2, Dreamer v4 and Genie-3, which all push beyond passive video prediction into richer, interactive environments. These models now support agent-conditioned rollouts, long-horizon temporal structure and even 3D persistence - capabilities that were unthinkable in early generative work. While these systems are yet to demonstrate sim-to-real transfer for robotics, this feels like their likely direction of travel.

This month, DeepMind’s SIMA-2 offered a clear glimpse of where this might go on the virtual side. In this system, a Gemini-powered agent plays 3D games, explains its plans and adapts to procedurally generated worlds created by models like Genie-3. Relatedly on the driving side, Wayve’s GAIA-2 uses a generative world model to roll out closed-loop driving trajectories in novel scenes, showing that similar ideas can be applied to real-world sensorimotor data.

Taken together, these systems suggest that world models are morphing into a high-potential training and evaluation substrate. Future embodied policies can be stress-tested cheaply, safely and at scale before being exposed to the real world.

Vision-Language-Action Models (VLAMs)

VLAMs have become the clearest expression of how the foundation‑model playbook is infusing new life into robotics. Instead of custom perception stacks and task‑specific controllers, robots are increasingly powered by large multimodal models that can interpret scenes, understand instructions and produce structured actions. This is a big deal.

Two architectural strategies currently dominate the research landscape:

• Insulated approaches, where the VLM backbone is kept largely fixed and robot‑specific learning happens in compact action experts trained on modest datasets (e.g. models like π₀.₅, GR00T N1, and to some extent SmolVLA). This offers stability, preserves pretrained knowledge and reduces the risk of catastrophic forgetting.

• End‑to‑end approaches, where perception, semantics and control are trained jointly (e.g. models like GR‑3, Gemini Robotics, RT-2). This unlocks richer grounding and enables models to internalise geometry, affordances and contact dynamics.

Despite their differences, these systems share a common thread: they consistently outperform older modular pipelines on generalisation, long‑horizon tasks and cross‑embodiment transfer. π₀.₅ executes multi‑step manipulation in entirely new homes. Gemini Robotics adapts a frontier foundation model to new robot bodies with limited additional data. GR00T N1 brings generalist humanoid control into the open‑weight ecosystem.

Taken together, they mark a meaningful concentration of progress toward large‑scale multimodal models that learn from broad data, reason over abstract goals and synthesize them into precise physical actions.

The rise of Chain-of-Action

A subtle but important shift is the move toward explicit planning layers. Instead of jumping straight from pixels to torques, newer systems produce a mid-level sequence of goal tokens, spatial anchors or trajectory hints - a kind of Chain-of-Action inspired by Chain-of-Thought in language models. This emerging pattern marks a notable departure from the traditional “end-to-end or bust” mentality that dominated early deep robotics.

These intermediate representations give embodied systems space to deliberate. They provide structure for breaking complex tasks into achievable steps, allowing models to reason about order, constraints and dependencies in a way that raw motor outputs simply cannot.

More importantly, explicit planning layers begin to answer one of the hardest questions in robotics: how to build systems that are both general and controllable. Planning tokens expose the model’s internal reasoning process, offering developers a handle on why a robot is doing something, not just what it is doing. This transparency is critical in environments where safety, verification and predictability matter.

Another advantage of using Chain-of-Action is that it creates a natural interface for cross-embodiment transfer. When a model outputs a symbolic plan followed by embodiment-specific execution, you can swap the robot without retraining the reasoning component. It is the same logic that allows large language models to generalise across tasks: abstract first, specialise later.

AI2’s Molmo-Act, Google DeepMind’s Gemini-ER and Sereact’s Cortex illustrate this trajectory in different ways. Molmo-Act uses language-conditioned reasoning steps to break down manipulation tasks. Gemini-ER enriches spatial and geometric understanding before delegating control. Cortex uses plan tokens to bridge a VLM-based perception stack with a high-frequency Motion Policy Expert.

Planning layers are still an emerging design choice, but they are exciting because they unlock something robotics has historically lacked: a modular, interpretable and reusable interface between understanding and action.

Sereact Cortex: real-world embodied AI in warehouses

Sereact is a Stuttgart/Boston‑based robotics company building embodied AI systems to automate warehouse picking. Its Cortex platform is an example of the modern embodied‑AI stack in action: a unified perception-planning-control system designed to operate under real‑world variability.

Sereact is deploying Cortex across Rohlik Group’s Knuspr and Gurkerl operations - part of a European e‑grocery leader serving more than one million customers and exceeding €1 billion in annual revenue. The rollout starts with 24 robots in Berlin and Vienna and expands toward 100+ systems across DACH. These robots run inside Rohlik’s Veloq fulfilment platform, working reliably in both chilled and ambient zones where traditional automation typically fails. Every action of every robot across the diverse fleet is captured and used as real-world training data to rapidly make Cortex smarter and smarter in a way that simulation and teleoperations cannot.

Cortex works by tokenising both perception and action. A dedicated VLM (“Lens”) interprets scenes and affordances; a planning layer converts these into discrete plan tokens describing the next sub‑steps of the task; and a Motion Policy Expert executes them as smooth, robot‑specific trajectories. This separation of understanding, planning and execution enables the same model family to generalise across arms, grippers and warehouse layouts while remaining debuggable in production.

At a technical level, Cortex compresses multi‑view RGB‑D inputs and deployment logs into discrete video tokens, and discretises short motion primitives into action tokens. The core model consumes text, image, video and action tokens in a single sequence, pretrained on mixed datasets and then finetuned with a continuous‑control expert using trajectory‑matching. A lightweight safety layer enforces limits and intervenes when uncertainty rises.

Wayve: Generalising public road autonomy

Outside of warehouse environments, London/SF-based Wayve is demonstrating what embodied AI generalisation looks like on public roads. Here, a single end-to-end driving model operated in 90 cities in 90 days, across Europe, North America and Asia - with no HD maps, no geofencing and no city-specific tuning. Importantly, 62% of the cities were entirely unseen and 14% of cities had zero prior data. This system logged more than 10,000 hours of AI driving.

Conditions spanned Tokyo’s narrow streets, Alpine terrains, dense European capitals, night driving, heavy rain and complex road geometry. The now-widely shared Tokyo sequence under typhoon conditions captures this vividly.

Wayve’s driving system offers some of the best empirical evidence to date that a single embodied model can handle diverse environments without retraining. No simulation benchmark comes close in terms of real-world diversity.

Humanoids are showing rapid progress

Humanoid robots have surged in investment, capability and architectural sophistication. NVIDIA’s GR00T N1 shows that modern multimodal representations and learned control stacks can operate increasingly complex bodies. Recent home robot prototype launches reinforce this momentum. Sunday Robotics debuted a home-focused humanoid designed for repeatable manipulation tasks, positioning itself explicitly as a pragmatic, deployment-driven alternative to more speculative projects. Meanwhile, 1X introduced its NEO platform with striking demos, though independent analysis indicates that much of the behaviour was teleoperated rather than autonomous, highlighting how uneven real capability remains across the category.

Despite this activity, there is still no equivalent of Wayve’s 90-city tour or Cortex’s multi‑site warehouse deployments in the humanoid space. Hardware reliability, safety certification and fleet‑scale data remain major constraints. As a result, the domain closely resembles autonomous driving circa 2017 - an ambitious technological trajectory, but a long road to safe, scaled deployment.

Into the next 12 months

Embodied AI is entering a phase where the question shifts from “can these systems work?” to “can they scale in real-world deployments?” The core ingredients are now in place: world models that provide increasingly realistic training grounds, VLAMs that unify perception and control under shared representations, and planning layers that expose a robot’s reasoning in a way that is inspectable and steerable. With Cortex operating in real warehouses and Wayve demonstrating continent‑scale generalisation on public roads, the gap between research prototypes and deployed systems is beginning to narrow.

On paper, it’s brilliant arbitrage. In practice, it’s a mirage. It rests on a fundamental category error: confusing operational improvement with business model transformation. Yes, AI can make workflows more efficient. No, that doesn’t turn a services company into a software company.

Today, French consulting giant Capgemini announced its $3.3B all-cash acquisition of WNS, a publicly listed BPO firm with 65,000 employees and deep roots in sectors like insurance, travel, and BFSI. Reuters framed the deal as a move into "business-process transformation" powered by generative and agentic AI.

But if you take Capgemini at its word, that AI capability is already present in WNS. On their website, WNS promotes a mature generative AI offering spanning customer service, KPO, and analytics. This isn’t some dusty call center, it’s a services company with AI traction.

So why did it sell for BPO multiples?

Once a BPO, always a BPO

Let’s look at the numbers:

• WNS FY2025 revenue: $1.315B

• Operating margin: 18.7%

• EBITDA FY2025: $236M

• Implied multiple: ~2.5× revenue, ~14× EBITDA

That’s a far cry from EV/EBITA multiples fetched by AI-transformed software peers.

This is exactly the category error we described: confusing operational improvement with business model transformation. Yes, AI can make workflows more efficient. No, that doesn’t turn a services company into a software company.

Read our original op-ed below

Eating rollups for returns

Across the technology investing world, investors are scaling their bets on a seductive thesis: Generative AI will transform low-margin service businesses into high-margin software companies. Several well-known platform venture firms have committed billions to this strategy and have begun to make their bets. Here’s how the thesis goes:

First, acquire traditional business process outsourcing (BPO) companies such as call centers and accounting firms at modest valuations of 1x revenue. These businesses typically operate at 10-15% EBITDA (earnings before interest, taxes, depreciation, and amortization) margins, weighed down by armies of human workers performing repetitive tasks, and automation faces the greatest structural resistance.

Second, deploy generative AI to automate core workflows, cut headcount, and expand EBITDA margins to 40% or more. What once required hundreds of accountants or call center agents can now be done by a handful of people managing AI systems.

Third, exit the newly minted AI-enabled services company at software multiples because buyers and public markets recognize you’ve transformed a human-heavy service business into a scalable AI business. Where traditional BPOs trade at 6x EBITDA, software companies command 20x or more.

On paper, it’s brilliant arbitrage. In practice, it’s a mirage. It rests on a fundamental category error: confusing operational improvement with business model transformation. Yes, AI can make workflows more efficient. No, that doesn’t turn a services company into a software company.

Indeed, five years ago, a now notable AI company ran this exact experiment, and walked away. Its findings should serve as a warning to today’s believers. Let’s dig in.

The 69x valuation canyon

The most damning evidence against the AI rollup thesis hides in plain sight on public markets. Today’s “AI-transformed” BPO firms that have invested heavily in automation—among them Concentrix, Genpact, and Infosys—trade at 5-23x EV/EBITDA (enterprise value to EBITDA). Their pure software counterparts, such as Salesforce, ServiceNow, and Workday, command valuations of 22-92x EV/EBITDA. Here is a chart to tell the story:

That’s not a gap that can be bridged with press releases about OpenAI, Anthropic, or Gemini partnerships. It’s a fundamental difference in how markets value human-dependent businesses versus true software platforms.

Consider Concentrix, often cited as a BPO transformation success story. Despite a major push in launching their gen-AI products in 2024 and now having deployments at over 1,000 customers, the company’s EV/EBITDA multiple remains stuck in the low single digits, and its EBITDA margin is still hovering around 10%. The market’s message is clear: Automating workflows doesn’t change your fundamental business model.

The PolyAI prophecy

In 2019 PolyAI, the leading conversational AI company, spent six months exploring whether to acquire incumbent human-driven contact centers to accelerate its growth. After analyzing the opportunity by visiting over 10 contact centers, building relationships with three major BPOs, and hiring industry advisors, the answer was a clear no.

“Business Process Outsourcing firms are not trusted to innovate, not rewarded for innovating, and not allowed to innovate,” read its board deck.

The structural barriers it identified remain unchanged today:

• The illusion of control: Buying a BPO doesn’t mean owning the business you’re supporting. You’re simply renting the right to supply labor on the client’s terms. Tech stacks, processes, and approvals remain firmly in the client’s hands. AI deployments still require their permission, integration, and oversight. You’re not in control; you’re a replaceable vendor.

• The pricing trap: Most service businesses bill by the hour. Efficiency improvements that reduce billable hours directly cannibalize revenue. As PolyAI discovered, BPOs promise innovation to win contracts, then revert to maximizing billable hours to protect margins. It’s a business model fundamentally at odds with automation.

• Zero switching costs: Where 10-year service contracts were once the norm, it’s now increasingly common to see three-year terms or less. This reduces the ability to recoup up-front AI investments, particularly when there’s little client lock-in, no network effects, and no moat.

PolyAI chose to remain a software company, partnering with BPOs rather than acquiring them. Today, it’s valued at over $500 million with customers like PG&E, Marriott, and FedEx. Meanwhile, the BPOs they considered buying still trade at single-digit multiples.

Share

Why this time isn’t different

Here’s what investors are missing: Services businesses aren’t inefficient by accident. They’re inefficient by design. The inefficiency is the product. Clients pay for flexibility, customization, and someone to blame when things go wrong.

Automating away the human doesn’t just reduce costs, it fundamentally changes what you’re selling. BPO technology capability has never been the constraint. And clients who wanted software would have already bought software.

The most successful services firms understand this. They use AI to augment their humans, not replace them. They maintain margins through pricing power and relationships, not through headcount reduction. Ultimately, they still trade at services multiples because that’s what they are.

The lessons of history

The AI rollup thesis represents a familiar pattern in technology investing: the conflation of technological capability with business model transformation. We’ve seen this movie before.

In the early 2000s, believers thought e-commerce would transform retail margins. Amazon proved them right by building a native digital retailer, not by acquiring and transforming Sears or Barnes & Noble. In the 2010s, investors believed software would eat traditional industries. The winners built new software-native businesses rather than retrofit old ones.

The same lesson applies today, but with a narrower scope. AI may well transform some corners of professional services, especially when existing firms are pushed to adopt new tools by private equity owners with clear control and incentives. We’ve seen this in sectors like health care and financial services, where PE firms have driven adoption of AI-driven tooling. But this is different from the AI rollup thesis that VCs are chasing—one that assumes low-margin, labor-heavy service businesses can be turned into software-like platforms simply by embedding AI. For those firms, transformation won’t come from owning the service layer. It will come from new, AI-native companies with fundamentally different economics.

The bottom line: Own the software, not the service

The AI rollup thesis is venture capital’s attempt to arbitrage the multiple gap between services and software. But that gap exists for a reason. Services businesses, even highly automated ones, face different constraints, different economics, and different customer relationships than software companies.

PolyAI saw it in 2019. Public markets see it now. The AI revolution is real. The opportunity to improve services businesses with AI is real. The idea that this improvement transforms them into software companies? It’s unlikely to be real today, just as it wasn’t in 2019.

AI rollups may still deliver returns, but not the kind VCs are underwriting. At best, they’re tech‑enabled private equity: operationally heavy, valuation‑capped, and unlikely to scale like software.

Share

[This article originally appeared on Fortune.com and can be read here for free]

The United Arab Emirates is spending $20 billion on OpenAI’s Stargate UAE. The project is billed as a sovereign AI capability, yet it relies entirely on American chips, software, and infrastructure. This is the sovereign AI paradox: The harder nations push for AI independence, the deeper their dependencies become.

The UAE is not alone. From Paris to New Delhi, governments are pouring billions into so-called “sovereign” frontier models. France backs Mistral. India promotes BharatGPT. Each promises strategic autonomy yet is dependent on a globalized stack.

The term “AI factories,” adopted by Nvidia CEO Jensen Huang, rebrands data centers as strategic infrastructure akin to power plants or shipyards. This is political branding, not technical reality. It aligns AI with the rhetoric of national self-reliance, even as the underlying systems remain foreign-made and globally entangled. Calling any national data center an “AI factory” does not make it sovereign any more than France’s Qwant became a European search engine by wrapping Microsoft Bing.

Model weights, once seen as crown jewels, now update faster than policy cycles. They are versioned, cloned, and surpassed in quarterly releases. What endures is the infrastructure: chips, data pipelines, and labor required to build, deploy, and serve models. Sovereignty at the top of the stack is symbolic if the foundations remain foreign.

France’s Mistral was hailed as a European sovereignty breakthrough, only to be surpassed by more efficient, open-sourced Chinese models like DeepSeek. Now France and the UAE are co-funding what is billed as Europe’s largest AI campus, again built on American infrastructure. These efforts highlight the depth of technological entanglement: Even in pursuit of sovereignty, nations remain dependent across the stack, from chips to data to middleware.

The deepest dependencies lie in the invisible layers. Training data is often annotated by outsourced labor abroad, while pipelines for filtering and tuning rely on proprietary U.S. tools that entrench vendor lock-in. As AI moves into complex fields like law and medicine, demand is shifting toward expert labor in developed markets. Yet owning weights while depending on fragmented global workforces and imported toolchains is hardly sovereignty—it is a repackaged dependency.

This reveals a new kind of digital colonialism. Not one where countries are denied access, but one where they are structurally bound into dependencies across every layer of the AI stack. A European lab may host its own weights on a data center in France, but that center runs on American hardware, software, and middleware. The illusion of control masks a dense web of interdependence.

Strategic leverage today lies not in model authorship, but in owning the connective infrastructure that links data to deployment. That means investing in domestic expert data capacity, security, building open-source engineering stacks, and cultivating chip independence—not to beat Nvidia, but to secure “good enough” alternatives and diversify risk. Countries that fail to grasp this are sleepwalking into vendor lock-in enforced not by licenses, but by geopolitical gravity.

Nations must think in ecosystems, not hero models. A vibrant AI sector will not emerge from a flagship GPT-X, but from an interdependent network of local tools, standards, infrastructure, and governance. The U.S. and China have ecosystems. Europe, as yet, does not.

Sovereign AI reflects a fundamental misunderstanding of modern technology. Unlike oil or steel, AI depends on global flows of data, chips, software, and talent. No country can meaningfully isolate itself. Sovereignty, pursued at the top of the stack, risks becoming a costly illusion.

The choice is clear: Pursue symbolic ownership or invest in strategic infrastructure. The harder nations chase the illusion of AI independence, the deeper their entrenchment in foreign dependencies becomes. At present, most are choosing the former—and will pay for it dearly.

Share

Revisiting my response to the GS report on 29 Nov 2016, it’s striking how many of the missing themes I identified have since gone mainstream. I got a few things wrong too. Turns out that we did finally get a 10x better chatbot that everyone now uses and developers are happy to call intelligence from third party APIs.

Hope you enjoy the read!

Overall, I think the report is great and the use cases are interesting. However:

- There's far more than 150 companies [in AI today].

- The report, however, does not touch on generative machine learning models. This class of models is distinct from discriminative models, which are tasked with separating, labelling, or categorising data. Generative models instead learn representations of data and can then output new data altogether. You can see examples where Facebook showed they can erase the middle of an image and then ask a generative model to fill in the blank on the basis of it's understanding of the image's content.

- It also talks a bunch about efficiency that is gained from ML when applied to optimisation problems, but it doesn't talk about the importance of creating ML models and algorithms that are less and less data "hungry" and thus less computationally and energy intensive (because the models are smaller at training and runtime). This is crucial.

- It also doesn't talk about reinforcement learning at all, which is a big oversight in my view, especially when considering problems that involve exploring (through trial and error) an environment (e.g. a robotic control task) to learn the optimal way of reaching a certain goal (e.g. Atari games). This hasn't really made it into real world production yet, but is coming.

- It also doesn't talk about transfer learning, which is a strategy to train an ML model on one data modality such as text and then take that model and apply it to images. These pre-trained networks have been shown to reduce the amount of new training data they need to solve tasks on a new data modality.

- The piece is very light on DeepMind, which I think is by far the most compelling organisation in the world right now. It also says that "In 2013, Google paid more than $400mn to acqui-hire DeepMind Technologies; according to press reports, a team of roughly a dozen.", which is false. The team was much bigger than 12 people!

- NVIDIA is also doing a hell of a lot more in AI than just making GPUs - that company is smashing it right now. If you want to get a good view, I suggest you come along to their GTC in May 2017 (if I got the date correctly). You'll be blown away. Markets clearly didn't appreciate the extent to which it is a market leader, as evidenced by the 30% pop. Should have trusted my own knowledge instead of reading market reports on that one :)

- Sentences like this in the description of Baidu brain just don't mean much tbh "1) An AI algorithm that mimics the human neural network, with a large number of parameters trained over hundreds of billions of samples." I think you gotta be careful with layman descriptions the verge on biological references or too handwaving reference to "lots of data". What data are you talking about.

- "Neural networks become more effective the more data that they have, meaning that as the amount of data increases the number of problems that machine learning can solve using that data increases." That's not technically true. It's not the volume of data that matters per se, it's the volume of quality data that specifically describes the task you want to solve that matters. The more this increases, the more problems we can consider solving, if we have the right approaches of course. This relies on both building models of problems and training algorithms.

- I find the description of deep learning rather confusing and they also don't actually explain exhibit 4 to the reader. It also misses an important point that unlike non-deep learning ML models, deep learning does not require the developer to engineer their own features (i.e. the variables in the data that have to be tweaked in order for the ML model to output a classification). That's super powerful.

- They should explain why over fitting is bad. Of course the problem is that your model will not generalise well to data beyond the training examples it has seen.

- I'm not so sure on the claim that horizontal APIs for ML as a service will be a good business. From research I've done with several hundred developers, basically no one wants to consume a paid for API for an ML task if it's going to be an important feature of their application. Developers hate 3rd party dependencies for functionality and especially in the AI world. Maybe larger corporates will who can't get hold of ML engineers would consume those APIs from Google, but very unlikely if it's a core part of the offering the corporate sells to its customers.

- "Industry consortiums in areas such as retail or advertising could pool data to better compete against larger competitors (e.g., retailers could pool data to better compete against Amazon’s recommendation engine)." This would be neat, but I'm not convinced it'll happen. Maybe amongst small players, but not the big boys. In fact, open standards for training data is an area the US government is pushing.

- I'm bearish on bots. I do not really buy that they're the future user interface. I challenge the authors to find an example product or service that is best procured/consumed with via a bot - I'm talking 10x better. I'm still looking. I think this area is overhyped.

- The piece is light on context awareness for digital assistants. I think this is the big area - if you can identify the intent in language or user behaviour, you can pre-empt their next actions. That's a holy grail and makes a big impact on user experiences.

Read more:

In recent months, a subtle but politically astute rebranding has propagated across the AI industry. Leading figures like NVIDIA’s Jensen Huang and OpenAI’s Sam Altman are no longer calling the backbone of AI infrastructure "AI data centers." Instead, they refer to them as "AI factories."

At first glance, this might seem like hype-driven marketing speak. But the intent and implications of this linguistic shift are anything but trivial. The rebranding is deeply political, aligning Silicon Valley’s ambitions with nationalist industrial policy, particularly the Trump-era vision of reshoring American manufacturing.

A new metaphor

Take Jensen Huang’s own words just this week:

“AI is now infrastructure, and this infrastructure, just like the internet, just like electricity, needs factories,” Huang said.

“These factories are essentially what we build today.” “They’re not data centers of the past,” Huang added. “These AI data centers, if you will, are improperly described. They are, in fact, AI factories. You apply energy to it, and it produces something incredibly valuable, and these things are called tokens.”

He paints a picture of AI infrastructure not as ephemeral cloud services, but as physical, job-creating, patriotic assets that produce physical things. At a recent event, Jensen questioned why America would onshore sneaker manufacturing but offshore AI manufacturing, rightly suggesting that outsourcing AI factories would be a strategic blunder. This framing is echoed in Sam Altman’s lobbying campaign to raise trillions for nuclear-powered AI infrastructure and further Stargate’s buildout. These aren’t server farms - they’re the largest industrial project in history.

Global supply, domestic illusion

Talk of AI factories wrap digital infrastructure — still globally sourced, largely automated, and low on direct job creation — in the comforting rhetoric of American industrial revival. And it’s working.

The term "factory" evokes a 20th-century mental model of production: smokestacks, union jobs, and steel. But traditional factories produce physical goods. Most critical infrastructure that doesn't – like power plants, roads, or communication networks – isn't labeled a factory. We don’t call a power plant a "power factory," or fiber optic networks "internet factories." Even a datacenter, arguably closer in function, isn’t called one. This shift in nomenclature stretches the metaphor to serve a political narrative.

The reality is far more global and intangible. The components of an "AI factory" other than the NVIDIA GPU – TSMC fabs, ASML lithography tools, SK Hynix and Samsung memory modules, Foxconn server assembly, and even rack and power systems – are designed and manufactured overseas. The CUDA and PyTorch software that runs them is built by researchers and engineers scattered across the globe. Yet the new language gives cover for governments to subsidize these facilities as if they were a domestic industrial renaissance.

AI factories as the new geopolitical lever

The strategic logic is obvious. Politically, an “AI factory” sounds like a jobs program. It plays to voters in swing states who want to believe that manufacturing is returning. It gives policymakers on both sides of the aisle, whether it was Biden with the CHIPS Act or now Trump with America First manufacturing, cover to write massive checks and ink trade deals. And it lends moral weight to Silicon Valley’s demands for subsidies, energy permits, and export control exemptions.

You can see this narrative being embraced abroad too. Just last week, Gulf states like the UAE and Saudi Arabia signed enormous AI infrastructure deals with the U.S. with the same gravity as oil, gas, and defense agreements. The analogy is clear: AI is the new petroleum, and owning the means of its production – its factories – is a source of sovereign power.

Under the new 'AI Acceleration Partnership' signed during President Trump's visit to Abu Dhabi, the U.S. secured a framework where Gulf partners will build large AI campuses powered by American chips and infrastructure, while also committing to invest billions into U.S.-based data centers. Saudi Arabia received similar terms. In return, American hyperscalers retain ownership of the compute clusters, and all advanced semiconductors remain under tight U.S. security controls—positioning the U.S. tech stack as the de facto standard for the region.

But how durable is this alignment? Unlike physical factories – like power plants that ship electrons – AI factories transmit packets over a global communications grid that’s already in place. Software models and AI services can be rerouted far more easily than oil or electricity. What’s to stop these same facilities from running Chinese foundation models or serving Chinese tech companies? The political allegiance of AI infrastructure may be more fragile than the industrial metaphor implies.

Factories for AGI

Jensen often draws an analogy to car companies to explain what an AI factory is. A car company doesn’t just make cars – it also builds and automates the systems that make cars. Now, AI is becoming the intelligence layer within both the product and the production process. The AI that powers vehicle autonomy, logistics, or manufacturing robots must be trained somewhere. That somewhere is the car company’s own AI factory. This narrative makes it easier to justify in-house GPU clusters, massive CapEx, and a closer alignment between physical manufacturing and digital intelligence.

This rhetorical pivot carries real influence. By recasting a globally dependent, capital-intensive, low-employment infrastructure as a patriotic asset, it encourages a narrative that aligns with strategic policy goals. It helps direct subsidies and public investment toward critical AI infrastructure – even if the on-the-ground economic impact differs from traditional manufacturing. While this language can be constructive in shaping national AI capacity, it also obscures the underlying reality: most of the economic value is captured by software platforms and global chipmakers, not American factory workers.

Narratives move markets

If the 20th century was defined by steel and oil, the 21st will be defined by semiconductors and AI models. The industrialization of AI infrastructure is accelerating – and nations are treating it as a matter of strategic importance.

OpenAI’s new 'OpenAI for Countries' initiative is just one recent example of how AI capabilities are being politicized as national assets and geopolitical levers. In that context, calling data centers “factories” isn’t just branding – it’s a deliberate rhetorical move that aligns the AI sector with the language of industrial policy.

There are two takeaways. First, we shouldn’t let this metaphor obscure the mismatch: these “factories” don’t produce physical goods, and the value they generate is neither local nor easily contained. Second, for companies building this infrastructure, aligning with the political narrative is now a strategic imperative. Jensen’s AI factories may win hearts in Washington – but we’d do well to remember where the real value accrues.

The motivation for this essay came from hearing founders—particularly highly technical teams—say they’re uncomfortable talking about their products before they’re (essentially) perfect. And God forbid hyping them…

Science vs. startups

Many founders who build companies in and around AI were educated through high-caliber graduate school programs in computer science, life science, and engineering. As a PhD grad, I empathise with the scientific method teaching us to not overreach on conclusions that aren’t explicitly supported by empirical evidence. It’s important to call out caveats behind experimental results. Research is heavily scrutinised internally and by external committee before it sees broad distribution. And even when it is, the academic community’s hawk eye doesn’t abate.

But building a startup is somewhat different: their existence is precarious from day one. Each week, startup teams must design, run and analyse experiments around their market, product, and growth levers. They must make decisions with imperfect information in rapid succession. The more shots on goal per unit time, the faster their learning rate. And, critically, they must attract financial resources, talent, and market attention. There isn’t really time to get everything right before the grand release. Perfect is the enemy of good, as they say.

So while perfection is rewarded in academia, it is a death sentence for startups.

Subscribe now

Narrative unlocks mindshare

A startup’s story isn’t just marketing—it’s how it attracts talent, capital, and customers before it even has a finished product.

Consider that the narrative arc of a startup can be viewed as a novel. Startups have a mission (the novel’s title), they develop products as they learn, mature and evolve (the chapters), they fight with competitors and develop relationships with partners (the characters), and eventually reach their grand master plan (the conclusion). Like an AI model refining a blurry image, a startup’s early direction is far clearer than its distant future.

It is therefore critical from day one to clearly and loudly articulate the scope of the startups ambitions in order to captivate the audience’s attention and harvest necessary resources for the rest of its journey. What markets, customers, technologies and products are in-scope for its ambitions? How will it get there? Why does it matter at all?

Carte blanche to competition

If a company doesn’t signal to the market that the whitespace they’ve found is theirs for the taking, it’ll leave the door open for competitors and investors looking to get exposure to the opportunity. For example, a defense tech startup that sets out to build a new platform that’s explicitly needed by the Army or Navy but doesn’t publicly project it’s doing so will invite competitors into the market who only see whitespace with no existing actors in their way. By the time the company might be ready to talk about their product, it’ll no longer be that coveted n=1 company, but will be compared to n=x alternatives.

Consider the famous story of General Magic, the startup that essentially invented the smartphone before Apple and Google. Formed in the early 1990s as a spinoff of Apple, General Magic basically pioneered touchscreen interfaces, mobile apps, and personal communication devices. But instead of aggressively shaping the public narrative and rallying industry support, it operated in secrecy for years. By the time it finally launched, the market had shifted, and its once-revolutionary ideas had been overtaken by nimbler, more public-facing players. When Apple unveiled the iPhone in 2007, it succeeded not just because of technology but because it owned the conversation around what a smartphone should be. General Magic had the right vision, but it stayed quiet—and in doing so, it lost the chance to define its own category.

Anointing winners

Modern venture capital thrives on identifying proto-winners, anointing them, and rapidly weaponizing their balance sheets to starve competitors of oxygen. Recent examples include OpenAI (AI), Wiz (cybersecurity), and Anduril (defense). Projecting your grandiose ambitions from day one is similar to effective patent writing, in which new inventions include as broad claims as possible to stake their turf. And this tends to work best when you’re the earliest in your peer set to do so.

Put another way, being first (and the most visible) is a moat. If you don’t own your narrative from day one, someone else will—and when the market moves, it won’t wait for you.

Subscribe now

Introduction

For those of us tracking the pulse of the AI vibes, a summary would probably look like this:

Progress slows or ‘hits a wall’, only to accelerate. Companies are down and out, only to be ‘cooking’ a few months later. We’ve seen this dynamic play out with a handful of big tech companies. Satya Nadella has turned Microsoft from a stagnating behemoth that didn’t know how to innovate to a 10x turnaround story. Much of his success stemmed from his decision to go all-in on OpenAI, even if the relationship today isn’t as happy as it might once have been. Following OpenAI’s board drama, Microsoft were left looking like the biggest winners. “Satya is playing 4D chess” was what the people chanted.

Meanwhile, Google went from being ‘unable to ship’ to building the viral AI product hit of late 2024 and stunning observers with a new quantum computing chip. Gemini 2.0 impressed users with its AI assistant and an agent that can control the user’s browser. Its Deep Research functionality, which generates a multi-step research plan based on user prompts, before executing on it, has also won fans.

So what’s going on? Are these reflections of real change or the vagaries of narrative wars?

Tech is hard

Building consumer-facing AI is hard. Anything involving relatively open-ended user prompting runs into a range of hot-button social and political issues. Not to mention the heightened chance of random events… Microsoft and Google both fell afoul of these at different stages.

Microsoft was first out the door with Tay. Launched in 2016, the company’s AI-powered Twitter bot lasted a whole 16 hours, before trolls managed to push it into releasing a series of increasingly inflammatory replies. This quickly led it to be enshrined as an example of reckless product design and release strategies, even years later.

Google had this in three waves, with its first adventures into generative AI. Bard shared inaccurate information in a promotional video, while the saga of Gemini’s “woke” image generation made the company a cross between a laughing stock and a punching bag. As a chaser, its AI-generated search previews came under scrutiny. Remember AI Overviews suggesting to use “non-toxic glue” to make cheese stick to pizza better?

These very visible, public errors that ordinary people could replicate for themselves understandably overshadowed the highly impressive benchmark scores we saw in Gemini releases. It didn’t help that Google was caught gaming one of its Gemini demos either… The company also hasn’t done itself any favors by pursuing a generally uninspiring approach to communicating new product news, often with well-concealed, dry updates that have attracted satirical comment. Not to mention if products were even available for use when they were announced.

Subscribe now

You only need to win once

It’s important to understand that to execute a vibe shift, you don’t have to win at everything. One or two big wins will more than patch over a good number of disappointments.

For example, Microsoft has won credit for its OpenAI investment, its nuclear investment, and data centre buildout. This in turn gives it cover for its misses. It doesn’t take a lot of scouring on social media or Microsoft’s Community forums to see that reception for Copilot has been far from overwhelmingly positive.

At the same time, if Menlo Ventures’ survey is to be believed, Google’s Gemini has failed to increase its share of enterprise spending over the past year, significantly lagging OpenAI, Anthropic, and Meta.

And yet, NotebookLM has won the company creds, even though there’s no data available on its actual adoption. While undoubtedly well-crafted, it’s not necessarily an economic gamechanger. Likewise, DeepResearch is clearly better at some domains than others. Meanwhile, Willow is sufficiently complicated that it has likely received even less expert scrutiny than the median AI product release.

In short, momentum and pace are so often the building blocks on which good vibes are built. Once the narrative is up and running, anything that doesn’t fit can be conveniently discarded.

In search of a narrative

Maybe opinion is simply shifting rapidly, because … the space is just inherently uncertain.

Consistent commentary relies on having a model of the world. In this context, we mean a coherent theory of how innovation works, what makes for a killer product, markers of success for a company, and a view of the direction in which the field is moving.

Of course, you should update this model as new evidence emerges, but without one, you will be endlessly buffeted by the tide of new announcements. It quickly becomes impossible to properly weigh new information which, depending on your personal biases, means either a bias towards dismissing or overinterpreting every new development.

The other dimension at work is pretty human. The perpetual race dynamic is kinda fun. In the most recent installment of the State of AI Report, we noted that you could just “close your eyes for a year and OpenAI is still #1”.

But we don’t want to do that.

The race is motivating for the participants, while a whole industry of commentators is dependent on keeping us in suspense. For a good story, we need someone to be ‘ahead’, someone to be ‘catching up’, and someone we’d counted out to be going through a surprise ‘comeback’. To spice things up, we need occasional new entrants (and controversies), even if we quickly forget them after they flame out.

Closing thoughts

Narrative, real progress, or demand-driven speculation? We’re probably seeing a combination of the three at play today. It is probably true that our vibes on Google, Microsoft or other big tech companies are over-determined by individual, high-profile decisions. But at the same time, these big moments conceal feedback, lessons learned, as well as fundamental research. Vibes may in fact, not be all you need in the longer term.

Subscribe now

Introduction

As we compiled this year’s State of AI Report, it was striking to see how one debate in particular had changed: safety.

While we had our longest ever section on safety-related research, spanning jailbreaking, ‘stealth’ attacks, national resilience efforts, and more, the debate seemed much cooler. This stands in stark contrast to the previous year, when Yann LeCun was going into battle every day on (then) Twitter against voices who wanted to pause AI development altogether.

In 2023, Anthropic CEO Dario Amodei had issued Congress a stark warning on biorisk. In 2024, he was writing a 15,000 word techno-optimist manifesto.

So what’s changed?

When the facts change, the vibes change with them

On one level, Team Safety got a lot of what they wanted. While the most extreme voices remain shut out, it is obviously true that governments clearly take these questions much more seriously.

Despite facing initial scepticism, the UK has managed to build a genuinely world-leading institution focused on AI safety. Anthropic, Google DeepMind, and OpenAI submit their most powerful models to the UK AI Safety Institute or its US counterpart for pre-deployment testing. Various US national security-focused bodies were given early previews of o1.

Provided you don’t want to bomb GPU clusters, you should be reasonably happy with the last year or so of regulatory progress.

Labs, which once underhired safety researchers (if they did at all), now publish mountains of safety-related research. And much of this research, just … isn’t that scary.

In this year’s report, for example, we covered some eye-catching results. It’s noteworthy, however, that many of them involved using highly-convoluted attacks that only worked under specific conditions that would be challenging to replicate in the real-world.

Secondly, AI commentary has been the victim of Amara’s Law: we overestimate the short-term, but underestimate the long-term impact of technology. Model capabilities have undoubtedly improved significantly over the past few years, but many of the most breathless predictions about dangerous capabilities simply haven’t come true.

As we’ve covered before, AI biorisk currently remains more hypothetical than real. Meanwhile, agentic systems remain too brittle to be useful for many of the most sophisticated malicious uses that were the subject of speculation. After all, we’re old enough to remember the panic around AutoGPT…

Every time a major election approaches, pundits predict that our information ecosystem will collapse as a result of deep fakes or AI-generated misinformation. This is yet to occur, while studies suggest that misinformation remains a peripheral, overhyped phenomenon.

This has likely removed some of the abject panic from the debate.

Subscribe now

Reality bites

There are, of course, various practical reasons for this change.

We’ve argued before against the notion that AI safety is some form of regulatory capture ploy or guerilla marketing tactic. While there’s a theoretical universe in which this could be true, it doesn’t actually explain the actions of any one company particularly well. If this were the case, you’d expect to see big tech companies support sweeping regulatory measures like SB1047 in California, which would undoubtedly have harmed their open source competitors. Instead, they either opposed the bill or lobbied hard to weaken it.

However, it is obviously the case that successful multi-billion dollar companies are swayed by the political climate to some extent. When a lifelong Democrat like Mark Zuckerberg gives money to Donald Trump’s inaugural fund, it’s not out of an altruistic desire to ensure the new President has a good party.

For a start, these companies do need to make money eventually. Regulation that hurts them, even if it hurts their competitors as well, isn’t particularly helpful when you’re trying to fundraise and it’s unclear if your business model actually works. Seeing intense x-risk impulses translated into regulatory proposals, California-style, likely made companies sit up and rethink.

Secondly, thanks to (geo)political shifts - expectations of companies have changed. Back in 2022/23, most tech companies were keen to emphasize their political neutrality. Sam Altman expressed the hope that OpenAI would never have to adjudicate values conflicts, leaving people to finetune models themselves, within certain very broad bounds.

As the return of Donald Trump to the White House became likelier, tech companies rushed to rebrand themselves as champions of American values. Meta, OpenAI, and Anthropic are now either allowing the military to use their technology, while the latter two are partnering with the likes of Anduril and Palantir.

The incoming administration is unlikely to be overly-concerned with safety. The Republican Party official platform pledged to repeal the Biden White House Executive Order on frontier models, while senior GOP politicians seem interested in only narrowly-targeted regulation. Now is not the time to appear opposed to faster American technological progress.

Closing thoughts

The ‘vibe shift’ on safety is a salutary reminder that even apparently powerful memes can collide with reality. However loud a debate gets inside the social media echo chamber, political and economic reality have a tendency to assert ourselves.

It’s also important to remember that just because a conversation moves on, it doesn’t mean nothing has changed. The x-risk wars of 2023 did change things: whether it’s new institutions or new research. Just because the narrative doesn’t determine reality, it often leaves its mark.

Subscribe now

Subscribe to Air Street Press so you don’t miss any of our writing - like our 2024 year in review.

1. Market commentators don’t always agree that AI exists

If you minus out US big tech, then over the last couple of years (since Oct 2022), the S&P has in fact not outperformed its European counterparts. We’ve seen many versions of this argument articulated routinely, most recently in the Financial Times’ Alphaville section. The FT included this handy chart:

We’re confused by this argument.

First, “October 2022” feels like a pretty arbitrary cut-off to measure market performance, considering that the SP500 and MSCI EMU indices have diverged significantly since 2009. One could make the argument that US acceleration has been high growth, tech-led, while European stocks are more mature and tend to pay out dividends. But if you compare accumulation ETFs to get a sense of total shareholder return over the past decade, the trend goes back much further than 2022…

Secondly, even if the divergence was entirely the result of tech - why would it matter? The US tech sector is made up of highly profitable, dynamic businesses that create products used by billions of people around the world. You may think they are overspending in the short-term on AI infrastructure, but these companies can afford to. Microsoft and Alphabet aren’t the manufacturers of VC-backed juicemakers. Talk of AI bubbles often suggests people are proceeding off the wrong mental model.

Subscribe now

2. The humanoids are coming

Taking the stage at CES in Las Vegas, a chic leather jacket-clad Jensen Huang, flanked by humanoids on either side, presented a menacing image to the world.

Rather than announcing the robot takeover, NVIDIA was in fact releasing a family of generative world foundation models called Cosmos. Trained on tens of millions of hours of humans moving and manipulating objects, the Cosmos family is designed to generate videos and world states for physical AI development that demonstrate an accurate grasp of physics.

As an anecdote about the importance of timing in tech adoption, back in 2018, a deep-learning startup (TwentyBN) where I worked pushed the research agenda of end-to-end learning on video streams of human actions to learn visual common sense. They had demonstrated this was a tractable path but, back then, the commercial use cases weren’t clear nor were the scalable transformer architectures…

Current users include humanoid start-ups such as Agility and Figure AI, as well as companies working on self-driving, like Wayve.

What should we make of all this?

Firstly, betting against NVIDIA is generally unwise. Unless some mysterious, unspecified new AI paradigm comes along, every time a potential use case gains promise, the company will almost always be the natural partner of choice for entrepreneurs. Who’s going to say no to GPUs and a ready-made software stack? Best case scenario: they plant themselves at the center of a new use case. Worst case: they make a model that lots of people use. Either way, the house never loses.

Secondly, we’re starting to see signs of an NVIDIA playbook emerging. The company was quick to move on self-driving almost a decade ago now, partnering with all the major players, while working on its own autonomy stack in the background. It seems highly likely that with Cosmo, or Gr00T, their general-purpose foundation model for robotics, that they are attempting the same move with physical AI. This is the side of NVIDIA’s business that the focus on the (obviously important) CUDA advantage sometimes overlooks.

3. An implicit reward for reaching the end

Work inspired by O1 reasoning is understandably popular at the moment. But traditional approaches to improving reasoning in LLMs have often been dependent on large volumes of annotated data. PRIME (Process Reinforcement through Implicit Rewards) introduces an implicit process reward model, which learns from overall outcomes (is this solution correct?) rather than needing detailed step-by-step feedback.

This model generates dense feedback for every step in a reasoning process, solving a common problem in reinforcement learning (RL) where rewards are too sparse to guide training effectively. Importantly, the PRM can start from an existing language model and improve itself dynamically during training, reducing the complexity and cost of creating a specialized reward system.

This allowed the authors’ model, trained from Qwen2.5-Math-7B-Base, to surpass GPT-4o and Qwen2.5-Math-7B-Instruct - using only 1/10 of the data.

If you want to learn more about Implicit PRM and some of the theory behind it, you should check out the original paper.

See you next week!

Subscribe now

As we reach the end of 2024 and things slow down just a bit, we’re sharing a few of our favorite essays from the past year for you to enjoy over the holidays.

We’ve got a busy schedule planned for 2025, so don’t forget to subscribe!

Subscribe now

Back in 2016, I came across a prescient guide on data acquisition strategies for AI start-ups written by Moritz Mueller-Freitag, then co-founder of Twenty Billion Neurons (TwentyBN).

We became good friends and even worked together for some time at TwentyBN. In 2021, the company was acquired by Qualcomm, where he continues to work as Director of Product Management.

A lot has changed in the last 8 years, so we teamed up with Moritz to produce an updated guide that’s relevant for AI-first founders building in 2024. The piece covers LLMs, synthetic data, scraping, open data, as well as some examples of techniques that have worked less well.

The 2024 State of AI Report devoted more space to the output of Chinese AI labs than any of its predecessors. This reflects the striking convergence in performance between US and Chinese labs over the past year. Over the summer, we looked at the strengths and weaknesses of Chinese lab output, how sanctions were and weren’t working, and some of the surrounding policy failures.

This essay was written before OpenAI unveiled o1 (and now o3) and sparked a series of Chinese imitators. We cover these in this month’s installment of the Guide to AI.

Speaking of the creative use of reinforcement learning to improve the performance of LLMs on reasoning tasks…This summer also saw us dive into open-endedness.

We wrote the piece in response to “agentic” becoming the new AI buzzword. It looked at how the poor performance of LLMs on tasks that genuinely tested reasoning (as opposed to knowledge retrieval) suggested that the agentic era wasn’t quite here, while looking at some promising areas of research. The release of o1 a couple of months after we wrote this, and now with the even more impressive reasoning results from o3, has only deepened our excitement for this domain.

We started 2024 with a lack of clarity around fair use and training data for generative AI … and we’re set to end 2024 without clarity around fair use and training data for generative AI. With almost every major lab currently embroiled in copyright-related legal battles, we make the case for compromise. Even if companies prevail in the courts, a number of other forces may still end up pressuring them to settle as amicably as possible.

We draw on past precedent to show why big tech companies rarely emerge from these kinds of social battles unscathed - so they can choose to negotiate a settlement or have one imposed upon them, potentially on much less favorable terms.

At Air Street, we believe strong storytelling is essential to stronger company building. Whether creating a new category, battling incumbents, or facing regulatory uncertainty, hoping that technology will speak for itself just isn’t good enough.

But too often, technical founders are uneasy about it: they view it as either ‘hype’ or ‘dumbing down’. So we made the case for why compelling storytelling is mission critical, why it needs to evolve as a company evolves, some successful case studies, and pitfalls to avoid. We find ourselves regularly referencing this essay…

Foundation models and pragmatism: a trilogy

You get three for the price of one here! Over the course of this year, we used the glamorous subject of foundation model economics to ask a bunch of questions about business models, chips, and AGI.

In our first piece, back in April, we expressed skepticism about the sustainability of frontier model builders continuing to make their most powerful models available at cheaper and cheaper prices. We suggested that we would eventually see a divergence, with the most powerful models being served at an economic price to the people who actually need them, while people with more everyday needs would end up using cheaper or smaller alternatives.

My friend, Eiso Kant, the Co-Founder and CTO of Poolside, was more optimistic when I interviewed him in November. He compared the unit economics of frontier models to the early days of blitz-scaled start-ups like Uber and Lyft - a temporary product of competitive dynamics that would figure themselves out.

We decided to take our theory a bit further and asked what it meant for the chip world.

If we moved away from the bigger is always better paradigm - does that loosen NVIDIA’s stranglehold on the AI chips business? Yes and no.

One of the main pushbacks on our argument about economics is that when AGI comes along, it will unlock such economic gains that our ideas about “capex” and “margin” will become quaint anachronisms.

We’re not so sure that you can wave away basic reality like that - were such a high-powered general model to exist, it would likely come with a hefty accompanying compute tag, from both building the model and running it at scale, especially in the current era of scaling inference-time compute. This would likely render it overpriced and overpowered for … quite a lot of things, which we covered in a reality check.

Sam Altman semi-agrees with us on this, saying in a recent interview that he expected “the economic disruption to take a little longer than people think, because there’s a lot of inertia in society”.

With big tech going nuclear, we took the opportunity to explore the vexed question of AI’s energy demands. We argue that nuclear likely is the solution, but the current buildout pace isn’t fast enough, in part because of regulation. It’s also clear that significant upgrades are needed to transmission infrastructure.

Following on from our essay earlier in the year about AI-first biology, we asked what this would mean for safety.

We’re yet to see evidence that the current wave of text-powered LLMs pose a grave biorisk, but think that AI-powered biological design tools pose more risk. Not only are they optimized for bio-related tasks, they often don’t require the sorts of huge GPU clusters that make actors easy to monitor. We explore their risk profile, some of the barriers to operationalizing them, and some questions that policymakers should be considering.

We’ll have plenty more to come in 2025 - so make sure you subscribe so you don’t miss out!

Subscribe now

NB. At the start of the book, you look at some of the earlier writing about AI, with the likes of Turing, Weizenbaum, and Minsky. Having read their work, predictions, and warnings - how do you think they’d feel about the current field and the quality of the debates we’re having about AI?

JK. One of the things I found most striking in researching Mastering AI is how the debates we’re currently having about AI were really present from the very dawn of AI research in the mid-20th Century—and never satisfactorily resolved. At the time, of course, many of these issues seemed hypothetical and esoteric. People were positing “thinking machines” but the early computers and AI software could really do very little. Now, these questions are urgent.

How these early AI pioneers would view the present varies. Turing would probably be pleased that so much of what he theorized has, in fact, come to pass. Minsky was, for much of his career, the bete noire of those working on neural network-based approaches. Terry Sejnowksi famously accused him of being “the devil” and setting back work on deep learning by decades.

Today, I think Minsky would grudgingly acknowledge deep learning’s remarkable progress—while still, perhaps, being skeptical that human-like AGI can be achieved with deep learning alone.

Weizenbaum is, to my mind, the most relevant of the three. More than anyone, he saw the fundamental moral challenges of AI. He is the one who said we should stop arguing about automation on the basis of capability—what can software do as well as a person— and instead shift our focus to moral considerations, i.e. which decisions should always remain human-driven because they depend on empathy, compassion, or mercy, which are all things AI can only mimic, not truly experience or express.

Subscribe now

NB. You write that by ‘helping’ us, technology risks making us intellectually and morally lazier. How do you see that playing out and how can we respond without just giving up on progress?

JK. This is one of the big risks from AI that I think doesn’t get enough attention. I worry that many people will feel they don’t need to learn or remember as much information. This already is true to some extent with a Google search—but I think the confident, self-contained answers AI chatbots provide make the temptation to simply rely on them even worse. I am concerned people will stop thinking critically about the information they receive from an AI assistant. They won’t pay enough attention to the sources that chatbot is using to produce the information and won’t question the pat response.

Generative AI technology also frames the act of writing—or creating art or music for that matter—as somehow separable from the act of thinking, that you can just jot down some bullet points and that AI can do the rest. I think this is particularly dangerous because it is through writing that we often refine and hone our ideas. We discover logical flaws and have to work through how to fix them. We also need to put our thoughts into some kind of linear order—to build a narrative of some sort around them—and this is often essential to making our ideas appealing to others. We have already seen that PowerPoint can actually obscure more than it illuminates and lead to lazy thinking—this is why Jeff Bezos famously banned PowerPoint at Amazon and made managers write long-form memos in advance of meetings—and I worry AI only exacerbates this trend.

To get the most out of AI technology without the risk of losing these vital human cognitive abilities, I think we need to do several things.

One is that media literacy will be essential, for both children and adults. Critical thinking and analysis, and writing, absolutely needs to continue to be taught in schools. And I think in our professional lives we can guard against cognitive deskilling by not having an AI copilot or chatbot write the first draft of that business memo or strategy report or set of recommendations. Instead, we should do the hard work of writing the first draft—it’s hard because it involves thinking and refining our arguments! But then we can use an AI model to act as an editor or critical reader of our draft, pointing out logical gaps and counterarguments, and suggesting improvements. This way we both avoid the risk of deskilling and still gain the benefits of AI.

The issue of moral deskilling is even more frightening. Governments are often keen to implement AI algorithms into decision-making precisely because it’s a way to escape accountability and moral culpability for their decisions. Worried that your policing practices will be accused of being racist? Well, use some kind of algorithm to decide where to send your police patrols and suddenly the police chief can claim he’s simply “following the data.” Find deciding who should be denied housing benefit too troubling? Just let an algorithm do it and then the official can throw up his hands and say, “well, I’m sorry, but the algorithm says no.”

We need to continue to practice moral decision-making—and there are some decisions that may be of such high consequence (in judicial proceedings, family courts, and healthcare settings) where we may simply want to ban AI technology completely, no matter how “good” it is. Even if research shows that an AI can make more “optimal” decisions than a person in some of these areas, there is something morally wrong about allowing an algorithm to decide matters of life and death—it dehumanizes both the decision-maker and the person subject to the decision.

I think it is fine to use AI systems to help inform human decision-makers, to surface information from a report that we might have overlooked or double-check that we’ve properly weighed various pieces of evidence. But we should not allow AI systems to make the final call when human life or freedom is at stake, nor should we create processes where the human decision-maker winds up just being a rubber-stamp the recommendation of a black box deep learning system.

Share

NB. Some of the concerns you mention (e.g. around filter bubbles) are actively being tackled by frontier labs’ safety and alignment teams. Do you believe companies are doing enough and that problems caused by technology can be fixed by it?

JK. Yes, it is true that the labs working on frontier AI systems are well aware of some of these issues—on filter bubbles, and on alignment and AI safety—and are working on them. But I worry about the incentives. And, when it comes to something like filter bubbles, or whose values an AI system is being aligned around, that ultimately needs to be a societal decision—and to some extent an individual decision. And there will be tension here over individual choices and societal effects. And we need to collectively come to some decision about how to balance those tensions. But it shouldn't be up to the AI labs to make that call.

Already we’ve seen Elon Musk and others attack Google and OpenAI for building “woke AI” and he’s supposedly made Grok much less guardrailed in its responses. I think someone like Musk might say our own AI chatbot or AI assistant should simply align around our own individual values and beliefs as users. That’s a very libertarian stance. But that means the AI system is not doing anything to combat filter bubbles or conspiracy theories or trying to push people to consider other viewpoints.

And that seems like a missed opportunity, because we’ve now seen a lot of very convincing research that AI can be an excellent tool for popping filter bubbles. It can surface a broader range of information and viewpoints than a person might typically get in a social media feed designed to maximize engagement, for example. There’s been great research out of Cornell University on how AI chatbots can be used to help people start to question their beliefs in conspiracy theories. This is a technology that really could be used to help solve some of the atomization of society—but only if we make a collective decision that we want the companies building these chatbots and AI assistants to have them act in this way.

I also worry about the commercial incentives. If I have an AI agent and I tell it to go out and research the best hiking boots for me, given the amount I hike and where I plan to hike, I want it to come back with options that really do seem like the best choices based on those criteria, not the Nike hiking boots because it turns out that, unbeknownst to me, Nike has paid Anthropic or OpenAI to surface its shoes higher in such queries than competing brands. I think at the very least we are going to need government regulation that demands transparency to the end user if there are any such arrangements. But I also think that given how persuasive AI chatbots can be, we probably will need some additional limits around how AI vendors can seek to monetize their users.

My same concern about incentives extends to AI Safety. I worry that the financial and competitive pressure on these labs mean they may not prioritize safety appropriately. Our own reporting at Fortune and that of The Wall Street Journal indicate that safety testing for OpenAI’s GPT4-o model and for o-1 preview may have been rushed for instance. We also reported that in at least one instance, OpenAI may have exceeded the thresholds set by its own “Preparedness Framework” for safe model release. OpenAI claimed that its researchers' post-release finding that the system was above its release threshold on a risk they call “Persuasion” was the result of an analytical mistake by that researcher and that further analysis showed the system did not exceed the threshold. But can we really trust them? They are grading their own homework here. Outside experts have also raised questions about exactly how they graded the results of their safety testing on a biosafety benchmark. I think the AI Safety Institutes the U.S., U.K., and other countries have created are vital and that they really need to be independently running these tests to make sure the labs aren’t tempted to cheat in their self-assessments.

Subscribe now

NB. As well as the way we work, you talk about how the dynamics of some industries are being overturned. Can you give an example of how that’s playing out for good and for ill?

JK. In “Mastering AI,” I make a bunch of informed predictions about how I think AI is going to affect particular industries, including tech itself, law, publishing, pharmaceuticals, Hollywood, and more. But it’s a bit early yet to tell for certain how this is playing out. But at a macro level, I think you can already see a lot of industries becoming a tale of the AI haves and have nots.

There are some players in each industry that are embracing AI wholeheartedly—such as JPMorgan Chase in finance or Sanofi in pharmaceuticals—and they are starting to see flywheel effects from those decisions. And I think that is ultimately going to put them well ahead of those companies that are laggards, who are taking a “wait and see” approach to generative AI technology. Often I think they are hoping someone will come along with enterprise software that just incorporates a lot of the AI features they want without them having to spend as much to build out those features or experiment with engineering genAI models into complex workflows. And they are also waiting for the costs to continue to fall further. But I think that approach is a mistake because by the time they begin to adopt some of this technology they may be too far behind to ever catch up.

In general, I think AI tends to further enhance existing power dynamics. It is a case of “thems that got, shall get.” The largest and most successful firms in each industry also tend to have the most and best quality data—and can also afford to spend more on compute and engineering talent. That means they are best positioned to use AI to enhance and extend their leadership. I think they only stumble if they are too risk adverse to seize their data advantage and actually use it to both enhance their internal productivity—so they are ringing more value out of each dollar spent than rivals—and also use AI to create new customer offerings that extend their leadership position.

The one countervailing trend that I see to this is that in some professional services settings—law or investment banking or management consulting—the thing that has kept some big rainmakers attached to these firms is the need to have a huge support infrastructure of junior analysts or associates and other support staff to service big multinational clients on complex deals or litigation and do so in a very timely manner. But AI potentially means that these rainmakers can take their Rolodexes—all their personal contacts—and go off and form boutique firms. Because now, thanks to AI assistants and agents, they might not need as many support staff to effectively service large clients on complex deals in a timely manner. So you could see a fracturing and fragmenting of some large professional services firms as some of their top talent leaves for boutiques.

NB. Considering how many expert predictions have been made about issues like technology’s economic impact or job losses that have then proven to be wrong, as a reporter, how do you weigh up claims about the future?

JK. With a good deal of skepticism—plenty of people who know this technology very well, have been wrong about its likely impacts before (see: Geoffrey Hinton and his predictions about radiologists or Elon Musk’s predictions about self-driving.) But for the book, I did have to make some educated forecasts. On the issue of job losses, I spent a lot of time looking at what had happened with previous technologies and talking to labor economists, not technologists. The lesson from every other technology ever invented is that, on a net basis, they created more jobs than they destroyed. I have no reason to think AI will be any different. And in so many sectors, we have a shortage of qualified people—so AI is actually helping us to fill a gap. That is true in accounting, for example, and it might be true in other areas, from medicine to construction. We also have a demographic issue in many developed countries where there are aging populations, declining birth rates, and not enough workers—which is again, a reason to think that AI is unlikely to put us all out of work. I am also skeptical about how quickly AGI will be achieved—and until it is, the current AI systems can help automate some tasks, but they can’t really replace people wholesale.

Share

NB. Your book is optimistic about the impact of AI on some areas of science, but more pessimistic on its ability to mitigate climate change. What determines your optimism or pessimism levels about certain sectors?

JK. I look at the applications we’ve seen so far and how big their impact is likely to be. In areas like drug discovery, there’s a good pipeline of promising new candidates already in the pipeline. And while so far, no AI-discovered drug has made it past Phase 2 human clinical trials, I really do think it is only a matter of time before some of these new protein and small-molecule based therapies make it through to FDA approval and start having a significant impact on human health. And if you look at how the AI-enabled processes are both dramatically increasing hit rates in pre-clinical studies and seem, in many cases, much more efficacious in preliminary tests, it is easy to surmise that the impact is likely to be pretty revolutionary.

But if you look at what has happened so far in the application of AI to combating climate change, it is nowhere near as dramatic. You do have impacts—better wind and solar forecasting, which helps grid operators balance supply and demand better, so that they don’t have to keep as many gas-fired turbines on spinning reserve; more efficient management of HVAC systems; computer vision systems that can spot gas flaring and other methane emissions or better monitor deforestation. But at the end of the day, all of these things help a bit at the margin. They aren’t going to “solve climate change.” And that’s because a lot of what we need to do to solve climate change is not really a science problem or an engineering problem. It’s a political problem. In many cases, we know what needs to be done, we just don’t have the political will to do it.

And, on the side of the equation, generative AI technology is incredibly power-consumptive. And while many of the hyperscalers are committed to using renewable power, they are buying up so much of the renewable supply to feed these vast numbers of GPUs, that other customers that want to use renewables are having to fall back on carbon emitting sources like gas.

So, on a net basis, I am not optimistic about AI’s impact on climate change. That said, if AI helps us achieve some incredible breakthrough in fusion power, then, sure, maybe AI can help us solve climate change. But right now, that is largely hypothetical. (DeepMind’s work on using AI to help control the plasma in a tokamak notwithstanding.)

Subscribe now

NB. Defense has gone from a fringe sector to highly buzzy among AI practitioners in the last few years. Have you noticed that same shift in governments and militaries?

JK. Militaries have been interested in AI technology for a long time. But in the past two years, most major militaries have really rushed to embrace AI and looked for ways to infuse it into weapons systems and also command-and-control systems. They are interested in acquiring increasingly autonomous weapons platforms and they are using LLMs to synthesize and analyze intelligence and to even recommend tactical decisions. I think seeing the impact that autonomous weapons—aerial drones, but also loitering munitions and unmanned kamikaze boats—have had in Ukraine and also the role of drones and AI-based targeting systems have had in Gaza and Lebanon has made many military thinkers realize that autonomous systems can convey a huge advantage. I think the U.S. and China certainly view powerful AI systems—and AGI, if that could be achieved—as critical strategic assets.

Meanwhile, tech companies that were once leery of working on military applications of AI—in part because they worried that their employees would view it as unethical or at least morally dubious—have now overcome this hesitancy and are racing one another to sell AI software to militaries. OpenAI, Anthropic, Microsoft, and Google are all pursuing Pentagon and other government contracts and several militaries—including groups affiliated with the Chinese military—have experimented with building systems on top of Meta’s Llama models.

NB. In this year’s State of AI Report, we saw a ‘vibe shift’ away from existential risk. Have you noticed that too? If so, what do you think is driving it?

JK. I think it’s subtle, but yes, there has been a bit of a shift away from highlighting existential risk. I think there may be several factors driving this. Cynically, you might argue that the major AI labs are under a lot of pressure to generate revenue and that their major backers—Microsoft, Amazon, and Google—are worried that too much X risk talk risks scaring away some corporate customers and also inviting regulation that might stymie further AI progress. I also think these companies need to convince investors that they are in the AGI race to win it, and that emphasizing their fears about AGI might make some investors doubt they will pursue AGI as relentlessly as a competitor that is less concerned about existential dangers. (Meta is a bit of a special case as Yann LeCun at Meta has never been a believer in x-risk.)

But I think there are some other factors, too. Companies are now starting to deploy generative AI systems at scale and I think they are asking governments for clear, practical guidance on things, like copyright, or what kinds of mitigations against bias are going to be required in high-risk use cases like healthcare and finance. Most businesses aren’t worried about x-risk. This may have shifted the focus of regulators and lawmakers a bit away from x-risk.

You’ve also had changes in governments in both the UK and the US. Rishi Sunak made international AI governance and avoiding catastrophic risks from AI a centerpiece of his government’s foreign policy and technology policy. But now Labour is in power and may be less inclined to speak about AI Safety just because it was seen as a Tory-driven focus area. The same may happen in the U.S., with Trump promising to scrap Biden’s Executive Order on AI, which had a fair bit of AI safety stuff in it. And certainly there are some Silicon Valley supporters of Trump, such as Marc Andreessen, who are clearly in the e/acc camp. But, on the other hand, Elon Musk has always been very concerned about AI’s X risk and he has Trump’s ear at the moment. And we know that Ivanka Trump liked Leopold Aschenbrenner’s “Situational Awareness” monograph. So it is a bit hard to tell what Trump’s AI policies will look like.

Share

NB. In the last couple of years, some in Silicon Valley have become increasingly critical of mainstream technology journalism. What do the critics get right or wrong, and have you noticed an impact on how journalists work?

JK. Many of the most vocal critics of mainstream technology journalism don’t seem to actually understand what the role of journalism is—and should be—in a democratic society. We’re not there to be industry mouthpieces or cheerleaders. At its best, journalism is supposed to “speak truth to power,” and “afflict the comfortable.” And these days, there are few people as powerful or as comfortable as many of these Silicon Valley moguls and the companies they founded and capitalized.

Good journalists are trained to be skeptical. And there is a structural bias towards highlighting, conflict or obstacles to be overcome—because that is what makes for interesting stories. There is also a structural bias towards writing about people rather than just products or ideas. That’s because people like to read about people. So there’s a natural tendency to focus on human drama. I think some people in the tech world, who tend to come from engineering backgrounds where they are fascinated with how things work and not as interested in how people work, don’t quite get that. The startup world is full of tension and conflict and challenges to overcome. Which is one reason journalists like to write about it—but we may not cover the aspects that venture capitalists or founders would like us to cover.

Which brings me to another point: if Silicon Valley is upset that the relationship with the tech industry has become increasingly adversarial, it has only itself to blame. It has created a “credibility gap” with the press by repeatedly lying—about founding stories, about product capabilities, about financial performance, and about growth prospects. “Fake it ‘til you make it,” has become a Silicon Valley mantra—but outside of the bubble of Silicon Valley, that’s just called lying, or in some cases, even fraud. So don’t be surprised if after the dot com crash, after Facebook getting caught concealing what it knew about the negative social effects of its own platform, after WeWork, and after Theranos and SBF—if the press becomes increasingly adversarial and distrusting of what tech CEOs, founders, and venture capitalists have to say.

Now, having said all that, I do think the critics of mainstream tech journalists do get one thing right, which is that in some cases we in the tech press have allowed what should be healthy skepticism and agnosticism to trip over into cynicism and pessimism. We assume, even in the absence of evidence, that tech founders and venture capitalists are lying about their true motivations and we underplay positive impacts that a new technology might have.

I think you see this in how some AI reporters cover talk of AGI and AI safety fears emanating from the top AI labs—there’s this assumption that it is all a deliberate marketing ploy to subtly reinforce the power of the technology they are building. But I don’t think we should assume that AI safety talk is insincere. I think the leaders of many of these AI labs genuinely think AGI is close to being achieved and genuinely concerned about the safety of the technology they are building. So I don’t think we should be so dismissive of these statements. I think they deserve coverage. But I also think they need to characterized appropriately—as beliefs, not statements of fact.

NB. What advice would you give to a founder about storytelling and working with the media?

JK. I would say that narratives matter and having a good story to tell—one that has some tension to it—can help generate coverage. But you should be honest and have humility—a little bit of self-deprecation can go a long way. Don’t lie! And don’t try to brush over inconvenient facts. Rather, you should acknowledge them, and try to address them head on. If your startup has just pivoted, you should be honest and say so.

Also, I always find it helps when the company goes into as much depth as possible about how the tech actually works—it will help you build credibility with the journalist. Don’t just throw buzzwords or jargon around, but actually try to explain how what you’ve built works. Allowing journalists to actually play around with the product helps build confidence that the tech is real too. And, of course, it always helps to have customers who are willing to go on the record talking about how they’ve found using your product or service.

Be willing to have several conversations with a journalist, even if they don’t lead to an immediate story, as it will help build a longer-term relationship that can prove helpful later for both sides of the relationship.

Subscribe now

Subscribe to Air Street Press so you don’t miss any of our writing - like this week’s deep dive examination of the importance of speed, frictions, and inertia for early-stage companies.

Road to nowhere

General Motors has announced that it’s shuttering embattled self-driving subsidiary Cruise, which it acquired in 2016. Despite investing $9B in the company, the wheels came off after a Cruise vehicle seriously injured a pedestrian in San Francisco in October 2023. GM initially seemed committed to saving the company, before concluding that the costs were too high.

There are two lessons from this story.

First, it doesn’t matter if AI is technically safer or better at a task than humans. Tech people will routinely complain about how we hold AI systems to a higher bar than humans (“people crash cars more often" or “humans make biased decisions”) and we have some sympathy. But there’s no point complaining to the referee - this is currently the bar and it means the margin for error is very slim.

Second, beware of big corporations bearing gifts. Considering the high costs associated with AI research and the ensuing non-stop fundraising dance, cash liquidity and scale are appealing. However, an acquirer’s early enthusiasm, as GM displayed with Cruise, doesn’t mean you get a hall pass for life.

It’s easy for founders to assume that because it would make no sense for a company to ditch or undermine its most innovative work, it never would do such a thing. But this is where the start-up and incumbent mindset often differ. Innovation is existential for an early-stage company. But for a large incumbent, innovation is a long-term process to be managed (hopefully by the next guy). Tesla is now the only automaker in the robotaxi game.

This means that if an acquired entity doesn’t align deeply with its new parent’s mission or demonstrate value quickly, there’s always a risk of being thrown overboard when the storms hit.

Remember you are mortal, especially if you can’t stand on your own two feet on stable ground.

Subscribe now

To Russia with love

Western chips seem to have a nasty habit of appearing in Russian military equipment on the frontline in Ukraine, whether it’s in tanks, drones, or missiles. Thanks to a new Bloomberg investigation, we’re beginning to get a sense of how it happens. Russian sites integrate live pricing and stock detail from the websites of providers like US-based Texas Instruments (TI). Customers place their orders through a network of shell corporations, third parties, and front companies.

There’s a parallel here in semiconductor smuggling to China, which we covered in this year’s State of AI Report. Sly tricks travel fast:

As Bloomberg notes, it’s hard for Russia to manufacture its own alternatives to these components, just as Chinese attempts to replicate high-end NVIDIA GPUs haven’t amounted to much.

In reality, bad actors will get their hands on as many smuggled chips or components as democratic governments let them. While deliberate export control violations attract fines, like the one Raytheon was handed a few weeks ago, inattentiveness or sloppiness tend to be punished less aggressively. For example, Texas Instruments didn’t require third party distributors to reveal their end customers and missed a number of other obvious red flags.

Inaction is likely fuelled by a mixture of inertia, a fear of damaging American companies, and of provoking retaliatory measures from our adversaries. But bluntly, as we know from banks on money-laundering, defense primes on bribery or arms control, berating or shaming big corporations alone isn’t going to change behavior. Ultimately, tolerating smuggling is a political choice.

Subscribe now

Gemini comes of age

We’ve long believed that models are not products. As the industry matures, building a good model, wrapping it in an API, and leaving users to figure out the rest is increasingly no longer cutting it. The evolution of models to products is a natural one - many of the most successful companies that developed killer technical edges have taken a product-first approach - like Apple, Google, or TikTok - instead of exposing their technology as a service at the outset.

OpenAI and Anthropic have been charging ahead over the past couple of months, seemingly unveiling new bells and whistles every few days. Meanwhile, Google DeepMind’s Gemini has struggled with lower adoption rates. But Gemini 2.0, released this week may be able to change the game.

As well as reporting the obligatory strong benchmark performance, Gemini has released a bunch of cool looking features. These include an AI assistant that can use Google Search, Lens and Maps, along with an agent that can control the user’s browser.

Along with 2.0, Google DeepMind is attempting to turn Gemini into a bigger platform. Following early hit NotebookLM, the company has introduced Deep Research, which has been unveiled for Gemini Advanced subscribers. The system generates a multi-step research plan based on user prompts, before executing on it.

And so far, early users seem impressed:

As ever, we have questions about the obscure, oddly quiet release strategy that often accompanies Gemini features. To compete with Anthropic and OpenAI, Google will have to remember that good technology doesn’t speak for itself…

See you next week!

Subscribe now

Introduction

For what feels like a long time in AI, 2-year old ElevenLabs had a pretty clear run in the voice cloning and text-to-speech market. It was up against half-hearted, underpowered offerings from big tech companies or slow, unintuitive open source models like Tortoise.

Although larger players like OpenAI built powerful text-to-speech models, they adopted slow and cautious release strategies.

It’s not difficult to understand why. With frontier labs already embroiled in a series of legal disputes around copyright, they probably didn’t want to be accused of flooding the Internet with deep fakes in the run-up to a series of globally significant elections. But this is already beginning to change.

So should start-ups like ElevenLabs be worried?

In a recent interview, Carles Reina, VP Revenue at ElevenLabs, shared an interesting perspective:

We're seeing mid-market logistics companies, hospitals, and 200-employee e-commerce companies all wanting to implement AI. They see the benefits but don't have the resources to do it themselves. This changes how I, at Eleven Labs, approach customers but also gives long-term moats, contracts, and success. Once you secure one of these non-tech customers, they're likely to stay with you - they don't want to change, unlike big tech companies that can easily switch APIs.

This touched on something important and often under-appreciated by those building in AI: disruption is actually pretty hard. Being better than the rest can compensate for not being first, but it’s not guaranteed.

This week on Air Street Press, we’re going to talk about the importance of speed, the challenge of friction and inertia, and what we can learn from this.

Subscribe now

Being first

When we get asked what we look for in founders, we obviously mention engineering and product brilliance, but equally important is a strong dose of technical pragmatism and customer obsession. We’ve met many gifted builders who sometimes forget that category-defining businesses aren’t built solely by publishing papers reporting impressive benchmark performance in a lab setting. Indeed, as we’ve written elsewhere, good technology does not speak for itself.

While it’s possible to make your fortune off the back of a consumer app that goes viral, this is rare to capture lightning in a bottle.

Instead, the companies that succeed often have to i) convince someone to buy what they are selling; ii) regear their existing systems to integrate it; iii) potentially retrain their team; and iv) experience some short-term disruption or reduction in productivity to unlock long-term advantage.

This is possible, but it’s hard and it’s why most start-ups don’t succeed.

But it also means that if you’re first, you have a strong moat.

ElevenLabs was prepared to endure early controversy around the misuse of its product that scared off big tech labs.

Thanks to a good product, an early lead, and mindshare built through Instagram and TikTok creators, it built strong loyalty as it moved upmarket, with users at over 60% of Fortune 500 companies. It will have significantly more data than potential competitors on what users like and don’t like, which features drive growth versus which aren’t worth the investment, promising sectors and geographies, regulatory experience, and investors aligned with the vision.

Someone who’s built a good text-to-speech model in a lab, even if it’s theoretically better, will have none of these advantages. While they might be able to convince a few AI-native users who enjoy hacking things together to trial their work, convincing enterprise users with better things to do that they should switch from a product they already like is much harder.

You could make a similar argument about Synthesia, the AI-first video avatar platform, which outpaced HeyGen and Hour One, to build significant Fortune 500 uptake. This is a daunting lead for a new entrant to overcome:

There’s also a point about mindshare. Perplexity’s AI-powered search product likely isn’t yet deeply intertwined with many org’s tech stacks, but the company has invested significantly in brand, simply to drown out the voice of any potential competitor - going to lengths that few others would.

Any new AI-first search challenger has to both create a competing product, convince investors that it’s superior to a better-funded Perplexity (not to mention incumbents with the best business model ever invented) and then build consumer awareness from scratch.

Share

Frictions

In this month’s Guide to AI we picked up on some numbers in Menlo Ventures’ (very originally named) State of Generative AI in the Enterprise Report.

One of the most striking headline results is how Llama’s enterprise market share hasn’t increased at all this year. Despite models in the Llama 3 family now being competitive with the frontier across a number of metrics, the numbers remain flat. Similarly, Mistral, which routinely reports impressive benchmark performance has seen its market share decline.

Something that’s both really good and free. What’s not to like?

But this isn’t wholly surprising. The Llama users we know tend to be in one of two categories: AI researchers or AI-first start-ups. In short, they’re sophisticated users who understand model deployment and hosting, who are prepared to accept trade-offs in simplicity for lower costs and greater customization.

That’s a valuable niche, but if you’re building at the foundation layer, you’re unlikely to become the dominant player by exclusively serving it.

Most users will likely want something more straightforward, where they can abstract away as much complexity as possible. In the past, we’ve written about how enterprises (and their legal and procurement teams) will likely be biased towards providers who can offer them familiar software license-like terms.

Switching from using Claude or an OpenAI model to Llama will objectively involve more work and some disruption. To Carles’ point at the beginning, if you’re a mid-market logistics company or a hospital - is it likely to be worth it?

But, looking at the chart - hasn’t OpenAI lost significant ground to Anthropic? Doesn’t that undercut our point on frictions?

We’d note two things.

Firstly, OpenAI and Anthropic primarily derive their revenue from different sources.

75% of OpenAI’s revenue comes from its consumer-facing product, while 85% of Anthropic’s revenue comes from its API. Menlo’s numbers don’t distinguish between the two - and the survey could well be including people who expense a ChatGPT subscription to their employer (functionally a consumer subscription) or pay for their own.

Switching between consumer chatbots is essentially a painless process. Switching from one proprietary LLM to another as an enterprise is harder, but as we’ve covered in the past, the lock-in effect is often smaller versus many other tools.

There’s a parallel dynamic in hardware, which we’ve covered before in our writing on Intel and NVIDIA. Companies looking to challenge NVIDIA’s stranglehold on the AI market have often attempted to compete on individual metrics (e.g. inference speed or performance-per-dollar and performance-per-watt). But this approach fails to understand that for a company that’s optimized its models for CUDA and whose team is accustomed to the software ecosystem, the bar for switching is incredibly high. This doesn’t even account for how NVIDIA has cornered the data center market. How many teams want to either abandon hardware they’ve bought or reconfigure a cluster they’re renting from scratch?

Disruption is hard

We’ve talked a lot about challenging high-performing incumbents. Taking on ElevenLabs or OpenAI is obviously difficult. But what about sectors where there’s been next to no innovation in recent years?

This is where inertia and lock-in take hold. Legacy tools or software can maintain a dominant market share for years after better alternatives have come along. Adobe Flash persisted to 2020, even after HTML5 provided better alternatives for web animation and interactivity, while new entrants took years to start chipping away at Oracle Database’s market share.

Perhaps one of the worst examples of this trend is in the world of Internet browsers. Internet Explorer 6 was released in 2001, bundled in with Windows XP. By the turn of the new millennium, Microsoft’s superior financial resources meant that it had convincingly beaten Netscape in the First Browser War. The combination of limited competition and Windows as a distribution channel led Microsoft to consider web browsers solved. As a result, they froze IE development, releasing no major updates between 2001 and 2006.

In this time, IE6 went from being a passable product to being an actively bad one. There was poor practice, like security vulnerabilities being left unpatched for years at a time. But, more importantly, the product did not keep up with the changing nature of the web.

From about 2004, Ajax and Web 2.0 began the shift from static web pages to interactive applications where content could update without refreshing the entire page. IE6 struggled with this new paradigm because it used non-standard ways to handle background data loading (requiring special IE6-specific code), had a slow JavaScript engine with memory leaks, and implemented web standards like CSS incorrectly.

However, new challengers like Firefox, which could both handle this shift and had modern features like tabbed browsing, gained little traction. Internet Explorer was often reported as ‘slipping’ in this period, but its market share remained comfortably over 80%.

Share

But why?

For most consumers in 2024, the act of switching browsers seems trivial. Downloading and installing software is a routine task, while transferring over your bookmarks and passwords is usually straightforward. Literacy and confidence levels were lower 20 years ago, while many users were likely unaware that better alternatives to IE6 existed.

Meanwhile, because of its dominance, many developers had to build applications tailored for IE6’s quirks. The time and expense in switching could run into millions of dollars for a large organization. This created a network effect-driven doom loop where people built for IE6, which meant sites worked best in IE6, which caused people to continue to use IE6, even though it sucked.

Perhaps the most notorious example is ActiveX, a now deprecated Microsoft software framework that allows browsers to share data or functionality with apps (e.g. displaying and playing a video on a web page). There was a panic in South Korea in 2022 when Microsoft moved to end IE support, 7 years after deprecating the browser. This was because much of the country’s payments infrastructure, along with web services run by the public sector, were still reliant on ActiveX.

Even as the world attempted to move on from IE6, developers still had to maintain two separate codebases - one for modern browsers and a simplified version for IE6.

Meanwhile, Microsoft had to build an IE6 “compatibility mode” into later browsers, so people could access sites and web apps built for it.

But sometimes even that wasn’t enough. Darkly, you can find old Internet discussions where people need to downgrade because of this.

But of course, IE6 eventually died, as did IE more broadly.

Then, Chrome launched in 2008. Not only was it paired with Google’s massive distribution, it had a decisively better ability to run web apps, as they were becoming prevalent. This made the performance difference both obvious and important. The rise of smart phones also rendered IE’s fragile desktop monopoly irrelevant, while raising user expectations about functionality.

What can we learn from this?

In short, no company is undefeatable. But even if a product is bad, if it achieves widespread enough early adoption, it can take years to displace it.

Subscribe now

All empires eventually crumble?

If we believe that AI is going to remake everything, could it also loosen the grip of legacy software?

Yes and no.

It’s possible to imagine tools produced by companies like Vercel or Replit accelerating development and deployment optimization. Vercel’s v0, for example, could help refactor and modernize legacy code, while Replit tools like Ghostwriter could probably help document existing codebases.

AI-first tools could help reduce switching costs by accelerating the development of compatibility layers or providing better tooling for incremental migration.

Alternatively, generative software may have a role to play. Tools produced by companies like our friends at Patina Systems, might help democratize the rapid creation of software that works for them.

We also shouldn’t underestimate how modern cloud services are reducing switching costs. Many enterprise applications are now containerized - meaning different tools have their own codebase, communicate with each other through well-defined APIs, and can be deployed independently. We’re unlikely to see a whole country’s payment infrastructure depending on one company’s browser plug-in again.

But this is no cause for start-ups to become complacent. The biggest frictions in enterprise software migration often aren’t technical.

It involves changes to business processes, user training, data migration, compliance requirements, and integration with other systems. Although, we would contend that our friends at Interloom could help with some of this, by turning unstructured institutional knowledge into structured, AI-workable data.

Sticking with South Korea’s bizarre ActiveX technical debt - that stemmed from institutional dependencies, many of which were the product of bad policy, rather than no one in one of the world’s most technologically advanced countries understanding browser plug-ins.

It’s easy to see a world of smaller, more agile enterprises chopping and changing tools rapidly. But start-ups hoping to break into institutions or big enterprises will likely need to fight quickly and fight hard for some time to come. It means thinking about these challenges from day one.

Share

Closing thoughts

At our SF State of AI launch meet-up, we were lucky to be joined by Henry Modisett, who heads design at Perplexity. He reminded us that building for normal people isn’t the same as building for AI experts. Users shouldn’t know what model a product uses (nor do they even care to know), they shouldn’t have to decode the meaning of words like “agentic”, or have to fiddle around with model settings.

If you are AI native and are only thinking about AI natives, you are unlikely to consider friction or inertia as real forces. Trying a tool for the first time or implementing a new system is a source of excitement for you. For 99% of the population, any change to technology they regularly use is likely to be a source of nervousness or dread. If you can’t show your AI product to a normal person and have them immediately grok what it’s about, you’re in for some heartache. Getting them to try once might be doable, but invoking repeat usage or a full product switch for the same job becomes a reach.

We are currently in an environment where, thanks to the speed of frontier model development, there’s currently a big gap between our current capabilities and the products we’ve managed to build. Instead of trying to build the next incremental improvement on something that already exists, only to hit up against a wall of inertia, think about the delightful products you can build now that will generate quality data for you to build models native to the problems your users have.

If you think you’re building something that’s ‘first’ or are helping to break the vice-like grip of a legacy tool - get in touch!

Subscribe now

Introduction

Last week, Intel CEO Pat Gelsinger announced his intention to retire after a tumultuous 4-year stint at the helm of the company. Amid a global boom for semiconductors, Intel has clearly struggled. In Q3 of this year, it posted a loss of $16.6B, the largest in its history. In stark contrast, semiconductor darling, NVIDIA, printed revenues of $35.1B during the same time.

With its share price plumbing depths not seen since the bursting of the dot-com bubble, a once leading American company is now regarded as a target for either break-up or M&A. Until recently, Qualcomm was considering harvesting the company for parts.

While NVIDIA is the undisputed leader in the AI race at the moment, it is striking just how badly Intel has done, even for an NVIDIA competitor. In the last quarter, NVIDIA’s diluted earnings per share were 0.78, but AMD still enjoyed a healthy 0.47. Intel? -3.88.

So how did Intel miss the boat on AI?

If you don’t disrupt yourself, AI will disrupt you

A big part of Intel’s woes stem from the company simply not being interested in AI, for a long time.

Intel had flirted with the idea of buying NVIDIA in 2005 for $20B, with some in the company believing NVIDIA’s graphics chips may one day have a role to play in data centers. But given the choice between embracing a new direction or doubling down on money-printing chips underpinned by its monopolistic x86 instruction set architecture, Intel took the latter option. It’s not for nothing that a former Intel CEO once compared the x86 to a creosote plant - a bush known for poisoning anything that grows around it.

From 2007 onwards, NVIDIA began to invest aggressively in its CUDA ecosystem - which allows users to harness the parallel processing power of GPUs beyond their traditional task of graphics rendering and acceleration. For most of this period, this work was regarded by competitors and Wall Street as an eccentric waste of money. We covered the story of how NVIDIA spotted the potential of GPUs early and moved boldly back in May.

After AlexNet in 2012 and DeepSpeech 2 in 2015, which saw deep learning training speeds and model performance accelerated by GPUs, the world began to realise that NVIDIA were onto something.

Intel fought back with a grab bag of internal programs and an M&A campaign. But even as it ramped up its expenditure, the house view was that AI was still a peripheral market and that the future remained CPU-bound.

In 2016, Intel bought Nervana Systems, a new AI-focused chip company that had yet to launch commercially, for $400M and placed its CEO in charge of the company’s Artificial Intelligence Product Group. This product allegedly received poor customer feedback when it launched in 2019. So in the same year, Intel acquired Israeli AI chipmaker Habana Labs for $2B, which developed separate training and inference processors optimized for data centre applications. A year later, Intel killed off Nervana, whose executive team left to start MosaicML in 2021, a cloud infrastructure player focused on generative AI. MosaicML was acquired for $1.3B by Databricks in 2023.

Share

You don’t get to choose what matters to your customer

Habana had created Gaudi - Intel’s AI accelerator. With Gaudi, Intel made a crucial strategic misstep - they chose to compete on the wrong metric. In 2019-2020, NVIDIA was the dominant player with the V100 and then the A100. Intel rightly clocked that they couldn’t compete on raw performance or tackle NVIDIA’s software ecosystem head-on. Instead, they tried to target better performance-per-dollar and performance-per-watt. This remains central to the Gaudi pitch.

Unfortunately, companies don’t get to choose what matters to their customers. Even with better efficiency metrics, the switching costs from NVIDIA’s ecosystem were just too high for most customers.

A theoretical edge in efficiency doesn’t mean all that much when your models are optimized for NVIDIA’s CUDA, your team is trained on NVIDIA tools, and you use an NVIDIA software ecosystem you love (and hate) every day. The frictions are too great. Our earlier examination of NVIDIA also covered how the company cornered the data center market by building (still) unrivalled networking capabilities.

Intel has conceded that it will likely never compete against NVIDIA for high-end training workloads with Gaudi 3, and instead it focused on pitching to businesses that might need cost-effective, open models. But considering the low cost of accessing open models via cloud providers like Azure or AWS - it remains unclear whether this strategy is grounded in a clear-headed assessment of the market or … sheer desperation for whitespace. Based on early results, it would appear to be the latter. Intel has gone from predicting AI contracts worth $2B over the course of a year, to $500M, to dropping its predictions.

Subscribe now

It’s possible to make the best of a bad situation

In the spirit of charity, let’s compare Intel to a company that was also late to the AI party. While AMD unveiled its first AI-focused accelerator, the Radeon Instinct MI25 in 2017, it didn’t have a rival to NVIDIA’s tensor core until 2020 with the arrival of the CDNA Architecture and matrix cores.

But AMD got a couple of crucial things right that allowed it to build a respectable second-tier AI business.

For a start, AMD realised it couldn’t opt out of building a software ecosystem altogether. This led it to build ROCm ecosystem, a CUDA rival and to develop relationships with developers, just as NVIDIA had done for decades. In 2021, PyTorch unveiled an installation option for ROCm, while AMD has worked with Microsoft on an AMD-enabled version of PyTorch library DeepSpeed to allow for efficient LLM training. This summer, AMD bought Finnish company Silo AI, the largest private AI lab in Europe, who were building open foundation models for enterprise on AMD hardware.

Rather than chopping-and-changing between a mixture of in-house and acquired hardware approaches only to rush out an underperforming product, AMD patiently chose an approach that worked and stuck to it. As a result, its H100 competitor, the MI300X performs respectably for a quarter of the price.

Of course, AMD has achieved nothing like NVIDIA’s stratospheric levels of success, but it’s a profitable business whose share price has increased by over 250% over the course of the past five years, versus loss-making Intel’s 60% decline. We can’t all be NVIDIA, but we don’t all have to fail.

Share

You can’t just spend your way out of trouble

In 2021, Intel launched its Foundry business to manufacture chips for external customers using Intel's process technology. The company calculated that a combination of national security concerns about the semiconductor supply chain and growing demand for chips could allow it to diversify its revenue streams.

This work has roped in billions of dollars in subsidies from the US government via the CHIPS Act. So far, however, the bet doesn’t appear to be paying off. In 2023, the Foundry business saw an operating loss of $7B, the yield rate is reportedly poor, and the business primarily services Intel rather than external customers.

Intel itself relies heavily on TSMC for its chip production. In 2021, the company had negotiated a 40% discount with TSMC, but after Pat Gelsinger offended them by saying on stage that “you don’t want all of your eggs in the basket of a Taiwan fab”, the discount was not honored.

Closing thoughts

The story of Intel’s ‘vibe shift’ tells us a couple of things. If you’re not going to be first, you have to compensate for it by being smarter. The company failed to do this. Even as the ground shifted under their feet, the x86 architecture continued to poison the plants around it. AI was outsourced to M&A and a taxpayer-funded foundry empire was started in a fit of hubris.

Our series will return soon with examples of companies that have adapted to tectonic change much better…

Subscribe now