Andrej Karpathy: Software Is Changing (Again)

The landscape of software development is undergoing a profound transformation, moving beyond traditional code (Software 1.0) and neural networks (Software 2.0) to a new paradigm: Software 3.0. This evolution is driven by Large Language Models (LLMs), which are essentially programmable through natural language prompts, effectively turning English into a new programming language. This shift is creating a massive opportunity for developers entering the industry, as there's a significant amount of rewriting and new software to be built across these different programming paradigms.

LLMs themselves are complex systems, akin to early operating systems from the 1960s, and are currently distributed as utilities through time-sharing. While they exhibit superhuman capabilities like vast knowledge, they also possess significant cognitive deficits, including hallucinations and inconsistent intelligence. Developing effective LLM applications involves creating "partial autonomy apps" that integrate human oversight with AI generation, emphasizing user interfaces (GUIs) for auditing and control, and employing an "autonomy slider" to manage the level of AI involvement. The future of software will likely involve building more "Iron Man suits" – augmentations that empower users – rather than fully autonomous agents, with a focus on speeding up the generation-verification loop for human programmers.

A significant aspect of this new era is "vibe coding," where natural language allows individuals to create software without deep technical expertise, acting as a gateway to software development. The development of software for agents, which are human-like entities interacting with digital information, is also a burgeoning field. This involves making documentation and infrastructure accessible to LLMs, often through formats like markdown, and transforming traditional interfaces to be agent-friendly, ultimately aiming to automate more complex tasks and allow for gradual increases in autonomy over time.

Software Evolution: From 1.0 to 3.0 [00:41]

• Software has not fundamentally changed for 70 years, with two rapid shifts in recent years.
• Software 1.0 refers to traditional computer code.
• Software 2.0 refers to neural networks, specifically their weights, tuned via data and optimizers. Hugging Face is considered the equivalent of GitHub for Software 2.0.
• Software 3.0 is introduced with programmable neural networks, particularly Large Language Models (LLMs), where prompts act as programs.
• Prompts for LLMs are written in English, making it a unique programming language.
• The example of sentiment classification illustrates the shift: writing Python code (1.0), training a neural net (2.0), or prompting an LLM (3.0).
• GitHub now includes English interspersed with code, indicating a new category of code.

Remarkably, we're now programming computers in English.

This section outlines the historical evolution of software development, contrasting traditional programming (Software 1.0) with the data-driven approach of neural networks (Software 2.0) and introducing the concept of Large Language Models (LLMs) as the foundation for Software 3.0, where natural language prompts serve as executable code. The speaker emphasizes the fundamental shift and the emergence of English as a programming language.

Related Video Summary

• The Future of AI Prompting: 5 Context Levels
• I copied TikTok trends and made millions with AI
• 7 AI Tools That Actually Make Your Life Easier

LLMs as a New Kind of Computer and Operating System [06:06]

• LLMs are being compared to electricity as a utility due to their widespread demand and metered access via APIs, requiring low latency, high uptime, and consistent quality.
• The infrastructure for LLMs involves significant capital expenditure (capex) for training models, similar to building power grids, and operational expenditure (opex) for serving them.
• Analogy to electricity utilities: just as one can switch between grid, solar, or battery power, one can potentially switch between different LLMs.
• LLMs are seen as having properties of both utilities and fabs (manufacturing plants), due to the high capex and rapidly evolving technology tree involved in their development.
• A stronger analogy is that LLMs are like operating systems, representing complex software ecosystems rather than simple commodities.
• The ecosystem mirrors operating systems with closed-source providers (like Windows/macOS) and open-source alternatives (like Linux), with LLMs having competing closed-source providers and the potential for open-source ecosystems like Llama.
• LLMs are described as a new type of computer, with the LLM itself being the CPU equivalent, context windows as memory, and orchestrating memory and compute for problem-solving.
• This resembles operating systems from the 1960s, where compute was expensive, leading to centralized cloud-based systems and time-sharing.
• The concept of personal computing for LLMs is still nascent, with Mac Minis being suitable for batch-one inference due to being memory-bound.
• Interacting with LLMs via text feels like communicating with an operating system through a terminal, with a graphical user interface (GUI) for LLMs still largely uninvented.

LLMs certainly feel like they have properties of utilities right now.

This section delves into the nature of LLMs, drawing parallels to utilities and fabs, but ultimately concluding they are more akin to operating systems. The speaker highlights the early-stage, 1960s-like computing environment LLMs inhabit and speculates on the future of personal LLM computing and the potential development of GUIs for these systems.