Your agent architecture has a half-life of 6 months — Dan Farrelly, CTO, Inngest

The Rapid Evolution of Agent Architectures [00:17]

• Agent architectures have a "half-life" of about 6 months, requiring frequent rewrites due to new models, frameworks, and tools.
• The speaker, Dan, CTO of Ingest, emphasizes practical experience in building AI infrastructure and agents.

  "if you've been building agents for more than 6 months, you've likely rewritten something, maybe more than once."

A Layered Approach to Agent Architecture [02:44]

• Traditional agent architecture discussions often focus on specific components rather than conceptual layers.
• The proposed mental model consists of three discrete layers: execution (brain), context (knowledge), and compute (hands).

  "there are three discrete layers. First, the execution layer. I think of this as the brain."

Understanding the "Half-Life" Concept [03:39]

• The "half-life" is a scientific term for the time it takes for something to decay by half.
• In agent architecture, prompts and models have short half-lives (weeks to months), while execution can last years if designed correctly.

  "And I think that your architecture has a half-life, also."

The Problem of Coupled Architectures [04:09]

• Most teams couple all layers together, causing the shorter half-life of one layer to drag down others, creating technical debt.
• Frameworks or custom solutions often suffer from merged layers, making components difficult to swap out.

  "And what happens then is that one layer's half-life kind of leaks and drags the other components down."

Focusing on the Stable Execution Layer [05:49]

• The execution layer is defined as the system responsible for reliably running code, managing how, when, and whether work completes, independent of infrastructure.
• It manages the full lifecycle: plan, call model, run code, invoke sub-agent, loop, retry, coordinate.

  "I define it as the execution layer as being the system responsible for running your code reliably, managing how, when, or whether each piece of work completes."

Key Requirements of the Execution Layer [06:47]

• Resumability: The system must pick up after failures without restarting from the beginning, crucial for long-running tasks.
• Durable and External State: State must live outside the immediate work process to enable resumability.
• Flexible Invocation Patterns: Support for crons, events, APIs, human-in-the-loop, and sub-agents is necessary.
• Observability: Providing visibility across the entire session, not just LLM or tool calls, is essential for debugging and improvement.

  "Your execution layer must enable your system to pick up after failures without restarting from the beginning."

Sandboxes vs. Execution Layer [09:37]

• Sandboxes are ephemeral and stateless, designed for executing code or browsing, not for durability or state management.
• The execution layer provides the sandbox with context, sequence, and durability, acting as the "brain" to the sandbox's "hands."

  "a sandbox is ephemeral and stateless by design. So, using it for durability, snapshots, or something in state, I think is an anti-pattern."

Emerging Agent Patterns and Execution Layer Needs [10:30]

• Future architectures like background agents, dynamic workflows, and autonomous loops are long-running, asynchronous, and delegated.
• These patterns require a fundamental execution layer for observability, inspectability, and the ability to mix and combine approaches.

  "They're all long-running. They're asynchronous. They're delegated. That means that you need to be able to observe them all down to the core."

Examples: Background Agents and Loop Architectures [11:33]

• Background agents can run for hours and involve numerous tool calls, necessitating robust infrastructure and observability.
• Loop architectures continuously assess system state against goals, requiring crons, sub-agent delegation, and inspectable history.

  "You can't even debug your background agent that's running asynchronously without the right infrastructure, without the observability, without everything that's going on in that process or multiple processes."

Building for the Future with Ingest [16:34]

• Ingest provides durable execution for AI agents, acting as the execution layer that can be combined with any context or compute layer.
• It offers durable steps, event triggers, scheduling, agent-to-agent coordination, and full session traces without managing infrastructure.

  "We're durable execution for AI agents. We're the execution layer."

Iterating and Scoring Agents [17:08]

• The execution layer is an ideal place to instrument and score agents by leveraging the data flowing through it.
• Outcome-based scoring, such as whether a triage action resulted in a PR, can be facilitated by attaching events to sessions.

  "When you have access to the data that you need that's all flowing through there or you've just executed something, it's a perfect place to run something after and defer some scoring for with the whole trace information, maybe the inputs and the outputs."

Conclusion: Embrace Change with a Solid Execution Layer [18:26]

• Build agent harnesses by understanding the layers and embracing the fast pace of change.
• Getting the execution layer right with the proper primitives allows other components to evolve more easily.

  "And I think if you can get your execution layer right and think about the right primitives, everything else can quickly evolve around it."