Arvind Krishna & Nikesh Arora: Quantum Threat, Open vs Frontier AI, AI Cyber Warfare & SaaS Collapse

The Quantum Computing Threat to Cryptography [0:00]

• The conversation pivots from AI to cryptography and the risks posed by quantum computing.
• A DARPA competition, Quantum Benchmarking Initiative (QBI), is underway to identify viable quantum computers.
• Experts predict that by 2030, quantum computers will be capable of breaking current cryptography.
• Data encrypted today could be compromised years later by future quantum computers.

  "Well, it will break to this cryptography. That's not a supposition. That's just a reality."

The Urgency of Post-Quantum Cryptography [1:15]

• Post-quantum cryptography is understood and its code is written, but adoption has been slow.
• The current moment is seen as a wake-up call for implementing quantum-resistant solutions.
• The rigor of quantum-proof cryptography is compared to the security of current cryptography against classical computers.

  "The good news is post-quantum cryptography is understood. It's well known. The code is written, but people have been too lazy to get to it."

Milestones in Quantum Computer Development [2:10]

• "Toy" quantum computers exist but are error-prone and not yet fully scalable.
• The first milestone is demonstrating error correction or mitigation on a running quantum computer, with claims expected by next year.
• Third-party evaluation will be crucial to validate these claims.
• Progress in error correction, though not at scale, is anticipated by 2027.

  "So question one will be, who's the first one who can demonstrate not a simulation, not on the side, but some level of error correction or mitigation on a running quantum computer?"

Scaling Quantum Computations [3:35]

• The second milestone involves scaling quantum computers to handle computations at a larger level, beyond the current ~10,000 gate computations.
• The pace of scaling is questioned, with a hypothetical "adding a zero each year" leading to 10 million gate computations by 2029.
• This level of scale could yield surprising capabilities.

  "Question two will be, can you run these things at some level of scale, not yet complete scale?"

Logical Qubits and Effective Capacity [4:25]

• The concept of "logical qubits" is discussed as a measure of error-corrected operating qubits.
• A large number of physical qubits can translate to a smaller number of logical qubits due to error correction.
• This metric helps understand the effective capacity of a quantum machine.

  "So if I begin to error-correct it, it in some sense decreases the effective capacity of the machine. So a logical then says, okay, what's the effective capacity of this machine is, I think, a reasonable way to look at it."

Identifying Vulnerabilities for Attackers [5:35]

• Red teams focus on identifying "honeypots" or vulnerabilities for potential attackers.
• Crypto keys are ubiquitous across IT infrastructure, making them prime targets.
• Anything not quantum-proof will be breakable.

  "The first thing that's going to happen is whatever is not quantum proof is going to be breakable."

The Need for Immediate Quantum-Safe Infrastructure [6:00]

• Waiting until 2029 or 2030 to transition to quantum-safe infrastructure is too late.
• Companies should be replacing their IT stacks with quantum-safe solutions now.
• Post-quantum ready keys can be used as a stopgap measure.

  "So you should be built, you should be replacing your IT stack today with quantum safe or quantum proof infrastructure right now."

AI's Role in Code-Breaking [7:00]

• A senior AI researcher suggested AI models could break cryptography before quantum computers do.
• This idea stems from AI's potential to discover new mathematical proofs and models.
• The OpenAI math dump is cited as a potential early indicator.

  "Well, I mean, his view was actually models, AI models today can get us there before quantum computers get us there."

Limitations of AI in Cryptography [7:50]

• Simulating quantum processes on classical hardware (including AI) is not expected to be more efficient.
• Quantum computers operate differently than classical computers, which rely on arithmetic.
• While AI can discover quantum algorithms, running them efficiently requires quantum hardware.

  "You cannot simulate something on a different architecture and expect a more efficient result."

Classical Approaches to Code-Breaking [8:55]

• It's possible to use classical computing with advanced number theory to break cryptography.
• Discovering better predictors for prime number distributions could enable classical models to break encryption.
• The key question is how long such an approach would take.

  "It is possible. The question is, how long will it take? Because that's always the question."

Open vs. Closed Source AI Models and Cost [10:00]

• The discussion shifts to the philosophy around open-source, closed-source, and open-weight AI models.
• Teams should be allowed to experiment with the best available models without immediate cost optimization.
• For practical enterprise use, smaller, task-specific, compressed models are often more suitable than large, expensive frontier models.

  "So my point is, what we're doing is we're trying to figure out what task requires what level of intelligence and what is the cost of that intelligence for that particular task."

Strategic Use of Frontier AI Models [11:30]

• Frontier LLMs are valuable in unique, high-stakes situations, such as during a security breach, where business impact justifies the cost.
• In such scenarios, complex reasoning and agent swarms might be employed for defense.
• For most enterprise use cases, training customized open-source models is more practical.

  "At that point in time, I can run agent swarms, I can do all kinds of stuff that you want from a defensive perspective."

Economics and IP Drive Open-Weight Models [13:00]

• Economics heavily favor open-weight models, with potential cost reductions of 10-40 times compared to proprietary models.
• Protecting intellectual property (IP) also drives companies to run models internally.
• Geopolitical factors, with leading open-weight models currently from China, also influence the adoption of Western open-weight alternatives.

  "So I think economics is important. The second is going to be, at the end of the day, you just talked previously about chain of thought, not just the end tokens."

Developer Behavior and Production Usage [14:30]

• Developers may default to more expensive, easier-to-use models unless controls are in place.
• While experimentation can allow for expensive models, skepticism exists about their value for production usage beyond niche cases.
• The ease of use can lead to inefficient choices if not managed.

  "Developers are lazy. They're going to use the most expensive, the most expensive answer unless you put a control on it, because it's easy."

AI Models and Vulnerability Detection [15:30]

• AI models, trained on code, can identify both good and bad code, including vulnerabilities.
• Attackers only need one vulnerability, while defenders must find and patch all of them.
• While models can identify vulnerabilities, human testing is still required to confirm and patch them.

  "Well, guess what? If you teach a model what good code looks like, it also can figure out what bad code looks like."

Learning from Warfare for Cybersecurity [17:00]

• Concepts from warfare, such as powerful alliances, asymmetry, and technological superiority, can inform cybersecurity strategies.
• Cybersecurity can benefit from collaboration, pooling vulnerability information, and distributing patches collectively.
• Achieving technological superiority in cybersecurity is a key goal for defenders.

  "And so our goal is, can we get technological superiority? And that's what Nikesh is referring to."

Agentic Code and Verifiability [18:30]

• Agentically written code can be superior to human-written code, especially when agents perform evaluations.
• However, fully verifying the correctness of such code is computationally infeasible with current technology.
• The challenge of verifying code becomes harder with unlimited input possibilities.

  "All that said, we're only finding all the flaws that we know how to find. And so you get into this weird loop in computer science."

The Inevitability of an AI Arms Race [20:00]

• An international agreement to halt AI development is unlikely, leading to an inevitable AI arms race.
• This necessitates a focus on defensive and offensive superiority, compute capacity, and protecting economies.
• The argument for technological superiority as a driver for defense is presented as a way to change the conversation around AI regulation.

  "The argument doesn't work. Yeah. Because you wrote the software."

Populism vs. Economic Advantage in AI Regulation [22:00]

• AI regulation is largely driven by populism aimed at gaining votes, rather than pure reality.
• There's a balancing act between political pressures and the long-term economic advantages of AI development.
• The potential for state-level bans on data centers is discussed, with a prediction that such widespread bans are unlikely.

  "The regulation question on AI and data centers is driven, I'm going to be sorry and cynical, is driven by populism, not by any reality."

The Digital Divide: Haves and Have-Nots [24:00]

• A potential separation between "haves" (Western markets) and "have-nots" in AI access is a concern.
• The goal is to make technology cheap and accessible to everyone.
• The primary challenge is the uneven distribution of compute development, with potential supply constraints limiting access outside major tech hubs.

  "I fear that we will see fast advances in compute development in two large countries in the world. We may not see that as evenly distributed across the rest of the world."

Labor vs. Capital in the AI Era [25:30]

• Historically, nations that invest more in capital over labor tend to win in economic cycles.
• Nations prioritizing labor protection at all costs may lose out to those embracing capital-intensive AI development.
• Job displacement is seen as potentially leading to better jobs, contingent on social philosophy.

  "Nations that get very hung up on social and left-leaning philosophies that labor has to be protected at all costs... are the ones that are going to lose."

Vertical Integration in Technology [27:00]

• Vertical integration, exemplified by the IBM mainframe, has ensured longevity and IP protection.
• Modern vertical integration focuses on controlling key aspects of the stack, like chip design, while commoditizing others (e.g., fabs).
• NVIDIA is presented as a modern example of vertical integration, controlling the stack from hardware to applications.

  "The IBM mainframe has lasted 65 years. Why? It was vertically integrated."

Open vs. Closed Stacks and Application Layer Strategy [29:00]

• The AI landscape is seen as a battle between open and closed stacks.
• As models become more complex, separating context, memory, and reasoning becomes difficult, favoring integrated stacks.
• Vertical integration at the application layer will likely emerge, creating a "mode" that is hard to migrate.

  "I think at some point in time, this notion of running something like an open router become impossible."

Horizontal Expansion and M&A in Enterprise Tech [31:00]

• Horizontal expansion through M&A has been a strategy to drive synergies and cross-sales.
• In cybersecurity, there's a trend towards consolidation, replacing numerous disparate solutions with integrated platforms.
• The value of software is being re-evaluated in the age of AI, leading to potential disruption and consolidation in the SaaS space.

  "We're working as fast as we can to take disparate parts of our industry and find the way to stitch them together to have the data work together."

Consolidation and the Future of Software [32:30]

• The average enterprise uses hundreds or thousands of software vendors, creating a massive attack surface.
• AI capabilities may replace many of these vendors, leading to consolidation around core platforms.
• Software with high interaction value is more susceptible to AI replacement than highly reliable, low-level functions like database lookups.

  "The average enterprise uses anywhere from 500 to 5,000 distinct software vendors."