Kimi K3: Can It Beat Opus 5 on 3D Worlds?

Gemini K3 Release and Task Setup [00:00]

• The release of Gemini weights is announced, positioning it as a top open model.
• A comparison is set up to test Gemini against Opus 5 for long-horizon tasks, specifically generating explorable 3D worlds from a single reference image.
• The output from Opus 5 is presented as the benchmark for quality, physics, visuals, and interactions.

  Okay, so Gemini weights are officially out. It's the best open model out there.

The Challenge: 3D World Generation [00:16]

• The test involves generating a 3D world from a single reference image, a task that requires the model to create a new world rather than recalling existing game code.
• This is defined as a long-horizon task requiring orchestration of multiple sub-agents.
• Generating the Opus 5 example consumed approximately 20% of the creator's weekly cloud code limits.

  And this is a long horizon task where the agent has to use a number of different sub-agents, orchestrate them.

Prompt and Testing Methodology [01:31]

• The goal is to test Gemini using the exact same prompt as used for Opus 5.
• The prompt encourages the creation of multiple sub-agents that verify their own work, introducing ambiguity to test the model's decision-making.
• The Klein harness, an open-source tool for large language models, is chosen for testing.

  So, this is going to be a very long horizon task with intentionally adding a lot of ambiguity, so the model can make its own decisions.

Introducing the Klein Harness [02:16]

• Klein is an open-source harness for LLMs, released under Apache 2.0 license with significant GitHub stars.
• It offers IDE extensions, a CLI, and an SDK.
• Klein Pass provides subscription-based access to open-weight models like GLM-5.2 and Gemini K3 with generous quotas.

  And for open models, they offer Client Pass, which is a single subscription with very generous quotas and reliable access to the best and latest open weight models.

Setting Up and Running Gemini K3 [03:30]

• The video demonstrates using the Klein CLI to install and run the harness.
• Gemini K3 is selected as the model for testing.
• The reference image and the prompt are provided to the Klein harness.

  So, in the rest of the video, we're going to be using the Client harness. I'm going to be using the CLI and it's very easy to set up.

Gemini K3 Performance and Initial Results [04:45]

• Gemini K3 is noted as being fast compared to its initial release.
• The model goes through a planning phase, utilizing seven sub-agents and an orchestrator.
• Initial results show a smooth visual experience, though not as refined as Opus 5's output.

  Okay, so Gemini K 3 is pretty fast compared to when initially it was released on Moonshot.

Iterative Refinement and Improvements [06:37]

• The process is iterative, with models refining their output over multiple passes.
• Improvements are observed in details like car tires and roof material quality.
• The agent's ability to iteratively refine output based on specifications is highlighted as a key capability.

  So, if you actually look at the car, even with the wheels this time it added iron wheel tires, right?

Final Observations and Recommendations [07:58]

• The latest version shows improved visual consistency and closer adherence to the original reference image, particularly in building placement.
• The creator finds Gemini K3 to be a capable workhorse model, suitable for implementation tasks.
• For design-intensive tasks, Opus 5 or Fable 5 are still recommended.

  So, Gemini 3 is a really capable open model. I personally think it's really great as a workhorse model.