I Gave 7 AI Agents the Same Swift Challenge. It Was BRUTAL!

Highlights

• Most AI coding models consistently underperform in Swift development compared to Python or Java due to a data gap in private repositories and frequent API changes.

• Benchmark bias is a significant issue, as models are often optimized for Python-centric tests like HumanEval rather than complex iOS UI tasks.

• Opus 4.6 emerged as the clear winner, successfully building a functional and aesthetically pleasing 'DogTinder' app on the first attempt.

• The new Xcode AI assistant feature significantly improved Gemini 3 Pro's performance, highlighting the value of environment-specific integration.

• Open-source models like Qwen 3 Coder Next and Grok Code Fast struggled heavily with Swift project structure and UI design.

• GLM 5, despite claims of superior coding ability, failed the Swift challenge with broken imagery and incomplete features.

Timeline

1. 00:00 - 02:24

   The Problem with AI and Swift Development

   The speaker introduces the central thesis that most AI models fail at iOS development despite their success with web-based languages. He cites a comparative study showing that models like GPT and Claude perform significantly worse in Swift than in Python or Java. Three main bottlenecks are identified: the data gap from private commercial repositories, the rapid 'API drift' in Apple's ecosystem, and the benchmarking bias of existing tests. These factors combine to leave AI models with outdated knowledge and a lack of training on complex UI logic. This section sets the stage for the 'DogTinder' challenge meant to test these specific weaknesses.

2. 02:25 - 08:28

   Lower Tier Performers: Qwen, Grok, and Kimi

   The coding challenge begins with the lowest-ranked models, starting with Qwen 3 Coder Next, which failed to produce an openable project file. Grok Code Fast performed slightly better by providing functional code, though the design was uninspired and required manual file management. Kimi K2.5 showed a marked improvement by creating a recognizable Tinder-like interface with swipe animations, though it remained buggy and finicky. Each of these models required significant 'handholding' through Xcode's AI assistant to reach a semi-functional state. The speaker notes that while these models might excel at web development, they are currently 'one-trick ponies' when it comes to Swift.

3. 08:29 - 11:53

   Mid-to-High Tier Performers: Gemini and GPT 5.3

   Gemini 3 Pro initially failed through its CLI but achieved impressive results when utilized through the native Xcode AI assistant, producing a high-quality UI on the first try. GPT 5.3 Codecs followed, distinguishing itself as the first model to generate a complete project package without any manual setup. However, GPT's version suffered from design flaws, such as images exceeding the bounds of the app frame. This section highlights the importance of the tool used to access the model, as Gemini's performance varied wildly depending on the interface. Both models represent a tier where professional-grade code generation becomes more realistic.

4. 11:54 - 15:01

   The Champion: Opus 4.6 and the GLM 5 Bonus

   Opus 4.6 is crowned the winner for acing the challenge on the first attempt using the Claude code CLI, delivering a beautiful and fluid user experience. The speaker then introduces a surprise test for GLM 5, which recently claimed to outperform Opus in coding tasks. Contrary to these claims, GLM 5 failed the Swift test miserably, requiring three rounds of bug fixes only to produce an app that couldn't load images or handle navigation. This comparison reinforces that specialized benchmarks are often misleading when applied to specific ecosystems like iOS. The speaker concludes that Opus remains the most reliable model for this particular use case.

5. 15:02 - 16:24

   Final Verdict and the Future of Coding Models

   The video concludes by reflecting on the lessons learned from the competition between these seven AI agents. The speaker emphasizes that large model size and strong reasoning capabilities currently provide the only bridge over the Swift 'data gap.' He suggests that the industry may need to shift toward language-specific models to handle the unique demands of bespoke ecosystems. Using internal AI tooling, like the Xcode assistant, is recommended for developers who want to maximize the performance of existing models. The session ends with a call for viewer feedback on whether specialized or general models will win in the long run.