Matthew Honnibal

LLMs have brought something like a post-scarcity mindset to software projects. We can generate code much much faster than we used to be able to. Much of the programming best practices we've built up feel hard to apply to the new workflows, because what it means for a team to be productive feels very different. However, many things we've learned about managing software complexity still apply. Agents are still not great at long-term considerations, so it's very helpful to have a good concept of what you should be steering the agent towards.

Software fundamentally doesn't "want" to scale. The default is for complexity to grow exponentially with the size of your codebase. This is why many non-professional programmers, such as researchers or domain experts, can fluently code programs to a certain size, but no larger. To create larger projects you need to pay attention to different considerations.

In this talk I'll focus on three things agents consistently get wrong in Python: error handling, interface contracts and architecture. These are the most important ways agents produce pieces of code that appear to work, but which ultimately assemble failure out of these seeming successes.