Using AI without outsourcing the learning

Anthropic published research on AI assistance and coding skills that made me think about what we learn while getting work done.

In a randomized study, 52 mostly junior software engineers with Python experience worked with Trio, an unfamiliar asynchronous programming library. Some had access to an AI assistant; others worked without it.

Three findings stood out:

• Learning: the AI group averaged 50% on the subsequent assessment, compared with 67% in the control group. That is a gap of 17 percentage points.
• Speed: the AI group finished about two minutes faster on average, but the difference was not statistically significant.
• Debugging: this was the area with the largest gap, which matters when the next task is to inspect and supervise generated code.

The researchers also looked at how participants used the assistant. Asking for explanations and building understanding were associated with better learning outcomes than delegating the work wholesale. Those interaction patterns came from small groups; they are useful signals, not proof that a particular prompting style causes better learning.

My takeaway is to treat AI as a comprehension partner when learning something new: ask why, inspect the result, and make sure I can explain and debug it myself. Finishing a task and understanding it are different outcomes, and I want to keep paying attention to both.

This study concerns a specific learning task and an immediate assessment. It does not establish what happens to long-term expertise or to experienced developers working in familiar codebases.