How do you stay familiar with the code when it's written by an LLM?
If you used an LLM to write code you shipped a month ago, how well do you remember it? Would you realize if the code you committed today would break it? If a customer reported a bug, would you quickly think, "Oh, I know what that is" in the same way you used to?
As you have LLMs write more code, you'll quickly realize you're less familiar with the code you ship. You have a harder time remembering the decisions you made. You no longer instinctively know where the code lives. If someone asks you a question about it, you have to go back to the LLM instead of answering off the top of your head. Over time, you have to delegate more work and more understanding to the LLM. Your guidance to the LLM gets worse as you lose knowledge of the code. And when the LLM struggles to write the next feature, you don't even know what the cause is, let alone how to help.
Is this just what software development is now? How can you be responsible for code you don't understand?
Does it matter?
Maybe you don't care! Maybe next month's LLM will just solve the problems caused by this month's model. Maybe you are entirely outcome-focused, and the code doesn't matter as long as it solves your problem. Maybe this is a one-off project that just replaces something you would have downloaded a package for, if only it existed — you wouldn't have looked at that code, right?
But if this is an app you or your company depend on, that you are crafting over time, it does matter. People rely on what you produce, and the work you do today has to live on indefinitely.
You have to care about the how, and not just the what.
Writing the code by hand, creating and fixing problems, deciding where this new code goes, refactoring old code to make it better — all these build your intuitive understanding of the code. When someone finds a bug and you immediately guess the answer, it's based on that intuition. It's how you know that changing a value over here will need a migration over there. It's why you have teams that focus on different areas of the code. And if you're not careful, code-generating LLMs will keep you from building that intuitive understanding until it's too late.
Meanwhile, AI uses three signals to decide how to write code: training data, the code it sees in your codebase, and your guidance. As you lose your ability to guide the LLM, and as it sees more of the code it wrote and less of the code you wrote, it doesn't matter what your programming processes and philosophies are. It's dragging your codebase to the mean.
So how can you stay familiar with the code as you write less of it?
Make mistakes
Have you ever answered something embarrassingly wrong on a test and still remember the right answer today? As programmers, that failure-to-surprise feeling is why a lot of obscure pieces of information stick with us. When an LLM is generating the code, you don't get to feel that. The process is too smooth.
It's like reading the answers in the back of the book without thinking about the questions first.
Instead, you have to cultivate that feeling of surprise. Look for opportunities to detect mistakes, or even create them. Tell an LLM to try a wild optimization, think about why it won't work, and then see if you're right. Have the LLM give you options without feedback, identify the pros and cons of each, then send it in the best direction (or, because it's an LLM, many directions at once!) and see what comes out. Create opportunities to be surprised and get things wrong. Those will be the parts that stick with you.
Type it yourself
LLMs are much faster than me at the initial implementation of a plan, but much slower at small refinements. When you were first getting used to LLMs, it was nice to push them anyway so you could practice guiding them. Now that you use them all the time, take some of that agency back.
Just like writing your notes helps you remember them, typing your code helps you remember it. If you're making small changes yourself, you have to understand what the LLM has done so far, just like you have to understand enough of any codebase in order to change it. You'll more strongly remember the code you typed and the code you had to learn about to get there.
Ask questions
There are an infinite number of ways to implement a feature. So if you try, you can think of an infinite number of questions about it. Ask these to yourself. How would I have implemented this? How did the LLM do it? Why did it do it that way? Did it miss anything I know about the code? Did I? What other options were there? Why did it pick that variable name? Is there a cleaner way to separate this?
That kind of curiosity is good to cultivate in general. It's especially important with LLMs, whose preferences you don't know as well as your teammates'. It also creates those opportunities for surprise that help things stick.
You can even quiz yourself. Without looking, can you name all of the objects involved in this change? Which areas would you expect this change to ripple out to? What fields now exist on each changed class, and what are their types? Could you sketch out the system's architecture after the change the LLM just made? Thinking, guessing, and checking is a great way to reinforce your memory of what you just read.
With LLM-generated code, you have the extra benefit that the LLM has a chat window open right there. Once you think through your own answers, you can ask the questions to the LLM and see what it comes up with. I've learned a lot this way, and they're things that have stuck. And the interactive review has helped the code behind the feature stick too.
Explore the code
I find GitHub's PR view very hard to use for large pull requests. For example, you only see the code context for the bits that changed. If I'm reviewing a PR for a part of the codebase I've never touched, the best I can be is an expensive linter: I can give local, tactical feedback, but can't see the bigger picture.
Reviewing LLM-generated code is similar. Look at the diff and conversation trace all you want — you're only seeing the parts it decided to change.
Instead of using GitHub for large PRs, I check the code out in my editor. That way, I can do searches, jump to definition, look at sibling files in the files pane, run it in a browser, set breakpoints and inspect values … it's an active review, not a passive one. This also works for LLM-generated code on your machine, and you don't even need to check it out first. Just like a large PR, take the time to explore. Understand the context, not just the changes, and take an active part in learning about what it did.
Generate an HTML explainer
If you have a big change with a lot of pieces, I like using the HTML explainer approach. That's where you ask the LLM to generate a single-page HTML file explaining the feature and its architecture. Have it include diagrams, editor links to the code, inline code snippets, and anything else you think will help you get context.
These are helpful when I'm not sure where to begin exploring. But it's not a complete answer. Just like reading a book about a programming language doesn't really teach you that language unless you start to write in that language, this won't help you learn about the feature by itself. Still, when you're dealing with a large or complex change, it's a great starting point to help with the suggestions above.
Review like a PR
I ban my agents from committing and pushing in our main repositories. I'm responsible for the code with my name on it, and I think it's rude to make someone else look at code I generated that I haven't looked at myself.
After the LLM has changed files, I review it in my editor's git tools as if I were doing a PR review for someone else. I leave comments, ask questions, and push for revisions until it's something I'd approve in GitHub.
I had the agent create a simple skill to read and address comments prefixed with AI? or AI: in the code. That way, I can annotate the code as I read and ask it to handle my feedback when I'm done. By making this a skill, the agent will sometimes pick up on it on its own if you mention leaving feedback, and it will check it in a consistent way. This makes it feel like a real code review, and that approach works very well for me.
Although this has the same problem of only seeing local changes that I mentioned earlier, it's a nice final gate before I allow code to go in and a good trigger for me to go through those earlier steps.
Build your awareness
When you're not as involved in writing the code, you have fewer opportunities to question the decisions that went into it. So those gut feelings when something looks wrong, overcomplicated, or broken are even more important.
You don't have to know what's wrong. You don't have to have the answer yourself. But you have to more actively notice when those feelings come up as you're reviewing the LLM-written code and ask it about them. "This seems complex," "I'm not so sure about X," or "I have a bad feeling about this" are often enough to guide it onto a different track. You'll feel more involved with how the code is generated, and you'll remember, "Oh, I helped guide this code into this shape."
You don't have to go through all of these every time, but they become natural habits when you do them frequently. Yes, it'll take longer than giving an agent total control of a feature until it ships. But if you're the one responsible for the development of a project, you owe it to yourself, your team, and your customers to understand it. Otherwise, you're betting your future on a future LLM to understand it for you and make better decisions than you. And at that point, why do they need you?
For additional perspectives on how AI is shifting how we write code, read the Aha! engineering blog.
