Thoughts and lessons learned from Going All-In with Claude Code

Focus on one thing and then iterate and refine and then iterate and refine some more.

Two Months with Claude Code: Seven Lessons from Going All-In

Anybody who’s spent enough time working with AI probably hit this: the models do something so good you rethink how you’ve been working for the past decade, then, twenty minutes later, it does something so wrong you wonder how that’s even possible.

I’ve spent the last two months using Claude Code for nearly everything: blog posts, cover letters, job applications, coding projects, research. Not casual tinkering. Daily, sustained, serious use. And the best summary of the experience so far is a concept Ethan Mollick calls the “jagged frontier of intelligence”: AI capabilities aren’t a smooth pyramid where the model is generally smart or generally dumb. They’re wildly uneven. Brilliant here, bafflingly wrong there. The frontier has peaks and valleys, and until you’ve walked the terrain yourself, you can’t predict where the edges are.

That unevenness is the defining experience. Not the hype, not the disappointment. The unevenness. And the real work has been building patterns to compensate for the valleys while leaning into the peaks.

Here’s what I’ve learned.

1. The jagged frontier is real, and the advantage is in the refinement

On some things, Claude Code is amazing and it blows your mind. On others, it’s completely useless and you can’t understand how it got you wrong. The gap between peak and valley isn’t subtle. It’s disorienting.

I built a writing mode for cover letters and resumes. I built a coding workflow with structured specs and test-driven development. I built a deep research pipeline. Each of these started rough and got dramatically better through trial and error, through discovering the specific ways the model falls short in each context and building compensations for those shortcomings.

That process is where the real value lies. Not in the tool itself, but in the accumulated refinement: the guidelines, the patterns, the small decisions about what to include in a prompt and what to leave out, the structural choices about how to break work into phases. If you’ve done that work for a particular use case, you’ve collected something that’s not trivial to replicate. Someone else using the same tool, with the same subscription, won’t get your results because they haven’t walked the same terrain.

This is the “unfair advantage”. Not the AI. The refinement you’ve layered on top of it. Which is no different than any other skilled job at any point in time: experience is the defining factor.

2. Sub-agents beat instructions for preventing fabrication

Fabrication is the central recurring problem across every use case I’ve tried. And I don’t mean the model hallucinating something wild and obviously wrong. I mean subtle, plausible fabrication that you might not catch unless you’re paying close attention.

Here’s a story. I built a mode to help me write resumes and cover letters, grounded in my actual career history. I gave it my real data. I told it not to make things up and, you guessed it, it still made stuff up. It extended position timelines to fill employment gaps. It changed job titles to match the target description. I have not been a principal staff product manager, but somehow it concluded that it was okay to rewrite one of my titles to say that, because I’d been a senior PM and a director, so close enough, right? It made stuff up in subtle and weird ways, and instructions alone couldn’t stop it.

I saw the same pattern in coding. People complain that AI agents doing test-driven development will change the test to match the code, or write code that technically passes but misses the intent, or outright lie about test results. My intuition is that this happens because the same context that wrote the code also evaluates its own work. It knows what it wrote. It has an incentive to fudge things in order to comply.

The breakthrough was separating the work into phases handled by different sub-agents, each with fresh context. One agent writes, another fact-checks. One agent implements, another verifies the tests. The verifier didn’t write the code, has no context about it, and starts with a clean slate. I completely gave up on instructions alone and started using a second pass with a sub-agent as a fact check.

What matters here is the mechanism: fresh context removes the incentive to be self-consistent at the expense of being correct. And you don’t need elaborate orchestration to do this. Claude Code has built-in task spawning that gives you a sub-agent with fresh context. No additional code, no tools, no frameworks. Just by doing that, you get a fresh context and you don’t need to pass anything except the specific task.

I’ve tried hard to resist adding complexity. There’s a temptation to model agent orchestration like a real team, with separate agents as coworkers and orchestration layers connecting them. That’s not necessarily wrong, but it introduces code and infrastructure. For my purposes, Claude Code’s native sub-agent spawning does the same job with less.

3. Context is everything (keep it under 50%)

There’s a piece of research from Chroma that showed model quality degrades noticeably once context usage passes about 56%. That number stuck with me, because it matched what I was seeing: the longer a session runs and the more context fills up, the worse the output gets. More repetition. More fabrication. More ignoring instructions that worked fine twenty minutes earlier.

I try to keep context usage under roughly 50%. Sub-agents help here too, because each one starts fresh instead of inheriting a bloated context window. But there’s a subtler problem that took me longer to see.

When you hit a quality issue, the instinct is to add more instructions. The model fabricated? Add a guideline. It ignored a constraint? Be more explicit. And each of those additions is reasonable on its own. But collectively they fill context with system-level instructions, which squeezes out room for the actual work. You end up refining, refining, refining, and as you refine and add more instructions, you also risk going overboard. The model gets confused not because it lacks guidance, but because there’s too much of it.

The tension is real: you need enough guidelines to prevent fabrication, but not so many that you overwhelm the context. You gotta give clear instructions, but no more than needed. I don’t have a clean answer for where that line sits. It shifts by use case. But the principle is consistent: treat context as a finite, valuable resource and be deliberate about what goes in.

The new 1M context window does not change that.

4. Structure your input

People love talking to their AI. Voice flow is natural, and it’s genuinely useful to walk and think out loud and let the tool capture your ideas. I do this regularly. I record audio while walking, transcribe it, and feed the transcript to the model.

But raw transcripts are terrible input. They’re rambling, unstructured, full of half-finished thoughts and tangents and verbal tics. That ambiguity creates quite a bit of opportunity for fabrication and confusing the model itself. When the input is vague, the model fills gaps with plausible-sounding content that may or may not reflect what you actually meant.

The fix was simple: add an intermediate step. Instead of feeding the raw transcript directly to the agent that does the work, I first run it through a briefing pass. That pass extracts the key ideas, themes, and arguments into a structured brief. Then the structured brief goes to the next agent.

The principle is broader than transcripts. Always try to give structured data to the model. If you’re passing in research, summarise the key findings first. If you’re describing a feature to build, write a spec rather than describing it conversationally. The more structured your input, the less room the model has to misinterpret, and the better the output.

This is, incidentally, the process I used to write this very post. I recorded my thoughts while walking, transcribed them, briefed the transcript, and then drafted from the brief. The brief you’d see behind this post is several pages of structured themes, arguments, and direct quotes from my own audio. You can read more about that here if you care.

5. Memory capture works; memory retrieval doesn’t

Everybody loves the memory. ChatGPT has it, Claude has it, and people get genuinely excited when the tool remembers something from a previous conversation. But these are very simplistic forms of memory. A flat list of facts, stored and retrieved without much intelligence about what’s relevant when.

I’ve built my own memory system: markdown files in a git repo, organised by topic. Career history, voice guides, project context, personal preferences. It’s simple, introspectable, and version-controlled. I can read the files, see exactly what the model will see, and edit them when something’s wrong. Compared to opaque memory systems hidden inside a product, I much prefer this.

Capture works. I have hooks that record learnings from each session, and I periodically process those into the memory files. The information gets saved, and it gets saved accurately.

Retrieval is where it breaks down. To just shove everything that you have in memory into a session is not very effective. It fills context (see lesson 3), and most of it isn’t relevant to the task at hand. What you actually need is selective retrieval: bring in the right memories for the right task, and leave the rest out.

This is where solutions like RAG, graph RAG, Mem0, and Letta are often brought in. They make sense in theory and they make sense for production systems. But for local agent use, they feel like overkill. You lose the introspectability of simple files. You add infrastructure. You trade a problem you understand (selective retrieval is hard) for a different problem (now your retrieval system is a black box).

I suspect the right answer involves a lightweight agent that reads the conversation and decides what to pull in, without requiring a full vector database, but I’m not there yet.

6. Bake research into the process

Here’s a story that crystallised something for me. I built a project to parse Claude Code’s session files. I used my spec process with test-driven development. The code was excellent: well-structured, clean, all tests passing, no cheating. I was impressed.

Then I ran it. And it failed. The file path it used for Claude Code’s session files was wrong. The model had outdated information about where those files live on disk.

You would think that a tool called Claude Code at a minimum knows very well how Claude Code works. But no, it does not. The model’s training data has a cutoff, and Claude Code’s file structure had changed since then.

This drove me to integrate deep research directly into the spec process. Now, when I write a spec for a task, the workflow also fires off a research pass that searches current documentation, GitHub repos, and implementation details. The research surfaces what’s actually true right now, not what was true when the model was trained.

I went through an evolution with this. I started with ChatGPT’s deep research feature via API, which is solid. Then I tried Perplexity, which is probably the best standalone deep research tool out there. But the friction of switching to a separate tool every time I needed to verify something was too high. So I built my own deep research skill that runs inside Claude Code, integrated directly into the workflow.

The broader principle is simple: the model’s knowledge is frozen at its training cutoff. If your task depends on what’s true right now, you need a research step baked in.

7. The $200 question

Let’s talk about cost, because nobody else seems to want to.

The $20/month Claude subscription is not feasible for serious, sustained use. It’s too limited. You can’t run heavy sessions, you hit rate limits almost immediately, you can’t do deep research for sure. It’s fine for casual use, but if you’re trying to build workflows and go all-in, it’s not enough.

The $200/month tier made a profound difference. The capability ceiling lifted. Sessions that would have been impossible at $20 became routine. The value, measured in time saved and quality of output, is clearly there. If you value your time at even minimum wage, the maths works out.

But here’s the thing. Heavy sessions with research easily reach $60. Sessions with multiple code reviews, fixes, security reviews, etc can easily come out at $100. That’s on a $200/month plan. The economic viability of this whole thing is definitely questionable. Not even questionable. It’s not a question that it is not viable for the average person at current prices. This is my spend just for the last 2 weeks (from ccusage | jq):

(Yes I hear you, “Don’t use Opus for everything you fool!)

I considered a multi-model strategy: $20 each for Gemini, ChatGPT, and Claude, roughly $60 total. Use each for its strengths. Gemini for research and image generation, Claude for coding, ChatGPT for writing and maybe PR review. In theory, this gets you decent coverage at a third of the cost, but it would most likely not cover all the work I do today. On top of that, the friction of switching between models and maintaining separate contexts and workflows makes it less appealing than it sounds.

Prices will come down. They always do. But we’re definitely not there today. If you’re considering going all-in on an AI tool for serious work, budget accordingly, and know that the per-session costs can be surprisingly high even on a premium plan.

What I’d tell someone starting out

The jagged frontier is real, but the jagged frontier is workable. The peaks are genuinely impressive and useful. The valleys are real and sometimes infuriating. The work is in the middle: figuring out where the edges are for your specific use cases, and building the patterns and refinements that compensate.

That accumulated refinement, the instructions, the workflows, the structural decisions, is the differentiator. Two people with the same tool and the same subscription will get very different results depending on how much they’ve invested in understanding the tool’s specific strengths and weaknesses for their work.

I’m still refining. The memory retrieval problem isn’t solved. The cost question looms. There are use cases I haven’t cracked yet. But two months in, I’m using Claude Code more, not less, and the output quality keeps climbing as the refinements accumulate.

If you’re experimenting with this or building your own patterns, I’d genuinely like to hear what’s working for you so feel free to find some time to chat. The terrain is different for everyone.