There’s a gap that most engineering teams are starting to feel but haven’t quite named yet. You spin up an agent, give it a task, and it goes... somewhere. It reads files, checks documentation, wanders down paths that don’t pan out. It’s not incompetent, it just doesn’t know how things work here.
Dennis Pilarinos has been thinking about this problem for years. He’s the founder and CEO of Unblocked, which he describes as the context layer for modern engineering teams: a system that surfaces the tribal knowledge inside an organization and makes it available to both the people doing the work and the agents working alongside them. Before Unblocked, Dennis spent years at Microsoft on the early Azure team, then moved through mobile CI and developer tooling at a time when, as he puts it, investors were telling founders “there’s no money in DevTools.”
There’s money in DevTools now. But the more interesting question is whether teams are building the right things into them. Rob and Dennis went deep on that question: why agents fail in ways humans don’t, what happens to the pull request when agents are writing the code, and whether we’re finally close to being able to draw a line from “we ran this agent” to “we moved this metric.”
1. Every agent you spin up starts on day one. Every time.
The framing Dennis uses here is hard to shake: imagine joining a company and being asked to fix a bug. You’re reasonably competent. But you’re up against someone who has three to seven years of context on that codebase. Who gets it done better? Almost certainly not you, not yet.
That’s the situation for every agent, every session. It doesn’t accumulate experience across runs. It doesn’t remember the incident channel discussion from three months ago, or why that particular API pattern was chosen over the obvious alternative. Each session starts cold.
“When you spin up an agent, it is basically like me joining CircleCI on day one. It has no meaningful historical context for which to draw upon. And so it does what I would do. I would probably dig through a whole bunch of documentation. I would go down paths that don’t actually lead to productive outcomes.”
Dennis Pilarinos, Founder and CEO, Unblocked
The practical takeaway is this: if your agents are producing messy, roundabout solutions, the problem is likely the missing context. What the agent needs is often sitting in Slack, in a closed issue, in a Confluence doc nobody’s opened in 18 months.
2. The code alone is not enough context.
Dennis is direct about something most teams discover the hard way: indexing the codebase is a starting point, not a solution. The reasoning behind the code lives elsewhere.
“I have never ever met a company that’s like, our docs are super well organized and completely up to date. That just doesn’t exist.”
Dennis Pilarinos, Founder and CEO, Unblocked
When Unblocked pulls in Slack threads, bug trackers, incident channels, and documentation alongside source code, the quality of outcomes goes up significantly. This isn’t a surprise in retrospect. It’s exactly how a person learns a codebase. They read pull request comments. They get feedback on their first few contributions. They sit in on postmortems. The “scar tissue,” as Dennis calls it, is where the real knowledge lives.
Teams building context systems that rely only on source code are building the equivalent of onboarding a new engineer with read-only access to the repo and no other communication. It doesn’t work for people either.
3. AI code review is only as good as the context behind it.
Unblocked started using its own context layer to build an internal code review tool after surveying the existing market and finding it underwhelming. The tools they evaluated felt like early-stage engineers leaving noise in PRs, sometimes literally leaving haikus. Not useful.
When they built review on top of a context layer that knew the history of decisions, past incidents, and team-specific conventions, the results were different. The reviewer understood why certain patterns existed.
“It’s not just limited to the source code. It knows conversations you’ve had about incidents, or some of the best practices that you might have written down somewhere else that other tools might not necessarily know.”
Dennis Pilarinos, Founder and CEO, Unblocked
If you’re evaluating AI code review tools and finding them shallow, this is worth paying attention to. The limiting factor is usually not the model’s ability to read code. It’s whether the model understands the context in which that code was written. Most tools don’t have that. The ones that will matter will.
4. The pull request is two tools in a trench coat.
Rob and Dennis had one of the more useful framings of this conversation on the subject of pull requests. The PR solves two distinct problems: it’s an information-sharing mechanism (here’s what changed and why) and a change management mechanism (here’s how we control what goes to production). Those are different problems, and they’ve been bundled together for so long that most teams have stopped asking whether they need to be.
As agents generate more code, the PR review bottleneck gets real fast. Teams that Dennis talks to are saying their engineers are backed up on review, not on writing. The question becomes: can you decompose what the PR was doing and solve each part better?
“Go back to first principles. Pull that apart and say, what would be the way we would solve for this piece of it, for this specific problem in this new context? It doesn’t have to be one solution. It just happens to be now, because it was there.” Rob Zuber, CTO, CircleCI
The implication for engineering leaders: look at every gate in your delivery pipeline and ask what problem it was originally solving. The answer might still be worth solving. The mechanism might not be the right one anymore.
5. AI adoption inside one company spans an enormous range.
This was one of the more grounding moments in the conversation. Dennis described organizations where some engineers are just discovering tab-completion and others are running thousands of autonomous agents handling multi-hour implementation tasks. Same company. Same week.
He puts roughly three to five percent of engineering teams in the “AI pilled” category: people who have experienced enough of the technology to feel acutely where the gaps are. They’re the ones who find the lack of organizational context genuinely painful, because they’ve hit the ceiling. The rest of the org might not have reached that ceiling yet.
This matters for how you roll out tooling and expectations. Someone whose primary interaction with AI is autocomplete doesn’t have the same mental model of what’s possible, or of what’s broken. The journey isn’t linear, and it’s not uniform.
6. The ROI problem is closer to being solvable than it’s ever been.
This was the most forward-looking thread in the conversation, and both Rob and Dennis were careful about timelines. But the observation is worth sitting with: agents are instrumented in ways that people never were. You can actually see what it cost to get to a solution. The question is whether you can connect that to business outcomes.
“We can say: what did it take in order for you to come to this conclusion or to this solution? And can you do it in a token-efficient way? But what will happen next is: how do we reconcile that execution relative to business objectives? Not just how much did it cost, but how did it actually move the needle for the business?”
Dennis Pilarinos, Founder and CEO, Unblocked
Finance teams are noticing the spike in costs. The engineering side hasn’t yet built the case for what that cost produced. Closing that gap, drawing the through-line from token spend to business outcome, is the work in front of engineering leaders right now. Dennis thinks we’re closer than we’ve ever been. Rob’s take: we’ve taken one step on a very long journey, but it’s a real step.
Confident Commit is CircleCI’s publication for engineering leaders and practitioners. Subscribe to get episodes and companion posts delivered directly.
