Planning, Specs, and Decision Logs: What Separates AI Engineers From Vibe Coders
In this lesson: the last mile separates a prototype from production, a spec is a contract with your coding agent, and decision logs keep your agents in check.
Top 3 takeaways
The last mile separates a prototype from production
AI can one-shot the first 70 percent of a build, and the last 30 percent is the edge cases, the error handling, the security, and the architecture. That last mile is where your focus belongs, and handling it well is what moves you from vibe coder to AI engineer.
A spec is a contract with your coding agent
A spec is a markdown file that names the exact outcome, the scope and files in play, the constraints, and the acceptance criteria. Any engineer can hand the same spec to any coding agent and get consistent output, which is why specs become real IP for a team.
Decision logs keep your agents in check
Decision logs, also called ADRs, are the non-negotiables your team has agreed on, such as the database you always use or the rules the front end must follow. Loaded into the context window, they stop an agent from going off the rails, and they are the first step toward building a harness.
Ash Tilawat
CTO, Gauntlet AI
CTO of Gauntlet AI, leading the company's technical direction and AI-native training programs. Has trained 1,200+ engineers across 104 companies and run multiple corporate trainings this year — including the AI sales course that firms like a16z, Mainsail, and PwC brought Gauntlet in to teach. Focused on turning AI from a prototype tool into something teams use in real production workflows, with an emphasis on evaluation and systems thinking.
Lesson notes
A written walkthrough of the lecture, covering the patterns, the practice, and the things that trip people up.
Vibe Coding Is a Starting Point
Ash Tilawat argues that vibe coding is a valuable skill. Learning to communicate effectively with AI benefits everyone—from engineering to marketing, sales, and product—and it's an excellent way to prototype ideas quickly.
AI engineering goes further. It requires ownership of architecture, planning, tradeoffs, testing, and production systems. The role shifts from writing code to directing AI, making decisions, and ensuring the final product is reliable.
Planning Is the New Bottleneck
Giving a team access to Claude Code or Cursor doesn't automatically make them faster.
The real bottleneck is planning. AI can generate code quickly, but it still needs clear specifications, well-defined requirements, and acceptance criteria. As organizations adopt AI, they often find they have more implementation capacity than well-scoped work ready to build.
The second bottleneck is code review. Even when AI writes the code, engineers remain responsible for validating it before it reaches production.
Spec-Driven Development
As AI takes over more implementation work, software development becomes increasingly driven by specifications rather than tickets.
A specification acts as a contract between the engineer and the coding agent. It clearly defines the objective, scope, constraints, files to modify, and acceptance criteria, allowing different engineers—or different agents—to consistently produce the same outcome.
The emphasis shifts from how to build something to what should be built and why.
What Makes a Good Specification
A strong specification is written in plain language and gives both humans and AI a shared understanding of the task.
Effective specs include:
- The desired outcome.
- Scope and boundaries.
- Constraints.
- Acceptance criteria.
- A task breakdown.
- Guidance for what to do when something fails.
Clear specifications reduce ambiguity, improve consistency, and produce better AI-generated code.
Decision Logs Preserve Engineering Judgment
Decision logs (ADRs) capture important architectural decisions that should remain consistent over time.
Rather than relying on tribal knowledge, teams document choices around technologies, infrastructure, and engineering standards so both people and AI agents understand the reasoning behind them. These records become long-term context that helps agents make decisions aligned with the team's established practices.
Harnesses Create Reliable AI Systems
A harness is the collection of tools, memory, context, and evaluations that guides an AI agent's behavior.
The same underlying model can perform very different roles depending on the harness surrounding it. At the individual level, a harness shapes how a single agent works. At the organizational level, it becomes the operating system for an entire software factory.
The model provides intelligence. The harness provides consistency.
AI Engineering Is an Iterative Loop
Modern AI engineering follows a repeatable cycle.
Intent becomes product requirements and architecture. Those plans become specifications. Coding agents generate implementations, evaluations verify the results, and the process repeats until the software meets its acceptance criteria.
The lecture's central message is that great AI engineers don't spend their time writing prompts. They spend it designing systems—creating plans, specifications, decision logs, and evaluation loops that allow AI to produce reliable software at scale.
FAQ
What separates an AI engineer from a vibe coder? +
What is spec-driven development? +
What goes inside a spec? +
What is a decision log or ADR, and why use one? +
What is a harness in AI engineering? +
Why is planning the bottleneck now? +
Do you write specs by hand or with AI? +
What's next?
Keep building with the rest of Night School, or apply to Gauntlet — twelve weeks of technical intensity with the best AI engineers we can find.