Open navigationClose navigation

Discover · Design · Develop · Deliver

Useful answers for building with AI.

How I approach the questions that arise across AI-first product engineering — from choosing the right problem to operating a useful, trusted capability.

These are working principles, not universal prescriptions. The right implementation depends on the customer, consequence, evidence and operating context.

Showing 18 of 18 questions

01

Discover · Find the leverage

Start with work worth changing.

Frame the outcome, the evidence and the boundaries before deciding what to build.

01

What does AI-first product engineering mean to you?

It means redesigning the whole product-engineering system around what AI now makes possible — not simply adding an AI coding tool to the development step.

I connect discovery, design, development, delivery and operation through shared evidence and fast feedback. A working prototype becomes an evolving product foundation: useful enough to test early, then progressively connected to production data, services and controls. The goal is not more code. It is better customer outcomes, shorter learning cycles and a system the team can safely own.

02

How do you use coding agents without losing engineering discipline?

I give agents bounded autonomy inside an engineering loop with explicit context, acceptance criteria and verification gates.

I frame the problem, define the smallest coherent slice and make architecture and constraints visible before asking an agent to build. The agent can inspect, plan, implement and test, but its output remains a proposal until the evidence is strong enough. I review the decisions that carry consequence: interfaces, data, security, failure behaviour, maintainability and the customer experience. Agents increase capacity; they do not remove accountability.

03

When do you use one agent, and when do you use several?

I parallelise only when the work has clean boundaries; otherwise coordination costs more than it saves.

A single agent working sequentially is often best while the architecture or problem is still changing. Multiple specialised agents help when responsibilities, interfaces, shared state and acceptance criteria are clear — for example, implementation, test design and security review on separable surfaces. Their work is integrated frequently against one architecture. Parallel output without integration discipline only creates faster divergence.

04

How do you decide what to build, buy or compose?

Build what differentiates the business; compose or buy what does not.

I look at strategic distinctiveness, required control, integration cost, speed, capability and switching cost. Models and commodity services should usually sit behind clear boundaries so they can change. The organisation should retain the context, workflow knowledge, customer decisions and evaluation assets that make the capability uniquely valuable. A vendor choice is an architectural dependency, not the strategy itself.

02

Design · Shape the system

Design for uncertainty.

Make trust, human judgement and proportionate control part of the experience from the beginning.

05

How do you deal with hallucinations?

I treat a model response as an untrusted proposal and control the consequence of being wrong.

First, I reduce the opportunity to invent: ground the task in authoritative sources, constrain the output, expose provenance and use tools for facts or calculations. Then I verify what matters through deterministic checks, independent review, evaluations or human approval. Finally, I design the experience so uncertainty is visible and failure is recoverable. The control should match the consequence: a draft product description and a safety-critical decision should never share the same autonomy.

06

How do you evaluate an AI system when outputs are non-deterministic?

I evaluate the behaviour that matters to the workflow, not whether the model reproduces one ideal sentence.

I build a representative set of normal, difficult and adversarial cases from real work, then score task success, factual grounding, policy compliance, tool use and escalation behaviour. Some checks are deterministic, some use structured human judgement and some can use a separate model judge that has itself been calibrated. I track the set over time so prompt, model or data changes cannot quietly weaken the system.

07

How do you test systems built with coding agents?

The speed of generation increases the need for layered, independent verification.

I use the usual engineering controls — types, static analysis, unit, integration, contract, end-to-end, performance and security tests — plus evaluation suites for model behaviour. I ask an agent to prove acceptance criteria, not merely report that it has finished. High-consequence changes receive independent review, and deliberately failing safety cases confirm that release gates actually block. Generated code earns trust the same way human-written code does: through evidence.

08

How do you approach security, privacy and prompt injection?

I assume model input and retrieved content can be hostile, and I keep authority outside the model.

Identity, permissions and policy are enforced by trusted services, with least-privilege tools, scoped credentials, data classification and auditable actions. Untrusted content is separated from instructions. Sensitive data is minimised and protected through its lifecycle. High-impact actions require validation or approval. I also threat-model the agent-specific paths — prompt injection, data exfiltration, excessive agency and poisoned context — and test them continuously.

03

Develop · Build the capability

Turn intent into working evidence.

Compose models, context, tools and controls into a coherent system that can be tested.

09

How do you manage latency in an agentic experience?

I manage both elapsed time and perceived time, starting with the service level the user actually needs.

I remove unnecessary model calls, run independent work in parallel, choose smaller models for simpler steps, cache stable results and stream useful progress. The experience should show what is happening and let people continue when possible. Longer reasoning belongs in asynchronous workflows with clear status and recovery. I measure latency by step so optimisation targets the real bottleneck rather than the most visible technology.

10

How do you control cost without compromising quality?

I route each task to the least expensive approach that can meet its quality and risk threshold.

That might be deterministic software, retrieval, a small fast model or a more capable reasoning model. I control context size, reuse stable outputs, limit tool loops and set budgets for tokens, time and retries. Unit cost only matters beside task success and business value, so I observe cost per successful outcome — not simply cost per model call.

11

How do you choose models and avoid vendor lock-in?

I choose models against evaluated tasks and isolate provider-specific behaviour behind purposeful interfaces.

Capability, latency, cost, privacy, availability and tool use all matter. I prefer a small routing portfolio over abstracting every difference into a lowest common denominator. Prompts, schemas, evaluation sets and business logic remain owned assets; adapters contain provider details. Portability has a cost, so I invest in it where dependency risk justifies it rather than pretending every component is interchangeable.

12

How do you give an agent useful context and memory?

Context is a governed product, not a pile of documents placed into a prompt.

I separate authoritative knowledge, current task state, durable memory and conversation history. Retrieval should respect identity and permissions, preserve provenance, favour relevant current evidence and fit within an intentional context budget. Structured data and knowledge graphs help when relationships matter. Memory is written selectively, with ownership, expiry and correction paths, so yesterday's inference does not silently become tomorrow's fact.

13

What do you observe in production?

I observe the whole decision path: inputs, context, model and tool behaviour, outcome, cost and human intervention.

Traces should make it possible to understand why the system acted, while protecting sensitive content. I monitor technical health, evaluation drift, task success, latency, cost, escalation, override and customer impact. Versioned prompts, models, tools and knowledge sources make changes explainable. Observability is not only for incidents; it is the learning loop that tells the team what to improve next.

14

What happens when a model or tool fails?

Failure is a designed state, with a safe boundary, a useful fallback and a clear recovery path.

Calls have timeouts, bounded retries and idempotency where actions may repeat. The system can fall back to another model, a simpler deterministic path, a saved draft or a person — depending on consequence. Partial work is preserved and external actions are reconciled. A graceful failure that protects trust is better than a plausible answer the system cannot support.

04

Deliver · Embed and improve

Make capability last.

Put the system into real work, observe its outcomes and leave the team able to carry it forward.

15

How do you move from an AI prototype to production?

I treat the prototype as evidence and a potential product foundation — not as proof that the hard work is finished.

Before scaling, I confirm the customer outcome, owner, risk tier and operating model. Then I progressively replace assumptions with production data, identity, permissions, integrations, evaluations, resilience and support. Investment is staged against evidence, with explicit stop criteria. The shortest path to production is usually a narrow end-to-end slice with real controls, not a broad demonstration that must later be rebuilt.

16

How do you build trust and adoption?

Trust grows when people help shape the system, understand its limits and retain meaningful agency.

I begin with the people who know the work: they define exceptions, quality and what must never happen. We make the agent's role, sources, uncertainty and escalation visible, then introduce autonomy progressively as evidence grows. Feedback must lead to observable improvement. Training helps, but adoption is primarily a product and work-design problem: the capability has to make the job genuinely better.

17

How do you measure whether an AI initiative is successful?

I connect model behaviour to workflow performance, customer outcome and business value.

The measures depend on the reason for changing the work: time to outcome, quality, conversion, avoidable effort, risk, satisfaction or a new revenue opportunity. I establish a baseline and watch for displaced effort and unintended consequences. Adoption and technical accuracy are leading indicators, not the finish line. A system is successful when the outcome improves sustainably and the organisation can operate it.

18

What changes for engineering teams when code becomes cheaper?

Judgement, architecture, integration and accountability become more valuable — not less.

Teams can explore more possibilities and create working evidence earlier, but generated volume can overwhelm review and operations. Strong engineers increasingly shape intent, decompose systems, define constraints, design tests, integrate continuously and learn from production. Leaders should reward outcomes and capability growth rather than lines of code. The opportunity is to remove handoff friction across disciplines, not automate one silo and accelerate its queue.

The short version

01 · DISCOVER

Start with curiosity.

Build a shared understanding of the people, work, evidence and constraints — then define the problem worth solving and what has become possible.

02 · DESIGN

Bound autonomy.

Match freedom and control to the evidence available and the consequence of error.

03 · DEVELOP

Verify the result.

Treat every output as a proposal until proportionate checks make it trustworthy.

04 · DELIVER

Learn in production.

Observe outcomes, improve the system and turn useful work into reusable capability.

A living FAQ

The questions will keep changing.
So will the answers.

AI capability, practice and consequences are evolving together. This page is a working record — revisiting familiar questions, asking better ones and updating the answers as evidence grows.

What are you curious about?

Bring the question you keep returning to, the work that could be better or the opportunity you can almost see. We can build a shared understanding of what is worth solving — and what AI might make possible.

Get in touch

Talk with Martyn

Choose the way that works best for you.

Call MartynReveal the phone number

Prefer email? Reveal the email address