← Back to shipped cases

02 / SHIPPED CASE

RelayOps: context you can trace, and uncertainty you can see

It reads documentation, finds code, and checks bounded live state. When evidence supports an answer, it cites the source. When it does not, it says what remains unknown.

PUBLIC CASE STUDY / The index figures are a 2026-07-17 snapshot. They describe system scale, not answer quality, adoption, or time saved. Repositories, paths, services, and live environments are generalised.

1,678KNOWLEDGE CHUNKS
657INDEXED SOURCE FILES
84AUTOMATED TESTS
30REAL CONVERSATIONS USED FOR PRODUCT LEARNING
ROLEProduct architecture · retrieval · agent · governance
TIME2026.06 — 2026.07
COLLABORATIONPlatform engineers · repository owners
SCOPERead-only repositories + bounded live investigation

THE PROBLEM

Engineering knowledge is not a search result. It is context that can be defended.

Real questions cross repositories, architecture notes, ownership, and current infrastructure state. Vector search alone misses exact paths and fresh runtime signals.

RelayOps starts with reproducible prefetch, allows a bounded read-only investigation, then applies citation checks, confidence caps, secret gates, and abstention.

SANITISED WALKTHROUGH

One engineering question, from retrieval to a visible unknown.

This example follows evaluation and QA-history fields. Internal repository, path, and service identifiers are removed.

QUESTION / REDACTEDCITED · PARTIAL
01 / INPUT“How does incident approval work?”

natural language, typo and follow-up tolerant

02 / RETRIEVEBM25 prefetch finds policy and approval documents

a reproducible candidate set

03 / INVESTIGATERead-only tools check current configuration and ownership

bounded budget; no mutation capability

04 / VERIFYEvery claim must map to a source seen in this run

invalid citations are removed; weak evidence triggers a cap or abstention

05 / ANSWERExplain the flow, cite it, and expose the exception

environment-specific approver mapping remains unknown

SANITISED ANSWER

Approval occurs after policy validation and before execution. The request includes the proposed action, scope, and rollback plan. Publicly available evidence does not establish the approver mapping for a specific environment, so the service owner must verify that detail.

SOURCES · approval-flow.md · policy-gates.md · live snapshot (read-only)

SAFETY MODEL

“Read-only” is a capability boundary, not a sentence in the prompt.

READ-ONLY

No mutating tools

Least-privilege infrastructure access sets a hard ceiling on what the agent can do.

SECRET GATE

Redaction at several layers

High-risk resources and sensitive tool output are filtered before the final answer.

CITATIONS

Claims must remain traceable

The verifier checks that cited evidence appeared in the current investigation.

ABSTAIN

“I do not know” is a feature

Conflicting, stale, or absent evidence becomes an explicit unknown and safe next step.

WHAT THE EVIDENCE SUPPORTS

Traceability is measurable. User impact still needs a publishable baseline.

0.90

Citation validity in the same controlled evaluation set.

0

False confidence for the hybrid architecture in adversarial cases.

≈4.3

Average tool calls, compared with approximately 8.4 for open exploration.

OPEN

Time saved, answer adoption, and correct-abstention rate are not publicly claimed yet.

REQUEST A WALKTHROUGH

Want to see citations, abstention, and the audit trail working together?

Email Ian ↗

RELATED SHIPPED CASE

See the same evidence boundary applied to incident remediation.

Read AI-SRE ↗