Build story · ·9 min read

I pointed autonomous AI agents at my own repos — they shipped verified PRs

By Dmitry · DigiWorks

Last week I stopped using AI agents as fancy autocomplete and started using them as teammates. Not the demo kind — the kind that opens a real pull request against a real repo, having already proven the change works. I pointed a set of autonomous workflows at my own web properties, told them to fix our SEO and AIEO gaps, and made one rule non-negotiable: an agent doesn't get to say "done." It has to prove it.

This is a write-up of what actually happened. Two properties, two separate repos, two workflows running in parallel. Two real PRs at the end — one on GitHub, one on our self-hosted Forgejo. Both green on every check. Both still sitting there waiting for me to click merge, because that part is still mine.

What I was actually fixing

Two flavors of the same problem: machines can't read the site properly.

SEO is the old one — Googlebot needs a valid sitemap.xml, a real robots.txt, canonical URLs, structured data. AIEO — AI-engine optimization — is the newer sibling: being crawlable and citable by GPTBot, ClaudeBot, and PerplexityBot so that when someone asks an assistant "who does X in my city," your pages are in the answer. Same core discipline: clean, honest, machine-legible metadata.

The two properties in scope were the DigiWorks agency site (a static single-page app served by nginx) and bodybar.studio (an Astro site for a client). Different stacks, same workflow shape.

The workflow topology

Each property got its own independent workflow, running as a pipeline. Because each is a separate repo, I ran them in parallel — no shared state, no cross-contamination.

per property (one repo each), run in parallel:

  ┌─────────┐   ┌──────┐   ┌──────────────┐   ┌──────────┐
  │  AUDIT  │ → │ FIX  │ → │  VERIFY GATE │ → │ OPEN PR  │
  └─────────┘   └──────┘   └──────────────┘   └──────────┘
       │            │              │                │
   find gaps    apply the    BUILD real output   only if
   (robots,     minimal      + assert against   ALL GREEN
   sitemap,     diff         it. red = stop.
   JSON-LD,                  no assertion,
   OG tags)                  no PR.

The interesting box is the third one. Everything before it is the kind of thing an LLM is already good at — read files, notice a missing tag, write a patch. The verify gate is what turns "an agent that edits files" into "an agent I'd let near my repo."

The verify gate is the whole game

Here is the failure mode I refuse to tolerate: an agent edits robots.txt, the file looks correct in the diff, the agent declares victory — and in production the SPA's catch-all fallback serves index.html for /robots.txt with a text/html content type. Googlebot asks for robots, gets a webpage, shrugs. The diff was perfect. The result was broken. A model that reads diffs would never catch this.

So for the agency site, the verify gate doesn't read files. It builds the actual nginx Docker container, runs it on a port, and curls it like a real crawler would:

# verify gate — agency static site (nginx)
docker build -t dw-verify . && docker run -d -p 8099:80 dw-verify

# robots.txt must be served as text/plain, NOT swallowed by SPA fallback
curl -sI localhost:8099/robots.txt | grep -qi 'content-type: text/plain' \
  || { echo 'FAIL: robots.txt not text/plain'; exit 1; }

# sitemap must be valid XML
curl -s localhost:8099/sitemap.xml | xmllint --noout - \
  || { echo 'FAIL: sitemap.xml not valid XML'; exit 1; }

# every JSON-LD block on the homepage must parse
curl -s localhost:8099/ \
  | grep -oP '(?<=application/ld\+json">).*?(?=</script>)' \
  | while read -r b; do echo "$b" | jq . >/dev/null \
      || { echo 'FAIL: JSON-LD does not parse'; exit 1; }; done

# canonical + OG + Twitter tags present
for tag in 'rel="canonical"' 'property="og:title"' 'name="twitter:card"'; do
  curl -s localhost:8099/ | grep -q "$tag" \
    || { echo "FAIL: missing $tag"; exit 1; }
done

echo 'ALL GREEN'  # ← only now is the agent allowed to open a PR

That ALL GREEN is the gate. No green, no PR. The agent physically cannot hand me a broken change and call it done, because the thing that authorizes the PR is a build-and-curl of the real artifact, not the agent's own optimism.

The Astro site had a lighter gate — it's a static build, so there's no container to spin up. There the gate ran npm run build and grepped the emitted dist/ for the assertions. Same principle, cheaper mechanics: assert against the built output, never against the source.

The honesty guardrail earned its keep

This is the part I did not expect and the part I'm proudest of.

On bodybar.studio, the audit agent found fabricated review markup baked into the page: schema.org/Review microdata plus a fake aggregateRating claiming 4.9 out of 5 from 500 reviews — a rating with no real data behind it. Somebody, at some point, had sprinkled star-rating structured data on the site to game rich results.

That's not an optimization. It's an integrity problem, and Google penalizes fabricated structured data when it catches it. My workflow had an explicit instruction: remove fake structured data, don't launder it. The agent stripped the Review microdata and the invented aggregateRating, and then — this is the good part — the verify gate asserted the removal:

# verify gate — bodybar (Astro), integrity assertion
npm run build

# the fabricated rating must NOT survive into the built output
! grep -rq 'aggregateRating' dist/ \
  || { echo 'FAIL: fabricated aggregateRating still present'; exit 1; }

! grep -rq 'schema.org/Review' dist/ \
  || { echo 'FAIL: fake Review markup still present'; exit 1; }

echo 'ALL GREEN'

The negative assertion (! grep) is doing real work here. It's not enough for the agent to say it removed the lie; the gate proves the lie is gone from what actually ships. An agent that fabricates data is a liability. An agent that refuses to ship someone else's fabrication, and proves it, is a colleague.

Self-correction: it rebased instead of clobbering

Here's the moment that told me these workflows had crossed a threshold. Partway through, the bodybar agent noticed that the repo's master branch had already independently fixed several of the items on its own to-do list — someone had done the work between my audit snapshot and the run.

A naive agent would have applied its full pre-computed patch and reverted those good changes back to the broken state — technically doing "what it was told," while destroying real improvements. This one didn't. It rebased its branch onto current master, recomputed the actual remaining gaps, and applied only what was genuinely still missing. It treated the existing good work as ground truth instead of an obstacle.

That's the difference between an automation and a teammate. An automation executes a plan. A teammate notices the world changed and adjusts.

The results

Both PRs passed every check. Neither agent touched a live server. The whole system is deliberately bounded: repo → PR → human merge. Agents build, agents verify, agents open the PR — and then they stop.

What still needed a human

I want to be precise about this, because the honest version is more useful than the hype version.

The lesson

An agent that says "done" is worthless. An agent that proves done against real, built output is a teammate. The whole difference between a toy and a tool is that third box in the pipeline — the verify gate that builds the actual artifact and asserts against it, including negative assertions that a bad thing is gone.

Everything else — the auditing, the fixing, the self-correction, the honesty about fabricated data — is enabled by the fact that the agent can't lie to the gate. And behind the gate, a human still merges. That's not a limitation I'm rushing to remove. That's the design.

Want agent workflows that prove their work before they ask you to trust it — instead of demos that fall over in production?

Let's talk →