Can a small app turn a typed feedback note into a shipped change, with no human in the loop, safely enough to leave on?

Hypothesis

For a small app with a handful of known users, most feedback is a one-line change: a label, a colour, a button size. The cost of getting that into production is mostly waiting and context switching, not difficulty. If a coding agent can take a feedback note all the way to a deploy, behind strict gates and a spending cap, the feedback loop could run in the background.

What we built

• In-app feedback. A feedback field, a status list and a detail page inside the app, so users see what happened to their note.
• The loop. A note becomes a GitHub issue labelled feedback. A workflow runs the claude-code-action agent, which opens a pull request on its own branch. A second workflow merges it when a guard check and the build both pass, closes the issue and dispatches the deploy.
• Gates. The agent may never edit the .github folder, where its own rules live. A guard job fails any pull request that touches it. Humans change the robot's behaviour through a normal pull request.
• Failure path. A red gate produces one comment per commit and marks the issue needs-human. Replying to the robot on the pull request asks it to fix the failing check, and a green push then merges by itself.
• A sweep. An hourly job re-gates open robot pull requests and closes issues whose pull request merged.
• Spend limits. A dedicated API key with a 30 USD monthly limit, stored only as a repository secret, a 2 USD cap and 60 turn cap per run. Measured cost so far is about 0.59 USD per run.
• Off switches. One repository variable stops new feedback runs, another stops merging and the sweep.
• A kill rule. A scheduled workflow opens a "trial result" issue after 10 robot pull requests or on 2026-11-12, whichever comes first, to decide whether the robot stays.

Learnings

• Small changes did ship. Over four days the app received a stream of small merged changes, from button sizes and wording to a search box on one screen. We have not yet counted how many came from the robot versus by hand.
• The robot wakes on any owner comment. Commenting a status note on a needs-human issue, from the same account that owns the repo, re-ran the robot. We changed it deliberately so any owner comment reruns it, and now avoid posting status notes there.
• Bot comments do not trigger workflows. Comments made with the default workflow token do not start other workflows, so a red pull request has no automatic retry. It would need a separate token or app.
• Review tools rate limit quickly. The hosted code reviewer rate limits fast on a busy small repo, so one review per pull request is configured and an independent subagent review is the fallback.
• Open question. Whether leaving a model free to change a live app is a good trade is the thing the trial exists to answer. We do not yet have a count of regressions or reverted robot changes, so we make no claim either way.

Log

• 2026-10-17: Any owner comment on a feedback plus needs-human issue reruns the robot. Automerge also checks needs-human on the linked issue.
• 2026-10-15: Fully hands-off: automerge, issue close and deploy dispatch chained.
• 2026-10-13: First version live: feedback issue to robot pull request.