Human-in-the-loop QC

The iteration loop is moving off your laptop.

QC Hub is the infrastructure that closes it — with humans in the loop where it counts.

The dev loop is moving

Where software gets written, tested, and iterated has shifted. Understanding where it is headed explains what QC Hub is for.

Then — iteration on the laptop

A developer prompted an LLM, ran the code locally, and looped until it worked. GitHub was the handoff. QC saw a finished build.

sequenceDiagram participant QC participant Github participant LLM participant Dev Github->>Dev: Issue loop Tight iteration on local machine Dev->>LLM: Some prompt LLM->>Dev: Code Dev->>LLM: Feedback + test result end Dev->>Github: Pull Request Github->>QC: Test Build QC->>Github: Pass / Fail

Now — iteration on deployed builds

Modern agentic workflows iterate against real deployments. Each cycle is a real build, so cycles are slower — and the developer is still the tester in the inner loop.

sequenceDiagram participant QC participant Github participant LLM participant Dev Dev->>LLM: Some prompt Github->>LLM: Issue loop Iteration on deployed test builds LLM->>Github: Pull Request Github->>Dev: Test Build Dev->>Github: Feedback + test result end Github->>QC: Test Build QC->>Github: Pass / Fail

Next — autonomous iteration, bounded by heuristics

The missing piece is a system that closes the feedback cycle without a human in the inner loop. The developer steps in only for architectural direction changes.

sequenceDiagram participant QC participant QCHub as QC Hub participant Github participant LLM participant Dev Dev->>Github: Architectural direction changes Github->>LLM: Issue loop Autonomous cycle — bounded by heuristics (max loops, etc.) LLM->>Github: Pull Request Github->>QCHub: Test Build QCHub->>QC: Test Build QC->>QCHub: Pass / Fail report + application logs Github->>LLM: Pass / Fail report + application logs + aggregated analysis end

How QC Hub closes the loop

QC Hub sits between GitHub and the testers. It is the piece that makes the autonomous loop work.

When a PR opens, QC Hub receives the test build, dispatches a structured review to human testers, and aggregates three things back: pass/fail per checklist item, application logs from the preview environment, and analysis synthesized across testers. That bundle becomes the feedback signal the agent iterates against.

Developers own architectural direction. Testers own quality verification — the perceived-quality judgments that no automated system can make. Everything in between is a candidate for automation.

Loops-per-task — the health metric

Once the loop runs itself, a new signal emerges: how many cycles the LLM needed to complete a task. A clean task lands in 1–2 loops. A task that needs 5, 8, 15 loops is telling you something — usually that the structure of the code around it is working against the agent. Loops-per-task makes structural debt visible as it accumulates, not months later when velocity has already stalled.

Principles

The loop is the product. QC Hub's job is to let one cycle complete — and then the next.
Humans close what automation cannot. Perceived quality and architectural direction are human calls.
Measure what you close. Every cycle generates data. Loops-per-task is a leading indicator of codebase health.
Blind, independent review produces honest signals. Testers do not see each other's work.
Ship the simple loop first. A checklist with pass/fail running end-to-end is more valuable than a sophisticated system shipped never.