How a Self-Critique Loop Doubled Our AI Editing Quality

We doubled the quality of our AI editing pipeline by adding one thing.

A critique loop.

At Threadline Studio, we build AI tools for professional video production. Our pipeline reads transcripts and assembles a first cut. For months it was single-pass. One prompt, one output, done.

Results were always fine. Never great.

So we added a second step: the model reviews its own edit like a producer would, then revises based on its own notes. Two to three rounds, then output.

But the real insight was not adding the loop. It was how we structured the feedback.

Why Numerical Scoring Failed

Our first version used numerical scores. "Pacing: 7/10. Narrative: 6/10." The model would look at these and make vague improvements. A 6 in pacing does not tell you what to fix.

Then we thought about how actual producers give notes.

No producer says "pacing is a 6." They say "the opening drags and we lose Sarah's thread after the second act." Specific. Actionable. Pointed.

We switched the critique to generate qualitative notes. Which clips run too long. Where the narrative loses focus. Whether the ending resolves or just stops.

The quality jump was immediate. The revision pass finally had real instructions to work with.

What We Learned

Self-critique is surprisingly cheap. One extra LLM call adds maybe 30 seconds and a few cents. The ROI is enormous.

Qualitative feedback dramatically outperforms numerical scoring. "The opening drags" beats "opening: 6/10" every time.

Three iterations is the ceiling. After that, changes go lateral, not upward.

If you are shipping first-pass LLM output in production, try adding a critique loop. It is the highest-leverage improvement we have made to our pipeline.

First drafts find the story. Second drafts tell it.