5Evaluating AI output in under a minuteDetermine

The loop: prompt, evaluate, refine, repeat

8 min read1,401 words

The Core Cycle

If you take one idea from this entire course and apply it to every AI interaction, let it be this: productive AI work is iterative.

The Describe-Determine loop is simple:

  1. Describe what you need (your prompt)
  2. Receive the AI's response
  3. Determine if it's good enough (using the evaluation skills from this module)
  4. Refine your description based on what you learned
  5. Repeat until you reach the quality threshold your situation requires

This isn't a failure mode. This isn't "prompting badly and having to fix it." This is how the process works when it's working well. The first response gives you information — not just about your question, but about how the AI interpreted your question, what it prioritised, what it missed, and where it went wrong. That information makes your second prompt better. And the second response gives you more information still.

Each loop through the cycle isn't just getting you closer to a good output — it's teaching you about the AI's capabilities and tendencies, which makes every future interaction more effective.

Why Single-Prompt Perfection Is a Myth

There is a persistent belief that expert AI users craft perfect prompts that produce perfect responses on the first try. This is largely a fantasy. Here's why:

You often don't fully know what you want until you see a first attempt. You might know you want a "project plan," but seeing the AI's first draft helps you realise you actually want something more like a stakeholder communication with timeline elements. The first response clarifies your own thinking.

AI interprets ambiguity in ways you can't always predict. Even a well-crafted prompt contains ambiguous terms, and the AI will resolve that ambiguity based on its training data, not your intentions. You won't know how it resolved the ambiguity until you see the response.

Complex tasks have too many variables to specify upfront. You could write a 500-word prompt trying to specify every aspect of what you want, or you could write a 50-word prompt, see what comes back, and then give targeted feedback. The second approach is usually faster and produces better results.

The point is not that careful prompting doesn't matter — it absolutely does, and that's what Module 4 (Articulate) was about. The point is that even excellent prompting is the beginning of a process, not the entirety of one.

When to Iterate, When to Accept, When to Start Over

Not every response needs five rounds of refinement. Part of the Determine skill is knowing which path to take.

Iterate when the response is on the right track but needs adjustment. The structure is right but the tone is wrong. The analysis is solid but it's missing one key consideration. The content is good but the format needs work. In these cases, targeted feedback will improve the next version efficiently.

Accept when the response meets your quality threshold for the situation. And be realistic about what that threshold is. A first draft of internal meeting notes doesn't need the same polish as a client-facing proposal. Perfectionism in AI evaluation wastes the very time that AI was meant to save. If it's good enough for the purpose, it's good enough.

Start over when the response reveals a fundamental misunderstanding that targeted feedback won't fix. If the AI has completely misinterpreted your task, built its response on wrong assumptions, or gone in a direction so different from what you need that correcting it would take more effort than starting fresh — close the conversation, think about what went wrong, and begin again with a better initial prompt.

A Real Example: The Describe-Determine Loop in Action

Let's walk through a realistic example to see how the loop works in practice.

The task: You're a team leader who needs to write a difficult email to a colleague whose project deliverables have been consistently late, affecting your team's work.

Loop 1 — First attempt

Your prompt: "Write an email to a colleague about their late deliverables."

What comes back: A generic, overly formal email that reads like an HR warning letter. It uses phrases like "failure to meet obligations" and "expected standards of performance." It's technically about late deliverables, but it's completely wrong in tone — this is a peer, not a direct report, and you want to preserve the relationship.

Your Determine evaluation:

  • Accuracy: N/A (it's a drafted email, not factual claims)
  • Relevance: Partially — it addresses lateness but treats it as a disciplinary matter, not a peer conversation
  • Completeness: Missing any acknowledgement that the colleague might have valid reasons for the delays
  • Bias: Assumes the colleague is at fault without considering systemic issues
  • Usability: Cannot send this — it would damage the relationship

What you learned: You need to specify the relationship dynamic, the desired tone, and the goal of the email.

Loop 2 — Refined attempt

Your prompt: "Rewrite this. The colleague is a peer I have a good relationship with. The tone should be collaborative, not accusatory. My goal is to find a solution together, not to reprimand. Acknowledge they might have valid reasons for the delays."

What comes back: Much better. The tone is warm and collaborative. It opens by acknowledging shared pressures. It raises the lateness issue gently and suggests a conversation to find a way forward together. But it's now so soft that the actual impact on your team — missed deadlines, weekend work to compensate, a client complaint — isn't mentioned at all. It reads more like a casual check-in than a substantive conversation about a real problem.

Your Determine evaluation:

  • Accuracy: Fine
  • Relevance: Better — the relationship dynamic is right
  • Completeness: Now missing the business impact, which is the reason for writing the email in the first place
  • Bias: Has swung too far in the other direction — now seems to minimise a genuine problem
  • Usability: Too soft — wouldn't achieve the goal of actually addressing the issue

What you learned: You need both warmth and substance. The email should be kind but clear about the concrete impact.

Loop 3 — Targeted refinement

Your prompt: "Good tone, but you've lost the substance. Add a brief, specific mention of the impact: our team had to work over the weekend twice to cover the delays, and one client flagged a concern. Keep the collaborative tone but make sure the seriousness of the impact comes through. The email should make clear this needs to change while making clear you want to solve it together."

What comes back: Now you have an email that opens warmly, mentions the specific impacts without being accusatory ("I wanted to flag something because the knock-on effects have been building"), expresses genuine interest in understanding the colleague's situation, and closes with a clear ask for a conversation to find a better way forward. It's honest, respectful, and actionable.

Your Determine evaluation:

  • Accuracy: The specific details are ones you provided, so they're correct
  • Relevance: Exactly addresses your situation
  • Completeness: Covers relationship, impact, openness to their perspective, and a clear next step
  • Bias: Balanced — neither accusatory nor dismissive of the problem
  • Usability: You could send this with minor personalisation tweaks

What the three loops achieved: Each round didn't just improve the email — it helped you articulate what you actually needed. The first response showed you that tone mattered more than you'd initially specified. The second showed you that tone without substance wasn't enough either. By the third round, you'd essentially taught the AI (and clarified for yourself) the precise balance you were looking for.

This is the Describe-Determine loop working as intended. Three prompts, perhaps five minutes total, and a result that would have taken you twenty minutes of agonising over wording if you'd written it from scratch.

Key Takeaways

  • 1Productive AI work is iterative — the Describe-Determine loop of prompting, evaluating, and refining is how the process works when it is working well.
  • 2Single-prompt perfection is a myth; even expert users expect to iterate because complex tasks have too many variables to specify upfront.
  • 3Each iteration teaches you about the AI's capabilities and clarifies your own thinking about what you actually need.
  • 4Know when to iterate (close but needs adjustment), accept (good enough for the purpose), or start over (fundamentally off track).