5Evaluating AI output in under a minuteDetermine

Exercise: Spot the errors a novice would miss

4 min read764 words

Time required: 20 minutes

This exercise is designed to make the relationship between expertise and evaluation viscerally obvious — not as a theory, but as something you experience directly.

Step 1Part A: Evaluate in Your Area of Expertise

Choose a topic you know deeply — your professional field, a subject you studied extensively, a skill you've practised for years.

Ask an AI the following (adapting to your domain):

"Explain the key principles of [your area of expertise] to someone who needs to apply them in practice. Include common mistakes people make and how to avoid them. Be specific and detailed."

What topic did you choose for your area of expertise?

0 wordsSaved

Now evaluate the response using all five quality checks:

  1. Accuracy — Read every claim carefully. Are the facts correct? Are the technical terms used properly? Are there subtle errors that a non-expert wouldn't notice?
  2. Relevance — Does this address practical application, as requested? Or has the AI drifted into theoretical overview?
  3. Completeness — What's missing? What would you add if a colleague showed you this? What crucial nuance has been flattened?
  4. Bias — Does it favour one school of thought, one methodology, one approach? Does it present debatable opinions as consensus?
  5. Usability — Could someone genuinely learn from and apply this? Or would following this advice lead them astray in important ways?

Record your findings below. Be specific — note exact phrases that are wrong, misleading, or incomplete.

Quality CheckRating (1–5)Specific Issues Found
Accuracy
Relevance
Completeness
Bias
Usability
0 of 10 cells filledSaved

Describe your most significant findings in detail. What errors, omissions, or misleading statements did you spot that a non-expert would likely miss?

0 wordsSaved
1 / 3