Chapter 4: Monitoring

About monitoring

Monitoring is about evaluating and improving the work you do in collaboration with AI. This is where you truly act as the critical human in the process: you assess quality, give feedback, and ensure that the result meets your professional standards.

It requires healthy scepticism, clear quality criteria, and strong professional judgement. In the end, you are the one who stands behind the quality of the work. Everything a chatbot helps you produce must, at a minimum, match what you yourself would be able to stand by – and preferably be better.

That is why it is important that you can evaluate the work and give constructive feedback to your chatbot along the way.

Monitoring is therefore about being able to:

• Assess the quality of the chatbot’s output
• Evaluate the process the chatbot went through to produce the output and improve it
• Reflect on whether you recognise yourself in the product of the collaboration and whether it lives up to your professional standards

Remember: working with AI is not an excuse to cut corners or lower your standards, but a good reason to make an even greater effort.

The three P’s

When you monitor your collaboration with the chatbot, you can once again keep the three P’s in mind:

1. Product: Does the product or result of the collaboration meet my quality criteria? Is the content correct? How much is still missing before the task is solved satisfactorily?
2. Process: Did the chatbot approach the task in the way you wanted? Did it follow the instructions? Can the process be improved with either fewer or more steps?
3. Preferences: Did the chatbot act in a desirable way, or was there something in its “manner” that annoyed you? Did you like the interaction at all?

You can use these questions to continuously evaluate your collaboration with your artificial assistant – and to tell it what improvements you want.

Improvement-oriented feedback

People learn from feedback. The same applies when you want to improve your collaboration with a chatbot. When you give feedback to your chatbot, you can follow this recipe for constructive feedback:

1. Point out the problem: What is wrong?
2. Explain why it is a problem – and what you want instead
3. Give your own suggestions for improvements
4. Highlight what the chatbot did well, so it can learn your preferences for next time

Ask the chatbot to update its memory if there is something you want it to remember the next time you work together.

Source criticism

Good old-fashioned source criticism never goes out of style. Always be sceptical of the answers you get from a chatbot, and check whether its claims hold up.

As mentioned earlier, a chatbot can easily invent an answer or a reference that does not exist in reality. It can also base its answer on sources that are unsound – which is why it is important that you look up the sources it refers to yourself.

Source criticism is more important than ever before!

Worth considering: Percentages rather than finished products

When you get assistance from chatbots in your work, you may be inclined to think in terms of finished products: ”Did the chatbot solve the task fully and satisfactorily?”

Often, the answer will be no.

Instead of assessing the collaboration in the dichotomy of ”finished or failure”, you are often better off thinking in terms of percentage completion of the task: What percentage of the task has the chatbot solved? Perhaps it has solved 80% of the task, so you only need to do the last 20 yourself. Perhaps it has only helped you get started on the first 10% – but a little also counts.

The machine helps, but the human completes the task. And takes responsibility for the quality.

EXERCISE: Polish your critical glasses

Here is an exercise to keep your critical eye sharp when you work with chatbots:

1. Ask Copilot (or another chatbot) a factual question within an area you know well. Ask for specific information with references.
2. Check the claims: Do the sources, articles, or names mentioned actually exist – and do they say what the chatbot claims? Are the figures, years, or definitions correct? Is anything left out that should have been included?
3. Consider what you found out: What was right, wrong, or difficult to verify? Were the errors obvious or hard to find? What should you pay attention to in the future when you seek knowledge through a chatbot?

Conclusion

You now know how to improve your collaboration with the chatbot – and why critical thinking and good old-fashioned source criticism are more important than ever before.

In the next chapter you will learn how to use AI safely at SDU – without running into trouble.

See you on the other, no… safe side!

Rules of thumb for mastering monitoring

1. Do not settle for the first and ”best” answer (and take responsibility for quality)
2. Give the chatbot constructive feedback on both product and process – and tell it about your specific preferences
3. Good, old-fashioned source criticism never, ever goes out of style