Start with an operational event
“Detect a person” is a model task. “Recognise a person remaining inside a hazardous zone while equipment is operating” is an operational scenario. Write the scenario in terms of the source, conditions, duration, evidence and intended response.
Agree who will review the event, how quickly it should arrive and which action it supports. These decisions shape camera placement, scenario rules and acceptance criteria.
Build a representative test set
Include the different lighting, viewpoints, motion, occlusions and working conditions that the deployment is expected to encounter. Positive examples alone cannot establish whether the system is usable. Add situations that resemble an incident but should not trigger an event.
- Keep a controlled reference set with reviewed labels.
- Separate development examples from acceptance examples.
- Record the source, model, scenario and parameter versions for each run.
- Review ambiguous episodes and agree the intended outcome.
Measure the complete result
Count missed events and false events against the reviewed reference. Track repeated notifications, processing delay and resource use on the intended hardware. Operator confirmation rates are useful operational signals, but they are not a substitute for precision and recall measured on a labelled set.
Test changes as changes
A new threshold or zone can alter behaviour even when model weights stay the same. Compare candidate versions on the same reference material, document the outcome and prepare a rollback path.
Agree the handover
Define the accepted operating conditions, known limitations, event ownership and maintenance responsibilities. A useful pilot produces a repeatable operating procedure as well as a technical result.
Read the technical guide to assets and test runs.