Remember the classic meme? A dog is lying behind a fence. The shadows of the bars fall across its body. A computer-vision algorithm confidently draws a box and announces: “Tiger. Accuracy: 100%.”
At that point, every human in the room starts laughing. It is obviously a dog. The stripes are shadows, not part of the animal. The fence is a separate object. The scene is outdoors. And tigers rarely wait for lunch outside an office building.
For older computer vision, it was not quite so obvious.
Why the mistakes were so funny
For a long time, classical computer vision lived in a world of separate objects. Here is an object. Here is a box. Here is a class. Here is a confidence score.
The system examines a piece of the image, finds a visual pattern and tries to match it to something it knows. Stripes? Tiger-like. Body shape? Well, perhaps. Context? What context?
That is how a model could be confidently wrong about something a person would barely need to think about. It saw features without understanding the scene.
In a customer’s real application, that difference matters enormously.
Detecting a helmet is not the same as understanding a violation
A conventional detector can find a helmet, a person, a cigarette or a mobile phone. The business question is rarely that simple.
- Is the person actually inside the hazardous zone, or passing behind a barrier?
- Is a worker breaking a rule or carrying out a normal operation?
- Is a vehicle dangerously close to a person, or are they in separate areas?
- Is the employee using a phone where it is prohibited, or holding an authorised work terminal?
- Is a tree inside the construction site, or outside its protective fence?
Those are questions about relationships, spatial logic, operating rules and context. That is where things become interesting.
From recognition to interpretation
Modern vision-language models — VLMs — change the questions we can ask of an image.
Classical computer vision asks, “What is in the picture?” A VLM can address questions such as: “What is happening here?”, “Is there a violation?”, “Why might this be dangerous?”, “How are these objects related?” and “Does this scene comply with the given rule?”
Take the tree and fence. A detector reports: “Tree detected. Fence detected.” A context-aware analysis can describe the relationship: “The tree is outside the construction site’s protective fence.”
In the real world, an object’s presence is only part of the answer. Its position, role and relationship to other objects may be what matters.
What changes for customers
The main change is not simply that AI has become “smarter”. The way we describe a task is moving closer to the way people actually think about their work.
Creating a new detector used to be a separate project: collect data, label images, train a model, evaluate it, revise it — and repeat. That could take weeks or months.
Some tasks can now begin with a rule in natural language:
- Determine whether a person in a hazardous zone is missing a protective helmet.
- Check whether an employee crosses the red line beside operating machinery.
- Find instances of a driver becoming distracted from the road.
- Determine whether an unauthorised person is at a dispatcher’s workstation.
- Check for smoking in an area where smoking is prohibited.
There is no magic here. Industrial scenarios still require configuration, testing, quality control and validation. But the process of developing a scenario becomes more flexible and more understandable to the business.
How we apply it in the platform
Our platform incorporates VLM-based analysis to examine a scene in context: where the person is, what they are doing, which objects are nearby, whether a rule is being broken and why the observation should become an event.
A particularly useful capability is the prompt editor. Users can describe a rule, configure the logic and test the result against a video stream or archive. For suitable tasks, this avoids starting a separate custom-model development project every time an operating rule changes.
This matters at facilities where procedures evolve and there are too many possible situations to pre-train a separate detector for each one.
Classical detectors still have a job
Do not swing to the opposite extreme. A VLM is not a replacement for all classical computer vision.
Conventional detectors remain well suited to recognising common objects quickly and consistently at scale: people, helmets, vests, vehicles, machinery, cigarettes and phones. VLMs add a further layer for interpreting relationships and meaning.
The useful architecture combines the two. Detectors find objects and candidate events. A VLM examines the context. The enterprise’s rules become scenarios. The operator receives an event with an explanation.
That combination helps reduce absurd mistakes of the dog-becomes-a-tiger variety.
A little provocation
The old “AI will take over the world” meme was funny because AI often failed to understand the obvious. It is becoming funny for another reason: modern models can handle more context than many people expected. They are no longer just counting stripes.
Customers once asked, “Can you train a detector for our task?” Increasingly, the question is, “Can we describe our operating rule so the system can start checking it?”
In my view, that is a much bigger shift than another percentage point of accuracy in a presentation.
Beyond the box
First we taught AI to find a helmet. Then a person without a helmet. Then a person without a helmet in a hazardous zone. Now the task is a breach of a particular rule in the particular context of an enterprise.
So use that old dog-and-tiger meme carefully. A modern model might look at it and say, “This is a dog. The stripes are shadows cast by the fence. The probability of a tiger is very low.”
And then, perhaps: “Still a good meme.”
Explore Platform 4.0 and the technical overview of multimodal analysis.