Video analytics latency on real footage: how to test

TL;DR: Measure notification latency on the real camera path by recording when the event appears, when the pipeline creates a signal and when the responsible person receives and reviews it. A demo timing or a vendor specification cannot stand in for the customer's camera, network and workflow. Discuss a measurable first scenario.

Latency across the video path

Latency is the time between a visible event in a camera scene and a usable signal reaching the agreed reviewer. That interval can contain capture, encoding, network transport, decoding, inference, rule evaluation, clip creation, notification delivery and human review. A project should name which boundary it is measuring instead of calling every delay “the model latency.”

For a practical test, record at least three timestamps: the source event, the generated signal and the review start. If the project needs a stronger technical record, separate the pipeline stages and write down the clock used by each system.

Timestamp record

NVIDIA DeepStream documents frame NTP timestamps and describes host-system time at stream-mux receipt as one method. It also describes timestamps from RTSP sender reports when the source provides them. The host-time method requires the host to be synchronized to NTP. This gives the test a concrete reason to check clocks before comparing events.

A useful record has the camera or source identifier, event description, source timestamp, processing timestamp, notification timestamp, review timestamp and any missing value. Do not hide a missing timestamp by replacing it with the time when someone opened a spreadsheet.

Comparison: lab demo and site test

A lab demo can show that a pipeline runs with prepared footage. A real site test also includes camera exposure, movement, network load, source clock behavior, notification routing and the operator's response. The real site therefore answers a broader question: can this specific workflow produce a reviewable signal under the agreed conditions?

The comparison is useful when the same event is replayed in both settings. Keep the scene, stream and clock assumptions visible. A faster lab result is not a measured result for a customer's camera.

Latency test sequence

Choose one camera, one event and one responsible reviewer.
Synchronize the clocks that the test uses and record the method.
Mark the event in the source footage with an auditable timestamp.
Record when the processing rule emits a signal and when the notification becomes available.
Record the review start and the operator's action.
Repeat the same scenario under the relevant lighting and network conditions.

DeepStream also documents component latency metadata with input and output timestamps. This can help an engineering team locate a slow stage when that toolkit is used. It does not say that an aiVISION project uses DeepStream, and it does not predict a customer's result.

Repeatability and reviewer separation

Repeat the same event definition, camera identifier and clock method when the test is repeated. If a reviewer changes, record that change rather than mixing human response time with pipeline time. The report should separate technical delay from the time a person needed to understand the clip.

Limits of a latency test

A small test cannot establish a permanent service-level number, a universal camera result or the absence of missed events. It can show the conditions that were tested and the gaps that remain. If the source clock is unknown, the event is ambiguous or the notification channel is only a mock, the result should stay marked as incomplete. NIST's AI RMF is a governance reference for keeping those unknowns visible.

Latency also interacts with detection quality. A fast signal that uses a poor view is not useful, and a clear event that arrives after the operator's decision window may not fit the process. Keep both dimensions in the acceptance record.

Latency test record

Camera and stream identifier.
Event description and scene conditions.
Clock and timestamp method.
Stage timestamps with missing values visible.
Notification route and human review action.

Repeatability matters as much as the average observation. Use the same event definition, the same camera identifier and the same clock method when the test is repeated. If a reviewer changes, record that change rather than mixing human response time with pipeline time. The report should separate technical delay from the time a person needed to understand the clip.

When a test produces a slow or incomplete result, preserve it. A failed run can identify a network bottleneck, a missing clock source or an unclear event rule. Changing the threshold until the table looks better removes the evidence that the team needs to decide whether the workflow is ready for a next stage.

The final report should avoid one blended number. Explain which interval was measured, which interval was unavailable and which part depended on the reviewer. That format gives an owner a decision they can understand: change the scene, change the path, change the rule or keep testing.

The same discipline applies to the notification route. A signal created inside the pipeline may still wait for a message gateway, a device or a person. Record the route instead of treating the first internal timestamp as the end of the process.

Frequently asked questions

Is a vendor's latency statement enough?

No. It may describe a defined test environment. The project needs a measurement on the relevant camera, network, processing path and notification workflow.

What if one timestamp is missing?

Keep the value unknown, record why it is missing and avoid presenting the remaining timestamps as a complete end-to-end result.

მასალა მომზადებულია AI-ის დახმარებით და გადამოწმებულია სარედაქციო ეტაპზე.