Multi-modal process capture: from camera footage to BPMN

Status, October 16, 2026: reviewed recording-to-BPMN drafting is supported in configured Pro video analysis. It requires deployment setup and human review. Earlier versions of this note reported accuracy figures without supporting evaluation evidence. Those claims remain withdrawn. No measured accuracy or automatic privacy protection is claimed here.

In configured deployments, Geist analyzes uploaded or archived video and links findings to exact frames and timestamps for review. The flow covered here creates an editable BPMN draft from activities reviewed by a person, with an explicitly confirmed scope and sequence.

Problem statement

Traditional process mining requires structured event logs. In practice, large parts of operational processes happen outside of IT systems: manual assembly steps, physical inspections, verbal handoffs between teams. These activities leave no digital trace in any event log.

Camera-based process capture promises to close this gap, but raw video is not an event log. Converting footage into structured process data requires solving several hard problems simultaneously.

Reviewed recording workflow

The workflow covers one recorded procedure. A user chooses the process boundaries, reviews candidate activities against the footage, clarifies uncertainty, confirms their sequence, and creates a BPMN draft. A live camera or ERP connection is not a prerequisite.

Stage 1: Select a recording

Start with a complete packing, inspection, or loading procedure. Narration may explain intent, but it must remain distinguishable from what is visible. The recording needs a known source, permitted use, and an appropriate retention policy.

The draft links back to the retained recording and reviewed activities, subject to source access permissions. A visible person or object is not treated as a known employee, order, or process case.

Stage 2: Propose activities

For each candidate activity, Pro analysis proposes a name, a clip interval, an evidence label, and any uncertainty it identifies. Visually observed actions, statements in narration, human confirmations, and model inferences remain distinguishable. These labels also need human review.

Occlusion, rapid actions, repeated steps, and incomplete clips can hide or confuse activity boundaries. A fluent summary is insufficient evidence that the proposed sequence is correct.

Stage 3: Review and clarify

Selecting an activity seeks to its supporting interval. The reviewer can rename, split, merge, reorder, accept, or reject activities. Accepted inferences and activities marked uncertain require clarification. Every activity needs a review decision before drafting.

Consequential questions need explicit answers: Does inspection always happen? What happens when it fails? Are two overlapping actions independent, or does one depend on the other? The draft follows the sequence confirmed by the reviewer. Unsupported branches are not inferred from one recording.

Stage 4: Create an editable draft

Accepted activities and clarifications feed the existing guided drafting and validation flow. The result is an editable BPMN draft with links back to the reviewed evidence. Users can reopen the saved review and draft after reloading the application and inspect the model before using it operationally.

One recording demonstrates one execution. It does not establish all business rules, exception paths, variant frequencies, conformance, or throughput.

Later research: repeated executions

Continuous camera-based process mining is a separate research direction. It would require several reviewed executions, reliable case identities, and reconciliation with relevant system records. Clock offsets, repeated activities, and overlapping work need explicit treatment before timing or frequency metrics are meaningful.

Camera observations alone cannot reveal hidden approvals, system-only actions, or the reason for a decision. These gaps require additional evidence and human clarification.

Evaluation and limits

No validated accuracy, correction-effort, latency, or cost results are published for this flow. A broader evaluation needs a fixed set of real recordings, including incomplete clips, occlusion, repetition, and ambiguous actions.

That evaluation should measure missing and invented steps, evidence accuracy, review effort, and total cost. A working upload, review, draft, save, reload, and evidence-navigation journey does not establish accuracy across other recordings or business processes.

Privacy controls must be verified for each deployment. This workflow does not claim automatic face blurring, edge-only processing, or suitability for employee monitoring.