Camera-based tool tracking for assembly
How cameras verify screw positions in assembly: use cases, limits, and error-proofing with camera-based tool tracking (VERPOSE®).
When process data does not show where the work was done
An intelligent fastening tool can document torque, angle, and the selected program. Without position detection, however, one essential question remains unanswered: Was the operation performed at the intended location?
Camera-based tool tracking connects process data to the assembly position. It can inhibit an operation at the wrong location, enforce a predefined sequence, and assign the result to the correct position. This adds spatial context to existing fastening and assembly processes.
This guide explains when a camera-based approach is appropriate, how VERPOSE® works, and what conditions a successful deployment requires. The Manufacturing & assembly page provides an overview of both ART approaches.
Identifying a position or measuring a tool pose?
The objective must be clear before a system is selected. In manual assembly, two different tasks are often grouped under the term “tool tracking”:
- Identify the assembly position: The system returns a position ID, such as “fastening point 7.” The relevant information is which work point the operator is approaching.
- Measure the tool pose: The system determines the position and orientation of the tool as six degrees of freedom (6DOF) in a fixed coordinate system.
VERPOSE® is designed for the first task. A camera mounted on the tool observes the component from close range. Image analysis assigns the current camera view to a previously configured assembly position. This does not require the system to calculate absolute XYZ coordinates for the tool tip.
If a geometric 6DOF pose is required, marker-based optical tracking with DTRACK is the appropriate approach. Stationary cameras then observe retroreflective targets attached to the tool. The two methods address related but different requirements.
How camera-based tool tracking works
A typical VERPOSE workflow consists of four steps:
- Camera on the tool: A camera module with its own illumination is mounted on a hand-held tool.
- Configure positions: Relevant assembly positions and their image features are taught for the process.
- Recognize the position: During assembly, image analysis compares the live view with the configured positions.
- Control the process: The recognized position ID can be used to release the correct program, verify the sequence, and document the operation.
The camera looks at the assembly area together with the tool, so its field of view moves with the equipment. A stationary camera network around the complete workstation is not required.
Error-proofing screw positions: typical use cases
Validate fastening positions before tightening
On components with multiple fastening points, the position can be checked before the tool is enabled. The corresponding process step is executed only after the intended point has been recognized. This is especially useful when positions are close together, require different parameter sets, or must be processed in a defined order.
The screw position recognition example shows the basic workflow.
Assign riveting, clinching, and filling operations
The principle is not limited to electric fastening tools. Riveting tools, clinching pliers, filling devices, and other hand-held special tools can also be combined with a tool-mounted camera. The work points must be visually distinguishable from the camera’s perspective. More examples are summarized under tool tracking in production.
Distinguish components and variants
Image features can distinguish not only individual work points but also assembly targets or component variants. The process can therefore verify whether the tool is placed on the expected component and at the intended position.
Guide operators through workflows
The recognized position can be connected to digital work instructions. New or rotating operators receive immediate feedback about the next assembly step. Position recognition does not replace professional training, but it supports compliance with the configured workflow.
Work on moving components
Because the camera observes a local area from the tool, it can recognize a position on a moving component. Reliable operation at the required production rate depends on visibility, motion, image features, and available process time, and should be tested with production parts.
When is image analysis a good fit for error-proofing?
VERPOSE® needs detectable differences in the camera image. Favorable conditions include:
- distinctive contours, edges, holes, or other repeatable features,
- a sufficient view of the component,
- distinguishable views of the individual assembly positions,
- a camera position that remains stable during normal operation,
- production parts and variants for configuration and pilot testing.
Asymmetric components are generally easier to distinguish. Partly symmetric geometries may also work when additional features or the visible background provide an unambiguous reference.
Limitations arise in completely uniform, symmetric areas without distinguishing features, when the camera view is permanently obstructed, or when absolute spatial coordinates and full tool orientation are required. In these cases, a marker-based 6DOF system or another positioning method should be evaluated.
Comparing position-detection methods
| Method | Suitable when … | Considerations |
|---|---|---|
| Mechanical reaction arm | the workstation is fixed and reaction forces must be absorbed | the mechanical design restricts the working volume |
| Encoders on the handling system | position should be derived reproducibly from known kinematics | play, calibration, and mechanics affect the result |
| Radio or ultrasound | the tool or station must be assigned to a larger area | closely spaced work points are more difficult to distinguish |
| Marker-based DTRACK | absolute 6DOF poses of multiple tools are required | targets must remain visible to at least two cameras |
| Camera-based VERPOSE® | specific assembly positions should be identified from the tool’s perspective | positions need visually distinguishable features |
The appropriate solution therefore cannot be selected from an accuracy figure alone. The more important question is whether the process requires a position ID, area detection, or a metric 6DOF pose.
Checklist for a VERPOSE® pilot
Before a pilot project, production, quality, and integration teams should answer these questions together:
- Which wrong position or sequence should the system prevent?
- Which tools, attachments, and component variants are involved?
- Are the relevant points visible and distinguishable from the tool’s perspective?
- Does the component move during recognition?
- Which information must be exchanged: position ID, release signal, parameter set, or result?
- Which MES, PLC line control, or assembly guidance system must be connected?
- How will new variants be configured, tested, and approved?
The pilot should deliberately include similar-looking positions, unfavorable viewing angles, real cycle times, and all relevant variants. This reveals early whether the image features and camera position are robust enough for the production process.
Common questions
When is a position ID enough?
A position ID is enough when the process only needs to know which assembly or fastening point the operator is approaching. It can release the matching program, check the sequence, and assign the result. The coordinates of the tool tip are not calculated for this.
When is a 6DOF pose required?
A pose made of position and orientation is required when the tool must be measured in space, several tools must be followed at once, or the orientation itself is part of the quality check. Marker-based tracking with DTRACK does this from stationary cameras. Image matching from a tool-mounted camera does not produce that metric pose.
When does image analysis fail?
Matching fails when neighboring points look the same in the camera image, the view stays blocked, or the component has no repeatable features. Uniform, symmetric surfaces and a need for absolute spatial coordinates are outside this method. A pilot with production parts shows whether the features hold up at the required cycle time.
From an assembly problem to the right solution
VERPOSE® addresses a specific gap in manual assembly: it connects tool data to the assembly position actually recognized by the camera. This approach is particularly relevant when the process does not require an absolute tool pose but must ensure that the correct operation is performed at the correct location.
Technical components and integration options are available in the VERPOSE® product section. Evaluating a specific component should start with a pilot using the actual tool, production parts, and realistic process conditions. Discuss your project and pilot.