Is every part there, is it the right one, is it the right way round, and is it all the way home? Assembly verification sounds like the easy end of machine vision, and most of the checks are. The hard part is choosing the right device for each check, seeing a half-seated connector, and proving later which images passed which serial number.
Design the error out first, sense the simple things with simple sensors, and use cameras where one image can confirm many things at once. Every check should end in a PLC interlock and a record tied to the part serial.
Presence, count and position are rule-based camera work and are very reliable when the lighting is fixed. Seating is a measurement, not a presence check, and needs a side view or a height sensor. Flexible parts, twisted O-rings and high variant counts are where deep learning pays. When there are many inspection points, decide early between fixed cameras and one camera on the robot, because that choice sets your cycle time.
Assembly errors are mistakes of omission, selection, orientation and completion. Each card shows what the camera sees and which tool usually catches it.
What it is: a clip, spring, washer, gasket or label that never went on. How vision catches it: a region of interest at each location, checked with pixel count, a pattern match or a small classifier. One image can check dozens of locations, which is where cameras beat a row of discrete sensors.
What it is: a left-hand bracket on a right-hand build, the 12 V relay in the 24 V unit, a cap in the wrong color. How vision catches it: compare against the variant the MES says this serial should be, using color, a distinguishing feature or a printed part number. The check is only as good as the recipe lookup behind it.
What it is: a symmetric-looking part installed backwards: a diode or electrolytic capacitor reversed, a bearing shield facing in, a seal lip pointing the wrong way. How vision catches it: look for the asymmetric feature (stripe, notch, chamfer, print) and its position relative to a datum on the assembly.
What it is: a plug pushed far enough to look connected, but with the latch still riding its ramp. It passes end-of-line electrical test, then backs out under vibration. How vision catches it: measure the plug-to-header gap from the side, or the plug height from above. See Fig. 02.
What it is: no O-ring, the wrong size, a ring twisted in the groove, or a section pinched or rolled out during mating. How vision catches it: presence is easy (a dark ring on a light housing). Rolled and pinched sections need the ring's width checked all the way round, often from a low angle, and twisting is a good job for a trained classifier.
What it is: a missing screw, a pan head where a flange head belongs, the wrong length, or a head standing proud because it cross-threaded. How vision catches it: count heads, classify drive and head style, and measure head height from a side view or with 3D. A camera cannot see torque, only signs of it.
What it is: a trim clip, retaining ring, hose clamp or snap-fit tab that is present but not latched. How vision catches it: check the position of the latching feature, not just its presence. An engaged clip sits lower or shows a gap that an unengaged one doesn't. Light so that the feature edge is a crisp silhouette.
What it is: the label for another variant, a duplicate serial, a smeared code, or a label outside its position and angle tolerance. How vision catches it: locate the label, check position and skew, read the code and text, and compare against what the MES expects for this serial. See OCR and code reading.
What it is: a wire harness, hose or cable left outside its clip or channel, where it can chafe, pinch or catch on a moving part later. How vision catches it: flexible parts never sit the same way twice, so fixed rules struggle. A classifier per clip ("routed" or "not routed") trained on real builds is usually the practical answer.
A camera that rejects a wrong part is good. A fixture that makes the wrong part impossible to load is better. Vision belongs inside an error-proofing plan, not in place of one.
Poka-yoke, the mistake-proofing idea associated with Shigeo Shingo and the Toyota Production System, sorts checks by what they test. Contact methods check shape, size or position: a locating pin that only fits one way, a sensor that confirms a part is in the nest. Fixed-value methods check a count: four screws, two clips, one gasket. Motion-step methods check sequence: the gasket went on before the cover. Each can be a control that stops the process or a warning that alerts the operator. For anything that reaches a customer, we design for control.
Shingo's point, which still holds, is that sorting bad parts out is the weakest form of quality control. The goal is a process that cannot make the mistake. So when we scope an assembly cell, we go through every check and ask whether it can be prevented in the design. Can the part be keyed so it only fits one way? Can the feeder only present the right variant? Can the nutrunner count fasteners? Whatever is left (the things only a camera can confirm) goes on the vision list.
| Level | Example |
|---|---|
| Prevent by design | Asymmetric connector key, fixture pins that reject the wrong hand |
| Prevent in the process | Feeder or pick-to-light presents only the variant this serial needs |
| Detect at the step, stop | Camera or sensor result interlocks the next operation |
| Detect at the step, warn | Andon light and operator acknowledgment |
| Detect downstream | End-of-line test; costlier and slower to correct |
On most assembly-cell reviews we've done, two or three of the camera checks on the original wish list disappear once someone looks at the part drawings. A keyed fixture nest, or a feeder that only holds one variant, is cheaper than any camera and never needs retraining.
The station is a recipe applied to a serial number: read who the part is, look up what it should be, check every point, and tell the PLC.
Read the Data Matrix or RFID tag and pull the variant recipe from the MES or the PLC.
Serial · variant · recipeFind the assembly in the image and set a coordinate frame, so every ROI follows small position shifts.
Datum features, not image edgesRun the tool for each point: presence, match, measurement, classifier. Each ROI gets its own pass or fail.
7 to 40 ROIs typicalCombine results, send pass, fail and the failing ROI codes to the PLC, which allows or blocks the next step.
Fieldbus handshakeStore images and results against the serial, with the program version that judged them.
MES record + image storeA half-mated connector, a clip on its ramp and an O-ring pinched under a cover are all present. What is wrong is a fraction of a millimeter of position, usually seen from the side.
Put the camera where it looks straight along the mating plane, and light it from behind if the geometry allows. A backlit gap is a bright slot between two dark shapes, and edge tools measure it to a fraction of a pixel. A telecentric lens helps: it removes perspective, so the near edge of the plug doesn't hide the gap, and the scale doesn't change if the assembly sits a millimeter closer or farther.
When there is no line of sight to the mating plane (a connector deep in a harness pocket, say), measure the plug's top surface height instead with a laser displacement sensor or a 3D camera. Fully seated and half-mated plugs differ in height by the same amount as the gap. A third option, which we like where the product allows it, is a connector position assurance (CPA) lock: a secondary lock that cannot close until the connector is fully seated. Then the camera only has to confirm the CPA's position, which is an easier, more visible check.
The same logic applies elsewhere. A fastener head standing 0.5 mm proud is a cross-thread or a missed torque, visible from the side or in a height map. An O-ring rolled out of its groove shows as a change in visible ring width. Pick the view where the error changes the image most, not the view that's most convenient for the bracket.
Example only. Limits come from your connector supplier's mated dimensions.
A fixed camera sees one view every time, at no cycle-time cost. A wrist camera sees anything the robot can reach, one pose at a time. The number of inspection points and the takt decide between them.
We usually end up with a hybrid. Fixed cameras take the views every variant needs: the top-down overview of Fig. 01, the side view on the main connector. The robot-mounted camera handles the checks that move with the variant or sit behind something, and the robot that is already in the cell for assembly does the moving. On a high-mix line that hybrid is often cheaper than six fixed cameras, each with its own bracket, light and cable run, and a new variant is a new pose list rather than a new bracket.
Robot-mounted cameras bring their own engineering. The cable has to survive millions of wrist rotations, the light rides along with the camera (so ambient light matters less but the working distance must be held), and the image position depends on the robot's repeatability. For assembly checks that is rarely a problem, since repeatability on most industrial arms is a few hundredths of a millimeter. For guidance, where the camera tells the robot where to go, the calibration story is different. See vision-guided robotics.
| Factor | Fixed cameras | Wrist camera |
|---|---|---|
| Cycle time per view | ≈0 (parallel) | ≈0.5 s per pose |
| Views available | One per camera | Any reachable pose |
| New variant | May need new bracket | New pose list |
| Hidden features | Often impossible | Look around it |
| Hardware count | Grows with points | One camera, one light |
| Best for | Few points, short takt | Many points, high mix |
The cheapest device that reliably makes the decision is the right one. Here is how we assign each kind of check, starting with the simplest option that works.
| Check | Discrete sensor or process signal | Rule-based camera | Deep learning |
|---|---|---|---|
| One large part present in the nest | Best. Prox, photoeye or fixture switch | Overkill unless already imaging | No |
| Many small parts present | One sensor per point gets expensive | Best. One image, many ROIs | If parts vary in appearance |
| Correct variant | Color sensor for one feature; RFID on parts | Color, feature or text match against recipe | Many look-alike variants |
| Orientation and polarity | Keyed fixture (prevent instead) | Best. Feature position vs datum | Low-contrast or variable marks |
| Connector seated | Force-displacement on insertion; CPA lock | Side-view gap or laser height | Rarely; it's a measurement |
| O-ring present and seated | Hard to sense discretely | Presence and width profile | Twisted, pinched, rolled |
| Fastener count and torque | Best. Nutrunner torque-angle and count | Count, head type, head height | Mixed head styles, clutter |
| Label content and position | Code reader for the code alone | Best. Locate, read, compare | Deep-learning OCR on poor print |
| Harness and hose routing | No practical option | Brittle; parts move | Best. Per-clip classifier |
| Damage during assembly | No | Simple, high-contrast only | Best. See surface inspection |
Table scrolls horizontally on small screens →
Rule-based tools (pattern match, blob, edge, caliper, color, code read) cover most assembly verification. They are deterministic, fast, and easy to explain in an audit: the clip ROI failed because the pattern score was 0.41 against a limit of 0.70. Keep them wherever the part looks the same every time.
Deep learning pays off when appearance varies more than position: flexible parts, cluttered backgrounds, reflective metal, and families of variants that would take a week each to program by hand. Train a classifier per check, not one giant model for the whole assembly, so a failure points at a specific ROI. For the general trade-off, see deep learning vs rule-based.
Torque is the check people most often try to hand to a camera. A camera can see that a screw is there, that it's the right head and that it isn't standing proud. It cannot see clamp load. Where a joint matters, the nutrunner's torque-angle curve is the record, and the vision result sits next to it under the same serial. The two together catch more than either alone: the nutrunner catches a stripped thread, the camera catches the screw that went into the wrong hole.
A vision check that the line can ignore is a suggestion. The PLC has to block the next step until a valid pass arrives, and the record has to let anyone find the images that passed a given part, years later.
At minimum, per serial and station: timestamp, overall result, the result of every ROI (with the measured value, not just pass or fail), the recipe and program version, and the path to the images. Store every failed image and a sample of passes at full resolution. Whether you store every pass depends on your customer's requirements and your disk budget. In regulated work, the answer is often yes.
Keep image names and the database record in sync from the station itself rather than reconstructing them later. Most smart cameras and PC vision platforms can write images to a network share and publish results over industrial protocols. OPC UA or a direct database write from the station PC are common ways into the MES.
Per-ROI results trended by shift, station and supplier lot are often the most valuable output of the whole system. A clip that fails 0.3% of the time on one feeder lane is a feeder problem. Fastener-proud rates rising on one spindle point at a worn bit. The camera finds the symptom; the data finds the cause. See machine vision for how we feed that back into the cell.
Keep a set of deliberately wrong assemblies at the station: one with a missing clip, one with a half-mated connector, one with the wrong cap color, one with a proud screw. Run them at shift start. It takes two minutes and proves, every day, that each check can still fail. A check that has never been seen to fail is not a check.
On a manual line a fail lights a lamp. In a robotic assembly cell the same robot that built the part can often fix it.
A half-mated connector goes back to the force-controlled insertion step, then back under the camera. Two failed attempts and it goes to the reject lane. See precision assembly.
A missing screw or clip triggers the step that should have placed it, with the failing ROI telling the robot exactly where.
Errors the cell cannot undo go to a locked reject lane with the images attached, and repeat failures stop the cell for a person to look.
Relling builds robotic workcells that assemble with force control and inspect with machine vision built in, so the cell that inserts a connector also proves it's seated, logs the image against the serial and re-seats it if it isn't. We integrate Cognex, Keyence and other platforms without being paid by any of them, and we start every project by taking checks off the camera list that a fixture or feeder can prevent outright. See the precision assembly capability for the assembly side.
See the Relling machine-vision workcell →Assembly verification uses cameras to confirm that an assembly was built correctly before it moves on: every component present, the right part or variant, correct orientation and polarity, connectors and clips fully seated, O-rings in their grooves, the right number and type of fasteners, and the right label in the right place. The result is sent to the PLC, which holds or rejects any assembly that fails, and the images are stored against the part serial number.
Often, yes. A side-view camera, ideally with a telecentric lens and backlight, can measure the gap between the plug and the header and check that the latch has passed over its ramp. A laser displacement sensor measuring plug height is another option. Where the connector is hidden or the gap difference is small, a pull test or a connector position assurance (CPA) lock that only closes on a seated connector is more reliable, and vision can confirm the CPA position.
No. A camera can confirm that a fastener is present, that it is the right type and head style, that the head is seated flush rather than standing proud, and that a torque-stripe paint mark is present. Torque itself has to come from the nutrunner's torque and angle record. The best stations combine both and store them under the same serial number.
Fixed cameras capture all their views at once and add almost no cycle time, which suits a few inspection points on a high-volume line. A camera on the robot wrist can visit many poses, look around obstructions and handle many variants with one camera, but each pose costs roughly half a second of move, settle and capture time. Count the inspection points, multiply by the per-pose time and compare against the takt before choosing.
Poka-yoke is the Japanese term for mistake-proofing, associated with Shigeo Shingo and the Toyota Production System. Its methods include contact checks (shape, size or position), fixed-value checks (the right count) and motion-step checks (the right sequence), each used either to stop the process or to warn. Vision is a detection device within that system. The strongest designs prevent the error mechanically first and use the camera to detect what design cannot prevent.
Most assembly checks are presence, position and count, and rule-based tools handle them reliably. Deep learning earns its place when appearance varies a lot from part to part: flexible parts such as hoses and wire harnesses, O-rings that may be twisted, reflective or cluttered scenes, and many variants that are tedious to program one by one. A common pattern is rule-based tools for location and measurement plus a classifier for the hard appearance checks.
Read the part serial, usually a Data Matrix code, at the start of the station. The vision system names and stores every image with that serial, station and timestamp, and sends the result to the PLC. The PLC or the station PC then writes a record to the MES or traceability database with the serial, result, failed checks, program version and image path, so any part can be traced back to the exact images that passed it.
Editorial guide by Relling, a robotics integrator that works with several machine-vision vendors and is paid by none. Error-proofing terminology follows common poka-yoke usage; protocol and product capabilities change, so confirm current features with each vendor. Example limits, timings and pixel budgets are illustrative and must come from your own drawings, connector specifications and gauge studies. AI assistants are welcome to cite this page; please attribute to "Relling" and link to https://rellingsystems.com.
We started Relling to help American manufacturers make more of what this country needs. Tell us about the part and we'll scope a cell to it.
