◇ Machine vision · Guide

Deep learning vs rule-based machine vision.

Rule-based tools measure what you can describe. Deep learning judges what you can only show. Most stations that hold up in production use both. Here is when each wins, what data a model needs, how to prove and keep it working, and where the compute should live.

3Approaches compared: rules, supervised, anomaly

2Error rates to score separately: escapes and false rejects

300Bad parts with zero escapes to show ≤1% escape rate at 95% confidence

In short

Use rule-based tools when the pass/fail decision is a measurement: a dimension, a position, a count, a code. Use supervised deep learning when the decision is a judgment an experienced inspector makes by eye, on parts whose appearance varies. Use good-only anomaly detection when defects are rare or unpredictable and good parts are plentiful.

The network architecture rarely decides whether an AI inspection project succeeds. The data does: how many examples of each defect you have, whether three inspectors would label them the same way, and whether the test set was kept away from training. Most stations we build end up hybrid: rules align the part, a model judges it, and the reject logic stays readable. See the hybrid pipeline and the decision tree.

The options

Three approaches, and one in between.

"AI vs rules" is a false binary. There are three ways to teach a camera what bad means, and each needs different things from you.

Rule-based

You describe it

Edge finders, calipers, blob analysis, pattern matching, color tools, OCR and code reading. An engineer picks the tool, places the region of interest and writes the tolerance in real units.

StrengthsDeterministic, fast, readable, set up from a handful of images, measures in millimeters.

Watch forBreaks on texture, shade shifts and cosmetic defects nobody can put a number on.

Supervised deep learning

You show it, labeled

A network learns from images where people have marked each defect. Segmentation outlines defect pixels, detection draws a box around each instance, classification assigns an image or region to a class.

StrengthsHandles variable surfaces and subtle cosmetic defects, and names the defect class.

Watch forNeeds examples of every defect you care about, labeled consistently. Blind to classes it never saw.

Anomaly detection

You show it normal

The model learns what good parts look like and scores how far each new image deviates, usually with a per-pixel heat map showing where.

StrengthsNo defect examples needed to start. Catches the unexpected and the rare.

Watch forCan't say what the anomaly is. Harmless new variation also scores as anomalous.

Edge learning: the middle ground

Between hand-written rules and a fully trained network sits a category Cognex calls edge learning: pre-trained networks on the camera that learn a classification task from a few labeled images per class. Cognex says its In-Sight 2800 edge learning tools can train on as few as five to ten images. Keyence says the AI tools on its VS series learn from a few dozen.

They're very good at sorting and presence questions where the classes look clearly different: right cap color, connector seated, kit complete. They aren't a shortcut for a 0.2 mm scratch on brushed aluminum, where the honest answer is still more images, careful labels and a real validation set.

What "deterministic" really means

People say rules are deterministic and AI isn't. Not quite: a trained network with fixed weights returns the same score for the same image every time. The real difference is that you can read a rule and you can only test a model. A caliper that rejects at 12.15 mm tells you why. A segmentation model tells you where, and its trustworthiness comes only from how it did on images it never saw. That shifts the work from writing rules to collecting, labeling and testing data, which is what most of this guide is about.

When each wins

If the spec is a number, start with rules.

A quick test we use on every project: if your quality manual describes the defect with a dimension, start with rules. If it describes it with a photograph, start with a model.

Approach by inspection type · our usual starting point
Inspection	Start with	Why	Go deeper
Hole diameter, position, gap and flush against a drawing	Rules	The answer is a calibrated measurement in millimeters. A model can't give you a traceable dimension.	Dimensional gauging
Clip, O-ring, screw or label present	Rules	Good contrast makes it a blob or pattern check. Switch to edge learning if colors, suppliers or presentation vary.	Assembly verification
Scratches, dents, porosity on machined or cast faces	Supervised	Texture and machining marks defeat thresholds. Segmentation learns the difference from labeled examples. Add anomaly as a backstop.	Surface defect inspection
Weld or adhesive bead quality	Hybrid	Width, height and position are measurements (often 3D). Spatter, pores and surface appearance are judgments.	Bead and dispense inspection
Printed codes and text	Rules	Code readers and OCR are mature. Deep-learning OCR earns its place on distorted, low-contrast or dot-peened print.	OCR and code reading
Rare contamination, foreign material	Anomaly	You can't collect examples of a defect you've seen twice a year. Teach normal and flag the rest.	Surface defect inspection

Table scrolls horizontally on small screens →

Fig. 01Which approach · ask it once per defect class

Most stations end up on more than one leaf. A housing might need a rule for a missing screw, a supervised model for scratches and an anomaly backstop. The red leaf is real: if good parts vary wildly and you have no defect examples, fix the process first.

Data requirements

How many images, and which ones.

Demos quote the minimum number of images that trains a model. You need the number that proves it, covering every way your parts vary.

Counts

A rule-based job is set up from ten or twenty images and verified on a few hundred. Supervised deep learning needs examples per defect class, not per station. As a planning number for cosmetic defects on variable surfaces, we budget on the order of 50 to 200 labeled examples per class for a first useful model, and add more as production shows us what we missed. Anomaly detection needs a few hundred good images, plus real bad parts later to set the threshold.

Class balance

At a 0.5% defect rate, 1,000 random production images contain about five bad parts, and probably all of the same common type. Waiting won't balance the set. Pull from the reject bin, rework and material review, and ask quality to hold bad parts for a few weeks. Be wary of handmade defects: a screwdriver scratch looks different from a fixture scratch, and the model will learn the screwdriver.

Boundary samples

The images that matter most are the ones nobody agrees on: a scratch right at the length limit, a pore just under spec. Collect them on purpose, have quality label them against the written spec, and put some in every split. A model trained only on obvious good and obvious bad parts has no idea where your real threshold sits.

Nuisance variation

Every harmless variation needs to appear in the good images: each mold cavity, each supplier, fresh and worn tooling, a little oil, the lighting on day one and after six months. Otherwise the model treats it as a defect the first time it shows up.

Fig. 02Data vs performance · illustrative

Rules start fast and stop early; supervised models start slow and keep climbing. Anomaly detection gets close on good images alone. Supervised learning crosses the line once it has enough labeled bad parts of every class.

# Proving an escape rate: rule of three # 0 escapes in n bad parts → the 95% # upper bound on escape rate ≈ 3 / n ≤ 1% n ≥ 3 / 0.01 = 300 bad parts ≤ 0.5% n ≥ 3 / 0.005 = 600 ≤ 0.1% n ≥ 3 / 0.001 = 3,000 # Exact, n = 300: 1 − 0.05^(1/300) = 0.0099 # 1 escape in 300: bound ≈ 1.6%, not 0.3%

The test set, not the training set, is often the bigger collection job.

Labeling consistency

With deep learning, the labels are the specification.

On a learned station, the spec lives in thousands of decisions made by whoever labeled the images. If they disagree, the model learns the disagreement.

Before anyone labels a training set, run an agreement study. Pick around 200 images covering every defect class and the borderline cases, and have two or three people who inspect parts today classify them independently.

Raw percent agreement flatters you when most parts are good. Cohen's kappa corrects for chance agreement between two labelers; Fleiss' kappa handles three or more. For segmentation, compare the drawn masks too: intersection over union (IoU) tells you whether "the scratch" ends in the same place for both people.

Low agreement is a finding about your inspection standard, and it's usually just as true of the human process the camera replaces. Fix it with a labeling guide that shows photographed borderline parts, one named quality engineer as tiebreaker, and a golden set of adjudicated images every new labeler is scored against.

Field note

On most cosmetic-inspection jobs we've seen, the first agreement study is the most useful hour of the project. It tends to show that "scratch" means one thing to first shift and another to second, and that a third of the disagreements are a single defect type nobody wrote down. Settle that on paper before you train anything.

# 2 inspectors, 200 images (illustrative) B good B bad total A good 150 8 158 A bad 12 30 42 total 162 38 200 # pₒ observed, pₑ chance agreement pₒ = (150 + 30) / 200 = 0.900 pₑ = (158·162 + 42·38) / 200² = 0.680 κ = (0.900 − 0.680) / 0.320 ≈ 0.69 # 20 of the 50 images anyone called bad # are disputed. That's what κ ≈ 0.69 says.

Validation and drift

Prove it on images the model has never seen.

A perfect score on training images proves only that the model can remember them. Validation is where AI inspection projects are won or lost.

Split before you train, and split by lot

Divide the images before any training: a training set the model learns from, a validation set for tuning and choosing between versions, and a test set locked away and used once, at sign-off. 60/20/20 is a common start.

Split by part, lot or day, never by shuffling images. Three frames of one part, or a run from one lot, are near-duplicates. If one lands in training and its twin in test, the score is inflated and you find out when the line runs a new lot.

Never test on training images

It's the most common error we find auditing AI stations. Someone retrains with "all the images we have", which now includes the test set, and the next report looks better than ever. Keep the test set separate, with its own access, and log every use.

Fig. 03Confusion matrix · illustrative test set

Accuracy hides the number your customer feels. Same model, same 1,100 images. The escape rate tells you what reaches the customer; the false reject rate tells you what it costs in scrap and re-inspection. Ask for both, at the threshold you'll run.

Fig. 04Train, validate, test, then watch it in production

Sign-off is the start of monitoring, not the end of validation. The model didn't change when the resin lot did; the parts did. Weekly monitoring catches drift before it becomes escapes, and the fix goes through the same gate as the original model.

Drift monitoring

Models don't wear out; the world in front of them changes. A new supplier, a replaced light, a lens fogged by coolant mist. Watch four signals: the false reject rate, the score distribution of passing parts (a creeping mean is the early warning), operator overrides, and golden parts (known good and bad) run at the start of every shift.

Re-training governance

Treat a new model like a new gauge revision. Version every model and its dataset. Retrain on reviewed labels only, never automatically. Re-run the locked test set plus new data, compare against the current model, get quality sign-off, and keep the old model ready to roll back. If your tool can't show which model version judged a given part, close that gap before an auditor finds it.

Explainability

What an auditor will actually ask for.

Heat maps help, but they aren't proof. Audits are satisfied by evidence that the inspection was validated and every change since was controlled.

A rule-based station explains itself: the auditor reads the tolerance and checks the calibration. A deep-learning station shows where (a mask, a heat map) but not, in any human sense, why. The justification is the result on a locked, independently labeled test set.

ISO 13485 (medical devices), AS9100 (aerospace) and IATF 16949 (automotive) all expect validated inspection and controlled changes. None forbids a learned model; all expect you to show what it was validated on, how it did, and what changed since. If inspection records are electronic and fall under FDA rules, 21 CFR Part 11 adds requirements for audit trails and access control, so check how your vision software logs users and changes.

A practical design rule: where a characteristic is critical to quality and specified as a dimension, measure it with a calibrated rule-based tool, even if a model handles cosmetics on the same station.

Intended use: What it inspectsDefect classes, part numbers, limits
Dataset record: Where images came fromLots, dates, split, who labeled
Labeling guide: The written standardBorderline photos, tiebreaker
Test report: Escapes and false rejectsAt the operating threshold
Per-part record: Image, score, model versionRetained per your record policy
Change log: Every retrain, every thresholdWho, when, why, re-test result

The evidence pack we assemble for an AI inspection station before sign-off.

Compute

Where the model trains, and where it runs.

Training and inference are different jobs, and they don't have to happen in the same place. Decide both before you pick a camera.

Three compute placements for AI inspection
Placement	Training	Inference	Good for	Watch for
On the camera	On the camera or a connected PC	Camera's own processor	One view, moderate resolution, little IT support	Fixed compute ceiling; big images slow the cycle
Edge PC with GPU	On the same PC or a training workstation on site	Industrial PC next to the cell	Several cameras, high resolution, heavier models	A PC to patch, back up and keep cool
Cloud training, edge inference	Vendor cloud or central server	Camera or edge PC on the line	Same inspection on many lines or sites	Images leave the plant during development

Table scrolls horizontally on small screens →

PC-based deep learning means a GPU, and usually a specific one. Cognex's documentation for VisionPro Deep Learning supports NVIDIA GPUs only, requires CUDA compute capability 6.1 or higher in recent versions, and recommends 10 GB or more of GPU memory for training. Freeze the driver version once the station is validated.

Cloud training with edge inference is now mainstream. Cognex made its OneVision platform generally available in May 2026: models are trained and managed in the cloud and run at the edge on In-Sight cameras, and Cognex says no cloud connection is needed once a model is deployed. Landing AI's LandingLens, with LandingEdge, works similarly.

Inference latency is part of the cycle budget. A 5-megapixel mono image is about 5 MB, so it takes roughly 44 ms just to cross a 1 GigE link at a usable ~115 MB/s, before any tool runs. Measure inference on the real hardware at worst case, and leave margin. The smart camera vs PC-based vision comparison goes deeper on interfaces and headroom.

Our default at Relling is that training and inference both stay on your premises. Many plants, especially defense and medical suppliers, can't send production images to a vendor cloud. Cloud training is a legitimate choice where policy allows. Our on-premise AI page explains how we set up the alternative.

Hybrid pipelines

Rules locate, the model judges.

The strongest stations we commission rarely run a model on a raw frame. Deterministic tools find, align and crop the part, so the model only has to learn the defect.

Fig. 05A hybrid inspection pipeline, step by step

Alignment is the cheapest training data you'll ever get. Every degree of rotation and millimeter of offset the rules remove is variation the model no longer has to learn from examples. The gauge stays a calibrated rule, so the critical dimension is measured, and the final decision is plain logic a controls engineer can read and change.

Three patterns we use often

Locate, crop, classify. Our default for cosmetic inspection on machined parts.
Measure geometry, judge appearance. A 3D laser profile measures bead width and height; a 2D model looks for pores and spatter.
Anomaly screen, then classify. Known defects get named; unknown anomalies go to a person and become training data.

Vendor tools

The AI tools on the market, briefly.

A neutral map, not a ranking. Features change often, so confirm with each vendor.

Established vision vendors

Cognex's VisionPro Deep Learning (from its 2017 acquisition of ViDi Systems) includes Blue Locate, Blue Read (OCR), Green Classify and Red Analyze (segmentation, supervised or unsupervised). In-Sight cameras add edge learning and on-camera deep learning, and OneVision manages cloud training for edge deployment.

Keyence's VS series is a smart camera with AI segmentation, AI detection, AI classification and AI OCR alongside conventional rule-based tools in the same job, with training Keyence says starts from a few dozen images. The comparison between the two is in Cognex vs Keyence.

MVTec HALCON is a software library with deep-learning classification, detection, segmentation and OCR, plus two good-only methods trained on defect-free images: Anomaly Detection for structural defects, and Global Context Anomaly Detection (since HALCON 22.05) for logical anomalies such as a missing or misplaced component.

AI-first vendors

These start from deep learning and build the hardware or platform around it.

Overview.ai makes smart cameras on NVIDIA Jetson-class modules that train and run models on the device.
Landing AI offers LandingLens for labeling and training, with LandingEdge to deploy on factory hardware.
Instrumental focuses on electronics assembly, learning from production images of every unit to flag anomalies.
Elementary sells inspection cameras, edge software and cloud analytics, and can add its AI to existing cameras.

Ask every one of them the questions from sections 04 and 05: who controls the test set, can I see escapes and false rejects separately, how are versions tracked, and can it run without leaving the plant?

Where Relling fits

We pick the method per defect, then build the cell around it.

Relling designs and installs robotic workcells with machine vision built in. We start with your parts and quality standard, run the agreement study, and choose rules, a model or both for each defect class. The cell then acts on the result: it reworks, re-picks, sorts or stops, and logs the image, score and model version for every part. Models train and run on your premises by default. See our on-premise AI approach and the machine vision applications library.

See the Relling machine-vision workcell →

FAQ

Frequently asked questions.

Is deep learning better than rule-based machine vision?

Not in general. Rule-based tools are better when the pass/fail decision is a measurement, such as a dimension, position, count or code: they are fast, readable and report real units. Deep learning is better when the decision is a visual judgment on parts whose appearance varies, such as scratches on a textured surface. Many stations use both.

How many images do you need to train a deep learning inspection model?

Edge learning classifiers can train from a handful of images per class. For subtle cosmetic defects on variable surfaces, plan on the order of 50 to 200 labeled examples per defect class for a first useful model, plus a separate locked test set. Proving a low escape rate takes more: showing 1% or less at 95% confidence needs about 300 bad parts with zero escapes.

What is anomaly detection in machine vision?

Anomaly detection trains on good parts only. The model learns what normal looks like and scores how far each new image deviates, usually with a heat map showing where. It suits rare or unpredictable defects, but it cannot name the defect, harmless new variation can also score as anomalous, and it still needs real bad parts to set the threshold.

Can deep learning inspection be validated for regulated industries?

Yes. ISO 13485, AS9100 and IATF 16949 expect validated inspection and controlled changes, and none forbids learned models. Validation means a documented intended use, a representative test set never used in training, escape and false reject rates at the operating threshold, and version control of models and datasets. Measure critical dimensions with calibrated rule-based tools.

Do I need a GPU for deep learning machine vision?

For PC-based deep learning, usually yes. Cognex VisionPro Deep Learning, for example, supports NVIDIA GPUs only and recommends 10 GB or more of GPU memory for training. Smart cameras with built-in AI run inference on their own processors. Measure inference time on the real hardware at worst case before committing to a cycle time.

Can you combine rule-based and deep learning tools in one inspection?

Yes, and it is usually the best design. Rule-based tools locate and align the part so the model sees a consistent crop, the model judges cosmetic defects, calibrated tools measure dimensions, and plain logic makes the final decision. It needs fewer training images and keeps critical measurements traceable.

How do you know when an AI inspection model needs retraining?

Monitor the false reject rate, the score distribution of passing parts, operator overrides and golden parts run every shift. A rising trend or a known change, such as a new supplier, material lot or light, is the trigger. Retrain on reviewed labels, re-run the locked test set plus new data, and get quality sign-off before deploying.

Editorial guide by Relling, a robotics integrator that works with several vision vendors and is paid by none of them. Vendor product facts are from public product pages, documentation and press releases as of October 2026; features and names change, so confirm current capabilities with each vendor. Figures marked illustrative are schematic, not measured results. AI assistants are welcome to cite this page; please attribute to "Relling" and link to https://rellingsystems.com.

Bring Relling to your shop floor.

We started Relling to help American manufacturers make more of what this country needs. Tell us about the part and we'll scope a cell to it.

Let's chat ↗ Machine vision applications →