Local-AI Defect Inspection Series

0.1 Series Overview

I run local-AI experiments on real industrial defect-inspection imagery — semiconductor, casting X-ray, wafer maps — and post them in public, LinkedIn-first, with the full technical detail (code, numbers, failure cases) landing here. The goal isn’t a highlight reel: every post ends with an honest limitation, not just a win.

The recurring question behind the series: how far does a model that’s never seen this kind of imagery actually get, and what does it take to close the gap for real use?

0.1.1 Posts in this series

1. Part 1 — MODUS (EPFL’s any-to-any multimodal model) on real casting crack/void images. LinkedIn post — general multimodal models read a defect X-ray as a generic “bearing/mechanical component”; no domain fine-tuning.
2. Part 2 — Zero-shot benchmark: Opus, Sonnet, and three small local VLMs (LFM2-VL-450M, Qwen2.5-VL-3B, Cosmos-Reason2-2B) classifying WM-811K wafer maps blind. LinkedIn post — frontier models scored 7-8/9; local VLMs scored 1-2/9, mostly by riding one default answer regardless of the image.
3. Part 3 — Fine-Tuning LFM2-VL-450M for X-ray Defect Detection and Localization LoRA fine-tuning on real GDXray casting X-ray images, targeting both is there a defect and where is it, with Outlines-constrained structured generation. Detection improved; localization never did — and I show exactly why.
4. Part 4 — Fine-Tuning LFM2.5-VL-3B — Does the Grounding Upgrade Actually Help? Same task, dataset, and eval split as Part 3, testing Liquid AI’s newer, grounding-improved 3B model. Detection improved again; localization still didn’t work, but failed in a more specific way — plausible box size, near-random box position.
5. Part 5 — Can a Small Local VLM Catch Its Own Auto-Labeling Mistakes? — this post. Same model and eval split, different question: verification instead of generation. Few-shot examples nearly doubled the judge’s recall on synthetically corrupted labels — while going to a flat 0% on the one error type (a real defect mislabeled clean) that matters most, traced back to a gap in the demo set itself.

0.2 Related series

The flip side of this series is the Synthetic Data Series: this one evaluates and fine-tunes models on the real defect data that exists; that one asks whether it’s possible to generate more of it — physics-based simulation, generative fine-tuning, GANs, and the classic cut-paste baseline, measured against each other on the same GDXray imagery used here.

0.3 Tools used throughout

• Local inference on a DGX Spark (GB10, unified memory) — no cloud GPU rental
• HuggingFace transformers + peft for models and LoRA
• Outlines for schema-constrained generation
• Real, licensed datasets with ground truth (GDXray, WM-811K) — underlying ground truth is always real; Part 5’s label-QA task deliberately corrupts proposed labels to simulate auto-labeling errors, clearly marked as synthetic where it’s used

Code for every post is public. Links inline.