Laser-Based Weed Detection
& Removal System
A precision-agriculture system that tells weeds from crops pixel by pixel on fused RGB + near-infrared imagery, then drives a laser to remove them — cutting herbicide use without touching the crop.
The problem
Herbicide is sprayed across an entire field to kill the small fraction of it that is weeds. The chemicals reach the soil, the groundwater and the crop, and the weeds steadily evolve resistance. The obvious alternative — remove weeds individually — has always been limited by the cost of labour.
The computer vision problem underneath it is genuinely hard, and it's a segmentation problem rather than a detection one. A weed growing between crop rows overlaps its neighbours; leaves interleave; both are green, both are leafy, and both are lit by the same variable sunlight. A bounding box is the wrong output — a laser needs to know which pixels are safe to fire at. Get it wrong and you burn the crop you were protecting.
Approach
- Add a channel the eye doesn't have. In visible light, weed and crop are both "green" — the signal separating them is weak. Healthy vegetation reflects strongly in near-infrared, and it does so differently by species and leaf structure. Fusing NIR with RGB gives the network a discriminative channel that simply doesn't exist in a normal photograph. This was the single decision that made the rest of the problem tractable.
- UNet for the segmentation. UNet's encoder–decoder with skip connections is built for exactly this shape of problem: the encoder captures the context needed to know what something is, and the skip connections carry the fine spatial detail forward so the decoder knows precisely where its boundaries are. For a laser targeting task, boundary precision is the whole game.
- A real-time inference pipeline, not a notebook. A model that segments correctly but slowly is useless on a moving rig. OpenCV handles capture and pre-processing, PyTorch runs inference, and the resulting weed-vs-crop mask feeds straight into targeting — one continuous loop from camera to laser.
- Fail safe by construction. The asymmetry matters: missing a weed costs a little yield, but hitting a crop destroys the plant. The targeting stage acts on confident weed regions, so uncertainty resolves toward not firing.
Stack
- UNet Designed for dense, pixel-level segmentation and known to work from relatively small annotated datasets — which is the normal situation in agriculture, where labelled RGB + NIR field imagery is scarce and expensive to produce.
- PyTorch Adapting a standard UNet to accept a fused multi-spectral input rather than three RGB channels means editing the model definition directly. PyTorch makes that a few lines instead of a fight with the framework.
- OpenCV Camera capture, channel alignment between the RGB and NIR sources, and per-frame pre-processing — fast enough in C++ under the hood to keep the loop real-time.
- RGB + NIR imagery The input choice that carries the accuracy. Near-infrared reflectance separates vegetation types that look nearly identical in visible light.
Outcome
~90%
segmentation accuracy on RGB + NIR imagery
Real-time
inference pipeline, camera through to targeting
↓ Herbicide
targeted removal in place of blanket spraying
The result I keep coming back to is that the biggest accuracy gain came from the input, not the architecture. Adding the NIR channel did more than any amount of tuning would have. It's the same instinct I use in backend work: before optimising the thing you built, check whether you're feeding it the right information in the first place.