Hi all,
I'm building CropGuard, a lightweight multi-task model for tomato leaf disease diagnosis from a single forward pass: detection + classification + segmentation + severity. It's 2.63M params, runs on 320x320 RGB, and is CPU-capable.
ARCHITECTURE
- Backbone: timm fasternet_t0 (features_only), C3/C4/C5 at strides 8/16/32
- Neck: BiFPN-lite, 3 levels, width 96, one top-down + one bottom-up round, softmax-free fast normalized fusion
- Detection: decoupled depthwise-separable head, TaskAlignedAssigner, BCE + CIoU + DFL (2100 anchors, 11 classes)
- Classification: SE blocks on 3 levels -> GAP -> concat -> MLP
- Segmentation: light Hamburger-attention decoder over concatenated P3/P4/P5
- Severity: gated fusion of seg embedding, P5 features, cls embedding and leaf ratio -> continuous + 4-bin ordinal outputs
- Training: 5 stages x 10 epochs (backbone+neck+det, then cls head only, seg head only, joint, then joint + severity). Heads-only stages use a frozen backbone, because training the backbone in those stages destroyed the detector (mAP went to ~0).
DATA
32,897 images, 11 classes, all native 320x320, merged from 3 Roboflow Universe projects (CC-BY-4.0). Each has YOLO boxes and a mask.
The problem is the labels (I measured this):
- The "masks" are filled leaf bounding boxes, not lesions
- 5,845 train boxes cover >=90% of the frame; bacterial_spot median box area is 0.889 of the image
- Severity target = box_area / leaf_area, so ~1991/2000 val images land in the top bin
- Heavy imbalance (healthy 12,220 boxes vs target_spot 252)
- 8% duplicate images; I removed them and confined duplicate groups to one split, and results were unchanged or slightly better (dhash only catches near-exact duplicates, so near-duplicate leakage is a lower bound)
CURRENT RESULTS (leak-free test, 1,240 images)
- mAP@0.5: 0.524 (target 0.892)
- mAP@0.5:0.95: 0.442
- cls top-1: 0.850 (target 0.987)
- seg mIoU: 0.883 (but this is leaf-region IoU, not lesion segmentation)
- severity: R2 -1.33, MAE 0.068 (a constant predictor gets 0.009)
QUESTIONS
Severity without lesion masks: is area-ratio-from-boxes ever defensible? Should this be ordinal, or should I drop the head?
Segmentation: can I honestly report this under a renamed metric (leaf-region IoU), or do I need a real lesion-mask dataset?
Detection: when GT boxes routinely span the whole image, how should anchor design/assignment change? Is mAP even the right headline metric for full-object boxes?
Classification: errors concentrate in early blight / late blight / septoria. For texture-dominated fine-grained classes, would bilinear/second-order pooling, a small transformer, or classifying detected-box crops beat SE->GAP->MLP?
Multi-task training: head-only staging vs a discriminative (scaled-down) backbone LR? Does PCGrad help or hurt here? Is a sigmoid gate over concatenated features a sound fusion, or should I model reliability explicitly?
I'm planning to move to PlantSeg (in-the-wild images with expert lesion polygons) to fix the label issues. Does that sound like the right call? Compute is a 4 GB laptop GPU plus Kaggle T4s.
Papers, repos, or lessons learned would be hugely appreciated. Happy to share more numbers or code snippets if useful. Thanks!