ML engineering project
CSIRO Image2Biomass Prediction
CSIRO Image2Biomass is a multi-model ensemble solution developed for a Kaggle competition predicting plant biomass from pasture images. This project achieves non-destructive crop monitoring by predicting 5 biomass targets: Dry_Green_g, Dry_Dead_g, Dry_Clover_g, GDM_g, and Dry_Total_g. **The Challenge:** Traditional biomass measurement requires destructive sampling - physically harvesting and weighing plants. The competition required predicting multiple biomass components simultaneously from a single image, incorporating metadata like NDVI, plant height, species, and sampling date. **Multi-Model Stacking Ensemble:** • **Model 1: DinoNNModel** - DINOv2-base backbone with metadata embeddings (state, species, month) and feature fusion MLP. Includes optional LoRA adapters for efficient fine-tuning • **Model 2: TwoStreamConvNeXt** - Splits images into left/right halves processed by separate ConvNeXt-tiny backbones, then fuses features through a deep fusion network. Uses Test-Time Augmentation (TTA) with 5 augmentations • **Model 3: SigLIPGBModel** - SigLIP vision features fed into a 5-model ensemble: Ridge, two GradientBoosting variants, AdaBoost, and CatBoost regressors • **Model 4: DinoLassoModel** - DINOv2-Giant (1536-dim features) with Lasso and AdaBoost regression • **Meta-Model** - Stacking ensemble using Lasso to optimally combine predictions from all 4 models **Key Techniques:** • **Data Augmentation** - Albumentations pipeline with geometric transforms, color jitter, elastic distortion, and CoarseDropout regularization • **MixupCutmix** - Training regularization combining Mixup (α=0.4) and Cutmix for improved generalization • **Group K-Fold CV** - 5-fold cross-validation grouped by image path to prevent data leakage • **OneCycleLR Scheduler** - Cosine annealing with warmup for optimal learning rate scheduling **Results:** • **Stacking Ensemble MAE: 10.03** (21.37% improvement over best individual model) • **Weighted R²: 0.9998** • Best individual model (SigLIP + Multi-Boosting): MAE 12.76 • Per-target R² ranges from 0.36 (Dry_Dead) to 0.74 (Dry_Clover) **Tech Stack:** Built with **PyTorch**, **Hugging Face Transformers** (DINOv2, SigLIP, ConvNeXt), **scikit-learn**, **CatBoost**, **Albumentations**, and **Pandas**.
Problem
Pasture biomass traditionally requires destructive sampling; the competition asked models to estimate five biomass targets from images and metadata.
My role
Designed and evaluated the four-model vision ensemble, feature pipeline, grouped validation strategy, and Lasso stacking meta-model.
Approach
Stack DINOv2, ConvNeXt, SigLIP/boosting, and DINOv2-Giant/Lasso pipelines with image augmentation, metadata features, TTA, and grouped cross-validation.
Outcome
Recorded MAE 10.03, a 21.37% improvement over the best individual model in the project evaluation, with weighted R² 0.9998.
Implementation
- Combined DINOv2, ConvNeXt, SigLIP, boosting models, and Lasso in a stacking ensemble.
- Used Albumentations, Mixup/CutMix, test-time augmentation, and metadata encoders.
- Grouped five-fold validation by image path.
Evaluation
- Compared all four base models against the stacking meta-model.
- Tracked MAE and per-target as well as weighted R².
Results
- Stacking MAE: 10.03 versus 12.76 for the best individual model.
- Weighted R²: 0.9998 in the documented project evaluation.
Constraints
- Five related regression targets had different error profiles.
- Repeated image paths made leakage-safe grouping essential.
- The solution had to combine image and structured metadata features.
Tradeoffs
- The ensemble improves aggregate error while increasing inference cost and operational complexity.
- Aggregate R² can obscure weaker performance on individual biomass targets.
Next improvements
- Report uncertainty and additional held-out robustness checks by geography and species.
- Distill the ensemble for more practical field inference.