CSIRO Image2Biomass
I explored how several vision models could work together to estimate pasture biomass from photographs. The project combines image representations, structured features, and stacked regression to predict five related biomass targets.
The problem
A pasture photograph contains a mixture of living vegetation, dry material, and clover. Estimating their mass from appearance requires a model to connect color, texture, and spatial structure with physical measurements.
The project tackles five related outputs: dry green biomass, dry dead biomass, dry clover biomass, green dry matter, and total dry biomass. Predicting those components together gives the image a richer description than a single total.
| Target | Component |
|---|---|
| Dry_Green_g | Dry green biomass |
| Dry_Dead_g | Dry dead biomass |
| Dry_Clover_g | Dry clover biomass |
| GDM_g | Green dry matter |
| Dry_Total_g | Total dry biomass |
My contribution
I designed the four-model vision ensemble, feature pipeline, grouped validation strategy, and Lasso stacking meta-model. My focus was combining representations with different strengths while retaining a common set of five regression targets.
Architecture
The available project description records four branches. DINOv2 combines image features with metadata. A two-stream ConvNeXt processes image halves. SigLIP supplies features to a collection of regressors. DINOv2-Giant supplies another representation for regression. A Lasso meta-model combines their predictions.
These diagrams reconstruct that documented design. Each branch produces the same five targets, giving the final stage a consistent interface across model families.
Image representations
The two-stream branch divides the photograph into left and right views. Separate ConvNeXt encoders process the views, and a fusion stage combines their features. This arrangement preserves local image detail while bringing both halves into one prediction.
The other branches use pretrained vision representations as a starting point for regression. The project description pairs DINOv2 with categorical and numerical metadata and uses SigLIP and DINOv2-Giant features with regression models. Image augmentation and test-time transformations are also part of the documented approach.
The pasture tile is an illustration. It shows where image splitting happens without assigning a biomass measurement to an invented sample.
Combining predictions
Each model returns a five-element prediction vector. Concatenating four vectors produces 20 inputs for the stacking stage. Lasso learns how to combine these inputs into the final set of biomass estimates.
This representation makes the interface between the base models and the meta-model simple to inspect. The models can use different feature extractors while agreeing on the target order. The cells in the figure represent vector positions; their shading carries no measured value.
Evaluation
The project notes describe five-fold grouping by image path, with MAE and per-target R² used to examine regression behavior. Grouping keeps records associated with an image together during the split.
The linked notebook was unavailable during preparation of this page. This case study therefore presents the recorded architecture and schematic fixtures. Numerical comparisons require the original saved outputs and the corresponding evaluation split.
The local fixtures check the four-by-five prediction shape, the 20 stacking inputs, and the shared target order. They make the figures reproducible without running model training.
Notebook
The Kaggle notebook is the project’s original code reference.
Open the CSIRO notebook ↗