Measure leaf area, length, and width in real-world units from a photo of leaves laid on a printed calibration template. MATS finds four fiducial markers with RF-DETR, corrects perspective, segments each leaf with a fast Otsu threshold by default (or the heavier BiRefNet model for tougher backgrounds), and writes a measurements CSV.
Pipeline in one line: detect markers → perspective-correct → segment leaf → measure → CSV.
Companion code for the manuscript (target journal: Plant Phenomics). BiRefNet is optional and runs entirely from a locally installed checkpoint (see Model weights). For USDA users the model weights are hosted on Agdatacommons and the pipeline is available on SciNET
MATS needs only Python ≥ 3.9 — QR codes are decoded with OpenCV, so the default install pulls everything from wheels with no system libraries and no conda required.
pip (recommended):
git clone https://github.com/Breeding-Insight/Morphometric-Analysis-Toolbox-for-Segmentation.git
cd Morphometric-Analysis-Toolbox-for-Segmentation
pip install -e ".[app]" # ".[app]" adds the Streamlit GUIRobust QR reading (optional). OpenCV decodes clean codes reliably. For
tougher photos (glare, skew, blur), add the pyzbar + qreader fallbacks:
pip install -e ".[app,qr]"This enables QReader without Conda. To enable the additional pyzbar
fallback, install its native zbar library with your existing environment:
# Linux: apt install libzbar0
# macOS: brew install zbar
# Existing Conda environment only: conda install -c conda-forge zbarIf a code can't be read, the pipeline continues — pass the finished sheet size
with --sheet-dimensions (for example, --sheet-dimensions 12x12in), so
enhanced QR is a convenience rather than a requirement.
Then fetch the model weights once and confirm the environment:
mats fetch-weights # fetches the ~134 MB RF-DETR checkpoint (mandatory, default)
mats fetch-weights --only birefnet --source lfs # optional: explicitly fetch the ~2.65 GB BiRefNet checkpoint
mats doctor # checks weights, GPU/CPU device, QR backendsMATS installs in a lightweight operating configuration for convenience. The
standard app includes fast Otsu segmentation and OpenCV's built-in QR reader,
but it does not automatically download the optional ~2.65 GB BiRefNet
checkpoint or install the pyzbar/QReader robust-QR fallbacks. The required
~134 MB RF-DETR marker checkpoint is also fetched explicitly with mats fetch-weights so installations never hide a model download.
This keeps the initial network and disk footprint predictable, avoids native
zbar failures on managed machines, and works better on HPC systems and
restricted networks. Start with the standard path, then add only what the
photographs require:
| Capability | Included initially | Add when needed |
|---|---|---|
| Otsu leaf segmentation | Yes | Nothing |
| Clear QR codes with OpenCV | Yes | Nothing |
| RF-DETR marker detection | Code included | mats fetch-weights |
| BiRefNet segmentation | No checkpoint | mats fetch-weights --only birefnet --source lfs |
| Robust QR fallbacks | No | pip install "mats-morpho[app,qr]" |
QReader can download its detector model when that fallback is first used.
pyzbar requires the native zbar library described above. The app's Preflight
and setup pages show exactly which readers and checkpoints are available before
a run.
- I want to click buttons → Using the app
- I want to script it → Using the command line
Both run the exact same pipeline and produce the same measurements.
mats appThis opens the Streamlit GUI in your browser. From there:
- Pick images — a local folder, or drag-and-drop uploads.
- Set the scale — enter the finished printed sheet's width, height, and
unit (for example,
12 x 12 in). MATS applies the Template Creator margin rules and derives the marker-centre calibration area automatically. Or tick Variable dimensions, read QR code to read each image's calibration. - Choose segmentation — Otsu threshold (fast, default) or BiRefNet (accurate when its optional local checkpoint is installed).
- Choose workers — the app detects the CPUs assigned to it. One worker uses CUDA/MPS when available; two or more workers use parallel CPU processing and disable CUDA/MPS for that run. A colored warning light shows CPU allocation; counts above 75% require a one-run break-glass acknowledgement.
- Run, then preview results and download a CSV or a ZIP of masks + boxes.
Getting oriented. The app's Help page (sidebar) ships three annotated sample photographs — including a real QR-read failure that shows why printed sheet entry is the most consistent option — a photography checklist, and a glossary for every results-CSV column. See docs/gui.md.
Printing templates. The app has a Template Creator page (in the sidebar) that accepts only the finished sheet's width and length, then automatically sizes the observation area and corner markers. Download the print-ready PDF or an editable Adobe InDesign IDML; the PDF is recommended for final printing. Print at 100% scale (no "fit to page"), lay your leaves inside the box, and photograph it flat. See docs/templates.md.
mats run -i ./images -o ./out -r results.csv --sheet-dimensions 12x12inCommon options (full reference in docs/cli.md):
| Flag | Meaning | Default |
|---|---|---|
-i, --input_dir |
Folder of images to measure | prompt |
-o, --output_dir |
Where masks / target boxes are written | prompt |
-r, --results_path |
Measurement CSV path | ./leaf_morpho_results.csv |
--sheet-dimensions |
Finished Template Creator sheet size, <w>x<h><unit> |
read from QR |
-t, --template_dimensions |
Legacy/custom marker-centre calibration area | unused |
--mask-method |
birefnet (accurate, GPU) or threshold (fast) |
threshold |
--threshold-level |
auto (Otsu) / low / medium / high |
auto |
--csv-schema |
full (area/width/length + per-axis pixels-per-selected-unit) or compact |
full |
--results-unit |
Measurement-output unit: mm, cm, or in |
cm |
-w, --workers |
Parallel workers (threshold path only) | auto |
--save-axes |
Also save length/width overlay images for QC | off |
Choosing a segmentation method. threshold (Otsu) is the default — fast,
no GPU, no extra download, and good for clean, high-contrast backgrounds where
a leaf sits on plain white. birefnet is more accurate on cluttered or
low-contrast backgrounds and uses a GPU when available (CPU works but is
slow), at the cost of the ~2.65 GB checkpoint — fetch it once with
mats fetch-weights --only birefnet.
Per image, in the output folder:
{sample_id}_target_box.jpg— the perspective-corrected observation box{sample_id}_mask.png— the leaf segmentation mask
Plus a measurements CSV. Choose mm, cm (the default), or in with
--results-unit in the CLI or the Result units control in the app. The
selection changes results, dashboard labels, and unit-bearing CSV column names;
it does not change calibration math. Two schemas:
- full (default, research schema) —
sample_id, leaf_area_cm2, width_cm, length_cm, px_per_cm_width, px_per_cm_height, scale_aspect_ratio, source. When dimensions are read from QR codes, it also appends a trace column for OpenCV and each optional decoder installed locally, showing which decoder succeeded or whether a fallback failed or was unused. Scaling is anisotropic: the x-extent (width_cm) is divided bypx_per_cm_width, the y-extent (length_cm) bypx_per_cm_height, and area by their product — each axis calibrated independently against the template, rather than one averaged scalar applied to everything.scale_aspect_ratio(px_per_cm_width / px_per_cm_height) is a QC signal: it should sit near 1.0, and a value far from 1.0 flags a calibration problem (skewed template print, lens distortion, a non-planar sheet) worth investigating. - compact —
sample_id, area_cm2, width_cm, length_cmby default. With millimeters or inches selected,cmis replaced consistently in the measurement column names.
A leaf_morpho_failures.csv records per-image warnings and failures.
Migration note: earlier versions reported three isotropic scale conventions (
*_meanscale,*_widthscale,*_heightscale). Old CSVs remain usable — the newleaf_area_cm2can be recovered from an old row withleaf_area_cm2_widthscale * (px_per_cm_width / px_per_cm_height), and the newwidth_cm/length_cmequal the oldwidth_cm_widthscale/length_cm_heightscale.
The checkpoints are tracked in this repository with Git LFS:
| Model | File | Size |
|---|---|---|
| RF-DETR marker detector | rf_detr_marker.pth |
~134 MB |
| BiRefNet leaf segmenter | birefnet_leaf.pth |
~2.65 GB |
RF-DETR is available in a normal checkout. BiRefNet is LFS-tracked but excluded from the default clone, so it is downloaded only through an explicit action:
- Otsu (default) — needs no BiRefNet checkpoint and never downloads one.
- BiRefNet (optional) — fetch explicitly with
mats fetch-weights --only birefnet --source lfs, or use the setup page. - Shared filesystem — set
MATS_WEIGHTS_DIR(e.g. a SCINet/projectpath) to read weights in place with no per-user copy.
Full detail and checksums: docs/weights.md.
An Open OnDemand Batch Connect app that serves the GUI on a compute node is in deploy/ondemand/mats/. See its README and docs/hpc.md.
MATS chains two models. RF-DETR (fine-tuned, single "Marker" class) detects
the four corner fiducials at 1120×1120 px; their centroids define a homography
that rectifies the observation box. The rectified box's pixel width and height
are compared against the template's known physical size, independently per
axis, to fix px_per_cm_width and px_per_cm_height. BiRefNet (fine-tuned
for leaf foreground) then segments the leaf, from which area (pixel count) and
length/width (bounding dimensions) are computed and converted to centimeters
using their respective axis scale. A classic Otsu threshold is offered as a
fast alternative to BiRefNet. See the manuscript for training and evaluation
detail.
Run MATS doctor first — it reports most of these.
- QR code not read / measurements need a scale — the default OpenCV decoder
couldn't read the code. Pass the finished sheet size with
--sheet-dimensions(e.g.--sheet-dimensions 12x12in), or add enhanced QR reading:pip install -e ".[qr]"plus thezbarsystem lib (Linux:apt install libzbar0; macOS:brew install zbar; conda:conda install -c conda-forge zbar). - CUDA out of memory (only relevant with
--mask-method birefnet) — process in smaller batches, or use--mask-method threshold(the default). - No markers detected — check print quality and that the marker color matches the template (the Template Creator uses the trained color); make sure all four corners are in frame.
- Blank page on Open OnDemand — almost always the reverse-proxy
baseUrlPathmismatch; see deploy/ondemand/mats/README.md.
If you use MATS, please cite the manuscript. See CITATION.cff.
MIT. The pipeline builds on RF-DETR (Apache-2.0) and BiRefNet (MIT); see docs/weights.md for model provenance.