.png&w=640&q=75)
Standardize WSI and ROI data, connect pluggable models, run gigapixel inference, and evaluate results at slide level, with reproducible configurations from experiment to deployment.
Triplet-SAM, trained and deployed using the PuzzleSeg framework, enables multi-scale interactive feature identification and uses natural-language reasoning to automatically segment relevant regions.
Three shared interfaces for data, execution, and experiment provenance span training, gigapixel inference, slide-level evaluation, and reproduction.
Preserve consistent coordinate and scale semantics from ingestion to gigapixel inference.
Level-0 coordinates and MPP unify reading, tiling, and preprocessing.
Tissue filtering, window fusion, and sparse storage balance memory and throughput.
Connect models, training strategies, and distributed runtimes through stable interfaces.
SegmentationOutput and registries connect multiple model families by configuration.
Compose losses, AMP, DDP, FSDP, ZeRO-3, and PEFT / LoRA.
Unify metrics, configuration, and tracking so every experiment is reliable and auditable.
Aggregate patch predictions by slide while decoupling inference, metrics, and visualization.
Checkpoints retain configuration alongside experiment tracking and deterministic settings.
Control, execution, and artifact layers preserve WSI + ROI, Level-0 spatial semantics, and Patch-to-Slide traceability.
Hydra YAML + CLI OverridesExperiment configs and CLI overrides managed centrally; config.yaml stored with the checkpoint · Registry: Registry-driven, extended by config · Composable: Modules compose by config · Reproducible: Experiments reproduce
Connect WSI, ROI, and public or private pathology datasets
Unify data representation while preserving Level-0 coordinate semantics
Unify models, training, and distributed execution
One engine covers Patch / WSI with sync and async execution
Retain models, configs, metrics, and visual results
Task type describes what a model outputs; input / prompt mode describes what drives the segmentation. They are independent dimensions, and integration status is tracked separately from task capability.
Task-specific training with a stable semantic segmentation baseline
IntegratedText or visual prompt segmentation with LoRA tuning and Patch / WSI evaluation
IntegratedGenerates masks from images and natural-language reasoning instructions with LoRA and ZeRO-3
IntegratedData schema and output interfaces support box annotations; a dedicated detection model is pending
PendingPanNuke Patch and Lizard WSI are reported separately under different protocols
Results are reported separately by dataset and protocol. Values from different datasets are not combined into a single benchmark.
Same 5 datasets · same single-tile record granularity
GlaS · same PuzzleSeg U-Net · Patch vs. Sliding-window aggregation
Magnified axis (.82—.95); Sliding-window closes more than half the gap to nnU-Net.
Triplet-V1 generates a structured pathology description. A resolver converts supported findings into concept prompts, and SAM3 grounds each prompt to image regions. This links selected report items to inspectable spatial masks.
A pathology multimodal model reads the slide scale by scale and writes a structured description.
Image→Structured findingsEach report item becomes a concept prompt that a segmentation model can consume.
Structured findings→Concept promptsTriplet-V1 and SAM3 reason jointly to produce one spatial mask per concept.
Concept prompts→Spatial masks
One example at each of three scales — 10× / 40× / 200×.
Selected report items map to localizable, inspectable mask regions.
Left is the raw field, center is the structured description from Triplet-V1, right is the mask Triplet-SAM grounds from it. The three rows are three magnifications of the same ROI, and the highlighted entry in the center column is what each mask was grounded on.
One breast core-biopsy WSI, narrowed from the whole slide to the ROI shared by the three scales below.
PuzzleSeg has been integrated into the PuzzleOmni pathology operating system, enabling localized research and development of both large and lightweight pathology segmentation models, with developed models further deployable to the PuzzleVision slide viewer for use in real-world clinical settings.
