Convert to OME-Zarr¶

Convert Micro-Manager TIFF formats and Nikon ND2 datasets to OME-Zarr.

Note

There is also a CLI command for conversion. See the CLI Reference or run iohub convert --help.

Supported inputs¶

Micro-Manager OME-TIFF
Micro-Manager NDTiff
Nikon ND2

Performance¶

Pixels are copied by a pool of worker threads (num_workers, default 4). Each worker converts one frame slab — a block of whole 2D frames of one FOV, bounded in T, C and Z.

NDTiff v3 datasets are read directly from the byte offsets in NDTiff.index (one contiguous read per volume, O_DIRECT where the filesystem allows it, plane metadata decoded from the same bytes), so throughput scales with num_workers until the storage saturates; on network filesystems 16 workers is a good setting. Other formats are read through their reader one slab at a time.

Scheduling across jobs¶

Work items are independent and enumerated deterministically, so a scheduler can split a conversion across processes or cluster jobs:

converter = TIFFConverter(src, dst, version="0.5")
converter.init_store()          # once
n = len(converter.slabs)
# in each job k of K (fresh TIFFConverter per job):
TIFFConverter(src, dst, version="0.5").convert(slice(n * k // K, n * (k + 1) // K))
# once, after every job finished:
TIFFConverter(src, dst, version="0.5").finalize()

Python¶

TIFFConverter ¶

TIFFConverter(
    input_dir: str | Path,
    output_dir: str | Path,
    grid_layout: int = False,
    chunks: tuple[int] | Literal["XY", "XYZ"] | None = None,
    hcs_plate: bool | None = None,
    version: Literal["0.4", "0.5"] = "0.4",
    implementation: str | None = None,
    num_workers: int = DEFAULT_NUM_WORKERS,
)

Convert supported microscopy datasets to OME-Zarr.

Supports Micro-Manager TIFF formats (OME-TIFF and NDTiff) and Nikon ND2. Each FOV is written to a separate well in the plate layout.

Parameters:

input_dir (str | Path) –

Input directory path
output_dir (str | Path) –

Output zarr directory path
grid_layout (bool, default: False ) –

Whether to lay out the positions in a grid-like format based on how the data was acquired (useful for tiled acquisitions), by default False
chunks (tuple[int] or Literal['XY', 'XYZ'], default: None ) –

Chunk size of the output Zarr arrays, by default None (chunk by XYZ volumes or 500 MB size limit, whichever is smaller)
hcs_plate (bool, default: None ) –

Create NGFF HCS layout based on position names from the HCS Site Generator in Micro-Manager (only available for OME-TIFF), and is ignored for other formats, by default None (attempt to apply to OME-TIFF datasets, disable this with False)
version (Literal['0.4', '0.5'], default: '0.4' ) –

OME-NGFF version for the output Zarr store, by default "0.4". "0.4" uses Zarr v2 format; "0.5" uses Zarr v3 format.
implementation (str, default: None ) –

Zarr backend implementation to use for writing. None (default) uses zarr-python. Pass "tensorstore" to write via TensorStore (requires the optional tensorstore dependency).
num_workers (int, default: DEFAULT_NUM_WORKERS ) –

Threads copying pixels concurrently, by default 4. Each in-flight slab holds a few times its size in memory (~150 MB slabs -> ~600 MB per worker). NDTiff datasets are read with byte-range I/O and scale with workers; other formats are read through their reader serially.

Notes

The image plane metadata for each FOV is aggregated into a JSON file, and placed under the Zarr array directory (e.g. /row/column/fov/0/image_plane_metadata.json).

Conversion is a list of independent :class:FrameSlab (blocks of whole frames of one FOV). converter() runs everything; schedulers may instead call :meth:init_store once, :meth:convert on disjoint slab slices in separate processes or jobs, then :meth:finalize once.

slabs `cached` `property` ¶

slabs: list[FrameSlab]

All slabs, in reader order; deterministic for a given dataset and chunking.

init_store ¶

init_store() -> None

Create the OME-Zarr store, positions, empty arrays and root attributes. Run once.

convert ¶

convert(slabs: slice | None = None) -> None

Copy pixels (and plane metadata) for slabs into an existing store.

With slabs=None every slab is converted and each FOV's plane metadata is written when its last slab lands. With a slice, metadata is written as a part file for :meth:finalize to merge, so disjoint slices may run in different processes or jobs concurrently.

finalize ¶

finalize() -> None

Merge plane metadata parts written by sliced :meth:convert runs. Run once, after all slices.

call ¶

__call__() -> None

Runs the conversion.

Examples:

>>> from iohub.convert import TIFFConverter
>>> converter = TIFFConverter("input/path/", "output/path/")
>>> converter()