Tutorials

Step-by-step Jupyter-notebook-style tutorials covering the gunz-cm processing pipeline: load → convert → visualize → random downsample.

The tutorials live as Jupyter notebooks in notebooks/tutorials/ at the root of the gunz-cm repository. The pages below are generated from those notebooks at build time (via nbconvert --to markdown --execute) so the rendered HTML always shows fresh outputs.

To run the tutorials locally with fresh outputs:

.. code-block:: bash

git clone https://github.com/sXperfect/gunz-cm.git cd gunz-cm mamba activate gunz_cm jupyter lab notebooks/

What you need

1. Python 3.11+ (gunz-cm does not support earlier versions)

2. gunz-cm installed in a virtual environment — see :doc:installation for setup instructions

3. The 18 tutorial notebooks in notebooks/tutorials/ (cloned from the repository)

4. (Optional) Sample data — every tutorial generates synthetic data inline, so you can run them without real Hi-C files

Core Operations (Getting Started)

Read data from diverse formats, convert between formats, and inspect/visualize Hi-C contact maps.

• Tutorial: Load HIC contact matrix data
• Tutorial: Load COOL / MCOOL contact matrix data
• Tutorial: Convert contact matrices to alternate formats
• Tutorial: Visualizing Hi-C contact matrices with gunz_cm.visualizations
• Tutorial: Load a real Hi-C dataset via GUNZ_CM_TUTORIAL_DATAThis tutorial walks through loading the canonical public GM12878chr1 Hi-C dataset from 4DNucleome (accession 4DNFI1UEG1HD)using the gunz_cm.loaders API. The path to the file isresolved at runtime from the GUNZ_CM_TUTORIAL_DATA env var, sono contributor’s filesystem path is ever hardcoded.## SetupBefore running, populate the data directory once:bashmkdir -p ~/gunz_cm_tutorial_datapython scripts/download_tutorial_data.py --name gm12878_chr1_1mb \    --target ~/gunz_cm_tutorial_dataexport GUNZ_CM_TUTORIAL_DATA=~/gunz_cm_tutorial_data## Learning Objectives* Resolve a canonical dataset via notebooks/_tutorial_data.load_tutorial_dataset.* Load a .hic contact matrix at a specific resolution + chromosome.* Inspect metadata (resolutions, balancing, chromosome info).* Apply KR balancing and read the matrix back as COO.## Estimated TimeApproximately 5 minutes after the data is downloaded.## Prerequisites* gunz-cm installed (this repo).* GUNZ_CM_TUTORIAL_DATA set to a directory containing 4DNFI1UEG1HD.hic.
• Tutorial: Load a real Micro-C dataset via GUNZ_CM_TUTORIAL_DATAThis tutorial walks through loading the canonical public mESCMicro-C chr19 dataset from 4DNucleome (accession4DNFI9GMP7J3) using the gunz_cm.loaders API.## Setupbashmkdir -p ~/gunz_cm_tutorial_datapython scripts/download_tutorial_data.py --name mESC_microc_chr19_50kb \    --target ~/gunz_cm_tutorial_dataexport GUNZ_CM_TUTORIAL_DATA=~/gunz_cm_tutorial_data## Learning Objectives* Resolve a multi-resolution .mcool file via the helper.* Inspect the available zoom levels (get_resolutions).* Load at the finest 50 kb resolution on chr19 with KR balancing.

Preprocessing & Normalization

Clean, balance, and downsample contact matrices for biologically accurate analyses.

• Tutorial: Filter and balance a contact matrix
• Tutorial: KR and ICE balancing for Hi-C contact matrices
• Tutorial: Downsample a Hi-C contact matrix to a coarser resolution
• Tutorial: Preprocessing Pipeline (matrices module)

3D Genome Reconstruction & Spatial Metrics

Reconstruct 3D chromosome models, assess reconstruction quality, and inspect standard datasets.

• Tutorial: 3D Structure Visualization
• Tutorial: The Full Procrustes Surface
• Tutorial: Reconstruction Performance Evaluators
• Tutorial: Typed Data Shapes in gunz_cm.structs
• Tutorial: PyTorch Dataset Zoo

Advanced GZCM Format & Deep Learning

Dive into custom codecs, adaptive picker metrics, and high-concurrency PyTorch training.

• Tutorial: Tutorial 25: GZCM Codec Registry and the v5.1 Wire-Format Contract
• Tutorial: Tutorial 26: GZCM Codec Picker and Adaptive Codec Selection
• Tutorial: Tutorial 27: Writing GZCM v4 Files with Adaptive Codec Picker
• Tutorial: Tutorial 28: Reading GZCM v4 Files with the LRU Tile Cache
• Tutorial: Tutorial 29: Using GzcmDataset with PyTorch DataLoader
• Tutorial: Tutorial 30: Sparse-COO DataLoader with SparseCODataset.from_coo
• Tutorial: Tutorial 31: End-to-End In-Memory Hi-C Pipeline (synth -> model)

Why the tutorials are pre-rendered

The MyST Notebook ecosystem (myst_parser + myst_nb) has a known collision documented in commit da827f6: both extensions try to register the same config value, and no version combination resolves it. We work around this by:

1. Executing the tutorials locally with jupyter nbconvert --execute (which captures current cell outputs)

2. Converting to MyST Markdown via nbconvert --to markdown

3. Rendering the static .md in Sphinx via the standard myst_parser

This guarantees the tutorials show on the website with the right code and the right outputs, even without myst_nb.

Pre-existing bugs documented in the tutorials

The tutorials work around the following pre-existing gunz-cm bugs (per the CHANGELOG):

• get_resolutions raises LoaderError for single-resolution .cool files

• get_balancing facade has an AttributeError for COOLER format

• Balancing.VC enum value exists but no vc_normalize function ships

• scipy.spatial.distance.mdscale was removed in SciPy 1.16 (use sklearn.manifold.MDS instead)

• GZCM v2: padded to block_size; original_shape not stored

• GZCM v3 + CMC: upper-triangular only (lossy in lower)

• NPY loader returns raw ndarray (not ContactMatrix)

• CSV loader expects base-pair coordinates (not bin IDs)

• display_contact_map and display_compartment_map are dead stubs in __init__.py

Where to go next

• :doc:installation — full installation guide

• :doc:quickstart — minimal 5-minute working example

• :doc:concepts — mental model behind the library

• :doc:modules — full API reference