CIMS–macroABM Setup Guide

This guide sets up the CIMS–macroABM linkage from scratch. By the end you will have three repositories cloned, three conda environments created, and a scenario ready to run.

The CIMS–macroABM workflow is independent of CIMS–COPPER–SILVER. You only need CIMS and macroABM (COPPER and SILVER are not required).

1 · Prerequisites

  • Conda (Anaconda or Miniconda) — for the orchestrator and CIMS.
  • uv — for macroABM (pip install uv). Not needed if you choose runner = "conda" for macroABM.
  • git.
  • The macroABM raw input data (raw_data/ — IO tables, HFCS, exchange rates, emission factors, …). Obtain it from the macroABM maintainers if you do not already have it.

2 · Clone the repositories

A convenient layout is to clone the model repos under M3-linkages/dependencies/ (the same convention the GUI installer uses), but any layout works as long as the repo_path entries in your scenario config point at them.

git clone -b cims-macro-linkage https://gitlab.com/sesit/M3-linkages.git
cd M3-linkages
mkdir dependencies

git clone -b main   https://gitlab.com/sesit/cims-models-fork.git dependencies/cims-models-fork
git clone -b cims-macro-linkage https://github.com/uvic-sesit/macroabm-ca.git dependencies/macroabm-ca

:: Optional — only needed to run the LabourABM stage (see step 7)
git clone -b cims-macro-labour-branch https://gitlab.com/sesit/macrocosm/labourabm/model.git dependencies/labourabm

Place (or symlink) the macroABM raw data at dependencies/macroabm-ca/raw_data/.

LabourABM branch. Clone the cims-macro-labour-branch (it carries a numpy≥2 compatibility fix the stage needs). If you instead use macrocosm-sesit, apply the equivalent .values.copy() fix in labour_abm_canada/data_bridge/bridge.py or the ABM will crash with "underlying array is read-only".

3 · Create the environments

The orchestrator and CIMS use conda environments (mirroring the CIMS–COPPER–SILVER pattern). macroABM does not need a conda environment — macroabm-ca is a uv-managed project, and the default runner = "uv" resolves its environment from uv.lock automatically on first run.

:: Orchestration environment (runs linkage_macroabm.py + the runner scripts)
conda create -n m3linkages_env python=3.11
conda activate m3linkages_env
pip install -r requirements.txt
conda deactivate

:: CIMS environment
conda create -n cims_env python=3.13
conda activate cims_env
pip install -r dependencies/cims-models-fork/requirements.txt
conda deactivate

:: macroABM — no conda env needed; just install uv once (any environment)
pip install uv

The conda environment names above must match the env_name /base_env values in your scenario config.toml (see Configuration Reference).

LabourABM environment (optional)

Like macroABM, LabourABM needs no conda environment when you use the default runner = "uv": uv run syncs its dependencies (torch, pandas, …) into a managed venv on first run, and the linkage runner adds h5py automatically. Just have uv installed (already covered above).

To use a conda env instead, create one, install LabourABM editable, and set runner = "conda" + env_name under [Models.LabourABM]:

conda create -n labourabm_env python=3.12
conda activate labourabm_env
pip install -e dependencies/labourabm
pip install h5py
conda deactivate

Prefer a conda env for macroABM instead?

If you would rather run macroABM in a conda env (e.g. on a cluster without uv), create one and set runner = "conda" + env_name in the scenario config:

conda create -n macroabm_env python=3.12
conda activate macroabm_env
pip install -e dependencies/macroabm-ca[dev]
conda deactivate

4 · Configure the scenario

Ready-made scenario folders ship with this branch:

  • scenarios/MacroABM Reference/ — full 2014→2050 linkage run
  • scenarios/MacroABM Smoke/ — same settings; use for quick resume/skip tests

Open scenarios/MacroABM Reference/config.toml and adjust paths and environment names for your machine:

[General]
    base_env = "m3linkages_env"
    models_to_run = ["CIMS", "MacroABM"]
    sim_end_year = 2050

[Models.CIMS]
    repo_path = "dependencies\\cims-models-fork"
    env_name  = "cims_env"
    base_scenario = "Reference"
    warm_start = true
    history_boundary_year = 2020

[Models.MacroABM]
    repo_path      = "dependencies\\macroabm-ca"
    runner         = "uv"        # default; no conda env needed
    python_version = "3.12"
    raw_data_path  = "dependencies\\macroabm-ca\\raw_data"
    warm_start     = true
    history_boundary_year = 2020

To create a new scenario, copy this folder to scenarios/<Your Scenario>/ and edit the config.

5 · (Automatic) CIMS results-logging template

The macroABM extractor needs CIMS to log new_stock and total_stock at the technology level. You do not need to do this manually — cims_macro_runner.py appends the two lines to dependencies/cims-models-fork/results/results_tech.txt if missing, and prints a NOTE: when it does.

This is the only change the linkage makes to anything in the CIMS repo, and it is a data file, not code. If you re-clone or update CIMS and the file gets reset, the runner simply re-applies it next run. See the overview note.

CIMS runtime patch (Python 3.13 / pandas 3.x)

On some newer stacks (notably Python 3.13 with recent pandas/polars), empty CSV cells are read as float NaN. That breaks CIMS in two ways:

  • Emissions_GWP — blank Sub_Context is treated as present, so model build fails with KeyError: 'year_value'.
  • DCC_class (and similar params) — blank year cells become NaN, so CIMS nests the value by context and _dcc_classes() later fails with TypeError: unhashable type: 'dict'.

You do not need to fix this manually — the linkage runners apply scripts/cims_runtime_patch.py automatically before calling CIMS.Model (via cims_warm_restart.py when warm_start = true, or cims_scenario_runner.py otherwise).

To verify the patch on your machine (from M3-linkages, inside cims_env):

python scripts/test_cims_runtime_patch.py

Expected output includes get_ghg_and_emissions OK (the failure mode this patch targets). A full CIMS.Model build OK line confirms end-to-end model load.

6 · (Optional) Adjust the sector mapping

The CIMS↔macroABM classification is defined in two editable CSVs inside the macroABM repo:

  • macro_data/processing/macroabm_cims_data_processing/data/cims_macro_sector_map.csv
  • macro_data/processing/macroabm_cims_data_processing/data/cims_macro_region_map.csv

Edit these to change which CIMS sectors/fuels map to which macroABM industries, which industries form the energy bundle, and which provinces aggregate into the CIMS AT (Atlantic) region. Unmapped rows are skipped with a warning rather than crashing the run. Column meanings are documented in the Configuration Reference.

7 · (Optional) Enable the LabourABM stage

LabourABM runs once, after the CIMS–macroABM loop converges (or hits --max_iterations), and translates the macroABM production result into occupation-level labour-market outcomes. To enable it:

  1. Clone LabourABM (step 2) and have uv installed (step 3).
  2. Add "LabourABM" to models_to_run and add a [Models.LabourABM] section to your scenario config.toml (the MacroABM Reference config ships with a ready-to-use block):

```toml [General] models_to_run = ["CIMS", "MacroABM", "LabourABM"]

[Models.LabourABM] repo_path = "dependencies\labourabm" runner = "uv" # or "conda" / "pip" python_version = "3.12" start_year = 2020 # A baseline macroABM run is needed for a meaningful labour-demand shock, # e.g. use a Reference scenario as the baseline for a Net Zero policy run: # baseline_scenario = "MacroABM Reference" # baseline_h5 = "path\to\baseline\simulation_results.h5" ```

LabourABM computes a per-industry shock as (policy − baseline) / production, so it needs two macroABM production HDF5s: the policy run (this scenario, used automatically) and a baseline run (baseline_scenario / baseline_h5). With no baseline it reuses the policy file (zero shock) — fine to check the plumbing, but set a baseline for real analysis. All keys are documented in the Configuration Reference.

Next steps

Head to the Quick-start tutorial to launch a test iteration.