CIMS–macroABM Configuration Reference¶

The CIMS–macroABM workflow reads a single config.toml from scenarios/<scenario>/config.toml. Only the [General], [Models.CIMS] and [Models.MacroABM] sections are used.

`[General]`¶

Key	Type	Description
`base_env`	string	Conda env that runs `linkage_macroabm.py` and the `scripts/` runners.
`models_to_run`	list	`["CIMS", "MacroABM"]` for the core workflow. Add `"LabourABM"` (i.e. `["CIMS", "MacroABM", "LabourABM"]`) to run the LabourABM stage once after the CIMS–macroABM loop converges (or hits `--max_iterations`). See `[Models.LabourABM]`.
`sim_end_year`	int	Shared linkage horizon (default `2050`). CIMS solves milestone years up to this value; macroABM simulates through this calendar year.

[General]
    base_env = "m3linkages_env"
    models_to_run = ["CIMS", "MacroABM"]
    sim_end_year = 2050

`[Models.CIMS]`¶

Key	Type	Description
`repo_path`	string	Path to `cims-models-fork`. Relative paths are also resolved against `../../`.
`env_name`	string	Conda env for CIMS.
`base_scenario`	string	CIMS scenario folder to start from (e.g. `Reference`).
`warm_start`	bool	When `true`, use `scripts/cims_warm_restart.py` instead of a full `scenario_runner.py` solve. Solves history once, checkpoints it, then re-solves only post-`history_boundary_year` years each iteration. Default `false`.
`history_boundary_year`	int	Last milestone year treated as fixed history when `warm_start = true` (default `2020`). Years `<=` this value are solved once and saved to `<scenario>_cims_checkpoint.pkl`; only later milestone years are re-solved on subsequent iterations.
`[Models.CIMS.Config]`	table	Optional overrides merged into the CIMS scenario `config.toml` (same structure as a CIMS config).

[Models.CIMS]
    repo_path = "dependencies\\cims-models-fork"
    env_name  = "cims_env"
    base_scenario = "Reference"
    warm_start = true              # optional; default false
    history_boundary_year = 2020     # only used when warm_start = true

    [Models.CIMS.Config]
        [Models.CIMS.Config.Capacity_Expansion_Linkage]
            OBPS_flag = true
            CER_flag = true
            coal_phase_out = true

CIMS warm-restart (optional efficiency mode)¶

When warm_start = true, cims_macro_runner.py invokes scripts/cims_warm_restart.py instead of CIMS' scenario_runner.py. The mechanism (validated in a spike test):

First run (cold): solve all milestone years <= history_boundary_year (e.g. 2000–2020), save the model to <scenario_root>/<scenario>_cims_checkpoint.pkl, then solve the remaining future years (e.g. 2025–2050).
Later iterations (warm): load the checkpoint, inject this iteration's macroabm_production feedback for future years via set_param, and re-solve only the post-boundary milestone years.

The checkpoint is shared across all iterations of a scenario (history is feedback-independent). Re-solving future years from the checkpoint reproduces a full CIMS solve bit-for-bit, while cutting per-iteration runtime roughly in proportion to the number of years skipped.

Important: model.years must stay the full milestone list during a warm-restart (CIMS enumerates stock vintages from it). The warm-restart helper keeps model.years intact and solves only the target years — do not truncate year_list in the config to achieve this. Instead set [General].sim_end_year in the linkage config; the runner passes --years-to-solve for post-boundary milestones up to that year.

Set warm_start = false (or omit it) to fall back to a full scenario_runner.py solve every iteration.

`[Models.MacroABM]`¶

Key	Type	Description
`repo_path`	string	Path to the `macroabm-ca` repo (cims-macro-linkage branch).
`runner`	string	How to launch the driver: `"uv"` (default — `macroabm-ca` is uv-managed, no conda env needed) or `"conda"`.
`python_version`	string	Python version passed to `uv run --python` (default `3.12`). Only used when `runner = "uv"`.
`env_name`	string	Conda env for macroABM. Required only when `runner = "conda"`.
`raw_data_path`	string	macroABM raw_data root used to build the data pickle (must contain `icio/`, `hfcs/`, `ihs_markit_goods_criticality/`, etc.). Do not point at the `icio/` subfolder alone — the provincial IO table is resolved automatically from `raw_data/icio/`.
`warm_start`	bool	When `true`, checkpoint macroABM at each CIMS milestone year and restart from the earliest non-converged period on later iterations. Defaults to the CIMS `warm_start` value when unset.
`history_boundary_year`	int	Last milestone year treated as fixed history (default `2020`, aligned with CIMS). Periods ending at or before this year are excluded from the convergence check.
`checkpoint_retention`	string	How milestone checkpoints under `<scenario>_macroabm_checkpoints/` are retained: `"rolling"` (default) keeps only the latest milestone file plus the shared history checkpoint; `"all"` keeps every milestone file (can use 100+ GB). With `"rolling"`, warm restarts from an intermediate milestone replay from the history checkpoint instead of loading a missing file.
`save_h5_on_convergence`	bool	When `true` (default), write a full HDF5 results file to the converged iteration's `macroabm/` folder after the linkage finishes. No HDF5 is written during non-converged iterations.
`h5_filename`	string	HDF5 filename used when `save_h5_on_convergence = true` (default `simulation_results.h5`).
`save_shallow_h5`	bool	When `true` (default), write a compact summary HDF5 on every macroABM run with per-province time series of GDP, unemployment, inflation, etc.
`shallow_h5_filename`	string	Shallow summary HDF5 filename (default `simulation_shallow.h5`).

`[Models.MacroABM.Config]`¶

Key	Default	Description
`sim_start_year`	`2014`	Calendar year of simulation step 0.
`steps_per_year`	`4`	Simulation steps per year (quarterly).
`cims_base_year`	`2015`	First CIMS milestone year is `cims_base_year + cims_year_step`.
`cims_year_step`	`5`	Interval (years) between CIMS milestone years.
`seed`	`0`	Random seed for the macroABM run.
`sector_map`	(unset)	Optional path to override the default sector-map CSV.
`region_map`	(unset)	Optional path to override the default region-map CSV.

`[Models.MacroABM.Config.Linkage]`¶

Controls how strongly CIMS engineering signals nudge macroABM productivity at each milestone year via firms.link(). Values are fractional damping steps applied per linkage call (e.g. 0.10 moves productivity by ±10% toward the CIMS-implied direction, not a full jump to the target).

Key	Default	Description
`intermediate_factor`	`0.10`	Damping step for intermediate (energy) input productivity when CIMS requested quantities differ from macro use.
`capital_factor`	`0.10`	Damping step for capital-input productivity when macro capital differs from the CIMS investment signal. No adjustment is applied when the CIMS energy-investment total for an industry is zero.
`capital_investment_boost`	`0.10`	Extra multiplier applied to negative capital productivity adjustments.

`[Models.MacroABM.Convergence]`¶

Key	Default	Description
`gdp_growth_tolerance`	`0.10`	The loop stops when per-period GDP growth changes by less than this fraction between iterations. Non-converged periods are written to `<scenario>_linkage_state.json`.
`mode`	`"national"`	Convergence scope: `"national"` compares summed provincial GDP growth per period (one check per period); `"provincial"` requires every province to pass for every period (legacy). National mode reads provincial GDP levels from `simulation_shallow.h5` (default) and records them under a `"national"` key in `gdp_growth_by_period.json`.

macroABM warm-restart (optional efficiency mode)¶

When warm_start = true, run_cims_linkage.py:

First run (cold): simulates through history_boundary_year, saves <scenario>_macroabm_history_checkpoint.pkl, then simulates through sim_end_year. Per-milestone checkpoints are saved under <scenario>_macroabm_checkpoints/ (see checkpoint_retention below).
Later iterations (warm): loads the checkpoint at the start of the earliest non-converged period and simulates forward from there with updated CIMS linkage inputs. With checkpoint_retention = "rolling", intermediate milestone files may have been deleted; the driver replays from the history checkpoint when needed.

Each checkpoint_<year>.pkl is a full simulation snapshot through that calendar year (all agent state and accumulated time series), not an incremental delta. Later checkpoints are larger but are not substitutes for earlier ones — you cannot restart from milestone 2025 using only checkpoint_2050.pkl.

Set checkpoint_retention = "rolling" (default) to delete superseded milestone files as newer ones are written, leaving roughly two large files on disk: <scenario>_macroabm_history_checkpoint.pkl and the latest checkpoint_<year>.pkl. Set checkpoint_retention = "all" only if you need every milestone file kept for manual inspection or offline restarts.

Set warm_start = false to run the full 2014→sim_end_year simulation every iteration.

HDF5 results on convergence¶

By default (save_h5_on_convergence = true) the linkage does not write a full HDF5 file on every macroABM run. Instead, after the convergence check passes, scripts/macroabm_export_h5.py exports <iteration>/macroabm/simulation_results.h5 from:

The warm-start milestone checkpoint at the final CIMS milestone year (when warm_start = true), or
A per-iteration simulation_export.pkl fallback (when warm_start = false).

Set save_h5_on_convergence = false to disable the export entirely.

Shallow HDF5 on every iteration (debugging)¶

When save_shallow_h5 = true (default), each macroABM run writes <iteration>/macroabm/simulation_shallow.h5 — a compact file with per-province time series of high-level indicators (gross output, unemployment rate, CPI/PPI, imports/exports, wages, etc.) via Simulation.shallow_hdf_save. This is much smaller than the full HDF5 and is available on non-converged iterations for debugging. The full simulation_results.h5 is still written only on convergence (when save_h5_on_convergence = true).

Per-period convergence¶

The convergence check compares GDP growth for each CIMS milestone period (e.g. 2020-2025) between iterations. By default (mode = "national") provincial GDP levels at period boundaries are summed and one national growth rate is computed per period; the loop stops when every period's national rate changes by less than gdp_growth_tolerance. Set mode = "provincial" to restore the legacy rule requiring every province to pass independently. When a period fails, the next iteration warm-restarts both CIMS and macroABM from that period's start milestone rather than from the base year. State is tracked in <scenario>_linkage_state.json at the scenario root.

[General]
    base_env = "m3linkages_env"
    models_to_run = ["CIMS", "MacroABM"]
    sim_end_year = 2050

[Models.MacroABM]
    repo_path      = "dependencies\\macroabm-ca"
    runner         = "uv"        # or "conda"
    python_version = "3.12"      # used when runner = "uv"
    # env_name     = "macroabm_env"   # required only when runner = "conda"
    raw_data_path  = "dependencies\\macroabm-ca\\raw_data"
    warm_start     = true
    history_boundary_year = 2020
    save_h5_on_convergence = true
    h5_filename = "simulation_results.h5"
    save_shallow_h5 = true
    shallow_h5_filename = "simulation_shallow.h5"

    [Models.MacroABM.Config]
        sim_start_year = 2014
        steps_per_year = 4
        cims_base_year = 2015
        cims_year_step = 5
        seed = 0

    [Models.MacroABM.Config.Linkage]
        intermediate_factor = 0.10
        capital_factor = 0.10
        capital_investment_boost = 0.10

    [Models.MacroABM.Convergence]
        gdp_growth_tolerance = 0.10
        mode = "national"

`[Models.LabourABM]` (optional)¶

Add "LabourABM" to General.models_to_run to run the LabourABM stage. It runs once, after the CIMS–macroABM loop has converged (or hit --max_iterations), driven by scripts/labourabm_runner.py. The runner reuses the LabourABM repo's existing one-way MacroABM → LabourABM bridge (the scripts/1_format_production_data.py → scripts/2_run_bridge.py → scripts/3_run_abm_macromodel.py pipeline). All linkage logic lives in M3-linkages and the LabourABM repo; nothing is added to macroABM.

The bridge converts a per-industry output shock, computed as (policy − baseline) / baseline, into occupation-level labour demand. LabourABM therefore needs two macroABM full HDF5 files: the policy run (this scenario) and a baseline run. Results are written to <scenario>_<iteration>/labourabm/ (results.h5 plus IAMC summary CSVs, the wide labour-demand scenario, and diagnostic plots).

Key	Type	Default	Description
`repo_path`	string	`dependencies\labourabm`	Path to the LabourABM repo. Relative paths are also resolved against `../../`.
`runner`	string	`"uv"`	How to launch the LabourABM scripts: `"uv"` (uv syncs the repo's deps into a managed venv — no manual install needed), `"conda"` (needs `env_name`, with the package `pip install -e .`'d), or `"pip"` (use `base_env`).
`python_version`	string	`"3.12"`	Python version passed to `uv run --python` (only used when `runner = "uv"`).
`env_name`	string	(unset)	Conda env for LabourABM. Required only when `runner = "conda"`.
`policy_h5`	string	(this scenario's converged macroABM HDF5)	Path to the policy production HDF5. Defaults to `<scenario>_<iteration>/macroabm/<h5_filename>`.
`baseline_h5`	string	(unset)	Path to a baseline macroABM full HDF5. Relative paths resolve against the scenario folder.
`baseline_scenario`	string	(unset)	Instead of `baseline_h5`, name another linkage scenario; its highest-iteration converged macroABM HDF5 is used as the baseline.
`start_year`	int	`sim_start_year`	Calendar year mapped to LabourABM time index 0 (`--start-year`).
`merge_macro`	bool	`false`	When `true`, merge LabourABM output with a macro IAMC file (otherwise `--no-merge-macro`).
`heatmap`	bool	`true`	When `false`, skip the heavy occupation-transition-matrix / heatmap outputs (`--no-heatmap`).

Baseline required for a meaningful shock. If neither baseline_h5 nor baseline_scenario resolves, the policy HDF5 is reused as the baseline, giving a zero labour-demand shock. This validates the wiring end-to-end but is not a substantive run — set a baseline for real analysis (e.g. a Reference scenario as the baseline for a Net Zero policy scenario).

[General]
    base_env = "m3linkages_env"
    models_to_run = ["CIMS", "MacroABM", "LabourABM"]
    sim_end_year = 2050

# ... [Models.CIMS] and [Models.MacroABM] as above ...

[Models.LabourABM]
    repo_path      = "dependencies\\labourabm"
    runner         = "uv"          # or "conda" / "pip"
    python_version = "3.12"        # used when runner = "uv"
    # env_name     = "labourabm_env"   # required only when runner = "conda"
    baseline_scenario = "MacroABM Reference"   # baseline run for the shock
    start_year     = 2020
    merge_macro    = false
    heatmap        = true

Sector and region mapping¶

These editable CSVs live in the macroABM repo at macro_data/processing/macroabm_cims_data_processing/data/. They are loaded by SectorMap; any unmapped or misspelled entry is skipped with a warning, so editing them can never crash a run.

`cims_macro_sector_map.csv`¶

One row per macroABM industry code.

Column	Description
`macro_code`	macroABM ISIC sub-code (e.g. `B05a`, `D01a`).
`macro_name`	Human-readable label (informational).
`cims_sector`	CIMS sector this industry maps to (e.g. `Coal Mining`). Drives both result extraction (producing-sector rows) and the production feedback. Blank = no CIMS counterpart.
`cims_fuel_names`	`;`-separated CIMS fuel/good leaf names this industry represents as an input good (e.g. `Diesel;Gasoline`). Used to place CIMS `quantity_requested` into the correct matrix column.
`is_energy_bundle`	`TRUE` if this code is an energy-bundle input good (columns updated by `firms.link()`).
`is_cims_comparable`	`TRUE` if this is a producing industry with a CIMS counterpart (rows updated by `firms.link()`).
`feedback_enabled`	`TRUE` if macroABM production for this code is written back to CIMS as a growth signal.

When several macro_code rows share a cims_sector (e.g. all electricity sub-codes map to Electricity), the first row in file order is used as the representative producing industry when extracting CIMS results; all of them contribute to the feedback aggregation.

`cims_macro_region_map.csv`¶

Maps macroABM provinces to CIMS regions. CIMS lumps the four Atlantic provinces into a single AT region, so several macro provinces can map to one CIMS region (their production is summed before computing growth).

Column	Description
`macro_region`	macroABM province code (e.g. `CAN_AB`).
`macro_region_name`	Human-readable label (informational).
`cims_region`	CIMS region code (`AB`, `BC`, `MB`, `ON`, `QC`, `SK`, or `AT`).

How CIMS results become macroABM inputs¶

For reference, the extractor (CIMSResultsExtractor) reads two standard CIMS output files from each iteration's cims/outputs/:

*_results_general.csv → quantity_requested rows build the requested-quantities matrix (producing industry × input good).
results_tech.csv → new_stock, capital cost, output build per-sector investment totals (new_stock * capital_cost / max(output, output_floor)), allocated across goods by the requested-quantity shares. The default output_floor is the 1st percentile of positive technology-level output values for that milestone year (all regions), which prevents near-zero outputs from creating pathological investment spikes.

Both are divided by steps_per_year to convert annual CIMS values to the macroABM's per-step values.

CIMS–macroABM Configuration Reference¶

[General]¶

[Models.CIMS]¶

CIMS warm-restart (optional efficiency mode)¶

[Models.MacroABM]¶

[Models.MacroABM.Config]¶

[Models.MacroABM.Config.Linkage]¶

[Models.MacroABM.Convergence]¶