Quick-start¶

0 · Prerequisites¶

See the setup guide: four repos (M3-linkages, macroabm-ca, labourabm on cims-macro-linkage-v3; cims-epm on EPM_working) and uv sync in dependencies/cims-epm. Plus the two macro-stage inputs that live on the shared drive, not in any repo — the raw_data/ root and the pinned provincial pickle — and the machine-specific raw_data_path / pkl_path edits they need (see the setup guide, section 6).

1 · Stage a scenario¶

A scenario folder is a CIMS-COPPER run folder with a config.toml in the root. Copy your run (the cims/, copper/, iterations/ layout) under scenarios/<name>/, then write the config.

Start from the template configs — do not write a config from scratch. The linkage only produces sensible results under a quite specific configuration: the full channel set (intensity targets + quantity anchors with the guard and total-conserving rescale, pinned prices, fossil pins, the capacity floor/ceiling/gate/D-floor set, the investment channels, the policy environment), the SC1 growth calibration, the labour-force tails, the seed, and the LabourABM absolute mode — each measured and promoted together, and each carrying a comment explaining the measured reason for its value. Use scenarios/Reference_2021_3_EPM/config.toml as the template for a baseline arm and scenarios/Conservative_platform_2021_3_EPM/config.toml for a policy arm (it additionally names its baseline via investment_baseline_scenario and enables the policy-only generation-investment multiplier). Change the scenario label and the file paths; keep the channel and calibration settings unless you are running a deliberate, measured A/B — the methodology records what each one does and what changing it cost when it was tried.

Running CIMS vs staging its outputs. With a [Models.CIMS] block (and "CIMS" first in models_to_run) the workflow runs CIMS-EPM itself on cims/inputs/ and writes cims/outputs/ (about an hour per scenario for 2000–2050). Without the block it consumes whatever CIMS outputs are staged in cims/outputs/. COPPER outputs are always staged.

Everything referenced by [Models.CIMSCOPPER] can be a relative path (resolved against the scenario folder). The macro stage reads its energy-data inputs from [Models.EnergyData] — the adapter-derived CSVs in data/ plus the mapping tables (the CER pipeline's original [Models.CER] name is still accepted, but every path in this workflow is a CIMS/COPPER-derived file):

[Models.CIMS]
    repo_path = "dependencies/cims-epm"
    runner    = "uv"
    inputs    = "cims/inputs"
    outputs   = "cims/outputs"
    strict_policies = true

[Models.CIMSCOPPER]
    cims_results_general = "cims/outputs/<name>_results_general.csv"
    copper_csv = "copper/outputs/<name>_output_summary_IDEA.csv"
    cims_results_tech = "cims/outputs/results_tech.csv"
    scenario_label = "<name>"
    # POLICY arm only -- names the baseline arm the ITC + generation-investment
    # multiplier are built against:
    investment_baseline_scenario = "<baseline scenario folder name>"

2 · Run¶

conda activate m3linkages_env
python linkage_cims_copper_macroabm.py

Stages, in order (the orchestrator fronts all data work before any model run):

CIMS-EPM (scripts/cims_epm_runner.py, only with [Models.CIMS]), per scenario → cims/outputs/: the SESIT scenario runner on cims/inputs/; skipped when outputs are already present (--refresh forces).
Adapter (scripts/cims_copper_data.py), per scenario → data/: end_use_demand.csv, electricity-capacity.csv, electricity-generation.csv, crude-oil-production.csv, natural-gas-production.csv, end-use-prices.csv, benchmark-prices.csv. Skipped when present; --refresh forces.
Investment pair prep (scripts/cims_copper_investment.py), once per pair (driven by investment_baseline_scenario in the policy config): appends both labels' ITC rows to scripts/data/cer_investment_tax_credit.csv and writes the policy arm's data/generation-investment-multiplier.csv (clamped, with COPPER's new interties priced in) and transmission-capex.csv. Capex comes from CODERS when the policy config sets Models.CIMSCOPPER.capex_override_csv to a table built by python scripts/build_copper_capex.py --key-inputs <policy>/copper/outputs/key_inputs --label <policy> (refresh the CODERS dump first with --from-coders --api-key KEY; methodology §6.1). Budget layer only: run python scripts/policy_budget_2026.py --policy-scenario <budget> --baseline-scenario <reference> after this step to build the transfers / credits / concessional tables the policy config points at (methodology §7b).
MacroABM (scripts/cer_macroabm_runner.py → run_cer_linkage.py) → macroabm/ (~15 min): simulation_shallow.h5, gdp_growth*.json, labour_force_index.csv, household_energy_wallet.csv (energy spending by income quintile, split into home electricity / EV charging / gas / motor fuel / heating oil and per 2021-census household, methodology §9), per-year linkage artefacts under cer_data/.
LabourABM (scripts/labourabm_runner.py, absolute mode) → labourabm/: results.h5, IAMC CSVs, the derived demand scenario, and labour_force_index_monthly.csv.
IDEA consolidation (scripts/build_idea_outputs.py) → IDEA/: the COPPER *output_summary* and CIMS *results_general_IDEA* files plus macro-labour-shallow_<scenario>.h5 — quarterly frames indexed 2022Q1 .. 2050Q4 (from the config's sim_start_year; the shallow file's row 0, the pre-simulation state, is dropped), employees ×1000 to persons, the household energy-wallet frames, plus a time_index frame (methodology §9).

3 · Read the outputs¶

File	Contents
`macroabm/simulation_shallow.h5`	per-province macro summary (GDP by three measures + deflator, prices, rates) and per-industry diagnostics
`labourabm/results.h5`	LabourABM employment/unemployment/vacancies/demand per geography × occupation × month
`labourabm/macromodel_integration_iamc_results.csv`	the same, IAMC format, occupational aggregates + unemployment rates
`IDEA/macro-labour-shallow_<scenario>.h5`	the macro summary with labour-market series overwritten by the LabourABM (see below)
`IDEA/output_summary`, `IDEA/results_general_IDEA`	COPPER and CIMS results in IDEA formats

macro-labour-shallow_<scenario>.h5: per province the Unemployment Rate column is the LabourABM's (quarterly, unit-matched), and Employment, Unemployment, Vacancies, Vacancy Rate columns are added in persons. The macro _firms_number_of_employees tables (model agents, industry dimension) are untouched.

Comparing scenarios. Both arms run in absolute mode, so compare LabourABM outputs directly between the two scenario folders — same seed and parameters, the difference is the scenario. Do not reconstruct the old relative-shock convention.

3b · Build the analysis report¶

python scripts/linkage_analysis.py                                    # the default pair
python scripts/linkage_analysis.py --scenarios "A,B" --out report.html  # any set of runs

writes one self-contained interactive HTML page (scenarios/linkage_analysis.html by default) with province dropdowns and scenario overlays; sectoral charts are grouped stacked bars at 5-year milestones with one bar per scenario, so however many scenarios you pass appear side by side.

Section 1 — linked variables: side-by-side trends (indexed, 2023 = 1.0) of every quantity that crosses a model boundary — CIMS price assumptions vs the macro model's realized sector-D prices (deflated by the pinning's exogenous_price_inflation escalator, or by CPI under exogenous_price_nominal_anchor = "cpi", so real is compared with real); COPPER generation vs sector-D production, including a value-structure-adjusted COPPER trace (COPPER × the CIMS demand value/PJ ratio) that separates the unit-structure share of any wedge from genuine model divergence; COPPER capacity vs D's capital-bound output ceiling; fuel use per carrier, population-matched (CIMS firm sectors vs firms' real purchases; the household-routed Transportation Personal + Residential vs households' real consumption) and value-weighted (PJ × CIMS's own per-sector electricity rates / per-fuel prices, so both sides are dollar-commensurable); COPPER build investment vs sector-D real GFCF; and macro employment (the LabourABM's absolute-mode demand driver) vs LabourABM employment.

Section 2 — key results per model: CIMS demand by fuel/sector and trend-only investment; COPPER generation/capacity by technology; MacroABM real GDP, sectoral GVA, GFCF, exports/imports; LabourABM unemployment nationally, by province and by occupation major group.

Chart notes carry the measured caveats (the pinned-price nominal/real conversion, the freight-base residual in the fuel anchors, the structural demand-supply wedge vs COPPER), so the report is the first place to sense-check a new run — and the right place to look before concluding a divergence is a defect: several apparent gaps are unit conventions, and the notes say which.

4 · Checks that a run is sound¶

Adapter sector list — the end-use log line must name all 22 CIMS sectors including Natural Gas Production (the newer CIMS vocabulary is normalised on read; a missing sector means a silent B06 demand drop).
ITC rows exist — the pair prep prints NATIONAL CREDIT, $bn/yr for both labels; if the macro log later says the ITC flag "is having NO effect", the pair prep did not run before the macro stage.
Labour-force index exported — macroabm/labour_force_index.csv must exist after the macro stage; without it the LabourABM warns and falls back to a fixed labour force, and growing absolute demand will hit the conserved-workforce ceiling.
LabourABM supply matches — in labourabm/, national labour-force growth over the run equals the initial-LF-weighted macro index by construction; provincial deviations are endogenous migration, not errors.
Ceiling headroom — this vintage runs the CER production ceiling (1.3). If sector D rations terminally (u → 90%+ everywhere late in the run), widen capacity_ceiling_margin and re-run before quoting anything.