EMERGING parser walkthrough#

This notebook is the practical guide for parsing the EMERGING MATLAB bundles supported by MARIO.

What this notebook covers#

which official Zenodo version records are relevant for MARIO;
how local file naming works for v1.0, v2.x, and variant="E" bundles;
how download_emerging(...) maps to the supported version records;
how year=, regions=, load_co2=, co2_path=, and variant= are used;
which caveats matter for very large EMERGING and EMERGING-E databases.

Supported official version records#

The relevant official records are:

Any other version reference should be treated as deprecated in the MARIO documentation.

Main entry point#

For normal user workflows, the public entry point is:

mario.parse_emerging(...)

The parser currently supports only the multiregional IOT workflow.

Key arguments#

The key public arguments are:

path: one EMERGING .mat file or a directory containing multiple yearly bundles;
table: currently only "IOT" is supported;
year: use it when one directory contains more than one EMERGING year;
regions: optional ISO3 subset to keep only one manageable part of the database;
load_co2: enable or disable automatic companion CO2 import;
co2_path: explicit path to one companion CO2 file when auto-detection is not enough;
labels_path: optional Excel workbook with explicit sector labels; for variant="E" it overrides workbook auto-detection next to the bundle;
variant: keep "standard" for the original EMERGING releases and use "E" for the EMERGING-E power-disaggregated bundle.

Local file naming conventions#

In practice MARIO accepts these local naming conventions:

global_mrio_<year>.mat for v1.0;
EMERGING_V2_<year>_m.mat for v2.x;
EMERGING_V2_<year>.mat for older local v2.x copies when the internal MATLAB structure is still compatible;
EMERGING_E_<year>.mat for the EMERGING-E power-disaggregated variant.

For local v2.x files, MARIO does not try to infer the exact sub-version 2.0 versus 2.1 versus 2.2 from the filename alone, because the public naming convention is shared across those releases.

Expected path structure#

path can point either to one yearly .mat file or to a directory containing the yearly EMERGING bundle and, optionally, the companion CO2 file.

Typical directory layout:

EMERGING/2.2/
├── EMERGING_V2_2023_m.mat
└── EMERGING_CO2_2023.mat

For the older v1.0 naming convention, the economic bundle is typically named like global_mrio_<year>.mat. For EMERGING-E, the local bundle is typically named like EMERGING_E_<year>.mat and may ship a sibling Figure data.xlsx workbook that MARIO uses to recover ISO3 region codes.

When path is a directory, use year= to select the economic bundle. With load_co2=True, MARIO first looks for the matching CO2 file in the same directory, for example EMERGING_CO2_2023.mat for year=2023. Use co2_path= when the companion file is stored elsewhere or has a non-standard name.

Download workflow#

Use mario.download_emerging(...) when you want MARIO to fetch one of the supported official version records, including the newer version="E" EMERGING-E record.

latest currently resolves to v2.2.

[1]:

import mario

Download one official version explicitly#

[ ]:

mario.download_emerging(
    path="/path/to/2.2",
    version="2.2",
    years=[2023],
)

{'source': 'https://doi.org/10.5281/zenodo.10956622',
 'version': '2.2',
 'version_record': 'https://doi.org/10.5281/zenodo.19461860',
 'download_dir': '/path/to/emerging_directory',
 'years': [2023],
 'files': ['/path/to/emerging_directory/EMERGING_CO2_2023.mat',
  '/path/to/EMERGING_V2_2023_m.mat']}

Download the EMERGING-E record#

Use version="E" when you want MARIO to fetch the public EMERGING-E Zenodo record. With years=[2018], MARIO keeps the main .mat file and the companion Figure data.xlsx workbook used to recover ISO3 region codes.

[ ]:

mario.download_emerging(
    path="/path/to/emerging-e",
    version="E",
    years=[2018],
)

Parse one local EMERGING bundle#

[3]:

db = mario.parse_emerging(
    path="/path/to/EMERGING_V2_2023_m.mat",
    table="IOT",
)

db

INFO Parser: reading EMERGING bundle EMERGING_V2_2023_m.mat.
INFO Parser: reading EMERGING CO2 file EMERGING_CO2_2023.mat.
INFO Parser: EMERGING parsed with 245 regions, 133 sectors, 735 final-demand columns and 7 satellite rows.
INFO Metadata: initialized.

[3]:

name = EMERGING 2023
table = IOT
scenarios = ['baseline']
Factor of production = 1
Satellite account = 7
Consumption category = 3
Region = 245
Sector = 133

Parse one local EMERGING-E bundle#

EMERGING-E is currently a local-file workflow. Use variant="E" and point path to EMERGING_E_<year>.mat.

[ ]:

db_e = mario.parse_emerging(
    # path="/path/to/EMERGING_E_2018.mat",
    path='/Users/lorenzorinaldi/Library/CloudStorage/OneDrive-SharedLibraries-PolitecnicodiMilano/DENG-SESAM - Documenti/c-Research/a-Datasets/_Input Output Databases/EMERGING/EMERGING-E/EMERGING_E_2018.mat',
    table="IOT",
    variant="E"
)

db_e

INFO Parser: reading EMERGING-E bundle EMERGING_E_2018.mat.
INFO Parser: EMERGING-E parsed with 245 regions, 146 sectors, 735 final-demand columns and 1 satellite rows.
INFO Metadata: initialized.

name = EMERGING-E 2018
table = IOT
scenarios = ['baseline']
Factor of production = 1
Satellite account = 1
Consumption category = 3
Region = 245
Sector = 146

Parse from a directory and select one year#

[3]:

db = mario.parse_emerging(
    path="/path/to/2.2",
    table="IOT",
    year=2023,
)

db

INFO Parser: reading EMERGING bundle EMERGING_V2_2023_m.mat.
INFO Parser: reading EMERGING CO2 file EMERGING_CO2_2023.mat.
INFO Parser: EMERGING parsed with 245 regions, 133 sectors, 735 final-demand columns and 7 satellite rows.
INFO Metadata: initialized.

[3]:

name = EMERGING 2023
table = IOT
scenarios = ['baseline']
Factor of production = 1
Satellite account = 7
Consumption category = 3
Region = 245
Sector = 133

Restrict the region set#

The full EMERGING matrix is very large, so regions= is often the right first step.

[4]:

db = mario.parse_emerging(
    path="/path/to/2.2",
    table="IOT",
    year=2023,
    regions=["DEU", "FRA", "ITA"],
)

db

INFO Parser: reading EMERGING bundle EMERGING_V2_2023_m.mat.
INFO Parser: reading EMERGING CO2 file EMERGING_CO2_2023.mat.
INFO Parser: EMERGING parsed with 3 regions, 133 sectors, 9 final-demand columns and 7 satellite rows.
INFO Metadata: initialized.

[4]:

name = EMERGING 2023
table = IOT
scenarios = ['baseline']
Factor of production = 1
Satellite account = 7
Consumption category = 3
Region = 3
Sector = 133

Control CO2 loading explicitly#

[5]:

db = mario.parse_emerging(
    path="/path/to/2.2",
    table="IOT",
    year=2023,
    co2_path="/path/to/EMERGING_CO2_2023.mat",
)

db

INFO Parser: reading EMERGING bundle EMERGING_V2_2023_m.mat.
INFO Parser: reading EMERGING CO2 file EMERGING_CO2_2023.mat.
INFO Parser: EMERGING parsed with 245 regions, 133 sectors, 735 final-demand columns and 7 satellite rows.
INFO Metadata: initialized.

[5]:

name = EMERGING 2023
table = IOT
scenarios = ['baseline']
Factor of production = 1
Satellite account = 7
Consumption category = 3
Region = 245
Sector = 133

Caveats#

EMERGING parsing currently supports only IOT tables;
local v2.x file names do not identify the exact official sub-version;
regions= is often necessary to keep the database manageable;
load_co2=False is useful when you want to parse the core IOT first and deal with extensions separately.

[6]:

db.table_type

[6]:

'IOT'

[7]:

db.get_index("Region")[:10]

[7]:

['ABW', 'AFG', 'AGO', 'AIA', 'ALB', 'AND', 'ANT', 'BES', 'CUW', 'ARE']

[8]:

db.get_index("Satellite account")

[8]:

['Coal',
 'Natural gas',
 'Oil products',
 'Crude, NGL, Ref Feeds.',
 'Other',
 'Oil shale & oil sands',
 'Peat & Peat products']

[9]:

db.meta.source

[9]:

'EMERGING MATLAB bundle via Zenodo; (concept DOI https://doi.org/10.5281/zenodo.10956622); Huo, J., Chen, P., Hubacek, K., Zheng, H., Meng, J., & Guan, D. (2022). Full-scale, near real-time multi-regional input-output table for the global emerging economies (EMERGING). Journal of Industrial Ecology, 26, 1218–1232. https://doi.org/10.1111/jiec.13264'

Download this notebook