Monetary EXIOBASE walkthrough#

This notebook is the practical walkthrough for parsing monetary EXIOBASE in MARIO.

It focuses on the two main cases:

  • monetary IOT

  • monetary SUT

What this notebook covers#

  • which parser entry points to use

  • what local dataset layout MARIO expects

  • how to download supported monetary releases

  • how to parse one monetary IOT

  • how to parse one monetary SUT

  • how to add SUT extensions from the matching IOT when available

Relevant Zenodo releases#

Main entry point#

For normal user workflows, the public entry points are:

  • mario.parse_exiobase(...)

Key arguments#

The key public arguments are:

  • table: choose "IOT" or "SUT"

  • unit: use "Monetary"

  • path: local EXIOBASE directory to parse

  • year: optional metadata override, usually inferred from the payload

  • add_extensions: monetary SUT-only helper to import extensions from a matching IOT

Local layout expectation#

The current monetary parsers expect the EXIOBASE dataset to be available as a local directory with the files required by the corresponding release. The parser auto-detects the layout and database version from the directory contents and metadata.

For SUT parsing, extensions can be added by pointing add_extensions to the matching monetary IOT directory.

Local layout details and caveats#

path points to one extracted monetary EXIOBASE release.

Typical layouts are:

EXIOBASE/3.9.4/
├── IOT_2013_ixi/
└── IOT_2013_pxp/

EXIOBASE/3.8.2/
└── MRSUT_2011/

When download=True, path becomes the destination/cache directory used by the downloader before parsing.

Practical caveats:

  • monetary SUT parsing is currently available only for 3.8.2;

  • the newer supported monetary releases are IOT-only;

  • EXIOBASE 1 and 2 are not handled by this parser family.

[1]:

import mario

Optional download step#

If you do not already have the monetary dataset locally, MARIO can download the supported monetary releases directly from Zenodo.

For IOT releases, you must specify the system ("ixi" or "pxp").

[ ]:

download_iot = mario.download_exiobase3(
    path="/path/to/3.10.2",
    years=[2024],
    table="IOT",
    system="ixi",
    version="3.10.2",
)

The SUT download path is currently available only for 3.8.2.

[ ]:

download_sut = mario.download_exiobase3(
    path="/path/to/3.8.2",
    years=[2019],
    table="SUT",
    version="3.8.2",
)

Parse a monetary IOT#

Use this path when you have a monetary EXIOBASE input-output table already available locally.

[ ]:

db_iot = mario.parse_exiobase(
    path="/path/to/IOT_2019_ixi",
    table="IOT",
    unit="Monetary",
)

INFO Parser: reading EXIOBASE IOT from /var/folders/7_/vg3_ld6n2pd73xyzt9dcdhlc0000gn/T/mario_exiobase_iot_2efvhkd9.
INFO Parser: reading EXIOBASE IOT extensions from /var/folders/7_/vg3_ld6n2pd73xyzt9dcdhlc0000gn/T/mario_exiobase_iot_2efvhkd9.
INFO Parser: using split extension layout with 7 extension directories.
INFO Parser: EXIOBASE IOT parsed with 163 sectors, 9 value-added rows and 724 extension rows.
INFO Metadata: initialized.

Parse a monetary SUT#

Use this path when the source dataset is the monetary supply-use table.

At the moment, this path is tied to EXIOBASE 3.8.2.

[ ]:

db_sut = mario.parse_exiobase(
    path="/path/to/MRSUT_2019",
    table="SUT",
    unit="Monetary",
)

INFO Parser: reading EXIOBASE SUT from /var/folders/7_/vg3_ld6n2pd73xyzt9dcdhlc0000gn/T/mario_exiobase_sut_3xaccws2/MRSUT_2022.
INFO Parser: no matching IOT provided; using empty satellite extensions.
INFO Parser: EXIOBASE SUT parsed with 163 activities, 200 commodities, 12 value-added rows.
INFO Metadata: initialized.

Add SUT extensions from the matching monetary IOT#

If you have both the monetary SUT and the corresponding monetary IOT, MARIO can read the extension blocks from the IOT and attach them to the SUT parse.

[ ]:

db_eesut = mario.parse_exiobase(
    path="/path/to/MRSUT_2019",
    table="SUT",
    unit="Monetary",
    add_extensions="/path/to/IOT_2019_ixi",
)

INFO Parser: reading EXIOBASE SUT from /var/folders/7_/vg3_ld6n2pd73xyzt9dcdhlc0000gn/T/mario_exiobase_sut_bcsb1waf/MRSUT_2022.
INFO Parser: importing satellite extensions from matching IOT at /Users/lorenzorinaldi/Library/CloudStorage/OneDrive-SharedLibraries-PolitecnicodiMilano/DENG-SESAM - Documenti/c-Research/a-Datasets/_Input Output Databases/EXIOBASE/3.10.2/IOT_2022_ixi.
INFO Parser: reading EXIOBASE IOT extensions from /Users/lorenzorinaldi/Library/CloudStorage/OneDrive-SharedLibraries-PolitecnicodiMilano/DENG-SESAM - Documenti/c-Research/a-Datasets/_Input Output Databases/EXIOBASE/3.10.2/IOT_2022_ixi.
INFO Parser: using split extension layout with 7 extension directories.
INFO Parser: EXIOBASE SUT parsed with 163 activities, 200 commodities, 12 value-added rows.
INFO Metadata: initialized.

Inspect the parsed database#

Once parsed, the result is a standard MARIO database. From here the normal exploration methods apply.

A simple way to inspect a parsed MARIO database is to evaluate the object name in a notebook cell (for example, db_iot or db_sut).
This prints a compact summary of its core properties.
[8]:

db_iot

[8]:

name = EXIO_IOT_2022_ixi
table = IOT
scenarios = ['baseline']
Factor of production = 9
Satellite account = 724
Consumption category = 7
Region = 49
Sector = 163

[9]:

db_sut

[9]:

name = MRSUT_2022
table = SUT
tech_assumption = industry-based
scenarios = ['baseline']
Activity = 163
Commodity = 200
Factor of production = 12
Satellite account = 1
Consumption category = 7
Region = 49

One may then check the list of sectors, activities, commodities or any other set of the databse

[10]:

db_iot.sectors

[10]:

['Cultivation of paddy rice',
 'Cultivation of wheat',
 'Cultivation of cereal grains nec',
 'Cultivation of vegetables, fruit, nuts',
 'Cultivation of oil seeds',
 'Cultivation of sugar cane, sugar beet',
 'Cultivation of plant-based fibers',
 'Cultivation of crops nec',
 'Cattle farming',
 'Pigs farming',
 'Poultry farming',
 'Meat animals nec',
 'Animal products nec',
 'Raw milk',
 'Wool, silk-worm cocoons',
 'Manure treatment (conventional), storage and land application',
 'Manure treatment (biogas), storage and land application',
 'Forestry, logging and related service activities (02)',
 'Fishing, operating of fish hatcheries and fish farms; service activities incidental to fishing (05)',
 'Mining of coal and lignite; extraction of peat (10)',
 'Extraction of crude petroleum and services related to crude oil extraction, excluding surveying',
 'Extraction of natural gas and services related to natural gas extraction, excluding surveying',
 'Extraction, liquefaction, and regasification of other petroleum and gaseous materials',
 'Mining of uranium and thorium ores (12)',
 'Mining of iron ores',
 'Mining of copper ores and concentrates',
 'Mining of nickel ores and concentrates',
 'Mining of aluminium ores and concentrates',
 'Mining of precious metal ores and concentrates',
 'Mining of lead, zinc and tin ores and concentrates',
 'Mining of other non-ferrous metal ores and concentrates',
 'Quarrying of stone',
 'Quarrying of sand and clay',
 'Mining of chemical and fertilizer minerals, production of salt, other mining and quarrying n.e.c.',
 'Processing of meat cattle',
 'Processing of meat pigs',
 'Processing of meat poultry',
 'Production of meat products nec',
 'Processing vegetable oils and fats',
 'Processing of dairy products',
 'Processed rice',
 'Sugar refining',
 'Processing of Food products nec',
 'Manufacture of beverages',
 'Manufacture of fish products',
 'Manufacture of tobacco products (16)',
 'Manufacture of textiles (17)',
 'Manufacture of wearing apparel; dressing and dyeing of fur (18)',
 'Tanning and dressing of leather; manufacture of luggage, handbags, saddlery, harness and footwear (19)',
 'Manufacture of wood and of products of wood and cork, except furniture; manufacture of articles of straw and plaiting materials (20)',
 'Re-processing of secondary wood material into new wood material',
 'Pulp',
 'Re-processing of secondary paper into new pulp',
 'Paper',
 'Publishing, printing and reproduction of recorded media (22)',
 'Manufacture of coke oven products',
 'Petroleum Refinery',
 'Processing of nuclear fuel',
 'Plastics, basic',
 'Re-processing of secondary plastic into new plastic',
 'N-fertiliser',
 'P- and other fertiliser',
 'Chemicals nec',
 'Manufacture of rubber and plastic products (25)',
 'Manufacture of glass and glass products',
 'Re-processing of secondary glass into new glass',
 'Manufacture of ceramic goods',
 'Manufacture of bricks, tiles and construction products, in baked clay',
 'Manufacture of cement, lime and plaster',
 'Re-processing of ash into clinker',
 'Manufacture of other non-metallic mineral products n.e.c.',
 'Manufacture of basic iron and steel and of ferro-alloys and first products thereof',
 'Re-processing of secondary steel into new steel',
 'Precious metals production',
 'Re-processing of secondary preciuos metals into new preciuos metals',
 'Aluminium production',
 'Re-processing of secondary aluminium into new aluminium',
 'Lead, zinc and tin production',
 'Re-processing of secondary lead into new lead, zinc and tin',
 'Copper production',
 'Re-processing of secondary copper into new copper',
 'Other non-ferrous metal production',
 'Re-processing of secondary other non-ferrous metals into new other non-ferrous metals',
 'Casting of metals',
 'Manufacture of fabricated metal products, except machinery and equipment (28)',
 'Manufacture of machinery and equipment n.e.c. (29)',
 'Manufacture of office machinery and computers (30)',
 'Manufacture of electrical machinery and apparatus n.e.c. (31)',
 'Manufacture of radio, television and communication equipment and apparatus (32)',
 'Manufacture of medical, precision and optical instruments, watches and clocks (33)',
 'Manufacture of motor vehicles, trailers and semi-trailers (34)',
 'Manufacture of other transport equipment (35)',
 'Manufacture of furniture; manufacturing n.e.c. (36)',
 'Recycling of waste and scrap',
 'Recycling of bottles by direct reuse',
 'Production of electricity by coal',
 'Production of electricity by gas',
 'Production of electricity by nuclear',
 'Production of electricity by hydro',
 'Production of electricity by wind',
 'Production of electricity by petroleum and other oil derivatives',
 'Production of electricity by biomass and waste',
 'Production of electricity by solar photovoltaic',
 'Production of electricity by solar thermal',
 'Production of electricity by tide, wave, ocean',
 'Production of electricity by Geothermal',
 'Production of electricity nec',
 'Transmission of electricity',
 'Distribution and trade of electricity',
 'Manufacture of gas; distribution of gaseous fuels through mains',
 'Steam and hot water supply',
 'Collection, purification and distribution of water (41)',
 'Construction (45)',
 'Re-processing of secondary construction material into aggregates',
 'Sale, maintenance, repair of motor vehicles, motor vehicles parts, motorcycles, motor cycles parts and accessoiries',
 'Retail sale of automotive fuel',
 'Wholesale trade and commission trade, except of motor vehicles and motorcycles (51)',
 'Retail trade, except of motor vehicles and motorcycles; repair of personal and household goods (52)',
 'Hotels and restaurants (55)',
 'Transport via railways',
 'Other land transport',
 'Transport via pipelines',
 'Sea and coastal water transport',
 'Inland water transport',
 'Air transport (62)',
 'Supporting and auxiliary transport activities; activities of travel agencies (63)',
 'Post and telecommunications (64)',
 'Financial intermediation, except insurance and pension funding (65)',
 'Insurance and pension funding, except compulsory social security (66)',
 'Activities auxiliary to financial intermediation (67)',
 'Real estate activities (70)',
 'Renting of machinery and equipment without operator and of personal and household goods (71)',
 'Computer and related activities (72)',
 'Research and development (73)',
 'Other business activities (74)',
 'Public administration and defence; compulsory social security (75)',
 'Education (80)',
 'Health and social work (85)',
 'Incineration of waste: Food',
 'Incineration of waste: Paper',
 'Incineration of waste: Plastic',
 'Incineration of waste: Metals and Inert materials',
 'Incineration of waste: Textiles',
 'Incineration of waste: Wood',
 'Incineration of waste: Oil/Hazardous waste',
 'Biogasification of food waste, incl. land application',
 'Biogasification of paper, incl. land application',
 'Biogasification of sewage slugde, incl. land application',
 'Composting of food waste, incl. land application',
 'Composting of paper and wood, incl. land application',
 'Waste water treatment, food',
 'Waste water treatment, other',
 'Landfill of waste: Food',
 'Landfill of waste: Paper',
 'Landfill of waste: Plastic',
 'Landfill of waste: Inert/metal/hazardous',
 'Landfill of waste: Textiles',
 'Landfill of waste: Wood',
 'Activities of membership organisation n.e.c. (91)',
 'Recreational, cultural and sporting activities (92)',
 'Other service activities (93)',
 'Private households with employed persons (95)',
 'Extra-territorial organizations and bodies']

[11]:

db_sut.commodities

[11]:

['Paddy rice',
 'Wheat',
 'Cereal grains nec',
 'Vegetables, fruit, nuts',
 'Oil seeds',
 'Sugar cane, sugar beet',
 'Plant-based fibers',
 'Crops nec',
 'Cattle',
 'Pigs',
 'Poultry',
 'Meat animals nec',
 'Animal products nec',
 'Raw milk',
 'Wool, silk-worm cocoons',
 'Manure (conventional treatment)',
 'Manure (biogas treatment)',
 'Products of forestry, logging and related services (02)',
 'Fish and other fishing products; services incidental of fishing (05)',
 'Anthracite',
 'Coking Coal',
 'Other Bituminous Coal',
 'Sub-Bituminous Coal',
 'Patent Fuel',
 'Lignite/Brown Coal',
 'BKB/Peat Briquettes',
 'Peat',
 'Crude petroleum and services related to crude oil extraction, excluding surveying',
 'Natural gas and services related to natural gas extraction, excluding surveying',
 'Natural Gas Liquids',
 'Other Hydrocarbons',
 'Uranium and thorium ores (12)',
 'Iron ores',
 'Copper ores and concentrates',
 'Nickel ores and concentrates',
 'Aluminium ores and concentrates',
 'Precious metal ores and concentrates',
 'Lead, zinc and tin ores and concentrates',
 'Other non-ferrous metal ores and concentrates',
 'Stone',
 'Sand and clay',
 'Chemical and fertilizer minerals, salt and other mining and quarrying products n.e.c.',
 'Products of meat cattle',
 'Products of meat pigs',
 'Products of meat poultry',
 'Meat products nec',
 'products of Vegetable oils and fats',
 'Dairy products',
 'Processed rice',
 'Sugar',
 'Food products nec',
 'Beverages',
 'Fish products',
 'Tobacco products (16)',
 'Textiles (17)',
 'Wearing apparel; furs (18)',
 'Leather and leather products (19)',
 'Wood and products of wood and cork (except furniture); articles of straw and plaiting materials (20)',
 'Wood material for treatment, Re-processing of secondary wood material into new wood material',
 'Pulp',
 'Secondary paper for treatment, Re-processing of secondary paper into new pulp',
 'Paper and paper products',
 'Printed matter and recorded media (22)',
 'Coke Oven Coke',
 'Gas Coke',
 'Coal Tar',
 'Motor Gasoline',
 'Aviation Gasoline',
 'Gasoline Type Jet Fuel',
 'Kerosene Type Jet Fuel',
 'Kerosene',
 'Gas/Diesel Oil',
 'Heavy Fuel Oil',
 'Refinery Gas',
 'Liquefied Petroleum Gases (LPG)',
 'Refinery Feedstocks',
 'Ethane',
 'Naphtha',
 'White Spirit & SBP',
 'Lubricants',
 'Bitumen',
 'Paraffin Waxes',
 'Petroleum Coke',
 'Non-specified Petroleum Products',
 'Nuclear fuel',
 'Plastics, basic',
 'Secondary plastic for treatment, Re-processing of secondary plastic into new plastic',
 'N-fertiliser',
 'P- and other fertiliser',
 'Chemicals nec',
 'Charcoal',
 'Additives/Blending Components',
 'Biogasoline',
 'Biodiesels',
 'Other Liquid Biofuels',
 'Rubber and plastic products (25)',
 'Glass and glass products',
 'Secondary glass for treatment, Re-processing of secondary glass into new glass',
 'Ceramic goods',
 'Bricks, tiles and construction products, in baked clay',
 'Cement, lime and plaster',
 'Ash for treatment, Re-processing of ash into clinker',
 'Other non-metallic mineral products',
 'Basic iron and steel and of ferro-alloys and first products thereof',
 'Secondary steel for treatment, Re-processing of secondary steel into new steel',
 'Precious metals',
 'Secondary preciuos metals for treatment, Re-processing of secondary preciuos metals into new preciuos metals',
 'Aluminium and aluminium products',
 'Secondary aluminium for treatment, Re-processing of secondary aluminium into new aluminium',
 'Lead, zinc and tin and products thereof',
 'Secondary lead for treatment, Re-processing of secondary lead into new lead',
 'Copper products',
 'Secondary copper for treatment, Re-processing of secondary copper into new copper',
 'Other non-ferrous metal products',
 'Secondary other non-ferrous metals for treatment, Re-processing of secondary other non-ferrous metals into new other non-ferrous metals',
 'Foundry work services',
 'Fabricated metal products, except machinery and equipment (28)',
 'Machinery and equipment n.e.c. (29)',
 'Office machinery and computers (30)',
 'Electrical machinery and apparatus n.e.c. (31)',
 'Radio, television and communication equipment and apparatus (32)',
 'Medical, precision and optical instruments, watches and clocks (33)',
 'Motor vehicles, trailers and semi-trailers (34)',
 'Other transport equipment (35)',
 'Furniture; other manufactured goods n.e.c. (36)',
 'Secondary raw materials',
 'Bottles for treatment, Recycling of bottles by direct reuse',
 'Electricity by coal',
 'Electricity by gas',
 'Electricity by nuclear',
 'Electricity by hydro',
 'Electricity by wind',
 'Electricity by petroleum and other oil derivatives',
 'Electricity by biomass and waste',
 'Electricity by solar photovoltaic',
 'Electricity by solar thermal',
 'Electricity by tide, wave, ocean',
 'Electricity by Geothermal',
 'Electricity nec',
 'Transmission services of electricity',
 'Distribution and trade services of electricity',
 'Coke oven gas',
 'Blast Furnace Gas',
 'Oxygen Steel Furnace Gas',
 'Gas Works Gas',
 'Biogas',
 'Distribution services of gaseous fuels through mains',
 'Steam and hot water supply services',
 'Collected and purified water, distribution services of water (41)',
 'Construction work (45)',
 'Secondary construction material for treatment, Re-processing of secondary construction material into aggregates',
 'Sale, maintenance, repair of motor vehicles, motor vehicles parts, motorcycles, motor cycles parts and accessoiries',
 'Retail trade services of motor fuel',
 'Wholesale trade and commission trade services, except of motor vehicles and motorcycles (51)',
 'Retail  trade services, except of motor vehicles and motorcycles; repair services of personal and household goods (52)',
 'Hotel and restaurant services (55)',
 'Railway transportation services',
 'Other land transportation services',
 'Transportation services via pipelines',
 'Sea and coastal water transportation services',
 'Inland water transportation services',
 'Air transport services (62)',
 'Supporting and auxiliary transport services; travel agency services (63)',
 'Post and telecommunication services (64)',
 'Financial intermediation services, except insurance and pension funding services (65)',
 'Insurance and pension funding services, except compulsory social security services (66)',
 'Services auxiliary to financial intermediation (67)',
 'Real estate services (70)',
 'Renting services of machinery and equipment without operator and of personal and household goods (71)',
 'Computer and related services (72)',
 'Research and development services (73)',
 'Other business services (74)',
 'Public administration and defence services; compulsory social security services (75)',
 'Education services (80)',
 'Health and social work services (85)',
 'Food waste for treatment: incineration',
 'Paper waste for treatment: incineration',
 'Plastic waste for treatment: incineration',
 'Intert/metal waste for treatment: incineration',
 'Textiles waste for treatment: incineration',
 'Wood waste for treatment: incineration',
 'Oil/hazardous waste for treatment: incineration',
 'Food waste for treatment: biogasification and land application',
 'Paper waste for treatment: biogasification and land application',
 'Sewage sludge for treatment: biogasification and land application',
 'Food waste for treatment: composting and land application',
 'Paper and wood waste for treatment: composting and land application',
 'Food waste for treatment: waste water treatment',
 'Other waste for treatment: waste water treatment',
 'Food waste for treatment: landfill',
 'Paper for treatment: landfill',
 'Plastic waste for treatment: landfill',
 'Inert/metal/hazardous waste for treatment: landfill',
 'Textiles waste for treatment: landfill',
 'Wood waste for treatment: landfill',
 'Membership organisation services n.e.c. (91)',
 'Recreational, cultural and sporting services (92)',
 'Other services (93)',
 'Private households with employed persons (95)',
 'Extra-territorial organizations and bodies']

For deeper and more targeted search, the search function can be used to explore specific text in specific sets

[12]:

db_iot.search("sectors","electricity")

[12]:

['Production of electricity by coal',
 'Production of electricity by gas',
 'Production of electricity by nuclear',
 'Production of electricity by hydro',
 'Production of electricity by wind',
 'Production of electricity by petroleum and other oil derivatives',
 'Production of electricity by biomass and waste',
 'Production of electricity by solar photovoltaic',
 'Production of electricity by solar thermal',
 'Production of electricity by tide, wave, ocean',
 'Production of electricity by Geothermal',
 'Production of electricity nec',
 'Transmission of electricity',
 'Distribution and trade of electricity']

[13]:

db_sut.search("commodities","gas")

[13]:

['Manure (biogas treatment)',
 'Natural gas and services related to natural gas extraction, excluding surveying',
 'Natural Gas Liquids',
 'Gas Coke',
 'Motor Gasoline',
 'Aviation Gasoline',
 'Gasoline Type Jet Fuel',
 'Gas/Diesel Oil',
 'Refinery Gas',
 'Liquefied Petroleum Gases (LPG)',
 'Biogasoline',
 'Electricity by gas',
 'Coke oven gas',
 'Blast Furnace Gas',
 'Oxygen Steel Furnace Gas',
 'Gas Works Gas',
 'Biogas',
 'Distribution services of gaseous fuels through mains',
 'Food waste for treatment: biogasification and land application',
 'Paper waste for treatment: biogasification and land application',
 'Sewage sludge for treatment: biogasification and land application']

Notes#

  • prefer the dispatcher when you want one stable public API;

  • derived matrices are not materialized eagerly unless you ask for them;

  • if you need a monetary SUT, use the 3.8.2 release rather than the newer monetary IOT-only releases.

Download this notebook