Ctrl+K

Getting started with ac3airborne

Getting started with ac3airborne#

Welcome to ac3airborne! We’re happy that you’re here and hope that you find this Python library to be useful for your needs. In order to help you to get started with ac3airborne, we’ve put together this collection of tutorials that should introduce you to the basic syntax and functionality of this library.

For installation instructions, please see our Github Repository.

This package intends to make the access to the airborne data collected within the various AC3 airborne campaigns more easy.

In some of the examples you will see an icon like the first one on this picture:

title

Just go with your mouse on it:

title

Clicking on Binder will start an virtual environment and the page will be shown as a Jupyter Notebook within a Jupyter Hub, so that you can try the examples yourself!

Functionality#

First of all we want to import the ac3airborne module:

import ac3airborne as ac3a

If you want to check which version of the package is currently installed:

ac3a.__version__

'0.1.0-2-g18be986'

Basic methods#

There are two basic methods which do most of the work in this library:

  • get_flight_segments

  • get_intake_catalog

ac3airborne.get_flight_segments#

meta = ac3a.get_flight_segments()

This method downloads and parses flight segmentation information. Let’s look into its output:

type(meta)

dict

As you can see, the method returns a dictionary. Let’s take a look to its keys:

list(meta.keys())

['ACLOUD', 'AFLUX', 'HALO-AC3', 'HAMAG', 'MOSAiC-ACA', 'PAMARCMiP']

The keys correspond to the different campaigns. We select one of them and take a look to the nested structure of the dictionary.

type(meta['ACLOUD'])

dict

list(meta['ACLOUD'])

['P5', 'P6']

We have two keys, corresponding to the Polar 5 and to the Polar 6 aircraft of the Alfred-Wegener-Institute for Polar and Marine research.

type(meta['ACLOUD']['P5'])

dict

list(meta['ACLOUD']['P5'].keys())

['ACLOUD_P5_RF04',
 'ACLOUD_P5_RF05',
 'ACLOUD_P5_RF06',
 'ACLOUD_P5_RF07',
 'ACLOUD_P5_RF08',
 'ACLOUD_P5_RF10',
 'ACLOUD_P5_RF11',
 'ACLOUD_P5_RF13',
 'ACLOUD_P5_RF14',
 'ACLOUD_P5_RF15',
 'ACLOUD_P5_RF16',
 'ACLOUD_P5_RF17',
 'ACLOUD_P5_RF18',
 'ACLOUD_P5_RF19',
 'ACLOUD_P5_RF20',
 'ACLOUD_P5_RF21',
 'ACLOUD_P5_RF22',
 'ACLOUD_P5_RF23',
 'ACLOUD_P5_RF25']

As you can see, once we selected the campaign and the aircraft, the third level of the nested dictionary consists of the names of the research flights. We can now select a specific flight and look deeper into it:

type(meta['ACLOUD']['P5']['ACLOUD_P5_RF14'])

dict

list(meta['ACLOUD']['P5']['ACLOUD_P5_RF14'].keys())

['co-location',
 'contacts',
 'date',
 'events',
 'flight_id',
 'flight_report',
 'landing',
 'mission',
 'name',
 'platform',
 'remarks',
 'segments',
 'takeoff']

We now see that this flight contains following informations:

  • contacts

  • date

  • events

  • flight_id

  • flight_report

  • landing

  • mission

  • name

  • platform

  • remarks

  • segments

  • takeoff

contacts flight_report#

The first it’s just the contact data for reference, for example:

meta['ACLOUD']['P5']['ACLOUD_P5_RF14']['contacts']

[{'email': 'mario.mech@uni-koeln.de',
  'name': 'Mario Mech',
  'tags': ['pi', 'lc']}]

The second is the location of the metadata:

meta['ACLOUD']['P5']['ACLOUD_P5_RF14']['flight_report']

'https://home.uni-leipzig.de/~ehrlich/ACLOUD_wiki_doku/lib/exe/fetch.php?media=14_20170608_p5_acloud_flight_report.pdf'
date takeoff landing#

The date of the flight, the time when it took of and when it landed:

meta['ACLOUD']['P5']['ACLOUD_P5_RF14']['date']

datetime.date(2017, 6, 8)

meta['ACLOUD']['P5']['ACLOUD_P5_RF14']['takeoff']

datetime.datetime(2017, 6, 8, 7, 36, 50)

meta['ACLOUD']['P5']['ACLOUD_P5_RF14']['landing']

datetime.datetime(2017, 6, 8, 12, 51, 25)

As you can see the time is given as datetime.date or datetime.datetimeobject.

events remarks#

Some metadata:

meta['ACLOUD']['P5']['ACLOUD_P5_RF14']['events']

['Ny-Alesund overflight with cross pattern',
 'Polarstern overflight with cross pattern',
 'A-train underflight',
 'Racetrack pattern over sea ice',
 'Radiometer calibration']

meta['ACLOUD']['P5']['ACLOUD_P5_RF14']['remarks']

['Thin broken clouds over sea ice']
other keys#
meta['ACLOUD']['P5']['ACLOUD_P5_RF14']['mission']

'ACLOUD'

meta['ACLOUD']['P5']['ACLOUD_P5_RF14']['platform']

'P5'

meta['ACLOUD']['P5']['ACLOUD_P5_RF14']['name']

'RF14'

meta['ACLOUD']['P5']['ACLOUD_P5_RF14']['flight_id']

'ACLOUD_P5_RF14'
The flight segments#

This is the key, which actually contains the segmented flight. Let’s take a closer look:

type(meta['ACLOUD']['P5']['ACLOUD_P5_RF14']['segments'])

list

As you can see the segments-key does not contain a dictionary, as the previous levels. Instead it contains a list! Let’s take the first element:

type(meta['ACLOUD']['P5']['ACLOUD_P5_RF14']['segments'][0])

dict

So the segment-key contains actually a list of dictionary as value. Let’s just print this first segment:

meta['ACLOUD']['P5']['ACLOUD_P5_RF14']['segments'][0]

{'dropsondes': [],
 'end': datetime.datetime(2017, 6, 8, 8, 8, 53),
 'irregularities': ['whale-watching loop'],
 'kinds': ['major_ascent', 'nya_overflight', 'sveabreen_glacier_overflight'],
 'levels': [100, 9700],
 'name': 'major ascent',
 'segment_id': 'ACLOUD_P5_RF14_ma',
 'start': datetime.datetime(2017, 6, 8, 7, 36, 50)}

and it’s keys:

list(meta['ACLOUD']['P5']['ACLOUD_P5_RF14']['segments'][0].keys())

['dropsondes',
 'end',
 'irregularities',
 'kinds',
 'levels',
 'name',
 'segment_id',
 'start']

As you can see a flight segment usually contains:

  • start and end time

  • levels height(s) of the flight

  • name and segment_id to identify it

  • kinds or to which type of segments it belongs (high_level, low_level and so on)

  • irregularities some comments about irregularies

  • dropsondes

Take a look at Querying the flight phase files for a more advanced selection of the segments.

ac3airborne.get_intake_catalog#

cat = ac3a.get_intake_catalog()

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[27], line 1
----> 1 cat = ac3a.get_intake_catalog()

File ~/miniconda3/envs/how_to_ac3airborne/lib/python3.12/site-packages/ac3airborne/__init__.py:60, in get_intake_catalog()
     53 def get_intake_catalog():
     54     """
     55     Open the intake data catalog.
     56 
     57     The catalog provides access to public AC3 airborne datasets without the need to
     58     manually specify URLs to the individual datasets.
     59     """
---> 60     return intake.open_catalog("https://raw.githubusercontent.com/igmk/ac3airborne-intake/main/catalog.yaml")

File ~/miniconda3/envs/how_to_ac3airborne/lib/python3.12/site-packages/intake/__init__.py:184, in open_catalog(uri, **kwargs)
    177     raise ValueError(
    178         f"Unknown catalog driver '{driver}'. "
    179         "Do you need to install a new driver from the plugin directory? "
    180         "https://intake.readthedocs.io/en/latest/plugin-directory.html\n"
    181         f"Current registry: {list(sorted(registry))}"
    182     )
    183 try:
--> 184     return registry[driver](uri, **kwargs)
    185 except VersionError:
    186     # warn that we are switching to V2? The file will be read twice
    187     return from_yaml_file(uri, **kwargs)

File ~/miniconda3/envs/how_to_ac3airborne/lib/python3.12/site-packages/intake/catalog/local.py:613, in YAMLFileCatalog.__init__(self, path, text, autoreload, **kwargs)
    611 self.filesystem = kwargs.pop("fs", None)
    612 self.access = "name" not in kwargs
--> 613 super(YAMLFileCatalog, self).__init__(**kwargs)

File ~/miniconda3/envs/how_to_ac3airborne/lib/python3.12/site-packages/intake/catalog/base.py:128, in Catalog.__init__(self, entries, name, description, metadata, ttl, getenv, getshell, persist_mode, storage_options, user_parameters)
    126 self.updated = time.time()
    127 self._entries = entries if entries is not None else self._make_entries_container()
--> 128 self.force_reload()

File ~/miniconda3/envs/how_to_ac3airborne/lib/python3.12/site-packages/intake/catalog/base.py:186, in Catalog.force_reload(self)
    184 """Imperative reload data now"""
    185 self.updated = time.time()
--> 186 self._load()

File ~/miniconda3/envs/how_to_ac3airborne/lib/python3.12/site-packages/intake/catalog/local.py:648, in YAMLFileCatalog._load(self, reload)
    646     logger.warning("Use of '!template' deprecated - fixing")
    647     text = text.replace("!template ", "")
--> 648 self.parse(text)

File ~/miniconda3/envs/how_to_ac3airborne/lib/python3.12/site-packages/intake/catalog/local.py:728, in YAMLFileCatalog.parse(self, text)
    726 result = CatalogParser(data, context=context, getenv=self.getenv, getshell=self.getshell)
    727 if result.errors:
--> 728     raise exceptions.ValidationError(
    729         "Catalog '{}' has validation errors:\n\n{}"
    730         "".format(self.path, "\n".join(result.errors)),
    731         result.errors,
    732     )
    734 cfg = result.data
    736 self._entries = {}

ValidationError: Catalog 'https://raw.githubusercontent.com/igmk/ac3airborne-intake/main/catalog.yaml' has validation errors:

("missing 'module'", {'module': 'intake_xarray'})

Open the intake data catalog. The catalog provides access to public AC3 airborne datasets without the need to manually specify URLs to the individual datasets. Let’s take a look into it:

type(cat)

intake.catalog.local.YAMLFileCatalog

As you can see, this is not a dictionary, but it has keys that can be selected as a dictionary. We can list them using .walk(depth=1) instead of keys():

list(cat.walk(depth=1))

['ACLOUD', 'PAMARCMiP', 'AFLUX', 'MOSAiC-ACA']

Let’s take again the polar 5 during ACLOUD:

list(cat['ACLOUD']['P5'].walk(depth=1))

['GPS_INS',
 'AMSR2_SIC',
 'CLOUD_TOP_HEIGHT',
 'MiRAC-A',
 'MiRAC-P',
 'DROPSONDES',
 'FISH_EYE_CAMERA',
 'BROADBAND_IRRADIANCE',
 'NOSE_BOOM']

These are the data set which are available for the polar 5. Let’s take GPS_INS:

list(cat['ACLOUD']['P5']['GPS_INS'].walk(depth=1))

['ACLOUD_P5_RF04',
 'ACLOUD_P5_RF05',
 'ACLOUD_P5_RF06',
 'ACLOUD_P5_RF07',
 'ACLOUD_P5_RF08',
 'ACLOUD_P5_RF10',
 'ACLOUD_P5_RF11',
 'ACLOUD_P5_RF13',
 'ACLOUD_P5_RF14',
 'ACLOUD_P5_RF15',
 'ACLOUD_P5_RF16',
 'ACLOUD_P5_RF17',
 'ACLOUD_P5_RF18',
 'ACLOUD_P5_RF19',
 'ACLOUD_P5_RF20',
 'ACLOUD_P5_RF21',
 'ACLOUD_P5_RF22',
 'ACLOUD_P5_RF23',
 'ACLOUD_P5_RF25']

These are the flights that are available for the given dataset. Let’s take again ACLOUD_P5_RF14:

type(cat['ACLOUD']['P5']['GPS_INS']['ACLOUD_P5_RF14'])

intake_xarray.netcdf.NetCDFSource

cat['ACLOUD']['P5']['GPS_INS']['ACLOUD_P5_RF14']

ACLOUD_P5_RF14:
  args:
    urlpath: https://atmos.meteo.uni-koeln.de/ac3/acloud/p5/gps_ins/gps_ins_ACLOUD_polar5_20170608_RF14.nc
  description: ''
  driver: intake_xarray.netcdf.NetCDFSource
  metadata:
    catalog_dir: https://raw.githubusercontent.com/igmk/ac3airborne-intake/main/ACLOUD/P5

In order to read the data set, use the method .to_dask() (or sometimes .read()) as explained in the examples under datasets

Constants#

ac3airborne comes with some predefined constants, such the coordinates of Ny-Ålesund and Longyearbyen give as dictionaries:

import ac3airborne.constants as ac3c

ac3c.NYA

{'name': 'Ny-Ålesund', 'lon': 11.922222, 'lat': 78.925}

ac3c.LYB

{'name': 'Longyearbyen', 'lon': 15.633333, 'lat': 78.216667}

ac3c.LYR

{'name': 'Svalbard Airport, Longyear',
 'icao': 'ENSB',
 'iata': 'LYR',
 'lon': 15.465556,
 'lat': 78.246111,
 'elev_m': 28,
 'elev_ft': 94}

ac3c.LYR['lon']

15.465556
Contents