Flight-phase separation

Flight-phase separation#

The idea behind flight-phase-separation is to easily select segments of research flights based on their properties as required for the user’s research. The selection can be done using any combination of attributes. In the following sections, the stored attributes and the structure of files under flight_phase_files is presented.

Interactive map of flight segments#

This example displays the Polar 5 flight segments from ACLOUD_P5_RF14 on an interactive map.

import ac3airborne
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.colors import to_hex
from matplotlib import cm
import ipyleaflet
from ipyleaflet import Polyline, Map, basemaps, basemap_to_tiles
from simplification.cutil import simplify_coords_idx
plt.style.use("mplstyle/book")


def colors(n):
    """Creates set of random colors of length n"""
    
    cmap = cm.get_cmap('gist_rainbow')
    rnd = np.random.uniform(low=0, high=1, size=n)
    cols = cmap(rnd)
    
    return cols


def simplify_dataset(ds, tolerance):
    indices_to_take = simplify_coords_idx(np.stack([ds.lat.values, ds.lon.values], axis=1), tolerance)
    return ds.isel(time=indices_to_take)

# load intake catalog and flight segments
cat = ac3airborne.get_intake_catalog()
meta = ac3airborne.get_flight_segments()

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[1], line 27
     24     return ds.isel(time=indices_to_take)
     26 # load intake catalog and flight segments
---> 27 cat = ac3airborne.get_intake_catalog()
     28 meta = ac3airborne.get_flight_segments()

File ~/miniconda3/envs/how_to_ac3airborne/lib/python3.12/site-packages/ac3airborne/__init__.py:60, in get_intake_catalog()
     53 def get_intake_catalog():
     54     """
     55     Open the intake data catalog.
     56 
     57     The catalog provides access to public AC3 airborne datasets without the need to
     58     manually specify URLs to the individual datasets.
     59     """
---> 60     return intake.open_catalog("https://raw.githubusercontent.com/igmk/ac3airborne-intake/main/catalog.yaml")

File ~/miniconda3/envs/how_to_ac3airborne/lib/python3.12/site-packages/intake/__init__.py:184, in open_catalog(uri, **kwargs)
    177     raise ValueError(
    178         f"Unknown catalog driver '{driver}'. "
    179         "Do you need to install a new driver from the plugin directory? "
    180         "https://intake.readthedocs.io/en/latest/plugin-directory.html\n"
    181         f"Current registry: {list(sorted(registry))}"
    182     )
    183 try:
--> 184     return registry[driver](uri, **kwargs)
    185 except VersionError:
    186     # warn that we are switching to V2? The file will be read twice
    187     return from_yaml_file(uri, **kwargs)

File ~/miniconda3/envs/how_to_ac3airborne/lib/python3.12/site-packages/intake/catalog/local.py:613, in YAMLFileCatalog.__init__(self, path, text, autoreload, **kwargs)
    611 self.filesystem = kwargs.pop("fs", None)
    612 self.access = "name" not in kwargs
--> 613 super(YAMLFileCatalog, self).__init__(**kwargs)

File ~/miniconda3/envs/how_to_ac3airborne/lib/python3.12/site-packages/intake/catalog/base.py:128, in Catalog.__init__(self, entries, name, description, metadata, ttl, getenv, getshell, persist_mode, storage_options, user_parameters)
    126 self.updated = time.time()
    127 self._entries = entries if entries is not None else self._make_entries_container()
--> 128 self.force_reload()

File ~/miniconda3/envs/how_to_ac3airborne/lib/python3.12/site-packages/intake/catalog/base.py:186, in Catalog.force_reload(self)
    184 """Imperative reload data now"""
    185 self.updated = time.time()
--> 186 self._load()

File ~/miniconda3/envs/how_to_ac3airborne/lib/python3.12/site-packages/intake/catalog/local.py:648, in YAMLFileCatalog._load(self, reload)
    646     logger.warning("Use of '!template' deprecated - fixing")
    647     text = text.replace("!template ", "")
--> 648 self.parse(text)

File ~/miniconda3/envs/how_to_ac3airborne/lib/python3.12/site-packages/intake/catalog/local.py:728, in YAMLFileCatalog.parse(self, text)
    726 result = CatalogParser(data, context=context, getenv=self.getenv, getshell=self.getshell)
    727 if result.errors:
--> 728     raise exceptions.ValidationError(
    729         "Catalog '{}' has validation errors:\n\n{}"
    730         "".format(self.path, "\n".join(result.errors)),
    731         result.errors,
    732     )
    734 cfg = result.data
    736 self._entries = {}

ValidationError: Catalog 'https://raw.githubusercontent.com/igmk/ac3airborne-intake/main/catalog.yaml' has validation errors:

("missing 'module'", {'module': 'intake_xarray'})

import os
from dotenv import load_dotenv

load_dotenv()

ac3cloud_username = os.environ['AC3_USER']
ac3cloud_password = os.environ['AC3_PASSWORD']

credentials = dict(user=ac3cloud_username, password=ac3cloud_password)

# local caching
kwds = {'simplecache': dict(
    cache_storage=os.environ['INTAKE_CACHE'], 
    same_names=True
)}

mission = 'MOSAiC-ACA'
platform = 'P5'
flight_id = 'MOSAiC-ACA_P5_RF11'

# read gps data of one flight
ds_gps_ins = cat[mission][platform]['GPS_INS'][flight_id].to_dask()
flight = meta[mission][platform][flight_id]

# plot map
m = Map(basemap=basemap_to_tiles(basemaps.NASAGIBS.ModisTerraTrueColorCR, str(flight['date'])),
        center=(80, 10),
        zoom=4,
        scroll_wheel_zoom=True)

col_segments = colors(len(flight['segments']))

for i, segment in enumerate(flight['segments']):
    
    # reduce flight segment
    ds_reduced = simplify_dataset(ds_gps_ins.sel(time=slice(segment['start'], 
                                                            segment['end'])), 
                                  1e-5)
    
    # plot flight segment
    line = Polyline(locations=np.stack([ds_reduced.lat.values, ds_reduced.lon.values], axis=1).tolist(),
                    color=to_hex(col_segments[i]),
                    fill=False,
                    weight=2,
                    name=str(i) + ': ' + segment['name'])
    m.add_layer(line)

m.add_control(ipyleaflet.ScaleControl())
m.add_control(ipyleaflet.LayersControl(position='topright'))
m.add_control(ipyleaflet.FullScreenControl())

display(m)

Flight keywords#

name: research flight number within campaign (e.g. RF10)

mission: name of campaign (ACLOUD, AFLUX, MOSAiC-ACA)

platform: aircraft short name (P5, P6)

flight_id: unique identifier for flight (campaign_platform_name)

contacts: information on mission PI and creator of .yaml file (name and email contact)

date: date of flight (will be read as python datetime.date object)

flight_report: link to the online flight report with detailed information on the flight (flight objectives, quicklooks, photos)

takeoff: flight takeoff at the airport (will be read as python datetime.datetime object)

landing: flight landing at the airport (will be read as python datetime.datetime object)

events: planned flight features, such as patterns, joint flights between aircraft, satellite overpasses, or instrument calibration

remarks: information on atmospheric conditions and instrument status

segments: list of flight segments

Keyword `segments`#

The following section provides an overview of the flight segments and their properties. They are ordered chronologically in a list, with the first segment starting at flight takeoff and the last segment ending at flight landing. The given start and end times of each segment are included.

Segment keywords#

kinds: list of properties, which describe the flight segment (see next section)

name: name of segment, that is unique within the flight

irregularities: list of irregularities in the flight track or caused by other factors, such as icing

segment_id: unique id of flight segment

start: start of segment (will be read as python datetime.datetime object)

end: end of segment (will be read as python datetime.datetime object)

levels: median flight level(s) or highest and lowest flight level for ascents/descents [in feet]

dropsondes: list of dropsondes released during that segment

parts: list of sub-segments (parts), which belong to a flight pattern (see next section)

Keyword `kinds`#

Making flight-phase-separation a useful tool for various communities, either working on in-situ or remotely sensed data, the various flight maneuvers are classified and separated. The following section provides an overview of the kinds used to describe the flight segments. Multiple keywords can be used for one flight segment. In total, 33 kinds are defined to classify the segments.

Takeoff and landing maneuvers#

The takeoff and landing maneuvers are labelled as major_ascent (takeoff) and major_descent (landing). Their start and end times correspond to the flight takeoff and landing keyword, respectively.

Name	Description	Ending of `segment_id`
`major_ascent`	takeoff from the airport until a constant flight level is reached	`ma`
`major_descent`	last descent from a constant flight level before landing at the airport	`md`

Straight segment with constant flight altitude#

Straight flight segments with constant flight altitude are classified into three height levels.

Name	Description	Ending of `segment_id`
`low_level`	Constant flight level at < 1 km altitude	`ll`
`mid_level`	Constant flight level at 1-2 km altitude	`ml`
`high_level`	Constant flight level at > 2 km altitude	`hl`

Ascents and descents#

Ascents and descents are classified by two properties: (1) the height difference between lowest and highest point, and (2) the vertical velocity. Note that sometimes ascents or descents may be flown within curves.

Name	Description	Ending of `segment_id`
`small_ascent`	Ascent with height difference < 1 km	`sa`
`small_descent`	Descent with height difference < 1 km	`sd`
`medium_ascent`	Ascent with height difference of 1-2 km	`ma`
`medium_descent`	Descent with height difference of 1-2 km	`md`
`large_ascent`	Ascent with height difference > 2 km	`la`
`large_descent`	Descent with height difference > 2 km	`ld`
`profiling`	Ascent or descent with vertical speed mostly between 5-10 m/s

Overflights, underflights and co-locations#

During many research flights, the research station in Ny-Alesund and the research vessel Polarstern are overflown, or the satellite constellation A-train is underflown. These segments are marked and easily accessible. Additionally, co-locations between aircraft are marked.

Name	Description	Ending of `segment_id`
`nya_overflight`	Overflight over the Ny-Alesund research station within 15 km. Might be combined with `cross_pattern` or `high_level`
`polarstern_overflight`	Overflight over RV Polarstern within 15 km. Might be combined with `cross_pattern`
`a-train_underflight`	Underflight of the A-Train satellite constellation. For Polar 5, it is usually combined with `high_level`. For Polar 6 it might be any profiling pattern
`sveabreen_glacier_overflight`	Overflight over the Sveabreen glacier, Svalbard, mostly during `major_ascent`
`p6_co-location`	Co-location of Polar 5 with Polar 6
`p5_co-location`	Co-location of Polar 6 with Polar 5

Instrument calibration and testing#

Especially in the beginning of campaigns, maneuvers are flown to test or calibrate instrumentation. The keywords for these segments are listed in the table below.

Name	Description	Ending of `segment_id`
`radiometer_calibration`	Segment dedicated for radiometer calibration
`instrument_testing`	Segment for instrument testing, usually at the beginning of campaigns, where various maneuvers are flown	`it`

Curves and circles#

Different curves and circles are flown, depending on the angle or flight direction. These segments are not described by further keywords. Note, that segments, which are only curves (without ascent or descent), do not have the keywords irregularities, segment_id, levels, and dropsondes.

Name	Description	Ending of `segment_id`
`short_turn`	Turn directly to another flight direction (cutting corner)
`long_turn`	Turn on the long way to another flight direction (without cutting corner)
`procedure_turn`	Turn of 180° to be directly on track again in opposite direction
`360_turn`	360° circle, which is not part of a specific pattern
`cross_pattern_turn`	Transfer leg during `cross_pattern`

Patterns#

Different flight patterns with each specific properties were flown during the campaigns. Each pattern segment consists of sub-segments, which are listed under the keyword parts. Every part can be one of the above mentioned segments.

Name	Description	Ending of `segment_id`
`cross_pattern`	Rectangular crosses with transfer legs in between, usually flown over Ny-Alesund or Polarstern	`cr`
`racetrack_pattern`	Back and forth on the same track, with possible height changes	`rt`
`holding_pattern`	Time spending or waiting	`ho`
`stairstep_pattern`	Step wise ascenting or descenting along a straight line	`ss`
`sawtooth_pattern`	Alternating ascents and descents along a straight line	`st`
`radiation_square`	Square pattern at high altitude for radiation measurements	`rs`
`noseboom_calibration_pattern`	Pattern dedicated to calibrate the nose boom	`np`

Template of `.yaml` file#

Below, a template of a flight-phase-separation .yaml file is shown. A description of the keywords is presented in the previous sections.

name: RF04
mission: ACLOUD
platform: P5
flight_id: ACLOUD_P5_RF04
contacts:
- name:
  email:
  tags:
  - pi  # PI of flight
- name:
  email:
  tags:
  - lc  # list creator
date: 2017-05-23
flight_report: null
takeoff: 2017-05-23 09:11:47
landing: 2017-05-23 14:23:40
events: []
remarks: []
segments:
- kinds:
  - major_ascent
  name: major ascent 1
  irregularities: []
  segment_id: ACLOUD_P5_RF04_ma
  start: 2017-05-23 09:11:47
  end: null
  levels: []
  dropsondes: []
- kinds:
  - add_new_segments_here
  name: null
  irregularities: []
  segment_id: null
  start: null
  end: null
  levels: []
  dropsondes: []
- kinds:
  - major_descent
  name: major descent 1
  irregularities: []
  segment_id: ACLOUD_P5_RF04_md
  start: null
  end: 2017-05-23 14:23:40
  levels: []
  dropsondes: []