API Reference

Top-level package for fv3config.

exception fv3config.ConfigError[source]

Bases: ValueError

class fv3config.DiagFieldConfig(module_name: str, field_name: str, output_name: str, time_sampling: str = 'all', reduction_method: str = 'none', regional_section: str = 'none', packing: fv3config.config.diag_table.Packing = <Packing.SINGLE_PRECISION: 2>)[source]

Bases: object

Object representing configuration for a field of a diagnostics file.

Parameters:
  • module_name – Name of Fortran module containing diagnostic.
  • field_name – Name of diagnostic within Fortran code.
  • output_name – Name of diagnostic to use in output NetCDF.
  • time_sampling – Always set to ‘all’.
  • reduction_method – One of ‘none’, ‘average’, ‘min’, ‘max’.
  • regional_section – ‘none’ or region specification.
  • packing – precision for output data.
packing = 2
reduction_method = 'none'
regional_section = 'none'
time_sampling = 'all'
class fv3config.DiagFileConfig(name: str, frequency: int, frequency_units: str, field_configs: Sequence[fv3config.config.diag_table.DiagFieldConfig], file_format: fv3config.config.diag_table.FileFormat = <FileFormat.NETCDF: 1>, time_axis_units: str = 'hours', time_axis_name: str = 'time')[source]

Bases: object

Object representing a diagnostics file configuration.

Parameters:
  • name – Name to use for NetCDF files, not including ‘.tile?.nc’.
  • frequency – Period between records in file.
  • frequency_units – One of ‘years’, ‘months’, ‘days’, ‘hours’, ‘minutes’, ‘seconds’
  • field_configs – Sequence of DiagFieldConfigs defining fields to save.
  • file_format – Always FileFormat.NETCDF.
  • time_axis_units – Units for time coordinate in output files. One of ‘years’, ‘months’, ‘days’, ‘hours’, ‘minutes’, ‘seconds’.
  • time_axis_name – Name for time coordinate in output files.
file_format = 1
time_axis_name = 'time'
time_axis_units = 'hours'
class fv3config.DiagTable(name: str, base_time: datetime.datetime, file_configs: Sequence[fv3config.config.diag_table.DiagFileConfig])[source]

Bases: object

Representation of diag_table, which controls Fortran diagnostics manager.

Note

This implementation is based on the diag_table specification described in

https://data1.gfdl.noaa.gov/summer-school/Lectures/July16/03_Seth1_DiagManager.pdf
The MOM6 documentation has a useful description as well: https://mom6.readthedocs.io/en/latest/api/generated/pages/Diagnostics.html.
Parameters:
  • name – label used as attribute in output diagnostic files. Cannot contain spaces.
  • base_time – time to be used as reference for time coordinate units.
  • file_configs – sequence of DiagFileConfig’s defining the diagnostics to be output.
asdict()[source]
classmethod from_dict(diag_table: dict)[source]
classmethod from_str(diag_table: str)[source]

Initialize DiagTable class from Fortran string representation.

class fv3config.FileFormat[source]

Bases: enum.Enum

An enumeration.

NETCDF = 1
exception fv3config.InvalidFileError[source]

Bases: FileNotFoundError

Raised when a specified file is invalid, either non-existent or not as expected.

class fv3config.Packing[source]

Bases: enum.Enum

An enumeration.

DOUBLE_PRECISION = 1
SINGLE_PRECISION = 2
fv3config.asset_list_from_path(from_location, target_location='', copy_method='copy')[source]

Return asset_list from all files within a given path.

Parameters:
  • location (str) – local path or google cloud storage url.
  • target_location (str, optional) – target_location used for generated assets. Defaults to ‘’ which is root of run-directory.
  • copy_method ('copy' or 'link', optional) – whether to copy or link assets, defaults to ‘copy’. If location is a google cloud storage url, this option is ignored and files are copied.
Returns:

a list of asset dictionaries

Return type:

list

fv3config.config_from_namelist(namelist_filename)[source]

Read a configuration dictionary from a namelist file.

Only reads the namelist configuration.

Parameters:namelist_filename (str) – a namelist filename
Returns:a configuration dictionary
Return type:return_dict (dict)
Raises:InvalidFileError – if the specified filename does not exist
fv3config.config_to_namelist(config, namelist_filename)[source]

Write the namelist of a configuration dictionary to a namelist file.

Parameters:
  • config (dict) – a configuration dictionary
  • namelist_filename (str) – filename to write, will be overwritten if present
fv3config.do_remote_caching(flag: bool)[source]

Set whether to cache remote files when accessed. Default is True.

fv3config.dump(config: Mapping[str, Any], f: TextIO)[source]

Serialize config to a file-like object using yaml encoding

Parameters:
  • config – an fv3config object
  • f – the file like object to write to
fv3config.enable_restart(config, initial_conditions)[source]

Apply namelist settings for initializing from model restart files.

Parameters:
  • config (dict) – a configuration dictionary
  • initial_conditions (str) – path to desired new initial conditions.
Returns:

a configuration dictionary

Return type:

dict

fv3config.ensure_data_is_downloaded()[source]

Removed, do not use.

fv3config.get_asset_dict(source_location, source_name, target_location='', target_name=None, copy_method='copy')[source]

Helper function to generate asset for a particular file

Parameters:
  • source_location (str) – path to directory containing source file
  • source_name (str) – filename of source file
  • target_location (str, optional) – sub-directory to which file will be written, relative to run directory root. Defaults to empty string (i.e. root of run directory).
  • target_name (str, optional) – filename to which file will be written. Defaults to None, in which case source_name is used.
  • copy_method (str, optional) – flag to determine whether file is copied (‘copy’) or hard-linked (‘link’). Defaults to ‘copy’.
Returns:

an asset dictionary

Return type:

dict

fv3config.get_bytes_asset_dict(data: bytes, target_location: str, target_name: str)[source]

Helper function to define the necessary fields for a binary asset to be saved at a given location.

Parameters:
  • data – the bytes to save
  • target_location – sub-directory to which file will be written, relative to run directory root. Defaults to empty string (i.e. root of run directory).
  • target_name – filename to which file will be written. Defaults to None, in which case source_name is used.
Returns:

an asset dictionary

Return type:

dict

fv3config.get_cache_dir()[source]
fv3config.get_default_config()[source]

Removed, do not use.

fv3config.get_nudging_assets(run_duration: datetime.timedelta, current_date: Sequence[int], nudge_path: str, nudge_filename_pattern: str = '%Y%m%d_%HZ_T85LR.nc', copy_method: str = 'copy', nudge_interval: datetime.timedelta = datetime.timedelta(seconds=21600)) → List[Mapping[KT, VT_co]][source]

Return list of assets of nudging files required for given run duration and start time.

This method defines file paths directly from its arguments, without determining whether the files themselves are present.

Parameters:
  • run_duration – length of fv3gfs run
  • current_date – start time of fv3gfs run as a sequence of 6 integers
  • nudge_path – local or remote path to nudging files
  • nudge_filename_pattern – template for nudging filenames. Pattern should follow style of datetime strptime and strftime ‘format’ argument. Defaults to ‘%Y%m%d_%HZ_T85LR.nc’.
  • copy_method – copy_method for nudging file assets. Defaults to ‘copy’.
  • nudge_interval – time between nudging files. Must be multiple of 1 hour. Defaults to 6 hours.
Returns:

list of all assets required for nudging run

Raises:

ConfigError – if copy_method is “link” and a remote path is given for nudge_path

fv3config.get_run_duration(config)[source]

Return a timedelta indicating the duration of the run.

Parameters:config (dict) – a configuration dictionary
Returns:the duration of the run
Return type:duration (timedelta)
Raises:ValueError – if the namelist contains a non-zero value for “months”
fv3config.get_timestep(config)[source]

Get the model timestep from a configuration dictionary.

Parameters:config (dict) – a configuration dictionary
Returns:the model timestep
Return type:datetime.timedelta
fv3config.load(f: TextIO) → Mapping[str, Any][source]

Load a configuration from a file-like object f

fv3config.run_docker(config_dict_or_location, outdir, docker_image, runfile=None, keyfile=None, capture_output=True)[source]

Run the FV3GFS model in a docker container with the given configuration.

Copies the resulting directory to a target location. Will use the Google cloud storage key at $GOOGLE_APPLICATION_CREDENTIALS by default. Requires the fv3gfs-python package and fv3config to be installed in the docker image.

Parameters:
  • config_dict_or_location (dict or str) – a configuration dictionary, or a location (local or on Google cloud storage) of a yaml file containing a configuration dictionary
  • outdir (str) – location to copy the resulting run directory
  • runfile (str, optional) – Python model script to use in place of the default.
  • docker_image (str, optional) – If given, run this command inside a container using this docker image. Image must have this package and fv3gfs-python installed.
  • keyfile (str, optional) – location of a Google cloud storage key to use inside the docker container
  • capture_output (bool, optional) – If true, then the stderr and stdout streams will be redirected to the files outdir/stderr.log and outdir/stdout.log respectively.
fv3config.run_kubernetes(config_location, outdir, docker_image, runfile=None, jobname=None, namespace='default', memory_gb=3.6, memory_gb_limit=None, cpu_count=1, gcp_secret=None, image_pull_policy='IfNotPresent', job_labels=None, submit=True, capture_output=True)[source]

Submit a kubernetes job to perform a fv3run operation.

Much of the configuration must be first saved to google cloud storage, and then supplied via paths to that configuration. The resulting run directory is copied out to a google cloud storage path. This call is non-blocking, and only submits a job.

Parameters:
  • config_location (str) – google cloud storage location of a yaml file containing a configuration dictionary
  • outdir (str) – google cloud storage location to upload the resulting run directory
  • docker_image (str) – docker image name to use for execution, which has fv3config installed with fv3run
  • runfile (str, optional) – location of a python file to execute as the model executable, either on google cloud storage or within the specified docker image
  • jobname (str, optional) – name to use for the kubernetes job, defaults to a random uuid.uuid4().hex
  • namespace (str, optional) – kubernetes namespace for the job, defaults to “default”
  • memory_gb (float, optional) – gigabytes of memory required for the kubernetes worker, defaults to 3.6GB
  • memory_gb_limit (float, optional) – maximum memory allowed for the kubernetes worker, defaults to the value set by memory_gb
  • cpu_count (int, optional) – number of CPUs to use on the kubernetes worker
  • gcp_secret (str, optional) – name of kubernetes secret to mount containing a file key.json to use as the google cloud storage key.
  • image_pull_policy (str, optional) – pull policy passed on to the kubernetes job. if set to “Always”, will always pull the latest image. When “IfNotPresent”, will only pull if no image has already been pulled. Defaults to “IfNotPresent”.
  • job_labels (Mapping[str, str], optional) – labels provided as key-value pairs to apply to job pod. Useful for grouping jobs together in status checks.
  • capture_output (bool, optional) – If True, then the stderr and stdout streams will be redirected to the files outdir/stderr.log and outdir/stdout.log respectively.
fv3config.run_native(config_dict_or_location, outdir, runfile=None, capture_output: bool = True)[source]

Run the FV3GFS model with the given configuration.

Copies the resulting directory to a target location. Will use the Google cloud storage key at $GOOGLE_APPLICATION_CREDENTIALS by default. Requires the fv3gfs-python package.

Parameters:
  • config_dict_or_location (dict or str) – a configuration dictionary, or a location (local or on Google cloud storage) of a yaml file containing a configuration dictionary
  • outdir (str) – location to copy the resulting run directory
  • runfile (str, optional) – Python model script to use in place of the default.
  • capture_output (bool, optional) – If true, then the stderr and stdout streams will be redirected to the files outdir/stderr.log and outdir/stdout.log respectively.
fv3config.set_cache_dir(parent_dirname)[source]
fv3config.set_run_duration(config: dict, duration: datetime.timedelta) → dict[source]

Set the run duration in the configuration dictionary.

Returns a new configuration dictionary.

Parameters:
  • config (dict) – a configuration dictionary
  • duration (timedelta) – the new run duration
Returns:

configuration dictionary with the new run duration

Return type:

new_config (dict)

Raises:

ConfigError – if the config dictionary is invalid

fv3config.update_config_for_nudging(config: Mapping[KT, VT_co])[source]

Update config object in place to include nudging file assets and associated file_names namelist entry. Requires ‘gfs_analysis_data’ entry in fv3config object with ‘url’ and ‘filename_pattern’ entries.

Parameters:config – configuration dictionary
Raises:ConfigError – if provided config does not contain “gfs_analysis_data” section.

Note

will delete any existing assets in ‘patch_files’ that match the given filename_pattern before new assets are added.

fv3config.write_run_directory(config, target_directory)[source]

Write a run directory based on a configuration dictionary.

Parameters:
  • config (dict) – a configuration dictionary
  • target_directory (str) – target directory, will be created if it does not exist