API Reference¶
Top-level package for fv3config.
-
class
fv3config.
DiagFieldConfig
(module_name: str, field_name: str, output_name: str, time_sampling: str = 'all', reduction_method: str = 'none', regional_section: str = 'none', packing: fv3config.config.diag_table.Packing = <Packing.SINGLE_PRECISION: 2>)[source]¶ Bases:
object
Object representing configuration for a field of a diagnostics file.
Parameters: - module_name – Name of Fortran module containing diagnostic.
- field_name – Name of diagnostic within Fortran code.
- output_name – Name of diagnostic to use in output NetCDF.
- time_sampling – Always set to ‘all’.
- reduction_method – One of ‘none’, ‘average’, ‘min’, ‘max’.
- regional_section – ‘none’ or region specification.
- packing – precision for output data.
-
packing
= 2¶
-
reduction_method
= 'none'¶
-
regional_section
= 'none'¶
-
time_sampling
= 'all'¶
-
class
fv3config.
DiagFileConfig
(name: str, frequency: int, frequency_units: str, field_configs: Sequence[fv3config.config.diag_table.DiagFieldConfig], file_format: fv3config.config.diag_table.FileFormat = <FileFormat.NETCDF: 1>, time_axis_units: str = 'hours', time_axis_name: str = 'time')[source]¶ Bases:
object
Object representing a diagnostics file configuration.
Parameters: - name – Name to use for NetCDF files, not including ‘.tile?.nc’.
- frequency – Period between records in file.
- frequency_units – One of ‘years’, ‘months’, ‘days’, ‘hours’, ‘minutes’, ‘seconds’
- field_configs – Sequence of DiagFieldConfigs defining fields to save.
- file_format – Always FileFormat.NETCDF.
- time_axis_units – Units for time coordinate in output files. One of ‘years’, ‘months’, ‘days’, ‘hours’, ‘minutes’, ‘seconds’.
- time_axis_name – Name for time coordinate in output files.
-
file_format
= 1¶
-
time_axis_name
= 'time'¶
-
time_axis_units
= 'hours'¶
-
class
fv3config.
DiagTable
(name: str, base_time: datetime.datetime, file_configs: Sequence[fv3config.config.diag_table.DiagFileConfig])[source]¶ Bases:
object
Representation of diag_table, which controls Fortran diagnostics manager.
Note
This implementation is based on the diag_table specification described in
- https://data1.gfdl.noaa.gov/summer-school/Lectures/July16/03_Seth1_DiagManager.pdf
- The MOM6 documentation has a useful description as well: https://mom6.readthedocs.io/en/latest/api/generated/pages/Diagnostics.html.
Parameters: - name – label used as attribute in output diagnostic files. Cannot contain spaces.
- base_time – time to be used as reference for time coordinate units.
- file_configs – sequence of DiagFileConfig’s defining the diagnostics to be output.
-
exception
fv3config.
InvalidFileError
[source]¶ Bases:
FileNotFoundError
Raised when a specified file is invalid, either non-existent or not as expected.
-
class
fv3config.
Packing
[source]¶ Bases:
enum.Enum
An enumeration.
-
DOUBLE_PRECISION
= 1¶
-
SINGLE_PRECISION
= 2¶
-
-
fv3config.
asset_list_from_path
(from_location, target_location='', copy_method='copy')[source]¶ Return asset_list from all files within a given path.
Parameters: - location (str) – local path or google cloud storage url.
- target_location (str, optional) – target_location used for generated assets. Defaults to ‘’ which is root of run-directory.
- copy_method ('copy' or 'link', optional) – whether to copy or link assets, defaults to ‘copy’. If location is a google cloud storage url, this option is ignored and files are copied.
Returns: a list of asset dictionaries
Return type: list
-
fv3config.
config_from_namelist
(namelist_filename)[source]¶ Read a configuration dictionary from a namelist file.
Only reads the namelist configuration.
Parameters: namelist_filename (str) – a namelist filename Returns: a configuration dictionary Return type: return_dict (dict) Raises: InvalidFileError
– if the specified filename does not exist
-
fv3config.
config_to_namelist
(config, namelist_filename)[source]¶ Write the namelist of a configuration dictionary to a namelist file.
Parameters: - config (dict) – a configuration dictionary
- namelist_filename (str) – filename to write, will be overwritten if present
-
fv3config.
do_remote_caching
(flag: bool)[source]¶ Set whether to cache remote files when accessed. Default is True.
-
fv3config.
dump
(config: Mapping[str, Any], f: TextIO)[source]¶ Serialize config to a file-like object using yaml encoding
Parameters: - config – an fv3config object
- f – the file like object to write to
-
fv3config.
enable_nudging
(config: Mapping[KT, VT_co]) → Mapping[KT, VT_co][source]¶ Return config object with necessary nudging file assets and associated file_names namelist entry. Requires ‘gfs_analysis_data’ entry in fv3config object with ‘url’ and ‘filename_pattern’ entries.
Parameters: config – configuration dictionary Raises: ConfigError
– if provided config does not contain “gfs_analysis_data” section.Note
will delete any existing assets in ‘patch_files’ that match the given filename_pattern before new assets are added.
-
fv3config.
enable_restart
(config, initial_conditions)[source]¶ Apply namelist settings for initializing from model restart files.
Parameters: - config (dict) – a configuration dictionary
- initial_conditions (str) – path to desired new initial conditions.
Returns: a configuration dictionary
Return type: dict
-
fv3config.
get_asset_dict
(source_location, source_name, target_location='', target_name=None, copy_method='copy')[source]¶ Helper function to generate asset for a particular file
Parameters: - source_location (str) – path to directory containing source file
- source_name (str) – filename of source file
- target_location (str, optional) – sub-directory to which file will be written, relative to run directory root. Defaults to empty string (i.e. root of run directory).
- target_name (str, optional) – filename to which file will be written. Defaults to None, in which case source_name is used.
- copy_method (str, optional) – flag to determine whether file is copied (‘copy’) or hard-linked (‘link’). Defaults to ‘copy’.
Returns: an asset dictionary
Return type: dict
-
fv3config.
get_bytes_asset_dict
(data: bytes, target_location: str, target_name: str)[source]¶ Helper function to define the necessary fields for a binary asset to be saved at a given location.
Parameters: - data – the bytes to save
- target_location – sub-directory to which file will be written, relative to run directory root. Defaults to empty string (i.e. root of run directory).
- target_name – filename to which file will be written. Defaults to None, in which case source_name is used.
Returns: an asset dictionary
Return type: dict
-
fv3config.
get_nudging_assets
(run_duration: datetime.timedelta, current_date: Sequence[int], nudge_path: str, nudge_filename_pattern: str = '%Y%m%d_%HZ_T85LR.nc', copy_method: str = 'copy', nudge_interval: datetime.timedelta = datetime.timedelta(seconds=21600)) → List[Mapping[KT, VT_co]][source]¶ Return list of assets of nudging files required for given run duration and start time.
This method defines file paths directly from its arguments, without determining whether the files themselves are present.
Parameters: - run_duration – length of fv3gfs run
- current_date – start time of fv3gfs run as a sequence of 6 integers
- nudge_path – local or remote path to nudging files
- nudge_filename_pattern – template for nudging filenames. Pattern should follow style of datetime strptime and strftime ‘format’ argument. Defaults to ‘%Y%m%d_%HZ_T85LR.nc’.
- copy_method – copy_method for nudging file assets. Defaults to ‘copy’.
- nudge_interval – time between nudging files. Must be multiple of 1 hour. Defaults to 6 hours.
Returns: list of all assets required for nudging run
Raises: ConfigError
– if copy_method is “link” and a remote path is given for nudge_path
-
fv3config.
get_run_duration
(config)[source]¶ Return a timedelta indicating the duration of the run.
Parameters: config (dict) – a configuration dictionary Returns: the duration of the run Return type: duration (timedelta) Raises: ValueError
– if the namelist contains a non-zero value for “months”
-
fv3config.
get_timestep
(config)[source]¶ Get the model timestep from a configuration dictionary.
Parameters: config (dict) – a configuration dictionary Returns: the model timestep Return type: datetime.timedelta
-
fv3config.
load
(f: TextIO) → Mapping[str, Any][source]¶ Load a configuration from a file-like object f
-
fv3config.
run_docker
(config_dict_or_location, outdir, docker_image, runfile=None, keyfile=None, capture_output=True)[source]¶ Run the FV3GFS model in a docker container with the given configuration.
Copies the resulting directory to a target location. Will use the Google cloud storage key at
$GOOGLE_APPLICATION_CREDENTIALS
by default. Requires the fv3gfs-python package and fv3config to be installed in the docker image.Parameters: - config_dict_or_location (dict or str) – a configuration dictionary, or a location (local or on Google cloud storage) of a yaml file containing a configuration dictionary
- outdir (str) – location to copy the resulting run directory
- runfile (str, optional) – Python model script to use in place of the default.
- docker_image (str, optional) – If given, run this command inside a container using this docker image. Image must have this package and fv3gfs-python installed.
- keyfile (str, optional) – location of a Google cloud storage key to use inside the docker container
- capture_output (bool, optional) – If true, then the stderr and stdout streams will be redirected to the files outdir/stderr.log and outdir/stdout.log respectively.
-
fv3config.
run_kubernetes
(config_location, outdir, docker_image, runfile=None, jobname=None, namespace='default', memory_gb=3.6, memory_gb_limit=None, cpu_count=1, gcp_secret=None, image_pull_policy='IfNotPresent', job_labels=None, submit=True, capture_output=True)[source]¶ Submit a kubernetes job to perform a fv3run operation.
Much of the configuration must be first saved to google cloud storage, and then supplied via paths to that configuration. The resulting run directory is copied out to a google cloud storage path. This call is non-blocking, and only submits a job.
Parameters: - config_location (str) – google cloud storage location of a yaml file containing a configuration dictionary
- outdir (str) – google cloud storage location to upload the resulting run directory
- docker_image (str) – docker image name to use for execution, which has fv3config installed with fv3run
- runfile (str, optional) – location of a python file to execute as the model executable, either on google cloud storage or within the specified docker image
- jobname (str, optional) – name to use for the kubernetes job, defaults to a random uuid.uuid4().hex
- namespace (str, optional) – kubernetes namespace for the job, defaults to “default”
- memory_gb (float, optional) – gigabytes of memory required for the kubernetes worker, defaults to 3.6GB
- memory_gb_limit (float, optional) – maximum memory allowed for the kubernetes worker, defaults to the value set by memory_gb
- cpu_count (int, optional) – number of CPUs to use on the kubernetes worker
- gcp_secret (str, optional) – name of kubernetes secret to mount containing a
file
key.json
to use as the google cloud storage key. - image_pull_policy (str, optional) – pull policy passed on to the kubernetes job. if set to “Always”, will always pull the latest image. When “IfNotPresent”, will only pull if no image has already been pulled. Defaults to “IfNotPresent”.
- job_labels (Mapping[str, str], optional) – labels provided as key-value pairs to apply to job pod. Useful for grouping jobs together in status checks.
- capture_output (bool, optional) – If True, then the stderr and stdout streams will be redirected to the files outdir/stderr.log and outdir/stdout.log respectively.
-
fv3config.
run_native
(config_dict_or_location, outdir, runfile=None, capture_output: bool = True)[source]¶ Run the FV3GFS model with the given configuration.
Copies the resulting directory to a target location. Will use the Google cloud storage key at
$GOOGLE_APPLICATION_CREDENTIALS
by default. Requires the fv3gfs-python package.Parameters: - config_dict_or_location (dict or str) – a configuration dictionary, or a location (local or on Google cloud storage) of a yaml file containing a configuration dictionary
- outdir (str) – location to copy the resulting run directory
- runfile (str, optional) – Python model script to use in place of the default.
- capture_output (bool, optional) – If true, then the stderr and stdout streams will be redirected to the files outdir/stderr.log and outdir/stdout.log respectively.
-
fv3config.
set_run_duration
(config: dict, duration: datetime.timedelta) → dict[source]¶ Set the run duration in the configuration dictionary.
Returns a new configuration dictionary.
Parameters: - config (dict) – a configuration dictionary
- duration (timedelta) – the new run duration
Returns: configuration dictionary with the new run duration
Return type: new_config (dict)
Raises: ConfigError
– if the config dictionary is invalid