out2nc v1.5.1

conversion utility from standard LPJ-GUESS output file format to netCDF. No warranties provided, but it should not eat your data (it never did before).

Usage

out2nc [-h] [-f FILE] [-d DIR] [--init-config [FILE]] [--overwrite] [-v,--version] [paths [paths ...]]

Options

-f FILE, --file FILE
Explicitly specify file to use as a configuration file. Without this switch program attempts to locate a .cfg file in the current working directory and in case only a single file found, it use it.
-d DIR, --dir DIR
Directory name where data files will be located, path to configuration files are not affected by the value of this parameter. The output files will be placed into this directory, so make sure you have write permissions.
-s YEAR, --start_year YEAR
Force first year in output. This value would be used for time.calendar attribute and it will override the content of the file. Useful when the simulation year is present in the out files, but the calendar year is desired for the netcdf files.
--init-config FILE
write sample config to file, filename is optional and defaults to default_config.cfg
--overwrite
Overwrite existing files with the newly created content
--ec-earth
Generate reduced Gaussian grid (N128 resolution). Certain metadata (offsets, bounding box and resolution) are ignored. This option also allows processing of the output which is split into <year-directory>/<processor-number>structure.
--verbose
Produce detailed output in case of an error
paths
Only these files will be considered for conversion, if none passed all the existing entries in configuration file will be considered. Note this paths are not affected by the -d/--dir switch, so that full relative or absolute paths need to be provided, rather than file names only. If these paths do not have corresponding entries in the configuration files, a strip-down conversion will be attempted.

Requirements

This program is written in Python and requires at minimum version 3.4 of the interpreter. Along with it the following packages are mandatory: numpy, netCDF4 and pandas. All of them can be installed using pip. On recent Debian-based distribution: installing python3-netcdf4 and python3-pandas would be sufficient.

Configuration

To create proper standard-compliant files, the tool relies on provision of metadata by the user. The configuration file should be in ini format (without relying on interpolation or Windows Registry features). The configuration filename can be passed using the -f/--file switch otherwise the program will try to use a .cfg file from the current directory (only when a single .cfg file found).

Without a specified configuration a strip-down conversion with default settings would be attempted, but is likely to be incorrect and not standard-compliant. A sample configuration file can be obtained by passing --init-config with an optional filename. Content of the configuration file should be as follows:

Metadata section

The section should be named metadata in lower case. This section may contain both freeform options that will be turned into global attributes as well as the following options:

extension
input filename extension, defaults to .out
format
one of netcdf4, netcdf4_classic, netcdf3_classic or netcdf3_64bit, defaults to netcdf4_classic. Is a format of a generated netCDF file.
missing
value used to fill missing values, defaults to 9.969e36
south, north, west, east
bounding box values in degrees
resolution
resolution of the (square) grid, defaults to 0.5
lon_offset, lat_offset
offset from the centre of the gridcell. E.g., lower left corner coordinates are both offset by -0.25 deg.
filename_format
output filename format, optional, defaults to {var}.nc

Possible elements:

for example, the following value of filename_format:

lpj-guess_{gparent}_{parent}_{var}_{tres}_{start}_{end}.nc4

could be interpolated to the final output filename:

lpj-guess_gcm_rcp_tair_year_1901_2001.nc4

For an annual file of air temperatures located in gcm/rcp directory.

Remaining sections

Each remaining section will represent a single variable that could be extracted from the .out files. The name of the section will be taken as a variable name as well as used as {var} element for filename. Therefore, section (variable) names must be unique. Each section may have the following options:

name, standard_name, units
Freeform values, but generally intended for CF convention compliance. Options' names and values are not checked for correctness
file
input filename, extension can be omitted. file does not have to exist, in which case give variable will be skipped
column
column name or 0-based index (excluding coordinate and year) in input file; should not be specified for monthly variables, will be ignored
conversion
only basic operations are currently supported, allowed operators: +, -, *, / followed by a number, e.g., +273.15, would convert from degrees Celsius to Kelvin.

Variable expansion

If the section name or a name option contain a {}, the variable will be replaced with a number of variables corresponding to the column names in the file. For example, the following section:

 [fpc_{}]
 name: FPC of {}
 standard_name: fpc
 file: fpc.out

referring to the fpc.out with columns: trees, grass and total; will be internally replaced by three variables as follows:

 [fpc_trees]
 name: FPC of trees
 standard_name: fpc
 file: fpc.out
 column: trees

 [fpc_grass]
 name: FPC of grass
 standard_name: fpc
 file: fpc.out
 column: grass

 [fpc_total]
 name: FPC of total
 standard_name: fpc
 file: fpc.out
 column: total

Only variable (section) name and name option are affected by this expansion. The '{}' can be present only in section name or name option, i.e., it does not have be found in both elements.

Details

Author: Michael Mischurow
Date:2016-08-26
Copyright:public domain
Version:1.5