Supported YAML Tags#

YAML allows defining custom so-called tags which can be distinguished during loading and serialization of objects. paramspace makes heavy use of this possibility, as it greatly simplifies the definition and usage of configuration files.

Parameter space tags#

The paramspace.yaml module implements constructors and representers for the following classes:

This is a very convenient way of defining these objects.

Hint

For the ParamDim and derived classes, there additionally are the !pdim-default and !coupled-pdim-default tags. These do not create a ParamDim objects but directly return the default value. By adding the -default in the end, they can be quickly deactivated inside the configuration file (as an alternative to commenting them out).

Python builtins and basic operators#

paramspace.yaml adds YAML constructors for a number of frequently used Python built-in functions and operators. Having these available while specifying configurations can make the definition of configurations files more versatile.

Warning

The YAML tags provided here are only meant to allow basic operations, i.e. summing two parameters to create a third. Don’t overdo it. Configuration files should remain easy to read.

The tags shown below call the equivalent Python builtin or the operators defined in the :py:mod`operator` module. Example:

any:      !any        [false, 0, true]    # == True
all:      !all        [true, 5, 0]        # == False

abs:      !abs        -1          # +1
int:      !int        1.23        # 1
round:    !round      9.87        # 10
sum:      !sum        [1, 2, 3]   # 6
prod:     !prod       [2, 3, 4]   # 24

min:      !min        [1, 2, 3]   # 1
max:      !max        [1, 2, 3]   # 3

sorted:   !sorted     [2, 1, 3]   # [1, 2, 3]
isorted:  !isorted    [2, 1, 3]   # [3, 2, 1]

# Operators
add:      !add        [1, 2]      # 1 + 2
sub:      !sub        [2, 1]      # 2 - 1
mul:      !mul        [3, 4]      # 3 * 4
mod:      !mod        [3, 2]      # 3 % 2
pow:      !pow        [2, 4]      # 2 ** 4
truediv:  !truediv    [3, 2]      # 3 / 2
floordiv: !floordiv   [3, 2]      # 3 // 2
pow_mod:  !pow        [2, 4, 3]   # 2 ** 4 % 3

not:      !not        [true]
and:      !and        [true, false]
or:       !or         [true, false]
xor:      !xor        [true, true]

lt:       !lt         [1, 2]      # 1 <  2
le:       !le         [2, 2]      # 2 <= 2
eq:       !eq         [3, 3]      # 3 == 3
ne:       !ne         [3, 1]      # 3 != 1
ge:       !ge         [2, 2]      # 2 >= 2
gt:       !gt         [4, 3]      # 4 >  3

negate:   !negate     [1]             # -1
invert:   !invert     [true]          # ~true
contains: !contains   [[1,2,3], 4]    # 4 in [1,2,3] == False

concat:   !concat     [[1,2,3], [4,5], [6,7,8]]  # […]+[…]+[…]+…

# List generation
# ... using the paramspace.tools.create_indices function
list1:    !listgen    [0, 10, 2]   # [0, 2, 4, 6, 8]
list2:    !listgen
  from_range: [0, 10, 3]
  unique: true
  append: [100]
  remove: [0]
  sort: true

# ... using np.linspace, np.logspace, np.arange
lin:      !linspace   [-1, 1, 5]   # [-1., -.5, 0., .5, 1.]
log:      !logspace   [1, 4, 4]    # [10., 100., 1000., 10000.]
arange:   !arange     [0, 1, .2]   # [0., .2, .4, .6, .8]

# String formatting
format1:  !format     ["{} is not {}", foo, bar]
format2:  !format
  fstr: "{some_key:}: {some_value:}"
  some_key: fish
  some_value: spam
format3:  !format
  fstr: "results: {stats[mean]:.2f} ± {stats[std]:.2f}"
  stats:
    mean: 1.632
    std:  0.026

# Joining and splitting strings
joined_words: !join    # -> "foo | bar | baz"
  - " | "
  - [foo, bar, baz]

words: !split          # -> [there, are, many, words, in this, sentence]
  - there are many words in this sentence
  - " "
  # - 3                # optional `maxsplit` argument

Reading environment variables and handling paths#

Sometimes, parameters depend on the machine that a program is running on. To not have to load certain variables manually, paramspace supports loading environment variables via os.environ and simple path manipulations using os.path.expanduser() and os.path.join().

# Reading environment variables, optionally with fallback
PATH:             !getenv PATH   # fails if variable is missing
username:         !getenv [USER, "unknown_user"]
home_directory:   !getenv [HOME, "/"]

# Expanding a path containing `~`
some_user_path:   !expanduser ~/some/path

# Joining paths
some_joined_path: !joinpath      # -> "~/foo/bar/../spam.txt"
  - "~"
  - foo
  - bar
  - ..
  - spam.txt

Recursively updating maps#

While YAML already provides the << operator to update a mapping, this operator does not work recursively. The !rec-update YAML tag supplies exactly that functionality using the recursive_update() function.

some_map: &some_map
  foo: bar
  spam: fish
some_other_map: &some_other_map
  foo:
    bar: baz
    baz: bar
  fish: spam

# Create a new map by recursively updating the first map with
# the second one (uses deep copies to avoid side effects)
merged: !rec-update [<<: *some_map, <<: *some_other_map]
# NOTE: Need to use  ^^-- inheritance here, otherwise this will
#       result in empty mappings (for some reason)

Warning

Always include via <<: *my_ref!

If supplying references to mappings (as shown in the example), the references have to be included using <<: *my_ref!

Otherwise, if using the simple *my_ref as argument, the YAML parser does not properly resolve the reference to the anchor but only returns an empty mapping.