First Steps with PDAF

When you have downloaded PDAF, a good starting point is to run a tutorial example. The directory tutorial/ contains files for tutorials demonstrating the implementation and application of PDAF with a simple 2-dimensional example.

Here we describe the steps for a Linux-based computer.

First Test Case - A Single Analysis Step

In this test case, a data assimilation program is used to read an ensemble of model fields from files and compute an analysis step. This is the so-called offline coupled mode of PDAF. A full description of this test case is provided in the implementation tutorial for the offline mode of PDAF.

Compiling

We recommend to first look at the tutorial offline_2D_serial, which is a single analysis step acting on an ensemble of 2D model fields without any parallelization.

Change to the tutorial directory with

cd tutorial/offline_2D_serial

and run

make PDAF_ARCH=linux_gfortran

This compiles the assimilation program including PDAF. The compilation should work for computers running Linux. If the compilation fails, please see below the section Compilation Problems.

Running

Having compiled the program, you can just run it by executing

./PDAF_offline

The program reads ensemble files and a file holding the observations from the directory tutorial/inputs_offline/. Then, it computes a single analysis step of the ensemble Kalman filter ESTKF. Running the program should only take seconds. The program generates the output files

• state_ana.txt (the analysis state)
• ens_01_ana.txt to ens_09_ana.txt (the analysis ensemble).

The screen output shows the progress of the program. For example, the ensemble standard deviation before and after the analysis step and final timing and memory information are shown. Lines starting with 'PDAF' are outputs from the core part of PDAF; other lines are from the user routines.

Plotting

You can plot using Python with, for example

import numpy as np
import matplotlib.pyplot as plt

file = 'state_ana.txt'

field = np.loadtxt(file)

plt.pcolor(field)
plt.show()

Analogously you can plot the observations (obs.txt) and the true state (true.txt) from which the observations have been generated. These files are in the input directory. In the observation file, only 28 grid points are observed, while non-observed grid points have the value -999.0. To get a meaningful plot, you can specify the color limits by

plt.clim([-1.5, 1.5])

before showing the plot.

Assimilation Options

There are various options you can set on the command line to modify the assimilation.

For example you can run

./PDAF_offline -filtertype 7

to apply the localized filter LESTKF instead of the global ESTKF. Without further settings, the localization radius is set to 0.0 so that only the observed grid points are changed by the assimilation. You can further set the localization cut-off radius with

./PDAF_offline -filtertype 7 -cradius 5.0

Now, the LESTKF is used with a localization radius of 5 grid points. This localization uses a constant weight of the observation. So you will see steps in the analysis fields around each observation. To add a tapering so that observations get less influence for increasing distance, use

./PDAF_offline -filtertype 7 -cradius 5.0 -locweight 2

Now, the filter is applied with the 5th-order polynomial function by Gaspari and Cohn (1999). As a result you get a smoothly varying analysis field. You can also change the ensemble size, e.g. running

./PDAF_offline -dim_ens 5

to run with an ensemble of 5 model states. (For this test case we only prepared 9 ensemble files, so only dim_ens<=9 is possible to run here. Please note that such ensemble size is usually too low for real cases) The standard deviation (RMS error) of the observation is set to 0.5 in the program. To change it to, e.g. 2.0, you can run

./PDAF_offline -rms_obs 2.0

Also the inflation can be specified on the command line. PDAF uses the so-called forgetting factor, which is a positive value <=1.0 (the ensemble variance is inflated by the inverse of the forgetting factor). One can specify the forgetting factor as

./PDAF_offline -forget 0.9

All the different options can be combined. For a complete list of possible options, see the file mod_assimilate.F90, which is the source code file in which the default values of options are declared and explained.

Second Test Case - A Sequence of Analysis Steps

As a second test case, we recommend to look at the tutorial online_2D_serialmodel. This case is again a simple 2D model field, but now coupled to PDAF with time stepping. This is the so-called online coupling of PDAF, in which the model code is augmented with data assimilation functionality provided by PDAF. A full description of this test case is provided in the implementation tutorial for the online mode of PDAF.

Compiling

Change to the tutorial directory with

cd ../online_2D_serialmodel

and run

make model_pdaf PDAF_ARCH=linux_gfortran_openmpi

This compiles the assimilation program including PDAF. The compilation should work for computers running Linux, but it requires that OpenMPI is installed on the computer. If it is not installed, please install the Linux package providing it. If the compilation still fails, please see below the section Compilation Problems.

Running

Having compiled the program, you can just run it by executing

mpirun -np 5 ./model_pdaf -dim_ens 5

The program computes a sequence of 9 analysis steps with of forecase phase of 2 time steps (thus two model time steps are computed in between subsequent analysis steps). The initial ensemble files are read from the directory tutorial/inputs_online/, where also the observation files are stored. The assimilation uses of the ensemble Kalman filter ESTKF. It should not take more than a few seconds. The program generates the output files

• state_stepX_ana.txt (the analysis state at time step X)
• ens_01_stepX_ana.txt to ens_09_stepX_ana.txt (the analysis ensemble at time step X)
• ens_01_stepX_for.txt to ens_09_stepX_for.txt (the forecast ensemble at time step X)

Plotting

You can plot the analysis fields, but also the observations, true fields and the initial state estimate as described in the first test case. However, the files for the observations and true fields are stored in the directory inputs_online/

You can plot using Python with, for example

import numpy as np
import matplotlib.pyplot as plt

file = 'state_step10_ana.txt'

field = np.loadtxt(file)

plt.pcolor(field)
plt.show()

Analogously, you can also plot the initial model field state_ini.txt, the true state (true_stepX.txt), or observations (obs_stepX.txt) with X the time step.

Assimilation Options

The same options as for the first test case can be used here, too.

In addition, one can specify the forecast length (number of time steps between two analysis steps) by

mpirun -np 9 ./model_pdaf -dim_ens 9 -delt_obs 6

To change the ensemble size to 6 states, you can use for example

mpirun -np 6 ./model_pdaf -dim_ens 6 -filtertype 7 -cradius 3.0

which chooses to run the LESTKF with a localization radius of 3 grid points. (Please note: The value behind -np must always set to be equal to the value given for -dim_ens. For this test case we only prepared 9 initial ensemble files, so only dim_obs<=9 is possible to run here.)

Compilation Problems

For the compilation, you need make. This should be installed on any computer running Linux, Unix or MacOS. If it is missing, you cannot compile and should install make.

The compilation might fail with an error mentioning blas or lapack. These are libraries for matrix computations, which are used by PDAF for performance reasons. Both libraries are usually installed on Linux computers. If these libraries are missing, please install them from the Linux packages of your Linux distribution. Then compile the tutorial example again.

For compilation on computers different from standard Linux or with a different compiler than gfortran, the directory make.arch/ provides include files for the compilation. To check for a suitable include file look into the directory make.arch/. There are files for compilation on different computers with differnt variants of the parallelization library MPI. If you don't find a suitable include file, you might also copy an existing file and edit it for your needs. To specify the include file for the compilation, you just need to set it when running make as

make PDAF_ARCH=FILENAME_WITHOUT_.h

On MacOS there is usually no gfortran installed. You can install it, e.g., using the Homebrew (brew.sh). Likewise, you can install OpenMPI. Then setting PDAF_ARCH=osx_gfortran_openmpi should work to compile the test cases.

Next steps

Having done your first experiments with PDAF, possible next steps can be the following:

Applying data assimilation with PDAF to the Lorenz-96 model

The implementation of PDAF with the Lorenz-96 model is a fully-featured example of PDAF coupled to this small model. Next to the Lorenz-96 model and assimilation user routines, the example provides tools for generating synthetic observation, a covariance matrix for ensemble generation and plotting scripts. We used this implementation for different publications where we studied the behavior of different assimilation methods.

Applying data assimilation with PDAF to the Lorenz-63 model

The implementation of PDAF with the Lorenz-63 model is a fully-featured example of PDAF coupled to this small model. Next to the Lorenz-63 model and assimilation user routines, the example provides tools for generating synthetic observation, a covariance matrix for ensemble generation and plotting scripts. The modul only has 3 state variables, so it's too small for localization, but it's a good example for applying the particle filter. We used this implementation for our publication on the hybrid nonlinear-Kalman filter, LKNETF.

Implementing PDAF with your model

If you plan to couple PDAF with your model, we recommend to study the PDAF implementation tutorials. The tutorial provides a step-by-step explanation of the implementation steps. Further, please see the description of the Implementation Concept of PDAF. PDAF also provides tools to generate an ensemble.

Beyond the more applied tutorials there are also the Implementation Guids for online coupled and offline coupled data assimilation systems.

Model Couplings

Coupling routines (also called 'model bindings') to different models, and related user routines for the data assimilation with PDAF are available in different repositories of ​https://github.com/PDAF

Current model bindings include the MITgcm general circulation model, NEMO ocean model, AWI climate model and FESOM ocean model, WRF. See the page: List of models connected to PDAF.

Full overview of PDAF code Package

To get an overview of what is overall contained in the PDAF package, please see the full description of the PDAF code package.