Recommendations on the Choice of Filter Algorithm

PDAF provides several filter algorithms. Here, we provide some guidance about which filter algorithm one should use.

The short story

In short, we recommend the LESTKF filter, if a local filter is needed. For a global filter, we recommend the ESKTF. For smoothing, we recommend the smoother extensions of these two filters. In our studies by now, we obtained the best results (i.e. smallest assimilation errors) with these filters and smoothers. Further, these filters are very efficient also with very high-dimensional models.

The full story

The filter algorithms that are currently implemented in PDAF have been used in comparison studies giving insight in the performance of different filter formulations. In particular, the Ensemble Kalman Filter (EnKF, Evensen, 1994) was compared with the SEEK and SEIK filters (Pham et al., 1998) in Nerger et al. (2005) (the links refer to the page listing the full references of the publications). The SEIK filter was then related to the ETKF (Bishop, 2002) in Nerger et al. (2012). This study also introduced the ESTKF. Nerger et al. (2006) introduced the localized SEIK filter (LSEIK) and Nerger and Gregg (2007) added observation localization to the LSEIK filter. The local filters LETKF and LESTKF use the same localization method as the LSEIK filter.

Based on our studies, we generally recommend to use the ESTKF or its localized variant LESTKF. Nerger et al. (2005) already showed advantages of the SEIK filter over the SEEK filter and the EnKF. Nerger et al. (2012) showed that the ESTKF combines the advantages for the ETKF and the SEIK filter. In particular, the ESTKF can be used with a deterministic minimum transformation as the ETKF, but at a slightly lower computational cost. Unlike the SEIK filter, the transformation of the ensemble in the ESTKF is independent of the order of the ensemble members in the ensemble matrix. The ESTKF can also be used with a random transformation, which is often used in the SEIK filter. In this case, the matrix square root can be computed by a Cholesky decomposition, which is faster than the singular value decomposition used to compute the symmetric square root that is required for the minimum transformation. While the ESTKF provides these advantages, it is computationally not more expensive than the SEIK filter.

The choice whether a global filter like ESTKF or a local formulation as LESTKF is used depends on the problem that is simulated. If the model represents only large scale features, the global filter should be a good choice. If the model fields represent many small scale features, the local filter is required. Also the availability of observations influences the choice. If only a very small number of observations is available, it might be better to use the global filter as the observational information can be spear out over a larger region. In contrast, spatially resolved observations, like surface temperature fields of the ocean detected by satellites, call for the local filter.

Special cases like data assimilation with a fixed covariance matrix or a static covariance matrix are also supported in PDAF. In case of a fixed covariance matrix one obtains an ensemble optimal interpolation (ensemble OI) algorithm in which the covariance matrix from the initialization is used for all filter analysis steps. With a static covariance matrix, the ensemble members representing the covariance matrix are updated during the analysis step. However, in the forecast phase only the ensemble mean state is integrated by the model. These special cases are provided for all filters, except EnKF, LEnKF, and the weight-based filters NETF, LNETF, and PF. They can be selected by specifying subtype=2 for the static covariance matrix and subtype=3 for the fixed covariance matrix.

Examples of parameter settings

To use the global ESTKF with minimum transformation one has to specify (following the naming of variables used in the tutorial, for example in /tutorial/online_2D_serialmodel):

parameter	value	comment
filtertype	6	Selecting ESTKF
type_sqrt	0	This specifies the SVD-based symmetric square root
type_trans	0	This specifies the deterministic ensemble transformation

For the global ESTKF with random transformation and cholesky decompostion one specifies:

parameter	value	comment
filtertype	6	Selecting ESTKF
type_sqrt	1	This specifies the Cholesky square root
type_trans	1	This specifies the random transformation

For the local LESTKF with minimum transformation a localization radius of 10 grid points and observation localization by a 5th order polynomial one specifies:

parameter	value	comment
filtertype	7	Selecting LESTKF
type_sqrt	0	This specifies the SVD-based symmetric square root
type_trans	0	This specifies the deterministic ensemble transformation
locweight	4	Regulated observation localization by 5th order polynomial function
cradius	10	Set locallization cut-off radius of observations to 10 grid points

For the global ESTKF with fixed covariance matrix one specifies:

parameter	value	comment
filtertype	6	Selecting ESTKF
subtype	3	Select fixed covariance matrix

The available configuration options of the filters are provided on the overview page on filter-specific options. Some parameters are only used by the user-supplied call back routines. Such options are, for example, locweight and cradius that specify the localization. These options can be found in the tutorial implementation in /tutorial/online_2D_serialmodel.