Changes between Version 5 and Version 6 of ImplementAnalysisENSRF_EAKF


Ignore:
Timestamp:
Mar 26, 2025, 10:43:03 AM (6 days ago)
Author:
lnerger
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • ImplementAnalysisENSRF_EAKF

    v5 v6  
    3939|| This page describes the implementation of the analysis step without using PDAF-OMI. Please see the [wiki:ImplementationofAnalysisStep page on the analysis with OMI] for the more modern and efficient implementation variant using PDAF-OMI. ||
    4040
    41 The ENSRF/EAKF were added with verson 3.0 of PDAF.
     41The ENSRF/EAKF filters were added with verson 3.0 of PDAF.
    4242
    4343== Overview ==
    4444
    45 The ENSRF and EAKF are ensemble Kalman filter variants using serial observation processing. The implementation follows Houtekamer and Hamill (2002) for the ENSRF and Anderson (2003) for the EAKF variant using local least squares regression. The variant of the serial-observation processing filter is selected by the `subtype` in the call to `PDAF_init` as follows:
     45The ENSRF and EAKF are ensemble Kalman filter variants using serial observation processing. The implementation in PDAF follows Houtekamer and Hamill (2002) for the ENSRF and Anderson (2003) for the EAKF variant using local least squares regression. The parallelization follows Anderson and Collins (2007), where the 'high-latency' variant is implemented, since their 'low-latency' variant appears to be overly optimistic while still leading to a partial serialization of the execution.
     46
     47The variant of the serial-observation processing filter is selected by the `subtype` in the call to `PDAF_init` as follows:
    4648
    4749||= subtype =||= Filter variant =||
    … …  
    4951||= 1 =|| EAKF linear regression (Anderson, 2003) ||
    5052
    51 For the analysis step of the ENSRF and EAKF different operations related to the observations are needed. These operations are requested by PDAF by calling user-supplied routines. Intentionally, the operations are split into separate routines in order to keep the operations rather elementary. This procedure should simplify the implementation. The names of the required routines are specified in the call to the routine `PDAF_assimilate_ensrf` for the fully-parallel and flexible parallelization implementations (alternatively `PDAF_put_state_lenkf` for the 'flexible' implementation). With regard to the parallelization, all these routines are executed by the filter processes (`filterpe=.true.`) only.
    52 
    53 For completeness we discuss here all user-supplied routines that are specified in the interface to `PDAF_assimilate_enskf`. Thus, some of the user-supplied routines that are explained on the page explaining the modification of the model code for the ensemble integration are repeated here.
    54 
    55 In our study Nerger et al., 2015) we discussed that applying localization can lead to stability issues of the ENSRF. The filter performs a loop over all single observations and with localization the assimilation result depends on the order in which the observations are assimilated. This actually result in the effect that the assimilation result at some grid point does not only depend on the observations with the localization radius **r**, but also on observations further away if the influence the state close to the observations at distance **r** if those observations are assimilated before the observations within the radius **r**. This effect has implications on the parallelization since keeping the observation order constant does lead to a pertial serialization of the algorithm. In the implementation in PDAF we use the parallelization approach that does not guarantee the some order of the observations. Usually, the differences are small, but the benefit is a better scaling since the serialization is avoided. Nonetheless, we generally recommend using LESTKF or LETKF, or their global variants ESTKF or ETKF, since they no not depend explicitly on the observation order, and they allow for non-diagonal observation error covariance matrices. However, the ENSRF/EAKF might have a good compute performance.
     53For the analysis step of the ENSRF and EAKF, different operations related to the observations are needed. These operations are requested by PDAF by calling user-supplied routines. Intentionally, the operations are split into separate routines in order to keep the operations rather elementary. This procedure should simplify the implementation. The names of the required routines are specified in the call to the routine `PDAF_assimilate_ensrf` for the fully-parallel and flexible parallelization implementations (alternatively `PDAF_put_state_lenkf` for the 'flexible' implementation and for the offline-coupled assimilation). With regard to the parallelization, all these routines are executed by the filter processes (`filterpe=.true.`) only.
     54
     55For completeness we discuss here all user-supplied routines that are specified in the interface to `PDAF_assimilate_ensrf`. Thus, some of the user-supplied routines that are explained on the page explaining the modification of the model code for the ensemble integration are repeated here.
     56
     57In our study Nerger et al. (2015), we discussed that applying localization can lead to stability issues of the ENSRF. The filter performs a loop over all single observations and with localization the assimilation result depends on the order in which the observations are assimilated. This actually leads to the effect that the assimilation result at some grid point does not only depend on the observations with the localization radius **r**, but also on observations further away. This happens if these observation at larger distance are assimilated before the observations within the radius **r** and if they influence the state close to distance **r**. In this case the innovation for the state close to distance **r** is already change when assimilating the observation within the radius and hence lead to a different result. This effect has implications on the parallelization since keeping the observation order constant over the full model domain leads to a partial serialization of the algorithm. In the implementation in PDAF, we use the parallelization approach that does not guarantee the same order of the observations. Usually, the differences when changing the observation order are small, but the benefit is a better scaling since the serialization is avoided. Nonetheless, we generally recommend using LESTKF or LETKF, or their global variants ESTKF or ETKF, since they no not depend explicitly on the observation order, and they allow for non-diagonal observation error covariance matrices. However, the ENSRF/EAKF might have a good compute performance.
    5658
    5759== `PDAF_assimilate_ensrf` ==
    5860
    59 The general aspects of the filter specific routines `PDAF_assimilate_*` have been described on the page [ModifyModelforEnsembleIntegration Modification of the model code for the ensemble integration] and its sub-page on [InsertAnalysisStep inserting the analysis step]. The routine is used in the fully-parallel and the flexible implementation variant of the data assimilation system. When the offline model is used the routines `PDAF_put_state_*` are used. These have also been used in previous PDAF releases for the 'flexible' implementation variant. Here, we list once more the full interface of the routine. Subsequently, the full set of user-supplied routines specified in the call to `PDAF_assimilate_ensrf` is explained.
     61The general aspects of the filter specific routines `PDAF_assimilate_*` have been described on the page [ModifyModelforEnsembleIntegration Modification of the model code for the ensemble integration] and its sub-page on [InsertAnalysisStep inserting the analysis step]. The routine is used in the fully-parallel and the flexible implementation variants of the data assimilation system. When the offline model is used, the routines `PDAF_put_state_*` are used. These have also been used in previous PDAF releases for the 'flexible' implementation variant. Here, we list once more the full interface of the routine. Subsequently, the full set of user-supplied routines specified in the call to `PDAF_assimilate_ensrf` is explained.
    6062
    6163The interface when using the LEnKF is the following:
    … …  
    7476 * [#U_init_obsvars_finit_obsvars_pdaf.F90 U_init_obsvars_f]: [[BR]]The name of the user-supplied routine that initializes the vector of observation error variances.
    7577 * [#U_localize_covar_seriallocalize_covar_serial_pdaf.F90 U_localize_covar_serial]: [[BR]]The name of the routine that applies the covariance localization for a single observation
    76  * [#U_prepoststepprepoststep_ens_pdaf.F90 U_prepoststep]: [[BR]]The name of the pre/poststep routine as in `PDAF_get_state`
     78 * [#U_prepoststepprepoststep_ens_pdaf.F90 U_prepoststep]: [[BR]]The name of the user-supplied pre/poststep routine as in `PDAF_get_state`
    7779 * [#U_next_observationnext_observation_pdaf.F90 U_next_observation]: [[BR]]The name of a user supplied routine that initializes the variables `nsteps`, `timenow`, and `doexit`. The same routine is also used in `PDAF_get_state`.
    7880 * `status_pdaf`: [[BR]]The integer status flag. It is zero, if the routine is exited without errors.
    … …  
    8082== `PDAF_put_state_ensrf` ==
    8183
    82 For the offline mode of PDAF, the routine `PDAF_put_state_lenkf` has to be used instead of `PDAF_assimilate_lenkf`. This routine can also be used when the 'flexible' implementation variant is chosen for the assimilation system,  The general aspects of the filter specific routines `PDAF_put_state_*` have been described on the page [ModifyModelforEnsembleIntegration Modification of the model code for the ensemble integration]. The interface of the routine is identical with that of `PDAF_assimilate_lenkf` with the exception that the arguments of the user-supplied routines `U_distribute_state` and `U_next_observation` are missing.
     84For the offline mode of PDAF, the routine `PDAF_put_state_ensrf` has to be used instead of `PDAF_assimilate_ensrf`. This routine can also be used when the 'flexible' implementation variant is chosen for the assimilation system,  The general aspects of the filter specific routines `PDAF_put_state_*` have been described on the page [ModifyModelforEnsembleIntegration Modification of the model code for the ensemble integration]. The interface of the routine is identical with that of `PDAF_assimilate_ensrf` with the exception that the arguments of the user-supplied routines `U_distribute_state` and `U_next_observation` are missing.
    8385
    8486The interface is the following:
    … …  
    126128
    127129Some hints:
    128  * We recommend to not only determine the size of the observation vector at this point. The routine is a good place to also already gather information about the corresponding indices of the state vector needed later to implement the observation operator. In addition, one can already prepare an array that holds the full observation vector and an array storing the coordinates of the observations. The required arrays can be defined in a module like `mod_assimilation`. The information can be used later in `U_localize_covar_serial`.
     130 * We recommend to not only determine the size of the observation vector at this point. The routine is a good place to also already gather information about the corresponding indices of the state vector needed later to implement the observation operator. In addition, one can already prepare an array that holds the full observation vector, an array storing the coordinates of the observations and possible an array storing observation error variances (if the observation error covariance matrix is diagonal). The required arrays can be defined in a module like `mod_assimilation`. The information can be used later in `U_localize_covar_serial`.
     131    || **Note**: PDAF-OMI provides a structured approach for implementing the observation functionality. Because of this we generally recommend to ue the PDAF3 interface that uses PDAF-OMI. ||
    129132 * The routine is similar to `init_dim_obs` used in the global filters. However, if the global filter is used with a domain-decomposed model, it only initializes the size of the observation vector for the local model sub-domain. This is different for the local filters, as the local analysis also requires observational data from neighboring model sub-domains. Nonetheless, one can base on an implemented routine `init_dim_obs` to implement `init_dim_obs_f`.
    130133