Changes between Initial Version and Version 1 of ImplementAnalysisENSRF_EAKF


Ignore:
Timestamp:
Mar 23, 2025, 7:57:45 PM (10 days ago)
Author:
lnerger
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • ImplementAnalysisENSRF_EAKF

    v1 v1  
     1= Implementation of the Analysis step for the ENSRF/EAKF (Ensemble Square Root/Adjustment Filter) =
     2
     3{{{
     4#!html
     5<div class="wiki-toc">
     6<h4>Implementation Guide</h4>
     7<ol><li><a href="ImplementationGuide">Main page</a></li>
     8<li><a href="AdaptParallelization">Adaptation of the parallelization</a></li>
     9<li><a href="InitPdaf">Initialization of PDAF</a></li>
     10<li><a href="ModifyModelforEnsembleIntegration">Modifications for ensemble integration</a></li>
     11<li><a href="ImplementationofAnalysisStep">Implementation of the analysis step</a></li>
     12<ol>
     13<li><a href="ImplementAnalysisestkf">Implementation for ESTKF</a></li>
     14<li><a href="ImplementAnalysislestkf">Implementation for LESTKF</a></li>
     15<li><a href="ImplementAnalysisetkf">Implementation for ETKF</a></li>
     16<li><a href="ImplementAnalysisletkf">Implementation for LETKF</a></li>
     17<li><a href="ImplementAnalysisseik">Implementation for SEIK</a></li>
     18<li><a href="ImplementAnalysislseik">Implementation for LSEIK</a></li>
     19<li><a href="ImplementAnalysisseek">Implementation for SEEK</a></li>
     20<li><a href="ImplementAnalysisenkf">Implementation for EnKF</a></li>
     21<li><a href="ImplementAnalysislenkf">Implementation for LEnKF</a></li>
     22<li>Implementation for ENSRF/EAKF</li>
     23<li><a href="ImplementAnalysisnetf">Implementation for NETF</a></li>
     24<li><a href="ImplementAnalysislnetf">Implementation for LNETF</a></li>
     25<li><a href="ImplementAnalysispf">Implementation for PF</a></li>
     26<li><a href="ImplementAnalysis_3DVar_classical">Implementation for 3D-Var</a></li>
     27<li><a href="ImplementAnalysis_3DEnVar_classical">Implementation for 3D Ensemble Var</a></li>
     28<li><a href="ImplementAnalysis_Hyb3DVar_classical">Implementation for Hybrid 3D-Var</a></li>
     29</ol>
     30<li><a href="AddingMemoryandTimingInformation">Memory and timing information</a></li>
     31<li><a href="EnsembleGeneration">Ensemble Generation</a></li>
     32<li><a href="DataAssimilationDiagnostics">Diagnostics</a></li>
     33</ol>
     34</div>
     35}}}
     36
     37[[PageOutline(2-3,Contents of this page)]]
     38
     39|| This page describes the implementation of the analysis step without using PDAF-OMI. Please see the [wiki:ImplementationofAnalysisStep page on the analysis with OMI] for the more modern and efficient implementation variant using PDAF-OMI. ||
     40
     41The LEnKF was added with verson 3.0 of PDAF.
     42
     43== Overview ==
     44
     45The ENSRF and EAKF are ensemble Kalman filter variants using serial observation processing. The implementation follows Houtekamer and Mitchell (2002) for the ENSRF and Anderson (2003) for the EAKF variant using local least squares regression.
     46
     47For the analysis step of the ENSRF and EAKF different operations related to the observations are needed. These operations are requested by PDAF by calling user-supplied routines. Intentionally, the operations are split into separate routines in order to keep the operations rather elementary. This procedure should simplify the implementation. The names of the required routines are specified in the call to the routine `PDAF_assimilate_ensrf` for the fully-parallel and flexible parallelization implementations (alternatively `PDAF_put_state_lenkf` for the 'flexible' implementation). With regard to the parallelization, all these routines are executed by the filter processes (`filterpe=.true.`) only.
     48
     49For completeness we discuss here all user-supplied routines that are specified in the interface to `PDAF_assimilate_enskf`. Thus, some of the user-supplied routines that are explained on the page explaining the modification of the model code for the ensemble integration are repeated here.
     50
     51In our study Nerger et al., 2015) we discussed that applying localization can lead to stability issues of the ENSRF. The filter performs a loop over all single observations and with localization the assimilation result depends on the order in which the observations are assimilated. This actually result in the effect that the assimilation result at some grid point does not only depend on the observations with the localization radius **r**, but also on observations further away if the influence the state close to the observations at distance **r** if those observations are assimilated before the observations within the radius **r**. This effect has implications on the parallelization since keeping the observation order constant does lead to a pertial serialization of the algorithm. In the implementation in PDAF we use the parallelization approach that does not guarantee the some order of the observations. Usually, the differences are small, but the benefit is a better scaling since the serialization is avoided. Nonetheless, we generally recommend using LESTKF or LETKF, or their global variants ESTKF or ETKF, since they no not depend explicitly on the observation order, and they allow for non-diagonal observation error covariance matrices. However, the ENSRF/EAKF might have a good compute performance.
     52
     53== `PDAF_assimilate_ensrf` ==
     54
     55The general aspects of the filter specific routines `PDAF_assimilate_*` have been described on the page [ModifyModelforEnsembleIntegration Modification of the model code for the ensemble integration] and its sub-page on [InsertAnalysisStep inserting the analysis step]. The routine is used in the fully-parallel and the flexible implementation variant of the data assimilation system. When the offline model is used the routines `PDAF_put_state_*' are used. These have also been used in previous PDAF releases for the 'flexible' implementation variant. Here, we list once more the full interface of the routine. Subsequently, the full set of user-supplied routines specified in the call to `PDAF_assimilate_ensrf` is explained.
     56
     57The interface when using the LEnKF is the following:
     58{{{
     59  SUBROUTINE PDAF_assimilate_lenkf(U_collect_state, U_distribute_state, U_init_dim_obs, &
     60                                 U_obs_op, U_init_obs, U_prepoststep, U_localize, &
     61                                 U_add_obs_err, U_init_obscovar, U_next_observation, status)
     62}}}
     63with the following arguments:
     64 * [#U_collect_statecollect_state_pdaf.F90 U_collect_state]: The name of the user-supplied routine that initializes a state vector from the array holding the ensemble of model states from the model fields. This is basically the inverse operation to `U_distribute_state` used in `PDAF_get_state` as well as here.
     65 * [#U_distribute_statedistribute_state_pdaf.F90 U_distribute_state]:  The name of a user supplied routine that initializes the model fields from the array holding the ensemble of model state vectors.
     66 * [#U_init_dim_obsinit_dim_obs_pdaf.F90 U_init_dim_obs]: The name of the user-supplied routine that provides the size of observation vector
     67 * [#U_obs_opobs_op_pdaf.F90 U_obs_op]: The name of the user-supplied routine that acts as the observation operator on some state vector
     68 * [#U_init_obsinit_obs_pdaf.F90 U_init_obs]: The name of the user-supplied routine that initializes the vector of observations
     69 * [#U_prepoststepprepoststep_ens_pdaf.F90 U_prepoststep]: The name of the pre/poststep routine as in `PDAF_get_state`
     70 * [#U_localizelocalize_covar_pdaf.F90 U_localize]: Apply covariance localization to the matrices HP and HPH^T^
     71 * [#U_add_obs_erradd_obs_err_pdaf.F90 U_add_obs_err]: The name of the user-supplied routine that adds the observation error covariance matrix to the ensemble covariance matrix projected onto the observation space.
     72 * [#U_init_obscovarinit_obscovar_pdaf.F90 U_init_obscovar]: The name of the user-supplied routine that initializes the observation error covariance matrix.
     73 * [#U_next_observationnext_observation_pdaf.F90 U_next_observation]: The name of a user supplied routine that initializes the variables `nsteps`, `timenow`, and `doexit`. The same routine is also used in `PDAF_get_state`.
     74 * `status`: The integer status flag. It is zero, if `PDAF_assimilate_lenkf` is exited without errors.
     75
     76== `PDAF_put_state_lenkf` ==
     77
     78When the 'flexible' implementation variant is chosen for the assimilation system, the routine `PDAF_put_state_lenkf` has to be used instead of `PDAF_assimilate_lenkf`. The general aspects of the filter specific routines `PDAF_put_state_*` have been described on the page [ModifyModelforEnsembleIntegration Modification of the model code for the ensemble integration]. The interface of the routine is identical with that of `PDAF_assimilate_lenkf` with the exception the specification of the user-supplied routines `U_distribute_state` and `U_next_observation` are missing.
     79
     80The interface when using the LEnKF is the following:
     81{{{
     82  SUBROUTINE PDAF_put_state_lenkf(U_collect_state, U_init_dim_obs, U_obs_op, &
     83                                 U_init_obs, U_prepoststep, U_localize, &
     84                                 U_add_obs_err, U_init_obscovar, status)
     85}}}
     86
     87
     88== User-supplied routines ==
     89
     90Here all user-supplied routines are described that are required in the call to `PDAF_assimilate_lenkf`. For some of the generic routines, we link to the page on [ModifyModelforEnsembleIntegration modifying the model code for the ensemble integration].
     91
     92To indicate user-supplied routines we use the prefix `U_`. In the tutorials in `tutorial/` and in the template directory `templates/` these routines exist without the prefix, but with the extension `_pdaf`. The files are named correspondingly. In the section titles below we provide the name of the template file in parentheses.
     93
     94In the subroutine interfaces some variables appear with the suffix `_p`. This suffix indicates that the variable is particular to a model sub-domain, if a domain decomposed model is used. Thus, the value(s) in the variable will be different for different model sub-domains.
     95
     96
     97=== `U_collect_state` (collect_state_pdaf.F90) ===
     98
     99This routine is independent of the filter algorithm used.
     100See the page on [InsertAnalysisStep#U_collect_statecollect_state_pdaf.F90 inserting the analysis step] for the description of this routine.
     101
     102
     103=== `U_distribute_state` (distribute_state_pdaf.F90) ===
     104
     105This routine is independent of the filter algorithm used.
     106See the page on [InsertAnalysisStep#U_distribute_statedistribute_state_pdaf.F90 inserting the analysis step] for the description of this routine.
     107
     108
     109=== `U_init_dim_obs` (init_dim_obs_pdaf.F90) ===
     110
     111This routine is used by all global filter algorithms (SEEK, SEIK, EnKF, ETKF) and by the LEnKF.
     112
     113The interface for this routine is:
     114{{{
     115SUBROUTINE init_dim_obs(step, dim_obs_p)
     116
     117  INTEGER, INTENT(in)  :: step       ! Current time step
     118  INTEGER, INTENT(out) :: dim_obs_p  ! Dimension of observation vector
     119}}}
     120
     121The routine is called at the beginning of each analysis step.  It has to initialize the size `dim_obs_p` of the observation vector according to the current time step. Without parallelization `dim_obs_p` will be the size for the full model domain. When a domain-decomposed model is used, `dim_obs_p` will be the size of the observation vector for the sub-domain of the calling process.
     122
     123Some hints:
     124 * It can be useful to not only determine the size of the observation vector at this point. One can also already gather information about the locations of the observations, which will be used later, e.g. to implement the observation operator. An array for the locations can be defined in a module like `mod_assimilation` of the example implementation.
     125
     126
     127=== `U_obs_op` (obs_op_pdaf.F90) ===
     128
     129This routine is used by all global filter algorithms (SEEK, SEIK, EnKF, ETKF) and the LEnKF.
     130
     131The interface for this routine is:
     132{{{
     133SUBROUTINE obs_op(step, dim_p, dim_obs_p, state_p, m_state_p)
     134
     135  INTEGER, INTENT(in) :: step               ! Currrent time step
     136  INTEGER, INTENT(in) :: dim_p              ! PE-local dimension of state
     137  INTEGER, INTENT(in) :: dim_obs_p          ! Dimension of observed state
     138  REAL, INTENT(in)    :: state_p(dim_p)     ! PE-local model state
     139  REAL, INTENT(out) :: m_state_p(dim_obs_p) ! PE-local observed state
     140}}}
     141
     142The routine is called during the analysis step. It has to perform the operation of the observation operator acting on a state vector that is provided as `state_p`. The observed state has to be returned in `m_state_p`.
     143
     144For a model using domain decomposition, the operation is on the PE-local sub-domain of the model and has to provide the observed sub-state for the PE-local domain.
     145
     146Hint:
     147 * If the observation operator involves a global operation, e.g. some global integration, while using domain-decomposition one has to gather the information from the other model domains using MPI communication.
     148
     149
     150=== `U_init_obs` (init_obs_pdaf.F90) ===
     151
     152This routine is used by all global filter algorithms (SEEK, SEIK, EnKF, ETKF) and the LEnKF.
     153
     154The interface for this routine is:
     155{{{
     156SUBROUTINE init_obs(step, dim_obs_p, observation_p)
     157
     158  INTEGER, INTENT(in) :: step             ! Current time step
     159  INTEGER, INTENT(in) :: dim_obs_p        ! PE-local dimension of obs. vector
     160  REAL, INTENT(out)   :: observation_p(dim_obs_p) ! PE-local observation vector
     161}}}
     162
     163The routine is called during the analysis step.
     164It has to provide the vector of observations in `observation_p` for the current time step.
     165
     166For a model using domain decomposition, the vector of observations that exist on the model sub-domain for the calling process has to be initialized.
     167
     168
     169=== `U_prepoststep` (prepoststep_ens_pdaf.F90) ===
     170
     171The general aspects of this routines have already been described on the [ModifyModelforEnsembleIntegration#U_prepoststepprepoststep_ens_pdaf.F90 page on modifying the model code for the ensemble integration] for the SEIK filter. For completeness, the description is repeated specifically for the EnKF:
     172
     173The interface of the routine is identical for all filters, but sizes can vary. Also, the particular operations that are performed in the routine can be specific for each filter algorithm.
     174
     175The interface for this routine is for the LEnKF
     176{{{
     177SUBROUTINE prepoststep(step, dim_p, dim_ens, dim_ens_p, dim_obs_p, &
     178                       state_p, Uinv, ens_p, flag)
     179
     180  INTEGER, INTENT(in) :: step        ! Current time step
     181                         ! (When the routine is called before the analysis -step is provided.)
     182  INTEGER, INTENT(in) :: dim_p       ! PE-local state dimension
     183  INTEGER, INTENT(in) :: dim_ens     ! Size of state ensemble
     184  INTEGER, INTENT(in) :: dim_ens_p   ! PE-local size of ensemble
     185  INTEGER, INTENT(in) :: dim_obs_p   ! PE-local dimension of observation vector
     186  REAL, INTENT(inout) :: state_p(dim_p) ! PE-local forecast/analysis state
     187                                     ! The array 'state_p' is not generally not initialized in the case of SEIK/EnKF/ETKF.
     188                                     ! It can be used freely in this routine.
     189  REAL, INTENT(inout) :: Uinv(1, 1)  ! Not used not LEnKF
     190  REAL, INTENT(inout) :: ens_p(dim_p, dim_ens)      ! PE-local state ensemble
     191  INTEGER, INTENT(in) :: flag        ! PDAF status flag
     192}}}
     193
     194The routine `U_prepoststep` is called once at the beginning of the assimilation process. In addition, it is called during the assimilation cycles before the analysis step and after the ensemble transformation. The routine is called by all filter processes (that is `filterpe=1`).
     195
     196The routine provides for the user the full access to the ensemble of model states. Thus, user-controlled pre- and post-step operations can be performed.  For example the forecast and the analysis states and ensemble covariance matrix can be analyzed, e.g. by computing the estimated variances. In addition, the estimates can be written to disk.
     197
     198Hint:
     199 * If a user considers to perform adjustments to the estimates (e.g. for balances), this routine is the right place for it.
     200 * Only for the SEEK filter the state vector (`state_p`) is initialized. For all other filters, the array is allocated, but it can be used freely during the execution of `U_prepoststep`.
     201 * The array `Uinv` is not used in the EnKF. Internally to PDAF, it is allocated to be of size (1,1).
     202 * Apart from the size of the array `Uinv`, the interface is identical for all ensemble filters (SEIK/ETKF/EnKF/LSEIK/LETKF/LEnKF). In general it should be possible to use an identical pre/poststep routine for all these filters.
     203 * The interface through which `U_prepoststep` is called does not include the array of smoothed ensembles. In order to access the smoother ensemble array one has to set a pointer to it using a call to the routine `PDAF_get_smootherens` (see page on [AuxiliaryRoutines auxiliary routines])
     204
     205
     206
     207=== `U_localize` (localize_covar_pdaf.F90) ===
     208
     209This routine is only used for the LEnKF.
     210
     211The interface for this routine is:
     212{{{
     213SUBROUTINE U_localize(dim_p, dim_obs, HP, HPH)
     214
     215  INTEGER, INTENT(in) :: dim_p                 ! PE-local state dimension
     216  INTEGER, INTENT(in) :: dim_obs               ! Dimension of global observation vector
     217  REAL, INTENT(inout) :: HP(dim_obs, dim_p)    ! Matrix HP
     218  REAL, INTENT(inout) :: HPH(dim_obs, dim_obs) ! Matrix HPH^T^
     219}}}
     220
     221The routine is called during the analysis step and has to apply the element-wise Schur product for the covariance localization of the two matrices'''HP''' and '''HPH^T^''', which are provided as input/output arguments.
     222
     223Notes:
     224 * In case of a parallelization with domain decomposition, `HP` contains only the columns of the matrix that resides on the model sub-domain of the calling process. The number of rows is that of the global number of observations
     225
     226Hints:
     227 * To compute the localization one can use the routine `PDAF_local_weight` after computing the distance between two elements in the matrix '''HP''' or '''HPH^T^'''.
     228
     229
     230=== `U_add_obs_err` (add_obs_err_pdaf.F90) ===
     231
     232This routine is only used for the EnKF and LEnKF.
     233
     234The interface for this routine is:
     235{{{
     236SUBROUTINE add_obs_err(step, dim_obs, C)
     237
     238  INTEGER, INTENT(in) :: step                ! Current time step
     239  INTEGER, INTENT(in) :: dim_obs             ! Dimension of obs. vector
     240  REAL, INTENT(inout) :: C(dim_obs, dim_obs) ! Matrix to that the observation
     241                                             !    error covariance matrix is added
     242}}}
     243
     244The routine is called during the analysis step. During the analysis step of the LEnKF, the projection of the ensemble covariance onto the observation space is computed. This matrix is provided to the routine as `C_p`. The routine has to add the observation error covariance matrix to `C_p`.
     245
     246The operation is for the global observation space. Thus, it is independent of whether the filter is executed with or without parallelization.
     247
     248Hints:
     249 * The routine does not require that the observation error covariance matrix is added as a full matrix. If the matrix is diagonal, only the diagonal elements have to be added.
     250
     251
     252
     253=== `U_init_obscovar` (init_obscovar_pdaf.F90) ===
     254
     255This routine is only used for the EnKF and LEnKF.
     256
     257The interface for this routine is:
     258{{{
     259SUBROUTINE init_obscovar(step, dim_obs, dim_obs_p, covar, m_state_p, &
     260     isdiag)
     261
     262  INTEGER, INTENT(in) :: step                ! Current time step
     263  INTEGER, INTENT(in) :: dim_obs             ! Dimension of observation vector
     264  INTEGER, INTENT(in) :: dim_obs_p           ! PE-local dimension of observation vector
     265  REAL, INTENT(out) :: covar(dim_obs, dim_obs) ! Observation error covariance matrix
     266  REAL, INTENT(in)  :: m_state_p(dim_obs_p)  ! PE-local observation vector
     267  LOGICAL, INTENT(out) :: isdiag             ! Whether the observation error covar. matrix is diagonal
     268}}}
     269
     270The routine is called during the analysis step and is required for the generation of an ensemble of observations. It has to initialize the global observation error covariance matrix `covar`. In addition, the flag `isdiag` has to be initialized to provide the information, whether the observation error covariance matrix is diagonal.
     271
     272The operation is for the global observation space. Thus, it is independent of whether the filter is executed with or without parallelization.
     273
     274Hints:
     275 * The local observation vector `m_state_p` is provided to the routine for the case that the observation errors are relative to the value of the observation.
     276
     277
     278=== `U_next_observation` (next_observation_pdaf.F90) ===
     279
     280This routine is independent of the filter algorithm used.
     281See the page on [InsertAnalysisStep#U_next_observationnext_observation_pdaf.F90 inserting the analysis step] for the description of this routine.
     282
     283
     284
     285== Execution order of user-supplied routines ==
     286
     287For the :EnKF, the user-supplied routines are essentially executed in the order they are listed in the interface to `PDAF_assimilate_lenkf`. The order can be important as some routines can perform preparatory work for later routines. For example, `U_init_dim_obs` can prepare an index array that provides the information for executing the observation operator in `U_obs_op`.
     288
     289Before the analysis step is called the following routine is executed:
     290 1. [#U_collect_statecollect_state_pdaf.F90 U_collect_state]
     291
     292The analysis step is executed when the ensemble integration of the forecast is completed. During the analysis step the following routines are executed in the given order:
     293 1. [#U_prepoststepprepoststep_ens_pdaf.F90 U_prepoststep] (Call to act on the forecast ensemble, called with negative value of the time step)
     294 1. [#U_init_dim_obsinit_dim_obs_pdaf.F90 U_init_dim_obs]
     295 1. [#U_obs_opobs_op_pdaf.F90 U_obs_op] (`dim_ens` calls: one call for each ensemble member)
     296 1. [#U_localizelocalize_covar_pdaf.F90 U_localize]
     297 1. [#U_add_obs_erradd_obs_err_pdaf.F90 U_add_obs_err]
     298 1. [#U_init_obsinit_obs_pdaf.F90 U_init_obs]
     299 1. [#U_init_obscovarinit_obscovar_pdaf.F90 U_init_obscovar]
     300 1. [#U_obs_opobs_op_pdaf.F90 U_obs_op] (`dim_ens` calls: one call for each ensemble member, repeated to reduce storage)
     301 1. [#U_prepoststepprepoststep_ens_pdaf.F90 U_prepoststep] (Call to act on the analysis ensemble, called with (positive) value of the time step)
     302
     303In case of the routine `PDAF_assimilate_enkf`, the following routines are executed after the analysis step:
     304 1. [#U_distribute_statedistribute_state_pdaf.F90 U_distribute_state]
     305 1. [#U_next_observationnext_observation_pdaf.F90 U_next_observation]