Changes between Version 15 and Version 16 of AddFilterAlgorithm_PDAF3


Ignore:
Timestamp:
Apr 17, 2026, 2:53:03 PM (5 weeks ago)
Author:
lnerger
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • AddFilterAlgorithm_PDAF3

    v15 v16  
    6868
    6969The call structure of an analysis step is shown in Figure 2.
    70 The interface routines `PDAF3_assimilate` and `PDAF3_assim_offline` (or the different variants of 3D-Var or the more specific routines for ensemble filters) are called directly from the model code. These generic routines call internally the method-specific routine (`PDAF_assimilate_X` or `PDAF_assim_offline_X`) according to the chosen filter. These routines control the ensemble forecasting for the online coupled mode and the ensemble handling for the offline mode. In implementations done for PDAF2, the routine `PDAF_put_state_X` might be called. `PDAF_assimilate_X` calls `PDAF_put_state_X`, controls the ensemble for the case that multiple ensemble states are propagated by a single model task (i.e. the `flexible parallelization` variant, and collects the ensemble from the  ensemble tasks before the assimilate update and distributes the ensemble to the model tasks afterwards.
     70The interface routines `PDAF3_assimilate` and `PDAF3_assim_offline` (or the different variants of 3D-Var or the more specific routines for ensemble filters) are called directly from the model code. These generic routines call internally the method-specific routine (`PDAF_assimilate_X` or `PDAF_assim_offline_X`) according to the chosen filter. These routines control the ensemble forecasting for the online coupled mode and the ensemble handling for the offline mode. In implementations done for PDAF2, the routine `PDAF_put_state_X` might be called. `PDAF_assimilate_X` calls `PDAF_put_state_X`, controls the ensemble for the case that multiple ensemble states are propagated by a single model task (i.e. the ''flexible parallelization'' variant, and collects the ensemble from the  ensemble tasks before the assimilate update and distributes the ensemble to the model tasks afterwards.
    7171
    7272At the time of an analysis step, the actual analysis routines are called. Here, `PDAFX_update` is the actual main routine for the DA method. In this routine, the observations are initialied and for domain-locallized filters, the local analysis loop is performed. The routine `PDAFX_analysis` then computes and applied the assimilation increment or computes the ensemble transformation of transform filters.
    7373
    7474[[Image(//pics/analysis_call_structure_PDAF3.png)]]
    75 [[BR]]'''Figure 2:''' Internal call structure for the analysis step. The universal interface routines `PDAF3_assimilate` and `PDAF3_Assim_offline` call a corresponding specific routine of the DA method. These specific routines  are `PDAF_assimilate_X` and `PDAF_assim_offline_X` which are members of the module `PDAFassimilate_X`. `PDAF_assimilate_X` calls `PDAF_put_state_X`. These two routines together control the online coupled mode, while `PDAF_assim_offline_X` controls the offline coupled mode. The actual analysis update is performed by the routines, `PDAFX_update` and `PDAFX_analysis`, each in their own module.
     75[[BR]]'''Figure 2:''' Internal call structure for the analysis step. The universal interface routines `PDAF3_assimilate` and `PDAF3_assim_offline` call a corresponding specific routine of the DA method. These specific routines  are `PDAF_assimilate_X` and `PDAF_assim_offline_X` which are members of the module `PDAFassimilate_X`. `PDAF_assimilate_X` calls `PDAF_put_state_X`. These two routines together control the online coupled mode, while `PDAF_assim_offline_X` controls the offline coupled mode. The actual analysis update is performed by the routines, `PDAFX_update` and `PDAFX_analysis`, each in their own module.
    7676
    7777Further below we provide a detailed description for the different routines. The routines `PDAF_assimilate_X`, `PDAF_assim_offline_X`, and `PDAF_put_state_X` are framework routines and the templates only need minimal changes when implementing a new DA method. Also `PDAFX_update` as control routine for the analysis needs likely only minor changes. The main work when implementing a new DA method is in `PDAFX_analysis` where the actual analysis algorithm is implemented.
    … …  
    9393=== Internal arrays ===
    9494
    95 When running `PDAF_init`, a DA method allocates in its routine `PDAF_X_alloc` several arrays of the PDAF infrastructure. These arrays are declared in `PDAF_mod_core`. These arrays remain allocated throughout the assimilation process.
     95When running `PDAF3_init`, a DA method allocates in its routine `PDAF_X_alloc` several arrays of the PDAF infrastructure. These arrays are declared in `PDAF_mod_core`. These arrays remain allocated throughout the assimilation process.
    9696
    9797For the processes that computes the analysis (those with `filterpe=.true.`) the following arrays are defined:
    … …  
    9999|| `state`|| `dim_p`   || State vector. Used in all DA methods. Inside the filter code, it's usually called `state_p` to indicate parallelization. ||
    100100|| `ens` || `dim_p` x `dim_ens` || Ensemble array. Used in all DA methods. Inside some filters the name is `ens_p` to indicate parallelization. ||
    101 || `Ainv` || `dim_ens-1` x `dim_ens-1` (SEIK, ESTKF)[[BR]] `dim_ens` x `dim_ens` (ETKF) || transform matrix **U**^-1^ (in SEIK) or **A**^-1^ (in ETKF, ESTKF). Not used in EnKF. ||
     101|| `Ainv` || `dim_ens-1` x `dim_ens-1` (SEIK, ESTKF)[[BR]] `dim_ens` x `dim_ens` (ETKF) || transform matrix **U**^-1^ (in SEIK) or **A**^-1^ (in ETKF, ESTKF). Not used in EnKF, EnSRF or EAKF. ||
    102102|| `sens` || `dim_p` x `dim_ens` x `dim_lag` || Ensemble array for smoothing, storing the ensembles of previous analysis steps. Only used if DA method supports smoothing and `dim_lag>0`. ||
    103103|| `bias` || `dim_bias_p` || Bias vector. Only used if a DA method implements bias estimation. ||
    … …  
    109109
    110110PDAF provides the routine `PDAF_alloc` to perform the allocation of these arrays, thus `PDAF_X_alloc` calls `PDAF_alloc` providing dimension information for the allocation. For more information see [wiki:PDAF_alloc].
     111
     112
     113== Utilizing PDAF-OMI
     114
     115Next to the framework infrastructure for the state-related variables, one utilizes the call-back routines for observations. 
     116Generally, each DA methods is implemented in a way that it invokes call-back routines to retrieve information on observations. These subroutine calls are independent of PDAF-OMI and a new DA method can define its own, possibly new, call-back routines. However, it is recommended to leverage the existing functionality because it directly provides functionality for observations. If only existing call-back functions are used, a new DA method can then be readily used in existing codes.
     117
     118The initialization of observations is provided in side PDAF by the routines `PDAFobs_init` and `PDAFobs_init_local`, while observation variances are provided by `PDAFomi_init_obsvar`. These routines are included in the file `src/PDAFobs.F90`. Please refer to this file for the details on the subroutine arguments.
     119
     120In the PDAF-OMI observation modules, the user provides the routines
     121||= Routine =||= Description =||
     122|| init_dim_obs_OBSTYPE || Initialize observation information and return `dim_obs` to PDAF ||
     123|| obs_op_OBSTYPE || Apply observation operator to a state vector ||
     124|| init_dim_obs_l_OBSTYPE || Initialize a local observation vector for domain-localized DA methods ||
     125|| obs_op_lin_OBSTYPE || Apply the linear observation operator in 3D-Var methods  ||
     126|| obs_op_adj_OBSTYPE || Apply the adjoint observation operator in 3D-Var methods ||
     127See the [wiki:OMI_observation_modules_PDAF3 description of PDAF-OMI observation modules] for details on these routines.
     128
     129Internally, PDAF-OMI provides several additional routines. All these are included in the file `src/ PDAFomi_callback.F90`. These OMI-internal routines are used in the calls for the specific DA assimilation routines in, e.g. `src/PDAF3_assimilate_ens.F90`.
     130
     131The available OMI-internal routines are
     132||= Routine =||= Description =||
     133|| PDAFomi_init_obs_f_cb || Return the full vector of observations ||
     134|| PDAFomi_init_obsvar_cb || Return the mean observation error variance; used for diagonal R matrices ||
     135|| PDAFomi_init_obsvars_f_cb || return the full vector of observation error variances (diagonal of obs. error covariance matrix R); used for diagonal R matrices||
     136|| PDAFomi_g2l_obs_cb || provided with a full observe state vector it return the corresponding local observed state vector for a local analysis domain ||
     137|| PDAFomi_init_obs_l_cb || Provide the local vector of observations for a local analysis domain ||
     138|| PDAFomi_init_obsvar_l_cb || Provide the local mean observation error variance; used for diagonal R matricces ||
     139|| PDAFomi_prodRinvA_l_cb || Return the product of the input matrix A with the inverse matrix R : R^-1^ A for a single local analysis domain; used for diagonal R matrices ||
     140|| PDAFomi_likelihood_l_cb || Return the local likelihood; used for diagonal R matrices ||
     141|| PDAFomi_prodRinvA_hyb_l_cb || Return product of the input matrix A with the inverse matrix R : R^-1^ A for a single local analysis domain with hybridization (used in the LKNETF); used for diagonal R matrices ||
     142|| PDAFomi_likelihood_hyb_l_cb || Return the local likelihood with hybridization (used in the LKNETF); used for diagonal R matrices ||
     143|| PDAFomi_prodRinvA_cb || Return the product of the input matrix A with the inverse matrix R : R^-1^ A for a global analysis step; used for diagonal R matrices||
     144|| PDAFomi_likelihood_cb || Return the global likelihood; used for diagonal R matrices ||
     145|| PDAFomi_add_obs_error_cb || Add the diagonal of the error covariance matrix to some matrix; used for diagonal R matrices; used the EnKF/LEnKF ||
     146|| PDAFomi_init_obscovar_cb || Return the global observation error covariance matrix; used the EnKF/LEnKF ||
     147|| PDAFomi_init_obserr_f_cb || return the vector of observation error standard deviations (from diagonal of obs. error covariance matrix R); used for diagonal R matrices ||
     148|| PDAFomi_localize_covar_cb || Apply covariance localization to the global matrices HP and HPH that occur, e.g. in the LEnKF ||
     149|| PDAFomi_localize_covar_serial_cb || Apply covariance localization to the distributed vectors HP_p and HXY_p for a single observation which occur e.g. in the EnSRF and EAKF methods. ||
     150|| PDAFomi_omit_by_inno_l_cb || Omit local observations if their value depends too much from ensemble mean ||
     151|| PDAFomi_omit_by_inno_cb || Omit full observations if their value depends too much from ensemble mean ||
     152Please refer to the file `src/PDAFomi_callback.F90` for details on the arguments of these routines.
     153
     154For non-diagonal observation error covariance matrices, there are additional routines that are provided by the user. See the [wiki:nondiagonal_observation_error_covariance_matrices_PDAF3 PDAF on the PDAF3 interface for non-diagonal R matrices] for their description.
     155
     156Notes:
     157* Routines with ‘_f_’ in their name return the ‘full’ observations. For global filters these are the observations on a process-subdomain, while for local filters these are usually all observation or those on a process sub-domain plus a halo of the size of the localization radius around it if [wiki:OMI_observation_modules_PDAF3#thisobsuse_global_obs thisobs%use_global_obs=0] is used.
     158 * Routines with ‘_l_’ in their name are used for domain localized methods and return the local observations
     159
    111160
    112161== Configuration routines of DA method ==
    … …  
    320369
    321370If one introduced a new call-back routine for the DA method one should check if this can be integrated into PDAF-OMI (if it is related to observations) or into PDAFlocal (if related to state localization). If this is not the case, the DA method is most likely not compatible with the universal interface routines. In this case, one has to generate a new for PDAF3_assimilate and for PDAF3_assim_offline. An example for such a routine is `PDAF3_assimilate_lenkf` in `src/PDAF2_assimilate_ens.F90`, which contains the additional routine `localize_covar_pdaf` for covariance localization. However, the LEnKF is also an example that one might be able to avoid additional call-back routines. The LEnKF is also integrated in the universal PDAF3 interface routine `PDAF3_assimilate`, despite the additional routine `localize_covlar_pdaf`. For this we created a call-back routine within PDAF-OMI which is used in combination with the routine `PDAFomi_set_localize_covar`. This routine is called in the observation-initialization routine (`U_init_dim_obs`) of each PDAF-OMI observation module and provides PDA-OMI with the necessary information for perform the covariance localization.
     371