Changes between Version 9 and Version 10 of AddFilterAlgorithm_PDAF3


Ignore:
Timestamp:
Jun 5, 2025, 6:34:51 PM (5 weeks ago)
Author:
lnerger
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • AddFilterAlgorithm_PDAF3

    v9 v10  
    2323The assimilation routines, here the universal routines `PDAF3_assim_offline` and `PDAF3_assimilate`, directly call the specific assimilation routine of the DA method. These are stored the module in `PDAF_assimilate_X.F90.
    2424
    25 [[Image(//pics/internal_interface_PDAF3_.png)]]
     25[[Image(//pics/internal_interface_PDAF3.png)]]
    2626[[BR]]'''Figure 1:''' Structure of the internal interface of PDAF. There are 7 internal interface routines (middle column) that connect the generic part with filter-specific routines. All these interface routines are collected in the module PDAF_utils filters. Each of the internal interface routines call one routine that is specific to the DA method. These routines are collected in the module PDAF_X, where 'X' would be the name of the DA method. The assimilation routines for the online and offline coupled modes are collected in the module PDAF_assimilate_X.
    2727
     
    6868At the time of an analysis step, the actual analysis routines are called. Here, `PDAFX_update` is the actual main routine for the DA method. In this routine, the observations are initialied and for domain-locallized filters, the local analysis loop is performed. The routine `PDAFX_analysis` then computes and applied the assimilation increment or computes the ensemble transformation of transform filters.
    6969
    70 [[Image(//pics/analysis_call_structure_PDAF3_.png)]]
     70[[Image(//pics/analysis_call_structure_PDAF3.png)]]
    7171[[BR]]'''Figure 2:''' Internal call structure for the analysis step. The universal interface routines `PDAF3_assimilate` and `PDAF3_Assim_offline` call a corresponding specific routine of the DA method. These specific routines  are `PDAF_assimilate_X` and `PDAF_assim_offline_X` which are members of the module `PDAFassimilate_X`. `PDAF_assimilate_X` calls `PDAF_put_state_X`. These two routines together control the online coupled mode, while `PDAF_assim_offline_X` controls the offline coupled mode. The actual analysis update is performed by the routines, `PDAFX_update` and `PDAFX_analysis`, each in their own module.
    7272
     
    229229Except for the status flag `outflag`, all arguments of `PDAF_assimilate_X` are the names of user-provided call-back routines.
    230230
    231 The routine is an infrastructure routine which is nearly identical for all DA methods. The template marks the lines which are generic and those which are specific to a DA method. Specific is the call to `PDAF_put_state_X`, both with regard to its name and to its arguments. Thus, when implementing a new DA method one has to adapt the argument list to those call-back routines that are used by the DA method.
     231The routine is an infrastructure routine which is nearly identical for all DA methods. The main functionality is to count time steps during the forecast time, perform possible operations during the forecast (for example apply incremental updates), and then call `PDAF_put_state_X` when the forecast of a state from the ensemble is complete. Afterwards, the routine `PDAF_get_state` is called to intialize the next forecast phase.
     232
     233The template marks the lines which are generic and those which are specific to a DA method. Specific is the call to `PDAF_put_state_X`, both with regard to its name and to its arguments. Thus, when implementing a new DA method one has to adapt the argument list to those call-back routines that are used by the DA method.
    232234
    233235Most of the specified call-back routines are used for all DA method. For example, the interface in the template for the global ensemble filter (`template/analysis_step/global/PDAF_assimilate_GLOBALTEMPLATE.F90`) is:
     
    237239       U_init_obsvar, U_next_observation, U_prepoststep, outflag)
    238240}}}
    239 Here, `U_` marks the user-provided call-back routines. This interface is identical to that in `PDAF_assimilate_etkf.F90`, `PDAF_assimilate_estkf.F90` and `PDAF_assimilate_seik.F90`. The only routines that are specific to the DA method are `U_prodRinvA` and `U_init_obsvar. Thus, only these should be changed (e.g. some  DA method do not used the product of **R**^-1^ with some matrix which is provided by `U_prodRinvA`, but compute an observation likelihood. Then one would rather use `U_likelihood`.)
     241Here, `U_` marks the user-provided call-back routines. This interface is identical to that in `PDAF_assimilate_etkf.F90`, `PDAF_assimilate_estkf.F90` and `PDAF_assimilate_seik.F90`.
     242
     243The only routines that are specific to the DA method are `U_prodRinvA` and `U_init_obsvar`. Thus, only these should be changed (e.g. some  DA method do not used the product of **R**^-1^ with some matrix which is provided by `U_prodRinvA`, but compute an observation likelihood. Then one would rather use `U_likelihood`.)
    240244
    241245The generic routines, which are always needed are:
    242 ||
    243 
    244 The domain-local filters (`template/analysis_step/global/PDAF_assimilate_LOCALTEMPLATE.F90`) have additional arguments to handle the localization of the state vectors and the localization of obsevations. However, they use the same generic routines
     246|| U_collect_state || Write model fields into a state vector ||
     247|| U_distribute_state || Write a state vector into model fields ||
     248|| U_init_dim_obs || Initialize observations, return observation dimension to PDAF. With PDAF-OMI, this is `init_dim_obs_pdafomi`. ||
     249|| U_obs_op || Observation operator, return observed model state to PDAF. With PDAF-OMI, this is `obs_op_dafomi` ||
     250|| U_init_obs || Return vector of observations to PDAF. When using PDAF-OMI, this routine is not visible to the user. ||
     251|| U_prodRinvA || Compute product of **R**^-1^ with some matrix **A**. Return **R**^-1^**A** to PDAF. When using PDAF-OMI, this routine is not visible to the user. ||
     252|| U_init_obsvar || Return mean observation error variance to PDAF. When using PDAF-OMI, this routine is not visible to the user.  ||
     253|| U_next_observation || Return number of time steps in next forecast phase, current model time and exit flag to PDAF ||
     254|| U_prepoststep ||  Pre/poststep routine ||
     255
     256The domain-local filters (`template/analysis_step/global/PDAF_assimilate_LOCALTEMPLATE.F90`) have additional arguments to handle the localization of the state vectors and the localization of obsevations. However, they use the same generic routines listed above.
     257
     258Some routines are hidden from the user when using the advanced interfaces like `PDAF3_assimilate` because they are provided by PDAF-OMI or PDAFlocal. However, the analysis routines are implemented without the assumption that PDAF-OMI or PDAFlocal are used.
     259
     260To get an overview of the available user-supplied call-back routines, you can looks into the file `src/PDAF_cb_procedures.F90`, which declared the interfaces of all call-back routines.
    245261
    246262
    247263=== `PDAF_assim_offline_X` ===
    248264
     265This routine is the counterpart of `PDAF_assimilate_X` for the offline-coupled implementation. It is called by `PDAF3_assim_offline` and other of the advanced 'assimilate'-routines.
     266
     267The routine is contained in the same file as `PDAF_assimilate_X` and is also an infrastructure routine. The routine is simpler, since no timestepping is done for the offline-coupled mode. As such the routine only writes out the configuration information and then calls the specific routine `PDAFX_update` for the analysis step.
     268
     269Except for the status flag `outflag`, all arguments of `PDAF_assim_offline_X` are the names of user-provided call-back routines. The user-supplied call-back routines specified as arguments are the same as for `PDAF_assimilate_X`, except that the routines `U_collect_state`, `U_distribute_state`, and `U_next_observation` are not present because these routines are only used for the online coupling.
     270
     271For implementing a new DA method, one mainly needs to adapt the call to PDAFX_update and the related specific call-back routines.
     272
    249273=== `PDAF_put_state_X` ===
    250274
     275This routine is the third infrastructure routine. This routine is usually called by `PDAF_assimilate_X`. Also `PDAF3_put_state` and other of the advanced 'assimilate'-routines call this routine.
     276
     277It provides the full interface in which all user-supplied routines are specified as arguments. For detailed information on using the routines with the full interface, please see the [wiki:ImplementationofAnalysisStep_noOMI Page on Implementing the Analysis Step using PDAF's full interface].
     278
     279The routine is an infrastructure routine which is nearly identical for all DA methods. Its functionality is to write the forecasted fields into a state vector from the ensemble array, and check for the completeness of the forecast phase (particularly relevant for the 'flexible parallelization' variant. When the analysis has to be computed, the routine gathers the ensemble on the filter processes and then calls `PDAFX_update` for the analysis step.
     280
     281Except for the status flag `outflag`, all arguments of `PDAF_put_state_X` are the names of user-provided call-back routines. The user-supplied call-back routines specified as arguments are the same as for `PDAF_assimilate_X`, except that the routines `U_distribute_state` and `U_next_observation` are not present because these routines are already used in `PDAF_assimilate_X` or in `PDAF_get_state` in implementations for PDAF2.
     282
     283For implementing a new DA method, one mainly needs to adapt the call to PDAFX_update and the related specific call-back routines.
     284
     285
    251286=== `PDAFX_update` ===
    252287
    253 The actual DA analysis update is computed in the routine `PDAF_X_update`. The routine is provided with the names of the call-back routines, the arrays as well of the relevant dimensions as describe above.
    254 
    255 The structure of the operations in `PDAF_X_update` can be designed freely when implementing a new DA method. The existing methods follow the recommended structure in which `PDAF_X_update` only performes preparations for the actual analysis update and then calls `PDAF_X_analysis` for the computation of the actual update. Here, the operations are different for global and local methods.
     288This is the main routine for the data assimilation update, i.e. the analysis step. The routine initializes the observations by calling `PDAFobs_init`. It also calls `U_prepoststep` before and after the actual analysis update.
     289
     290For the domain-local filters also the local analysis loop is performed in this routine. In this loop the local state vector and the local observations are intialized before the routine for the actual local analysis update is called.
     291
     292The arguments of this routine are the call-back routines which are needed for the analysis step, and the framework arrays from PDAF (i.e. `state_p`, `ens_p`, `Ainv`, for smoothers also `sens_p`) that are used in the analysis step.
     293
     294The structure of the operations in `PDAF_X_update` can be designed freely when implementing a new DA method. To keep the initializations clearly separate of the numical calculations we recommend the provided structure in which the actual analysis update is computed in `PDAFX_analysis`. Note, that it is possible to have multiple PDAFX_analysis routines with unique names. Then, one can call different analysis algorithms according to the `subtype`. This is, for example used for the EnSRF and EAKF, which are implemted as different subtypes in `src/PDAF_assimilate_ensrf.F90`
     295
     296The template files in `PDAF_GLOBALTEMPLATE_update.F90` and `PDAF_LOCALTEMPLATE_update.F90` explain the typical steps which should be usable for other DA methods. 
    256297
    257298=== `PDAFX_analysis` ===
    258299
    259 
    260 
    261 === Global Methods ===
    262 
    263 For global methods, e.g. ESTKF, ETKF, EnKF, NETF, the general structure of `PDAF_X_update` is:
    264 
    265 {{{
    266    IF (fixed ensemble, i.e. subtype==2 or subtype==3) THEN
    267       Add central state and ensemble perturbations
    268    ENDIF
    269 
    270    CALL U_prepoststep    # Perform pre/poststep before analysis update
    271 
    272    CALL PDAF_X_analysis  # Compute global analysis update of ensemble
    273 
    274    CALL PDAF_smoother    # Compute smoother update (Only for filters for which a smoother is implemented)
    275  
    276    CALL U_prepoststep    # Perform pre/poststep after analysis update
    277 }}}   
    278 You will find this structure e.g. in `PDAF_ESTKF_update.F90`. This structure is also valid for the LEnKF. The actual implementation, e.g. in `PDAF_ESTKF_update.F90` contains additional functionality e.g. calls for timers and normal screen output or debug outputs. Also there can be several different calls `PDAF_X_analysis` depending on the subtype of a filter.
    279 
    280 This general structure should be widely usable. Thus, when implementing a new global ensemble method one could copy an existing file and adapt it for the new method. The main functionality of the new DA method would then be implemented in `PDAF_X_analysis`.
    281 
    282 
    283 === Local Methods ===
    284 
    285 For domain-localized Methods, e.g. LESTKF, LETKF or LNETF the general structure of `PDAF_X_update` is different from the global methods because the local analysis loop is contained in `PDAF_X_update` and only a local analysis routine is called. The general structure is:
    286 
    287 {{{
    288    IF (fixed ensemble, i.e. subtype==2 or subtype==3) THEN
    289       Add central state and ensemble perturbations
    290    ENDIF
    291 
    292    CALL U_prepoststep          # Perform pre/poststep before analysis update
    293 
    294    CALL U_init_n_domains       # Determine number of local analysis domains (n_domains_p)
    295 
    296    # --- Perform global operations on observations and ensemble
    297    CALL U_init_dim_obs         # Determine number of observations (PDAF-OMI routine init_dim_obs_pdafomi)
    298    DO i = 1, dim_ens
    299       CALL U_obs_op            # Apply observation operator to all ensemble state (PDAF-OMI routine obs_op_pdafomi)
    300    ENDDO
    301    Compute mean forecast state (state_p)
    302    Compute mean observed forecast state (HXbar_f)
    303    # --- End of global operations
    304 
    305    # --- Perform local analysis loop
    306    DO i = 1, n_domains_p 
    307      CALL U_init_dim_l         # Set size of local state vector
    308      CALL U_init_dim_obs_l     # Set number of observations within radius (PDAF-OMI routine init_dim_obs_l_pdafomi)
    309      DO i = 1, dim_ens
    310         CALL U_g2l_state       # Get local state vector for all ensemble states
    311      ENDDO
    312 
    313      CALL PDAF_X_analysis      # Compute local analysis update of ensemble
    314 
    315      DO i = 1, dim_ens
    316         CALL U_l2g_state       # Update global state vector for all ensemble states
    317      ENDDO
    318      CALL U_l2g_state          # Update global state vector for ensemble mean state
    319 
    320      CALL PDAF_smoother_local  # Only for filters for which a smoother is implemented
    321    ENDDO
    322    # --- End of local analysis loop
    323  
    324    CALL U_prepoststep          # Perform pre/poststep after analysis update
    325 }}}   
    326 You will find this structure e.g. in `PDAF_LESTKF_update.F90`. This structure is not used for the LEnKF because this filter doe snot use a local analysis loop. The actual implementation, e.g. in `PDAF_LESTKF_update.F90` contains additional functionality e.g. calls for timers and normal screen output or debug outputs. Also there can be different calls `PDAF_X_analysis` depending on the subtype of a filter.
    327 
    328 This general structure should be widely usable for DA methods that use domain localization. Accordingly, when implementing a new local ensemble method one could copy an existing file and adapt it for the new method. The main functionality of the new DA method would then be implemented in `PDAF_X_analysis`.
     300This routine computes the actual analysis update. thus, the actual numerics are in this routine. For ensemble-based Kalman filters and nonlinear filters, this is mainly linear algebra. For these, we recommend to use BLAS and LAPACK library functions, because they usually lead to optimal compute performance.
     301
     302The provided templates contain some of the of common parts of the analysis step computation, which are stepwise explained. This is based on the computations in the ETKF and LETKF methods.
     303
     304Note that for the 3D-Var schemes, different solver algorithms are used. These are organized in further subroutines (see e.g. `src/PAF_3dvar_analysis_cvt.F90`).
     305
     306