Changes between Version 4 and Version 5 of Implement3DVarAnalysisPDAF3_Hyb3DVar


Ignore:
Timestamp:
May 27, 2025, 2:03:07 PM (7 weeks ago)
Author:
lnerger
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Implement3DVarAnalysisPDAF3_Hyb3DVar

    v4 v5  
    3636This page describes the recommended implementation of the analysis step for the hybrid 3D-Var schemes using the PDAF3 interface of.
    3737
    38 || The interface for hybrid 3D-Var using the localized LESTKF for the transformation is the universal interface. If one intends to implement particularly for the variant using the global filter ESTKF, there is a separate interface for this special case. ||
    39 
    40 For the analysis step of 3D-Var we need different operations related to the observations. These operations are requested by PDAF by call-back routines supplied by the user and provided in the OMI structure. The names of the routines that are provided by the user are specified in the call to the assimilation routines. With regard to the parallelization, all these routines (except `collect_state_pdaf`, `distribute_state_pdaf`, and `next_observation_pdaf`) are executed by the filter processes (`filterpe=.true.`) only.
    41 
    42 The different 3D-Var methods in PDAF were explained on the [wiki:Implement3DVarAnalysisOverview page providing the verview of the Analysis Step for 3D-Var Methods]. Depending the type of 3D-Var, the background covariance matrix '''B''' is represented either in a parameterized form, by an ensemble, or by a combination of both. The 3D-Var methods that use an ensemble need to transform the ensemble perturbations using an ensemble Kalman filter. PDAF uses for this the error-subspace transform filter ESTKF. There are two variants: The first uses the localized filter LESTKF, while the second uses the global filter ESTKF.
    43 
    44 For completeness we discuss here all user-supplied routines that are specified in the interface to `PDAF3_assimilate_hyb3dvar_X`. Thus, some of the user-supplied routines that are explained on the page describing the modification of the model code for the ensemble integration are repeated here.
     38|| The interface for hybrid 3D-Var using the localized LESTKF for the transformation is the [wiki:Implement3DVarAnalysisPDAF3Universal universal interface]. If one intends to implement particularly for the variant using the global filter ESTKF, there is a separate interface for this special case. ||
     39
     40The different 3D-Var methods in PDAF were explained on the [wiki:Implement3DVarAnalysisOverviewPDAF3 page providing the verview of the Analysis Step for 3D-Var Methods]. Depending the type of 3D-Var, the background covariance matrix '''B''' is represented either in a parameterized form, by an ensemble, or by a combination of both. The 3D-Var methods that use an ensemble need to transform the ensemble perturbations using an ensemble Kalman filter. PDAF uses for this the error-subspace transform filter ESTKF. There are two variants: The first uses the localized filter LESTKF, while the second uses the global filter ESTKF.
     41
     42
     43For the [wiki:Implement3DVarAnalysisOverviewPDAF3 analysis step of the 3D-Var methods] we need different operations related to the observations. These operations are requested by PDAF by call-back routines supplied by the user and provided in the OMI structure. The names of the routines that are provided by the user are specified in the call to the assimilation routines.
     44
     45For completeness we discuss here all user-supplied routines that are specified as arguments. Thus, some of the user-supplied routines, which were explained on the page describing the modification of the model code for the ensemble integration, are repeated here.
    4546
    4647
    4748== Analysis Routines ==
    4849
    49 The general aspects of the filter (or solver) specific routines `PDAF3_assimilate_*` have been described on the page [OnlineModifyModelforEnsembleIntegration_PDAF3 Modification of the model code for the ensemble integration]. Here, we list the full interface of the routine. Subsequently, the user-supplied routines specified in the call is explained.
     50The general aspects of the filter (or solver) specific routines for the 3D-Var analysis step have been described on the page [wiki:OnlineModifyModelforEnsembleIntegration_PDAF3 Modification of the model code for the ensemble integration]. Here, we list the full interface of the routine. Subsequently, the user-supplied routines specified in the call is explained.
    5051
    5152There are two variants that either compute the transformataion of the ensemble transformation using the local LESTKF method, or the global ESTKF.
     53
    5254
    5355=== `PDAF3_assimilate_3dvar_all` ===
     
    8688=== `PDAF3_assim_offline_3dvar_all` ===
    8789
    88 For the offline mode of PDAF, the routine `PDAF3_assim_offline_3dvar_all` is used to perform the analysis step.
    89 The interface of the routine is identical with that of `PDAF3_assimilate_3dvar_all`, except that the user-supplied routines `U_distribute_state`, `U_collect_state` and `U_next_observation` are missing.
    90 
    91 The interface when using one of the global filters is the following:
    92 {{{
    93   SUBROUTINE PDAF3_assim_offline_3dvar_all(&
    94                                  U_init_dim_obs_pdafomi, U_obs_op_pdafomi, &
    95                                  U_cvt_ens, U_cvt_adj_ens, U_cvt, U_cvt_adj, &
    96                                  U_obs_op_lin_pdafomi, U_obs_op_adj_pdafomi, &
    97                                  U_init_n_domains_p, U_init_dim_l, U_init_dim_obs_l_pdafomi, &
    98                                  U_prepoststep, outflag)
    99 }}}
     90This routine is used to perform the analysis step for the offline mode of PDAF.
     91The interface of the routine is identical with that of `PDAF3_assimilate_3dvar_all`, except that the user-supplied routines `distribute_state_pdaf`, `collect_state` and `next_observation_pdaf` are missing.
     92
     93The interface is:
     94{{{
     95  SUBROUTINE PDAF3_assim_offline_3dvar_all( &
     96                                 init_dim_obs_pdafomi, obs_op_pdafomi, &
     97                                 cvt_ens_pdaf, cvt_adj_ens_pdaf, cvt_pdaf, cvt_adj_pdaf, &
     98                                 obs_op_lin_pdafomi, obs_op_adj_pdafomi, &
     99                                 init_n_domains_p_pdaf, init_dim_l_pdaf, init_dim_obs_l_pdafomi, &
     100                                 prepoststep_pdaf, outflag)
     101}}}
     102where all arguments, except the last one, are the names of call-back routines. See the description of the arguments for `PDAF3_assimilate_3dvar_all`.
     103
     104
    100105
    101106=== `PDAF3_put_state_3dvar_all` ===
    102107
    103 This routine exists for backward-compatibility. In implementations that were done before the release of PDAF V3.0, a 'put_state' routine was used for the `flexible` parallelization variant and for the offline mode.
    104 When the 'flexible' implementation variant is chosen for the assimilation system, the routine. The routine `PDAF3_put_state_3dvar_all` allows to port such implemnetations to the PDAF3 interface with minimal changes.
    105 The interface of the routine is identical with that of `PDAF3_assimilate_3dvar_all`, except that the user-supplied routines `U_distribute_state` and `U_next_observation` are missing.
    106 
    107 The interface when using one of the global filters is the following:
    108 {{{
    109   SUBROUTINE PDAF3_put_state_3dvar_all(U_collect_state, &
    110                                  U_init_dim_obs_pdafomi, U_obs_op_pdafomi, &
    111                                  U_cvt_ens, U_cvt_adj_ens, U_cvt, U_cvt_adj, &
    112                                  U_obs_op_lin_pdafomi, U_obs_op_adj_pdafomi, &
    113                                  U_init_n_domains_p, U_init_dim_l, U_init_dim_obs_l_pdafomi, &
    114                                  U_prepoststep, outflag)
    115 }}}
     108This routine exists for backward-compatibility. In implementations that were done before the release of PDAF V3.0, a 'put_state' routine was used for the ''flexible'' parallelization variant and for the offline mode.
     109When the ''flexible'' implementation variant is chosen for the assimilation system, the routine. This routine allows to port such implementations to the PDAF3 interface with minimal changes.
     110The interface of the routine is identical with that of `PDAF3_assimilate_3dvar_all`, except that the user-supplied routines `distribute_state_pdaf` and `next_observation_padf` are missing.
     111
     112The interface is:
     113{{{
     114  SUBROUTINE PDAF3_assimilate_3dvar_all(collect_state_pdaf, &
     115                                 init_dim_obs_pdafomi, obs_op_pdafomi, &
     116                                 cvt_ens_pdaf, cvt_adj_ens_pdaf, cvt_pdaf, cvt_adj_pdaf, &
     117                                 obs_op_lin_pdafomi, obs_op_adj_pdafomi, &
     118                                 init_n_domains_p_pdaf, init_dim_l_pdaf, init_dim_obs_l_pdafomi, &
     119                                 prepoststep_pdaf, outflag)
     120}}}
     121where all arguments, except the last one, are the names of call-back routines. See the description of the arguments for `PDAF3_assimilate_3dvar_all`.
    116122
    117123
     
    141147
    142148This routine is used to perform the analysis step for the offline mode.
    143 The interface of the routine is identical with that of `PDAF3_assimilate_hyb3dvar_estkf`, except that the user-supplied routines `U_distribute_state`, `U_collect_state` and `U_next_observation` are missing.
    144 
    145 The interface is:
    146 {{{
    147 SUBROUTINE PDAF3_assimilate_hyb3dvar_estkf(&
     149The interface of the routine is identical with that of `PDAF3_assimilate_hyb3dvar_estkf`, except that the user-supplied routines `distribute_state_pdaf`, `collect_state` and `next_observation_pdaf` are missing.
     150
     151The interface is:
     152{{{
     153SUBROUTINE PDAF3_assim_offline_hyb3dvar_estkf(&
    148154                                 init_dim_obs_pdafomi, obs_op_pdafomi, &
    149155                                 cvt_ens_pdaf, cvt_adj_ens_pdaf, cvt_pdaf, cvt_adj_pdaf, &
     
    151157                                 prepoststep_pdaf, outflag)
    152158}}}
     159where all arguments, except the last one, are the names of call-back routines. See the description of the arguments for `PDAF3_assimilate_hyb3dvar_estkf`.
     160
     161
     162
     163=== `PDAF3_put_state_hyb3dvar_estkf` ===
     164
     165This routine exists for backward-compatibility. In implementations that were done before the release of PDAF V3.0, a 'put_state' routine was used for the ''flexible'' parallelization variant and for the offline mode.
     166When the ''flexible'' implementation variant is chosen for the assimilation system, this routine allows to port such implementations to the PDAF3 interface with minimal changes.
     167The interface of the routine is identical with that of `PDAF3_assimilate_hyb3dvar_estkf`, except that the user-supplied routines `distribute_state_pdaf` and `next_observation_padf` are missing.
     168
     169This routine is particular for the ESTKF. One can use it if one exclusively uses the global filter. In the argument list of this routine, the call-back routine for localization are not present and hence the argument list is shorter than that of `PDAF3_put_state_3dvar_all`.
     170
     171The interface is:
     172{{{
     173SUBROUTINE PDAF3_put_state_hyb3dvar_estkf(collect_state_pdaf, &
     174                                 init_dim_obs_pdafomi, obs_op_pdafomi, &
     175                                 cvt_ens_pdaf, cvt_adj_ens_pdaf, cvt_pdaf, cvt_adj_pdaf, &
     176                                 obs_op_lin_pdafomi, obs_op_adj_pdafomi, &
     177                                 prepoststep_pdaf, outflag)
     178}}}
    153179where all arguments, except the last one, are the names of call-back routines. See the description of the arguments for `PDAF3_assimilate_3dvar_all`.
    154180
    155181
    156 
    157 === `PDAF3_put_state_hyb3dvar_estkf` ===
    158 
    159 This routine exists for backward-compatibility. In implementations that were done before the release of PDAF V3.0, a 'put_state' routine was used for the `flexible` parallelization variant and for the offline mode.
    160 When the 'flexible' implementation variant is chosen for the assimilation system, the routine. The routine `PDAF3_put_state_hyb3dvar_estkf` allows to port such implemnetations to the PDAF3 interface with minimal changes.
    161 The interface of the routine is identical with that of `PDAF3_assimilate_hyb3dvar_estkf`, except that the user-supplied routines `U_distribute_state` and `U_next_observation` are missing.
    162 
    163 This routine is particular for the ESTKF. One can use it if one exclusively uses the global filter. In the argument list of this routine, the call-back routine for localization are not present and hence the argument list is shorter than that of `PDAF3_put_state_3dvar_all`.
    164 
    165 The interface is:
    166 {{{
    167 SUBROUTINE PDAF3_assimilate_hyb3dvar_estkf(collect_state_pdaf, &
    168                                  init_dim_obs_pdafomi, obs_op_pdafomi, &
    169                                  cvt_ens_pdaf, cvt_adj_ens_pdaf, cvt_pdaf, cvt_adj_pdaf, &
    170                                  obs_op_lin_pdafomi, obs_op_adj_pdafomi, &
    171                                  prepoststep_pdaf, outflag)
    172 }}}
    173 where all arguments, except the last one, are the names of call-back routines. See the description of the arguments for `PDAF3_assimilate_3dvar_all`.
    174 
    175 
    176 
    177 
    178182== User-supplied routines ==
    179183
    180 Here all user-supplied routines are described that are required in the call to the assimilation routines for hybrid 3D-Var. For some of the generic routines, we link to the page on [wiki:OnlineModifyModelforEnsembleIntegration_PDAF3 modifying the model code for the ensemble integration].
    181 
    182 To indicate user-supplied routines we use the prefix `U_`. In the template directory `templates/` as well as in the tutorial implementations in `tutorial/` these routines exist without the prefix, but with the extension `_pdaf.F90`. The user-routines relating to OMI are collected in the file `callback_obs_pdafomi.F90`. In the section titles below we provide the name of the template file in parentheses.
    183 
    184 In the subroutine interfaces some variables appear with the suffix `_p`. This suffix indicates that the variable is particular to a model sub-domain, if a domain decomposed model is used. Thus, the value(s) in the variable will be different for different model sub-domains.
    185 
     184
     185Here, all user-supplied routines are described that are required in the call to the assimilation routines for hybrid 3D-Var. For some of the generic routines, we link to the page on [wiki:OnlineModifyModelforEnsembleIntegration_PDAF3 modifying the model code for the ensemble integration].
     186
     187The names of the user-suppled routines routines ending on `_pdaf` relate to operations on the model state, while those ensing on `_pdafomi` handle observations using the structured appraoch guided by [wiki:PDAF_OMI_Overview PDAF-OMI]. The user-routines relating to PDAF-OMI are collected in the file `callback_obs_pdafomi.F90`. In the section titles below we provide the name of the template file in parentheses.
     188
     189In the subroutine interfaces some variables appear with the suffix `_p`. This suffix indicates that the variable is particular to a model sub-domain, if a domain decomposed model is used. Thus, the value(s) in the variable will be different for different model sub-domains. Further, the suffix `_l` indices variables that are specific for a local analysis domain in the LESTKF.
    186190
    187191=== `collect_state_pdaf` (collect_state_pdaf.F90) ===
     
    189193This routine is independent of the filter algorithm used.
    190194
    191 See the page on [ModifyModelforEnsembleIntegration#collect_state_pdafcollect_state_pdaf.F90 modifying the model code for the ensemble integration] for the description of this routine.
     195See the page on [wiki:OnlineModifyModelforEnsembleIntegration_PDAF3#collect_state_pdafcollect_state_pdaf.F90 modifying the model code for the ensemble integration] for the description of this routine.
    192196
    193197=== `distribute_state_pdaf` (distribute_state_pdaf.F90) ===
     
    195199This routine is independent of the filter algorithm used.
    196200
    197 See the page on [ModifyModelforEnsembleIntegration#distribute_state_pdafdistribute_state_pdaf.F90 modifying the model code for the ensemble integration] for the description of this routine.
     201See the page on [wiki:OnlineModifyModelforEnsembleIntegration_PDAF3#distribute_state_pdafdistribute_state_pdaf.F90 modifying the model code for the ensemble integration] for the description of this routine.
    198202
    199203
     
    231235
    232236The routine is called during the analysis step during the iterative minimization of the cost function.
    233 It has to apply the control vector transformation to the control vector and return the transformed result vector. Usually this transformation is the multiplication with the square-root of the background error covariance matrix '''B'''. For the 3D Ensemble Var, this square root is usually expressed through the ensemble.
     237It has to apply the control vector transformation to the control vector and return the transformed result state vector. Usually this transformation is the multiplication with the square-root of the background error covariance matrix '''B'''. For the 3D Ensemble Var, this square root is usually expressed through the ensemble.  More complex transformation, including the combination with a parameterized covariance matrix, are possible and the routine permits the flexiblity to implement any transformation.
    234238
    235239If the control vector is decomposed in case of parallelization it first needs to the gathered on each processor and afterwards the transformation is computed on the potentially domain-decomposed state vector.
     
    252256
    253257The routine is called during the analysis step during the iterative minimization of the cost function.
    254 It has to apply the adjoint control vector transformation to a state vector and return the control vector. Usually this transformation is the multiplication with transpose of the square-root of the background error covariance matrix '''B'''. or the 3D Ensemble Var, this square root is usually expressed through the ensemble.
     258It has to apply the adjoint control vector transformation to a state vector and return the control vector. Usually this transformation is the multiplication with transpose of the square-root of the background error covariance matrix '''B'''. For the 3D Ensemble Var, this square root is usually expressed through the ensemble. More complex transformation, including the combination with a parameterized covariance matrix, are possible and the routine permits the flexiblity to implement any transformation.
    255259
    256260If the state vector is decomposed in case of parallelization one needs to take care that the application of the trasformation is complete. This usually requries a comminucation with MPI_Allreduce to obtain a global sun.
     261
    257262
    258263
     
    271276
    272277The routine is called during the analysis step during the iterative minimization of the cost function.
    273 It has to apply the control vector transformation to the control vector and return the transformed result vector. Usually this transformation is the multiplication with the square-root of the background error covariance matrix '''B'''.
     278It has to apply the control vector transformation to the control vector and return the transformed result state vector. Usually this transformation is the multiplication with the square-root of the background error covariance matrix '''B''' in its parameterized form.
    274279
    275280If the control vector is decomposed in case of parallelization it first needs to the gathered on each processor and afterwards the transformation is computed on the potentially domain-decomposed state vector.
     
    290295
    291296The routine is called during the analysis step during the iterative minimization of the cost function.
    292 It has to apply the adjoint control vector transformation to a state vector and return the control vector. Usually this transformation is the multiplication with transposed of the square-root of the background error covariance matrix '''B'''.
    293 
    294 If the state vector is decomposed in case of parallelization one needs to take care that the application of the trasformation is complete. This usually requries a comminucation with MPI_Allreduce to obtain a global sun.
    295 
     297It has to apply the adjoint control vector transformation to a state vector and return the control vector. Usually this transformation is the multiplication with transposed of the square-root of the background error covariance matrix '''B''' in its parameterized form.
     298
     299If the state vector is decomposed in case of parallelization one needs to take care that the application of the trasformation is complete. This usually requries a comminucation with MPI_Allreduce to obtain a global sum.
    296300
    297301
     
    309313
    310314See the [wiki:OMI_Callback_obs_pdafomi documentation on callback_obs_pdafomi.F90] for more information.
    311 
    312315
    313316
     
    373376
    374377
     378
    375379=== `init_dim_obs_l_pdafomi` (callback_obs_pdafomi.F90) ===
    376380
     
    384388The routine has already been described for modifying the model for the ensemble integration and for inserting the analysis step.
    385389
    386 See the page on [ModifyModelforEnsembleIntegration#distribute_state_pdafdistribute_state_pdaf.F90 modifying the model code for the ensemble integration] for the description of this routine.
     390See the page on [wiki:OnlineModifyModelforEnsembleIntegration_PDAF3#distribute_state_pdafdistribute_state_pdaf.F90 modifying the model code for the ensemble integration] for the description of this routine.
    387391
    388392
     
    391395This routine is independent of the filter algorithm used.
    392396
    393 See the page on [ModifyModelforEnsembleIntegration#distribute_state_pdafdistribute_state_pdaf.F90 modifying the model code for the ensemble integration] for the description of this routine.
     397See the page on [wiki:OnlineModifyModelforEnsembleIntegration_PDAF3#distribute_state_pdafdistribute_state_pdaf.F90 modifying the model code for the ensemble integration] for the description of this routine.
     398
     399
    394400
    395401== Execution order of user-supplied routines ==
     
    410416 1. [#obs_op_lin_pdafomicallback_obs_pdafomi.F90 obs_op_lin_pdafomi]
    411417 1. [#obs_op_adj_pdafomicallback_obs_pdafomi.F90 obs_op_adj_pdafomi]
     418 1. [#cvt_adj_ens_pdafcvt_adj_ens_pdaf.F90 cvt_adj_ens_pdaf]
    412419 1. [#cvt_adj_pdafcvt_adj_pdaf.F90 cvt_adj_pdaf]
    413  1. [#cvt_adj_ens_pdafcvt_adj_ens_pdaf.F90 cvt_adj_ens_pdaf]
    414420
    415421After the iterative optimization the following routines are executes to complte the analysis step:
     
    418424 1. [#prepoststep_pdafprepoststep_ens_pdaf.F90 prepoststep_pdaf] (Call to act on the analysis ensemble, called with (positive) value of the time step)
    419425
    420 The iterative optimization abovve computes an updated ensemble mean state. Subsequently, the ensemble perturbations are updated using the LESTKF or ESTKF. The execution of the routines for these filters is described on the [wiki:ImplementAnalysisPDAF3Universal page on implementing the local filter analysis step] .
     426The iterative optimization above computes an updated ensemble mean state. Subsequently, the ensemble perturbations are updated using the LESTKF or ESTKF. The execution of the routines for these filters is described on the [wiki:ImplementAnalysisPDAF3Universal page on implementing the local filter analysis step] .
    421427
    422428In case of the routine `PDAF3_assimilate_3dvar_all`, the following routines are executed after the analysis step: