Changes between Version 50 and Version 51 of InitPdaf


Ignore:
Timestamp:
Feb 22, 2023, 12:28:08 PM (16 months ago)
Author:
lnerger
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • InitPdaf

    v50 v51  
    1919== Overview ==
    2020
    21 After the adaption of the parallelization, the initialization of PDAF has to be implemented. It is performed by the routine `PDAF_init`. This subroutine has several arguments. Typically, the initializations of all variables required for the call to `PDAF_init` can be collected into a single subroutine. This implementation strategy is useful, as only a single additional subroutine call has to be inserted to the model source code. In the example in `testsuite/src/dummymodel_1D` the routine in the file `init_pdaf.F90` collects all initializations for PDAF. The routine has no arguments. Likewise, the tutorial implementations use a routine `init_pdaf.F90`. The file `init_pdaf.F90` in `templates/` provides a commented template for this routine, which can be used as the basis of the implementation.
     21After the adaption of the parallelization, the initialization of PDAF has to be implemented. It is performed by the routine `PDAF_init`. This subroutine has several arguments. To ensure a clean code, one can collect the initializations of all variables required for the call to `PDAF_init` a single subroutine. With this only a single additional subroutine call has to be inserted to the model source code for the initialization. In the example in `tutorial/online_2D_serialmodel` the routine in the file `init_pdaf.F90` collects all initializations for PDAF. The routine has no arguments. Likewise, the templates (in sub-directories of `templates/`) provide a commented template for this routine, which can be used as the basis of the implementation.
    2222
    23 `PDAF_init` itself calls a user-supplied routine to initialize the ensemble of model states through its interface. In the example there are separate routines for ensemble-based and mode-based filters. They are in the files `init_seik_pdaf.F90` (for SEIK, LSEIK, ETKF, LETKF), `init_enkf_pdaf.F90` (for EnKF), and `init_seek_pdaf.F90` (for SEEK).
     23`PDAF_init` itself calls a user-supplied call-back routine to initialize the ensemble of model states. In the example and templates, this routine can be found in `init_ens_pdaf.F90`.
    2424
    2525== Inserting `init_pdaf` ==
    2626
    27 The right place to insert a routine like `init_pdaf` to the model code is in between the initialization part of the model and its time stepping loop. I.e. the regular model initialization should be completed, which allows PDAF to initialize the ensemble of model states.
     27The right place to insert a routine like `init_pdaf` into the model code is in between the initialization part of the model and the time stepping loop. Thus, the regular model initialization should be completed, which allows PDAF to initialize the ensemble of model states.
    2828
    29 In the routine a number of variables have to be defined that are used in the call to `PDAF_init` as described in '[#RequiredargumentsforPDAF_init Required arguments for `PDAF_init`]' below. (Please note: All names of subroutines that start with `PDAF_` are core routines of PDAF, while subroutines whose name end with `_pdaf` are generally user-supplied interface routines) There are also a few variables that are initialized in `init_pdaf` but not used in the call to `PDAF_init`. These are variables that are specific for the data assimilation system, but only shared in between the user-supplied routines. For the example case, these variables are described in '[#Othervariablesfortheassimilation Other variables for the assimilation]'
     29In `init_pdaf` a number of variables have to be defined that are used in the call to `PDAF_init` as described below. (Please note: All names of subroutines that start with `PDAF_` are core routines of PDAF, while subroutines whose name end with `_pdaf` are generally user-supplied interface routines) There are also a few variables that are initialized in `init_pdaf` but not used in the call to `PDAF_init`. These are variables that are specific for the data assimilation system, but only shared in between the user-supplied routines. For the example case, these variables are described below in '[#Othervariablesfortheassimilation Other variables for the assimilation]'.
    3030
    31 The example and tutorial implementations, as well as the template routines allow to parse all variables through a command line parser by call in `init_pdaf_parse.F90`. This method provides a convenient way to define an experiment and could also be used for other models. The parser module is provided as `templates/parser_mpi.F90`. Alternatively, one could implement an initialization of the variables with Fortran namelists or could just rely on hard-coded numbers in the source code.
     31The tutorial implementations as well as the template routines allow to parse all variables through a command line parser by calls in `init_pdaf_parse.F90`. This method provides a convenient way to define an experiment and could also be used for other models. The parser module is provided as  `templates/online_omi/parser_mpi.F90` and also used in the tutorials. Alternatively, one could implement an initialization of the variables with Fortran namelists or could just rely on hard-coded numbers in the source code. Generally we recommend to use a configuration strategy consistent with the model code used.
    3232
    3333== Required arguments for `PDAF_init` ==
     
    5757  * 9: NETF
    5858  * 10: LNETF
    59   * [11: GENOBS (moved to 100 in PDAF V2.0)]
     59  * 11: LKNETF (new in PDAF V2.1)
    6060  * 12: PF
    61   * 100: GENOBS (generate synthetic observations - from PDAF V2.0)
    62   * 200: 3D-Var (introduced in PDAF V2.0)
    63  * `subtype`: An integer defining the sub-type of the filter algorithm (see the example code in `testsuite/src/dummymodel_1D` for choices). If `PDAF_init` is called with `subtype=-1` the available options are shown for the selected filter algorithm.
     61  * 100: GENOBS (generate synthetic observations, was filtertype=11 before PDAF V2.0)
     62  * 200: 3D-Var methods (new in PDAF V2.0)
     63 * `subtype`: An integer defining the sub-type of the filter algorithm (see the example code in `templates/online_omi/init_pdaf.F90` for choices).
     64  * If `PDAF_init` is called with `subtype=-1` the available options are shown for the selected filter algorithm.
    6465 * `step_null`: An integer defining the initial time step. For some cases it can use useful to set `step_null` larger to 0.
    6566 * `filter_param_i`: Integer array collecting several variables for PDAF. The first two variables are mandatory and equal for all filters. Further variables are optional (see example code or use `subtype=-1` to display available options.). The mandatory variables are in the following order:
    6667  * The size of the local state vector for the current process.
    67   * The ensemble size for all ensemble-based filters (or the rank of the state covariance matrix for mode-based filters like SEEK)
    68  * `length_filter_param_i`: An Integer defining the length of the array `filter_param_i`. The entries in the array are parsed up to this index.
    69  * `filter_param_r`: Array of reals collecting floating point variables for PDAF. The first variable is mandatory and equal for all filters.  Further variables are optional (see example code in `testsuite/src/dummymodel_1D` or use `subtype=-1` to display available options.). The mandatory variable is:
     68  * The ensemble size for all ensemble-based filters
     69 * `length_filter_param_i`: An integer defining the length of the array `filter_param_i`. The entries in the array are parsed up to this index.
     70 * `filter_param_r`: Array of reals collecting floating point variables for PDAF. The first variable is mandatory and equal for all filters.  Further variables are optional (see `templates/online_omi/init_pdaf.F90` or use `subtype=-1` to display available options.). The mandatory variable is:
    7071  * The value of the forgetting factor controlling covariance inflation (required to be larger than zero; common are values between 0.9 and 1.0. For 1.0 the ensemble is not inflated.)
    7172 * `length_filter_param_r`: An Integer defining the length of the array `filter_param_r`. The entries in the array are parsed up to this index.
     
    7374 * `COMM_filter`: The communicator variable `COMM_filter` as initialized by `init_parallel_pdaf`. (Usually stored in the module `mod_assimilation`)
    7475 * `COMM_couple`: The communicator variable `COMM_couple` as initialized by `init_parallel_pdaf`. (Usually stored in the module `mod_assimilation`)
    75  * `task_id`: The index of the model tasks as initialized by `init_parallel_pdaf`. (Usually stored in the module `mod_assimilation`)
     76 * `task_id`: The model task index as initialized by `init_parallel_pdaf`. (Usually stored in the module `mod_assimilation`)
    7677 * `n_modeltasks`: The number of model tasks as defined before the call to `init_parallel_pdaf`. (Usually stored in the module `mod_assimilation`)
    7778 * `filterpe`: A logical flag showing whether a process belongs to `COMM_filter` as initialized by `init_parallel_pdaf`. (Usually stored in the module `mod_assimilation`)
    78  * `U_init_ens`: The name of the user-supplied routine that is called by `PDAF_init` to initialize the ensemble of model states. (See '[#User-suppliedroutineU_init_ensinit_ens_pdaf.F90 User-supplied routine U_init_ens]'
     79 * `U_init_ens`: The name of the user-supplied routine that is called by `PDAF_init` to initialize the ensemble of model states. (See '[#User-suppliedroutineU_init_ensinit_ens_pdaf.F90 User-supplied routine U_init_ens]')
    7980 * `screen`: An integer defining whether information output is written to the screen (i.e. standard output). The following choices are available:
    8081  * 0: quiet mode - no information is displayed.
    8182  * 1: Display standard information (recommended)
    8283  * 2: as 1 plus display of timing information during the assimilation process
    83   * 3: Display detailed information for debugging
    8484 * `status_pdaf`: An integer used as status flag of PDAF. If `status_pdaf` is zero upon exit from `PDAF_init` the initialization was successful. An error occurred for non-zero values. (The error codes are documented in the routine `PDAF_init`.)
    8585
    8686An overview of available options for each filter an be found on the [wiki:AvailableOptionsforInitPDAF overview page on options].
    8787
    88 It is recommended that the value of `status_pdaf` is checked in the program after PDAF_init is executed. Only if its value is 0 the initialization was successful.
    89 
    90 PDAF also has a [PdafSimplifiedInterface Simplified Interface] providing the routine `PDAF_init_si`. In the simplified interface, the name of the user-supplied routine `U_init_ens` is predefined to `init_ens_pdaf` such that it does not appear in the call to `PDAF_init_si`.  More information on the pre-defined names is provided in the [PdafSimplifiedInterface documentation of PDAF's simplified interface].
     88'''Notes:'''
     89 * It is recommended that the value of `status_pdaf` is checked in the program after PDAF_init is executed. Only if its value is 0 the initialization was successful.
     90 * PDAF also has a [PdafSimplifiedInterface Simplified Interface] providing the routine `PDAF_init_si`. In the simplified interface, the name of the user-supplied routine `U_init_ens` is predefined to `init_ens_pdaf` such that it does not appear in the call to `PDAF_init_si`.  More information on the pre-defined names is provided in the [PdafSimplifiedInterface documentation of PDAF's simplified interface].
    9191
    9292== Other variables for the assimilation ==
     
    9898 * `sradius`: support radius, if observation errors are weighted (i.e. `locweight>0`)
    9999 * `locweight`: Type of localizing weight
    100 It is useful to define variables like these at this central position. Of course, their definition has to be adapted to the particular model used.
     100It is useful to define variables like these at this central position. Of course, their definition has to be adapted to the particular model used. The example codes describe the options for `locweight`.
    101101
    102102== User-supplied routine `U_init_ens` (init_ens_pdaf.F90) ==
    103103
    104 The user-supplied routine the we named `U_init_ens` here, is called by PDAF through the defined interface described below. The routine is called by all MPI processes that compute the filter analysis step (i.e. those for which 'filterpe' is set to true. In the standard configuration of `init_parallel_pdaf` these are all processes of the first model task, i.e. task_id=1.) `U_init_ens_pdaf` is only called by `PDAF_init` if no error occurred before; thus the status flag is zero.
     104The user-supplied routine the we named `U_init_ens` in the interface description above, is called by PDAF through the defined interface described below (Here, `U_` marks the variable to describe a user-supplied routine while in the actual code we use the name `init_ens_pdaf`). The routine is called by all MPI processes that compute the filter analysis step (i.e. those for which 'filterpe' is set to true. In the standard configuration of `init_parallel_pdaf` these are all processes of the first model task, i.e. task_id=1.) `init_ens_pdaf` is only called by `PDAF_init` if no error occurred before; thus the status flag is zero.
    105105
    106106The interface is the following:
     
    110110}}}
    111111with
    112  * `filtertype`: The integer defining the type of filter algorithm as given in the call to `PDAF_init`
    113  * `dim_p`: An integer holding the size of the state dimension for the calling process as specified in the call to `PDAF_init`
    114  * `dim_ens`: An integer holding the size of the ensemble (or the rank of the state covariance matrix for SEEK)
    115  * `state_p`: A real array of size (`dim_p`) for the local model state of the calling process (Only relevant for mode-based filters)
    116  * `Uinv`: A real array of size (`dim_ens-1`, `dim_ens-1`) for the inverse of matrix '''U''' from the decomposition of the state error covariance matrix '''P''' = '''VUV^T^''' (Only relevant for mode-based filters like SEEK.)
    117  * `ens_p`: A real array of size (`dim_p`, `dim_ens`) the has to hold upon exit the ensemble of model states.
    118  * `flag`: Status flag for PDAF. It is 0 upon entry and can be set by in the user-supplied routine, depending on the success of the ensemble initialization.  Preferably, values above 102 should be used for failures to avoid conflicts with the error codes defined within PDAF_init.
     112 * `filtertype`, `integer, intent(in)`:[[BR]]The type of filter algorithm as given in the call to `PDAF_init`
     113 * `dim_p`, `integer, intent(in)`:[[BR]] The size of the state dimension for the calling process as specified in the call to `PDAF_init`
     114 * `dim_ens`, `integer, intent(in)`:[[BR]]The size of the ensemble (or the rank of the state covariance matrix for SEEK)
     115 * `state_p`, `real, dimension(dim_p), intent(inout)`:[[BR]]Array for the local model state of the calling process (Only relevant for mode-based filters)
     116 * `Uinv`, `real, dimension(dim_ens-1, dim_ens-1), intent(inout)`:[[BR]]A real array for the inverse of matrix '''U''' from the decomposition of the state error covariance matrix '''P''' = '''VUV^T^''' (Only relevant for the SEEK filter.)
     117 * `ens_p`, `real, dimension(dim_p, dim_ens), intent(inout)`:[[BR]]The ensemble array. It has to hold upon exit the ensemble of model states.
     118 * `flag`, `integer, intent(inout)`:[[BR]]Status flag for PDAF. It is 0 upon entry and can be set by in the user-supplied routine, depending on the success of the ensemble initialization.  Preferably, values above 102 should be used for failures to avoid conflicts with the error codes defined within PDAF_init.
    119119
    120120=== Defining the state vector ===
     
    127127=== Initialization for ensemble-based filters ===
    128128
    129 In the initialization routine `U_init_ens_pdaf` one has to distinguish between ensemble-based and mode-based filters. The only mode based filter supplied with PDAF is SEEK, while all other methods are ensemble-based. Generally, the filters SEIK, LSEIK, EnKF, ETKF, LETKF, ESTKF, and LESTKF are ensemble-based filters. For these filters only the array `ens_p` needs to be initialized by the ensemble of model states. If a parallel model with domain decomposition is used, the full ensemble for the local sub-domain of the MPI process has to be initialized.
     129For the ensemble-based filters and the ensembel/hybrid 3D-Var methods only the array `ens_p` needs to be initialized by the ensemble of model states. If a parallel model with domain decomposition is used, the full ensemble for the local sub-domain has to be initialized.
    130130
    131131The arrays `state_p` and `Uinv` are allocated to their correct sizes because they are used during the assimilation cycles. They are not yet initialized and it is allowed to use these arrays in the initialization. An exception from this is EnKF for which `Uinv` is allocated only with size (1,1), because `Uinv` is not used for EnKF.
     
    134134=== Initialization for mode-based filters ===
    135135
    136 The only mode-based filter supplied with PDAF is currenly the SEEK filter. For this filter the initialization bases on the decomposition of the state error covariance matrix in the form '''P''' = '''VUV^T^'''. According to this decomposition, the array `ens_p` has to be initialized to hold the modes from matrix '''V''' and `Uinv` holds the inverse of matrix '''U'''. In addition `state_p` has to be initialized with the initial state estimate.  If a parallel model with domain decomposition is used, the part of all modes for the local sub-domain of the MPI process and the corresponding part of the state vector has to be initialized. ```Uinv``` will be identical for all MPI processes.
     136The only mode-based filter supplied with PDAF is currenly the SEEK filter. For this filter, the initialization bases on the decomposition of the state error covariance matrix in the form '''P''' = '''VUV^T^'''. According to this decomposition, the array `ens_p` has to be initialized to hold the modes from matrix '''V''' and `Uinv` holds the inverse of matrix '''U'''. In addition `state_p` has to be initialized with the initial state estimate.  If a parallel model with domain decomposition is used, the part of all modes for the local sub-domain of the MPI process and the corresponding part of the state vector has to be initialized. '''Uinv''' will be identical for all MPI processes.
    137137
    138138== Testing the PDAF initialization ==
     
    142142'''Remark:''' For the compilation with a real MPI library, one has to ensure that MPI-module (`USE MPI`) of the MPI-library is USED for both the model and PDAF. (Thus in the include file for make in `make.arch`, one might have set `MPI_INC` to the directory holding the module.)
    143143
    144 At this stage it will not be meaningful to perform an actual model time stepping. However, one can test whether the initialization in PDAF_init is successful and whether the ensemble array is correctly initialized.
     144At this stage it will not be meaningful to perform an actual model time stepping. However, one can test whether the initialization in PDAF_init is successful and whether the ensemble array is correctly initialized. For this one can also [wiki:PDAF_debugging activate the PDAF debugging output].
    145145
    146146Standard output from PDAF_init should look like the following: