Changes between Version 20 and Version 21 of OMI_observation_modules_PDAF3


Ignore:
Timestamp:
May 31, 2026, 10:43:40 AM (4 hours ago)
Author:
lnerger
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • OMI_observation_modules_PDAF3

    v20 v21  
    2828== Overview ==
    2929
    30 The implementation of the observations with OMI is done in observation modules (obs-modules). For each observation type a separate module should be created.
    31 
    32 Each obs-module contains three routines (where 'TYPE' will be replaced by the name of the observation):
    33 
    34  - **init_dim_obs_OBSTYPE** initializes all variables holding the information about one observation type. The information about the observation type is stored in a data structure (Fortran 'type' variale).
    35  - **obs_op_OBSTYPE** applies the observation operator to a state vector. One can call an observation operator routine provided by PDAF, or one can to implement a new operator.
    36  - **init_dim_obs_l_OBSTYPE** calls a PDAF-OMI routine to obtain the local observation information corresponding to a local analysis domain. i.e. observations within the localization radius. One can set localization parameters, like the localization radius, for each observation type.
    37 
    38 The template file `obs_OBSTYPE_pdafomi_TEMPLATE.F90` shows the different steps needed when implementing these routines. The main work is to implement `init_dim_obs_OBSTYPE`, while the other routines mainly call a subroutine provided by PDAF-OMI.
    39 
    40 In the obs-module, the subroutines are named according to the observation type. The template file uses generic names which should be replaced by the user. Having distinct names for each observation type is relevant to include the subroutine from the module in the call-back routine with ‘use’. In the header of each obs-module, the user can declare further variables, e.g. `rms_obs_OBSTYPE` as a uniform observation error value.
     30The implementation of the observations with OMI is done in observation modules (//obs-modules//). For each observation type a separate module should be created. This encapsulates each observation type.
     31
     32Each obs-module contains three routines (where 'OBSTYPE' will be replaced by the name of the observation):
     33
     34 - **init_dim_obs_OBSTYPE** initializes all variables holding the information about one observation type and provides these to PDAF-OMI.
     35 - **obs_op_OBSTYPE** applies the observation operator to a state vector. One can call an observation operator routine provided by PDAF-OMI, or one can to implement a new operator.
     36 - **init_dim_obs_l_OBSTYPE** is used in local filters. It calls a PDAF-OMI routine to obtain the local observations for a local analysis domain, i.e. observations within the localization radius.
     37
     38The template file `obs_OBSTYPE_pdafomi.F90` (or `obs_OBSTYPE_pdafomi_TEMPLATE.F90` in older versions of PDAF) shows the different steps needed when implementing these routines. The main work is to implement `init_dim_obs_OBSTYPE`, while the other routines mainly call a subroutine provided by PDAF-OMI.
     39
     40In each obs-module, the subroutines are named according to the observation type. The template file uses generic names which should be replaced by the user. Having distinct names for each observation type is relevant to include the subroutine from the module in the call-back routine with ‘use’. In the header of each obs-module, the user can declare further variables, to store observation-specific values, e.g. parameters whose values are specified at run time.
    4141
    4242
    4343== Data type `obs_f` ==
    4444
    45 To ensure the functionality within each obs-module, we rely on a Fortran data type called `obs_f` that contains all information about the observation. The data type is declared by PDAF-OMI and one instance of this data type is allocated in each obs-module with the generic variable name `thisobs`. A few of the elements of `obs_f` are initialized by the user when the observation information is initialized on `init_dim_obs_OBSTYPE`. Further variables is set in a call to the routine `PDAFomi_gather_obs`. This information is then used by all other routines in the obs-module. The template file `obs_OBSTYPE_pdafomi_TEMPLATE.F90` shows the different steps needed to initialize `thisobs`.
    46 
    47 The '''mandatory variables''' in `obs_f` that need to be set by the user are:
    48 {{{
    49   TYPE obs_f
    50      ! ---- Mandatory variables to be set in INIT_DIM_OBS ----
    51      INTEGER :: doassim=0                 ! Whether to assimilate this observation type
    52      INTEGER :: disttype                  ! Type of distance computation to use for localization
     45To ensure the functionality within each obs-module, PDAF-OMI stores all information about one observation type in a variable called `thisobs`. This variable is, in Fortran, a 'type' variable. Thus, it combines the storage of value sof different types. PDAF-OMI uses for this a Fortran data type called `obs_f`. The data type is declared by PDAF-OMI and one instance of this data type is allocated in each obs-module with the generic variable name `thisobs`. Each variable within `thisobs` is accessed using '%', e.g. `thisobs%ncoord` for the number of coordinates.
     46
     47A few of the elements of `obs_f` are set directly by the user when the observation information is initialized on `init_dim_obs_OBSTYPE`. Further variables are set in a call to the routine `PDAFomi_gather_obs`, which registers the observation with PDAF-OMI. This information is then used by all other routines in the obs-module. The template file `obs_OBSTYPE_pdafomi.F90` explains the different steps needed to initialize `thisobs`.
     48
     49The '''mandatory variables''' of `thisobs` that need to be set by the user are:
     50{{{
     51  INTEGER :: thisobs%doassim              ! Whether to assimilate this observation type; default=0
     52  INTEGER :: thisobs%disttype             ! Type of distance computation to use for localization
    5353                                          !   (0) Cartesian, (1) Cartesian periodic
    5454                                          !   (2) simplified geographic, (3) geographic haversine function
    5555                                          !   (10) Cartesian 2+1D factorized, (11) Cartesian periodic 2+1D factorized
    5656                                          !   (12) simplified geographic 2+1D factorized
    57                                           ! (13) geographic haversine function  2+1D factorized
    58      INTEGER :: ncoord                    ! Number of coordinates use for distnce computation
    59      INTEGER, ALLOCATABLE :: id_obs_p(:,:) ! Indices of process-local observed field in state vector
    60      ...
    61   END TYPE obs_f
     57                                          !   (13) geographic haversine function  2+1D factorized
     58  INTEGER :: thisobs%ncoord               ! Number of coordinates use for distnce computation
     59  INTEGER, ALLOCATABLE :: thiobs%id_obs_p(:,:)   ! Indices of process-local observed field in state vector
    6260}}}
    6361
    6462In addition there are '''optional variables''' that the be used:
    6563{{{
    66   TYPE obs_f
    67      ...
    68      ! ---- Optional variables - they can be set in INIT_DIM_OBS ----
    69      REAL, ALLOCATABLE :: icoeff_p(:,:)   ! Interpolation coefficients for obs. operator (optional)
    70      REAL, ALLOCATABLE :: domainsize(:)   ! Size of domain for periodicity (<=0 for no periodicity) (optional)
    71 
    72      ! ---- Variables with predefined values - they can be changed in INIT_DIM_OBS  ----
    73      INTEGER :: obs_err_type=0            ! Type of observation error: (0) Gauss, (1) Laplace
    74      INTEGER :: use_global_obs=1          ! Whether to use (1) global full obs.
    75                                           ! or (0) obs. restricted to those relevant for a process domain
    76      CHARACTER(len=20) :: name=""         ! Name of observation type
    77      ...
    78   END TYPE obs_f
     64  ! ---- Optional variables - they can be set in INIT_DIM_OBS ----
     65  REAL, ALLOCATABLE :: thisobs%icoeff_p(:,:)   ! Interpolation coefficients for obs. operator (optional)
     66  REAL, ALLOCATABLE :: thisobs%domainsize(:)   ! Size of domain for periodicity (<=0 for no periodicity) (optional)
     67
     68  ! ---- Variables with predefined values - they can be changed in INIT_DIM_OBS  ----
     69  INTEGER :: thisobs%obs_err_type=0            ! Type of observation error: (0) Gauss, (1) Laplace
     70  INTEGER :: thisobs%use_global_obs=1          ! Whether to use (1) global full obs.
     71                                               ! or (0) obs. restricted to those relevant for a process domain
     72  CHARACTER(len=20) :: thisobs%name            ! Name of observation type
    7973}}}
    8074
     
    8276
    8377
    84 Next to the data type `obs_f`, there is a data type `obs_l` for localization. This is only used internally. It will be filled in the routine `PDAFomi_init_dim_obs_l` which is called by `init_dim_obs_l_OBSTYPE`.
     78Next to `thisobs`, there is the variable `thisobs_l` for localization. This is declared in each observation operator and will be filled in the routine `PDAFomi_init_dim_obs_l` which is called by `init_dim_obs_l_OBSTYPE`.
    8579
    8680== Required routines ==
     
    114108 * [wiki:OMI_observation_modules_PDAF3#ivar_obs_p ivar_obs_p]: store the inverse error variance of each observation for the default mode of OMI, which assumes a diagonal observation error covariance matrix
    115109
    116 Further possible initializations are:
     110Further optional initializations are:
    117111 * [wiki:OMI_observation_modules_PDAF3#thisobsicoeff_p thisobs%icoeff_p]: When one uses PDAF-OMI's observation operator with interpolation, one further needs to initialize this array of interpolation coefficients.
    118112 * [wiki:OMI_observation_modules#thisobsdomainsize thisobs%domainsize]: This needs to be set for Cartesian distance computation with periodicity.
     
    125119** 2. Call PDAFomi_gather_obs**
    126120
    127 After these variables are filled, one calls the routine `PDAFomi_gather_obs` to initialize an observation type for PDAF-OMI. The routine also performs the necessary operations to ensure that the parallelization is taken into account by PDAF.
    128 
    129 The routine is called as:
    130 {{{
    131     CALL PDAFomi_gather_obs(thisobs, dim_obs_p, obs_p, ivar_obs_p, ocoord_p, &
    132          thisobs%ncoord, lradius, dim_obs)
    133 }}}
    134 This routine will complete all required initializations for OMI. As such it is mandatory to call the routine.
     121After these variables are filled, one calls the routine `PDAFomi_gather_obs` to initialize an observation type for PDAF-OMI. The routine also performs the necessary operations to ensure that a parallelization is taken into account by PDAF.
     122It is mandatory to call the routine.
     123
     124The call to the routine is implemented in the template using the variable names used above (see also [wiki:PDAFomi_gather_obs]).
     125
    135126
    136127The routine `PDAFomi_gather_obs` returns the number of observations `dim_obs` which is the return variable for PDAF. 
    137128
    138 Notes:
    139  * Here, `lradius` is the maximum localization cut-off radius around the process sub-domain. The value is `lradius` is only used if [wiki:OMI_observation_modules_PDAF3#thisobsuse_global_obs thisobs%use_global_obs=0]. 
    140   * `lradius` is always a single value. It should be set of the largest radius of the directions in which the process domain is split by parallelization. It defines the radius within which observations from neighboring process sub-domains are taken into account.
    141129
    142130** 3. Optionally initialize covariance localization **
     
    164152=== `init_dim_obs_l_OBSTYPE` ===
    165153
    166 This routine initializes local observation information. The routine is only used by the domain-localized filters (LESTKF, LETKF, LSEIK, LNETF, LKNETF) and by ensemble 3D-Var using the LESTKF.
     154This routine initializes local observation information. The routine is only used by the domain-localized filters (LESTKF, LETKF, LSEIK, LNETF, LKNETF) and by ensemble 3D-Var and hybrid 3D-Var using the LESTKF.
    167155
    168156For the initialization the following routine is called:
     
    172160}}}
    173161
    174 Here, `thisobs` and `thisobs_l` are the data-type variables `obs_f` and `obs_l`. `dim_obs_l`, the local size of the observation vector, is the direct output of the routine.
     162Here, `thisobs` and `thisobs_l` are observation-type variables explained before. `dim_obs_l`, the local size of the observation vector, is the direct output of the routine. For details see [wiki:PDAFomi_init_dim_obs_l]
    175163
    176164''Implementation steps:''
    177  - Ensure that `coords_l` is filled and that the unit of `coords_l` is the same as that used for the observation coordinates. Usually `coords_l` is filled in `init_dim_l_pdaf` as shown in the tempaltes and tutorial.
    178  - Specify the localization variables (These variables are usually set in `init_pdaf` and included with `use mod_assimilation`)
    179    - ''locweight'': Type of localization (see table below)
    180    - ''cradius'': The localization cut-off radius or directional radii (cut-off radius for the observations, weight is always =0 for distances > cradius)
    181    - ''sradius'': The support radius (or directional radii) of the localization weight function. Usually, we set `sradius=cradius`. Using different radii is an option to tune the localization.
     165 - Ensure that `coords_l` is filled and that the unit of `coords_l` is the same as that used for the observation coordinates. Usually `coords_l` is filled in `init_dim_l_pdaf` as shown in the templates and tutorial.
     166 - Specify the localization variables (In the template and tuturial codes, they are set in `init_pdaf` and included with `use mod_assimilation`)
     167   - `locweight`: Type of localization (see table below)
     168   - `cradius`: The localization cut-off radius (cut-off radius for the observations, weight is always =0 for distances > cradius)
     169   - `sradius`: The support radius of the localization weight function. Usually, we set `sradius=cradius`. Using different radii is an option to tune the localization.
    182170
    183171Note, that for PDAF V2.2.1 and later these three variables can be either scalar values (for isotropic localization), or arrays (for non-isotropic localization). In the latter case one can additionally choose separate weight functions for the horizontal and vertical directions (see the notes below for more information)
    184172
    185 The setting of `locweight` influences the weight function for the localization. The choices are standardized as follows
     173The setting of `locweight` defines the weight function for the localization. The choices are standardized as follows
    186174
    187175||= '''locweight''' =||= '''0''' =||= '''1''' =||= '''2''' =||= '''3''' =||= '''4''' =||
     
    191179||= '''sradius''' =||  no impact  ||  weight = exp(-d / sradius)  ||||||||  weight = 0 if d >= sradius[[BR]] else[[BR]] weight = f(sradius, distance)  ||
    192180
    193 Here, 'regulation' refers to the regulated localization introduced in Nerger, L., et al. (2012). A regulated localization scheme for ensemble-based Kalman filters. Quarterly Journal of the Royal Meteorological Society, 138, 802-812. ​[https://doi.org/10.1002/qj.945 doi:10.1002/qj.945].
     181Here, ;d; is the distance between the local analysis domain and the observation. 'regulation' refers to the regulated localization introduced in Nerger, L., et al. (2012). A regulated localization scheme for ensemble-based Kalman filters. Quarterly Journal of the Royal Meteorological Society, 138, 802-812. ​[https://doi.org/10.1002/qj.945 doi:10.1002/qj.945].
    194182
    195183**Notes:**
    196 - **isotropic localization**: If `cradius` and `sradius` are scalar values, the localization is isotropic. Thus, it uses the same `cradius` in all directions. If different localization scales should be applied e.g. in the vertical compared to the horizonal one needs to scale the vertical coordinates.
    197 - **non-isotropic localization**: Nonisotropic localization was introduced with PDAF V2.2: `cradius` and `sradius` can be declared as vectors of length `thisobs%ncoords` and each element can get a different value. In this case, the values define a non-isotropic localization according to the values specified in `cradius` and `sradius`. PDAF-OMI will use these values to compute a directional localization radius.
    198 - **2D+1D factorized non-isotropic localization**: With PDAF V2.2.1 a factorized 2D+1D localization can be specified (see [wiki:OMI_observation_modules#thisobsdisttype explanation of disttype]. If the non-isotropic localization is used one can specify different weight functions for the vertical and horizontal directions. This is achieved by declaring `loweight` as a vector of size 2. Now the first element specifies the weight function (according to the table above) for the horizontal direction and the second element specified the wieght function for the vertical direction. When 'locweight' is used as a scalar variable, it specified the weight function in the horizontal direction while the weight function in the vertical dircetion is a constant value of one.
    199184- A common choice is to use `locweight=2` or `locweight=4` in combination with `cradius=sradius`. Choosing `sradius>cradius` is possible, but `sradius<cradius` should be avoided (one would set the weights of distant observation to zero, but would still assimilate them).
    200185- **optimizing the search operation**: With PDAF V3.1 and later one can optimize the performance of the search for local observations in `PDAFomi_init_dim_obs_l` by using the routine [wiki:PDAFomi_set_searchtype]. The speed of the search operation depends on different factors and this routine allows to switch the type of the search operation which can improve the speed.
     186- **isotropic localization**: If `cradius` and `sradius` are scalar values, the localization is isotropic. Thus, it uses the same `cradius` in all directions. If different localization scales should be applied e.g. in the vertical compared to the horizonal one would need to scale the vertical coordinates.
     187- **non-isotropic localization**: `cradius` and `sradius` can be declared as vectors of length `thisobs%ncoords` and each element can get a different value. The values define a non-isotropic localization with different radii in each direction. PDAF-OMI will use these values to compute a directional localization radius. (Nonisotropic localization is supported for PDAF V2.2 and later)
     188- **2D+1D factorized non-isotropic localization**:  This localization computes the localization weight as a product of separate weights for the horizontal and vertical directions ([wiki:OMI_observation_modules#thisobsdisttype see explanation of disttype]). In this case, one can specify different weight functions for the vertical and horizontal directions. When `locweight` is declared as a scalar variable, it specifies the weight function in the horizontal direction while the weight function in the vertical dircetion is a constant value of one. Alternatively, one can declare `loweight` as a vector of size 2. Now, the first element specifies the weight function (according to the table above) for the horizontal direction and the second element specifies the weight function for the vertical direction.  (factorized 2D+1D localization is supported for PDAF V2.2.1 and later)
    201189
    202190== Routine for backward compatibility ==
     
    204192=== `localize_covar_OBSTYPE` ===
    205193
    206 || This routine exists for backward compatibility for implementations started before PDAF V3.0. In PDAF V3.0, we added the routine [wiki:OMI_observation_modules_PDAF3#Initializingcovariancelocalization PDAFomi_set_localize_covar], which is described below. When `PDAFomi_set_localize_covar` is used, the user-provided routine `localize_covar_OBSTYPE` is no longer required. ||
     194|| This routine exists for backward compatibility for implementations started before PDAF V3.0. In PDAF V3.0, we added the routine [wiki:OMI_observation_modules_PDAF3#Initializingcovariancelocalization PDAFomi_set_localize_covar], which is described below. When `PDAFomi_set_localize_covar` is used, the user-provided routine `localize_covar_OBSTYPE` is no longer needed. ||
    207195
    208196This routine initializes local observation information. The routine is only used by the local EnKF (LEnKF).
     
    264252== Initializing covariance localization ==
    265253
     254Covariance localization is used in the LEnKF, EnSRF and EAKF methods.
    266255The covariance localization should be initialized in `init_dim_obs_OBSTYPE`. This is achieved by calling `PDAFomi_set_localize_covar`. The routine can only be called after the call to `PDAFomi_gather_obs`.
    267256
    268 The interface is for isotropic localization:
    269 {{{
    270   SUBROUTINE PDAFomi_set_localize_covar(thisobs, dim_p, ncoords, coords, &
    271        locweight, cradius, sradius)
    272 
    273     TYPE(obs_f), INTENT(inout) :: thisobs           ! Data type with full observation
    274     INTEGER, INTENT(in) :: dim_p                    ! State dimension
    275     INTEGER, INTENT(in) :: ncoords                  ! Number of coordinate directions
    276     REAL, INTENT(in)    :: coords_p(dim_p,ncoords)  ! Coordinates of state vector elements
    277     INTEGER, INTENT(in) :: locweight                ! Localization weight type
    278     REAL, INTENT(in)    :: cradius                  ! localization radius
    279     REAL, INTENT(in)    :: sradius                  ! Support radius for weight functions
    280 }}}
    281 
    282 '''Notes:'''
    283  * The routine allows to specify the localization radius and support radius (`lradius`, `sradius`) and localization function (`locweight`) individually for each observation type.
    284  * The coordinate array `coords_p` specifies the coordinates of each element in the state vector for the process local sub-domain. The coordinate units have to be consistent with those used to specify the coordinates of observations.
    285  * The routine only supports a fixed localization radius throughout the domain.
    286  * One can also directly call the routine `PDAFomi_set_localize_covar_iso`, e.g. when calling from a program coded in C.
    287 
    288 There is also a variant of this routine for non-isotropic localization. See the [wiki:PDAFomi_set_localize_covar Documentation on PDAFomi_set_localize_covar] for more information.
     257See the [wiki:PDAFomi_set_localize_covar Documentation on PDAFomi_set_localize_covar] for more information.
    289258
    290259
     
    292261
    293262To implement a new observation type, the approach is generally as follows:
    294 1.      Create a copy of `obs_OBSTYPE_pdafomi_TEMPLATE.F90`
     2631.      Create a copy of `obs_OBSTYPE_pdafomi.F90` (`obs_OBSTYPE_pdafomi_TEMPLATE.F90` in older version of PDAF)
    2952641.      Rename the module and its subroutines according to the observation (replacing ‘OBSTYPE’ by name of observation).
    296 1.      Implement `init_dim_obs` for the observation type following the instructions in the template
    297 1.      Adapt `obs_op` for the observation type
    298 1.      Adapt `init_dim_obs_l` for the observation type (if using a domain_localized filter)
    299 1.      Adapt `localize_covar` for the observation type (if using a the local EnKF)
     2651.      Implement `init_dim_obs_OBSTYPE` for the observation type following the instructions in the template
     2661.      Adapt `obs_op_OBSTYPE` for the observation type
     2671.      Adapt `init_dim_obs_l_OBSTYPE` for the observation type (if using a domain_localized filter)
    3002681.      Add subroutine calls for the new observation type into the routines in `callback_obs_pdafomi.F90`
    301269