LnDevel – PDAF - Parallel Data Assimilation Framework

LESTKF LETKF LSEIK	PDAF3_assimilate_local_nondiagR !	PDAF3_put_state_local_nondiagR !
LNETF	PDAF3_assimilate_lnetf_nondiagR !	PDAF3_put_state_lnetf_nondiagR !
LKNETF	PDAF3_assimilate_lknetf_nondiagR !	PDAF3_put_state_lknetf_nondiagR !
ESTKF ETKF SEIK	PDAF3_assimilate_global_nondiagR !	PDAF3_put_state_global_nondiagR !
NETF PF	PDAF3_assimilate_nonlin_nondiagR !	PDAF3_put_state_nonlin_nondiagR !
EnKF LEnKF	PDAF3_assimilate_enkf_nondiagR !	PDAF3_put_state_enkf_nondiagR !

3DVar	PDAF3_assimilate_3dvar !	PDAF3_put_state_3dvar !
En3DVar with ESTKF	PDAF3_assimilate_en3dvar_estkf !	PDAF3_put_state_en3dvar_estkf !
Hyb3DVar with ESTKF	PDAF3_assimilate_hyb3dvar_estkf !	PDAF3_put_state_hyb3dvar_estkf !
En3DVar with LESTKF	PDAF3_assimilate_en3dvar_lestkf !	PDAF3_put_state_en3dvar_lestkf !
Hyb3DVar with ESTKF	PDAF3_assimilate_hyb3dvar_lestkf !	PDAF3_put_state_hyb3dvar_lestkf !

Method	fully parallel & flexible parallel	flexible parallel (backward-compatibility)
3DVar	PDAF3_assimilate_3dvar_nondiagR	PDAF3_put_state_3dvar_nondiagR
En3DVar with LESKTF	PDAF3_assimilate_en3dvar_lestkf_nondiagR	PDAF3_put_state_en3dvar_lestkf_nondiagR
En3DVar with ESTKF	PDAF3_assimilate_en3dvar_estkf_nondiagR	PDAF3_put_state_en3dvar_estkf_nondiagR
Hyb3DVar with LESTKF	PDAF3_assimilate_hyb3dvar_lestkf_nondiagR	PDAF3_put_state_hyb3dvar_lestkf_nondiagR
Hyb3DVar with ESTKF	PDAF3_assimilate_hyb3dvar_estkf_nondiagR	PDAF3_put_state_hyb3dvar_estkf_nondiagR

For using PDAF_assimilate in flexible mode

PDAF_get_fcst_info

Need to adapt

ExternalModelLoop

ImplementAnalysisLocal

ImplementAnalysisGlobal

PDAFlocal_overview

ImplementAnalysisLocal

ImplementAnalysisGlobal

OfflineInitPdaf subtype=5

PdafSimplifiedInterface Note on no longer supported

PDAFomi_additional_functionality Add hint to PDAFomi_diag

Porting to PDAF-3

PDAF_interfaces_module now named PDAF; mandatory to use
subtype=5 for specifying the offline mode was removed; use PDAF_set_offline_mode
additional parameter in PDAF_assimilate_lnetf and PDAF_put_state_lnetf
porting from PDAFomi to PDAF3 for global filters: Just change the name of the routine
porting from PDAFlocalomi to PDAF3 for local filters: Just change the name of the routine
porting from PDAFomi to PDAF3 for local filters: The PDAF3 interface uses PDAFlocal so that the routines g2l_state_pdaf and l2g_state_pdaf are not present. Follow the implementation guide for init_dim_l to initialize the information for PDAFlocal
porting from classical PDAF interface with fully specification of all call-back routines: See implementation guide (PDAF3 uses both PDAF-OMI and PDAF-local to provide compact interfaces)
porting from PDAFomi or PDAFlocalomi for 3D-Var: Change the name of the routine
check subtype: 0 and 1 are unchanged; for others: 2=>10, 3=>11, 4=>2, 6=>3, 7=>4
the option to run the offline mode as 'subtype=5' was removed. It is now require to call "PDAF_set_offline_mode"

Version 3.0beta - April XX, 2025

This is a major release. The whole PDAF code was revised and modernized.

Important: Due to changes in the code, PDAF V3.0 is not fully backward compatible. Thus, with this release it is not possible to just compile and link with existing implementations. The required adaptions are no large. Please see ..... for a description of how to port your existing code to PDAF V3.0

We have marked this release as a beta-release. While we have tested this release with different applications, we cannot exclude that there might be some hidden issue due to the extensive changes in the code. We like to motivate all users to test PDAF V3.0 and to report possible issues to use (using the Issues in Github or by email at pdaf\@awi.de). This will help us to fix such issues so that the final release for PDAF V3.0 can hopefull ussor in 2-3 months. During this time, we will also finalize the documentation to reflect all changes.

Major changes with effect on backward-compatibility

replaced the optional module 'PDAF_interfaces_module' by the module 'PDAF'. In Fortran code, it is now required to include this module with 'USE PDAF'.
Change for offline mode: It is no longer spossible to specify the offline mode as 'subtype=5'. The offline model can now only be activated with a call to 'PDAF_set_offline_mode'.
Changes in subtypes (Only rleevant if one used other subtpyes the 0 or 1): The subtypes for the ensemble filters have been renumbered: subtypes 0 and 1 are unchanged, the other dynamic subtypes are changed as 4=>2, 6=>3, 7=>4; the variant with fixed ensmeble are changed as 2=>10, 3=>11.
Changes in the index of selected parameters: The index of some of the parameters set in PDAF_init has changed for better overall consistency. (See .... for an overview of the parameters)
The SEEK filter (filtertype=0) was finally removed after it was marked deprecated for a longer period.
Removed 'src/Makefile'. Now only the new Makefile in the main directory exists. However, if one has a workflow basing on src/Makefile and can copy the Makefile and adapt it according to the description within the file.

Major changes and additions:

Revision of the initialization of observations: Now the observations are initialized before the actual analysis step is computed. Also the observartion operator is called before the actual analysis step. This also allows the option to initialize observations before prepoststep_pdaf is called for the forecast ensemble. Related to this the screen output for the observations was revised and is displayed at a different location from before
New PDAF3 interface: The selection of possible calls to PDAF*_put_state and PDAF*_assimilate has grown over the years. We introduced a new interface PDAF3_assimilate_X and PDAF3_put_state_X, which uses the latesd features that were introduced in previous PDAF releases and allows for a minimum number of arguments. The previous routines still exist, but we recommend to change to the new routines.
New routine PDAF_get_fcst_info: This routine allows to retrieve the return values of PDAF_get_state (nsteps, time, doexit) for the case that PDAF_assimilate routines, which do not return these values, are used.
Change for flexible parallelization variant: For this variant it is now possible to use PDAF_assimilate routines, while PDAF_put_state routines had to be called before. Using PDAF_assimilate in combination with 'PDAF_get_fcst_info' is the recommend choice for PDAF V3. Since PDAF_assimilate routines are called at each time step during the forecast phase this allows to easy use of the IAU functionality.
Added observation diagnostics module for OMI: This is a set of routines that give the user access to the observation information including the observed ensemble and the observed ensemble mean. In addition, there are routines computing statistics for the difference beetween the observations and the observed ensemble mean.
New routine PDAFomi_set_localize_covar: This routine is called in the OMI observation module in init_dim_obs_pdafomi. It sets the information for localization in the LEnKF and ENSRF/EAKF methods. (optional for LEnKF)
Calling LEnKF with the common routine PDAF3_assimilate_global: In PDAF 2.3 and before, the LEnKF had its own interface due to the additional call-back routine localize_covar_pdafomi required by this filter. With the new routine PDAFomi_set_localize_covar that is called in the OMI observation module in init_dim_obs_pdafomi the additional user-supplied subroutine localize_covar_pdafomi is no longer required which lowers the amount of implementation work for the user.
Incremental Analysis Updates (IAU): The previous IAU of PDAF was recoded. Now a fully featured IAU is supported.
ENSRF/EAKF: The ensemble square-root filter (ENSRF, Whitaker & Hamill, 2002) and ensemble adjustment filter with local linear regression (EAKF, Anderson, 2003) were added. These filters perform the analysis step with a serial processing of single observations.
Added more diagnostic routines, e.g. for computing ensemble mean (PDAF_diag_ensmean), ensemble mean standard deviation (PDAF_diag_stdev), variance state vector (PDAF_diag_variance) and higher order moments (PDAF_diag_compute_moments).
Revised setting parameters for PDAF: Before the parameters were all set in the call to PDAF_init. Now there are new routines PDAF_set_iparam and PDAF_set_rparam to set selected parameters. These make the specification of parameters for PDAF for flexible and can also be used during a run to change parameter values.
New routine PDAF_set_seedset: This routine can be used to seed the randum number seed for genering observations
Introduced named variables for choosing the filter type: The filter type can now also be specified by a named parameter of the form PDAF_DA_X, where 'X' is the DA method, e.g. 'ESTKF' or '3DVAR'. With this one does not need to look up the number of the filter type.
Added routine 'PDAF_print_filter_types': This routine writes a list of the available filter types and named parameters PDAF_DA_X.
Added Fortran function PDAF_localfilter as alternative to subroutine PDAF_get_localfilter to make code more compact
Added subroutine PDAF_get_local_type and Fortran function PDAF_local_type. These return information which of the different localization variants a filter uses (no localization, local analysis, covariance localization, covariance localization with serial observation processing)
The screen output of the configration of the DA method was revised and provides now a complete configuration overview

Other changes:

The tutoral input files have been regenerated using Python scripts instead of the previous Matlab scripts. This changed their numbers.
Changed statistics for locally omitted observations: Now these are only counted for analysis domains with observations.
Added information output on allocated memory in PDAF-OMI
Further change in PDAFomi_init_dim_obs_l to improve the performance of the local observation search
Added templates for implementing global or local ensemble filters in templates/analysis/
Revision of PDAF file headers: It is now possible to use 'Doxygen' to generate a documentation and structure overview of PDAF.

Bug corrections:

Corrected a parallel race condition in the LKNETF NHK variant (subtype=0) in case of OpenMP parallelization
Corrected convergence check for the CG+ solver in the 3D-Var methods in case of parallelized control vector.
Corrected computation of likelihood for the case that dim_obs_l=0 for some observation type, but overall dim_obs_l>0. This could lead to a segmentatoin fault in the LKNETF)
Correction of type_forget=2: For the case that the adaptive forgetting factor would compute a negative value, the forgetting factor of the previous domain was used. Now, the prescribed default value is used. Here also a possible race condition when using OpenMP was fixed.
Corrected call to U_obs_op_adj. Here inside PDAF a rank-1 array was provided as argument, while it needed to be a rank-2 array with only one column.

Note: See TracWiki for help on using the wiki.

Download in other formats:

Plain Text