Changes between Version 2 and Version 3 of AdaptParallelization


Ignore:
Timestamp:
Aug 23, 2010, 11:05:41 AM (10 years ago)
Author:
lnerger
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • AdaptParallelization

    v2 v3  
    11= Adapting a model's parallelization for PDAF =
    22
    3 PDAF uses the MPI standard for the parallelization, like many numerical models. In the description below, we assume that the model is parallelized using MPI.
     3Like many numerical models, PDAF uses the MPI standard for the parallelization. In the description below, we assume that the model is parallelized using MPI.
    44
    5 If the parallelization possibilities of PDAF should be used, the parallelization for parallel ensemble integrations (below denoted as model tasks) has to be initialized. If the numerical model itself is parallelized, this parallelization has to be adapted and modified for the 2-level parallelization of the data assimilation system generated by adding PDAF to the model.
     5PDAF supports a 2-level parallelization: First, the numerical model can be parallelized and can be executed using several processors. Second, several model tasks can be computed in parallel, i.e. a parallel ensemble integration can be performed. This 2-level parallelization has to be initialized before it can be used. The example in `testsuite/dummymodel_1D/` includes the file `init_parallel_pdaf.F90` that can be used as a template for the initialization. The required variables are defined in `testsuite/main/mod_parallel.F90`. If the numerical model itself is parallelized, this parallelization has to be adapted and modified for the 2-level parallelization of the data assimilation system generated by adding PDAF to the model. The necessary steps are described below.
    66
    7 The parallel region of an MPI parallel program is initialized by calling MPI_init. MPI uses communicators to define sets of parallel processes. By calling MPI_init, the communicator MPI_COMM_WORLD is initialized. This communicator contains all processes of the MPI-parallel program. Often it is sufficient to conduct all parallel communication using this communicator. Thus, numerical models often use only MPI_COMM_WORLD to control all communication. However, this approach will not allow for parallel model tasks. In order to allow parallel model tasks, it is required to replace MPI_COMM_WORLD by an alternative communicator that is split for the model tasks. We will denote this communicator COMM_MODEL.
     7
     8== Three communicators ==
     9
     10MPI uses so-called 'communicators' to define sets of parallel processes. In order to provide the 2-level parallelism, three communicators need to be initialized that define the processes that are involved in different tasks of the data assimilation system.
     11The required communicators are initialized in the routine `init_parallel_pdaf` and called
     12 * `COMM_model` - defines the processes that are involved in the model integrations
     13 * `COMM_filter` - defines the processes that perform the filter analysis step
     14 * `COMM_couple` - defines the processes that are involved when data are transferred between the model and the filter
     15
     16The parallel region of an MPI parallel program is initialized by calling `MPI_init`.  By calling `MPI_init`, the communicator `MPI_COMM_WORLD` is initialized. This communicator is pre-defined by MPI to contain all processes of the MPI-parallel program. Often it is sufficient to conduct all parallel communication using only `MPI_COMM_WORLD`. Thus, numerical models often use only this communicator to control all communication. However, as `MPI_COMM_WORLD` contains all processes of the program, this approach will not allow for parallel model tasks. In order to allow parallel model tasks, it is required to replace `MPI_COMM_WORLD` by an alternative communicator that is split for the model tasks. We will denote this communicator `COMM_model`.
     17
     18== Using COMM_model ==
     19
    820Frequently the parallelization is initialized in the model by the lines:
    921{{{
     
    1224      CALL MPI_Comm_Size(MPI_COMM_WORLD, size, ierr)
    1325}}}
    14 Subsequently, one can define COMM_MODEL by adding
     26(The call to `MPI_init` is mandatory, while the second an third line are optional)
     27Subsequently, one can define `COMM_model` by adding
    1528{{{
    1629      COMM_MODEL = MPI_COMM_WORLD