Changes between Version 2 and Version 3 of AdaptParallelization

Timestamp:

Aug 23, 2010, 11:05:41 AM (14 years ago)

Author:

lnerger

Comment:

--

AdaptParallelization

@@ -1 +1 @@
 = Adapting a model's parallelization for PDAF =
 
-PDAF uses the MPI standard for the parallelization, like many numerical models. In the description below, we assume that the model is parallelized using MPI.
+Like many numerical models, PDAF uses the MPI standard for the parallelization. In the description below, we assume that the model is parallelized using MPI.
 
-If the parallelization possibilities of PDAF should be used, the parallelization for parallel ensemble integrations (below denoted as model tasks) has to be initialized. If the numerical model itself is parallelized, this parallelization has to be adapted and modified for the 2-level parallelization of the data assimilation system generated by adding PDAF to the model. 
+PDAF supports a 2-level parallelization: First, the numerical model can be parallelized and can be executed using several processors. Second, several model tasks can be computed in parallel, i.e. a parallel ensemble integration can be performed. This 2-level parallelization has to be initialized before it can be used. The example in `testsuite/dummymodel_1D/` includes the file `init_parallel_pdaf.F90` that can be used as a template for the initialization. The required variables are defined in `testsuite/main/mod_parallel.F90`. If the numerical model itself is parallelized, this parallelization has to be adapted and modified for the 2-level parallelization of the data assimilation system generated by adding PDAF to the model. The necessary steps are described below.
 
-The parallel region of an MPI parallel program is initialized by calling MPI_init. MPI uses communicators to define sets of parallel processes. By calling MPI_init, the communicator MPI_COMM_WORLD is initialized. This communicator contains all processes of the MPI-parallel program. Often it is sufficient to conduct all parallel communication using this communicator. Thus, numerical models often use only MPI_COMM_WORLD to control all communication. However, this approach will not allow for parallel model tasks. In order to allow parallel model tasks, it is required to replace MPI_COMM_WORLD by an alternative communicator that is split for the model tasks. We will denote this communicator COMM_MODEL.
+
+== Three communicators ==
+
+MPI uses so-called 'communicators' to define sets of parallel processes. In order to provide the 2-level parallelism, three communicators need to be initialized that define the processes that are involved in different tasks of the data assimilation system. 
+The required communicators are initialized in the routine `init_parallel_pdaf` and called
+ * `COMM_model` - defines the processes that are involved in the model integrations
+ * `COMM_filter` - defines the processes that perform the filter analysis step
+ * `COMM_couple` - defines the processes that are involved when data are transferred between the model and the filter
+
+The parallel region of an MPI parallel program is initialized by calling `MPI_init`.  By calling `MPI_init`, the communicator `MPI_COMM_WORLD` is initialized. This communicator is pre-defined by MPI to contain all processes of the MPI-parallel program. Often it is sufficient to conduct all parallel communication using only `MPI_COMM_WORLD`. Thus, numerical models often use only this communicator to control all communication. However, as `MPI_COMM_WORLD` contains all processes of the program, this approach will not allow for parallel model tasks. In order to allow parallel model tasks, it is required to replace `MPI_COMM_WORLD` by an alternative communicator that is split for the model tasks. We will denote this communicator `COMM_model`.
+
+== Using COMM_model ==
+
 Frequently the parallelization is initialized in the model by the lines:
 {{{
@@ -12 +24 @@
       CALL MPI_Comm_Size(MPI_COMM_WORLD, size, ierr)
 }}}
-Subsequently, one can define COMM_MODEL by adding
+(The call to `MPI_init` is mandatory, while the second an third line are optional)
+Subsequently, one can define `COMM_model` by adding
 {{{
       COMM_MODEL = MPI_COMM_WORLD