Changes between Version 5 and Version 6 of OnlineorOfflineImplementation

Timestamp:

Oct 3, 2010, 9:52:12 PM (14 years ago)

Author:

lnerger

Comment:

--

OnlineorOfflineImplementation

@@ -15 +15 @@
 When the assimilation system uses a large-scale model, there are more considerations about online and offline systems. In the online mode, a single ensemble has to be run using a rather large number of processors. In case of the offline mode, the integration of each ensemble member state can be performed using a single run with a smaller number of processors. When these runs are completed, the assimilation executable is run. Here the number of processors can be set rather independently from the number user for the model runs (This will depend on file output). In this respect the consideration is how to obtain the best throughput of the assimilation system. As large-scale models are typically run on computers using a batch or scheduling system one has to takes its configuration into account. Is it more likely that the single but large job is executed, or are a larger number of smaller jobs more likely to be executed. If the batch system only allows a very limited number of concurrent jobs of a single user, then the ensemble integration wil become essentially serial. (However, there can be possibilities to execute several parallel model integrations within a single batch job)
 
-Actually, the consideration of an offline or online implementation is not an ultimate one. In fact, one can consider to implement first an offline system and later move on to an online system. This is due to the fact that the observation-related routines (observation operator, initialization of array holding observations) perform the same operations in online and offline systems. However, in an offline system all information about the model fields like the location of individual grid points is read from the model files. In contrast, in an online implementation, this information is usually taken from some array of the model code. Spatial information about observations is read from a files in both implementation variants. In addition, one need routines to write state estimates into files in both implementation variants. 
+Apart from these considerations, we have to stress that PDAF does not provide scripts controlling the ensemble integration in the offline case. The actual scripts used to start a model are often very specific to the model (e.g. including operations to prepare forcing files). In addition, the submission to batch systems, is individual for the particular configuration of the batch system of the machine used.  Thus, a generic script to control the execution of model-specific scripts could be rather complex. For this reason, we decided to leave it to the user to control the ensemble integration of the offline implementation. 
+
+Finally, we like to stress that the consideration whether an offline or online implementation is performed is not an ultimate one. In fact, one can consider to implement first an offline system and later move on to an online system. This is due to the fact that the observation-related routines (observation operator, initialization of array holding observations) perform the same operations in online and offline systems. However, in an offline system all information about the model fields like the location of individual grid points is read from the model files. In contrast, in an online implementation, this information is usually taken from some array of the model code. Spatial information about observations is read from a files in both implementation variants. In addition, one need routines to write state estimates into files in both implementation variants. 
 Over all, the changes to the observation-related routines should be rather limited when moving from an offline to an online implementation. Thus, the major work will be the adaption of the parallelization, and the extension of the model code for the ensemble integration.