commit de david

[hpcc2014.git] / hpcc.tex

diff --git a/hpcc.tex b/hpcc.tex
index 62922bcbae3b99f759e7d8dc5a1acf1910b9c8f4..d1ce8c8580c0ede5a7b47809038b6b48a14520e0 100644 (file)
--- a/hpcc.tex
+++ b/hpcc.tex
@@ -10,7 +10,7 @@
 \usepackage{graphicx}
 \usepackage[american]{babel}
 % Extension pour les liens intra-documents (tagged PDF)
 \usepackage{graphicx}
 \usepackage[american]{babel}
 % Extension pour les liens intra-documents (tagged PDF)

-% et l'affichage correct des URL (commande \url{http://example.com})
+% et l'affichage correct des UR (commande \url{http://example.com})

 %\usepackage{hyperref}
 
 \usepackage{url}
 %\usepackage{hyperref}
 
 \usepackage{url}


@@ -77,8 +77,8 @@ Synchronous  iterative  algorithms  are  often less  scalable  than  asynchronou
 iterative  ones.  Performing  large  scale experiments  with  different kind  of
 network parameters is not easy  because with supercomputers such parameters are
 fixed. So, one  solution consists in using simulations first  in order to analyze
 iterative  ones.  Performing  large  scale experiments  with  different kind  of
 network parameters is not easy  because with supercomputers such parameters are
 fixed. So, one  solution consists in using simulations first  in order to analyze

-what parameters  could influence or not  the behaviors of an  algorithm. In this
-paper, we show  that it is interesting to use SimGrid  to simulate the behaviors
+what parameters  could influence or not  the behavior of an  algorithm. In this
+paper, we show  that it is interesting to use SimGrid  to simulate the behavior

 of asynchronous  iterative algorithms. For that,  we compare the  behavior of a
 synchronous  GMRES  algorithm  with  an  asynchronous  multisplitting  one  with
 simulations  which let us easily choose  some parameters.   Both  codes  are real  MPI
 of asynchronous  iterative algorithms. For that,  we compare the  behavior of a
 synchronous  GMRES  algorithm  with  an  asynchronous  multisplitting  one  with
 simulations  which let us easily choose  some parameters.   Both  codes  are real  MPI


@@ -177,29 +177,29 @@ out will be presented before some concluding remarks and future works.
  
 \section{Motivations and scientific context}
 
  
 \section{Motivations and scientific context}
 

-As exposed in  the introduction, parallel iterative methods  are now widely used
-in  many scientific  domains.   They can  be  classified in  three main  classes
-depending on  how iterations  and communications are  managed (for  more details
-readers  can refer  to~\cite{bcvc06:ij}). In  the synchronous  iterations model,
-data are exchanged  at the end of each iteration. All  the processors must begin
-the same iteration  at the same time and important idle  times on processors are
+As described in the introduction, parallel iterative methods are now widely used
+in many scientific domains.  They can be classified in three main classes
+depending on how iterations and communications are managed (for more details
+readers can refer to~\cite{bcvc06:ij}). In the synchronous iterations model,
+data are exchanged at the end of each iteration. All the processors must begin
+the same iteration at the same time and important and useless idle times used for synchronization on processors are

 generated.  It is possible to use asynchronous communications, in this case, the
 generated.  It is possible to use asynchronous communications, in this case, the

-model can be  compared to the previous one except that  data required on another
-processor are  sent asynchronously i.e.  without  stopping current computations.
-This technique  allows communications to be partially  overlapped by  computations but
-unfortunately, the overlapping is only  partial and important idle times remain.
-It is clear that, in a grid computing context, where the number of computational
-nodes is large,  heterogeneous and widely distributed, the  idle times generated
-by synchronizations are very penalizing. One  way to overcome this problem is to
-use the asynchronous iterations model.   Here, local computations do not need to
-wait for  required data. Processors can  then perform their  iterations with the
-data present  at that time.  Figure~\ref{fig:aiac} illustrates  this model where
-the gray blocks represent the  computation phases.  With this algorithmic model,
-the number  of iterations required  before the convergence is  generally greater
-than  for the  two former  classes.  But,  and as  detailed in~\cite{bcvc06:ij},
-asynchronous  iterative algorithms  can significantly  reduce  overall execution
-times by  suppressing idle  times due to  synchronizations especially in  a grid
-computing context.
+model can be compared to the previous one except that data required on another
+processor are sent asynchronously i.e.  without stopping current computations.
+This technique allows communications to be partially overlapped by computations
+but unfortunately, the overlapping is only partial and useless idle times used for synchronization remain.  
+It is clear that, in a grid computing context, where the number of
+computational nodes is large, heterogeneous and widely distributed, the idle
+times generated by synchronizations are very penalizing. One way to overcome
+this problem is to use the asynchronous iterations model.  Here, local
+computations do not need to wait for required data. Processors can then perform
+their iterations with the data present at that time.  Figure~\ref{fig:aiac}
+illustrates this model where the gray blocks represent the computation phases.
+With this algorithmic model, the number of iterations required before the
+convergence is generally greater than for the two former classes.  But, and as
+detailed in~\cite{bcvc06:ij}, asynchronous iterative algorithms can
+significantly reduce overall execution times by suppressing idle times due to
+synchronizations especially in a grid computing context.

 
 \begin{figure}[!t]
   \centering
 
 \begin{figure}[!t]
   \centering


@@ -235,7 +235,7 @@ algorithm  (number   of  splittings  with  the   multisplitting  algorithm),  th
 multisplitting code  will obtain the solution  more or less  quickly. Of course,
 the GMRES method also depends on the same parameters. As it is difficult to have
 access to  many clusters,  grids or supercomputers  with many  different network
 multisplitting code  will obtain the solution  more or less  quickly. Of course,
 the GMRES method also depends on the same parameters. As it is difficult to have
 access to  many clusters,  grids or supercomputers  with many  different network

-parameters,  it  is  interesting  to  be  able  to  simulate  the  behaviors  of
+parameters,  it  is  interesting  to  be  able  to  simulate  the  behavior  of

 asynchronous iterative algorithms before being able to run real experiments.
 
 
 asynchronous iterative algorithms before being able to run real experiments.
 
 


@@ -245,29 +245,30 @@ asynchronous iterative algorithms before being able to run real experiments.
 
 \section{SimGrid}
 
 
 \section{SimGrid}
 

-SimGrid~\cite{SimGrid,casanova+legrand+quinson.2008.simgrid} is a simulation
-framework to study the behavior of large-scale distributed systems.  As its name
-suggests, it emanates from the grid computing community, but is nowadays used to
-study grids, clouds, HPC or peer-to-peer systems.  The early versions of SimGrid
-date back from 1999, but it is still actively developed and distributed as an open
-source software.  Today, it is one of the major generic tools in the field of
-simulation for large-scale distributed systems.
+SimGrid~\cite{SimGrid,casanova+legrand+quinson.2008.simgrid,casanova+giersch+legrand+al.2014.versatile}
+is a simulation framework to study the behavior of large-scale distributed
+systems.  As its name suggests, it emanates from the grid computing community,
+but is nowadays used to study grids, clouds, HPC or peer-to-peer systems.  The
+early versions of SimGrid date back from 1999, but it is still actively
+developed and distributed as an open source software.  Today, it is one of the
+major generic tools in the field of simulation for large-scale distributed
+systems.

 
 SimGrid provides several programming interfaces: MSG to simulate Concurrent
 Sequential Processes, SimDAG to simulate DAGs of (parallel) tasks, and SMPI to
 run real applications written in MPI~\cite{MPI}.  Apart from the native C
 interface, SimGrid provides bindings for the C++, Java, Lua and Ruby programming
 
 SimGrid provides several programming interfaces: MSG to simulate Concurrent
 Sequential Processes, SimDAG to simulate DAGs of (parallel) tasks, and SMPI to
 run real applications written in MPI~\cite{MPI}.  Apart from the native C
 interface, SimGrid provides bindings for the C++, Java, Lua and Ruby programming

-languages.  SMPI is the interface that has been used for the work exposed in
+languages.  SMPI is the interface that has been used for the work described in

 this paper.  The SMPI interface implements about \np[\%]{80} of the MPI 2.0
 standard~\cite{bedaride+degomme+genaud+al.2013.toward}, and supports
 this paper.  The SMPI interface implements about \np[\%]{80} of the MPI 2.0
 standard~\cite{bedaride+degomme+genaud+al.2013.toward}, and supports

-applications written in C or Fortran, with little or no modifications.
+applications written in C or Fortran, with little or no modifications (cf Section IV - paragraph B).

 
 Within SimGrid, the execution of a distributed application is simulated by a
 single process.  The application code is really executed, but some operations,
 like communications, are intercepted, and their running time is computed
 according to the characteristics of the simulated execution platform.  The
 description of this target platform is given as an input for the execution, by
 
 Within SimGrid, the execution of a distributed application is simulated by a
 single process.  The application code is really executed, but some operations,
 like communications, are intercepted, and their running time is computed
 according to the characteristics of the simulated execution platform.  The
 description of this target platform is given as an input for the execution, by

-the mean of an XML file.  It describes the properties of the platform, such as
+means of an XML file.  It describes the properties of the platform, such as

 the computing nodes with their computing power, the interconnection links with
 their bandwidth and latency, and the routing strategy.  The scheduling of the
 simulated processes, as well as the simulated running time of the application
 the computing nodes with their computing power, the interconnection links with
 their bandwidth and latency, and the routing strategy.  The scheduling of the
 simulated processes, as well as the simulated running time of the application


@@ -415,11 +416,11 @@ cluster 1 broadcasts a stop message to the masters of other clusters. In this wo
 the local convergence on each cluster $\ell$ is detected when the following
 condition is satisfied
 \begin{equation*}
 the local convergence on each cluster $\ell$ is detected when the following
 condition is satisfied
 \begin{equation*}

-  (k\leq \MI) \text{ or } (\|X_\ell^k - X_\ell^{k+1}\|_{\infty}\leq\epsilon)
+  (k=\MI) \text{ or } (\|X_\ell^k - X_\ell^{k+1}\|_{\infty}\leq\epsilon)

 \end{equation*}
 where $\MI$ is the maximum number of outer iterations and $\epsilon$ is the
 tolerance threshold of the error computed between two successive local solution
 \end{equation*}
 where $\MI$ is the maximum number of outer iterations and $\epsilon$ is the
 tolerance threshold of the error computed between two successive local solution

-$X_\ell^k$ and $X_\ell^{k+1}$.
+$X_\ell^k$ and $X_\ell^{k+1}$. It should be noted that with asynchronous iterative algorithms, we cannot use a classical norm (which would require to synchronize all processors), such as $\|X_\ell^k - X_\ell^{k+1}\|_{2}$ for example. Nevertheless, in our experiments, we check that the final result is correct, for this we compute the precision with $max_i | A*x-b |_i$.

 
 
 
 
 
 


@@ -460,8 +461,8 @@ The parallel solving of the 3D Poisson problem with our multisplitting method re
 
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

-We did not encounter major blocking problems when adapting the multisplitting algorithm previously described to a simulation environment like SimGrid unless some code 
-debugging. Indeed, apart from the review of the program sequence for asynchronous exchanges between processors within a cluster or between clusters, the algorithm was executed successfully with SMPI and provided identical outputs as those obtained with direct execution under MPI. For the synchronous GMRES method, the execution of the program raised no particular issue but in the asynchronous multisplitting method, the review of the sequence of \texttt{MPI\_Isend, MPI\_Irecv} and \texttt{MPI\_Waitall} instructions
+We did not encounter major blocking problems when adapting the multisplitting algorithm previously described to a simulation environment like SimGrid. Only, some code 
+debugging has been required. Indeed, apart from the review of the program sequence for asynchronous exchanges between processors within a cluster or between clusters, the algorithm was executed successfully with SMPI and provided identical outputs as those obtained with direct execution under MPI. For the synchronous GMRES method, the execution of the program raised no particular issue but in the asynchronous multisplitting method, the review of the sequence of \texttt{MPI\_Isend, MPI\_Irecv} and \texttt{MPI\_Waitall} instructions

 and with the addition of the primitive \texttt{MPI\_Test} was needed to avoid a memory fault due to an infinite loop resulting from the non-convergence of the algorithm.
 %\CER{On voulait en fait montrer la simplicité de l'adaptation de l'algo a SimGrid. Les problèmes rencontrés décrits dans ce paragraphe concerne surtout le mode async}\LZK{OK. J'aurais préféré avoir un peu plus de détails sur l'adaptation de la version async} 
 %\CER{Le problème majeur sur l'adaptation MPI vers SMPI pour la partie asynchrone de l'algorithme a été le plantage en SMPI de Waitall après un Isend et Irecv. J'avais proposé un workaround en utilisant un MPI\_wait séparé pour chaque échange a la place d'un waitall unique pour TOUTES les échanges, une instruction qui semble bien fonctionner en MPI. Ce workaround aussi fonctionne bien. Mais après, tu as modifié le programme avec l'ajout d'un MPI\_Test, au niveau de la routine de détection de la convergence et du coup, l'échange global avec waitall a aussi fonctionné.}
 and with the addition of the primitive \texttt{MPI\_Test} was needed to avoid a memory fault due to an infinite loop resulting from the non-convergence of the algorithm.
 %\CER{On voulait en fait montrer la simplicité de l'adaptation de l'algo a SimGrid. Les problèmes rencontrés décrits dans ce paragraphe concerne surtout le mode async}\LZK{OK. J'aurais préféré avoir un peu plus de détails sur l'adaptation de la version async} 
 %\CER{Le problème majeur sur l'adaptation MPI vers SMPI pour la partie asynchrone de l'algorithme a été le plantage en SMPI de Waitall après un Isend et Irecv. J'avais proposé un workaround en utilisant un MPI\_wait séparé pour chaque échange a la place d'un waitall unique pour TOUTES les échanges, une instruction qui semble bien fonctionner en MPI. Ce workaround aussi fonctionne bien. Mais après, tu as modifié le programme avec l'ajout d'un MPI\_Test, au niveau de la routine de détection de la convergence et du coup, l'échange global avec waitall a aussi fonctionné.}


@@ -484,7 +485,7 @@ study that the results depend on the following parameters:
 \begin{itemize}
 \item At the network level, we found that the most critical values are the
   bandwidth and the network latency.
 \begin{itemize}
 \item At the network level, we found that the most critical values are the
   bandwidth and the network latency.

-\item Hosts processors power (GFlops) can also influence the results.
+\item Host processor power (GFlops) can also influence the results.

 \item Finally, when submitting job batches for execution, the arguments values
   passed to the program like the maximum number of iterations or the precision are critical. They allow us to ensure not only the convergence of the
   algorithm but also to get the main objective in getting an execution time with the asynchronous multisplitting  less than with synchronous GMRES. 
 \item Finally, when submitting job batches for execution, the arguments values
   passed to the program like the maximum number of iterations or the precision are critical. They allow us to ensure not only the convergence of the
   algorithm but also to get the main objective in getting an execution time with the asynchronous multisplitting  less than with synchronous GMRES. 


@@ -682,7 +683,7 @@ stable even if the residual error precision varies from \np{E-5} to \np{E-9}. By
 increasing the matrix size up to $100^3$ elements, it was necessary to increase the
 CPU power by \np[\%]{50} to \np[GFlops]{1.5} to get the algorithm convergence and the same order of asynchronous mode efficiency.  Maintaining  a relative gain of $2.5$ and such processor power but increasing network throughput inter cluster up to \np[Mbit/s]{50},  is obtained with
 high external precision of \np{E-11} for a matrix size from $110^3$ to $150^3$ side
 increasing the matrix size up to $100^3$ elements, it was necessary to increase the
 CPU power by \np[\%]{50} to \np[GFlops]{1.5} to get the algorithm convergence and the same order of asynchronous mode efficiency.  Maintaining  a relative gain of $2.5$ and such processor power but increasing network throughput inter cluster up to \np[Mbit/s]{50},  is obtained with
 high external precision of \np{E-11} for a matrix size from $110^3$ to $150^3$ side

-elements.
+elements. 

 
 %For the 3 clusters architecture including a total of 100 hosts,
 %Table~\ref{tab.cluster.3x33} shows that it was difficult to have a combination
 
 %For the 3 clusters architecture including a total of 100 hosts,
 %Table~\ref{tab.cluster.3x33} shows that it was difficult to have a combination


@@ -705,7 +706,7 @@ elements.
 %\CER{Définitivement, les paramètres réseaux variables ici se rapportent au réseau INTER cluster.}
 \section{Conclusion}
 The simulation of the execution of parallel asynchronous iterative algorithms on large scale  clusters has been presented. 
 %\CER{Définitivement, les paramètres réseaux variables ici se rapportent au réseau INTER cluster.}
 \section{Conclusion}
 The simulation of the execution of parallel asynchronous iterative algorithms on large scale  clusters has been presented. 

-In this work, we show that SimGrid is an efficient simulation tool that has enabled us to 
+In this work, we show that SimGrid is one of efficient simulation tool that has enabled us to 

 reach the following two objectives: 
 
 \begin{enumerate}
 reach the following two objectives: 
 
 \begin{enumerate}