thanks to mesocentre

[mpi-energy2.git] / mpi-energy2-extension / review / review.tex

diff --git a/mpi-energy2-extension/review/review.tex b/mpi-energy2-extension/review/review.tex
index 4e5eef6248bdc2c1c8f6033e3b70b8f5a64be09f..a7253f6a3a83755128b9c2d12761182c3514dc0b 100644 (file)
--- a/mpi-energy2-extension/review/review.tex
+++ b/mpi-energy2-extension/review/review.tex
@@ -54,6 +54,9 @@
 
 \title{Answers to the questions of the reviewers}
 \maketitle
 
 \title{Answers to the questions of the reviewers}
 \maketitle

+
+We would like to thank the reviewers for taking time to review our paper. Their remarks were very constructive and allowed us to improve our paper and clarify some ambiguous points. We took in consideration all the remarks of the reviewers and modified the paper accordingly. In the following sections, the reviewers can find our answers to their questions:
+

 \section{Questions and remarks of the first reviewer} 
 
 \begin{enumerate}
 \section{Questions and remarks of the first reviewer} 
 
 \begin{enumerate}


@@ -85,7 +88,7 @@ machines for example. In this paper experiments, only 16 and 32 nodes
 where considered.
 
 \textbf{Answer:}
 where considered.
 
 \textbf{Answer:}

-We agree with the reviewer that the algorithm is centralized and might be a bottleneck if it was applied to an application running on many thousands of nodes. However, up to 144 nodes in a heterogeneous cluster, the overhead of the algorithm was very small, 0.15 ms, as presented in the simulation results of [6]. We did not execute experiments with more than 32 nodes on Grid'5000 because it does not have many nodes that allow DVFS operations and have energy measurement tools. 
+We agree with the reviewer that the algorithm is centralized and might be a bottleneck if it was applied to an application running on many thousands of nodes. However, up to 144 nodes in a heterogeneous cluster, the overhead of the algorithm was very small, 0.15 ms, as presented in the simulation results of \cite{5}. We did not execute experiments with more than 32 nodes on Grid'5000 because it does not have many nodes that allow DVFS operations and have energy measurement tools. 

 
 On the other hand, the scalability of the proposed algorithm can be improved if we use asynchronous computations or if the algorithm was distributed in a hierarchical manner where a leader is chosen for each cluster or a group of nodes  to compute their scaled frequencies. Improving the scalability of the algorithm is beyond the scope of this paper.
 
 
 On the other hand, the scalability of the proposed algorithm can be improved if we use asynchronous computations or if the algorithm was distributed in a hierarchical manner where a leader is chosen for each cluster or a group of nodes  to compute their scaled frequencies. Improving the scalability of the algorithm is beyond the scope of this paper.
 


@@ -184,126 +187,143 @@ For the variability issue, please refer to the answer to question 1.
 \section{Questions and remarks of the second reviewer} 
 
 \begin{enumerate}
 \section{Questions and remarks of the second reviewer} 
 
 \begin{enumerate}

-\item Move the contributions from related work to introduction
+\item Move the contributions from related work to introduction.

 
 

-\item why emphasize it is a grid platform? the presentation of related work follows the logic of heterogeneous CPUs. Grid is only a type of platform with heterogeneous CPUs
+\textbf{Answer:} The contributions were moved to the introduction section.

 
 

-\textcolor{blue}{Answer: The proposed algorithm was adapted to the grid platform which is composed 
-of homogeneous clusters and interconnected by a wide area network which is slower than the local 
-network in each cluster. The algorithm can also work on other heterogeneous platforms.} 
+\item Why emphasize it is a grid platform? the presentation of related work follows the logic of heterogeneous CPUs. Grid is only a type of platform with heterogeneous CPUs.

 
 

+\textbf{Answer:} We agree with the reviewer that a grid is a type of heterogeneous architecture and the proposed algorithm can also work on any heterogeneous architecture. 

 
 

-\item Define what iterative message passing applications are and give exemplar applications of them targeted by this method. 
+  In \cite{5}, we have proposed a frequency selection algorithm for distributed applications running on heterogeneous clusters, while in this work,  the proposed algorithm was adapted to the grid architecture which is composed 
+of homogeneous clusters interconnected by a wide area network which is slower than the local 
+network in each cluster. 

 
 

-\textcolor{blue}{Answer: The message passing applications with iterations compute
-the same block of operations several times, starting from the initial solution until reaching
-the acceptable approximation of the exact solution (we have added this sentence to the paper page 21). 
-There are many example for these applications 
-such as JACOBI, GAUSS-SEIDE, Successive over-relaxation and  conjugate gradient (CG) method and etc.
-Similarly, an offline method uses DVFS and applied to these applications is in \cite{1} } 
+\item Define what iterative message passing applications are and give exemplar applications of them targeted by this method. 

 
 

+\textbf{Answer:} In order to clarify things, we have replaced in the paper the sentence ``the iterative message passing applications'' with ``the message passing applications with iterations''. Therefore, the proposed algorithm can be applied to any application that executes the same block of instructions many times and it is not limited to iterative methods that repeat the same block of instructions until convergence. 

 
 

-\item Figure 1 is not clearly explained. Where is the slack time in figure 1 and why slack time =0 for task 1?
+Many problems are solved by methods with iterations such as  solving a linear system of equations with Jacobi and Gauss-Seidel methods, Image processing with methods that apply the same instructions for each pixel or block of pixels, etc. 

 
 

-\textcolor{blue}{Answer: We agree with the reviewer, this figure was re-plotted to show the slack time. 
- In the figure, we assumed that task 1 is the slower task. So, there are no slack time (waiting time)
- in the slower task because it is not wait for the others while other tasks wait for it.}

 
 

-\item define the parameters in eq. 1.
+\item Figure 1 is not clearly explained. Where is the slack time in figure 1 and why slack time =0 for task 1?

 
 

-\textcolor{blue}{Answer: We have defined Fmax and Fnew in the text.}

 
 

-\item eq. 2: are you assuming each cluster has the same number of nodes?
+\textbf{Answer:} Figure 1 was redrawn, the white   space before the barrier is the slack time. Slack times occur when a node has to wait for another node to finish its computation to synchronously communicate with it.  In Figure 1, task 1 was assumed to be the slowest task. All the other tasks will finish their computations before the slowest task and wait until it finishes its computation before being able to synchronously communicate with it. This waiting time is the slack time and since the slowest task do not have to wait for the other tasks it has almost no slack time.

 
 

+\item Define the parameters in eq. 1.

 
 

+\textbf{Answer:} Fmax and Fnew have been defined as follows in the paper: ``$\Fmax$ is the maximum frequency before applying any DVFS and $\Fnew$  is the new frequency after applying DVFS''. 

 
 

+\item Eq. 2: are you assuming each cluster has the same number of nodes?

 
 

-\textcolor{blue}{Answer: No, we assume each cluster has different number of node, so in the equation we
-have replaced $M_i$ instead of $M$ and the same for $F$ replaced with $F_j$ in the all equations of the paper.}
+\textbf{Answer:}  No, each cluster can have a different number of nodes. Therefore, in the paper, $M$, the number of nodes in a cluster, was replaced by  $M_i$, the number of nodes in cluster $i$, in all the equations.  

 
 \item Eq.2 implicitly assumes that there is no overlapping between computation and communication. Is it reasonable?
 
 
 \item Eq.2 implicitly assumes that there is no overlapping between computation and communication. Is it reasonable?
 

-\textcolor{blue}{Answer: The communications between the computing nodes are synchronized, where each node 
-need to wait for the others to finished their jobs before computing the next iteration. So, there is no
-overlapping between computations and communications for a node. The overlapping happens when the communications are asynchronous or the computations are not depend on the data sent by the neighbouring nodes. }
+\textbf{Answer:} In this paper, only   message passing applications with synchronous and blocking communications were considered. In such applications, there is no overlapping between computations and communications on a node. 

 
 

-\item eq. 2 is not clear: 
+Asynchronous applications are beyond the scope of this paper and will be considered in a future work. 

 
 

--how to define and determine the slowest cluster h? the one before scaling or after scaling? 
+\item Eq. 2 is not clear: 
+\begin{enumerate}
+\item How to define and determine the slowest cluster h? the one before scaling or after scaling? 

 
 

-\textcolor{blue}{Answer: The slower task is the task which gives maximum execution time before scaling the frequency of the node. We have added this sentence to the paper (page 8).}
+\textbf{Answer:}  The slowest node $h$ is the node which takes the  maximum execution time to execute an iteration  before scaling down its  frequency. The previous sentence has been added to the paper.

  
 
  
 

-- what is the communication time without slack time
+\item What is the communication time without slack time?
+
+\textbf{Answer:} There is no synchronous communications with zero slack times, but if a node sends a message to another node who is already waiting for that message. The latter will acknowledge the reception of the message from the sender without any delay. On the other hand, if the receiving  node is still computing the sender has to wait for it to finish its computation to acknowledge the reception of the message. This time is called the slack time. 
+
+
+\item In equation, min operation is used to get the communication time, but in text, it says to use the slowest communication time, which should use the max operation then.

 
 

-\textcolor{blue}{Answer: There is no synchronous communications with zero slack times, but if a node send a message to another node which is already waiting for that message. The latter will acknowledge the reception of the message from the sender without any delay. On the other hand, if the receiving  node is still computing the sender has to wait for it to finish its computation to acknowledge the reception of the message. This time is called the slack time. }
+\textbf{Answer:} We agree with the reviewer and the sentence  "the slowest communication time" has been changed to "the communication time of the slowest node" in the paper.

 
 

+\end{enumerate}

 
 

-- in equation, min operation is used to get the communication time, but in text, it says to use the slowest communication time, which should use the max operation then.

 
 

-\textcolor{blue}{Answer: We agree with the reviewer and the sentence  "slower communication time" changed to "communication time of the slower node" in the paper.}
+\item Discuss the difference between eq. 2 and the prediction model in references \cite{4} and \cite{5}.

 
 

-\item discuss the difference between eq. 2 and the prediction model in references [5] and [6]
+\textbf{Answer:}  The prediction models  in \cite{4} and \cite{5} are for homogeneous and heterogeneous clusters respectively, while the model in Equation 2 is adapted for  grids. We have adapted the prediction models to the used architecture. Each architecture has its own characteristics. For example, in a homogeneous cluster all the nodes have the same specifications and only one scaling factor is computed by the algorithm to all the nodes of the cluster. 
+On the other hand,  in a  heterogeneous cluster, the nodes may have different specifications and a scaling factor should be computed to each node. The prediction models of a heterogeneous cluster can be used for a homogeneous cluster. In the same the models in this paper take more characteristics into considerations such as different networks to be adapted for grids and they can also be applied to a heterogeneous cluster.  Therefore, the  models presented in this paper are more complete than those presented in  \cite{4} and \cite{5} and take more characteristics into consideration. 

 
 

-\textcolor{blue}{Answer: The prediction models  in [5] and [6] are for homogeneous and heterogeneous clusters respectively, while eq. 2 is for a grid. where the homogeneous cluster predication model was used one scaling factor denoted as $S$, because all the nodes in the cluster have the same computing powers. Whereas, in heterogeneous cluster prediction model all the nodes have different scales and the scaling factors have denoted as one dimensional vector $(S_1, S_2, \dots, S_N)$. The execution time prediction model for a grid  Equation (2) defines a two dimensional array of scales  
-$(S_{11}, S_{12},\dots, S_{NM_i})$. We have added this to the paper (page 8).}

 
 \item Eq. 10: Can the authors comment on the energy consumed by communications?
 
 
 \item Eq. 10: Can the authors comment on the energy consumed by communications?
 

-\textcolor{blue}{Answer: The CPU during  communications consumed only the static power power. While
-in computations the CPU consumes both the dynamic and static communication, refer to \cite{3}. We have added this sentience to the paper, page 11.}
+\textbf{Answer:} During communications, the CPU only consumes the static power power and during computations it consumes both  dynamic and static power. For more information the reviewer can refer to \cite{3}. 

 
 \item This work assume homogeneous cpu in one cluster. Line 55 says: even if the distributed message
 passing iterative application is load balanced, the computation time of each cpu j in cluster i may be different  Why?
 
 
 \item This work assume homogeneous cpu in one cluster. Line 55 says: even if the distributed message
 passing iterative application is load balanced, the computation time of each cpu j in cluster i may be different  Why?
 

-\textcolor{blue}{Answer: The computation times may be slightly different due to the delay caused
-by the scheduler of the operating system. We have added this in the paper.}
+\textbf{Answer:} In a homogeneous cluster executing a load balanced distributed  application,  the computation time of each node might be slightly different than the others due to some delay caused
+by the scheduler of the operating system of the node.

 
 \item Comment why the applications in NAS parallel benchmark are iterative application? These applications are normally run in one cluster. Describe in more detail how they are run across multiple clusters. 
 
 
 \item Comment why the applications in NAS parallel benchmark are iterative application? These applications are normally run in one cluster. Describe in more detail how they are run across multiple clusters. 
 

-\textcolor{blue}{Answer: The applications in NAS parallel benchmark are  application with iterations  because they iterate the same block of instructions (communications and computations) many times. All the benchmarks are MPI programs that allowed to be executed on any distributed memory platform such as clusters and grids with no required modifications. Since, we have deployed the same operating system on all the nodes, we just compile the source on one cluster and then copied the executable program on all the clusters. }
+\textbf{Answer:} The sentence ``iterative applications'' was replaced by ``applications with iterations'' because the proposed algorithm can be applied to any application that executes the same block of instructions many times and it is not limited to iterative methods that terminate when they converge.  The NAS parallel benchmarks are  application with iterations  because they iterate the same block of instructions until convergence or for fixed number of iterations. These benchmarks can be executed on any distributed memory platform such as clusters or grids with no required modifications. Since, we have deployed the same operating system on all the nodes, we just compile the source on one node and then copy the executable program on all the nodes. The application can then be executed with an ``mpirun'' command that takes three arguments:
+\begin{itemize}
+\item the name of the application to execute
+\item the number of processes required to execute the application 
+\item the architecture file that contains the names of the nodes that will execute the application. They could be from different clusters.

 
 

-\item broken sentence in line 28 on page 12
+\end{itemize}
+\item Broken sentence in line 28 on page 12.

 
 

-\textcolor{blue}{Answer: The word "were" replaced with "where".}
+\textbf{Answer:} The sentence was corrected.

 
 \item Why $T_{old}$ is computed using eq. 12, which applies MAX over computation time and communication time, while in  $T_{new}$, max and min operations are applied over computation and communication separately?
 
 
 \item Why $T_{old}$ is computed using eq. 12, which applies MAX over computation time and communication time, while in  $T_{new}$, max and min operations are applied over computation and communication separately?
 

-\textcolor{blue}{Answer:  We  agree with the reviewer, $T_{old}$ is the maximum execution time of the application before scaling the frequency and it is computed as in $T_{new}$ equation without scaling factors. So, we have changed the $T_{old}$ in the paper as as follows:
+\textbf{Answer:} Both forms can be used for computing $T_{old}$ and $T_{new}$. To avoid this confusion, the same form was used for both equations in the paper.
+

 \begin{equation}
   \label{eq:told}
    T_{old} = \mathop{\max_{i=1,2,\dots,N}}_{j=1,2,\dots,M_i} (\Tcp[ij]) + 
 \begin{equation}
   \label{eq:told}
    T_{old} = \mathop{\max_{i=1,2,\dots,N}}_{j=1,2,\dots,M_i} (\Tcp[ij]) + 

-   \mathop{\min_{i=1,2,\dots,N}} (\Tcm[hj] )           
+   \mathop{\min_{j=1,2,\dots,M_h}} (\Tcm[hj] )           

 \end{equation}
 \end{equation}

-}
+where $h$ is the index of the slowest cluster. 
+
+   

 
 \item Line 55 on page 16 is to define the slack time, which should be introduced at the beginning of the paper, such as in figure 1.
 
 
 \item Line 55 on page 16 is to define the slack time, which should be introduced at the beginning of the paper, such as in figure 1.
 

-\textcolor{blue}{Answer: We have changed it in the paper and added to page 6.}
+\textbf{Answer:} We agree with the reviewer and the slack time is now presented at the beginning of the paper.

 
 
 \item Authors comment whether (and how) the proposed methods can be applied/extended to other programming models and/or platform, such as mapreduce, heterogeneous cluster with CPU+GPU.
 
 
 \item Authors comment whether (and how) the proposed methods can be applied/extended to other programming models and/or platform, such as mapreduce, heterogeneous cluster with CPU+GPU.

-Revision 

 
 

-\textcolor{blue}{Answer: The proposed method can only be applied to parallel programming with iteration  
-and with or without message passing. Indeed, the proposed method can be applied to the parallel application  with mapreduce if it is a regular application with iterations. Therefore,  the time of each map and reduce operations (communications) and the computation times in the program must be computed at the first iterations to predict the energy consumption and the execution time. After, the proposed algorithm can be used as it to select the best frequencies. 
-The proposed method can be applied to a heterogeneous platform composed from GPUs and CPUs, since modern GPUs like CPUs allow the use of DVFS operation.}
+\textbf{Answer:}  The proposed method can only be applied to parallel models with iterations  
+and with or without message passing. If only a few map and reduce operations are executed in the application and these operations are not iterative, the proposed algorithm cannot be adapted to that type of applications. On the other hand, if the  map or reduce  operations are iterative, the proposed algorithm can be applied  when executing these operations. Finally, if in the application, the same map and reduce operations are executed many times iteratively, the proposed algorithm can then be applied to the whole application while considering that an iteration consists of a map operation followed by a reduce operation.  
+
+The proposed method with some adaptations can be applied to applications with iterations running on heterogeneous platforms composed of GPUs and CPUs because modern GPUs like CPUs allow the use of DVFS.

 \end{enumerate}
 
 \section{Questions and remarks of the third reviewer}
 \begin{enumerate}
 
 \end{enumerate}
 
 \section{Questions and remarks of the third reviewer}
 \begin{enumerate}
 

-\item suggest the authors to use much larger size of nodes, instead of on 16 nodes, distributed on three clusters, to see the scalability of the energy saving
+\item Suggest the authors to use much larger size of nodes, instead of on 16 nodes, distributed on three clusters, to see the scalability of the energy saving
+
+\textbf{Answer:} The experiments were not only conducted over 16  nodes, but they were also executed  over 32 nodes distributed over three clusters. 
+In \cite{5} the algorithm was evaluated  on a simulated heterogeneous cluster composed of up to 144 nodes. The overhead of the algorithm was very small, just 0.15 ms.  
+
+ The experiments were not conducted on more than 32 nodes of Grid'5000 because it does not have many nodes that allow DVFS operations and have energy measurement tools. We agree with the reviewer that  experiments using much more nodes should be conducted to evaluate the scalability of the proposed algorithm and when we will have access to such platforms,  we will evaluate the proposed method over a larger number of nodes.
+
+\item  The energy saving is actually calculated by the quantitative formula instead of the real measurements.  Can you have any  discussions on the real measurements?
+
+\textbf{Answer:}  This paper does not focus on measuring the energy consumption of CPUs in a grid. It presents models to predict the energy consumption and the performance of an application with iterations running on a grid. These models use the given  dynamic and static powers to predict the energy consumption of each CPU with different scaling factors. Moreover, since we do not have physical access to the nodes of the grid which are geographically distributed on many sites in France, we cannot use hardware tools to measure  the consumption of CPUs. Therefore, we used  Grid'5000's  tool which measures the overall power consumption of a node in real-time. These values were used to deduce the dynamic power of the node when computing with the maximum frequency. 
+ 
+ As a future work, it would be  interesting  to compare the accuracy of the results of the proposed energy model to the values given by instruments  that measure the energy consumptions of CPUs during the execution time, as in \cite{2}.
+
+\item The overhead is not measured, can you present something on this as well to demonstrate what the authors claimed "has a small overhead and works without training or profiling"?

 
 

-\textcolor{blue}{Answer: We have made the experiments not only on 16  nodes, but we have also made them  over 32 nodes distributed over three clusters and in the near future we will apply the proposed method over a larger number of nodes.}
+\textbf{Answer:} In the comparison section 6.5, we have presented the execution time of the algorithm when it is executed over 32 nodes from three clusters and  located in two different sites.  It takes on average 0.01 $ms$. In \cite{5} the algorithm was evaluated  on a simulated heterogeneous cluster composed of up to 144 nodes. The overhead of the algorithm was  just 0.15 ms.  

 
 

-\item  the energy saving is actually calculated by the quantitative formula instead of the real measurements.  Can you have any  discussions on the real measurements?

 
 

-\textcolor{blue}{Answer: The scope of this paper is not mainly focuses on the energy measurements, but it focuses on modelling and optimizing the energy and performance of grid systems. The proposed energy model depends on the dynamic and static power values for each CPU. We have used a real power measurement tools allowed in Grid'5000 sites to measure the dynamic power consumption. Moreover, the real measurements are difficult  for a grid platform when the nodes are geographically distributed. As a future work, it is interesting  to compare the accuracy of the proposed energy model with a real instruments  to measure the energy consumption for local clusters such as the measurement tools presented in \cite{2}.}
+The algorithm works online without profiling means it only uses the measured communication and computation times during the run-time and do not require to profile the application before run-time. Some methods use profilers before executing the application to gather a lot of information about the application such as computations to communication ratios and dependencies between tasks. The gathered information is used to scale the frequency of each node before executing the application.  

 
 

-\item the overhead is not measured, can you present something on this as well to demonstrate what the authors claimed "has a small overhead and works without training or profiling"?
+The algorithm works without training because it does not require the partial or the total execution of the application before run-time. Indeed, some applications run parts of the application while using various frequencies to measure in advance their energy consumption. Then, using these values they select the the frequency of each node before executing the application.  

 
 

-\textcolor{blue}{Answer: In the comparison section 6.5, we have presented the execution time of the algorithm when it is executed over 32 nodes distributed over three sites located at two different sites, it takes on average 0.01 $ms$. The algorithm works online without training which means it only uses the measured communication and computation times during the runtime and do not require any profiling or training executed before runtime.}

 \end{enumerate}
 \bibliographystyle{plain}
 \bibliography{ref.bib}
 \end{enumerate}
 \bibliographystyle{plain}
 \bibliography{ref.bib}