new

[mpi-energy2.git] / mpi-energy2-extension / Heter_paper.tex

diff --git a/mpi-energy2-extension/Heter_paper.tex b/mpi-energy2-extension/Heter_paper.tex
index caa294f079fa8d0df9d0bd7c99606e1c44716840..6de325c679123bc04ef8b526fd1a796fafbdd40f 100644 (file)
--- a/mpi-energy2-extension/Heter_paper.tex
+++ b/mpi-energy2-extension/Heter_paper.tex
@@ -1,11 +1,49 @@
-\documentclass[3p,times]{elsarticle-1}
-\usepackage{ecrc}
+\documentclass[review]{elsarticle}
+
+\usepackage{lineno,hyperref}
+\modulolinenumbers[5]
+
+\journal{Journal of Computational Science}
+
+%%%%%%%%%%%%%%%%%%%%%%%
+%% Elsevier bibliography styles
+%%%%%%%%%%%%%%%%%%%%%%%
+%% To change the style, put a % in front of the second line of the current style and
+%% remove the % from the second line of the style you would like to use.
+%%%%%%%%%%%%%%%%%%%%%%%
+
+%% Numbered
+%\bibliographystyle{model1-num-names}
+
+%% Numbered without titles
+%\bibliographystyle{model1a-num-names}
+
+%% Harvard
+%\bibliographystyle{model2-names.bst}\biboptions{authoryear}
+
+%% Vancouver numbered
+%\usepackage{numcompress}\bibliographystyle{model3-num-names}
+
+%% Vancouver name/year
+%\usepackage{numcompress}\bibliographystyle{model4-names}\biboptions{authoryear}
+
+%% APA style
+%\bibliographystyle{model5-names}\biboptions{authoryear}
+
+%% AMA style
+%\usepackage{numcompress}\bibliographystyle{model6-num-names}
+
+%% `Elsevier LaTeX' style
+\bibliographystyle{elsarticle-num}
+%%%%%%%%%%%%%%%%%%%%%%%
+

 \usepackage[T1]{fontenc}
 \usepackage[utf8]{inputenc}
 \usepackage[english]{babel}
 \usepackage{algpseudocode}
 \usepackage{graphicx}
 \usepackage{algorithm}
 \usepackage[T1]{fontenc}
 \usepackage[utf8]{inputenc}
 \usepackage[english]{babel}
 \usepackage{algpseudocode}
 \usepackage{graphicx}
 \usepackage{algorithm}

+\usepackage{setspace}

 \usepackage{subfig}
 \usepackage{amsmath}
 \usepackage{url}
 \usepackage{subfig}
 \usepackage{amsmath}
 \usepackage{url}


@@ -61,122 +99,20 @@
 \newcommand{\Told}{\Xsub{T}{Old}}
 
 
 \newcommand{\Told}{\Xsub{T}{Old}}
 
 

-%% The ecrc package defines commands needed for running heads and logos.
-%% For running heads, you can set the journal name, the volume, the starting page and the authors
-
-%% set the volume if you know. Otherwise `00'
-\volume{00}
-
-%% set the starting page if not 1
-\firstpage{1}
-
-%% Give the name of the journal
-\journalname{Procedia Computer Science}
-
-%% Give the author list to appear in the running head
-%% Example \runauth{C.V. Radhakrishnan et al.}
-\runauth{}
-
-%% The choice of journal logo is determined by the \jid and \jnltitlelogo commands.
-%% A user-supplied logo with the name <\jid>logo.pdf will be inserted if present.
-%% e.g. if \jid{yspmi} the system will look for a file yspmilogo.pdf
-%% Otherwise the content of \jnltitlelogo will be set between horizontal lines as a default logo
-
-%% Give the abbreviation of the Journal.
-\jid{procs}
-
-%% Give a short journal name for the dummy logo (if needed)
-\jnltitlelogo{Procedia Computer Science}
-
-%% Hereafter the template follows `elsarticle'.
-%% For more details see the existing template files elsarticle-template-harv.tex and elsarticle-template-num.tex.
-
-%% Elsevier CRC generally uses a numbered reference style
-%% For this, the conventions of elsarticle-template-num.tex should be followed (included below)
-%% If using BibTeX, use the style file elsarticle-num.bst
-
-%% End of ecrc-specific commands
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-
-%% The amssymb package provides various useful mathematical symbols
-\usepackage{amssymb}
-%% The amsthm package provides extended theorem environments
-%% \usepackage{amsthm}
-
-%% The lineno packages adds line numbers. Start line numbering with
-%% \begin{linenumbers}, end it with \end{linenumbers}. Or switch it on
-%% for the whole article with \linenumbers after \end{frontmatter}.
-%% \usepackage{lineno}
-
-%% natbib.sty is loaded by default. However, natbib options can be
-%% provided with \biboptions{...} command. Following options are
-%% valid:
-
-%%   round  -  round parentheses are used (default)
-%%   square -  square brackets are used   [option]
-%%   curly  -  curly braces are used      {option}
-%%   angle  -  angle brackets are used    <option>
-%%   semicolon  -  multiple citations separated by semi-colon
-%%   colon  - same as semicolon, an earlier confusion
-%%   comma  -  separated by comma
-%%   numbers-  selects numerical citations
-%%   super  -  numerical citations as superscripts
-%%   sort   -  sorts multiple citations according to order in ref. list
-%%   sort&compress   -  like sort, but also compresses numerical citations
-%%   compress - compresses without sorting
-%%
-%% \biboptions{comma,round}
-
-% \biboptions{}
-
-% if you have landscape tables
-\usepackage[figuresright]{rotating}
-
-% put your own definitions here:
-%   \newcommand{\cZ}{\cal{Z}}
-%   \newtheorem{def}{Definition}[section]
-%   ...
-
-% add words to TeX's hyphenation exception list
-%\hyphenation{author another created financial paper re-commend-ed Post-Script}

 
 

-% declarations for front matter

 
 \begin{document}
 
 \begin{frontmatter}
 
 
 \begin{document}
 
 \begin{frontmatter}
 

-%% Title, authors and addresses
-
-%% use the tnoteref command within \title for footnotes;
-%% use the tnotetext command for the associated footnote;
-%% use the fnref command within \author or \address for footnotes;
-%% use the fntext command for the associated footnote;
-%% use the corref command within \author for corresponding author footnotes;
-%% use the cortext command for the associated footnote;
-%% use the ead command for the email address,
-%% and the form \ead[url] for the home page:
-%%
-%% \title{Title\tnoteref{label1}}
-%% \tnotetext[label1]{}
-%% \author{Name\corref{cor1}\fnref{label2}}
-%% \ead{email address}
-%% \ead[url]{home page}
-%% \fntext[label2]{}
-%% \cortext[cor1]{}
-%% \address{Address\fnref{label3}}
-%% \fntext[label3]{}
-
-\dochead{}
-%% Use \dochead if there is an article header, e.g. \dochead{Short communication}
-\title{Energy Consumption Reduction with DVFS for Message Passing \\
-       Iterative Applications on Grid Architecture} 
+
+
+\title{Energy Consumption Reduction with DVFS for Message \\
+         Passing Iterative Applications on \\
+                    Grid Architecture} 

   
 
   
 

-%% use optional labels to link authors explicitly to addresses:
-%% \author[label1,label2]{<author name>}
-%% \address[label1]{<address>}
-%% \address[label2]{<address>}
+

 
 \author{Ahmed Fanfakh,
         Jean-Claude Charr,
 
 \author{Ahmed Fanfakh,
         Jean-Claude Charr,


@@ -288,11 +224,9 @@ consumption while minimizing the degradation of the program's performance.
 Section~\ref{sec.optim} details the proposed frequencies selecting algorithm.
 Section~\ref{sec.expe} presents the results of applying the algorithm on the 
 NAS parallel benchmarks and executing them on the grid'5000 testbed. 
 Section~\ref{sec.optim} details the proposed frequencies selecting algorithm.
 Section~\ref{sec.expe} presents the results of applying the algorithm on the 
 NAS parallel benchmarks and executing them on the grid'5000 testbed. 

-%It shows the results of running different scenarios using multi-cores and one core per node and comparing them. 
-It also evaluates the algorithm over three different power scenarios. Moreover, it shows the
+It also evaluates the algorithm over multi-cores per node architectures and over three different power scenarios. Moreover, it shows the

 comparison results between the proposed method and an existing method.  Finally,
 in Section~\ref{sec.concl} the paper ends with a summary and some future works.
 comparison results between the proposed method and an existing method.  Finally,
 in Section~\ref{sec.concl} the paper ends with a summary and some future works.

-

 \section{Related works}
 \label{sec.relwork}
 
 \section{Related works}
 \label{sec.relwork}
 


@@ -511,11 +445,11 @@ follows:
 \end{equation}
 Replacing $\Fnew$ in (\ref{eq:pd}) as in (\ref{eq:fnew}) gives the following
 equation for dynamic power consumption:
 \end{equation}
 Replacing $\Fnew$ in (\ref{eq:pd}) as in (\ref{eq:fnew}) gives the following
 equation for dynamic power consumption:

-\begin{equation}
+\begin{multline}

   \label{eq:pdnew}
   \label{eq:pdnew}

-   \PdNew = \alpha \cdot \CL \cdot V^2 \cdot \Fnew = \alpha \cdot \CL \cdot \beta^2 \cdot \Fnew^3 
-    = \alpha \cdot \CL \cdot V^2 \cdot \Fmax \cdot S^{-3} = \PdOld \cdot S^{-3}
-\end{equation}
+   \PdNew = \alpha \cdot \CL \cdot V^2 \cdot \Fnew = \alpha \cdot \CL \cdot \beta^2 \cdot \Fnew^3 \\
+   {} = \alpha \cdot \CL \cdot V^2 \cdot \Fmax \cdot S^{-3} = \PdOld \cdot S^{-3}
+\end{multline}

 where $\PdNew$  and $\PdOld$ are the  dynamic power consumed with the
 new frequency and the maximum frequency respectively.
 
 where $\PdNew$  and $\PdOld$ are the  dynamic power consumed with the
 new frequency and the maximum frequency respectively.
 


@@ -561,13 +495,14 @@ of one iteration multiplied by the static power of each processor.  The overall
 energy consumption of a message passing distributed application executed over a
 heterogeneous grid platform during one iteration is the summation of all dynamic and
 static energies for $M$ processors in $N$ clusters.  It is computed as follows:
 energy consumption of a message passing distributed application executed over a
 heterogeneous grid platform during one iteration is the summation of all dynamic and
 static energies for $M$ processors in $N$ clusters.  It is computed as follows:

-\begin{equation}
+\begin{multline}

   \label{eq:energy}
  E = \sum_{i=1}^{N} \sum_{i=1}^{M} {(S_{ij}^{-2} \cdot \Pd[ij] \cdot  \Tcp[ij])} +  
   \label{eq:energy}
  E = \sum_{i=1}^{N} \sum_{i=1}^{M} {(S_{ij}^{-2} \cdot \Pd[ij] \cdot  \Tcp[ij])} +  

- \sum_{i=1}^{N} \sum_{j=1}^{M} (\Ps[ij] \cdot 
+ \sum_{i=1}^{N} \sum_{j=1}^{M} (\Ps[ij] \cdot {} \\

   (\mathop{\max_{i=1,\dots N}}_{j=1,\dots,M}({\Tcp[ij]} \cdot S_{ij}) 
   +\mathop{\min_{j=1,\dots M}} (\Tcm[hj]) ))
   (\mathop{\max_{i=1,\dots N}}_{j=1,\dots,M}({\Tcp[ij]} \cdot S_{ij}) 
   +\mathop{\min_{j=1,\dots M}} (\Tcm[hj]) ))

-\end{equation}
+\end{multline}
+

 
 Reducing the frequencies of the processors according to the vector of scaling
 factors $(S_{11}, S_{12},\dots, S_{NM})$ may degrade the performance of the application
 
 Reducing the frequencies of the processors according to the vector of scaling
 factors $(S_{11}, S_{12},\dots, S_{NM})$ may degrade the performance of the application


@@ -694,11 +629,13 @@ in~\cite{Zhuo_Energy.efficient.Dynamic.Task.Scheduling,Rauber_Analytical.Modelin
 \label{sec.optim}
 
 \begin{algorithm}
 \label{sec.optim}
 
 \begin{algorithm}

+\setstretch{1}

   \begin{algorithmic}[1]
     % \footnotesize
   \begin{algorithmic}[1]
     % \footnotesize

+    

     \Require ~
     \begin{description}
     \Require ~
     \begin{description}

-    \item [{$N$}] number of clusters in the grid.
+    \item [{$N$}] number of clusters in the grid. 

     \item [{$M$}] number of nodes in each cluster.
     \item[{$\Tcp[ij]$}] array of all computation times for all nodes during one iteration and with the highest frequency.
     \item[{$\Tcm[ij]$}] array of all communication times for all nodes during one iteration and with the highest frequency.
     \item [{$M$}] number of nodes in each cluster.
     \item[{$\Tcp[ij]$}] array of all computation times for all nodes during one iteration and with the highest frequency.
     \item[{$\Tcm[ij]$}] array of all communication times for all nodes during one iteration and with the highest frequency.


@@ -726,8 +663,7 @@ in~\cite{Zhuo_Energy.efficient.Dynamic.Task.Scheduling,Rauber_Analytical.Modelin
         \EndIf
        \State $\Tnew \gets $ computed as  in equations (\ref{eq:perf}). 
        \State $\Ereduced \gets $ computed as  in equations (\ref{eq:energy}). 
         \EndIf
        \State $\Tnew \gets $ computed as  in equations (\ref{eq:perf}). 
        \State $\Ereduced \gets $ computed as  in equations (\ref{eq:energy}). 

-       \State $\Pnorm \gets \frac{\Told}{\Tnew}$
-       \State $\Enorm\gets \frac{\Ereduced}{\Eoriginal}$
+       \State $\Pnorm \gets \frac{\Told}{\Tnew}$,  $\Enorm\gets \frac{\Ereduced}{\Eoriginal}$

       \If{$(\Pnorm - \Enorm > \Dist)$}
         \State $\Sopt[ij] \gets S_{ij},~i=1,\dots,N,~j=1,\dots,M_i. $
         \State $\Dist \gets \Pnorm - \Enorm$
       \If{$(\Pnorm - \Enorm > \Dist)$}
         \State $\Sopt[ij] \gets S_{ij},~i=1,\dots,N,~j=1,\dots,M_i. $
         \State $\Dist \gets \Pnorm - \Enorm$


@@ -923,20 +859,20 @@ The benchmarks have seven different classes, S, W, A, B, C, D and E, that repres
     Name        & model       & Freq. & Freq. & Freq. & per CPU         & of one core     \\
                 &             & GHz   & GHz   & GHz   &                 &           \\
     \hline
     Name        & model       & Freq. & Freq. & Freq. & per CPU         & of one core     \\
                 &             & GHz   & GHz   & GHz   &                 &           \\
     \hline

-    Taurus      & Intel       & 2.3  & 1.2  & 0.1     & 6               & \np[W]{35} \\
-                & Xeon        &       &       &       &                 &            \\
+                & Intel       & 2.3  & 1.2  & 0.1     & 6               & \np[W]{35} \\
+    Taurus      & Xeon        &       &       &       &                 &            \\

                 & E5-2630     &       &       &       &                 &            \\         
     \hline
                 & E5-2630     &       &       &       &                 &            \\         
     \hline

-    Graphene    & Intel       & 2.53  & 1.2   & 0.133 & 4               & \np[W]{23} \\
-                & Xeon        &       &       &       &                 &            \\
+                & Intel       & 2.53  & 1.2   & 0.133 & 4               & \np[W]{23} \\
+    Graphene    & Xeon        &       &       &       &                 &            \\

                 & X3440       &       &       &       &                 &            \\    
     \hline
                 & X3440       &       &       &       &                 &            \\    
     \hline

-    Griffon     & Intel       & 2.5   & 2     & 0.5   & 4               & \np[W]{46} \\
-                & Xeon        &       &       &       &                 &            \\
+                & Intel       & 2.5   & 2     & 0.5   & 4               & \np[W]{46} \\
+    Griffon     & Xeon        &       &       &       &                 &            \\

                 & L5420       &       &       &       &                 &            \\  
     \hline
                 & L5420       &       &       &       &                 &            \\  
     \hline

-    Graphite    & Intel       & 2     & 1.2   & 0.1   & 8               & \np[W]{35} \\
-                & Xeon        &       &       &       &                 &            \\
+                & Intel       & 2     & 1.2   & 0.1   & 8               & \np[W]{35} \\
+     Graphite   & Xeon        &       &       &       &                 &            \\

                 & E5-2650     &       &       &       &                 &            \\  
     \hline
   \end{tabular}
                 & E5-2650     &       &       &       &                 &            \\  
     \hline
   \end{tabular}


@@ -1042,26 +978,15 @@ energy saving percentage in one site scenario when executed over 16 nodes compar
   \centering
   \subfloat[The energy consumption by the nodes wile executing the NAS benchmarks over different scenarios    
            ]{%
   \centering
   \subfloat[The energy consumption by the nodes wile executing the NAS benchmarks over different scenarios    
            ]{%

-    \includegraphics[width=.4\textwidth]{fig/eng_con_scenarios.eps}\label{fig:eng_sen}} \hspace{1cm}%
+    \includegraphics[width=.48\textwidth]{fig/eng_con_scenarios.eps}\label{fig:eng_sen}} \hspace{0.4cm}%

   \subfloat[The execution times of the NAS benchmarks over different scenarios]{%
   \subfloat[The execution times of the NAS benchmarks over different scenarios]{%

-    \includegraphics[width=.4\textwidth]{fig/time_scenarios.eps}\label{fig:time_sen}}
+    \includegraphics[width=.48\textwidth]{fig/time_scenarios.eps}\label{fig:time_sen}}

   \label{fig:exp-time-energy}
   \caption{The  energy consumption and execution time of NAS  Benchmarks over different scenarios}
 \end{figure}
 
 
   \label{fig:exp-time-energy}
   \caption{The  energy consumption and execution time of NAS  Benchmarks over different scenarios}
 \end{figure}
 
 

-\begin{figure}
-  \centering
-  \subfloat[The energy reduction while executing the NAS benchmarks over different scenarios ]{%
-    \includegraphics[width=.4\textwidth]{fig/eng_s.eps}\label{fig:eng_s}} \hspace{2cm}%
-  \subfloat[The performance degradation of the NAS benchmarks over different scenarios]{%
-    \includegraphics[width=.4\textwidth]{fig/per_d.eps}\label{fig:per_d}}\hspace{2cm}%
-    \subfloat[The tradeoff distance between the energy reduction and the performance of the NAS benchmarks  
-      over different scenarios]{%
-    \includegraphics[width=.4\textwidth]{fig/dist.eps}\label{fig:dist}}
-  \label{fig:exp-res}
-  \caption{The experimental results of different scenarios}
-\end{figure}
+

 
 The energy saving percentage is reduced for all the benchmarks because of the long distance communications in the two sites 
 scenario, except for the   EP benchmark which has  no communications. Therefore, the energy saving percentage of this benchmark is 
 
 The energy saving percentage is reduced for all the benchmarks because of the long distance communications in the two sites 
 scenario, except for the   EP benchmark which has  no communications. Therefore, the energy saving percentage of this benchmark is 


@@ -1076,7 +1001,18 @@ algorithm  select smaller frequencies for the powerful nodes which
 produces less energy consumption and thus more energy saving.
 The best energy saving percentage was obtained in the one site scenario with 16 nodes, the energy consumption was on average reduced up to 30\%.
 
 produces less energy consumption and thus more energy saving.
 The best energy saving percentage was obtained in the one site scenario with 16 nodes, the energy consumption was on average reduced up to 30\%.
 

-
+\begin{figure}
+  \centering
+  \subfloat[The energy reduction while executing the NAS benchmarks over different scenarios ]{%
+    \includegraphics[width=.48\textwidth]{fig/eng_s.eps}\label{fig:eng_s}} \hspace{0.4cm}%
+  \subfloat[The performance degradation of the NAS benchmarks over different scenarios]{%
+    \includegraphics[width=.48\textwidth]{fig/per_d.eps}\label{fig:per_d}}\hspace{0.4cm}%
+    \subfloat[The tradeoff distance between the energy reduction and the performance of the NAS benchmarks  
+      over different scenarios]{%
+    \includegraphics[width=.48\textwidth]{fig/dist.eps}\label{fig:dist}}
+  \label{fig:exp-res}
+  \caption{The experimental results of different scenarios}
+\end{figure}

 Figure \ref{fig:per_d} presents the performance degradation percentages for all benchmarks over the two scenarios.
 The performance degradation percentage for the benchmarks running on two sites  with
 16 or 32  nodes is on average equal to 8.3\% or 4.7\% respectively. 
 Figure \ref{fig:per_d} presents the performance degradation percentages for all benchmarks over the two scenarios.
 The performance degradation percentage for the benchmarks running on two sites  with
 16 or 32  nodes is on average equal to 8.3\% or 4.7\% respectively. 


@@ -1139,9 +1075,9 @@ Scenario name                          & Cluster name & \begin{tabular}[c]{@{}c@
 \begin{figure}
   \centering
   \subfloat[Comparing the  execution times of running NAS benchmarks over one core and multicores scenarios]{%
 \begin{figure}
   \centering
   \subfloat[Comparing the  execution times of running NAS benchmarks over one core and multicores scenarios]{%

-    \includegraphics[width=.4\textwidth]{fig/time.eps}\label{fig:time-mc}} \hspace{1cm}%
+    \includegraphics[width=.48\textwidth]{fig/time.eps}\label{fig:time-mc}} \hspace{0.4cm}%

   \subfloat[Comparing the  energy consumptions of running NAS benchmarks over one core and multi-cores scenarios]{%
   \subfloat[Comparing the  energy consumptions of running NAS benchmarks over one core and multi-cores scenarios]{%

-    \includegraphics[width=.4\textwidth]{fig/eng_con.eps}\label{fig:eng-cons-mc}}
+    \includegraphics[width=.48\textwidth]{fig/eng_con.eps}\label{fig:eng-cons-mc}}

     \label{fig:eng-cons}
   \caption{The energy consumptions and execution times of NAS benchmarks over one core and multi-cores per node architectures}
 \end{figure}
     \label{fig:eng-cons}
   \caption{The energy consumptions and execution times of NAS benchmarks over one core and multi-cores per node architectures}
 \end{figure}


@@ -1179,12 +1115,12 @@ in the figure \ref{fig:dist-mc}. These  tradeoff distance between energy consump
 \begin{figure}
   \centering
     \subfloat[The energy saving of running NAS benchmarks over one core and multicores scenarios]{%
 \begin{figure}
   \centering
     \subfloat[The energy saving of running NAS benchmarks over one core and multicores scenarios]{%

-    \includegraphics[width=.4\textwidth]{fig/eng_s_mc.eps}\label{fig:eng-s-mc}} \hspace{2cm}%
+    \includegraphics[width=.48\textwidth]{fig/eng_s_mc.eps}\label{fig:eng-s-mc}} \hspace{0.4cm}%

     \subfloat[The performance degradation of running NAS benchmarks over one core and multicores scenarios
       ]{%
     \subfloat[The performance degradation of running NAS benchmarks over one core and multicores scenarios
       ]{%

-    \includegraphics[width=.4\textwidth]{fig/per_d_mc.eps}\label{fig:per-d-mc}}\hspace{2cm}%
+    \includegraphics[width=.48\textwidth]{fig/per_d_mc.eps}\label{fig:per-d-mc}}\hspace{0.4cm}%

     \subfloat[The tradeoff distance of running NAS benchmarks over one core and multicores scenarios]{%
     \subfloat[The tradeoff distance of running NAS benchmarks over one core and multicores scenarios]{%

-    \includegraphics[width=.4\textwidth]{fig/dist_mc.eps}\label{fig:dist-mc}}
+    \includegraphics[width=.48\textwidth]{fig/dist_mc.eps}\label{fig:dist-mc}}

   \label{fig:exp-res}
   \caption{The experimental results of one core and multi-cores scenarios}
 \end{figure}
   \label{fig:exp-res}
   \caption{The experimental results of one core and multi-cores scenarios}
 \end{figure}


@@ -1205,12 +1141,12 @@ In these experiments, the class D of the NAS parallel benchmarks are executed ov
 \begin{figure}
   \centering
   \subfloat[The energy saving percentages for the nodes executing the NAS benchmarks over the three power scenarios]{%
 \begin{figure}
   \centering
   \subfloat[The energy saving percentages for the nodes executing the NAS benchmarks over the three power scenarios]{%

-    \includegraphics[width=.4\textwidth]{fig/eng_pow.eps}\label{fig:eng-pow}} \hspace{2cm}%
+    \includegraphics[width=.48\textwidth]{fig/eng_pow.eps}\label{fig:eng-pow}} \hspace{0.4cm}%

   \subfloat[The performance degradation percentages for the NAS benchmarks over the three power scenarios]{%
   \subfloat[The performance degradation percentages for the NAS benchmarks over the three power scenarios]{%

-    \includegraphics[width=.4\textwidth]{fig/per_pow.eps}\label{fig:per-pow}}\hspace{2cm}%
+    \includegraphics[width=.48\textwidth]{fig/per_pow.eps}\label{fig:per-pow}}\hspace{0.4cm}%

     \subfloat[The tradeoff distance between the energy reduction and the performance of the NAS benchmarks over the three power scenarios]{%
       
     \subfloat[The tradeoff distance between the energy reduction and the performance of the NAS benchmarks over the three power scenarios]{%
       

-    \includegraphics[width=.4\textwidth]{fig/dist_pow.eps}\label{fig:dist-pow}}
+    \includegraphics[width=.48\textwidth]{fig/dist_pow.eps}\label{fig:dist-pow}}

   \label{fig:exp-pow}
   \caption{The experimental results of different static power scenarios}
 \end{figure}
   \label{fig:exp-pow}
   \caption{The experimental results of different static power scenarios}
 \end{figure}


@@ -1270,11 +1206,11 @@ presented in the figures \ref{fig:edp-eng}, \ref{fig:edp-perf} and \ref{fig:edp-
 \begin{figure}
   \centering
   \subfloat[The energy reduction induced by the Maxdist method and the EDP method]{%
 \begin{figure}
   \centering
   \subfloat[The energy reduction induced by the Maxdist method and the EDP method]{%

-    \includegraphics[width=.4\textwidth]{fig/edp_eng}\label{fig:edp-eng}} \hspace{2cm}%
+    \includegraphics[width=.48\textwidth]{fig/edp_eng}\label{fig:edp-eng}} \hspace{0.4cm}%

     \subfloat[The performance degradation induced by  the Maxdist method and the EDP method]{%
     \subfloat[The performance degradation induced by  the Maxdist method and the EDP method]{%

-    \includegraphics[width=.4\textwidth]{fig/edp_per}\label{fig:edp-perf}}\hspace{2cm}%
+    \includegraphics[width=.48\textwidth]{fig/edp_per}\label{fig:edp-perf}}\hspace{0.4cm}%

     \subfloat[The tradeoff distance between the energy consumption reduction and the performance for the Maxdist method and the  EDP method]{%
     \subfloat[The tradeoff distance between the energy consumption reduction and the performance for the Maxdist method and the  EDP method]{%

-    \includegraphics[width=.4\textwidth]{fig/edp_dist}\label{fig:edp-dist}}
+    \includegraphics[width=.48\textwidth]{fig/edp_dist}\label{fig:edp-dist}}

   \label{fig:edp-comparison}
   \caption{The comparison results}
 \end{figure}
   \label{fig:edp-comparison}
   \caption{The comparison results}
 \end{figure}


@@ -1304,7 +1240,7 @@ To evaluate the proposed method on a real heterogeneous grid platform, it was ap
  NAS parallel benchmarks   and the  class D instance was executed over the  grid'5000 testbed platform. 
  The experimental results showed that the algorithm reduces  on average 30\% of the energy consumption
 for all the NAS benchmarks   while  only degrading by 3.2\% on average  the performance. 
  NAS parallel benchmarks   and the  class D instance was executed over the  grid'5000 testbed platform. 
  The experimental results showed that the algorithm reduces  on average 30\% of the energy consumption
 for all the NAS benchmarks   while  only degrading by 3.2\% on average  the performance. 

-The Maxdist algorithm was also evaluated in different scenarios that vary in the distribution of the computing nodes between different clusters' sites or \textcolor{blue}{between using one core and multi-cores per node} or in the values of the consumed static power. The algorithm selects different vector of frequencies according to the 
+The Maxdist algorithm was also evaluated in different scenarios that vary in the distribution of the computing nodes between different clusters' sites or  use multi-cores per node architecture or consume different static power values. The algorithm selects different vector of frequencies according to the 

 computations and communication times ratios, and  the values of the static and measured dynamic powers of the CPUs. 
 Finally, the proposed algorithm was compared to another method that uses
 the well known energy and delay product as an objective function. The comparison results showed 
 computations and communication times ratios, and  the values of the static and measured dynamic powers of the CPUs. 
 Finally, the proposed algorithm was compared to another method that uses
 the well known energy and delay product as an objective function. The comparison results showed 


@@ -1326,8 +1262,7 @@ This work  has been  partially supported by  the Labex ACTION  project (contract
 Mr. Ahmed  Fanfakh, would  like to  thank the University  of Babylon  (Iraq) for
 supporting his work.
 
 Mr. Ahmed  Fanfakh, would  like to  thank the University  of Babylon  (Iraq) for
 supporting his work.
 

-
-\bibliographystyle{elsarticle-num}
+%\section*{References}

 \bibliography{my_reference}
 
 \end{document}
 \bibliography{my_reference}
 
 \end{document}