-The proposed algorithm was applied to seven MPI programs of the NAS benchmarks (EP, CG, MG, FT, BT, LU and SP) NPB v3.3
-\cite{44}, which were run with three classes (A, B and C).
-In this experiments we are interested to run the class C, the biggest class compared to A and B, on different number of
-nodes, from 4 to 128 or 144 nodes according to the type of the iterative MPI program. Depending on the proposed energy consumption model EQ(\ref{eq:energy}),
- we are measure the energy consumption for all NAS MPI programs. The dynamic and static power values are used under the same assumption used by \cite{45,3}, we are used a percentage of 80\% for dynamic power and 20\% for static of the total power consumption of a CPU. The heterogeneous nodes in table (\ref{table:platform}) have different simulated computing power (FLOPS), ranked from the node of type 1 with smaller computing power (FLOPS) to the highest computing power (FLOPS) for node of type 4. Therefore, the power values are used proportionally increased from nodes of type 1 to nodes of type 4 that with highest computing power. Then, we are used an assumption that the power consumption is increased linearly when the computing power (FLOPS) of the processor is increased, see \cite{48}.
-
-\begin{table}[htb]
- \caption{Running NAS benchmarks on 4 nodes }
- % title of Table
- \centering
- \begin{tabular}{|*{7}{l|}}
- \hline
- Method & Execution & Energy & Energy & Performance & Distance \\
- name & time/s & consumption/J & saving\% & degradation\% & \\
- \hline
- CG & 64.64 & 3560.39 &34.16 &6.72 &27.44 \\
- \hline
- MG & 18.89 & 1074.87 &35.37 &4.34 &31.03 \\
- \hline
- EP &79.73 &5521.04 &26.83 &3.04 &23.79 \\
- \hline
- LU &308.65 &21126.00 &34.00 &6.16 &27.84 \\
- \hline
- BT &360.12 &21505.55 &35.36 &8.49 &26.87 \\
- \hline
- SP &234.24 &13572.16 &35.22 &5.70 &29.52 \\
- \hline
- FT &81.58 &4151.48 &35.58 &0.99 &34.59 \\
-\hline
- \end{tabular}
- \label{table:res_4n}
-\end{table}
+Nodes from distinct clusters in a grid have different computing powers, thus
+while executing message passing iterative synchronous applications, fast nodes
+have to wait for the slower ones to finish their computations before being able
+to synchronously communicate with them as in Figure~\ref{fig:heter}. These
+periods are called idle or slack times. The algorithm takes into account this
+problem and tries to reduce these slack times when selecting the vector of the frequency
+scaling factors. At first, it selects initial frequency scaling factors
+that increase the execution times of fast nodes and minimize the differences
+between the computation times of fast and slow nodes. The value of the initial
+frequency scaling factor for each node is inversely proportional to its
+computation time that was gathered from the first iteration. These initial
+frequency scaling factors are computed as a ratio between the computation time
+of the slowest node and the computation time of the node $i$ as follows:
+\begin{equation}
+ \label{eq:Scp}
+ \Scp[ij] = \frac{ \mathop{\max_{i=1,\dots N}}_{j=1,\dots,M}(\Tcp[ij])} {\Tcp[ij]}
+\end{equation}
+Using the initial frequency scaling factors computed in (\ref{eq:Scp}), the
+algorithm computes the initial frequencies for all nodes as a ratio between the
+maximum frequency of node $i$ and the computation scaling factor $\Scp[i]$ as
+follows:
+\begin{equation}
+ \label{eq:Fint}
+ F_{ij} = \frac{\Fmax[ij]}{\Scp[ij]},~{i=1,2,\dots,N},~{j=1,\dots,M}
+\end{equation}
+If the computed initial frequency for a node is not available in the gears of
+that node, it is replaced by the nearest available frequency. In
+Figure~\ref{fig:st_freq}, the nodes are sorted by their computing powers in
+ascending order and the frequencies of the faster nodes are scaled down
+according to the computed initial frequency scaling factors. The resulting new
+frequencies are highlighted in Figure~\ref{fig:st_freq}. This set of
+frequencies can be considered as a higher bound for the search space of the
+optimal vector of frequencies because selecting higher frequencies
+than the higher bound will not improve the performance of the application and it
+will increase its overall energy consumption. Therefore the algorithm that
+selects the frequency scaling factors starts the search method from these
+initial frequencies and takes a downward search direction toward lower
+frequencies until reaching the nodes' minimum frequencies or lower bounds. A node's frequency is considered its lower bound if the computed distance between the energy and performance at this frequency is less than zero.
+A negative distance means that the performance degradation ratio is higher than the energy saving ratio.
+In this situation, the algorithm must stop the downward search because it has reached the lower bound and it is useless to test the lower frequencies. Indeed, they will all give worse distances.
+
+Therefore, the algorithm iterates on all remaining frequencies, from the higher
+bound until all nodes reach their minimum frequencies or their lower bounds, to compute the overall
+energy consumption and performance and selects the optimal vector of the frequency scaling
+factors. At each iteration the algorithm determines the slowest node
+according to the equation (\ref{eq:perf}) and keeps its frequency unchanged,
+while it lowers the frequency of all other nodes by one gear. The new overall
+energy consumption and execution time are computed according to the new scaling
+factors. The optimal set of frequency scaling factors is the set that gives the
+highest distance according to the objective function (\ref{eq:max}).
+
+Figures~\ref{fig:r1} and \ref{fig:r2} illustrate the normalized performance and
+consumed energy for an application running on a homogeneous cluster and a
+ grid platform respectively while increasing the scaling factors. It can
+be noticed that in a homogeneous cluster the search for the optimal scaling
+factor should start from the maximum frequency because the performance and the
+consumed energy decrease from the beginning of the plot. On the other hand, in
+the grid platform the performance is maintained at the beginning of the
+plot even if the frequencies of the faster nodes decrease until the computing
+power of scaled down nodes are lower than the slowest node. In other words,
+until they reach the higher bound. It can also be noticed that the higher the
+difference between the faster nodes and the slower nodes is, the bigger the
+maximum distance between the energy curve and the performance curve is, which results in bigger energy savings.
