+
+\begin{figure}
+ \centering
+ \includegraphics[scale=0.5]{fig/time_scenarios.eps}
+ \caption{The execution times of NAS benchmarks over different scenarios }
+ \label{fig:time_sen}
+\end{figure}
+
+The NAS parallel benchmarks are executed over these two platform
+ with different number of nodes, as in Table \ref{tab:sc}.
+The overall energy consumption of all the benchmarks solving the class D instance and
+using the proposed frequency selection algorithm is measured
+using the equation of the reduced energy consumption, equation
+(\ref{eq:energy}). This model uses the measured dynamic and static
+power values showed in Table \ref{table:grid5000}. The execution
+time is measured for all the benchmarks over these different scenarios.
+
+The energy consumptions and the execution times for all the benchmarks are
+presented in the plots \ref{fig:eng_sen} and \ref{fig:time_sen} respectively.
+
+For the majority of the benchmarks, the energy consumed while executing the NAS benchmarks over one site scenario
+for 16 and 32 nodes is lower than the energy consumed while using two sites.
+The long distance communications between the two distributed sites increase the idle time which leads to more static energy consumption.
+ The execution times of these benchmarks
+over one site with 16 and 32 nodes are also lower when compared to those of the two sites
+scenario.
+
+
+
+However, the execution times and the energy consumptions of EP and MG benchmarks, which have no or small communications, are not significantly affected
+ in both scenarios. Even when the number of nodes is doubled. On the other hand, the communications of the rest of the benchmarks increases when using long distance communications between two sites or increasing the number of computing nodes.
+
+\begin{figure}
+ \centering
+ \includegraphics[scale=0.5]{fig/eng_s.eps}
+ \caption{The energy saving of NAS benchmarks over different scenarios }
+ \label{fig:eng_s}
+\end{figure}
+
+
+\begin{figure}
+ \centering
+ \includegraphics[scale=0.5]{fig/per_d.eps}
+ \caption{The performance degradation of NAS benchmarks over different scenarios }
+ \label{fig:per_d}
+\end{figure}
+
+
+\begin{figure}
+ \centering
+ \includegraphics[scale=0.5]{fig/dist.eps}
+ \caption{The tradeoff distance of NAS benchmarks over different scenarios }
+ \label{fig:dist}
+\end{figure}
+
+The energy saving percentage is computed as the ratio between the reduced
+energy consumption, equation (\ref{eq:energy}), and the original energy consumption,
+equation (\ref{eq:eorginal}), for all benchmarks as in figure \ref{fig:eng_s}.
+This figure shows that the energy saving percentages of one site scenario for
+16 and 32 nodes are bigger than those of the two sites scenario which is due
+to the higher computations to communications ratio in the first scenario
+than in the second one. Moreover, the frequency selecting algorithm selects smaller frequencies when the computations times are higher than the communication times which
+results in a lower energy consumption. Indeed, the dynamic consumed power
+is exponentially related to the CPU's frequency value. On the other side, the increase in the number of computing nodes can
+increase the communication times and thus produces less energy saving depending on the
+benchmarks being executed. The results of the benchmarks CG, MG, BT and FT show more
+energy saving percentage in one site scenario when executed over 16 nodes comparing to 32 nodes. While, LU and SP consume more energy with 16 nodes than 32 in one site because there computations to
+communications ratio is not affected by the increase of the number of local communications.
+
+
+While all benchmarks are effected by the long distance communications in the two sites
+scenarios, except EP benchmarks. In EP benchmark there is no communications
+in their iterations, then it is independent from the effect of local and long
+distance communications. Therefore, the energy saving percentage of this benchmarks is
+depend on differences between the computing powers of the computing nodes, for example
+in the one site scenario, the graphite cluster is selected but in the two sits scenario
+this cluster is replaced with Taurus cluster that be more powerful in computing power.
+Therefore, the energy saving of EP benchmarks are bigger in the two site scenario due
+to increase in the differences between the computing powers of the nodes. This means, the higher
+differences between the nodes' computing powers make the proposed frequencies selecting
+algorithm to selects smaller frequencies in the nodes of the higher computing power,
+producing less energy consumption and thus more energy saving.
+The best energy saving percentage was for one site scenario with 16 nodes, on average it
+saves the energy consumption up to 30\%.
+
+Figure \ref{fig:per_d}, presents the performance degradation percentages for all benchmarks.
+It shows that the performance degradation percentages of the one site scenario with
+32 nodes, on average equal to 10\%, is higher than the performance degradation of one 16 nodes,
+which on average equal to 3\%. This because selecting smaller frequencies in the one site scenarios,
+when the computations grater than the communications , increase the number of the critical nodes
+when the number of nodes increased. The inverse happens in the tow sites scenario,
+this due to the lower computations to communications ratio that decreased with highest
+communications. Therefore, the number of the critical nodes are decreased. The average performance
+degradation for the two sites scenario with 16 nodes is equal to 8\% and for 32 nodes is equal to 4\%.
+The EP benchmarks is gives the bigger performance degradation ratio, because there is no
+communications and no slack times in this benchmarks that is always their performance effected
+by selecting big or small frequencies.
+The tradeoff between these scenarios can be computed as in the trade-off function \ref{eq:max}.
+Figure \ref{fig:dist}, presents the tradeoff distance for all benchmarks over all
+platform scenarios. The one site scenario with 16 and 32 nodes had the best tradeoff distance
+compared to the two sites scenarios, because the increase in the communications as mentioned before.
+The one site scenario with 16 nodes is the best scenario in term of energy and performance tradeoff,
+which on average is up 26\%. Then, the tradeoff distance is related linearly to the energy saving
+percentage. Finally, the best energy and performance tradeoff depends on the increase in all of:
+1) the computations to communications ratio, 2) the differences in computing powers
+between the computing nodes and 3) the differences in static and the dynamic powers of the nodes.
