Energy Consumption Reduction with DVFS for Message Passing Iterative Applications on

Heterogeneous Architectures

Jean-Claude Charr, Raphaël Couturier, Ahmed Fanfakh and Arnaud Giersch

FEMTO-ST Institute, University of Franche-Comté

IUT de Belfort-Montbéliard, 19 avenue du Maréchal Juin, BP 527, 90016 Belfort cedex, France Email: {jean-claude.charr,raphael.couturier,ahmed.fanfakh_badri_muslim,arnaud.giersch}@univ-fcomte.fr

Figure 1: Parallel tasks on a heterogeneous platform

Abstract—

I. Introduction

II. Related works

III. The performance and energy consumption measurements on heterogeneous architecture

A. The execution time of message passing distributed iterative applications on a heterogeneous platform

In this paper, we are interested in reducing the energy consumption of message passing distributed iterative synchronous applications running over heterogeneous grid platforms. A heterogeneous grid platform is defined as a collection of heterogeneous computing clusters interconnected via a long distance network (the internet network). Each computing cluster in the grid composed from homogeneous nodes, where are connected together via high speed homogeneous network. Therefore, each cluster has different characteristics such as computing power (FLOPS), energy consumption, CPU’s frequency range, network bandwidth and latency.

The overall execution time of a distributed iterative synchronous application over a heterogeneous grid consists of the sum of the computation time and the communication time for every iteration on a node. However, due to the heterogeneous computation power of the computing clusters, slack times may occur when fast nodes have to wait, during synchronous communications, for the slower nodes to finish their computations (see Figure 1). Therefore, the overall execution time of the program is the execution time of the slowest task which has the highest computation time and no slack time.

Dynamic Voltage and Frequency Scaling (DVFS) is a process, implemented in modern processors, that reduces the energy consumption of a CPU by scaling down its voltage and frequency. Since DVFS lowers the frequency of a CPU and consequently its computing power, the execution time of a program running over that scaled down processor may increase, especially if the program is compute bound. The frequency reduction process can be expressed by the scaling factor S which is the ratio between the maximum and the new frequency of a CPU as in (1).

S = \frac{F_{max}}{F_{new}}  (1)

The execution time of a compute bound sequential program is linearly proportional to the frequency scaling factor S. On the other hand, message passing distributed applications consist of two parts: computation and communication. The execution time of the computation part is linearly proportional to the frequency scaling factor S but the communication time is not affected by the scaling factor because the processors involved remain idle during the communications [1]. The communication time for a task is the summation of periods of time that begin with an MPI call for sending or receiving a message until the message is synchronously sent or received.

Since in a heterogeneous grid each cluster has different characteristics, especially different frequency gears, when applying DVFS operations on the nodes of these clusters, they may get different scaling factors represented by a scaling vector: (S_{11}, S_{12}, …, S_{NM}) where S_{ij} is the scaling factor of processor j in cluster i . To be able to predict the execution time of message passing synchronous iterative applications running over a heterogeneous grid, for different vectors of scaling factors, the communication time and the computation time for all the tasks must be measured during the first iteration before applying any DVFS operation. Then the execution time for one iteration of the application with any vector of scaling factors can be predicted using (2).

T_{New} = max_{i=1,...N}_{j=1,...,M}(T_{cpOld}_{ij} · S_{ij}) + min_{i=1,...N}_{j=1,...,M}(T_{cm}_{ij}) (2)

where N is the number of the clusters in the grid, M is the number of the nodes in each cluster, T_{cpOld}_{ij} and T_{cm}_{ij} is the computation and communication times of processor i in the cluster j during the first iteration. The model computes the maximum computation time with scaling factor from each node added to the communication time of the slowest node. It means only the communication time without any slack time is taken into account. Therefore, the execution time of the iterative application is equal to the execution time of one iteration as in (2) multiplied by the number of iterations of that application.

This prediction model is developed from the model to predict the execution time of message passing distributed applications for homogeneous and heterogeneous clusters [2], [3]. The execution time prediction model is used in the method to optimize both the energy consumption and the performance of iterative methods, which is presented in the following sections.

B. Energy model for heterogeneous platform

Many researchers [4], [5], [6], [7] divide the power consumed by a processor into two power metrics: the static and the dynamic power. While the first one is consumed as long as the computing unit is turned on, the latter is only consumed during computation times. The dynamic power P_{d} is related to the switching activity α, load capacitance C_{L}, the supply voltage V and operational frequency F, as shown in (3).

P_{d} = α · C_{L} · V^{2} · F  (3)

The static power P_{s} captures the leakage power as follows:

P_{s} = V · N_{trans} · K_{design} · I_{leak}  (4)

where V is the supply voltage, N_{trans} is the number of transistors, K_{design} is a design dependent parameter and I_{leak} is a technology dependent parameter. The energy consumed by an individual processor to execute a given program can be computed as:

E_{ind} = P_{d} · T_{cp} + P_{s} · T  (5)

where T is the execution time of the program, T_{cp} is the computation time and T_{cp} ≤ T. T_{cp} may be equal to T if there is no communication and no slack time.

The main objective of DVFS operation is to reduce the overall energy consumption [8]. The operational frequency F depends linearly on the supply voltage V, i.e., V = β · F with some constant β. This equation is used to study the change of the dynamic voltage with respect to various frequency values in [5]. The reduction process of the frequency can be expressed by the scaling factor S which is the ratio between the maximum and the new frequency as in (1). The CPU governors are power schemes supplied by the operating system’s kernel to lower a core’s frequency. The new frequency F_{new} from (1) can be calculated as follows:

F_{new} = S^{−1} · F_{max}  (6)

Replacing F_{new} in (3) as in (6) gives the following equation for dynamic power consumption:

P_{dNew} = α · C_{L} · V^{2} · F_{new} = α · C_{L} · β^{2} · F_{new}^{3} = α · C_{L} · V^{2} · F_{max} · S^{−3} = P_{dOld} · S^{−3} (7)

where P_{dNew} and P_{dOld} are the dynamic power consumed with the new frequency and the maximum frequency respectively.

According to (7) the dynamic power is reduced by a factor of S^{−3} when reducing the frequency by a factor of S [5]. Since the FLOPS of a CPU is proportional to the frequency of a CPU, the computation time is increased proportionally to S. The new dynamic energy is the dynamic power multiplied by the new time of computation and is given by the following equation:

E_{dNew} = P_{dOld} · S^{−3} · (T_{cp} · S) = S^{−2} · P_{dOld} · T_{cp} (8)

The static power is related to the power leakage of the CPU and is consumed during computation and even when idle. As in [5], [6], the static power of a processor is considered as constant during idle and computation periods, and for all its available frequencies. The static energy is the static power multiplied by the execution time of the program. According to the execution time model in (2), the execution time of the program is the sum of the computation and the communication times. The computation time is linearly related to the frequency scaling factor, while this scaling factor does not affect the communication time. The static energy of a processor after scaling its frequency is computed as follows:

E_{S} = P_{s} · (T_{cp} · S + T_{cm} )  (9)

In the considered heterogeneous grid platform, each node j in cluster i may have different dynamic and static powers from the nodes of the other clusters, noted as P_{d}_{ij} and P_{s}_{ij} respectively. Therefore, even if the distributed message passing iterative application is load balanced, the computation time of each CPU j in cluster i noted T_{cp}_{ij} may be different and different frequency scaling factors may be computed in order to decrease the overall energy consumption of the application and reduce slack times. The communication time of a processor j in cluster i is noted as T_{cm}_{ij} and could contain slack times when communicating with slower nodes, see Figure 1. Therefore, all nodes do not have equal communication times. While the dynamic energy is computed according to the frequency scaling factor and the dynamic power of each node as in (8), the static energy is computed as the sum of the execution time of one iteration multiplied by the static power of each processor. The overall energy consumption of a message passing distributed application executed over a heterogeneous grid platform during one iteration is the summation of all dynamic and static energies for M processors in N clusters. It is computed as follows:

E = ∑_{i=1}^{N} ∑_{i=1}^{M} (S_{ij}^{−2} · P_{d}_{ij} · T_{cp}_{ij}) + ∑_{i=1}^{N} ∑_{j=1}^{M}(P_{s}_{ij} ·

( max_{i=1,...N}_{j=1,...,M}(T_{cp}_{ij} · S_{ij}) + min_{i=1,...N}_{j=1,...,M}(T_{cm}_{ij}))) (10)

Reducing the frequencies of the processors according to the vector of scaling factors (S_{11}, S_{12}, …, S_{NM}) may degrade the performance of the application and thus, increase the static energy because the execution time is increased [9]. The overall energy consumption for the iterative application can be measured by measuring the energy consumption for one iteration as in (10) multiplied by the number of iterations of that application.

IV. Optimization of both energy consumption and performance

Using the lowest frequency for each processor does not necessarily give the most energy efficient execution of an application. Indeed, even though the dynamic power is reduced while scaling down the frequency of a processor, its computation power is proportionally decreased. Hence, the execution time might be drastically increased and during that time, dynamic and static powers are being consumed.

Therefore, it might cancel any gains achieved by scaling down the frequency of all nodes to the minimum and the overall energy consumption of the application might not be the optimal one. It is not trivial to select the appropriate frequency scaling factor for each processor while considering the characteristics of each processor (computation power, range of frequencies, dynamic and static powers) and the task executed (computation/communication ratio). The aim being to reduce the overall energy consumption and to avoid increasing significantly the execution time. In our previous work [2], [3], we proposed a method that selects the optimal frequency scaling factor for a homogeneous and heterogeneous clusters executing a message passing iterative synchronous application while giving the best trade-off between the energy consumption and the performance for such applications. In this work we are interested in heterogeneous grid as described above. Due to the heterogeneity of the processors, a vector of scaling factors should be selected and it must give the best trade-off between energy consumption and performance.

The relation between the energy consumption and the execution time for an application is complex and nonlinear, Thus, unlike the relation between the execution time and the scaling factor, the relation between the energy and the frequency scaling factors is nonlinear, for more details refer to [1]. Moreover, these relations are not measured using the same metric. To solve this problem, the execution time is normalized by computing the ratio between the new execution time (after scaling down the frequencies of some processors) and the initial one (with maximum frequency for all nodes) as follows:

P_{Norm} = \frac{T_{New}}{T_{Old}}  (11)

Where T new is computed as in (2) and T old is computed as in (12)

T_{Old} = max_{i=1,2,...,N}_{j=1,2,...,M}(T_{cp}_{ij} + T_{cm}_{ij})  (12)

In the same way, the energy is normalized by computing the ratio between the consumed energy while scaling down the frequency and the consumed energy with maximum frequency for all nodes:

E_{Norm} = \frac{E_{Reduced}}{E_{Original}}  (13)

Where E_{Reduced} is computed using (10) and E_{Original} is computed as in ().

E_{Original} = ∑_{i=1}^{N} ∑_{j=1}^{M}(P_{d}_{ij} ·T_{cp}_{ij})+∑_{i=1}^{N} ∑_{j=1}^{M}(P_{s}_{ij} ·T_{Old}) (14)

While the main goal is to optimize the energy and execution time at the same time, the normalized energy and execution time curves do not evolve (increase/decrease) in the same way. According to the equations (11) and (13), the vector of frequency scaling factors S_{1}, S_{2}, …, S_{N} reduce both the energy and the execution time simultaneously. But the main objective is to produce maximum energy reduction with minimum execution time reduction.

(a) Homogeneous cluster

(b) Heterogeneous grid

Figure 2: The energy and performance relation

This problem can be solved by making the optimization process for energy and execution time follow the same evolution according to the vector of scaling factors (S_{11}, S_{12}, …, S_{NM}). Therefore, the equation of the normalized execution time is inverted which gives the normalized performance equation, as follows:

P_{Norm} = \frac{T_{Old}}{T_{New}}  (15)

Then, the objective function can be modeled in order to find the maximum distance between the energy curve (13) and the performance curve (15) over all available sets of scaling factors. This represents the minimum energy consumption with minimum execution time (maximum performance) at the same time, see Figure 2a or Figure 2b. Then the objective function has the following form:

MaxDist = max_{i=1,...N}_{j=1,...,M}_{k=1,...,F}(\overbrace{P_{Norm}(S_{ijk})}^{Maximize} − \overbrace{E_{Norm}(S_{ijk})}^{Minimize}) (16)

where N is the number of clusters, M is the number of nodes in each cluster and F is the number of available frequencies for each node. Then, the optimal set of scaling factors that satisfies (16) can be selected. The objective function can work with any energy model or any power values for each node (static and dynamic powers). However, the most important energy reduction gain can be achieved when the energy curve has a convex form as shown in [6], [5], [10].

V. The scaling factors selection algorithm for heterogeneous platforms

A. The algorithm details

In this section, Algorithm 1 is presented. It selects the frequency scaling factors vector that gives the best trade-off between minimizing the energy consumption and maximizing Algorithm 1 Scaling factors selection algorithm Require:

N number of clusters in the grid. M number of nodes in each cluster. T_{cp}_{ij} array of all computation times for all nodes

during one iteration and with the highest frequency.

T_{cm}_{ij} array of all communication times for all nodes during one iteration and with the highest frequency.

F_{max}_{ij} array of the maximum frequencies for all

nodes. P_{d}_{ij} array of the dynamic powers for all nodes. P_{s}_{ij} array of the static powers for all nodes. F_{diff}_{ij} array of the differences between two succes-

sive frequencies for all nodes.

Ensure: S_{opt}_{11}, S_{opt}_{12} …, S_{opt}_{NM_{i}}, a vector of scaling factors that gives the optimal tradeoff between energy consumption and execution time

1: S_{cp}_{ij} ← \frac{max_{i=1,2,...,N} (max_{j=1,2,...,M_{i}} (T_{cp}_{ij}))}{T_{cp}_{ij}}

2: F_{ij} ← \begin{matrix}F_{max} & _{ij} \\ S_{cp} & _{i} \end{matrix} , i = 1, 2, ⋯, N, j = 1, 2, …, M_{i}.

3: Round the computed initial frequencies F_{i} to the closest

available frequency for each node. 4: if (not the first frequency) then 5: F_{ij} ← F_{ij} + F_{diff}_{ij}, i = 1, …, N, j = 1, …, M_{i}. 6: end if 7: T_{Old} ← computed as in equations (12). 8: E_{Original} ← computed as in equations (14) . 9: S_{opt}_{ij} ← 1, i = 1, …, N, j = 1, …, M_{i}. 10: Dist ← 0 11: while (all nodes have not reached their minimum

frequency or P_{Norm} − E_{Norm} < 0) do

12: if (not the last freq. and not the slowest node) then 13: F_{ij} ← F_{ij} −F_{diff}_{ij}, i = 1, …, N, j = 1, …, M_{i}.

14: S_{ij} ← \frac{F_{max}_{ij}}{F_{ij}} , i = 1, …, N, j = 1, …, M_{i}. 15: end if 16: T_{New} ← computed as in equations (2). 17: E_{Reduced} ← computed as in equations (10). 18: P_{Norm} ← \frac{T_{Old}}{T_{New}} 19: E_{Norm} ← \frac{E_{Reduced}}{E_{Original}} 20: if (P_{Norm} − E_{Norm} > Dist) then 21: S_{opt}_{ij} ← S_{ij}, i = 1, …, N, j = 1, …, M_{i}. 22: Dist ← P_{Norm} − E_{Norm} 23: end if 24: end while 25: Return S_{opt}_{11}, S_{opt}_{12}, …, S_{opt}_{NM_{i}} Algorithm 2 DVFS algorithm 1: for k = 1 to some iterations do 2: Computations section. 3: Communications section. 4: if (k = 1) then 5: Gather all times of computation and

communication from each node.

6: Call Algorithm 1. 7: Compute the new frequencies from the

returned optimal scaling factors.

8: Set the new frequencies to nodes. 9: end if 10: end for

Figure 3: Selecting the initial frequencies

the performance of a message passing synchronous iterative application executed on a heterogeneous grid platform. It works online during the execution time of the iterative message passing program. It uses information gathered during the first iteration such as the computation time and the communication time in one iteration for each node. The algorithm is executed after the first iteration and returns a vector of optimal frequency scaling factors that satisfies the objective function (16). The program applies DVFS operations to change the frequencies of the CPUs according to the computed scaling factors. This algorithm is called just once during the execution of the program. Algorithm 2 shows where and when the proposed scaling algorithm is called in the iterative MPI program.

The nodes in a heterogeneous grid have different computing powers, thus while executing message passing iterative synchronous applications, fast nodes have to wait for the slower ones to finish their computations before being able to synchronously communicate with them as in Figure 1. These periods are called idle or slack times. The algorithm takes into account this problem and tries to reduce these slack times when selecting the frequency scaling factors vector. At first, it selects initial frequency scaling factors that increase the execution times of fast nodes and minimize the differences between the computation times of fast and slow nodes. The value of the initial frequency scaling factor for each node is inversely proportional to its computation time that was gathered from the first iteration. These initial frequency scaling factors are computed as a ratio between the computation time of the slowest node and the computation time of the node i as follows:

S_{cp}_{ij} = \frac{max_{i=1,...N}_{j=1,...,M}(T_{cp}_{ij})}{T_{cp}_{ij}}  (17)

Using the initial frequency scaling factors computed in (17), the algorithm computes the initial frequencies for all nodes as a ratio between the maximum frequency of node i and the computation scaling factor S_{cp}_{i} as follows:

F_{ij} = \begin{matrix}F_{max} & ij \\ S_{cp} & ij \end{matrix} , i = 1, 2, …, N, j = 1, …, M  (18)

If the computed initial frequency for a node is not available in the gears of that node, it is replaced by the nearest available frequency. In Figure 3, the nodes are sorted by their computing power in ascending order and the frequencies of the faster nodes are scaled down according to the computed initial frequency scaling factors. The resulting new frequencies are highlighted in Figure 3. This set of frequencies can be considered as a higher bound for the search space of the optimal vector of frequencies because selecting scaling factors higher than the higher bound will not improve the performance of the application and it will increase its overall energy consumption. Therefore the algorithm that selects the frequency scaling factors starts the search method from these initial frequencies and takes a downward search direction toward lower frequencies or reaching to the lower bound. The lower bound is used to stop the algorithm search process when the new computed distance between the energy and performance is less than zero. The new negative distance is mean that the performance degradation ratio is higher than energy saving ratio. Therefore, the algorithm must stop the iterations before reaching to the end of the search space, the minimum frequencies, because the all the coming new distances are negative values. The algorithm iterates on all remaining frequencies, from the higher bound until all nodes reach their minimum frequencies or to the lower bound, to compute their overall energy consumption and performance, and select the optimal frequency scaling factors vector. At each iteration the algorithm determines the slowest node according to the equation (2) and keeps its frequency unchanged, while it lowers the frequency of all other nodes by one gear. The new overall energy consumption and execution time are computed according to the new scaling factors. The optimal set of frequency scaling factors is the set that gives the highest distance according to the objective function (16).

Figures 2a and 2b illustrate the normalized performance and consumed energy for an application running on a homogeneous platform and a heterogeneous grid platform respectively while increasing the scaling factors. It can be noticed that in a homogeneous platform the search for the optimal scaling factor should start from the maximum frequency because the performance and the consumed energy decrease from the beginning of the plot. On the other hand, in the heterogeneous grid platform the performance is maintained at the beginning of the plot even if the frequencies of the faster nodes decrease until the computing power of scaled down nodes are lower than the slowest node. In other words, until they reach the higher bound. It can also be noticed that the higher the difference between the faster nodes and the slower nodes is, the bigger the maximum distance between the energy curve and the performance curve is while the scaling factors are varying which results in bigger energy savings.

VI. Experimental results

A. Grid’5000 architature and power consumption

The grid’5000 is a large-scale testbed found in France [11]. The grid infrastructure consist of ten sites distributed over all France metropolitan regions. Each site in grid’5000 composed from number of heterogenous computing clusters, while each cluster includes collection from homogeneous nodes. In general, grid’5000 had one thousand of heterogeneous nodes and eight thousand of cores. All sites are connected together via special long distance network called RENATER, which is the French National Telecommunication Network for Technology.

Figure 4: The selected two sites of grid’5000

Table I: CPUs characteristics of the selected clusters

Cluster  CPU  Max  Min  Diff. no. of cores dynamic power
Name  model  Freq. Freq. Freq.  per node  of one core
GHz GHz  GHz
Taurus Intel 2.3 1.2 0.1 6 35 W

Xeon E5-2630

Graphene Intel 2.53 1.2 0.133 4 23 W

Xeon X3440

Griffon Intel 2.5 2 0.5 4 46 W

Xeon L5420

Graphite Intel 2 1.2 0.1 8 35 W

Xeon E5-2650

Whereas inside each site the clusters and their nodes are connected throw high speed local area networks. There are different types of local networks used such as Ethernet and Infiniband netwoks, which allowed different gigabits bandwidth and latencies. On the other hand, the nodes inside each cluster are homogeneous, while they are different from the nodes of the other clusters. Therefore, there are a wide diversity of processors in grid’5000, that mainly had different processors families such as Intel Xeon and AMD Opteron families.

In this paper we are interested to run NAS parallel v3.3 [12] over grid’5000. We are used seven benchmarks, CG, MG, EP, LU, BT, SP and FT. These benchmark used seven different types of classes. These classes are S, W, A, B, C, D, E, where S represents the smaller problem size that used by benchmark and E is represents the biggest class. In this work, class D is used in all the experiment that will showed in the coming sections. Moreover, the NAS parallel benchmarks have different computations and communications ratios, then the goal is appeared to study their energy consumption and their performance on real testbed such as grid’5000. In this work NAS benchmarks are executed over two sites, Lyon and Nancy sites, of grid’5000. These two sites had seven different types of computing clusters as in figure (4).

Four clusters from the two sites are selected in the experiments, one cluster from the Lyon site, Taurus cluster, and three clusters from the Nancy site where are Graphene, Griffon and Graphite. Each one of these clusters has homogeneous nodes inside, while they are different from the nodes of other cluster in many aspects such as: computing power, power consumption, available frequencies ranges and the network features, the bandwidth and the latencies. The table I show the details characteristics of these four clusters.

The grid’5000 testbed provided some monitoring and measurement features to captured the power consumption for each node in any cluster in the Lyon and Nancy sites. The power consumed for each node from the interested four clusters is measured. While the power consumed by any computing node is a collection of the power consumed by hard drive, mainboard, memory and node’s computing cores, for more detail refer to [?]. Therefore, the dynamic power consumed by one core is not allowed to measured alone. To overcome this problem, firstly, we measured the power consumed by one Figure 5: The power consumption of one core from Taurus cluster

node when it not make any computation, when the CPU is idle. The second step, we run EP benchmark, where there is no communication in this benchmarks, over one core of the desired node and capturing the power consumed by a node, this representing the peak power of the node with one core . The difference in the power consumed in peak state and idle state representing the dynamic power consumption of that core, for example see figure(5). We did this operation for all nodes type and their dynamic values are recorded in table I.

B. The experimental results of the scaling algorithm

C. The experimental results of multi-cores clusters

D. The results for different power consumption scenarios

E. The comparison of the proposed scaling algorithm

VII. Conclusion

Acknowledgment

This work has been partially supported by the Labex ACTION project (contract “ANR-11-LABX-01-01”). Computations have been performed on the supercomputer facilities of the Mésocentre de calcul de Franche-Comté. As a PhD student, Mr. Ahmed Fanfakh, would like to thank the University of Babylon (Iraq) for supporting his work.

References

[1] V. W. Freeh, F. Pan, N. Kappiah, D. K. Lowenthal, and R. Springer, “Exploring the energy-time tradeoff in MPI programs on a powerscalable cluster,” in Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS’05) - Papers - Volume 01, ser. IPDPS ’05. Washington, DC, USA: IEEE Computer Society, 2005, pp. 4a–4a.

[2] J.-C. Charr, R. Couturier, A. Fanfakh, and A. Giersch, “Dynamic frequency scaling for energy consumption reduction in distributed MPI programs,” in ISPA 2014: 12th IEEE International Symposium on Parallel and Distributed Processing with Applications. Milan, Italy: IEEE Computer Society, Aug. 2014, pp. 225–230.

[3] J.-C. Charr, R. A. Couturier, A. Fanfakh, and A. Giersch, “Energy consumption reduction with dvfs for message passing iterative applications on heterogeneous architectures,” in PDSEC 2015, 16th IEEE Int. Workshop on Parallel and Distributed Scientific and Engineering Computing (in conjuction with IPDPS 2015), Hyderabad, India, May 2015.

[4] K. Malkowski, “Co-adapting scientific applications and architectures toward energy-efficient high performance computing,” Ph.D. dissertation, The Pennsylvania State University, USA, 2009.

[5] T. Rauber and G. Rünger, “Analytical modeling and simulation of the energy consumption of independent tasks,” in Proceedings of the Winter Simulation Conference, ser. WSC ’12. Winter Simulation Conference, 2012, pp. 245:1–245:13.

[6] J. Zhuo and C. Chakrabarti, “Energy-efficient dynamic task scheduling algorithms for dvs systems,” ACM Trans. Embed. Comput. Syst., vol. 7, no. 2, pp. 17:1–17:25, Jan. 2008.

[7] N. B. Rizvandi, J. Taheri, and A. Y. Zomaya, “Some observations on optimal frequency selection in DVFS-based energy consumption minimization,” J. Parallel Distrib. Comput., vol. 71, no. 8, pp. 1154– 1164, Aug. 2011.

[8] E. Le Sueur and G. Heiser, “Dynamic voltage and frequency scaling: The laws of diminishing returns,” in Proceedings of the 2010 Workshop on Power Aware Computing and Systems (HotPower’10), Oct. 2010.

[9] N. S. Kim, T. Austin, D. Blaauw, T. Mudge, K. Flautner, J. S. Hu, M. J. Irwin, M. Kandemir, and V. Narayanan, “Leakage current: Moore’s law meets static power,” Computer, vol. 36, no. 12, pp. 68–75, Dec. 2003.

[10] H. Shen, J. Lu, and Q. Qiu, “Learning based DVFS for simultaneous temperature, performance and energy management,” in ISQED, 2012, pp. 747–754.

[11] “grid5000.” [Online]. Available: http://www.grid5000.fr/

[12] NASA Advanced Supercomputing Division, “NAS parallel benchmarks,” Mar. 2012. [Online]. Available: http://www.nas.nasa. gov/publications/npb.html