From: Arnaud Giersch <arnaud.giersch@iut-bm.univ-fcomte.fr>
Date: Fri, 11 Jan 2013 16:33:58 +0000 (+0100)
Subject: Describe the metrics.
X-Git-Url: https://bilbo.iut-bm.univ-fcomte.fr/and/gitweb/loba-papers.git/commitdiff_plain/a17baac0d900ec0555cc2002c02c3af3fd38ade7

Describe the metrics.
---

diff --git a/supercomp11/supercomp11.tex b/supercomp11/supercomp11.tex
index 056e185..f41425c 100644
--- a/supercomp11/supercomp11.tex
+++ b/supercomp11/supercomp11.tex
@@ -526,8 +526,10 @@ nodes: 16, 64, 256, and 1024 nodes.
 The distributed processes of the application were then logically organized along
 three possible topologies: a line, a torus or an hypercube.  We ran tests where
 the total load was initially on an only node (at one end for the line topology),
-and other tests where the load was initially randomly distributed across all
-the participating nodes.
+and other tests where the load was initially randomly distributed across all the
+participating nodes.  The total amount of load was fixed to a number of load
+units equal to 1000 times the number of node.  The average load is then of 1000
+load units.
 
 For each of the preceding configuration, we finally had to choose the
 computation and communication costs of a load unit.  We chose them, such as to
@@ -564,16 +566,44 @@ time.
 
 \paragraph{Metrics}
 
-In order to evaluate and compare the different load balancing strategies, we
-choose to measure the following metrics:
+In order to evaluate and compare the different load balancing strategies we had
+to define several metrics.  Our goal, when choosing these metrics, was to have
+something tending to a constant value, i.e. to have a measure which is not
+changing anymore once the convergence state is reached.  Moreover, we wanted to
+have some normalized value, in order to be able to compare them across different
+settings.
 
+With these constraints in mind, we defined the following metrics:
+%
 \begin{description}
-\item[\textbf{average idle time:}]
-\item[\textbf{average convergence date:}]
-\item[\textbf{maximum convergence date:}]
-\item[\textbf{data transfer amount:}] relative to the total data amount
+\item[\textbf{average idle time:}] that's the total time spent, when the nodes
+  don't hold any share of load, and thus have nothing to compute.  This total
+  time is divided by the number of participating nodes, such as to have a number
+  that can be compared between simulations of different sizes.
+
+  This metric is expected to give an idea of the ability of the strategy to
+  diffuse the load quickly, lesser is better.
+
+\item[\textbf{average convergence date:}] that's the average of the dates when
+  all nodes reached the convergence state.  The dates are measured as a number
+  of (simulated) seconds since the beginning of the simulation.
+
+\item[\textbf{maximum convergence date:}] that's the date when the last node
+  reached the convergence state.
+
+  These two dates give an idea of the time needed by the strategy to reach the
+  equilibrium state, lesser is doubtlessly better.
+
+\item[\textbf{data transfer amount:}] that's the sum of the amount of data of
+  all transfers during the simulation.  This sum is then normalized by dividing
+  it by the total amount of data present in the system.
+
+  This metric is expected to give an idea of the efficiency of the strategy in
+  terms of data movements, i.e. its ability to reach the equilibrium with fewer
+  transfers.
+
 \end{description}
-\FIXME{dire à chaque fois ce que ça représente, et motiver le choix}
+
 
 \subsection{Validation of our approaches}
 \label{Results}