1 /*! \page tracing Tracing Simulations for Visualization
3 \htmlinclude .tracing.doc.toc
5 \section tracing_tracing Tracing Simulations for Visualization
7 The trace visualization is widely used to observe and understand the behavior
8 of parallel applications and distributed algorithms. Usually, this is done in a
9 two-step fashion: the user instruments the application and the traces are
10 analyzed after the end of the execution. The visualization itself can highlights
11 unexpected behaviors, bottlenecks and sometimes can be used to correct
12 distributed algorithms. The SimGrid team has instrumented the library
13 in order to let users trace their simulations and analyze them. This part of the
14 user manual explains how the tracing-related features can be enabled and used
15 during the development of simulators using the SimGrid library.
17 \subsection tracing_tracing_howitworks How it works
19 For now, the SimGrid library is instrumented so users can trace the <b>platform
20 utilization</b> using the MSG, SimDAG and SMPI interface. This means that the tracing will
21 register how much power is used for each host and how much bandwidth is used for
22 each link of the platform. The idea with this type of tracing is to observe the
23 overall view of resources utilization in the first place, especially the
24 identification of bottlenecks, load-balancing among hosts, and so on.
26 The idea of the tracing facilities is to give SimGrid users to possibility to
27 classify MSG and SimDAG tasks by category, tracing the platform utilization
28 (hosts and links) for each of the categories. For that,
29 the tracing interface enables the declaration of categories and a function to
30 mark a task with a previously declared category. <em>The tasks that are not
31 classified according to a category are not traced</em>. Even if the user
32 does not specify any category, the simulations can still be traced in terms
33 of resource utilization by using a special parameter that is detailed below.
35 \subsection tracing_tracing_enabling Enabling using CMake
37 With the sources of SimGrid, it is possible to enable the tracing
38 using the parameter <b>-Denable_tracing=ON</b> when the cmake is executed.
39 The section \ref tracing_tracing_functions describes all the functions available
40 when this Cmake options is activated. These functions will have no effect
41 if SimGrid is configured without this option (they are wiped-out by the
45 $ cmake -Denable_tracing=ON .
49 \subsection tracing_tracing_functions Tracing Functions
51 \li <b>\c TRACE_category (const char *category)</b>: This function should be used
52 to define a user category. The category can be used to differentiate the tasks
53 that are created during the simulation (for example, tasks from server1,
54 server2, or request tasks, computation tasks, communication tasks).
55 All resource utilization (host power and link bandwidth) will be
56 classified according to the task category. Tasks that do not belong to a
57 category are not traced. The color for the category that is being declared
58 is random (use next function to specify a color).
60 \li <b>\c TRACE_category_with_color (const char *category, const char *color)</b>: Same
61 as TRACE_category, but let user specify a color encoded as a RGB-like string with
62 three floats from 0 to 1. So, to specify a red color, the user can pass "1 0 0" as
63 color parameter. A light-gray color can be specified using "0.7 0.7 0.7" as color.
65 \li <b>\c TRACE_msg_set_task_category (m_task_t task, const char *category)</b>:
66 This function should be called after the creation of a MSG task, to define the
67 category of that task. The first parameter \c task must contain a task that was
68 created with the function \c MSG_task_create. The second parameter
69 \c category must contain a category that was previously defined by the function
72 \li <b>\c TRACE_sd_set_task_category (SD_task_t task, const char *category)</b>:
73 This function should be called after the creation of a SimDAG task, to define the
74 category of that task. The first parameter \c task must contain a task that was
75 created with the function \c MSG_task_create. The second parameter
76 \c category must contain a category that was previously defined by the function
79 \li <b>\c TRACE_declare_mark(const char *mark_type)</b>: This function
80 declares a new Paje event type in the trace file that can be used by
81 simulators to declare application-level marks. This function is
82 independent of which API is used in SimGrid.
84 \li <b>\c TRACE_mark(const char *mark_type, const char *mark_value)</b>:
85 This function creates a mark in the trace file. The first parameter
86 had to be previously declared using \c TRACE_declare_mark, the second
87 is the identifier for this mark instance. We recommend that the \c
88 mark_value (the second parameter) is a unique value for the whole
89 trace file (the whole simulation). Nevertheless, this is not a strong
90 requirement: the trace will be valid if there are multiple mark
91 identifiers for the same trace.
93 \li <b>\c TRACE_[host|link]_variable_declare (const char *variable)</b>:
94 Declare a user variable that will be associated to host/link. A variable can
95 be used to trace user variables such as the number of tasks in a server,
96 the number of clients in an application (for hosts), and so on.
98 \li <b>\c TRACE_[host|link]_variable_declare_with_color (const char
99 *var, const char *color)</b>: Same as \c
100 TRACE_[host|link]_variable_declare, but user decides which color will
101 be assigned to the variable. The color needs to be a string with three
102 numbers separated by spaces in the range [0,1]. A light-gray color can
103 be specified using "0.7 0.7 0.7" as color.
105 \li <b>\c TRACE_[host|link]_variable_[set|add|sub] (const char *[host|link], const char *variable, double value)</b>:
106 Set the value of a given user variable for a given host/link. The value
107 of this variable is always associated to the host/link. The host/link
108 parameters should be its name as the one listed in the platform file.
110 \li <b>\c TRACE_[host|link]_variable_[set|add|sub]_with_time (double time, const char *[host|link], const char *variable, double value)</b>:
111 Same as TRACE_[host|link]_variable_[set|add|sub], but let user specify
112 the time used to trace it. Users can specify a time that is not the
113 simulated clock time as defined by the core simulator. This allows
114 a fine-grain control of time definition, but should be used with
115 caution since the trace can be inconsistent if resource utilization
116 traces are also traced.
118 \li <b>\c TRACE_link_srcdst_variable_[set|add|sub] (const char *src, const char *dst, const char *variable, double value)</b>:
119 Same as TRACE_link_variable_[set|add|sub], but now users specify a source and
120 destination hosts (as the names from the platform file). The tracing library
121 will get the corresponding route that connects those two hosts (src and dst) and
122 [set|add|sub] the value's variable for all the links of the route.
124 \li <b>\c TRACE_link_srcdst_variable_[set|add|sub]_with_time (double time, const char *src, const char *dst, const char *variable, double value)</b>:
125 Same as TRACE_link_srcdst_variable_[set|add|sub], but user specify a time different from the simulated time.
127 \subsection tracing_tracing_options Tracing configuration Options
129 To check which tracing options are available for your simulator, you
130 can just run it with the option <b>--help-tracing</b>. These are the
131 options accepted by the tracing system of SimGrid as of today, you
132 can use them by running your simulator with the <b>--cfg=</b> switch:
137 Safe switch. It activates (or deactivates) the tracing system.
138 No other tracing options take effect if this one is not activated.
146 It activates the categorized resource utilization tracing. It should
147 be enabled if tracing categories are used by this simulator.
149 --cfg=tracing/categorized:1
153 tracing/uncategorized
155 It activates the uncategorized resource utilization tracing. Use it if
156 this simulator do not use tracing categories and resource use have to be
159 --cfg=tracing/uncategorized:1
165 A file with this name will be created to register the simulation. The file
166 is in the Paje format and can be analyzed using Triva or Paje visualization
167 tools. More information can be found in these webpages:
168 <a href="http://triva.gforge.inria.fr/">http://triva.gforge.inria.fr/</a>
169 <a href="http://paje.sourceforge.net/">http://paje.sourceforge.net/</a>
171 --cfg=tracing/filename:mytracefile.trace
173 If you do not provide this parameter, the trace file will be named simgrid.trace.
178 By default, the tracing system uses all routes in the platform file
179 to re-create a "graph" of the platform and register it in the trace file.
180 This option let the user tell the tracing system to use only the routes
181 that are composed with just one link.
183 --cfg=tracing/onelink_only:1
189 This option only has effect if this simulator is SMPI-based. Traces the MPI
190 interface and generates a trace that can be analyzed using Gantt-like
191 visualizations. Every MPI function (implemented by SMPI) is transformed in a
192 state, and point-to-point communications can be analyzed with arrows.
200 This option only has effect if this simulator is SMPI-based. The processes
201 are grouped by the hosts where they were executed.
203 --cfg=tracing/smpi/group:1
209 This option only has effect if this simulator is MSG-based. It traces the
210 behavior of all categorized MSG tasks, grouping them by hosts.
212 --cfg=tracing/msg/task:1
218 This option only has effect if this simulator is MSG-based. It traces the
219 behavior of all categorized MSG processes, grouping them by hosts. This option
220 can be used to track process location if this simulator has process migration.
222 --cfg=tracing/msg/process:1
228 This option put some events in a time-ordered buffer using the
229 insertion sort algorithm. The process of acquiring and releasing
230 locks to access this buffer and the cost of the sorting algorithm
231 make this process slow. The simulator performance can be severely
232 impacted if this option is activated, but you are sure to get a trace
233 file with events sorted.
235 --cfg=tracing/buffer:1
241 This option changes the way SimGrid register its platform on the trace
242 file. Normally, the tracing considers all routes (no matter their
243 size) on the platform file to re-create the resource topology. If this
244 option is activated, only the routes with one link are used to
245 register the topology within an AS. Routes among AS continue to be
248 --cfg=tracing/onelink_only:1
252 tracing/disable_destroy
254 Disable the destruction of containers at the end of simulation. This
255 can be used with simulators that have a different notion of time
256 (different from the simulated time).
258 --cfg=tracing/disable_destroy:1
264 This option generates a graph configuration file for Triva considering
265 categorized resource utilization.
267 --cfg=triva/categorized:graph_categorized.plist
273 This option generates a graph configuration file for Triva considering
274 uncategorized resource utilization.
276 --cfg=triva/categorized:graph_uncategorized.plist
279 \subsection tracing_tracing_example_parameters Case studies
281 Some scenarios that might help you decide which tracing options
282 you should use to analyze your simulator.
284 \li I want to trace the resource utilization of all hosts
285 and links of the platform, and my simulator <b>does not</b> use
286 the tracing API. For that, you can run a uncategorized trace
287 with the following parameters (it will work with <b>any</b> Simgrid
292 --cfg=tracing/uncategorized:1 \
293 --cfg=tracing/filename:mytracefile.trace \
294 --cfg=triva/uncategorized:uncat.plist
297 \li I want to trace only a subset of my MSG (or SimDAG) tasks.
298 For that, you will need to create tracing categories using the
299 <b>TRACE_category (...)</b> function (as explained above),
300 and then classify your tasks to a previously declared category
301 using the <b>TRACE_msg_set_task_category (...)</b>
302 (or <b>TRACE_sd_set_task_category (...)</b> for SimDAG tasks). After
303 recompiling, run your simulator with the following parameters:
307 --cfg=tracing/categorized:1 \
308 --cfg=tracing/filename:mytracefile.trace \
309 --cfg=triva/categorized:cat.plist
313 \subsection tracing_tracing_example Example of Instrumentation
315 A simplified example using the tracing mandatory functions.
318 int main (int argc, char **argv)
320 MSG_global_init (&argc, &argv);
322 //(... after deployment ...)
324 //note that category declaration must be called after MSG_create_environment
325 TRACE_category_with_color ("request", "1 0 0");
326 TRACE_category_with_color ("computation", "0.3 1 0.4");
327 TRACE_category ("finalize");
329 m_task_t req1 = MSG_task_create("1st_request_task", 10, 10, NULL);
330 m_task_t req2 = MSG_task_create("2nd_request_task", 10, 10, NULL);
331 m_task_t req3 = MSG_task_create("3rd_request_task", 10, 10, NULL);
332 m_task_t req4 = MSG_task_create("4th_request_task", 10, 10, NULL);
333 TRACE_msg_set_task_category (req1, "request");
334 TRACE_msg_set_task_category (req2, "request");
335 TRACE_msg_set_task_category (req3, "request");
336 TRACE_msg_set_task_category (req4, "request");
338 m_task_t comp = MSG_task_create ("comp_task", 100, 100, NULL);
339 TRACE_msg_set_task_category (comp, "computation");
341 m_task_t finalize = MSG_task_create ("finalize", 0, 0, NULL);
342 TRACE_msg_set_task_category (finalize, "finalize");
351 \subsection tracing_tracing_analyzing Analyzing the SimGrid Traces
353 The SimGrid library, during an instrumented simulation, creates a trace file in
354 the Paje file format that contains the platform utilization for the simulation
355 that was executed. The visualization analysis of this file is performed with the
356 visualization tool <a href="http://triva.gforge.inria.fr">Triva</a>, with
357 special configurations tunned to SimGrid needs. This part of the documentation
358 explains how to configure and use Triva to analyse a SimGrid trace file.
360 - <b>Installing Triva</b>: the tool is available in the INRIAGforge,
361 at <a href="http://triva.gforge.inria.fr">http://triva.gforge.inria.fr</a>.
362 Use the following command to get the sources, and then check the file
363 <i>INSTALL</i>. This file contains instructions to install
364 the tool's dependencies in a Ubuntu/Debian Linux. The tool can also
365 be compiled in MacOSes natively, check <i>INSTALL.mac</i> file.
367 $ svn checkout svn://scm.gforge.inria.fr/svn/triva
372 - <b>Executing Triva</b>: a binary called <i>Triva</i> is available after the
373 installation (you can execute it passing <em>--help</em> to check its
374 options). If the triva binary is not available after following the
375 installation instructions, you may want to execute the following command to
376 initialize the GNUstep environment variables. We strongly recommend that you
377 use the latest GNUstep packages, and not the packages available through apt-get
378 in Ubuntu/Debian packaging systems. If you install GNUstep using the latest
379 available packages, you can execute this command:
381 $ source /usr/GNUstep/System/Library/Makefiles/GNUstep.sh
383 You should be able to see this output after the installation of triva:
385 $ ./Triva.app/Triva --help
386 Usage: Triva [OPTIONS...] TRACE0 [TRACE1]
387 Trace Analysis through Visualization
390 --ti_frequency {double} Animation: frequency of updates
391 --ti_hide Hide the TimeInterval window
392 --ti_forward {double} Animation: value to move time-slice
393 --ti_apply Apply the configuration
394 --ti_update Update on slider change
395 --ti_animate Start animation
396 --ti_start {double} Start of time slice
397 --ti_size {double} Size of time slice
399 --comparison Compare Trace Files (Experimental)
400 --graph Configurable Graph
401 --list Print Trace Type Hierarchy
402 --hierarchy Export Trace Type Hierarchy (dot)
403 --stat Trace Statistics and Memory Utilization
404 --instances List All Trace Entities
405 --linkview Link View (Experimental)
406 --treemap Squarified Treemap
407 --merge Merge Trace Files (Experimental)
408 --check Check Trace File Integrity
410 --gc_conf {file} Graph Configuration in Property List Format
411 --gc_apply Apply the configuration
412 --gc_hide Hide the GraphConfiguration window
414 Triva expects that the user choose one of the available options
415 (currently <em>--graph</em> or <em>--treemap</em> for a visualization analysis)
416 and the trace file from the simulation.
418 - <b>Understanding Triva - time-slice</b>: the analysis of a trace file using
419 the tool always takes into account the concept of the <em>time-slice</em>.
420 This concept means that what is being visualized in the screen is always
421 calculated considering a specific time frame, with its beggining and end
422 timestamp. The time-slice is configured by the user and can be changed
423 dynamically through the window called <em>Time Interval</em> that is opened
424 whenever a trace file is being analyzed. The next figure depicts the time-slice
425 configuration window.
426 In the top of the window, in the space named <i>Trace Time</i>,
427 the two fields show the beggining of the trace (which usually starts in 0) and
428 the end (that depends on the time simulated by SimGrid). The middle of the
429 window, in the square named <i>Time Slice Configuration</i>, contains the
430 aspects related to the time-slice, including its <i>start</i> and its
431 <i>size</i>. The gray rectangle in the bottom of this part indicates the
432 <i>current time-slice</i> that is considered for the drawings. If the checkbox
433 <i>Update Drawings on Sliders Change</i> is not selected, the button
434 <i>Apply</i> must be clicked in order to inform triva that the
435 new time-slice must be considered. The bottom part of the window, in the space
436 indicated by the square <i>Time Slice Animation</i> can be used to advance
437 the time-frame automatically. The user configures the amount of time that the
438 time-frame will forward and how frequent this update will happen. Once this is
439 configured, the user clicks the <i>Play</i> button in order to see the dynamic
440 changes on the drawings.
443 <a href="triva-time_interval.png" border=0><img src="triva-time_interval.png" width="50%" border=0></a>
446 <b>Remarks:</b> when the trace has too many hosts or links, the computation to
447 take into account a new time-slice can be expensive. When this happens, the
448 <i>Frequency</i> parameter, but also updates caused by change on configurations
449 when the checkbox <i>Update Drawings on Sliders
450 Change</i> is selected will not be followed.
452 - <b>Understanding Triva - graph</b>: this part of the documention explains how
453 to analyze the traces using the graph view of Triva, when the user executes
454 the tool passing <em>--graph</em> as parameter. Triva opens three windows when
455 this parameter is used: the <i>Time Interval</i> window (previously described),
456 the <i>Graph Representation</i> window, and the <em>Graph Configuration</em>
457 window. The Graph Representation is the window where drawings take place.
458 Initially, it is completely white waiting for a proper graph configuration input
459 by the user. We start the description of this type of analysis by describing the
460 <i>Graph Configuration</i> window (depicted below). By using a particular
462 can be used to customize the graph drawing according to
463 the SimGrid trace that was created with user-specific categories. Before delving
464 into the details of this customization, let us first explain the major parts of
465 the graph configuration window. The buttons located in the top-right corner can
466 be used to delete, copy and create a new configuration. The checkbox in the
467 top-middle part of the window indicates if the configuration typed in the
468 textfield is syntactically correct (we are using the non-XML
469 <a href="http://en.wikipedia.org/wiki/Property_list">Property List Format</a> to
470 describe the configuration). The pop-up button located on the top-left corner
471 indicates the selected configuration (the user can have multiple graph
472 configurations). The bottom-left text field contains the name of the current
473 configuration (updates on this field must be followed by typing enter on the
474 keyboard to take into account the name change). The bottom-right <em>Apply</em>
475 button activates the current configuration, resulting on an update on the graph
479 <a href="triva-graph_configuration.png" border=0><img src="triva-graph_configuration.png" width="50%" border=0></a>
482 <b>Basic SimGrid Configuration</b>: The figure shows in the big textfield the
483 basic configuration that should be used during the analysis of a SimGrid trace
484 file. The basic logic of the configuration is as follows:
490 The nodes of the graph will be created based on the <i>node</i> parameter, which
491 in this case is the different <em>"HOST"</em>s of the platform
492 used to simulate. The <i>edge</i> parameter indicates that the edges of the
493 graph will be created based on the <em>"LINK"</em>s of the platform. After the
494 definition of these two parameters, the configuration must detail how
495 <em>HOST</em>s and <em>LINK</em>s should be drawn. For that, the configuration
496 must have an entry for each of the types used. For <em>HOST</em>, as basic
497 configuration, we have:
504 The parameter <em>size</em> indicates which variable from the trace file will be
505 used to define the size of the node HOST in the visualization. If the simulation
506 was executed with availability traces, the size of the nodes will be changed
507 according to these traces. The parameter <em>scale</em> indicates if the value
508 of the variable is <em>global</em> or <em>local</em>. If it is global, the value
509 will be relative to the power of all other hosts, if it is local, the value will
511 For <em>LINK</em> we have:
521 For the types specified in the <em>edge</em> parameter (such as <em>LINK</em>),
522 the configuration must contain two additional parameters: <em>src</em> and
523 <em>dst</em> that are used to properly identify which nodes this edge is
524 connecting. The values <em>source</em> and <em>destination</em> are always present
525 in the SimGrid trace file and should not be changed in the configuration. The
526 parameter <em>size</em> for the LINK, in this case, is configured as the
527 variable <em>bandwidth</em>, with a <em>global</em> scale. The scale meaning
528 here is exactly the same used for nodes. The last parameter is the GraphViz
529 algorithm used to calculate the position of the nodes in the graph
532 graphviz-algorithm = neato;
535 <b>Customizing the Graph Representation</b>: triva is capable to handle
536 a customized graph representation based on the variables present in the trace
537 file. In the case of SimGrid, every time a category is created for tasks, two
538 variables in the trace file are defined: one to indicate node utilization (how
539 much power was used by that task category), and another to indicate link
540 utilization (how much bandwidth was used by that category). For instance, if the
541 user declares a category named <i>request</i>, there will be variables named
542 <b>p</b><i>request</i> and a <b>b</b><i>request</i> (<b>p</b> for power and
543 <b>b</b> for bandwidth). It is important to notice that the variable
544 <i>prequest</i> in this case is only available for HOST, and
545 <i>brequest</i> is only available for LINK. <b>Example</b>: suppose there are
546 two categories for tasks: request and compute. To create a customized graph
547 representation with a proportional separation of host and link utilization, use
548 as configuration for HOST and LINK this:
557 values = (prequest, pcomputation);
570 values = (brequest, bcomputation);
574 Where <i>sep_host</i> contains a composition of type <i>separation</i> where
575 its max size is the <i>power</i> of the host and the variables <i>prequest</i>
576 and <i>pcomputation</i> are drawn proportionally to the size of the HOST. And
577 <i>sep_link</i> is also a separation where max is defined as the
578 <i>bandwidth</i> of the link, and the variables <i>brequest</i> and
579 <i>bcomputation</i> are drawn proportionally within a LINK.
580 <i>This configuration enables the analysis of resource utilization by MSG tasks,
581 and the identification of load-balancing issues, network bottlenecks, for
583 <b>Other compositions</b>: besides <i>separation</i>, it is possible to use
584 other types of compositions, such as gradients, and colors, like this:
589 values = (numberOfTasks);
593 values = (is_server);
596 Where <i>gra_host</i> creates a gradient within a node of the graph, using a
597 global scale and using as value a variable called <i>numberOfTasks</i>, that
598 could be declared by the user using the optional tracing functions of SimGrid.
599 If scale is global, the max and min value for the gradient will be equal to the
600 max and min numberOfTasks among all hosts, and if scale is local, the max and
601 min value based on the value of numberOfTasks locally in each host.
602 And <i>color_host</i> composition draws a square based on a positive value of
603 the variable <i>is_server</i>, that could also be defined by the user using the
604 SimGrid tracing functions. \n
605 <b>The Graph Visualization</b>: The next figure shows a graph visualization of a
606 given time-slice of the masterslave_forwarder example (present in the SimGrid
607 sources). The red color indicates tasks from the <i>compute</i> category. This
608 visualization was generated with the following configuration:
621 values = (pcompute, pfinalize);
627 size = bandwidth;\section tracing_tracing Tracing Simulations for Visualization
629 The trace visualization is widely used to observe and understand the behavior
630 of parallel applications and distributed algorithms. Usually, this is done in a
631 two-step fashion: the user instruments the application and the traces are
632 analyzed after the end of the execution. The visualization itself can highlights
633 unexpected behaviors, bottlenecks and sometimes can be used to correct
634 distributed algorithms. The SimGrid team has instrumented the library
635 in order to let users trace their simulations and analyze them. This part of the
636 user manual explains how the tracing-related features can be enabled and used
637 during the development of simulators using the SimGrid library.
639 \subsection tracing_tracing_howitworks How it works
641 For now, the SimGrid library is instrumented so users can trace the <b>platform
642 utilization</b> using the MSG, SimDAG and SMPI interface. This means that the tracing will
643 register how much power is used for each host and how much bandwidth is used for
644 each link of the platform. The idea with this type of tracing is to observe the
645 overall view of resources utilization in the first place, especially the
646 identification of bottlenecks, load-balancing among hosts, and so on.
648 The idea of the tracing facilities is to give SimGrid users to possibility to
649 classify MSG and SimDAG tasks by category, tracing the platform utilization
650 (hosts and links) for each of the categories. For that,
651 the tracing interface enables the declaration of categories and a function to
652 mark a task with a previously declared category. <em>The tasks that are not
653 classified according to a category are not traced</em>. Even if the user
654 does not specify any category, the simulations can still be traced in terms
655 of resource utilization by using a special parameter that is detailed below.
657 \subsection tracing_tracing_enabling Enabling using CMake
659 With the sources of SimGrid, it is possible to enable the tracing
660 using the parameter <b>-Denable_tracing=ON</b> when the cmake is executed.
661 The section \ref tracing_tracing_functions describes all the functions available
662 when this Cmake options is activated. These functions will have no effect
663 if SimGrid is configured without this option (they are wiped-out by the
667 $ cmake -Denable_tracing=ON .
671 \subsection tracing_tracing_functions Tracing Functions
673 \li <b>\c TRACE_category (const char *category)</b>: This function should be used
674 to define a user category. The category can be used to differentiate the tasks
675 that are created during the simulation (for example, tasks from server1,
676 server2, or request tasks, computation tasks, communication tasks).
677 All resource utilization (host power and link bandwidth) will be
678 classified according to the task category. Tasks that do not belong to a
679 category are not traced. The color for the category that is being declared
680 is random (use next function to specify a color).
682 \li <b>\c TRACE_category_with_color (const char *category, const char *color)</b>: Same
683 as TRACE_category, but let user specify a color encoded as a RGB-like string with
684 three floats from 0 to 1. So, to specify a red color, the user can pass "1 0 0" as
685 color parameter. A light-gray color can be specified using "0.7 0.7 0.7" as color.
687 \li <b>\c TRACE_msg_set_task_category (m_task_t task, const char *category)</b>:
688 This function should be called after the creation of a MSG task, to define the
689 category of that task. The first parameter \c task must contain a task that was
690 created with the function \c MSG_task_create. The second parameter
691 \c category must contain a category that was previously defined by the function
694 \li <b>\c TRACE_sd_set_task_category (SD_task_t task, const char *category)</b>:
695 This function should be called after the creation of a SimDAG task, to define the
696 category of that task. The first parameter \c task must contain a task that was
697 created with the function \c MSG_task_create. The second parameter
698 \c category must contain a category that was previously defined by the function
701 \li <b>\c TRACE_[host|link]_variable_declare (const char *variable)</b>:
702 Declare a user variable that will be associated to host/link. A variable can
703 be used to trace user variables such as the number of tasks in a server,
704 the number of clients in an application (for hosts), and so on.
706 \li <b>\c TRACE_[host|link]_variable_[set|add|sub] (const char *[host|link], const char *variable, double value)</b>:
707 Set the value of a given user variable for a given host/link. The value
708 of this variable is always associated to the host/link. The host/link
709 parameters should be its name as the one listed in the platform file.
711 \li <b>\c TRACE_[host|link]_variable_[set|add|sub]_with_time (double time, const char *[host|link], const char *variable, double value)</b>:
712 Same as TRACE_[host|link]_variable_[set|add|sub], but let user specify
713 the time used to trace it. Users can specify a time that is not the
714 simulated clock time as defined by the core simulator. This allows
715 a fine-grain control of time definition, but should be used with
716 caution since the trace can be inconsistent if resource utilization
717 traces are also traced.
719 \li <b>\c TRACE_link_srcdst_variable_[set|add|sub] (const char *src, const char *dst, const char *variable, double value)</b>:
720 Same as TRACE_link_variable_[set|add|sub], but now users specify a source and
721 destination hosts (as the names from the platform file). The tracing library
722 will get the corresponding route that connects those two hosts (src and dst) and
723 [set|add|sub] the value's variable for all the links of the route.
725 \li <b>\c TRACE_link_srcdst_variable_[set|add|sub]_with_time (double time, const char *src, const char *dst, const char *variable, double value)</b>:
726 Same as TRACE_link_srcdst_variable_[set|add|sub], but user specify a time different from the simulated time.
728 \subsection tracing_tracing_options Tracing configuration Options
730 These are the options accepted by the tracing system of SimGrid:
735 Safe switch. It activates (or deactivates) the tracing system.
736 No other tracing options take effect if this one is not activated.
741 Register the simulation platform in the trace file.
746 By default, the tracing system uses all routes in the platform file
747 to re-create a "graph" of the platform and register it in the trace file.
748 This option let the user tell the tracing system to use only the routes
749 that are composed with just one link.
754 It activates the categorized resource utilization tracing. It should
755 be enabled if tracing categories are used by this simulator.
758 tracing/uncategorized
760 It activates the uncategorized resource utilization tracing. Use it if
761 this simulator do not use tracing categories and resource use have to be
767 A file with this name will be created to register the simulation. The file
768 is in the Paje format and can be analyzed using Triva or Paje visualization
769 tools. More information can be found in these webpages:
770 <a href="http://triva.gforge.inria.fr/">http://triva.gforge.inria.fr/</a>
771 <a href="http://paje.sourceforge.net/">http://paje.sourceforge.net/</a>
776 This option only has effect if this simulator is SMPI-based. Traces the MPI
777 interface and generates a trace that can be analyzed using Gantt-like
778 visualizations. Every MPI function (implemented by SMPI) is transformed in a
779 state, and point-to-point communications can be analyzed with arrows.
784 This option only has effect if this simulator is SMPI-based. The processes
785 are grouped by the hosts where they were executed.
790 This option only has effect if this simulator is MSG-based. It traces the
791 behavior of all categorized MSG tasks, grouping them by hosts.
796 This option only has effect if this simulator is MSG-based. It traces the
797 behavior of all categorized MSG processes, grouping them by hosts. This option
798 can be used to track process location if this simulator has process migration.
802 triva/categorized:graph_categorized.plist
804 This option generates a graph configuration file for Triva considering
805 categorized resource utilization.
808 triva/uncategorized:graph_uncategorized.plist
810 This option generates a graph configuration file for Triva considering
811 uncategorized resource utilization.
813 \subsection tracing_tracing_example Example of Instrumentation
815 A simplified example using the tracing mandatory functions.
818 int main (int argc, char **argv)
820 MSG_global_init (&argc, &argv);
822 //(... after deployment ...)
824 //note that category declaration must be called after MSG_create_environment
825 TRACE_category_with_color ("request", "1 0 0");
826 TRACE_category_with_color ("computation", "0.3 1 0.4");
827 TRACE_category ("finalize");
829 m_task_t req1 = MSG_task_create("1st_request_task", 10, 10, NULL);
830 m_task_t req2 = MSG_task_create("2nd_request_task", 10, 10, NULL);
831 m_task_t req3 = MSG_task_create("3rd_request_task", 10, 10, NULL);
832 m_task_t req4 = MSG_task_create("4th_request_task", 10, 10, NULL);
833 TRACE_msg_set_task_category (req1, "request");
834 TRACE_msg_set_task_category (req2, "request");
835 TRACE_msg_set_task_category (req3, "request");
836 TRACE_msg_set_task_category (req4, "request");
838 m_task_t comp = MSG_task_create ("comp_task", 100, 100, NULL);
839 TRACE_msg_set_task_category (comp, "computation");
841 m_task_t finalize = MSG_task_create ("finalize", 0, 0, NULL);
842 TRACE_msg_set_task_category (finalize, "finalize");
851 \subsection tracing_tracing_analyzing Analyzing the SimGrid Traces
853 The SimGrid library, during an instrumented simulation, creates a trace file in
854 the Paje file format that contains the platform utilization for the simulation
855 that was executed. The visualization analysis of this file is performed with the
856 visualization tool <a href="http://triva.gforge.inria.fr">Triva</a>, with
857 special configurations tunned to SimGrid needs. This part of the documentation
858 explains how to configure and use Triva to analyse a SimGrid trace file.
860 - <b>Installing Triva</b>: the tool is available in the INRIAGforge,
861 at <a href="http://triva.gforge.inria.fr">http://triva.gforge.inria.fr</a>.
862 Use the following command to get the sources, and then check the file
863 <i>INSTALL</i>. This file contains instructions to install
864 the tool's dependencies in a Ubuntu/Debian Linux. The tool can also
865 be compiled in MacOSes natively, check <i>INSTALL.mac</i> file.
867 $ svn checkout svn://scm.gforge.inria.fr/svn/triva
872 - <b>Executing Triva</b>: a binary called <i>Triva</i> is available after the
873 installation (you can execute it passing <em>--help</em> to check its
874 options). If the triva binary is not available after following the
875 installation instructions, you may want to execute the following command to
876 initialize the GNUstep environment variables. We strongly recommend that you
877 use the latest GNUstep packages, and not the packages available through apt-get
878 in Ubuntu/Debian packaging systems. If you install GNUstep using the latest
879 available packages, you can execute this command:
881 $ source /usr/GNUstep/System/Library/Makefiles/GNUstep.sh
883 You should be able to see this output after the installation of triva:
885 $ ./Triva.app/Triva --help
886 Usage: Triva [OPTIONS...] TRACE0 [TRACE1]
887 Trace Analysis through Visualization
890 --ti_frequency {double} Animation: frequency of updates
891 --ti_hide Hide the TimeInterval window
892 --ti_forward {double} Animation: value to move time-slice
893 --ti_apply Apply the configuration
894 --ti_update Update on slider change
895 --ti_animate Start animation
896 --ti_start {double} Start of time slice
897 --ti_size {double} Size of time slice
899 --comparison Compare Trace Files (Experimental)
900 --graph Configurable Graph
901 --list Print Trace Type Hierarchy
902 --hierarchy Export Trace Type Hierarchy (dot)
903 --stat Trace Statistics and Memory Utilization
904 --instances List All Trace Entities
905 --linkview Link View (Experimental)
906 --treemap Squarified Treemap
907 --merge Merge Trace Files (Experimental)
908 --check Check Trace File Integrity
910 --gc_conf {file} Graph Configuration in Property List Format
911 --gc_apply Apply the configuration
912 --gc_hide Hide the GraphConfiguration window
914 Triva expects that the user choose one of the available options
915 (currently <em>--graph</em> or <em>--treemap</em> for a visualization analysis)
916 and the trace file from the simulation.
918 - <b>Understanding Triva - time-slice</b>: the analysis of a trace file using
919 the tool always takes into account the concept of the <em>time-slice</em>.
920 This concept means that what is being visualized in the screen is always
921 calculated considering a specific time frame, with its beggining and end
922 timestamp. The time-slice is configured by the user and can be changed
923 dynamically through the window called <em>Time Interval</em> that is opened
924 whenever a trace file is being analyzed. The next figure depicts the time-slice
925 configuration window.
926 In the top of the window, in the space named <i>Trace Time</i>,
927 the two fields show the beggining of the trace (which usually starts in 0) and
928 the end (that depends on the time simulated by SimGrid). The middle of the
929 window, in the square named <i>Time Slice Configuration</i>, contains the
930 aspects related to the time-slice, including its <i>start</i> and its
931 <i>size</i>. The gray rectangle in the bottom of this part indicates the
932 <i>current time-slice</i> that is considered for the drawings. If the checkbox
933 <i>Update Drawings on Sliders Change</i> is not selected, the button
934 <i>Apply</i> must be clicked in order to inform triva that the
935 new time-slice must be considered. The bottom part of the window, in the space
936 indicated by the square <i>Time Slice Animation</i> can be used to advance
937 the time-frame automatically. The user configures the amount of time that the
938 time-frame will forward and how frequent this update will happen. Once this is
939 configured, the user clicks the <i>Play</i> button in order to see the dynamic
940 changes on the drawings.
943 <a href="triva-time_interval.png" border=0><img src="triva-time_interval.png" width="50%" border=0></a>
946 <b>Remarks:</b> when the trace has too many hosts or links, the computation to
947 take into account a new time-slice can be expensive. When this happens, the
948 <i>Frequency</i> parameter, but also updates caused by change on configurations
949 when the checkbox <i>Update Drawings on Sliders
950 Change</i> is selected will not be followed.
952 - <b>Understanding Triva - graph</b>: this part of the documention explains how
953 to analyze the traces using the graph view of Triva, when the user executes
954 the tool passing <em>--graph</em> as parameter. Triva opens three windows when
955 this parameter is used: the <i>Time Interval</i> window (previously described),
956 the <i>Graph Representation</i> window, and the <em>Graph Configuration</em>
957 window. The Graph Representation is the window where drawings take place.
958 Initially, it is completely white waiting for a proper graph configuration input
959 by the user. We start the description of this type of analysis by describing the
960 <i>Graph Configuration</i> window (depicted below). By using a particular
962 can be used to customize the graph drawing according to
963 the SimGrid trace that was created with user-specific categories. Before delving
964 into the details of this customization, let us first explain the major parts of
965 the graph configuration window. The buttons located in the top-right corner can
966 be used to delete, copy and create a new configuration. The checkbox in the
967 top-middle part of the window indicates if the configuration typed in the
968 textfield is syntactically correct (we are using the non-XML
969 <a href="http://en.wikipedia.org/wiki/Property_list">Property List Format</a> to
970 describe the configuration). The pop-up button located on the top-left corner
971 indicates the selected configuration (the user can have multiple graph
972 configurations). The bottom-left text field contains the name of the current
973 configuration (updates on this field must be followed by typing enter on the
974 keyboard to take into account the name change). The bottom-right <em>Apply</em>
975 button activates the current configuration, resulting on an update on the graph
979 <a href="triva-graph_configuration.png" border=0><img src="triva-graph_configuration.png" width="50%" border=0></a>
982 <b>Basic SimGrid Configuration</b>: The figure shows in the big textfield the
983 basic configuration that should be used during the analysis of a SimGrid trace
984 file. The basic logic of the configuration is as follows:
990 The nodes of the graph will be created based on the <i>node</i> parameter, which
991 in this case is the different <em>"HOST"</em>s of the platform
992 used to simulate. The <i>edge</i> parameter indicates that the edges of the
993 graph will be created based on the <em>"LINK"</em>s of the platform. After the
994 definition of these two parameters, the configuration must detail how
995 <em>HOST</em>s and <em>LINK</em>s should be drawn. For that, the configuration
996 must have an entry for each of the types used. For <em>HOST</em>, as basic
997 configuration, we have:
1004 The parameter <em>size</em> indicates which variable from the trace file will be
1005 used to define the size of the node HOST in the visualization. If the simulation
1006 was executed with availability traces, the size of the nodes will be changed
1007 according to these traces. The parameter <em>scale</em> indicates if the value
1008 of the variable is <em>global</em> or <em>local</em>. If it is global, the value
1009 will be relative to the power of all other hosts, if it is local, the value will
1010 be relative locally.
1011 For <em>LINK</em> we have:
1021 For the types specified in the <em>edge</em> parameter (such as <em>LINK</em>),
1022 the configuration must contain two additional parameters: <em>src</em> and
1023 <em>dst</em> that are used to properly identify which nodes this edge is
1024 connecting. The values <em>source</em> and <em>destination</em> are always present
1025 in the SimGrid trace file and should not be changed in the configuration. The
1026 parameter <em>size</em> for the LINK, in this case, is configured as the
1027 variable <em>bandwidth</em>, with a <em>global</em> scale. The scale meaning
1028 here is exactly the same used for nodes. The last parameter is the GraphViz
1029 algorithm used to calculate the position of the nodes in the graph
1032 graphviz-algorithm = neato;
1035 <b>Customizing the Graph Representation</b>: triva is capable to handle
1036 a customized graph representation based on the variables present in the trace
1037 file. In the case of SimGrid, every time a category is created for tasks, two
1038 variables in the trace file are defined: one to indicate node utilization (how
1039 much power was used by that task category), and another to indicate link
1040 utilization (how much bandwidth was used by that category). For instance, if the
1041 user declares a category named <i>request</i>, there will be variables named
1042 <b>p</b><i>request</i> and a <b>b</b><i>request</i> (<b>p</b> for power and
1043 <b>b</b> for bandwidth). It is important to notice that the variable
1044 <i>prequest</i> in this case is only available for HOST, and
1045 <i>brequest</i> is only available for LINK. <b>Example</b>: suppose there are
1046 two categories for tasks: request and compute. To create a customized graph
1047 representation with a proportional separation of host and link utilization, use
1048 as configuration for HOST and LINK this:
1057 values = (prequest, pcomputation);
1070 values = (brequest, bcomputation);
1074 Where <i>sep_host</i> contains a composition of type <i>separation</i> where
1075 its max size is the <i>power</i> of the host and the variables <i>prequest</i>
1076 and <i>pcomputation</i> are drawn proportionally to the size of the HOST. And
1077 <i>sep_link</i> is also a separation where max is defined as the
1078 <i>bandwidth</i> of the link, and the variables <i>brequest</i> and
1079 <i>bcomputation</i> are drawn proportionally within a LINK.
1080 <i>This configuration enables the analysis of resource utilization by MSG tasks,
1081 and the identification of load-balancing issues, network bottlenecks, for
1083 <b>Other compositions</b>: besides <i>separation</i>, it is possible to use
1084 other types of compositions, such as gradients, and colors, like this:
1089 values = (numberOfTasks);
1093 values = (is_server);
1096 Where <i>gra_host</i> creates a gradient within a node of the graph, using a
1097 global scale and using as value a variable called <i>numberOfTasks</i>, that
1098 could be declared by the user using the optional tracing functions of SimGrid.
1099 If scale is global, the max and min value for the gradient will be equal to the
1100 max and min numberOfTasks among all hosts, and if scale is local, the max and
1101 min value based on the value of numberOfTasks locally in each host.
1102 And <i>color_host</i> composition draws a square based on a positive value of
1103 the variable <i>is_server</i>, that could also be defined by the user using the
1104 SimGrid tracing functions. \n
1105 <b>The Graph Visualization</b>: The next figure shows a graph visualization of a
1106 given time-slice of the masterslave_forwarder example (present in the SimGrid
1107 sources). The red color indicates tasks from the <i>compute</i> category. This
1108 visualization was generated with the following configuration:
1121 values = (pcompute, pfinalize);
1133 values = (bcompute, bfinalize);
1136 graphviz-algorithm = neato;
1141 <a href="triva-graph_visualization.png" border=0><img src="triva-graph_visualization.png" width="50%" border=0></a>
1145 - <b>Understading Triva - colors</b>: An important issue when using Triva is how
1146 to define colors. To do that, we have to know which variables are defined in
1147 the trace file generated by the SimGrid library. The parameter <em>--list</em>
1148 lists the variables for a given trace file:
1150 $ Triva -l masterslave_forwarder.trace
1168 We can see that HOST has seven variables (from power to pfinalize) and LINK has
1169 four (from bandwidth to bfinalize). To define a red color for the
1170 <i>pcompute</i> and <i>bcompute</i> (which are defined based on user category
1171 <i>compute</i>), execute:
1173 $ defaults write Triva 'pcompute Color' '1 0 0'
1174 $ defaults write Triva 'bcompute Color' '1 0 0'
1176 Where the three numbers in each line are the RGB color with values from 0 to 1.
1183 values = (bcompute, bfinalize);
1186 graphviz-algorithm = neato;
1191 <a href="triva-graph_visualization.png" border=0><img src="triva-graph_visualization.png" width="50%" border=0></a>
1195 - <b>Understading Triva - colors</b>: An important issue when using Triva is how
1196 to define colors. To do that, we have to know which variables are defined in
1197 the trace file generated by the SimGrid library. The parameter <em>--list</em>
1198 lists the variables for a given trace file:
1200 $ Triva -l masterslave_forwarder.trace
1218 We can see that HOST has seven variables (from power to pfinalize) and LINK has
1219 four (from bandwidth to bfinalize). To define a red color for the
1220 <i>pcompute</i> and <i>bcompute</i> (which are defined based on user category
1221 <i>compute</i>), execute:
1223 $ defaults write Triva 'pcompute Color' '1 0 0'
1224 $ defaults write Triva 'bcompute Color' '1 0 0'
1226 Where the three numbers in each line are the RGB color with values from 0 to 1.