3 <object id="TOC" data="graphical-toc.svg" type="image/svg+xml"></object>
5 window.onload=function() { // Wait for the SVG to be loaded before changing it
6 var elem=document.querySelector("#TOC").contentDocument.getElementById("ModelBox")
7 elem.style="opacity:0.93999999;fill:#ff0000;fill-opacity:0.1;stroke:#000000;stroke-width:0.35277778;stroke-linecap:round;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-dashoffset:0;stroke-opacity:1";
18 As for any simulator, models are very important components of the SimGrid toolkit. This page first introduces the
19 different kind of models used in SimGrid before focusing on the **performance models** that compute the duration of
20 :ref:`every activities <S4U_main_concepts>` in the simulator depending on the platform characteristics and on the
21 other activities that are currently sharing the resources.
23 The **routing models** constitute advanced elements of the platform description. This description naturally entails
24 :ref:`components<platform>` that are very related to the performance models. For instance, determining the execution
25 time of a task obviously depends on the characteristics of the machine that executes this task. Furthermore, networking
26 zones can be interconnected to compose larger platforms `in a scalable way <http://hal.inria.fr/hal-00650233/>`_. Each
27 of these zones can be given a specific :ref:`routing model<platform_routing>` that efficiently computes the list of
28 links forming a network path between two given hosts.
30 The model checker uses an abstraction of the performance simulations. Mc SimGrid explores every causally possible
31 execution paths of the application, completely abstracting the performance away. The simulated time is not even
32 computed in this mode! The abstraction involved in this process also models the mutual impacts among actions, to not
33 re-explore histories that only differ by the order of independent and unrelated actions. As with the rest of the model
34 checker, these models are unfortunately still to be documented properly.
36 Finally, the `SimGrid-FMI external plugin <https://framagit.org/simgrid/simgrid-FMI>`_ can be used to integrate any
37 FMI-based model into SimGrid. This was used to accurately study a *Smart grid* through co-simulation: `PandaPower
38 <http://www.pandapower.org/>`_ was used to simulate the power grid, `ns-3 <https://nsnam.org/>`_ was used the
39 communication network while SimGrid was simulating the IT infrastructure. Please refer to the `relevant publication
40 <https://hal.archives-ouvertes.fr/hal-01762540/>`_ for more details.
45 The main objective of SimGrid is to provide timing information for three following kind of resources: network, CPU,
48 The **network models** have been improved and regularly assessed for almost 20 years. It should be possible to get
49 accurate predictions once you properly :ref:`calibrate the models for your settings<models_calibration>`. As detailed
50 in the next sections, SimGrid provides several network models. Two plugins can also be used to compute the network
51 energy consumption: One for the :ref:`wired networks<plugin_link_energy>`, and another one for the :ref:`Wi-Fi networks
52 <plugin_link_energy>`. Some users find :ref:`TCP simulated performance counter-intuitive<understanding_lv08>` at first
53 in SimGrid, sometimes because of a misunderstanding of the TCP behavior in real networks.
55 The **computing models** are less developed in SimGrid. Through the S4U interface, the user specifies the amount of
56 computational work (expressed in FLOPs, for floating point operations) that each computation "consumes", and the model
57 simply divides this amount by the host's FLOP rate to compute the duration of this execution. In SMPI, the user code
58 is automatically timed, and the :ref:`computing speed<cfg=smpi/host-speed>` of the host machine is used to evaluate
59 the corresponding amount of FLOPs. This model should be sufficient for most users, even though assuming a constant FLOP
60 rate for each machine remains a crude simplification. In reality, the flops rate varies because of I/O, memory, and
61 cache effects. It is somehow possible to :ref:`overcome this simplification<cfg=smpi/comp-adjustment-file>`, but the
62 required calibration process is rather intricate and not documented yet (feel free to
63 :ref:`contact the community<community>` on need).
64 In the future, more advanced models may be added but the existing model proved good enough for all experiments done on
65 distributed applications during the last two decades. The CPU energy consumption can be computed with the
66 :ref:`relevant plugin<plugin_host_energy>`.
68 The **disk models** of SimGrid are more recent than those for the network and computing resources, but they should
69 still be correct for most users. `Studies have shown <https://hal.inria.fr/hal-01197128>`_ that they are sensitive
70 under some conditions, and a :ref:`calibration process<howto_disk>` is provided. As usual, you probably want to
71 double-check their predictions through an appropriate validation campaign.
76 SimGrid aims at the sweet spot between accuracy and simulation speed. About accuracy, our goal is to report correct
77 performance trends when comparing competing designs with a minimal burden on the user, while allowing power users to
78 fine tune the simulation models for predictions that are within 5% or less of the results on real machines. For
79 example, we determined the `speedup achieved by the Tibidabo ARM-based cluster <http://hal.inria.fr/hal-00919507>`_
80 before it was even built. About simulation speed, the tool must be fast and scalable enough to study modern IT systems
81 at scale. SimGrid was for example used to simulate `a Chord ring involving millions of actors
82 <https://hal.inria.fr/inria-00602216>`_ (even though that has not really been more instructive than smaller scale
83 simulations for this protocol), or `a qualification run at full-scale of the Stampede supercomputer
84 <https://hal.inria.fr/hal-02096571>`_.
86 Most of our models are based on a linear max-min solver (LMM), as depicted below. The actors' activities are
87 represented by actions in the simulation kernel, accounting for both the initial amount of work of the corresponding
88 activity (in FLOPs for computing activities or bytes for networking and disk activities), and the currently remaining
89 amount of work to process.
91 At each simulation step, the instantaneous computing and communicating capacity of each action is computed according
92 to the model. A set of constraints is used to express for example that the cumulated instantaneous consumption of a
93 given resource by a set actions must remain smaller than the nominal capacity speed of that resource. In the example
94 below, it is stated that the speed :math:`\varrho_1` of activity 1 plus the speed :math:`\varrho_n`
95 of activity :math:`n` must remain smaller than the capacity :math:`C_A` of the corresponding host A.
97 .. image:: img/lmm-overview.svg
99 There are obviously many valuations of :math:`\varrho_1, \ldots{}, \varrho_n` that respect such as set of constraints.
100 SimGrid usually computes the instantaneous speeds according to a Max-Min objective function, that is maximizing the
101 minimum over all :math:`\varrho_i`. The coefficients associated to each variable in the inequalities are used to model
102 some performance effects, such as the fact that TCP tends to favor communications with small RTTs. These coefficients
103 are computed from both hardcoded values and :ref:`latency and bandwidth factors<cfg=network/latency-factor>`.
105 Once the instantaneous speeds are computed, the simulation kernel determines what is the earliest terminating action
106 from their respective speeds and remaining amounts of work. The simulated time is then updated along with the values
107 in the LMM. As some actions have nothing left to do, the corresponding activities thus terminate, which in turn
108 unblocks the corresponding actors that can further execute.
110 Most of the SimGrid models build upon the LMM solver, that they adapt and configure for their respective usage. For
111 network activities, **CM02** is the simplest LMM-based model as it does not introduce any correction factors. This model
112 should be used if you prefer understandable results over realistic ones. **LV08** (the default model) uses constant
113 factors that are intended to capture common effects such as slow-start, the fact that TCP headers reduce the *effective*
114 bandwidth, or TCP's ACK messages. **SMPI** uses more advanced factors that also capture the MPI-specific effects such as
115 the switch between the eager vs. rendez-vous communication modes. You can :ref:`select another
116 model<options_model_select>` on command line, and these models can be :ref:`further configured<options_model>`. For CPU
117 and disk activities, the LMM-based models are respectively named **Cas01** and **S19**.
121 **L07** is rather distinct because it uses another objective function called *bottleneck*. This is because this model is intended to be used for parallel tasks that are actions mixing flops and bytes while the Max-Min objective function requires that all variables are expressed using the same unit. This is also why in reality, we have one LMM system per resource kind in the simulation, but the idea remains similar.
123 .. _understanding_lv08:
125 The default TCP model
126 =====================
128 When simulating a data transfer between two hosts, you may be surprised by the obtained simulation time. Lets consider the
133 <host id="A" speed="1Gf" />
134 <host id="B" speed="1Gf" />
136 <link id="link1" latency="10ms" bandwidth="1Mbps" />
138 <route src="A" dst="B">
139 <link_ctn id="link1" />
142 If host `A` sends `100kB` (a hundred kilobytes) to host `B`, one can expect that this communication would take `0.81`
143 seconds to complete according to a simple latency-plus-size-divided-by-bandwidth model (0.01 + 8e5/1e6 = 0.81).
144 However, the default TCP model of SimGrid is a bit more complex than that. It accounts for three phenomena that
145 directly impact the simulation time even on such a simple example:
147 - The size of a message at the application level (i.e., 100kB in this example) is not the size that is actually
148 transferred over the network. To mimic the fact that TCP and IP headers are added to each packet of the original
149 payload, the TCP model of SimGrid empirically considers that `only 97% of the nominal bandwidth` are available. In
150 other words, the size of your message is increased by a few percents, whatever this size be.
152 - In the real world, the TCP protocol is not able to fully exploit the bandwidth of a link from the emission of the
153 first packet. To reflect this `slow start` phenomenon, the latency declared in the platform file is multiplied by
154 `a factor of 13.01`. Here again, this is an empirically determined value that may not correspond to every TCP
155 implementations on every networks. It can be tuned when more realistic simulated times for the transfer of short
156 messages are needed though.
158 - When data is transferred from A to B, some TCP ACK messages travel in the opposite direction. To reflect the impact
159 of this `cross-traffic`, SimGrid simulates a flow from B to A that represents an additional bandwidth consumption
160 of `0.05%`. The route from B to A is implicitly declared in the platform file and uses the same link `link1` as if
161 the two hosts were connected through a communication bus. The bandwidth share allocated to a data transfer from A
162 to B is then the available bandwidth of `link1` (i.e., 97% of the nominal bandwidth of 1Mb/s) divided by 1.05
163 (i.e., the total consumption). This feature, activated by default, can be disabled by adding the
164 `--cfg=network/crosstraffic:0` flag to the command line.
166 As a consequence, the time to transfer 100kB from A to B as simulated by the default TCP model of SimGrid is not 0.81
169 .. code-block:: python
171 0.01 * 13.01 + 800000 / ((0.97 * 1e6) / 1.05) = 0.996079 seconds.
177 In SimGrid, WiFi networks are modeled with WiFi zones, where a zone contains the access point of the WiFi network and
178 the hosts connected to it (called `stations` in the WiFi world). The network inside a WiFi zone is modeled by declaring
179 a single regular link with a specific attribute. This link is then added to the routes to and from the stations within
180 this WiFi zone. The main difference of WiFi networks is that their performance is not determined by some link bandwidth
181 and latency but by both the access point WiFi characteristics and the distance between that access point and a given
184 Such WiFi zones can be used with the LMM-based model or ns-3, and are supposed to behave similarly in both cases.
186 Declaring a WiFi zone
187 ---------------------
189 To declare a new WiFi network, simply declare a network zone with the ``WIFI`` routing attribute.
193 <zone id="SSID_1" routing="WIFI">
195 Inside this zone you must declare which host or router will be the access point of the WiFi network.
199 <prop id="access_point" value="alice"/>
201 Then simply declare the stations (hosts) and routers inside the WiFi network. Remember that one must have the same name
202 as the "access point" property.
206 <router id="alice" speed="1Gf"/>
207 <host id="STA0-0" speed="1Gf"/>
208 <host id="STA0-1" speed="1Gf"/>
210 Finally, close the WiFi zone.
216 The WiFi zone may be connected to another zone using a traditional link and a zoneRoute. Note that the connection between two
217 zones is always wired.
221 <link id="wireline" bandwidth="100Mbps" latency="2ms" sharing_policy="SHARED"/>
223 <zoneRoute src="SSID_1" dst="SSID_2" gw_src="alice" gw_dst="bob">
224 <link_ctn id="wireline"/>
227 WiFi network performance
228 ------------------------
230 The performance of a wifi network is controlled by the three following properties:
232 * ``mcs`` (`Modulation and Coding Scheme <https://en.wikipedia.org/wiki/Link_adaptation>`_)
233 is a property of the WiFi zone. Roughly speaking, it defines the speed at which the access point is exchanging data
234 with all the stations. It depends on the access point's model and configuration. Possible values for the MCS can be
235 found on Wikipedia for example.
236 |br| By default, ``mcs=3``.
237 * ``nss`` (Number of Spatial Streams, or `number of antennas <https://en.wikipedia.org/wiki/IEEE_802.11n-2009#Number_of_antennas>`_) is another property of the WiFi zone. It defines the amount of simultaneous data streams that the access
238 point can sustain. Not all values of MCS and NSS are valid nor compatible (cf. `802.11n standard <https://en.wikipedia.org/wiki/IEEE_802.11n-2009#Data_rates>`_).
239 |br| By default, ``nss=1``.
240 * ``wifi_distance`` is the distance from a station to the access point. Each station can have its own specific value.
241 It is thus a property of the stations declared inside the WiFi zone.
242 |br| By default, ``wifi_distance=10``.
244 Here is an example of a zone with non-default ``mcs`` and ``nss`` values.
248 <zone id="SSID_1" routing="WIFI">
249 <prop id="access_point" value="alice"/>
250 <prop id="mcs" value="2"/>
251 <prop id="nss" value="2"/>
255 Here is an example of setting the ``wifi_distance`` of a given station.
259 <host id="STA0-0" speed="1Gf">
260 <prop id="wifi_distance" value="37"/>
266 SimGrid provides two other network models in addition to the LMM-based ones.
268 First, the **constant-time model** is a simplistic network model where all communication take a constant time (one
269 second). It provides the lowest level of realism, but is marginally faster and much simpler to understand. This model
270 may reveal interesting if you plan to study abstract distributed algorithms such as leader election or causal broadcast.
272 On the contrary, the **ns-3 based model** is the most accurate network model that you can get in SimGrid. It relies on
273 the well-known `ns-3 packet-level network simulator <http://www.nsnam.org>`_ to compute every timing information related
274 to the network transfers of your simulation.
275 For instance, this may be used to investigate the validity of a simulation. Note that this model is much slower than the
276 LMM-based models, because ns-3 simulates the movement of every network packet involved in every communication while
277 SimGrid only recomputes the respective instantaneous speeds of the currently ongoing communications when one
278 communication starts or stops.
281 Both simulators are linear in the size of their input, but ns-3 has a much larger input in case of large steady communications.
283 ns-3 as a SimGrid model
284 =======================
286 You need to install ns-3 and recompile SimGrid accordingly to use this model.
288 The SimGrid/ns-3 binding only contains features that are common to both systems. Not all ns-3 models are available from
289 SimGrid (only the TCP and WiFi ones are), while not all SimGrid platform files can be used in conjunction with ns-3
290 (routes must be of length 1). Also, the platform built in ns-3 from the SimGrid
291 description is very basic. Finally, communicating from a host to
292 itself is forbidden in ns-3, so every such communication completes
293 immediately upon startup.
296 Compiling the ns-3/SimGrid binding
297 ----------------------------------
302 SimGrid requires ns-3 version 3.26 or higher, and you probably want the most
303 recent version of both SimGrid and ns-3. While the Debian package of SimGrid
304 does not have the ns-3 bindings activated, you can still use the packaged version
305 of ns-3 by grabbing the ``libns3-dev ns3`` packages. Alternatively, you can
306 install ns-3 from scratch (see the `ns-3 documentation <http://www.nsnam.org>`_).
308 Enabling ns-3 in SimGrid
309 ^^^^^^^^^^^^^^^^^^^^^^^^
311 SimGrid must be recompiled with the ``enable_ns3`` option activated in cmake.
312 Optionally, use ``NS3_HINT`` to tell cmake where ns3 is installed on
315 .. code-block:: console
317 $ cmake . -Denable_ns3=ON -DNS3_HINT=/opt/ns3 # or change the path if needed
319 By the end of the configuration, cmake reports whether ns-3 was found,
320 and this information is also available in ``include/simgrid/config.h``
321 If your local copy defines the variable ``SIMGRID_HAVE_NS3`` to 1, then ns-3
322 was correctly detected. Otherwise, explore ``CMakeFiles/CMakeOutput.log`` and
323 ``CMakeFiles/CMakeError.log`` to diagnose the problem.
325 Test that ns-3 was successfully integrated with the following command (executed from your SimGrid
326 build directory). It will run all SimGrid tests that are related to the ns-3
327 integration. If no test is run at all, you probably forgot to enable ns-3 in cmake.
329 .. code-block:: console
336 If you use a version of ns-3 that is not known to SimGrid yet, edit
337 ``tools/cmake/Modules/FindNS3.cmake`` in your SimGrid tree, according to the
338 comments on top of this file. Conversely, if something goes wrong with an old
339 version of either SimGrid or ns-3, try upgrading everything.
341 Note that there is a known bug with the version 3.31 of ns3 when it is built with
342 MPI support, like it is with the libns3-dev package in Debian 11 « Bullseye ».
343 A simple workaround is to edit the file
344 ``/usr/include/ns3.31/ns3/point-to-point-helper.h`` to remove the ``#ifdef NS3_MPI``
345 include guard. This can be achieved with the following command (as root):
347 .. code-block:: console
349 # sed -i '/^#ifdef NS3_MPI/,+2s,^#,//&,' /usr/include/ns3.31/ns3/point-to-point-helper.h
353 Using ns-3 from SimGrid
354 -----------------------
356 Platform files compatibility
357 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
359 Any route longer than one will be ignored when using ns-3. They are
360 harmless, but you still need to connect your hosts using one-hop routes.
361 The best solution is to add routers to split your route. Here is an
362 example of an invalid platform:
366 <?xml version='1.0'?>
367 <!DOCTYPE platform SYSTEM "https://simgrid.org/simgrid.dtd">
368 <platform version="4.1">
369 <zone id="zone0" routing="Floyd">
370 <host id="alice" speed="1Gf" />
371 <host id="bob" speed="1Gf" />
373 <link id="l1" bandwidth="1Mbps" latency="5ms" />
374 <link id="l2" bandwidth="1Mbps" latency="5ms" />
376 <route src="alice" dst="bob">
377 <link_ctn id="l1"/> <!-- !!!! IGNORED WHEN USED WITH ns-3 !!!! -->
378 <link_ctn id="l2"/> <!-- !!!! ROUTES MUST CONTAIN ONE LINK ONLY !!!! -->
383 This can be reformulated as follows to make it usable with the ns-3 binding.
384 There is no direct connection from alice to bob, but that's OK because ns-3
385 automatically routes from point to point (using
386 ``ns3::Ipv4GlobalRoutingHelper::PopulateRoutingTables``).
390 <?xml version='1.0'?>
391 <!DOCTYPE platform SYSTEM "https://simgrid.org/simgrid.dtd">
392 <platform version="4.1">
393 <zone id="zone0" routing="Full">
394 <host id="alice" speed="1Gf" />
395 <host id="bob" speed="1Gf" />
397 <router id="r1" /> <!-- routers are compute-less hosts -->
399 <link id="l1" bandwidth="1Mbps" latency="5ms"/>
400 <link id="l2" bandwidth="1Mbps" latency="5ms"/>
402 <route src="alice" dst="r1">
406 <route src="r1" dst="bob">
412 Once your platform is OK, just change the :ref:`network/model
413 <options_model_select>` configuration option to `ns-3` as follows. The other
414 options can be used as usual.
416 .. code-block:: console
418 $ ./network-ns3 --cfg=network/model:ns-3 (other parameters)
420 Many other files from the ``examples/platform`` directory are usable with the
421 ns-3 model, such as `examples/platforms/dogbone.xml <https://framagit.org/simgrid/simgrid/tree/master/examples/platforms/dogbone.xml>`_.
422 Check the file `examples/cpp/network-ns3/network-ns3.tesh <https://framagit.org/simgrid/simgrid/tree/master/examples/cpp/network-ns3/network-ns3.tesh>`_
423 to see which ones are used in our regression tests.
425 Alternatively, you can manually modify the ns-3 settings by retrieving
426 the ns-3 node from any given host with the
427 :cpp:func:`simgrid::get_ns3node_from_sghost` function (defined in
428 ``simgrid/plugins/ns3.hpp``).
430 .. doxygenfunction:: simgrid::get_ns3node_from_sghost
434 It is possible to define a fixed or random seed to the ns3 random number generator using the config tag.
438 <?xml version='1.0'?><!DOCTYPE platform SYSTEM "https://simgrid.org/simgrid.dtd">
439 <platform version="4.1">
441 <prop id = "network/model" value = "ns-3" />
442 <prop id = "ns3/seed" value = "time" />
447 The first property defines that this platform will be used with the ns3 model.
448 The second property defines the seed that will be used. Defined to ``time``,
449 it will use a random seed, defined to a number it will use this number as
455 A ns-3 platform is automatically created from the provided SimGrid
456 platform. However, there are some known caveats:
458 * The default values (e.g., TCP parameters) are the ns-3 default values.
459 * ns-3 networks are routed using the shortest path algorithm, using ``ns3::Ipv4GlobalRoutingHelper::PopulateRoutingTables``.
460 * End hosts cannot have more than one interface card. So, your SimGrid hosts
461 should be connected to the platform through only one link. Otherwise, your
462 SimGrid host will be considered as a router (FIXME: is it still true?).
464 Our goal is to keep the ns-3 plugin of SimGrid as easy (and hopefully readable)
465 as possible. If the current state does not fit your needs, you should modify
466 this plugin, and/or create your own plugin from the existing one. If you come up
467 with interesting improvements, please contribute them back.
472 If your simulation hangs in a communication, this is probably because one host
473 is sending data that is not routable in your platform. Make sure that you only
474 use routes of length 1, and that any host is connected to the platform.
475 Arguably, SimGrid could detect this situation and report it, but unfortunately,
476 this still has to be done.