application. We will detail each part of the code and the necessary
configuration to make it work. After this tour, several exercises
are proposed to let you discover some of the SimGrid features, hands
-on the keyboard. This practical session will be given in C++ or Python,
+on the keyboard. This practical session will be given in C++ or Python,
which you are supposed to know beforehand.
.. image:: /tuto_s4u/img/intro.svg
:align: center
-The provided code dispatches these tasks in `round-robin scheduling <https://en.wikipedia.org/wiki/Round-robin_scheduling>`_,
+The provided code dispatches these tasks in `round-robin scheduling <https://en.wikipedia.org/wiki/Round-robin_scheduling>`_,
i.e. in circular order: tasks are dispatched to each worker one after the other, until all tasks are dispatched.
You will improve this scheme later in this tutorial.
.. group-tab:: Python
- That being said, an algorithm alone is not enough to define a simulation:
+ That being said, an algorithm alone is not enough to define a simulation:
you need a main block to setup the simulation and its components as follows.
This code creates a SimGrid simulation engine (on line 4), registers the actor
functions to the engine (on lines 7 and 8), loads the simulated platform
Using Docker
............
-The easiest way to take the tutorial is to use the dedicated Docker image.
+The easiest way to take the tutorial is to use the dedicated Docker image.
Once you `installed Docker itself <https://docs.docker.com/install/>`_, simply do the following:
.. code-block:: console
optional in this tutorial, it is not installed to reduce the image
size.
+The docker does not run as root, so that the files can easily be exchanged between within the container and the outer world.
+If you need to run a command as root within the container, simply type the following in another terminal to join the same container as root:
+
+.. code-block:: console
+
+ $ docker container ls
+ # This lists all containers running on your machine. For example:
+ # CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
+ # 7e921b1b18a7 simgrid/stable "bash" 7 minutes ago Up 7 minutes simgrid
+
+ $ docker exec --user root -it simgrid bash
+
The code template is available under ``/source/simgrid-template-s4u.git``
in the image. You should copy it to your working directory and
recompile it when you first log in:
.. code-block:: console
- $ cp -r /source/simgrid-template-s4u.git/* /source/tutorial
- $ cd /source/tutorial
- $ cmake .
- $ make
+ $ # Make sure the simgrid-tutorial directory can be read and written by the non-root user
+ $ sudo chown $UID:$GID ~/simgrid-tutorial
+ $ # Connect to the running container if needed
+ $ docker exec --user $UID:$GID -ti simgrid bash
+ $container) cp -r /source/simgrid-template-s4u.git/* /source/tutorial
+ $container) cd /source/tutorial
+ $container) cmake .
+ $container) make
Using your Computer Natively
............................
.. group-tab:: Python
To take the tutorial on your machine, you first need to :ref:`install
- a recent version of SimGrid <install>` and ``pajeng`` to visualize the
+ a recent version of SimGrid <install>` and ``pajeng`` to visualize the
traces. You may want to install `Vite <http://vite.gforge.inria.fr/>`_ to get a first glance at the traces.
On Debian and Ubuntu for example, you can get them as follows:
$ sudo apt install simgrid pajeng vite
- An initial version of the source code is provided on framagit.
+ An initial version of the source code is provided on framagit.
If SimGrid is correctly installed, you should be able to clone the `repository
<https://framagit.org/simgrid/simgrid-template-s4u>`_ and execute it as follows:
your :ref:`SimGrid installation <install>`. On need, please refer to
the :ref:`Troubleshooting your Project Setup <install_yours_troubleshooting>` section.
+.. warning::
+
+ If you use the stable version of Debian 11, Ubuntu 21.04 or Ubuntu 21.10, then you need the right version of this tutorial
+ (add ``--branch simgrid-v3.25`` as below). These distributions only contain SimGrid v3.25 while the latest version of this
+ tutorial needs at least SimGrid v3.27.
+
+ .. code-block:: console
+
+ $ git clone --branch simgrid-v3.25 https://framagit.org/simgrid/simgrid-template-s4u.git
For R analysis of the produced traces, you may want to install R
and the `pajengr <https://github.com/schnorr/pajengr#installation/>`_ package.
$ make master-workers
$ ./master-workers small_platform.xml master-workers_d.xml
+ If you get an error message complaining that ``simgrid::s4u::Mailbox::get()`` does not exist,
+ then your version of SimGrid is too old for the version of the tutorial that you got. Check again previous section.
+
.. group-tab:: Python
Please execute the provided simulator as follows:
$ python master-workers.py small_platform.xml master-workers_d.xml
-For a more "fancy" output, you can use simgrid-colorizer.
-
-.. code-block:: console
-
- # Run C++ code
- $ ./master-workers small_platform.xml master-workers_d.xml 2>&1 | simgrid-colorizer
-
- # Run Python code
- $ python master-workers.py small_platform.xml master-workers_d.xml 2>&1 | simgrid-colorizer
-
-If you installed SimGrid to a non-standard path, you may have to
-specify the full path to simgrid-colorizer on the above line, such as
-``/opt/simgrid/bin/simgrid-colorizer``. If you did not install it at all,
-you can find it in <simgrid_root_directory>/bin/colorize.
+ If you get an error stating that the simgrid module does not exist, you need to get a newer version of SimGrid.
+ You may want to take the tutorial from the docker to get the newest version.
For a classical Gantt-Chart visualization, you can use `Vite
<http://vite.gforge.inria.fr/>`_ if you have it installed, as
.. image:: /tuto_s4u/img/vite-screenshot.png
:align: center
-.. note::
-
- If you use an older version of SimGrid (before v3.26), you should use
- ``--cfg=tracing/msg/process:yes`` instead of ``--cfg=tracing/actor:yes``.
-
If you want the full power to visualize SimGrid traces, you need
to use R. As a start, you can download this `starter script
<https://framagit.org/simgrid/simgrid/raw/master/docs/source/tuto_s4u/draw_gantt.R>`_
.. rst-class:: compact-list
- **Learning goals:**
+ **Learning goals:**
* Get your hands on the code and change the communication pattern
* Discover the Mailbox mechanism
.. code-block:: cpp
for (int i = 0; i < tasks_count; i++) {
- std::string worker_rank = std::to_string(i % workers_count);
- std::string mailbox_name = std::string("worker-") + worker_rank;
+ std::string worker_rank = std::to_string(i % workers_count);
+ std::string mailbox_name = "worker-" + worker_rank;
simgrid::s4u::Mailbox* mailbox = simgrid::s4u::Mailbox::by_name(mailbox_name);
mailbox->put(...);
.. code-block:: cpp
- for i in range(tasks_count):
+ for i in range(tasks_count):
mailbox = Mailbox.by_name(str(i % worker_count))
mailbox.put(...)
initiators' location and then the real communication occurs between
the involved parties.
-Please refer to the full `Mailboxes' documentation <app_s4u.html#s4u-mailbox>`_
+Please refer to the full `Mailboxes' documentation <app_s4u.html#s4u-mailbox>`_
for more details.
.. rst-class:: compact-list
- **Learning goals:**
+ **Learning goals:**
* Interact with the platform (get the list of all hosts)
* Create actors directly from your program instead of the deployment file
.. group-tab:: Python
For that, the master needs to retrieve the list of hosts declared in
- the platform with :py:func:`simgrid.Engine.get_all_hosts`. Since this method is not static,
+ the platform with :py:func:`simgrid.Engine.get_all_hosts`. Since this method is not static,
you may want to call it on the Engine instance, as in ``Engine.instance().get_all_hosts()``.
Then, the master should start the worker actors with :py:func:`simgrid.Actor.create`.
.. rst-class:: compact-list
- **Learning goals:**
+ **Learning goals:**
* Forcefully kill actors, and stop the simulation at a given point of time
* Control the logging verbosity
Of course, usual time functions like ``gettimeofday`` will give you the
time on your real machine, which is pretty useless in the
simulation. Instead, retrieve the time in the simulated world with
-:cpp:func:`simgrid::s4u::Engine::get_clock` (C++) or
+:cpp:func:`simgrid::s4u::Engine::get_clock` (C++) or
:py:func:`simgrid.Engine.get_clock()`) (Python).
You can still stop your workers with a specific task as previously,
.................................
Not all messages are equally informative, so you probably want to
-change some of the *info* messages (C: :c:macro:`XBT_INFO`; Python: :py:func:`simgrid.this_actor.info`)
+change some of the *info* messages (C: :c:macro:`XBT_INFO`; Python: :py:func:`simgrid.this_actor.info`)
into *debug* messages`(C: :c:macro:`XBT_DEBUG`; Python: :py:func:`simgrid.this_actor.debug`) so that they are
hidden by default. For example, you may want to use an *info* message once
every 100 tasks and *debug* when sending all the other tasks. Or
.. rst-class:: compact-list
- **Learning goals:**
+ **Learning goals:**
* Change the platform characteristics during the simulation.
* Explore other communication patterns.
...................
Attach a profile to your hosts, so that their computational speed automatically vary over time, modeling an external load on these machines.
-This can be done with :cpp:func:`simgrid::s4u::Host::set_speed_profile` (C++) or :py:func:`simgrid.Host.set_speed_profile` (Python).
+This can be done with :cpp:func:`simgrid::s4u::Host::set_speed_profile` (C++) or :py:func:`simgrid.Host.set_speed_profile` (Python).
Make it so that one of the hosts get really really slow, and observe how your whole application performance decreases.
-This is because one slow host slows down the whole process. Instead of a round-robin dispatch push,
+This is because one slow host slows down the whole process. Instead of a round-robin dispatch push,
you should completely reorganize your application in a First-Come First-Served manner (FCFS).
Actors should pull a task whenever they are ready, so that fast actors can overpass slow ones in the queue.
to what would happen with communications based on BSD sockets while the second is closer to message queues. You could also decide to
model your socket application in the second manner if you want to neglect these details and keep your simulator simple. It's your decision.
-Changing the communication schema can be a bit hairy, but once it works, you will see that such as simple FCFS schema allows one to greatly
+Changing the communication schema can be a bit hairy, but once it works, you will see that such as simple FCFS schema allows one to greatly
increase the amount of tasks handled over time here. Things may be different with another platform file.
Communication speed
You can even have the bandwidth automatically vary over time with :cpp:func:`simgrid::s4u::Link::set_bandwidth_profile` (C++) or :py:func:`simgrid.Link.set_bandwidth_profile` (python).
Once implemented, you will notice that slow communications may still result in situations
-where one worker only works at a given point of time. To overcome that, your master needs
-to send data to several workers in parallel, using
+where one worker only works at a given point of time. To overcome that, your master needs
+to send data to several workers in parallel, using
:cpp:func:`simgrid::s4u::Mailbox::put_async` (C++) or :py:func:`simgrid.Mailbox.put_async` (Python)
to start several communications in parallel, and
-:cpp:func:`simgrid::s4u::Comm::wait_any` (C++) or and :py:func:`simgrid.Comm.wait_any` (Python)
-to react to the completion of one of these communications. Actually, since this code somewhat tricky
-to write, it's provided as :ref:`an example <s4u_ex_communication>` in the distribution (search for
-``wait_any`` in that page).
+:cpp:func:`simgrid::s4u::Comm::wait_any` (C++) or and :py:func:`simgrid.Comm.wait_any` (Python)
+to react to the completion of one of these communications. Actually, since this code somewhat tricky
+to write, it's provided as :ref:`an example <s4u_ex_communication>` in the distribution (search for
+``wait_any`` in that page).
Dealing with failures
.....................
-Turn a given link off with :cpp:func:`simgrid::s4u::Link::turn_off` (C++) or :py:func:`simgrid.Link.turn_off` (python).
-You can even implement churn where a link automatically turn off and on again over time with :cpp:func:`simgrid::s4u::Link::set_state_profile` (C++) or :py:func:`simgrid.Link.set_state_profile` (python).
+Turn a given link off with :cpp:func:`simgrid::s4u::Link::turn_off` (C++) or :py:func:`simgrid.Link.turn_off` (python).
+You can even implement churn where a link automatically turn off and on again over time with :cpp:func:`simgrid::s4u::Link::set_state_profile` (C++) or :py:func:`simgrid.Link.set_state_profile` (python).
-If a link fails while you try to use it, ``wait()`` will raise a ``NetworkFailureException`` that you need to catch.
+If a link fails while you try to use it, ``wait()`` will raise a ``NetworkFailureException`` that you need to catch.
Again, there is a nice example demoing this feature, :ref:`under platform-failures <s4u_ex_communication>`.
Lab 5: Competing Applications
.. rst-class:: compact-list
- **Learning goals:**
+ **Learning goals:**
* Advanced vizualization through tracing categories
Instead of starting the execution in one function call only with
``this_actor::execute(cost)``, you need to
-create the execution activity, set its tracing category, start it
+create the execution activity, set its tracing category, start it
and wait for its completion, as follows.
.. tabs::
This tutorial is now terminated. You could keep reading the online documentation and
tutorials, or you could head up to the :ref:`example section <s4u_examples>` to read some code.
-.. todo::
-
- Things to improve in the future:
-
- - Propose equivalent exercises and skeleton in Java once we fix the Java binding.
-
.. |br| raw:: html
<br />