- os: windows
script:
- mv "C:/Program Files/Git/usr/bin/sh.exe" "sh-ignored.exe"
+ - df -h
+ - du -hs /tmp || true
- choco install boost-msvc-12 python jdk8
- export CC=gcc
- export CXX=g++
- export BOOST_INCLUDEDIR='C:/local/boost_1_58_0/include'
- export JAVA_HOME='C:/Program Files/Java/jdk1.8.0_211'
- cmake -G "MinGW Makefiles" -Denable_lua=OFF -Denable_documentation=OFF -Denable_java=ON -Denable_smpi=OFF -Denable_mallocators=OFF -Denable_lto=OFF .
- - mingw32-make.exe VERBOSE=1 java-all && ctest --output-on-failure -R java
+ - df -h
+ - du -hs /tmp || true
+ - mingw32-make.exe VERBOSE=1 java-all
+ - df -h
+ - du -hs /tmp || true
+ - ctest --output-on-failure -R java
+ - df -h
+ - du -hs /tmp || true
+# - mingw32-make.exe VERBOSE=1 java-all && ctest --output-on-failure -R java
- os: osx
osx_image: xcode11
script:
#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#-#
set(SIMGRID_VERSION_MAJOR "3")
-set(SIMGRID_VERSION_MINOR "23")
-set(SIMGRID_VERSION_PATCH "3") # odd => git branch; even => stable release or released snapshot
+set(SIMGRID_VERSION_MINOR "24")
+set(SIMGRID_VERSION_PATCH "0") # odd => git branch; even => stable release or released snapshot
set(SIMGRID_VERSION_DATE "2019") # Year for copyright information
----------------------------------------------------------------------------
-SimGrid (3.23.3) NOT RELEASED YET (v3.24 expected September 23. 7:50 UTC)
+SimGrid (3.24) October 9. 2019
+
+The Clean Disk Release.
S4U:
- Introduce a s4u::Disk interface to manage the newly introduced <disk>
MPI_C_COMPILER, MPI_CXX_COMPILER, MPI_Fortran_COMPILER variables.
- Add support for MPI Errhandlers in Comm, File or Win. Default errhandler is now
MPI_ERRORS_ARE_FATAL, so codes which were sending warnings may start failing.
- - trace-call-location can now be used with TI traces, and replayed.
+ - trace-call-location can be used with TI traces, and replayed, and is compatible
+ with smpi/comp-adjustment-file.
+ - sleep events are now correctly traced/replayed.
- Default for trace-call-location is now to use file names and not full paths.
To revert to previous behaviour (in case of collision of filenames), option
"smpi/trace-call-use-absolute-path" can be set to yes.
only three attributes (id, read_bw, and write_bw). All the other information
that was declared with the storage related tags now has to be expressed as
properties. An example of platform using this new tag is available at
- examples/platforms/hostsè_with_disks.xml
+ examples/platforms/hosts_with_disks.xml
+
+tesh:
+ - 'expect signal' can now accept more than one potential signal.
Fixed bugs (FG#.. -> framagit bugs; FG!.. -> framagit merge requests):
- FG#28: add sg_actor_self (and other wrappers on this_actor methods)
- FG!16: Fix the ns-3 bindings when several flows are simultaneously finishing
- FG!17: ns-3: unblock the right number of communications + others issues
- FG!18: Improving the performance of the ns-3 bindings
+ - GH#189: Energy consumption of parallel tasks
- GH#219: Error in the throughput of TCP transfer
- GH!330: Minor polishing of English: Configuring_Simgrid
- GH!331: Fix a doc error about actors (Tutorial_algorithms)
- GH!332: Add '-O0' flag for: cmake -DCMAKE_BUILD_TYPE=Debug
+ - GH!334: Rework energy plugin, again
----------------------------------------------------------------------------
include examples/smpi/smpi_msg_masterslave/msg_smpi.tesh
include examples/smpi/trace/trace.c
include examples/smpi/trace/trace.tesh
+include examples/smpi/trace_call_location/adjust
include examples/smpi/trace_call_location/trace_call_location.c
include examples/smpi/trace_call_location/trace_call_location.tesh
include examples/smpi/trace_simple/trace_simple.c
include tools/normalize-pointers.py
include tools/pkg-config/simgrid.pc.in
include tools/sanitizers.supp
-include tools/sg_xml_energy_ponecore_to_pepsilon.py
include tools/sg_xml_unit_converter.py
include tools/simgrid.supp
include tools/simgrid2vite.sed
include docs/source/Installing_SimGrid.rst
include docs/source/Introduction.rst
include docs/source/Platform_Examples.rst
+include docs/source/Plugins.rst
include docs/source/Start_Your_Own_Project.rst
include docs/source/Tutorial_Algorithms.rst
include docs/source/Tutorial_MPI_Applications.rst
\ \ / / _ \ '__/ __| |/ _ \| '_ \ |_ \ __) | || |_
\ V / __/ | \__ \ | (_) | | | | ___) | / __/|__ _|
\_/ \___|_| |___/_|\___/|_| |_| |____(_)_____| |_|
- (not released yet)
+ October 9. 2019
The Clean Disk Release.
`!' metacommand, which can be one of:
`timeout' <integer>|no
- `expect signal' <signal name>
+ `expect signal' <signal name>[|<signal name>]*
`expect return' <integer>
`output' <ignore|display>
`output sort' [integer]
Several ``--cfg`` command line arguments can naturally be used. If you
need to include spaces in the argument, don't forget to quote the
-argument. You can even escape the included quotes (write @' for ' if
-you have your argument between ').
+argument. You can even escape the included quotes (write ``@'`` for ``'`` if
+you have your argument between simple quotes).
Another solution is to use the ``<config>`` tag in the platform file. The
only restriction is that this tag must occur before the first
the host energy plugin by adding ``--cfg=plugin:host_energy`` to your
command line.
-Here is the full list of plugins that can be activated this way:
+Here is a partial list of plugins that can be activated this way. You can get
+the full list by passing ``--cfg=plugin:help`` to your simulator.
- - **host_energy:** keeps track of the energy dissipated by
- computations. More details in @ref plugin_energy.
- - **link_energy:** keeps track of the energy dissipated by
- communications. More details in @ref SURF_plugin_energy.
- - **host_load:** keeps track of the computational load.
- More details in @ref plugin_load.
+ - :ref:`Host Energy <plugin_host_energy>`: models the energy dissipation of the compute units.
+ - :ref:`Link Energy <plugin_link_energy>`: models the energy dissipation of the network.
+ - :ref:`Host Load <plugin_host_load>`: monitors the load of the compute units.
.. _options_modelchecking:
If you want to specify liveness properties, you have to pass them on
the command line, specifying the name of the file containing the
-property, as formatted by the ltl2ba program. Note that ltl2ba is not
-part of SimGrid and must be installed separatly.
+property, as formatted by the `ltl2ba <https://github.com/utwente-fmt/ltl2ba>`_ program.
+Note that ltl2ba is not part of SimGrid and must be installed separatly.
.. code-block:: shell
communication determinism mode of the model checker, which checks
determinism properties of the communications of an application.
+.. _options_mc_perf:
+
Verification Performance Considerations
.......................................
consumption when using model-checking. By default, each snapshot will
save a copy of the whole stacks and not only of the part that is
really meaningful: you should expect the contribution of the memory
-consumption of the snapshots to be @f$ @mbox{number of processes}
-@times @mbox{stack size} @times @mbox{number of states} @f$.
+consumption of the snapshots to be:
+:math:`\text{number of processes} \times \text{stack size} \times \text{number of states}`.
When compiled against the model checker, the stacks are not
protected with guards: if the stack size is too small for your
execution path. All options (but the model checker related ones) must
remain the same. In particular, if you ran your application with
``smpirun -wrapper simgrid-mc``, then do it again. Remove all
-MC-related options, keep the other ones and add
-``--cfg=model-check/replay``.
+MC-related options, keep non-MC-related ones and add
+``--cfg=model-check/replay:???``.
Currently, if the path is of the form ``X;Y;Z``, each number denotes
the actor's pid that is selected at each indecision point. If it's of
to **no**. This option just ignores the timings in your simulation; it
still executes the computations itself. If you want to stop SMPI from
doing that, you should check the SMPI_SAMPLE macros, documented in
-Section :ref:`SMPI_adapting_speed`.
+Section :ref:`SMPI_use_faster`.
+------------------------------------+-------------------------+-----------------------------+
| Solution | Computations executed? | Computations simulated? |
only consume 1 MiB in memory.
You can disable this behavior and come back to regular mallocs (for
-example for debugging purposes) using @c "no" as a value.
+example for debugging purposes) using ``no`` as a value.
If you want to keep private some parts of the buffer, for instance if these
parts are used by the application logic and should not be corrupted, you
.. _plugins:
+SimGrid Plugins
+###############
+
.. raw:: html
<object id="TOC" data="graphical-toc.svg" width="100%" type="image/svg+xml"></object>
<br/>
<br/>
-SimGrid Plugins
-###############
-
You can extend SimGrid without modifying it, thanks to our plugin
mechanism. This page describes how to write your own plugin, and
-documents some of the plugins distributed with SimGrid.
+documents some of the plugins distributed with SimGrid:
+
+ - :ref:`Host Energy <plugin_host_energy>`: models the energy dissipation of the compute units.
+ - :ref:`Link Energy <plugin_link_energy>`: models the energy dissipation of the network.
+ - :ref:`Host Load <plugin_host_load>`: monitors the load of the compute units.
-- :ref:`Host Energy <plugin_host_energy>`: models the energy dissipation of the compute units.
-- :ref:`Link Energy <plugin_link_energy>`: models the energy dissipation of the network.
-- :ref:`Host Load <plugin_host_load>`: monitors the load of the compute units.
+You can activate these plugins with the :ref:`--cfg=plugin <cfg=plugin>` command
+line option, for example with ``--cfg=plugin:host_energy``. You can get the full
+list of existing plugins with ``--cfg=plugin:help``.
Defining a Plugin
*****************
This feature is demoed by the example file
`examples/smpi/NAS/dt.c <https://framagit.org/simgrid/simgrid/tree/master/examples/smpi/NAS/dt.c>`_
+.. _SMPI_use_faster:
+
.........................
Toward Faster Simulations
.........................
author = u'The SimGrid Team'
# The short X.Y version
-version = u'3.23.3'
-# The full version, including alpha/beta/rc tags
-#release = u'3.23 alpha'
+version = u'3.24'
# -- General configuration ---------------------------------------------------
Msg.info("Send a request to the coordinator");
req.send("coordinator");
Msg.info("Wait for a grant from the coordinator");
- GrantTask.receive(getName());
+ Task.receive(getName());
Task compute = new Task("CS", csTime, 0);
compute.execute();
ReleaseTask release = new ReleaseTask();
<platform version="4.1">
<zone id="AS0" routing="Full">
<host id="bob" speed="1Gf">
+ <prop id="ram" value="100B" />
<disk id="Disk1" read_bw="100MBps" write_bw="40MBps">
<prop id="size" value="500GiB"/>
<prop id="mount" value="/scratch"/>
<disk id="Disk1" read_bw="200MBps" write_bw="80MBps">
<prop id="content" value="storage/content/small_content.txt"/>
</disk>
+ <prop id="ram" value="100B" />
</host>
<host id="carl" speed="1Gf">
simgrid::s4u::Engine e(&argc, argv);
e.load_platform(argv[1]);
+ /* - Display Host properties */
+ for (auto h : e.get_all_hosts()) {
+ XBT_INFO("*** %s properties ****", h->get_cname());
+ for (auto kv : *h->get_properties())
+ XBT_INFO(" %s -> %s", kv.first.c_str(), kv.second.c_str());
+ }
+
simgrid::s4u::Actor::create("", simgrid::s4u::Host::by_name("bob"), host);
e.run();
#!/usr/bin/env tesh
$ ${bindir}/s4u-io-disk-raw ${platfdir}/hosts_with_disks.xml "--log=root.fmt:[%10.6r]%e(%i:%P@%h)%e%m%n"
+> [ 0.000000] (0:maestro@) *** alice properties ****
+> [ 0.000000] (0:maestro@) ram -> 100B
+> [ 0.000000] (0:maestro@) *** bob properties ****
+> [ 0.000000] (0:maestro@) ram -> 100B
+> [ 0.000000] (0:maestro@) *** carl properties ****
+> [ 0.000000] (0:maestro@) remote_disk -> /scratch:Disk1:bob
> [ 0.000000] (1:@bob) *** Storage info on bob ***
> [ 0.000000] (1:@bob) Disk name: Disk1
> [ 0.000000] (1:@bob) Disk name: Disk2
XBT_PUBLIC void sg_host_energy_update_all();
XBT_PUBLIC double sg_host_get_consumed_energy(sg_host_t host);
XBT_PUBLIC double sg_host_get_idle_consumption(sg_host_t host);
+XBT_PUBLIC double sg_host_get_idle_consumption_at(sg_host_t host, int pstate);
XBT_PUBLIC double sg_host_get_wattmin_at(sg_host_t host, int pstate);
XBT_PUBLIC double sg_host_get_wattmax_at(sg_host_t host, int pstate);
XBT_PUBLIC double sg_host_get_power_range_slope_at(sg_host_t host, int pstate);
#define MSG_host_energy_plugin_init() sg_host_energy_plugin_init()
#define MSG_host_get_consumed_energy(host) sg_host_get_consumed_energy(host)
+#define MSG_host_get_idle_consumption_at(host,pstate) sg_host_get_idle_consumption_at((host), (pstate))
#define MSG_host_get_wattmin_at(host,pstate) sg_host_get_wattmin_at((host), (pstate))
#define MSG_host_get_wattmax_at(host,pstate) sg_host_get_wattmax_at((host), (pstate))
#define MSG_host_get_power_range_slope_at(host,pstate) sg_host_get_power_range_slope_at((host), (pstate))
double bound_ = 0.0;
double timeout_ = 0.0;
std::atomic_int_fast32_t refcount_{0};
- Host* host_ = nullptr;
protected:
Exec();
Exec* start() override;
ExecPtr set_host(Host* host) override;
- Host* get_host();
double get_remaining() override;
double get_remaining_ratio() override;
setup(
name='simgrid',
- version='3.23.3',
+ version='3.24',
author='Da SimGrid Team',
author_email='simgrid-devel@lists.gforge.inria.fr',
description='Toolkit for scalable simulation of distributed applications',
sonar.organization=simgrid
sonar.projectKey=simgrid_simgrid
sonar.projectName=SimGrid
-sonar.projectVersion=3.23.3
+sonar.projectVersion=3.24
sonar.links.homepage=https://simgrid.org
sonar.links.issue=https://framagit.org/simgrid/simgrid/issues
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Path;
+import java.util.stream.Stream;
/** Helper class loading the native functions of SimGrid that we use for downcalls
*
}
@Override
public void run() {
- try {
- for (File f : dir.listFiles())
- if (! f.delete() && !f.getAbsolutePath().contains("travis") && !f.getAbsolutePath().contains("appveyor")) // Be silent on Travis to not break the tests. Ugly trick :)
- System.out.println("Unable to clean temporary file "+f.getAbsolutePath()+" during shutdown.");
- if (! dir.delete() && !dir.getAbsolutePath().contains("travis") && !dir.getAbsolutePath().contains("appveyor"))
- System.out.println("Unable to clean temporary file "+dir.getAbsolutePath()+" during shutdown.");
+ try (Stream<Path> paths = Files.walk(dir.toPath())) {
+ paths.sorted(java.util.Comparator.reverseOrder())
+ .map(java.nio.file.Path::toFile)
+ //.peek(System.out::println) // Prints what gets removed
+ .forEach(java.io.File::delete);
} catch(Exception e) {
System.out.println("Error while cleaning temporary file "+dir.getAbsolutePath()+" during shutdown: "+e.getCause());
e.printStackTrace();
- }
+ }
}
}
}
public native void migrate(Host host);
/**
* Makes the current process sleep until millis milliseconds have elapsed.
- * You should note that unlike "waitFor" which takes seconds, this method takes milliseconds.
- * FIXME: Not optimal, maybe we should have two native functions.
+ * You should note that unlike "waitFor" which takes seconds (as usual in SimGrid), this method takes milliseconds (as usual for sleep() in Java).
+ *
* @param millis the length of time to sleep in milliseconds.
*/
public static void sleep(long millis) throws HostFailureException {
/** @defgroup plugin_host_energy
+ @rst
This is the energy plugin, enabling to account not only for computation time, but also for the dissipated energy in the
simulated platform.
-To activate this plugin, first call sg_host_energy_plugin_init() before your #MSG_init(), and then use
-MSG_host_get_consumed_energy() to retrieve the consumption of a given host.
+To activate this plugin, first call :cpp:func:`sg_host_energy_plugin_init()` before your :cpp:func:`MSG_init()`, and then use
+:cpp:func:`MSG_host_get_consumed_energy()` to retrieve the consumption of a given host.
When the host is on, this energy consumption naturally depends on both the current CPU load and the host energy profile.
According to our measurements, the consumption is somehow linear in the amount of cores at full speed, with an
-abnormality when all the cores are idle. The full details are in
-<a href="https://hal.inria.fr/hal-01523608">our scientific paper</a> on that topic.
+abnormality when all the cores are idle. The full details are in `our scientific paper <https://hal.inria.fr/hal-01523608>`_
+on that topic.
As a result, our energy model takes 4 parameters:
- - @b Idle: wattage (i.e., instantaneous consumption in Watt) when your host is up and running, but without anything to
-do.
- - @b Epsilon: wattage when all cores are at 0 or epsilon%, but not in Idle state.
- - @b AllCores: wattage when all cores of the host are at 100%.
- - @b Off: wattage when the host is turned off.
+ - ``Idle`` wattage (i.e., instantaneous consumption in Watt) when your host is up and running, but without anything to do.
+ - ``Epsilon`` wattage when all cores are at 0 or epsilon%, but not in Idle state.
+ - ``AllCores`` wattage when all cores of the host are at 100%.
+ - ``Off`` wattage when the host is turned off.
Here is an example of XML declaration:
-@code{.xml}
-<host id="HostA" speed="100.0Mf" core="4">
- <prop id="wattage_per_state" value="100.0:120.0:200.0" />
- <prop id="wattage_off" value="10" />
-</host>
-@endcode
+.. code-block:: xml
-If the 'Epsilon' parameter is omitted in the XML declaration, 'Idle' is used instead.
+ <host id="HostA" speed="100.0Mf" core="4">
+ <prop id="wattage_per_state" value="100.0:120.0:200.0" />
+ <prop id="wattage_off" value="10" />
+ </host>
-This example gives the following parameters: @b Off is 10 Watts; @b Idle is 100 Watts; @b Epsilon is 120 Watts and @b
-AllCores is 200 Watts.
+If only two values are given, ``Idle`` is used for the missing ``Epsilon`` value.
+
+This example gives the following parameters: ``Off`` is 10 Watts; ``Idle`` is 100 Watts; ``Epsilon`` is 120 Watts and
+``AllCores`` is 200 Watts.
This is enough to compute the wattage as a function of the amount of loaded cores:
-<table>
-<tr><th>@#Cores loaded</th><th>Wattage</th><th>Explanation</th></tr>
-<tr><td>0 (idle)</td><td> 100 Watts</td><td>Idle value</td></tr>
-<tr><td>0 (not idle)</td><td> 120 Watts</td><td>Epsilon value</td></tr>
-<tr><td>1</td><td> 140 Watts</td><td>linear extrapolation between Epsilon and AllCores</td></tr>
-<tr><td>2</td><td> 160 Watts</td><td>linear extrapolation between Epsilon and AllCores</td></tr>
-<tr><td>3</td><td> 180 Watts</td><td>linear extrapolation between Epsilon and AllCores</td></tr>
-<tr><td>4</td><td> 200 Watts</td><td>AllCores value</td></tr>
-</table>
+.. raw:: html
+
+ <table border="1">
+ <tr><th>#Cores loaded</th><th>Wattage</th><th>Explanation</th></tr>
+ <tr><td>0 (idle)</td><td> 100 Watts </td><td> Idle value</td></tr>
+ <tr><td>0 (not idle)</td><td> 120 Watts</td><td> Epsilon value</td></tr>
+ <tr><td>1</td><td> 140 Watts</td><td> Linear extrapolation between Epsilon and AllCores</td></tr>
+ <tr><td>2</td><td> 160 Watts</td><td> Linear extrapolation between Epsilon and AllCores</td></tr>
+ <tr><td>3</td><td> 180 Watts</td><td> Linear extrapolation between Epsilon and AllCores</td></tr>
+ <tr><td>4</td><td> 200 Watts</td><td> AllCores value</td></tr>
+ </table>
-### How does DVFS interact with the host energy model?
+.. raw:: html
+
+ <h4>How does DVFS interact with the host energy model?</h4>
If your host has several DVFS levels (several pstates), then you should give the energetic profile of each pstate level:
-@code{.xml}
-<host id="HostC" speed="100.0Mf,50.0Mf,20.0Mf" core="4">
- <prop id="wattage_per_state" value="95.0:120.0:200.0, 93.0:115.0:170.0, 90.0:110.0:150.0" />
- <prop id="wattage_off" value="10" />
-</host>
-@endcode
-
-This encodes the following values
-<table>
-<tr><th>pstate</th><th>Performance</th><th>Idle</th><th>Epsilon</th><th>AllCores</th></tr>
-<tr><td>0</td><td>100 Mflop/s</td><td>95 Watts</td><td>120 Watts</td><td>200 Watts</td></tr>
-<tr><td>1</td><td>50 Mflop/s</td><td>93 Watts</td><td>115 Watts</td><td>170 Watts</td></tr>
-<tr><td>2</td><td>20 Mflop/s</td><td>90 Watts</td><td>110 Watts</td><td>150 Watts</td></tr>
-</table>
+.. code-block:: xml
+
+ <host id="HostC" speed="100.0Mf,50.0Mf,20.0Mf" core="4">
+ <prop id="wattage_per_state"
+ value="95.0:120.0:200.0, 93.0:115.0:170.0, 90.0:110.0:150.0" />
+ <prop id="wattage_off" value="10" />
+ </host>
+
+This encodes the following values:
+
+.. raw:: html
+
+ <table border="1">
+ <tr><th>pstate</th><th>Performance</th><th>Idle</th><th>Epsilon</th><th>AllCores</th></tr>
+ <tr><td>0</td><td>100 Mflop/s</td><td>95 Watts</td><td>120 Watts</td><td>200 Watts</td></tr>
+ <tr><td>1</td><td>50 Mflop/s</td><td>93 Watts</td><td>115 Watts</td><td>170 Watts</td></tr>
+ <tr><td>2</td><td>20 Mflop/s</td><td>90 Watts</td><td>110 Watts</td><td>150 Watts</td></tr>
+ </table>
To change the pstate of a given CPU, use the following functions:
-#MSG_host_get_nb_pstates(), simgrid#s4u#Host#setPstate(), #MSG_host_get_power_peak_at().
+:cpp:func:`MSG_host_get_nb_pstates()`, :cpp:func:`simgrid::s4u::Host::set_pstate()`, :cpp:func:`MSG_host_get_power_peak_at()`.
+
+.. raw:: html
-### How accurate are these models?
+ <h4>How accurate are these models?</h4>
This model cannot be more accurate than your instantiation: with the default values, your result will not be accurate at
all. You can still get accurate energy prediction, provided that you carefully instantiate the model.
The first step is to ensure that your timing prediction match perfectly. But this is only the first step of the path,
-and you really want to read <a href="https://hal.inria.fr/hal-01523608">this paper</a> to see all what you need to do
+and you really want to read `this paper <https://hal.inria.fr/hal-01523608>`_ to see all what you need to do
before you can get accurate energy predictions.
+
+ @endrst
*/
XBT_LOG_NEW_DEFAULT_SUBCATEGORY(surf_energy, surf, "Logging specific to the SURF energy plugin");
double get_current_watts_value();
double get_current_watts_value(double cpu_load);
double get_consumed_energy();
- double get_idle_consumption();
+ double get_watt_idle_at(int pstate);
double get_watt_min_at(int pstate);
double get_watt_max_at(int pstate);
double get_power_range_slope_at(int pstate);
HostEnergy::~HostEnergy() = default;
-double HostEnergy::get_idle_consumption()
+double HostEnergy::get_watt_idle_at(int pstate)
{
xbt_assert(not power_range_watts_list_.empty(), "No power range properties specified for host %s",
host_->get_cname());
-
- return power_range_watts_list_[0].idle_;
+ return power_range_watts_list_[pstate].idle_;
}
double HostEnergy::get_watt_min_at(int pstate)
double HostEnergy::get_power_range_slope_at(int pstate)
{
- xbt_assert(not power_range_watts_list_.empty(), "No power range properties specified for host %s",
- host_->get_cname());
- return power_range_watts_list_[pstate].slope_;
+ xbt_assert(not power_range_watts_list_.empty(), "No power range properties specified for host %s",
+ host_->get_cname());
+ return power_range_watts_list_[pstate].slope_;
}
/** @brief Computes the power consumed by the host according to the current situation
/* Divide by the number of cores here to have a value between 0 and 1 */
cpu_load /= host_->pimpl_cpu->get_core_count();
- if (cpu_load > 1) // A machine with a load > 1 consumes as much as a fully loaded machine, not more
- cpu_load = 1;
+ if (cpu_load > 1) // This condition is true for energy_ptask on 32 bits, even if cpu_load is displayed as 1.000000
+ cpu_load = 1; // That may be an harmless rounding error?
if (cpu_load > 0)
host_was_used_ = true;
}
std::vector<std::string> all_power_values;
boost::split(all_power_values, old_prop, boost::is_any_of(","));
- std::string msg = std::string("DEPRECATION WARNING: Property 'watt_per_state' will not work after v3.28.\n");
+ xbt_assert(all_power_values.size() == (unsigned)host_->get_pstate_count(),
+ "Invalid XML file. Found %zu energetic profiles for %d pstates", all_power_values.size(),
+ host_->get_pstate_count());
+
+ // XBT_ATTRIB_DEPRECATED_v327: puting this macro name here so that we find it during the deprecation cleanups of v3.28
+ std::string msg = std::string("DEPRECATION WARNING: Property 'watt_per_state' will only work until v3.28.\n");
msg += std::string("The old syntax 'Idle:OneCore:AllCores' must be converted into 'Idle:Epsilon:AllCores' to "
"properly model the consumption of non-whole tasks on mono-core hosts. Here are the values to "
"use for host '") +
double p_full;
double p_epsilon;
- if (current_power_values.size() == 2) { // Case: Idle:AllCores
- p_full = xbt_str_parse_double((current_power_values.at(1)).c_str(),
- "Invalid obsolete XML file. Fix your watt_per_state property.");
- p_epsilon = p_full;
- } else { // Case: Idle:Epsilon:AllCores
+ if (current_power_values.size() == 3) {
p_one_core = xbt_str_parse_double((current_power_values.at(1)).c_str(),
"Invalid obsolete XML file. Fix your watt_per_state property.");
p_full = xbt_str_parse_double((current_power_values.at(2)).c_str(),
"Invalid obsolete XML file. Fix your watt_per_state property.");
- if (host_->get_core_count() == 1)
+ if (host_->get_core_count() == 1) {
p_epsilon = p_full;
- else
+ } else {
p_epsilon = p_one_core - ((p_full - p_one_core) / (host_->get_core_count() - 1));
+ }
+ } else { // consuption given with idle and full only
+ p_full = xbt_str_parse_double((current_power_values.at(1)).c_str(),
+ "Invalid obsolete XML file. Fix your watt_per_state property.");
+ if (host_->get_core_count() == 1) {
+ p_epsilon = p_full;
+ } else {
+ p_epsilon = p_idle;
+ }
}
+
PowerRange range(p_idle, p_epsilon, p_full);
power_range_watts_list_.push_back(range);
XBT_WARN("%s", msg.c_str());
return;
}
+
const char* all_power_values_str = host_->get_property("wattage_per_state");
- if (all_power_values_str == nullptr)
+ if (all_power_values_str == nullptr) {
+ /* If no power values are given, we assume it's 0 everywhere */
+ XBT_DEBUG("No energetic profiles given for host %s, using 0 W by default.", host_->get_cname());
+ for (int i = 0; i < host_->get_pstate_count(); ++i) {
+ PowerRange range(0,0,0);
+ power_range_watts_list_.push_back(range);
+ }
return;
+ }
std::vector<std::string> all_power_values;
boost::split(all_power_values, all_power_values_str, boost::is_any_of(","));
XBT_DEBUG("%s: power properties: %s", host_->get_cname(), all_power_values_str);
+ xbt_assert(all_power_values.size() == (unsigned)host_->get_pstate_count(),
+ "Invalid XML file. Found %zu energetic profiles for %d pstates", all_power_values.size(),
+ host_->get_pstate_count());
+
int i = 0;
for (auto const& current_power_values_str : all_power_values) {
/* retrieve the power values associated with the pstate i */
{
xbt_assert(HostEnergy::EXTENSION_ID.valid(),
"The Energy plugin is not active. Please call sg_host_energy_plugin_init() during initialization.");
- return host->extension<HostEnergy>()->get_idle_consumption();
+ return host->extension<HostEnergy>()->get_watt_idle_at(0);
}
/** @ingroup plugin_host_energy
* @brief Get the amount of watt dissipated at the given pstate when the host is idling
*/
+double sg_host_get_idle_consumption_at(sg_host_t host, int pstate)
+{
+ xbt_assert(HostEnergy::EXTENSION_ID.valid(),
+ "The Energy plugin is not active. Please call sg_host_energy_plugin_init() during initialization.");
+ return host->extension<HostEnergy>()->get_watt_idle_at(pstate);
+}
+
+/** @ingroup plugin_host_energy
+ * @brief Get the amount of watt dissipated at the given pstate when the host is at 0 or epsilon% CPU usage.
+ */
double sg_host_get_wattmin_at(sg_host_t host, int pstate)
{
xbt_assert(HostEnergy::EXTENSION_ID.valid(),
#include "src/plugins/vm/VirtualMachineImpl.hpp"
#include <simgrid/s4u.hpp>
-// Makes sure that this plugin can be activated from the command line with ``--plugin=name``
+// Makes sure that this plugin can be activated from the command line with ``--cfg=plugin:host_load``
SIMGRID_REGISTER_PLUGIN(host_load, "Cpu load", &sg_host_load_plugin_init)
/** @defgroup plugin_host_load
return this;
}
+ExecPtr Exec::set_name(const std::string& name)
+{
+ xbt_assert(state_ == State::INITED, "Cannot change the name of an exec after its start");
+ name_ = name;
+ return this;
+}
+
+/** @brief Retrieve the host on which this activity takes place.
+ * If it runs on more than one host, only the first host is returned.
+ */
Host* Exec::get_host() const
{
return static_cast<kernel::activity::ExecImpl*>(pimpl_.get())->get_host();
"Cannot change the host of an exec once it's done (state: %d)", (int)state_);
if (state_ == State::STARTED)
boost::static_pointer_cast<simgrid::kernel::activity::ExecImpl>(pimpl_)->migrate(host);
- host_ = host;
boost::static_pointer_cast<simgrid::kernel::activity::ExecImpl>(pimpl_)->set_host(host);
return this;
}
-
-/** @brief Retrieve the host on which this activity takes place. */
-Host* ExecSeq::get_host()
-{
- return host_;
-}
-
/** @brief Returns the amount of flops that remain to be done */
double ExecSeq::get_remaining()
{
return new_zone;
}
+void sg_platf_new_Zone_set_properties(std::unordered_map<std::string, std::string>* props)
+{
+ xbt_assert(current_routing, "Cannot set properties of the current Zone: none under construction");
+
+ for (auto kv = props->begin(); kv != props->end(); ++kv)
+ current_routing->get_iface()->set_property(kv->first, kv->second);
+}
+
/**
* @brief Specify that the description of the current AS is finished
*
XBT_PUBLIC simgrid::kernel::routing::NetZoneImpl*
sg_platf_new_Zone_begin(simgrid::kernel::routing::ZoneCreationArgs* zone); // Begin description of new Zone
+XBT_PUBLIC void sg_platf_new_Zone_set_properties(std::unordered_map<std::string, std::string>* props);
XBT_PUBLIC void sg_platf_new_Zone_seal(); // That Zone is fully described
XBT_PUBLIC void sg_platf_new_host(simgrid::kernel::routing::HostCreationArgs* host); // Add a host to the current Zone
/* make sure these symbols are defined as strong ones in this file so that the linker can resolve them */
+std::vector<std::unordered_map<std::string, std::string>*> property_sets;
+
/* The default current property receiver. Setup in the corresponding opening callbacks. */
-std::unordered_map<std::string, std::string>* current_property_set = nullptr;
std::unordered_map<std::string, std::string>* current_model_property_set = nullptr;
-int ZONE_TAG = 0; // Whether we just opened a zone tag (to see what to do with the properties)
FILE *surf_file_to_parse = nullptr;
/* Stuff relative to storage */
void STag_surfxml_storage()
{
- ZONE_TAG = 0;
XBT_DEBUG("STag_surfxml_storage");
- xbt_assert(current_property_set == nullptr, "Someone forgot to reset the property set to nullptr in its closing tag (or XML malformed)");
+ property_sets.push_back(new std::unordered_map<std::string, std::string>());
}
void ETag_surfxml_storage()
{
simgrid::kernel::routing::StorageCreationArgs storage;
- storage.properties = current_property_set;
- current_property_set = nullptr;
+ storage.properties = property_sets.back();
+ property_sets.pop_back();
storage.id = A_surfxml_storage_id;
storage.type_id = A_surfxml_storage_typeId;
}
void STag_surfxml_storage___type()
{
- ZONE_TAG = 0;
XBT_DEBUG("STag_surfxml_storage___type");
- xbt_assert(current_property_set == nullptr, "Someone forgot to reset the property set to nullptr in its closing tag (or XML malformed)");
+ property_sets.push_back(new std::unordered_map<std::string, std::string>());
xbt_assert(current_model_property_set == nullptr, "Someone forgot to reset the model property set to nullptr in its closing tag (or XML malformed)");
}
void ETag_surfxml_storage___type()
{
simgrid::kernel::routing::StorageTypeCreationArgs storage_type;
- storage_type.properties = current_property_set;
- current_property_set = nullptr;
+ storage_type.properties = property_sets.back();
+ property_sets.pop_back();
storage_type.model_properties = current_model_property_set;
current_model_property_set = nullptr;
}
void STag_surfxml_host(){
- ZONE_TAG = 0;
- xbt_assert(current_property_set == nullptr, "Someone forgot to reset the property set to nullptr in its closing tag (or XML malformed)");
+ property_sets.push_back(new std::unordered_map<std::string, std::string>());
}
void STag_surfxml_prop()
{
- if (ZONE_TAG) { // We need to retrieve the most recently opened zone
- XBT_DEBUG("Set zone property %s -> %s", A_surfxml_prop_id, A_surfxml_prop_value);
- simgrid::s4u::NetZone* netzone = simgrid::s4u::Engine::get_instance()->netzone_by_name_or_null(A_surfxml_zone_id);
-
- netzone->set_property(std::string(A_surfxml_prop_id), A_surfxml_prop_value);
- } else {
- if (not current_property_set)
- current_property_set = new std::unordered_map<std::string, std::string>; // Maybe, it should raise an error
- current_property_set->insert({A_surfxml_prop_id, A_surfxml_prop_value});
- XBT_DEBUG("add prop %s=%s into current property set %p", A_surfxml_prop_id, A_surfxml_prop_value,
- current_property_set);
- }
+ property_sets.back()->insert({A_surfxml_prop_id, A_surfxml_prop_value});
+ XBT_DEBUG("add prop %s=%s into current property set %p", A_surfxml_prop_id, A_surfxml_prop_value,
+ property_sets.back());
}
void ETag_surfxml_host() {
simgrid::kernel::routing::HostCreationArgs host;
- host.properties = current_property_set;
- current_property_set = nullptr;
+ host.properties = property_sets.back();
+ property_sets.pop_back();
host.id = A_surfxml_host_id;
}
void STag_surfxml_disk() {
- ZONE_TAG = 0;
- xbt_assert(current_property_set == nullptr,
- "Someone forgot to reset the property set to nullptr in its closing tag (or XML malformed)");
+ property_sets.push_back(new std::unordered_map<std::string, std::string>());
}
void ETag_surfxml_disk() {
simgrid::kernel::routing::DiskCreationArgs disk;
- disk.properties = current_property_set;
- current_property_set = nullptr;
+ disk.properties = property_sets.back();
+ property_sets.pop_back();
disk.id = A_surfxml_disk_id;
disk.read_bw = surf_parse_get_bandwidth(A_surfxml_disk_read___bw, "read_bw of disk ", disk.id);
void ETag_surfxml_cluster(){
simgrid::kernel::routing::ClusterCreationArgs cluster;
- cluster.properties = current_property_set;
- current_property_set = nullptr;
+ cluster.properties = property_sets.back();
+ property_sets.pop_back();
cluster.id = A_surfxml_cluster_id;
cluster.prefix = A_surfxml_cluster_prefix;
}
void STag_surfxml_cluster(){
- ZONE_TAG = 0;
- xbt_assert(current_property_set == nullptr, "Someone forgot to reset the property set to nullptr in its closing tag (or XML malformed)");
+ property_sets.push_back(new std::unordered_map<std::string, std::string>());
}
void STag_surfxml_cabinet(){
}
void STag_surfxml_link(){
- ZONE_TAG = 0;
- xbt_assert(current_property_set == nullptr, "Someone forgot to reset the property set to nullptr in its closing tag (or XML malformed)");
+ property_sets.push_back(new std::unordered_map<std::string, std::string>());
}
void ETag_surfxml_link(){
simgrid::kernel::routing::LinkCreationArgs link;
- link.properties = current_property_set;
- current_property_set = nullptr;
+ link.properties = property_sets.back();
+ property_sets.pop_back();
link.id = std::string(A_surfxml_link_id);
link.bandwidths = surf_parse_get_bandwidths(A_surfxml_link_bandwidth, "bandwidth of link", link.id.c_str());
void STag_surfxml_zone()
{
- ZONE_TAG = 1;
+ property_sets.push_back(new std::unordered_map<std::string, std::string>());
simgrid::kernel::routing::ZoneCreationArgs zone;
zone.id = A_surfxml_zone_id;
zone.routing = static_cast<int>(A_surfxml_zone_routing);
-
sg_platf_new_Zone_begin(&zone);
}
void ETag_surfxml_zone()
{
+ sg_platf_new_Zone_set_properties(property_sets.back());
+ delete property_sets.back();
+ property_sets.pop_back();
+
sg_platf_new_Zone_seal();
}
void STag_surfxml_config()
{
- ZONE_TAG = 0;
- xbt_assert(current_property_set == nullptr,
- "Someone forgot to reset the property set to nullptr in its closing tag (or XML malformed)");
+ property_sets.push_back(new std::unordered_map<std::string, std::string>());
XBT_DEBUG("START configuration name = %s",A_surfxml_config_id);
if (_sg_cfg_init_status == 2) {
surf_parse_error("All <config> tags must be given before any platform elements (such as <zone>, <host>, <cluster>, "
{
// Sort config elements before applying.
// That's a little waste of time, but not doing so would break the tests
+ auto current_property_set = property_sets.back();
+
std::vector<std::string> keys;
for (auto const& kv : *current_property_set) {
keys.push_back(kv.first);
XBT_DEBUG("End configuration name = %s",A_surfxml_config_id);
delete current_property_set;
- current_property_set = nullptr;
+ property_sets.pop_back();
}
static std::vector<std::string> arguments;
void STag_surfxml_actor()
{
- ZONE_TAG = 0;
+ property_sets.push_back(new std::unordered_map<std::string, std::string>());
arguments.assign(1, A_surfxml_actor_function);
- xbt_assert(current_property_set == nullptr, "Someone forgot to reset the property set to nullptr in its closing tag (or XML malformed)");
}
void ETag_surfxml_process()
{
simgrid::kernel::routing::ActorCreationArgs actor;
- actor.properties = current_property_set;
- current_property_set = nullptr;
+ actor.properties = property_sets.back();
+ property_sets.pop_back();
actor.args.swap(arguments);
actor.host = A_surfxml_actor_host;
if (NOT enable_memcheck AND NOT enable_address_sanitizer AND NOT enable_thread_sanitizer)
ADD_TESH_FACTORIES(stack-overflow "ucontext;raw;boost" --setenv bindir=${CMAKE_BINARY_DIR}/teshsuite/simix/stack-overflow --setenv srcdir=${CMAKE_HOME_DIRECTORY} --cd ${CMAKE_HOME_DIRECTORY}/teshsuite/simix/stack-overflow stack-overflow.tesh)
- if (release AND (WIN32 OR CMAKE_SYSTEM_NAME MATCHES "Darwin"))
+ if (WIN32)
SET_TESH_PROPERTIES(stack-overflow "ucontext;raw;boost" WILL_FAIL true)
endif()
endif()
-! expect signal SIGSEGV
+! expect signal SIGSEGV|SIGBUS
$ ${bindir:=.}/stack-overflow --cfg=contexts/stack-size:96 ${srcdir:=.}/examples/platforms/small_platform.xml
> [Tremblay:master:(1) 0.000000] [test/INFO] Launching our nice bugged recursive function...
> Access violation or Bus error detected.
${CMAKE_CURRENT_SOURCE_DIR}/generate-dwarf-functions
${CMAKE_CURRENT_SOURCE_DIR}/normalize-pointers.py
${CMAKE_CURRENT_SOURCE_DIR}/sg_xml_unit_converter.py
- ${CMAKE_CURRENT_SOURCE_DIR}/sg_xml_energy_ponecore_to_pepsilon.py
${CMAKE_CURRENT_SOURCE_DIR}/simgrid_update_xml.pl
${CMAKE_CURRENT_SOURCE_DIR}/simgrid_convert_TI_traces.py
${CMAKE_CURRENT_SOURCE_DIR}/doxygen/fig2dev_postprocessor.pl
docs/source/platform.rst
docs/source/platform_howtos.rst
docs/source/Platform_Examples.rst
+ docs/source/Plugins.rst
docs/source/XML_Reference.rst
docs/source/Tutorial_Algorithms.rst
+++ /dev/null
-#!/usr/bin/env python3
-# -*- coding: utf-8 -*-
-
-# Copyright (c) 2019. The SimGrid Team.
-# All rights reserved.
-
-# This program is free software; you can redistribute it and/or modify it
-# under the terms of the license (GNU LGPL) which comes with this package.
-
-'''Update 3-part energy consumption syntax in SimGrid XML platform files.
-
-- watt_per_state: "pIdle:pOneCore:pFull" -> "pIdle:pEpsilon:pFull"
- This is done by computing pEpsilon from pOneCore, pFull and #core.'''
-import fnmatch
-import os
-import sys
-import xml.etree.ElementTree as ET
-
-class TreeBuilderWithComments(ET.TreeBuilder):
- def comment(self, data):
- self.start(ET.Comment, {})
- self.data(data)
- self.end(ET.Comment)
-
-def update_platform_file(filename):
- comment_tb = TreeBuilderWithComments()
- tree = ET.parse(filename, parser=ET.XMLParser(target=comment_tb))
- root = tree.getroot()
-
- parent_dict = {c:p for p in root.iter() for c in p}
-
- for prop in root.iter('prop'):
- if 'id' in prop.attrib and prop.attrib['id'] == 'watt_per_state':
- # Parse energy consumption and core count
- values_str = "".join(prop.attrib['value'].split()) # remove whitespaces
- values = values_str.split(',')
- nb_core = 1
- if 'core' in parent_dict[prop].attrib:
- nb_core = int(parent_dict[prop].attrib['core'])
- if nb_core < 1: raise Exception(f'Invalid core count: {nb_core}')
-
- # If a middle value is given, pIdle:pOneCore:pFull is assumed
- # and converted to pIdle:pEpsilon:pFull
- consumption_per_pstate = []
- update_required = False
- for value in values:
- powers = value.split(':')
- if len(powers) == 3:
- update_required = True
- (pIdle, p1, pFull) = [float(x) for x in powers]
- if nb_core == 1:
- if p1 != pFull:
- raise Exception('Invalid energy consumption: ' +
- "A 1-core host has pOneCore != pFull " +
- f'({p1} != {pFull}).\n' +
- 'Original watt_per_state value: "{}"'.format(prop.attrib['value']))
- pEpsilon = pFull
- else:
- pEpsilon = p1 - ((pFull - p1) / (nb_core - 1))
- consumption_per_pstate.append(f"{pIdle}:{pEpsilon}:{pFull}")
- if pIdle > pEpsilon:
- print(f"WARNING: pIdle > pEpsilon ({pIdle} > {pEpsilon})")
- else: # len(powers) == 2
- if nb_core == 1:
- update_required = True
- (pIdle, pFull) = [float(x) for x in powers]
- pEpsilon = pFull
- consumption_per_pstate.append(f"{pIdle}:{pEpsilon}:{pFull}")
- print(f"WARNING: putting {pFull} as pEpsilon by default for a single core")
- else:
- consumption_per_pstate.append(value)
-
- if update_required:
- updated_value = ', '.join(consumption_per_pstate)
- print(f'"{values_str}" -> "{updated_value}" (core={nb_core})')
- prop.attrib['value'] = updated_value
-
- with open(filename, 'w', encoding='utf-8') as output_file:
- # xml.etree.ElementTree does not handle doctypes =/
- # https://stackoverflow.com/questions/15304229/convert-python-elementtree-to-string
- content = '''<?xml version='1.0' encoding='utf-8'?>
-<!DOCTYPE platform SYSTEM "https://simgrid.org/simgrid.dtd">
-{}
-'''.format(ET.tostring(root, encoding="unicode"))
- output_file.write(content)
-
-if __name__ == '__main__':
- usage = "usage: {cmd} FILE\n\n{doc}".format(cmd=sys.argv[0], doc=__doc__)
-
- if len(sys.argv) != 2:
- print(usage)
- sys.exit(1)
-
- update_platform_file(sys.argv[1])
self.cwd = os.getcwd()
self.ignore_output = False
- self.expect_return = 0
+ self.expect_return = [0]
self.output_display = False
print('\n'.join(logs))
return
- if proc.returncode != self.expect_return:
+ if not proc.returncode in self.expect_return:
if proc.returncode >= 0:
logs.append("Test suite `{file}': NOK (<{cmd}> returned code {code})".format(
file=FileReader().filename, cmd=cmdName, code=proc.returncode))
cmd.output_display = True
cmd.ignore_output = True
elif line[0:15] == "! expect return":
- cmd.expect_return = int(line[16:])
+ cmd.expect_return = [int(line[16:])]
#print("expect return "+str(int(line[16:])))
elif line[0:15] == "! expect signal":
- sig = line[16:]
- # get the signal integer value from the signal module
- if sig not in signal.__dict__:
- fatal_error("unrecognized signal '" + sig + "'")
- sig = int(signal.__dict__[sig])
- # popen return -signal when a process ends with a signal
- cmd.expect_return = -sig
+ cmd.expect_return = []
+ for sig in (line[16:]).split("|"):
+ # get the signal integer value from the signal module
+ if sig not in signal.__dict__:
+ fatal_error("unrecognized signal '" + sig + "'")
+ sig = int(signal.__dict__[sig])
+ # popen return -signal when a process ends with a signal
+ cmd.expect_return.append(-sig)
elif line[0:len("! timeout ")] == "! timeout ":
if "no" in line[len("! timeout "):]:
cmd.timeout = None
