These are instructions for reproducing the results from the SIGCOMM article. You must follow these steps, which are described in the sections below:
We are also offering an archive with emulation results that reproduce the results here.
Checkout the repository containing the setup used in our article:
git clone https://git.epfl.ch/repo/line-sigcomm14.git
The setup uses 3 machines:
The control center computer must satisfy a single requirement: it must run Ubuntu 12.04 LTS 64-bits.
The hardware requirements for the traffic generator and the network emulator are described in section Architecture of this document.
Please follow the instructions from section System configuration of this document.
Additionally, to generate the plots on the control center you must install the matplotlib library:
apt-get install matplotlib
In general, to run an emulation the following steps must be followed:
For the article, we used 23 experiments. Configuring them by hand in the GUI is possible but time-consuming and error-prone. To simplify the process, we added to the interface a set of buttons that automate the process, simulating the actions of the user to generate exactly the configurations used in the paper. They can be seen in this screenshot. The parameter values set by the buttons are described in detail for each figure below. Additionally, you can also find the complete configuration of the topology and the traffic used in each experiment in a file named graph.txt placed in the experiment directory (the traffic generator syntax is documented in line-traffic/config-file-howto.txt in the repository); other parameters passed to the emulator are saved in a file named run-params.txt. You can find these files in the archive we provide with our experimental data.
For generating this figure we run 4 experiments using the "quad" topology.
The shared link is neutral. To make things difficult for our algorithm, we try to create network conditions that could be misinterpreted as non-neutrality.
Traffic class 1 and class 2 have the same properties (such as RTT, TCP congestion control algorithm) except the flow transfer size: the flow transfer size of paths 1 and 2 is 1Mb, while that of paths 3 and 4 varies: 1Mb, 10Mb, 40Mb and 10Gb.
The four experiments used for the figure can be configured using the button "Sigcomm 7a" from the "Custom controls" window. The function that generates the configuration is:
line-gui/customcontrols_qos.cpp: void CustomControls::on_btnSigcomm7a_clicked()
After clicking the button, there should be four experiments in the queue. Click main window > "Queue" > "Start all" to start the emulations.
Here is a summary of the parameters set by the configuration code:
For generating this figure we run 4 experiments using the "quad" topology.
The shared link is neutral. To make things difficult for our algorithm, we try to create network conditions that could be misinterpreted as non-neutrality.
Traffic class 1 and class 2 have the same properties (such as transfer size, TCP congestion control algorithm) except RTT: the RTT of paths 1 and 2 is 50ms, while that of paths 3 and 4 varies from 50ms to 200ms.
The four experiments used for the figure can be configured using the button "Sigcomm 7b" from the "Custom controls" window. The function that generates the configuration is:
line-gui/customcontrols_qos.cpp: void CustomControls::on_btnSigcomm7b_clicked()
After clicking the button, there should be four experiments in the queue. Click main window > "Queue" > "Start all" to start the emulations.
Here is a summary of the parameters set by the configuration code:
For generating this figure we run 2 experiments using the "quad" topology.
The shared link is neutral. To make things difficult for our algorithm, we try to create network conditions that could be misinterpreted as non-neutrality.
Traffic class 1 and class 2 have the same properties (such as RTT, transfer size) except the TCP congestion control algorithms: in one experiment all paths use TCP CUBIC, while in the other paths 1 and 2 use CUBIC and paths 3 and 4 use NewReno.
The two experiments used for the figure can be configured using the button "Sigcomm 7c" from the "Custom controls" window. The function that generates the configuration is:
line-gui/customcontrols_qos.cpp: void CustomControls::on_btnSigcomm7c_clicked()
After clicking the button, there should be two experiments in the queue. Click main window > "Queue" > "Start all" to start the emulations.
Here is a summary of the parameters set by the configuration code:
For generating this figure we run 4 experiments using the "quad" topology.
The shared link is non-neutral, policing the traffic in class 2. Here, the difficult scenarios for our algorithm are the ones where all the paths carry the same kind of traffic. In this plot we vary the transfer size for both traffic classes between 1, 10, 40 and 10000 Mb.
The four experiments used for the figure can be configured using the button "Sigcomm 7d" from the "Custom controls" window. The function that generates the configuration is:
line-gui/customcontrols_qos.cpp: void CustomControls::on_btnSigcomm7d_clicked()
After clicking the button, there should be four experiments in the queue. Click main window > "Queue" > "Start all" to start the emulations.
Here is a summary of the parameters set by the configuration code:
For generating this figure we run 4 experiments using the "quad" topology.
The shared link is non-neutral, policing the traffic in class 2. Here, the difficult scenarios for our algorithm are the ones where all the paths carry the same kind of traffic. In this plot we vary the RTT for both traffic classes, between 50, 80, 120 and 200 ms.
The four experiments used for the figure can be configured using the button "Sigcomm 7e" from the "Custom controls" window. The function that generates the configuration is:
line-gui/customcontrols_qos.cpp: void CustomControls::on_btnSigcomm7e_clicked()
After clicking the button, there should be four experiments in the queue. Click main window > "Queue" > "Start all" to start the emulations.
Here is a summary of the parameters set by the configuration code:
For generating this figure we run 4 experiments using the "quad" topology.
The shared link is non-neutral, policing the traffic in class 2. Here, the difficult scenarios for our algorithm are the ones where all the paths carry the same kind of traffic. In this plot we vary the policing weights of the shared link, between (1.0,0.2), (1.0,0.3), (1.0,0.4) and (1.0,0.5). The first number represents the fraction of the link rate allocated to class 1, the second number represents the fraction of the link rate allocated to class 2.
The four experiments used for the figure can be configured using the button "Sigcomm 7f" from the "Custom controls" window. The function that generates the configuration is:
line-gui/customcontrols_qos.cpp: void CustomControls::on_btnSigcomm7f_clicked()
After clicking the button, there should be four experiments in the queue. Click main window > "Queue" > "Start all" to start the emulations.
Here is a summary of the parameters set by the configuration code:
For generating this figure we run an experiment using the "pegasus" topology. This is a large topology with 4 types of traffic (first number = transfer size, second number = number of parallel flows):
There are 3 non-neutral links, policing the traffic in class 2. There are two difficulties in this scenario: (1) that there is a large number of shared links and bottlenecks, but only a few of them are non-neutral; and (2) that measurements are taken for only some of the traffic (i.e. not every end-host in the network is a vantage point taking performance measurements).
The experiment used for the figure can be configured using the button "Sigcomm 8a/b" from the "Custom controls" window. The function that generates the configuration is:
line-gui/customcontrols_qos.cpp: void CustomControls::on_btnSigcomm8ab_clicked()
After clicking the button, there should be two experiments in the queue. Click main window > "Queue" > "Start all" to start the emulations.
Here is a summary of the parameters set by the configuration code:
After running the experiments, open a terminal in a directory used to store the plot (e.g. repro-plots/7a). Copy the code from the text box in the "Custom controls" window to copy there the data from the emulations, then clear the text box in the "Custom controls" window.
To generate the plot, run:
../../plotting-scripts/plot-data.py --plot vary-transfer-size
The script simply reads the file path-congestion-probs.txt from each experiment, selects the line corresponding to the desired loss threshold (0.001, i.e. the first column in the file) and plots the path congestion probabilities (columns 2-5 in the file).
The plot is found in:
image-plot-7a-*/2. stats prob-cong-path - transferSize.png
After running the experiments, open a terminal in a directory used to store the plot (e.g. repro-plots/7b). Copy the code from the text box in the "Custom controls" window to copy there the data from the emulations, then clear the text box in the "Custom controls" window.
To generate the plot, run:
../../plotting-scripts/plot-data.py --plot vary-rtt
The script simply reads the file path-congestion-probs.txt from each experiment, selects the line corresponding to the desired loss threshold (0.001, i.e. the first column in the file) and plots the path congestion probabilities (columns 2-5 in the file).
The plot is found in:
image-plot-7b-*/9. stats prob-cong-path - rtt.png
After running the experiments, open a terminal in a directory used to store the plot (e.g. repro-plots/7c). Copy the code from the text box in the "Custom controls" window to copy there the data from the emulations, then clear the text box in the "Custom controls" window.
Unfortunately the script incorrectly places the two experiments in different directories. Move them to the same directory (it does not matter which one).
To generate the plot, run:
../../plotting-scripts/plot-data.py --plot vary-tcp
The script simply reads the file path-congestion-probs.txt from each experiment, selects the line corresponding to the desired loss threshold (0.001, i.e. the first column in the file) and plots the path congestion probabilities (columns 2-5 in the file).
The plot is found in:
image-plot-7c-*/10. stats prob-cong-path - tcp.png
After running the experiments, open a terminal in a directory used to store the plot (e.g. repro-plots/7d). Copy the code from the text box in the "Custom controls" window to copy there the data from the emulations, then clear the text box in the "Custom controls" window.
To generate the plot, run:
../../plotting-scripts/plot-data.py --plot vary-transfer-size
The script simply reads the file path-congestion-probs.txt from each experiment, selects the line corresponding to the desired loss threshold (0.001, i.e. the first column in the file) and plots the path congestion probabilities (columns 2-5 in the file).
The plot is found in:
image-plot-7d-*/2. stats prob-cong-path - transferSize.png
After running the experiments, open a terminal in a directory used to store the plot (e.g. repro-plots/7e). Copy the code from the text box in the "Custom controls" window to copy there the data from the emulations, then clear the text box in the "Custom controls" window.
To generate the plot, run:
../../plotting-scripts/plot-data.py --plot vary-rtt
The script simply reads the file path-congestion-probs.txt from each experiment, selects the line corresponding to the desired loss threshold (0.001, i.e. the first column in the file) and plots the path congestion probabilities (columns 2-5 in the file).
The plot is found in:
image-plot-7e-*/9. stats prob-cong-path - rtt.png
After running the experiments, open a terminal in a directory used to store the plot (e.g. repro-plots/7f). Copy the code from the text box in the "Custom controls" window to copy there the data from the emulations, then clear the text box in the "Custom controls" window.
To generate the plot, run:
../../plotting-scripts/plot-data.py --plot vary-qos
The script simply reads the file path-congestion-probs.txt from each experiment, selects the line corresponding to the desired loss threshold (0.001, i.e. the first column in the file) and plots the path congestion probabilities (columns 2-5 in the file).
The plot is found in:
image-plot-7f-*/1. stats prob-cong-path - policing.png
After running the experiments, open a terminal in a directory used to store the plot (e.g. repro-plots; do not create a subdirectory for the figure). Copy the code from the text box in the "Custom controls" window to copy there the data from the emulations, then clear the text box in the "Custom controls" window.
Note that there are two experiments; we are only interested in the experiment ending with "non-neutral-links-5-14-20". The other one contains an identical configuration except all links are neutral, but it was not included in the paper.
Open a terminal to the directory of the the experiment ending with "non-neutral-links-5-14-20". To run the inference algorithm and generate the plot, run:
../../../build-line-runner/line-runner --process-results $(pwd) numResamplings 3 lossThreshold 0.05
The image file for figure 8a is located in:
link-analysis-0.01/class-path-cong-probs-links.png
The image file for figure 8b is located in:
seq-analysis-*-lossth-0.05-*/link-seq-cong-prob-all-inferred.png