SunnyQjm/Orca
1
0 Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Compare commits

merge into: SunnyQjm/Orca:master
SunnyQjm/Orca:master SunnyQjm/Orca:gh-pages
..
pull from: SunnyQjm/Orca:gh-pages
SunnyQjm/Orca:master SunnyQjm/Orca:gh-pages
1 Commits
master .. gh-pages
Author SHA1 Message Date
Soheil Abbasloo c914363b17 Create gh-pages branch via GitHub 2020-07-15 17:34:54 -04:00
54 changed files with 927 additions and 18789353 deletions
Expand all files Collapse all files

No files matched your search

README.md
-125
View File
@@ -1,125 +0,0 @@
# Orca v1.0
This release presents the source code and materials used for the experiments in our SIGCOMM'20 paper: "Classic Meets Modern: A Pragmatic Learning-Based Congestion Control for the Internet" (https://dl.acm.org/doi/abs/10.1145/3387514.3405892).
For the source code of our DeepCC plug-in reported in our IEEE JSAC'21 paper: "Wanna Make Your TCP Scheme Great for Cellular Networks? Let Machines Do It for You!", please checkout https://github.com/Soheil-ab/DeepCC.v1.0/ (The paper can be accessed here: https://ieeexplore.ieee.org/document/9252929)
Installation Guide
==================
### Installing Required Tools
Install Mahimahi (http://mahimahi.mit.edu/#getting)
```sh
sudo apt-get install build-essential git debhelper autotools-dev dh-autoreconf iptables protobuf-compiler libprotobuf-dev pkg-config libssl-dev dnsmasq-base ssl-cert libxcb-present-dev libcairo2-dev libpango1.0-dev iproute2 apache2-dev apache2-bin iptables dnsmasq-base gnuplot iproute2 apache2-api-20120211 libwww-perl
git clone https://github.com/ravinet/mahimahi
cd mahimahi
./autogen.sh && ./configure && make
sudo make install
sudo sysctl -w net.ipv4.ip_forward=1
```
### Orca's DRL Agent
#### Prerequisites
- Python >= 3.4
- Tensorflow 1.14.0
##### Create a virtul environment
```
mkdir ~/venv
sudo apt update
sudo apt install python3-pip
sudo pip3 install -U virtualenv
virtualenv ~/venv -p python3
```
If you change the path of the virtual environment, you need to change the source code accordingly. The source code assumes that virtual environment is set @ ~/venv .
##### Install packages
```
source ~/venv/bin/activate
pip install --upgrade pip
pip install gym
pip install tensorflow==1.14
pip install sysv_ipc
```
Verify Installation
```
python -c "import tensorflow as tf; tf.enable_eager_execution(); print(tf.reduce_sum(tf.random_normal([1000, 1000])))"
```
To deactivate venv
```
(venv) $ deactivate
```
### Patching Orca's Kernel: (Option 1)
Simplest option to install Orca's patched Kernels is to install the prepared debian packages:
```
cd linux
sudo dpkg -i linux-image*
sudo dpkg -i linux-header*
sudo reboot
```
### Patching Orca's Kernel: (Option 2)
If you have already done the option 1, skip this part!
Another option is to compile your own kernel using the provided patch. You can use the instructions provided here to do that: https://github.com/Soheil-ab/C2TCP-IFIP/
The source code is available in linux folder (https://github.com/Soheil-ab/Orca/blob/master/linux/linux-4-13-1-orca-0521%2Bc2tcp.patch)
### Verify the new kernel
After installing the Orca's kernel and restarting your system, use the following command to make sure that system is using the new kernel:
```
uname -r
```
The output should be 4.13.1-0521*. If not, you need to bring the 4.13.1-0521* Kernel image on top of the grub list. For instance, you can use grub-customizer application. Install the grub-customizer using following:
```
sudo add-apt-repository ppa:danielrichter2007/grub-customizer
sudo apt-get update
sudo apt-get install grub-customizer
sudo grub-customizer
```
### Build Orca's Server-Client Apps
To build the required applications, run the following:
```
./build.sh
```
This release includes two versions: A standalone actor version and an actor-learner version.
In the Standalone actor version, no learner will be initiated. This can be usefull when you simply wanna test the current model over an emulated link. However, the actor-learner version requires a learner being initiated before any actor can be started. The second mode is usefull for learning a new/better model and also, it still can be used for performing a simple test over emulated links.
### Run a Sample Test with the standalone version using the provided learned model
```
./orca-standalone-emulation.sh 44444
```
### Run a Sample Test with the acotr-learner version using the provided learned model
```
./orca.sh 4 44444
```
Results will be generated automatically in rl-module/log/*
You can check out the summary of results at rl-module/log/sum-*
### Some notes on the training:
1. To distribute the actors over remote servers, you need to change "learner_ip" and "actor_ip" fields of the params.json file to the corresponding server IPs.
2. Start the learner first.
3. Set `remote:true` when using remote servers.
4. Set `num_actors: N`, N is the number of actors.
### Cellular traces:
To use traces avaialbe @ https://github.com/Soheil-ab/Cellular-Traces-NYC , copy them to the traces folder of the project.
actor.sh
-48
View File
@@ -1,48 +0,0 @@
#!/bin/bash
if [ $# != 12 ]
then
echo -e "usage:$0 port period first_time [underlying scheme:cubic , vegas , westwood , illinois , bbr, yeah , veno, scal , htcp , cdg , hybla ,... ] [path to ddpg.py] [actor id] [downlink] [uplink] [one-way link delay] [time time] [Qsize] [Max iterations per run]"
exit
fi
port=$1
period=$2
first_time=$3
x=100
scheme=$4
path=$5
id=$6
down=$7
up=$8
latency=$9
finish_time=${10}
qsize=${11}
max_it=${12}
echo "Running orca-$scheme: $down"
trace=""
scheme_des="orca-$scheme-$latency-$period-$qsize"
log="orca-$scheme-$down-$up-$latency-${period}-$qsize"
#Bring up the actor i:
echo "will be done in $finish_time seconds ..."
echo "$path/orca-server-mahimahi $port $path ${period} ${first_time} $scheme $id $down $up $latency $log $finish_time $qsize $max_it"
$path/orca-server-mahimahi $port $path ${period} ${first_time} $scheme $id $down $up $latency $log $finish_time $qsize $max_it
#sudo killall -s15 python
#sleep 10
echo "Finished."
if [ ${first_time} -eq 2 ] || [ ${first_time} -eq 4 ]
then
echo "Doing Some Analysis ..."
out="sum-${log}.tr"
echo $log >> $path/log/$out
$path/mm-thr 500 $path/log/down-${log} 1>tmp 2>res_tmp
cat res_tmp >>$path/log/$out
echo "------------------------------" >> $path/log/$out
rm *tmp
fi
echo "Done"
build.sh
-8
View File
@@ -1,8 +0,0 @@
g++ -pthread src/orca-server-mahimahi.cc src/flow.cc -o orca-server-mahimahi
g++ src/client.c -o client
cp client rl-module/
mv orca-server* rl-module/
sudo chmod +x rl-module/client
sudo chmod +x rl-module/orca-server-mahimahi
client
BIN
View File
Binary file not shown.
copyright
-19
View File
@@ -1,19 +0,0 @@
MIT License
Copyright (c) 2020 - Soheil Abbasloo and Chen-Yu Yen
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
index.html
+127
View File
@@ -0,0 +1,127 @@
<!DOCTYPE html>
<html lang="en-us">
<head>
<meta charset="UTF-8">
<title>Orca by Soheil-ab</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" type="text/css" href="stylesheets/normalize.css" media="screen">
<link href='https://fonts.googleapis.com/css?family=Open+Sans:400,700' rel='stylesheet' type='text/css'>
<link rel="stylesheet" type="text/css" href="stylesheets/stylesheet.css" media="screen">
<link rel="stylesheet" type="text/css" href="stylesheets/github-light.css" media="screen">
</head>
<body>
<section class="page-header">
<h1 class="project-name">Orca</h1>
<h2 class="project-tagline">Orca: Towards Mastering Congestion Control In the Internet</h2>
<a href="https://github.com/Soheil-ab/Orca" class="btn">View on GitHub</a>
<a href="https://github.com/Soheil-ab/Orca/zipball/master" class="btn">Download .zip</a>
<a href="https://github.com/Soheil-ab/Orca/tarball/master" class="btn">Download .tar.gz</a>
</section>
<section class="main-content">
<h1>
<a id="orca-v10" class="anchor" href="#orca-v10" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Orca v1.0</h1>
<p>This release presents the source code and materials used for the experiments in our SIGCOMM'20 paper: "Classic Meets Modern: A Pragmatic Learning-Based Congestion Control for the Internet".</p>
<h1>
<a id="installation-guide" class="anchor" href="#installation-guide" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Installation Guide</h1>
<h3>
<a id="installing-required-tools" class="anchor" href="#installing-required-tools" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Installing Required Tools</h3>
<p>Install Mahimahi (<a href="http://mahimahi.mit.edu/#getting">http://mahimahi.mit.edu/#getting</a>)</p>
<pre><code>```sh
sudo apt-get install build-essential git debhelper autotools-dev dh-autoreconf iptables protobuf-compiler libprotobuf-dev pkg-config libssl-dev dnsmasq-base ssl-cert libxcb-present-dev libcairo2-dev libpango1.0-dev iproute2 apache2-dev apache2-bin iptables dnsmasq-base gnuplot iproute2 apache2-api-20120211 libwww-perl
git clone https://github.com/ravinet/mahimahi
cd mahimahi
./autogen.sh &amp;&amp; ./configure &amp;&amp; make
sudo make install
sudo sysctl -w net.ipv4.ip_forward=1
```
</code></pre>
<h3>
<a id="orcas-drl-agent" class="anchor" href="#orcas-drl-agent" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Orca's DRL Agent</h3>
<h4>
<a id="prerequisites" class="anchor" href="#prerequisites" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Prerequisites</h4>
<ul>
<li>Python &gt;= 3.4</li>
<li>Tensorflow 1.14.0</li>
</ul>
<h5>
<a id="create-a-virtul-environment" class="anchor" href="#create-a-virtul-environment" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Create a virtul environment</h5>
<pre><code>mkdir ~/venv
sudo apt update
sudo apt install python3-pip
sudo pip3 install -U virtualenv
virtualenv ~/venv -p python3
</code></pre>
<p>If you change the path of the virtual environment, you need to change the source code accordingly. The source code assumes that virtual environment is set @ ~/venv .</p>
<h5>
<a id="install-packages" class="anchor" href="#install-packages" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Install packages</h5>
<pre><code>source ~/venv/bin/activate
pip install --upgrade pip
pip install gym
pip install tensorflow==1.14
pip install sysv_ipc
</code></pre>
<p>Verify Installation</p>
<pre><code>python -c "import tensorflow as tf; tf.enable_eager_execution(); print(tf.reduce_sum(tf.random_normal([1000, 1000])))"
</code></pre>
<p>To deactivate venv</p>
<pre><code>(venv) $ deactivate
</code></pre>
<h3>
<a id="patching-orca-kernel-install-the-prepared-debian-packages" class="anchor" href="#patching-orca-kernel-install-the-prepared-debian-packages" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Patching Orca Kernel: Install the prepared debian packages.</h3>
<pre><code>cd linux
sudo dpkg -i linux-image*
sudo dpkg -i linux-header*
sudo reboot
</code></pre>
<h3>
<a id="verify-the-new-kernel" class="anchor" href="#verify-the-new-kernel" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Verify the new kernel</h3>
<p>Use the following command to make sure that new kernel is installed:</p>
<pre><code>uname -r
</code></pre>
<p>The output should be 4.13.1-0521*. If not you need to bring the 4.13.1-0521* Kernel image on top of the grub list. For instance, you can use grub-customizer application. Install the grub-customizer using following:</p>
<pre><code>sudo add-apt-repository ppa:danielrichter2007/grub-customizer
sudo apt-get update
sudo apt-get install grub-customizer
sudo grub-customizer
</code></pre>
<h3>
<a id="build-orcas-server-client-apps" class="anchor" href="#build-orcas-server-client-apps" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Build Orca's Server-Client Apps</h3>
<p>To build the required applications, run the following:</p>
<pre><code>./build.sh
</code></pre>
<p>This release includes two versions: A standalone actor version and an actor-learner version.
In the Standalone actor version, no learner will be initiated. This can be usefull when you simply wanna test the current model over an emulated link. However, the actor-learner version requires a learner being initiated before any actor can be started. The second mode is usefull for learning a new/better model and also, it still can be used for performing a simple test over emulated links.</p>
<h3>
<a id="run-a-sample-test-with-the-standalone-version-using-the-provided-learned-model" class="anchor" href="#run-a-sample-test-with-the-standalone-version-using-the-provided-learned-model" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Run a Sample Test with the standalone version using the provided learned model</h3>
<pre><code>./orca-standalone-emulation.sh 44444
</code></pre>
<h3>
<a id="run-a-sample-test-with-the-acotr-learner-version-using-the-provided-learned-model" class="anchor" href="#run-a-sample-test-with-the-acotr-learner-version-using-the-provided-learned-model" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Run a Sample Test with the acotr-learner version using the provided learned model</h3>
<pre><code>./orca.sh 4 44444
</code></pre>
<p>Results will be generated automatically in rl-module/log/*
You can check out the summary of results at rl-module/log/sum-*</p>
<h3>
<a id="some-notes-on-the-training" class="anchor" href="#some-notes-on-the-training" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Some notes on the training:</h3>
<ol>
<li>To distribute the actors over remote servers, you need to change "learner_ip" and "actor_ip" fields of the params.json file to the corresponding server IPs.</li>
<li>Start the learner first.</li>
<li>Set <code>remote:true</code> when using remote servers.</li>
<li>Set <code>num_actors: N</code>, N is the number of actors.</li>
</ol>
<h3>
<a id="cellular-traces" class="anchor" href="#cellular-traces" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Cellular traces:</h3>
<p>To use traces avaialbe @ <a href="https://github.com/Soheil-ab/Cellular-Traces-NYC">https://github.com/Soheil-ab/Cellular-Traces-NYC</a> , copy them to the traces folder of the project.</p>
<footer class="site-footer">
<span class="site-footer-owner"><a href="https://github.com/Soheil-ab/Orca">Orca</a> is maintained by <a href="https://github.com/Soheil-ab">Soheil-ab</a>.</span>
<span class="site-footer-credits">This page was generated by <a href="https://pages.github.com">GitHub Pages</a> using the <a href="https://github.com/jasonlong/cayman-theme">Cayman theme</a> by <a href="https://twitter.com/jasonlong">Jason Long</a>.</span>
</footer>
</section>
</body>
</html>
learner.sh
-22
View File
@@ -1,22 +0,0 @@
#!/bin/bash
if [ $# != 2 ]
then
echo -e "usage:$0 [path to train_dir & d5.py] [first_time==1]"
echo "$@"
echo "$#"
exit
fi
path=$1
first_time=$2
##Bring up the learner:
if [ $first_time -eq 1 ];
then
/home/`whoami`/venv/bin/python $path/d5.py --job_name=learner --task=0 --base_path=$path &
elif [ $first_time -eq 4 ]
then
/home/`whoami`/venv/bin/python $path/d5.py --job_name=learner --task=0 --base_path=$path --load --eval &
else
/home/`whoami`/venv/bin/python $path/d5.py --job_name=learner --task=0 --base_path=$path --load &
fi
linux/copyright
-19
View File
@@ -1,19 +0,0 @@
MIT License
Copyright (c) 2020 - Soheil Abbasloo
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
linux/linux-4-13-1-orca-0521+c2tcp.patch
-1268
View File
File diff suppressed because it is too large. Load diff
linux/linux-headers-4.13.1-0521-learner_4.13.1-0521-learner-10.00.Custom_amd64.deb
BIN
View File
Binary file not shown.
linux/linux-image-4.13.1-0521-learner_4.13.1-0521-learner-10.00.Custom_amd64.deb
BIN
View File
Binary file not shown.
models/checkpoint
-1
View File
@@ -1 +0,0 @@
model_checkpoint_path: "model.ckpt-1283529"
models/model.ckpt-1283529.data-00000-of-00001
BIN
View File
Binary file not shown.
models/model.ckpt-1283529.index
BIN
View File
Binary file not shown.
models/model.ckpt-1283529.meta
BIN
View File
Binary file not shown.
orca-standalone-emulation.sh
-63
View File
@@ -1,63 +0,0 @@
if [ $# -eq 1 ]
then
source setup.sh
first_time=4
port_base=$1
max_steps=500000
eval_duration=30
num_actors=1
epoch=20
memory_size=$((max_steps*num_actors))
scheme_="cubic"
cur_dir=`pwd -P`
dir="${cur_dir}/rl-module"
sed "s/\"num_actors\"\: 1/\"num_actors\"\: $num_actors/" $cur_dir/params_base_eval.json > "${dir}/params.json"
sed -i "s/\"memsize\"\: 2553600/\"memsize\"\: $memory_size/" "${dir}/params.json"
sudo killall -s9 python client orca-server-mahimahi
#Bring up the actor(s):
act_id=0
act_port=$port_base
#Use a wired48 link for both down & up links (you just need to change the downlink side, if you need.)
dl=48
downl="wired$dl"
upl="wired48"
#one-way delay=10ms
del=10
bdp=$((2*dl*del/12)) #12Mbps=1pkt per 1 ms ==> BDP=2*del*BW=2*del*dl/12
qs=$((2*bdp))
# For the Step-scenraio, you can use follwoing parameters:
#downl="step-10s-3-level"
#qs=1000
#eval_duration=60
./actor.sh ${act_port} $epoch ${first_time} $scheme_ $dir $act_id $downl $upl $del $eval_duration $qs 0 &
pids="$pids $!"
#in case you need more actors: increase number actor's id and port. and run actor.sh again ...
#act_id=$((act_id+1))
#act_port=$((port_base+act_id))
#sleep 2
#Wait for them ...
for pid in $pids
do
echo "waiting for $pid"
wait $pid
done
#Make sure that learner and actors are down ...
for i in `seq 0 $((num_actors))`
do
sudo killall -s15 python
sudo killall -s15 orca-server-mahimahi
sudo killall -s15 client
done
else
echo "usage: $0 [base port number]"
fi
orca.sh
-121
View File
@@ -1,121 +0,0 @@
if [ $# -eq 2 ]
then
source setup.sh
first_time=$1
port_base=$2
cur_dir=`pwd -P`
scheme_="cubic"
max_steps=500000 #Run untill you collect 50k samples per actor
eval_duration=30
num_actors=1
memory_size=$((max_steps*num_actors))
dir="${cur_dir}/rl-module"
sed "s/\"num_actors\"\: 1/\"num_actors\"\: $num_actors/" $cur_dir/params_base.json > "${dir}/params.json"
sed -i "s/\"memsize\"\: 5320000/\"memsize\"\: $memory_size/" "${dir}/params.json"
sudo killall -s9 python client orca-server-mahimahi
epoch=20
act_port=$port_base
if [ $1 -eq 4 ]
then
# If you are here: You are going to perform an evaluation over an emulated link
num_actors=1
sed "s/\"num_actors\"\: 1/\"num_actors\"\: $num_actors/" $cur_dir/params_base.json > "${dir}/params.json"
echo "./learner.sh $dir $first_time &"
./learner.sh $dir ${first_time} &
#Bring up the actors:
act_id=0
for dl in 48
do
downl="wired$dl"
upl="wired48"
for del in 10
do
bdp=$((2*dl*del/12)) #12Mbps=1pkt per 1 ms ==> BDP=2*del*BW=2*del*dl/12
for qs in $((2*bdp))
do
./actor.sh ${act_port} $epoch ${first_time} $scheme_ $dir $act_id $downl $upl $del $eval_duration $qs 0 &
pids="$pids $!"
act_id=$((act_id+1))
act_port=$((port_base+act_id))
sleep 2
done
done
done
for pid in $pids
do
echo "waiting for $pid"
wait $pid
done
#Bring down the learner and actors ...
for i in `seq 0 $((num_actors))`
do
sudo killall -s15 python
sudo killall -s15 orca-server-mahimahi
sudo killall -s15 client
done
else
# If you are here: You are going to start/continue learning a better model!
#Bring up the learner:
echo "./learner.sh $dir $first_time &"
if [ $1 -eq 1 ];
then
# Start the learning from the scratch
/home/`logname`/venv/bin/python ${dir}/d5.py --job_name=learner --task=0 --base_path=${dir} &
lpid=$!
else
# Continue the learning on top of previous model
/home/`logname`/venv/bin/python ${dir}/d5.py --job_name=learner --task=0 --base_path=${dir} --load &
lpid=$!
fi
sleep 10
#Bring up the actors:
# Here, we go with single actor
act_id=0
for dl in 48
do
downl="wired$dl"
upl=$downl
for del in 10
do
bdp=$((2*dl*del/12)) #12Mbps=1pkt per 1 ms ==> BDP=2*del*BW=2*del*dl/12
for qs in $((2*bdp))
do
./actor.sh ${act_port} $epoch ${first_time} $scheme_ $dir $act_id $downl $upl $del 0 $qs $max_steps
pids="$pids $!"
act_id=$((act_id+1))
act_port=$((port_base+act_id))
sleep 2
done
done
done
for pid in $pids
do
echo "waiting for $pid"
wait $pid
done
#Kill the learner
sudo kill -s15 $lpid
#Wait if it needs to save somthing!
sleep 30
#Make sure all are down ...
for i in `seq 0 $((num_actors))`
do
sudo killall -s15 python
sudo killall -s15 orca-server-mahimahi
done
fi
else
echo "usage: $0 [{Learning from scratch=1} {Continue your learning=0} {Just Do Evaluation=4}] [base port number ]"
fi
params.json
+1
View File
@@ -0,0 +1 @@
{"name":"Orca","tagline":"Orca: Towards Mastering Congestion Control In the Internet","body":"# Orca v1.0\r\n\r\nThis release presents the source code and materials used for the experiments in our SIGCOMM'20 paper: \"Classic Meets Modern: A Pragmatic Learning-Based Congestion Control for the Internet\".\r\n\r\nInstallation Guide\r\n==================\r\n \r\n### Installing Required Tools\r\n\r\nInstall Mahimahi (http://mahimahi.mit.edu/#getting)\r\n\r\n\t```sh \r\n\tsudo apt-get install build-essential git debhelper autotools-dev dh-autoreconf iptables protobuf-compiler libprotobuf-dev pkg-config libssl-dev dnsmasq-base ssl-cert libxcb-present-dev libcairo2-dev libpango1.0-dev iproute2 apache2-dev apache2-bin iptables dnsmasq-base gnuplot iproute2 apache2-api-20120211 libwww-perl\r\n\tgit clone https://github.com/ravinet/mahimahi \r\n\tcd mahimahi\r\n\t./autogen.sh && ./configure && make\r\n\tsudo make install\r\n\tsudo sysctl -w net.ipv4.ip_forward=1\r\n\t```\r\n\r\n### Orca's DRL Agent \r\n\r\n#### Prerequisites\r\n\r\n- Python >= 3.4\r\n- Tensorflow 1.14.0 \r\n\r\n##### Create a virtul environment\r\n```\r\nmkdir ~/venv\r\nsudo apt update\r\nsudo apt install python3-pip\r\nsudo pip3 install -U virtualenv\r\nvirtualenv ~/venv -p python3\r\n```\r\nIf you change the path of the virtual environment, you need to change the source code accordingly. The source code assumes that virtual environment is set @ ~/venv .\r\n\r\n##### Install packages\r\n```\r\nsource ~/venv/bin/activate\r\npip install --upgrade pip\r\npip install gym\r\npip install tensorflow==1.14\r\npip install sysv_ipc\r\n```\r\n\r\nVerify Installation\r\n```\r\npython -c \"import tensorflow as tf; tf.enable_eager_execution(); print(tf.reduce_sum(tf.random_normal([1000, 1000])))\"\r\n```\r\n\r\nTo deactivate venv\r\n```\r\n(venv) $ deactivate\r\n```\r\n\r\n### Patching Orca Kernel: Install the prepared debian packages.\r\n\r\n```\r\ncd linux\r\nsudo dpkg -i linux-image*\r\nsudo dpkg -i linux-header*\r\nsudo reboot \r\n```\r\n \r\n### Verify the new kernel\r\nUse the following command to make sure that new kernel is installed:\r\n\r\n```\r\nuname -r\r\n```\r\n\r\nThe output should be 4.13.1-0521*. If not you need to bring the 4.13.1-0521* Kernel image on top of the grub list. For instance, you can use grub-customizer application. Install the grub-customizer using following:\r\n\r\n```\r\nsudo add-apt-repository ppa:danielrichter2007/grub-customizer\r\nsudo apt-get update\r\nsudo apt-get install grub-customizer\r\nsudo grub-customizer\r\n```\r\n\r\n### Build Orca's Server-Client Apps\r\n To build the required applications, run the following:\r\n\r\n```\r\n./build.sh\r\n```\r\n\r\n This release includes two versions: A standalone actor version and an actor-learner version.\r\nIn the Standalone actor version, no learner will be initiated. This can be usefull when you simply wanna test the current model over an emulated link. However, the actor-learner version requires a learner being initiated before any actor can be started. The second mode is usefull for learning a new/better model and also, it still can be used for performing a simple test over emulated links.\r\n\r\n### Run a Sample Test with the standalone version using the provided learned model\r\n \r\n```\r\n./orca-standalone-emulation.sh 44444\r\n``` \r\n\r\n### Run a Sample Test with the acotr-learner version using the provided learned model\r\n\r\n```\r\n./orca.sh 4 44444\r\n``` \r\n\r\n\r\nResults will be generated automatically in rl-module/log/*\r\nYou can check out the summary of results at rl-module/log/sum-*\r\n\r\n### Some notes on the training:\r\n1. To distribute the actors over remote servers, you need to change \"learner_ip\" and \"actor_ip\" fields of the params.json file to the corresponding server IPs.\r\n2. Start the learner first.\r\n3. Set `remote:true` when using remote servers.\r\n4. Set `num_actors: N`, N is the number of actors.\r\n\r\n### Cellular traces:\r\nTo use traces avaialbe @ https://github.com/Soheil-ab/Cellular-Traces-NYC , copy them to the traces folder of the project.\r\n","note":"Don't delete this file! It's used internally to help with page regeneration."}
params_base.json
-120
View File
@@ -1,120 +0,0 @@
{
"TARGET_MARGIN": 1,
"device": "/cpu",
"PER": false,
"CDQ": true,
"single_actor_eval": false,
"LOSS_TYPE": "MSE",
"use_hard_target": false,
"hard_target": 500,
"MVWIN": 2,
"batch_size": 512,
"use_TCP": true,
"use_normalizer": false,
"h1_shape": 256,
"h2_shape": 256,
"recurrent": true,
"rec_dim": 10,
"noise_type": 3,
"noise_exp": 50000,
"stddev": 0.2,
"update_delay": 5,
"max_eps_steps": 64,
"max_epochs": 1000000,
"eval_frequency": 1000,
"state_dim": 7,
"input_dim": 15,
"action_dim": 1,
"delay_margin_coef": 1.25,
"remote": true,
"dequeue_length": 100,
"lr_a": 0.0001,
"lr_c": 0.001,
"tau": 0.001,
"gamma": 0.995,
"memsize": 2553600,
"tb_interval": 1,
"ckptdir": null,
"scheme": null,
"num_actors": 1,
"logdir": "train_dir",
"learner_ip": "127.0.0.1:33330",
"actor_ip": [
"127.0.0.1:33331",
"127.0.0.1:33332",
"127.0.0.1:33333",
"127.0.0.1:33334",
"127.0.0.1:33335",
"127.0.0.1:33336",
"127.0.0.1:33337",
"127.0.0.1:33338",
"127.0.0.1:33339",
"127.0.0.1:33340",
"127.0.0.1:33341",
"127.0.0.1:33342",
"127.0.0.1:33343",
"127.0.0.1:33344",
"127.0.0.1:33345",
"127.0.0.1:33346",
"127.0.0.1:33347",
"127.0.0.1:33348",
"127.0.0.1:33349",
"127.0.0.1:33350",
"127.0.0.1:33351",
"127.0.0.1:33352",
"127.0.0.1:33353",
"127.0.0.1:33354",
"127.0.0.1:33355",
"127.0.0.1:33356",
"127.0.0.1:33357",
"127.0.0.1:33358",
"127.0.0.1:33359",
"127.0.0.1:33360",
"127.0.0.1:33361",
"127.0.0.1:33362",
"127.0.0.1:33363",
"127.0.0.1:33364",
"127.0.0.1:33365",
"127.0.0.1:33366",
"127.0.0.1:33367",
"127.0.0.1:33368",
"127.0.0.1:33369",
"127.0.0.1:33370",
"127.0.0.1:33381",
"127.0.0.1:33382",
"127.0.0.1:33383",
"127.0.0.1:33384",
"127.0.0.1:33385",
"127.0.0.1:33386",
"127.0.0.1:33387",
"127.0.0.1:33388",
"127.0.0.1:33389",
"127.0.0.1:33380",
"127.0.0.1:33391",
"127.0.0.1:33392",
"127.0.0.1:33393",
"127.0.0.1:33394",
"127.0.0.1:33395",
"127.0.0.1:33396",
"127.0.0.1:33397",
"127.0.0.1:33398",
"127.0.0.1:33399",
"127.0.0.1:33400",
"127.0.0.1:33401",
"127.0.0.1:33402",
"127.0.0.1:33403",
"127.0.0.1:33404",
"127.0.0.1:33405",
"127.0.0.1:33406",
"127.0.0.1:33407",
"127.0.0.1:33408",
"127.0.0.1:33409",
"127.0.0.1:33410"
]
}
params_base_eval.json
-120
View File
@@ -1,120 +0,0 @@
{
"TARGET_MARGIN": 1,
"device": "/cpu",
"PER": false,
"CDQ": true,
"single_actor_eval": true,
"LOSS_TYPE": "MSE",
"use_hard_target": false,
"hard_target": 500,
"MVWIN": 2,
"batch_size": 512,
"use_TCP": true,
"use_normalizer": false,
"h1_shape": 256,
"h2_shape": 256,
"recurrent": true,
"rec_dim": 10,
"noise_type": 3,
"noise_exp": 50000,
"stddev": 0.2,
"update_delay": 5,
"max_eps_steps": 64,
"max_epochs": 1000000,
"eval_frequency": 1000,
"state_dim": 7,
"input_dim": 15,
"action_dim": 1,
"delay_margin_coef": 1.25,
"remote": true,
"dequeue_length": 100,
"lr_a": 0.0001,
"lr_c": 0.001,
"tau": 0.001,
"gamma": 0.995,
"memsize": 2553600,
"tb_interval": 1,
"ckptdir": "train_dir/learner0",
"scheme": null,
"num_actors": 1,
"logdir": "train_dir",
"learner_ip": "127.0.0.1:33330",
"actor_ip": [
"127.0.0.1:33331",
"127.0.0.1:33332",
"127.0.0.1:33333",
"127.0.0.1:33334",
"127.0.0.1:33335",
"127.0.0.1:33336",
"127.0.0.1:33337",
"127.0.0.1:33338",
"127.0.0.1:33339",
"127.0.0.1:33340",
"127.0.0.1:33341",
"127.0.0.1:33342",
"127.0.0.1:33343",
"127.0.0.1:33344",
"127.0.0.1:33345",
"127.0.0.1:33346",
"127.0.0.1:33347",
"127.0.0.1:33348",
"127.0.0.1:33349",
"127.0.0.1:33350",
"127.0.0.1:33351",
"127.0.0.1:33352",
"127.0.0.1:33353",
"127.0.0.1:33354",
"127.0.0.1:33355",
"127.0.0.1:33356",
"127.0.0.1:33357",
"127.0.0.1:33358",
"127.0.0.1:33359",
"127.0.0.1:33360",
"127.0.0.1:33361",
"127.0.0.1:33362",
"127.0.0.1:33363",
"127.0.0.1:33364",
"127.0.0.1:33365",
"127.0.0.1:33366",
"127.0.0.1:33367",
"127.0.0.1:33368",
"127.0.0.1:33369",
"127.0.0.1:33370",
"127.0.0.1:33381",
"127.0.0.1:33382",
"127.0.0.1:33383",
"127.0.0.1:33384",
"127.0.0.1:33385",
"127.0.0.1:33386",
"127.0.0.1:33387",
"127.0.0.1:33388",
"127.0.0.1:33389",
"127.0.0.1:33380",
"127.0.0.1:33391",
"127.0.0.1:33392",
"127.0.0.1:33393",
"127.0.0.1:33394",
"127.0.0.1:33395",
"127.0.0.1:33396",
"127.0.0.1:33397",
"127.0.0.1:33398",
"127.0.0.1:33399",
"127.0.0.1:33400",
"127.0.0.1:33401",
"127.0.0.1:33402",
"127.0.0.1:33403",
"127.0.0.1:33404",
"127.0.0.1:33405",
"127.0.0.1:33406",
"127.0.0.1:33407",
"127.0.0.1:33408",
"127.0.0.1:33409",
"127.0.0.1:33410"
]
}
rl-module/agent.py
-411
View File
@@ -1,411 +0,0 @@
'''
MIT License
Copyright (c) Chen-Yu Yen 2020
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
'''
import tensorflow as tf
import numpy as np
import os
import time
EXPLORE = 4000
STDDEV = 0.1
NSTEP = 0.3
from utils import OU_Noise, ReplayBuffer, G_Noise, Prioritized_ReplayBuffer
def create_input_op_shape(obs, tensor):
input_shape = [x or -1 for x in tensor.shape.as_list()]
return np.reshape(obs, input_shape)
class Actor():
def __init__(self, s_dim, a_dim,h1_shape,h2_shape, action_scale=1.0, name='actor'):
self.s_dim = s_dim
self.a_dim = a_dim
self.name = name
self.action_scale = action_scale
self.h1_shape = h1_shape
self.h2_shape = h2_shape
def train_var(self):
return tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope=self.name)
def build(self, s, is_training):
with tf.variable_scope(self.name, reuse=tf.AUTO_REUSE):
h1 = tf.layers.dense(s, units=self.h1_shape, name='fc1')
h1 = tf.layers.batch_normalization(h1, training=is_training, scale=False)
h1 = tf.nn.leaky_relu(h1)
h2 = tf.layers.dense(h1, units=self.h2_shape, name='fc2')
h2 = tf.layers.batch_normalization(h2, training=is_training, scale=False)
h2 = tf.nn.leaky_relu(h2)
output = tf.layers.dense(h2, units=self.a_dim, activation=tf.nn.tanh)
scale_output = tf.multiply(output, self.action_scale)
return scale_output
class Critic():
def __init__(self, s_dim, a_dim,h1_shape,h2_shape, action_scale=1.0, name='critic'):
self.a_dim = a_dim
self.s_dim = s_dim
self.name = name
self.action_scale = action_scale
self.h1_shape = h1_shape
self.h2_shape = h2_shape
def train_var(self):
return tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope=self.name)
def build(self, s, action):
with tf.variable_scope(self.name, reuse=tf.AUTO_REUSE):
h1 = tf.layers.dense(s, units=self.h1_shape, activation=tf.nn.leaky_relu, name='fc1')
h2 = tf.layers.dense(tf.concat([h1, action], -1), units=self.h2_shape, activation=tf.nn.leaky_relu, name='fc2')
output = tf.layers.dense(h2, units=1)
return output
class Agent():
def __init__(self, s_dim, a_dim, h1_shape,h2_shape,gamma=0.995, batch_size=8, lr_a=1e-4, lr_c=1e-3, tau=1e-3, mem_size=1e5,action_scale=1.0, action_range=(-1.0, 1.0),
noise_type=3, noise_exp=50000, summary=None,stddev=0.1, PER=False, alpha=0.6, CDQ=True, LOSS_TYPE='HUBERT'):
self.PER = PER
self.CDQ = CDQ
self.LOSS_TYPE = LOSS_TYPE
self.lr_a = lr_a
self.lr_c = lr_c
self.s_dim = s_dim
self.a_dim = a_dim
self.gamma = gamma
self.train_dir = './train_dir'
self.step_epochs = tf.Variable(0, trainable=False, name='epoch')
self.global_step = tf.train.get_or_create_global_step(graph=None)
self.s0 = tf.placeholder(tf.float32, shape=[None, self.s_dim], name='s0')
self.s1 = tf.placeholder(tf.float32, shape=[None, self.s_dim], name='s1')
self.is_training = tf.placeholder(tf.bool, name='Actor_is_training')
self.action = tf.placeholder(tf.float32, shape=[None, a_dim], name='action')
self.noise_type = noise_type
self.noise_exp = noise_exp
self.action_range = action_range
self.h1_shape=h1_shape
self.h2_shape=h2_shape
self.stddev=stddev
if not self.PER:
self.rp_buffer = ReplayBuffer(int(mem_size), s_dim, a_dim, batch_size=batch_size)
else:
self.rp_buffer = Prioritized_ReplayBuffer(int(mem_size), s_dim, a_dim, batch_size=batch_size, alpha=alpha)
if noise_type == 1:
self.actor_noise = OU_Noise(mu=np.zeros(a_dim), sigma=float(self.stddev) * np.ones(a_dim),dt=1,exp=self.noise_exp)
elif noise_type == 2:
## Gaussian with gradually decay
self.actor_noise = G_Noise(mu=np.zeros(a_dim), sigma=float(self.stddev) * np.ones(a_dim), explore =self.noise_exp)
elif noise_type == 3:
## Gaussian without gradually decay
self.actor_noise = G_Noise(mu=np.zeros(a_dim), sigma=float(self.stddev) * np.ones(a_dim), explore = None,theta=0.1)
elif noise_type == 4:
## Gaussian without gradually decay
self.actor_noise = G_Noise(mu=np.zeros(a_dim), sigma=float(self.stddev) * np.ones(a_dim), explore = EXPLORE,theta=0.1,mode="step",step=NSTEP)
elif noise_type == 5:
self.actor_noise = None
else:
self.actor_noise = OU_Noise(mu=np.zeros(a_dim), sigma=float(self.stddev) * np.ones(a_dim),dt=0.5)
# Main Actor/Critic Network
self.actor = Actor(self.s_dim, self.a_dim, action_scale=action_scale,h1_shape=self.h1_shape,h2_shape=self.h2_shape)
self.critic = Critic(self.s_dim, self.a_dim, action_scale=action_scale,h1_shape=self.h1_shape,h2_shape=self.h2_shape)
self.critic2 = Critic(self.s_dim, self.a_dim, action_scale=action_scale, name='critic2',h1_shape=self.h1_shape,h2_shape=self.h2_shape)
self.actor_out = self.actor.build(self.s0, self.is_training)
self.critic_out = self.critic.build(self.s0, self.action)
self.critic_out2 = self.critic2.build(self.s0, self.action)
self.critic_actor_out = self.critic.build(self.s0, self.actor_out)
# Target Actor/Critic network
self.target_actor = Actor(self.s_dim, self.a_dim, action_scale=action_scale,h1_shape=self.h1_shape,h2_shape=self.h2_shape,name="target_actor")
self.target_critic = Critic(self.s_dim, self.a_dim, action_scale=action_scale ,h1_shape=self.h1_shape,h2_shape=self.h2_shape,name='target_critic')
self.target_critic2 = Critic(self.s_dim, self.a_dim, action_scale=action_scale, name='target_critic2',h1_shape=self.h1_shape,h2_shape=self.h2_shape)
self.target_actor_out = self.target_actor.build(self.s1, self.is_training)
self.target_actor_policy = self.get_target_actor_policy()
self.target_critic_actor_out = self.target_critic.build(self.s1, self.target_actor_policy)
self.target_critic_actor_out2 = self.target_critic2.build(self.s1, self.target_actor_policy)
self.target_actor_update_op = self.target_update_op(self.target_actor.train_var(), self.actor.train_var(), tau)
self.target_critic_update_op = self.target_update_op(self.target_critic.train_var(), self.critic.train_var(), tau)
self.target_critic_update_op2 = self.target_update_op(self.target_critic2.train_var(), self.critic2.train_var(), tau)
self.target_act_init_op = self.target_init(self.target_actor.train_var(), self.actor.train_var())
self.target_cri_init_op = self.target_init(self.target_critic.train_var(), self.critic.train_var())
self.target_cri_init_op2 = self.target_init(self.target_critic2.train_var(), self.critic2.train_var())
self.extra_update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
self.terminal = tf.placeholder(tf.float32, shape=[None, 1], name='is_terminal')
self.reward = tf.placeholder(tf.float32, shape=[None, 1], name='reward')
self.y = self.reward + self.gamma * (1-self.terminal) * self.target_critic_actor_out
self.y2 = self.reward + self.gamma * (1-self.terminal) * self.target_critic_actor_out2
self.importance = tf.placeholder(tf.float32, [None,1], name='imporance_weights')
self.td_error = self.critic_out - self.y
self.summary_writer = summary
def build_learn(self):
self.actor_optimizer = tf.train.AdamOptimizer(self.lr_a)
self.critic_optimizer = tf.train.AdamOptimizer(self.lr_c)
self.actor_train_op = self.build_actor_train_op()
use_huber = True
if use_huber:
self.critic_train_op = self.build_critic_train_op_huber()
else:
self.critic_train_op = self.build_critic_train_op()
def build_critic_train_op_huber(self):
def f1(y,pred, weights=1.0):
error = tf.square(y - pred)
weighted_error = tf.reduce_mean(error*weights)
return weighted_error
loss_function = {
'HUBER':tf.compat.v1.losses.huber_loss,
'MSE': f1
}
if self.CDQ:
q_min_target = tf.minimum(self.y, self.y2)
if self.PER:
self.c_loss = loss_function[self.LOSS_TYPE](q_min_target, self.critic_out, weights=self.importance)
self.c_loss2 = loss_function[self.LOSS_TYPE](q_min_target, self.critic_out2, weights=self.importance)
else:
self.c_loss = loss_function[self.LOSS_TYPE](q_min_target, self.critic_out)
self.c_loss2 = loss_function[self.LOSS_TYPE](q_min_target, self.critic_out2)
critic_op = []
critic_op.append(self.critic_optimizer.minimize(self.c_loss, var_list=self.critic.train_var(), global_step = self.global_step))
critic_op.append(self.critic_optimizer.minimize(self.c_loss2, var_list=self.critic2.train_var(), global_step = self.global_step))
return critic_op
else:
if self.PER:
self.critic_loss = loss_function[self.LOSS_TYPE](self.y, self.critic_out, weights=self.importance)
else:
self.critic_loss = loss_function[self.LOSS_TYPE](self.y, self.critic_out)
loss_op = self.critic_optimizer.minimize(self.critic_loss, var_list=self.critic.train_var(), global_step = self.global_step)
return loss_op
def create_tf_summary(self):
if self.CDQ:
tf.summary.scalar('Loss/critic_loss:', self.c_loss)
tf.summary.scalar('Loss/critic_loss_2:', self.c_loss2)
else:
tf.summary.scalar('Loss/critic_loss:', self.critic_loss)
tf.summary.scalar('Loss/actor_loss:', self.a_loss)
self.summary_op = tf.summary.merge_all()
def init_target(self):
self.sess.run(self.target_act_init_op)
self.sess.run(self.target_cri_init_op)
self.sess.run(self.target_cri_init_op2)
def get_target_actor_policy(self):
eps = tf.random_normal(tf.shape(self.target_actor_out), stddev=0.1)
eps = tf.clip_by_value(eps, -0.2, 0.2)
t_a = self.target_actor_out + eps
t_a = tf.clip_by_value(t_a, -1.0, 1.0)
return t_a
def assign_sess(self, sess):
self.sess = sess
def build_critic_train_op(self):
critic_op = []
_q_min_target = tf.minimum(self.y, self.y2)
q_min_target = _q_min_target
self.c_loss = tf.reduce_mean(tf.square(q_min_target - self.critic_out))
self.c_loss2 = tf.reduce_mean(tf.square(q_min_target - self.critic_out2))
critic_op.append(self.critic_optimizer.minimize(self.c_loss, var_list=self.critic.train_var(), global_step = self.global_step))
critic_op.append(self.critic_optimizer.minimize(self.c_loss2, var_list=self.critic2.train_var(), global_step = self.global_step))
tf.summary.scalar('Loss/critic_loss:', self.c_loss)
tf.summary.scalar('Loss/critic_loss_2:', self.c_loss2)
return critic_op
def build_actor_train_op(self):
self.a_loss = -tf.reduce_mean(self.critic_actor_out)
return self.actor_optimizer.minimize(self.a_loss, var_list=self.actor.train_var(), global_step = self.global_step)
def target_init(self, target, vars):
return [tf.assign(target[i], vars[i]) for i in range(len(vars))]
def target_update_op(self, target, vars, tau):
return [tf.assign(target[i], vars[i] * tau + target[i] * (1 - tau)) for i in range(len(vars))]
def target_update_hard_op(self, target, vars):
return [tf.assign(target[i], vars[i]) for i in range(len(vars))]
def target_update(self):
self.sess.run([self.target_actor_update_op, self.target_critic_update_op, self.target_critic_update_op2])
def actor_clone_update(self):
self.sess.run(self.actor_clone_update_op)
def get_action(self, s, use_noise=True):
fd = {self.s0: create_input_op_shape(s, self.s0), self.is_training:False}
action = self.sess.run([self.actor_out], feed_dict=fd)
if use_noise:
noise = self.actor_noise(action[0])
action += noise
action = np.clip(action, self.action_range[0], self.action_range[1])
return action
def get_q(self, s, a):
fd = {self.s0: create_input_op_shape(s, self.s0),
self.action: create_input_op_shape(a, self.action)}
return self.sess.run([self.critic_out], feed_dict=fd)
def get_q_actor(self, s):
fd = {self.s0: create_input_op_shape(s, self.s0)}
return self.sess.run([self.critic_actor_out], feed_dict=fd)
def store_experience(self, s0, a, r, s1, terminal):
self.rp_buffer.store(s0, a, r, s1, terminal)
def store_many_experience(self, s0, a, r, s1, terminal, length):
if self.PER:
for i in range(length):
self.rp_buffer.store(s0[i], a[i], r[i], s1[i], terminal[i])
else:
self.rp_buffer.store_many(s0, a, r, s1, terminal, length)
def sample_experince(self):
return self.rp_buffer.sample()
def train_step_td(self):
return None
def train_step(self):
extra_update_ops = [v for v in tf.get_collection(tf.GraphKeys.UPDATE_OPS) if
"actor" in v.name and "target" not in v.name]
if self.PER == True:
batch_samples, weights, idxes = self.rp_buffer.sample()
t2 = time.time()
fd = {self.s0: create_input_op_shape(batch_samples[0], self.s0),
self.action: create_input_op_shape(batch_samples[1], self.action),
self.reward: create_input_op_shape(batch_samples[2], self.reward),
self.s1: create_input_op_shape(batch_samples[3], self.s1),
self.terminal: create_input_op_shape(batch_samples[4], self.terminal),
self.is_training: True,
self.importance: np.expand_dims(weights,axis=1)
}
else:
batch_samples = self.rp_buffer.sample()
t2 = time.time()
fd = {self.s0: create_input_op_shape(batch_samples[0], self.s0),
self.action: create_input_op_shape(batch_samples[1], self.action),
self.reward: create_input_op_shape(batch_samples[2], self.reward),
self.s1: create_input_op_shape(batch_samples[3], self.s1),
self.terminal: create_input_op_shape(batch_samples[4], self.terminal),
self.is_training: True
}
if self.PER:
_, td_errors = self.sess.run([self.critic_train_op, self.td_error], feed_dict=fd)
else:
self.sess.run([self.critic_train_op], feed_dict=fd)
self.sess.run([self.actor_train_op, extra_update_ops], feed_dict=fd)
summary, step = self.sess.run([self.summary_op, self.global_step], feed_dict=fd)
self.summary_writer.add_summary(summary, global_step=step)
if self.PER:
new_priorities = np.abs(np.squeeze(td_errors)) + 1e-6
self.rp_buffer.update_priorities(idxes, new_priorities)
def log_tf(self, val, tag=None, step_counter=0):
summary = tf.Summary()
summary.value.add(tag= tag, simple_value=val)
self.summary_writer.add_summary(summary, step_counter)
def save_model(self, step=None):
self.saver.save(self.sess, os.path.join(self.train_dir, 'model'), global_step =step)
def load_model(self, name=None):
if name is not None:
print(os.path.join(self.train_dir, name))
self.saver.restore(self.sess, os.path.join(self.train_dir, name))
else:
self.saver.restore(self.sess, tf.train.latest_checkpoint(self.train_dir))
def updat_step_epochs(self, epoch):
self.sess.run(tf.assign(self.step_epochs, epoch))
def get_step_epochs(self):
return self.sess.run(self.step_epochs)
rl-module/client
BIN
View File
Binary file not shown.
rl-module/d5.py
-458
View File
@@ -1,458 +0,0 @@
'''
MIT License
Copyright (c) Chen-Yu Yen - Soheil Abbasloo 2020
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
'''
import threading
import logging
import tensorflow as tf
import sys
from agent import Agent
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
import argparse
import gym
import numpy as np
import time
import random
import datetime
import sysv_ipc
import signal
import pickle
from utils import logger, Params
from envwrapper import Env_Wrapper, TCP_Env_Wrapper, GYM_Env_Wrapper
def create_input_op_shape(obs, tensor):
input_shape = [x or -1 for x in tensor.shape.as_list()]
return np.reshape(obs, input_shape)
def evaluate_TCP(env, agent, epoch, summary_writer, params, s0_rec_buffer, eval_step_counter):
score_list = []
eval_times = 1
eval_length = params.dict['max_eps_steps']
start_time = time.time()
for _ in range(eval_times):
step_counter = 0
ep_r = 0.0
if not params.dict['use_TCP']:
s0 = env.reset()
if params.dict['recurrent']:
a = agent.get_action(s0_rec_buffer, False)
else:
a = agent.get_action(s0, False)
a = a[0][0]
env.write_action(a)
while True:
eval_step_counter += 1
step_counter += 1
s1, r, terminal, error_code = env.step(a, eval_=True)
if error_code == True:
s1_rec_buffer = np.concatenate( (s0_rec_buffer[params.dict['state_dim']:], s1) )
if params.dict['recurrent']:
a1 = agent.get_action(s1_rec_buffer, False)
else:
a1 = agent.get_action(s1, False)
a1 = a1[0][0]
env.write_action(a1)
else:
print("Invalid state received...\n")
env.write_action(a)
continue
ep_r = ep_r+r
if (step_counter+1) % params.dict['tb_interval'] == 0:
summary = tf.summary.Summary()
summary.value.add(tag='Eval/Step/0-Actions', simple_value=env.map_action(a))
summary.value.add(tag='Eval/Step/2-Reward', simple_value=r)
summary_writer.add_summary(summary, eval_step_counter)
s0 = s1
a = a1
if params.dict['recurrent']:
s0_rec_buffer = s1_rec_buffer
if step_counter == eval_length or terminal:
score_list.append(ep_r)
break
summary = tf.summary.Summary()
summary.value.add(tag='Eval/Return', simple_value=np.mean(score_list))
summary_writer.add_summary(summary, epoch)
return eval_step_counter
class learner_killer():
def __init__(self, buffer):
self.replay_buf = buffer
print("learner register sigterm")
signal.signal(signal.SIGTERM, self.handler_term)
print("test length:", self.replay_buf.length_buf)
def handler_term(self, signum, frame):
if not config.eval:
with open(os.path.join(params.dict['train_dir'], "replay_memory.pkl"), "wb") as fp:
pickle.dump(self.replay_buf, fp)
print("test length:", self.replay_buf.length_buf)
print("--------------------------Learner: Saving rp memory--------------------------")
print("-----------------------Learner's killed---------------------")
sys.exit(0)
def main():
tf.get_logger().setLevel(logging.ERROR)
parser = argparse.ArgumentParser()
parser.add_argument('--load', action='store_true', default=False, help='default is %(default)s')
parser.add_argument('--eval', action='store_true', default=False, help='default is %(default)s')
parser.add_argument('--tb_interval', type=int, default=1)
parser.add_argument('--train_dir', type=str, default=None)
parser.add_argument('--mem_r', type=int, default = 123456)
parser.add_argument('--mem_w', type=int, default = 12345)
parser.add_argument('--base_path',type=str, required=True)
parser.add_argument('--job_name', type=str, choices=['learner', 'actor'], required=True, help='Job name: either {\'learner\', actor}')
parser.add_argument('--task', type=int, required=True, help='Task id')
## parameters from parser
global config
global params
config = parser.parse_args()
## parameters from file
params = Params(os.path.join(config.base_path,'params.json'))
if params.dict['single_actor_eval']:
local_job_device = ''
shared_job_device = ''
def is_actor_fn(i): return True
global_variable_device = '/cpu'
is_learner = False
server = tf.train.Server.create_local_server()
filters = []
else:
local_job_device = '/job:%s/task:%d' % (config.job_name, config.task)
shared_job_device = '/job:learner/task:0'
is_learner = config.job_name == 'learner'
global_variable_device = shared_job_device + '/cpu'
def is_actor_fn(i): return config.job_name == 'actor' and i == config.task
if params.dict['remote']:
cluster = tf.train.ClusterSpec({
'actor': params.dict['actor_ip'][:params.dict['num_actors']],
'learner': [params.dict['learner_ip']]
})
else:
cluster = tf.train.ClusterSpec({
'actor': ['localhost:%d' % (8001 + i) for i in range(params.dict['num_actors'])],
'learner': ['localhost:8000']
})
server = tf.train.Server(cluster, job_name=config.job_name,
task_index=config.task)
filters = [shared_job_device, local_job_device]
if params.dict['use_TCP']:
env_str = "TCP"
env_peek = TCP_Env_Wrapper(env_str, params,use_normalizer=params.dict['use_normalizer'])
else:
env_str = 'YourEnvironment'
env_peek = Env_Wrapper(env_str)
s_dim, a_dim = env_peek.get_dims_info()
action_scale, action_range = env_peek.get_action_info()
if not params.dict['use_TCP']:
params.dict['state_dim'] = s_dim
if params.dict['recurrent']:
s_dim = s_dim * params.dict['rec_dim']
if params.dict['use_hard_target'] == True:
params.dict['tau'] = 1.0
with tf.Graph().as_default(),\
tf.device(local_job_device + '/cpu'):
tf.set_random_seed(1234)
random.seed(1234)
np.random.seed(1234)
actor_op = []
now = datetime.datetime.now()
tfeventdir = os.path.join( config.base_path, params.dict['logdir'], config.job_name+str(config.task) )
params.dict['train_dir'] = tfeventdir
if not os.path.exists(tfeventdir):
os.makedirs(tfeventdir)
summary_writer = tf.summary.FileWriterCache.get(tfeventdir)
with tf.device(shared_job_device):
agent = Agent(s_dim, a_dim, batch_size=params.dict['batch_size'], summary=summary_writer,h1_shape=params.dict['h1_shape'],
h2_shape=params.dict['h2_shape'],stddev=params.dict['stddev'],mem_size=params.dict['memsize'],gamma=params.dict['gamma'],
lr_c=params.dict['lr_c'],lr_a=params.dict['lr_a'],tau=params.dict['tau'],PER=params.dict['PER'],CDQ=params.dict['CDQ'],
LOSS_TYPE=params.dict['LOSS_TYPE'],noise_type=params.dict['noise_type'],noise_exp=params.dict['noise_exp'])
dtypes = [tf.float32, tf.float32, tf.float32, tf.float32, tf.float32]
shapes = [[s_dim], [a_dim], [1], [s_dim], [1]]
queue = tf.FIFOQueue(10000, dtypes, shapes, shared_name="rp_buf")
if is_learner:
with tf.device(params.dict['device']):
agent.build_learn()
agent.create_tf_summary()
if config.load is True and config.eval==False:
if os.path.isfile(os.path.join(params.dict['train_dir'], "replay_memory.pkl")):
with open(os.path.join(params.dict['train_dir'], "replay_memory.pkl"), 'rb') as fp:
replay_memory = pickle.load(fp)
_killsignal = learner_killer(agent.rp_buffer)
for i in range(params.dict['num_actors']):
if is_actor_fn(i):
if params.dict['use_TCP']:
shrmem_r = sysv_ipc.SharedMemory(config.mem_r)
shrmem_w = sysv_ipc.SharedMemory(config.mem_w)
env = TCP_Env_Wrapper(env_str, params, config=config, for_init_only=False, shrmem_r=shrmem_r, shrmem_w=shrmem_w,use_normalizer=params.dict['use_normalizer'])
else:
env = GYM_Env_Wrapper(env_str, params)
a_s0 = tf.placeholder(tf.float32, shape=[s_dim], name='a_s0')
a_action = tf.placeholder(tf.float32, shape=[a_dim], name='a_action')
a_reward = tf.placeholder(tf.float32, shape=[1], name='a_reward')
a_s1 = tf.placeholder(tf.float32, shape=[s_dim], name='a_s1')
a_terminal = tf.placeholder(tf.float32, shape=[1], name='a_terminal')
a_buf = [a_s0, a_action, a_reward, a_s1, a_terminal]
with tf.device(shared_job_device):
actor_op.append(queue.enqueue(a_buf))
if is_learner:
Dequeue_Length = params.dict['dequeue_length']
dequeue = queue.dequeue_many(Dequeue_Length)
queuesize_op = queue.size()
if params.dict['ckptdir'] is not None:
params.dict['ckptdir'] = os.path.join( config.base_path, params.dict['ckptdir'])
print("## checkpoint dir:", params.dict['ckptdir'])
isckpt = os.path.isfile(os.path.join(params.dict['ckptdir'], 'checkpoint') )
print("## checkpoint exists?:", isckpt)
if isckpt== False:
print("\n# # # # # # Warning ! ! ! No checkpoint is loaded, use random model! ! ! # # # # # #\n")
else:
params.dict['ckptdir'] = tfeventdir
tfconfig = tf.ConfigProto(allow_soft_placement=True)
if params.dict['single_actor_eval']:
mon_sess = tf.train.SingularMonitoredSession(
checkpoint_dir=params.dict['ckptdir'])
else:
mon_sess = tf.train.MonitoredTrainingSession(master=server.target,
save_checkpoint_secs=15,
save_summaries_secs=None,
save_summaries_steps=None,
is_chief=is_learner,
checkpoint_dir=params.dict['ckptdir'],
config=tfconfig,
hooks=None)
agent.assign_sess(mon_sess)
if is_learner:
if config.eval is True:
print("=========================Learner is up===================")
while not mon_sess.should_stop():
time.sleep(1)
continue
if config.load is False:
agent.init_target()
counter = 0
start = time.time()
dequeue_thread = threading.Thread(target=learner_dequeue_thread, args=(agent,params, mon_sess, dequeue, queuesize_op, Dequeue_Length),daemon=True)
first_time=True
while not mon_sess.should_stop():
if first_time == True:
dequeue_thread.start()
first_time=False
up_del_tmp=params.dict['update_delay']/1000.0
time.sleep(up_del_tmp)
if agent.rp_buffer.ptr>200 or agent.rp_buffer.full :
agent.train_step()
if params.dict['use_hard_target'] == False:
agent.target_update()
if counter %params.dict['hard_target'] == 0 :
current_opt_step = agent.sess.run(agent.global_step)
logger.info("Optimize step:{}".format(current_opt_step))
logger.info("rp_buffer ptr:{}".format(agent.rp_buffer.ptr))
else:
if counter %params.dict['hard_target'] == 0 :
agent.target_update()
current_opt_step = agent.sess.run(agent.global_step)
logger.info("Optimize step:{}".format(current_opt_step))
logger.info("rp_buffer ptr:{}".format(agent.rp_buffer.ptr))
counter += 1
else:
start = time.time()
step_counter = np.int64(0)
eval_step_counter = np.int64(0)
s0 = env.reset()
s0_rec_buffer = np.zeros([s_dim])
s1_rec_buffer = np.zeros([s_dim])
s0_rec_buffer[-1*params.dict['state_dim']:] = s0
if params.dict['recurrent']:
a = agent.get_action(s0_rec_buffer,not config.eval)
else:
a = agent.get_action(s0, not config.eval)
a = a[0][0]
env.write_action(a)
epoch = 0
ep_r = 0.0
start = time.time()
while True:
start = time.time()
epoch += 1
step_counter += 1
s1, r, terminal, error_code = env.step(a,eval_=config.eval)
if error_code == True:
s1_rec_buffer = np.concatenate( (s0_rec_buffer[params.dict['state_dim']:], s1) )
if params.dict['recurrent']:
a1 = agent.get_action(s1_rec_buffer, not config.eval)
else:
a1 = agent.get_action(s1,not config.eval)
a1 = a1[0][0]
env.write_action(a1)
else:
print("TaskID:"+str(config.task)+"Invalid state received...\n")
env.write_action(a)
continue
if params.dict['recurrent']:
fd = {a_s0:s0_rec_buffer, a_action:a, a_reward:np.array([r]), a_s1:s1_rec_buffer, a_terminal:np.array([terminal], np.float)}
else:
fd = {a_s0:s0, a_action:a, a_reward:np.array([r]), a_s1:s1, a_terminal:np.array([terminal], np.float)}
if not config.eval:
mon_sess.run(actor_op, feed_dict=fd)
s0 = s1
a = a1
if params.dict['recurrent']:
s0_rec_buffer = s1_rec_buffer
if not params.dict['use_TCP'] and (terminal):
if agent.actor_noise != None:
agent.actor_noise.reset()
if (epoch% params.dict['eval_frequency'] == 0):
eval_step_counter = evaluate_TCP(env, agent, epoch, summary_writer, params, s0_rec_buffer, eval_step_counter)
print("total time:", time.time()-start)
def learner_dequeue_thread(agent,params, mon_sess, dequeue, queuesize_op, Dequeue_Length):
ct = 0
while True:
ct = ct + 1
data = mon_sess.run(dequeue)
agent.store_many_experience(data[0], data[1], data[2], data[3], data[4], Dequeue_Length)
time.sleep(0.01)
def learner_update_thread(agent,params):
delay=params.dict['update_delay']/1000.0
ct = 0
while True:
agent.train_step()
agent.target_update()
time.sleep(delay)
if __name__ == "__main__":
main()
rl-module/envwrapper.py
-457
View File
@@ -1,457 +0,0 @@
'''
MIT License
Copyright (c) Soheil Abbasloo - Chen-Yu Yen 2020
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
'''
import numpy as np
import gym
import math
import collections
from utils import logger
import os
import sysv_ipc
import signal
import sys
from time import sleep
class Env_Wrapper(object):
def __init__(self, name):
self.env = gym.make(name)
self.env.reset()
def get_dims_info(self):
return self.env.observation_space.shape[0], self.env.action_space.shape[0]
def get_action_info(self):
action_scale = self.env.action_space.high
action_range = (self.env.action_space.low, self.env.action_space.high)
return action_scale, action_range
def reset(self):
return self.env.reset()
def step(self, action,eval_=False):
s1, r, done, _ = self.env.step(action)
if done:
s1 = self.env.reset()
return s1, r, done, _
class GYM_Env_Wrapper(Env_Wrapper):
def __init__(self,name, params, for_init_only=True, shrmemory_id=None, shrmem_r=None, shrmem_w=None):
super().__init__(name)
logger.info("USE_GYM_Env_Wrapper")
if not for_init_only:
self.shrmemory_id = shrmemory_id
self.shrmem_r = shrmem_r
self.shrmem_w = shrmem_w
self.prev_rid = 99
def test(self):
print("Hello")
def get_state(self):
prev_rid = 0
s1 = None
delay_ = 0
rew0 = 10
error_code = True
return prev_rid, s1, delay_, rew0, error_code
def write_action(self, action):
pass
def map_action(self, action):
out = math.pow(4, action)
out *= 100
out = int(out)
return out
def step(self, action, eval_=False):
s1, r, done, _ = self.env.step(action)
if done:
s1 = self.env.reset()
return s1, r, done, True
class TCP_Env_Wrapper(object):
def __init__(self,name, params, config=None, for_init_only=True, shrmem_r=None, shrmem_w=None,use_normalizer=True):
self.params = params
if not for_init_only:
self.params = params
self.shrmem_r = shrmem_r
self.shrmem_w = shrmem_w
self.prev_rid = 99999
self.wid = 23
self.local_counter=0
self.pre_samples = 0.0
self.new_samples = 0.0
self.avg_delay = 0.0
self.avg_thr = 0.0
self.thr_ = 0.0
self.del_ = 0.0
self.max_bw = 0.0
self.max_cwnd = 0.0
self.max_smp = 0.0
self.min_del = 9999999.0
self.use_normalizer=use_normalizer
if self.use_normalizer==True:
self.normalizer=Normalizer(params, config)
else:
self.normalizer=None
self.del_moving_win = Moving_Win(params.dict['MVWIN'])
self.thr_moving_win = Moving_Win(params.dict['MVWIN'])
self.config = config
signal.signal(signal.SIGINT, self.handler_term)
signal.signal(signal.SIGTERM, self.handler_term)
if self.config.load is not None and self.use_normalizer==True:
_ = self.normalizer.load_stats()
def handler_term(self, signum, frame):
print("python program terminated usking Kill -15")
if not self.config.eval and self.use_normalizer==True:
print("save stats by kill -15")
self.normalizer.save_stats()
sys.exit(0)
def get_dims_info(self):
return self.params.dict['state_dim'], self.params.dict['action_dim']
def get_action_info(self):
action_scale = np.array([1.])
action_range = (-action_scale,action_scale)
print('action_scale & action_range')
return action_scale, action_range
def reset(self):
# start signal
self.shrmem_w.write(str(99999) + " " + str(99999) + "\0")
state, delay_, rew0, error_code = self.get_state()
return state
def test(self):
print("Hello")
def get_state(self, evaluation=False):
succeed = False
error_cnt=0
while(1):
# Read value from shared memory
try:
memory_value = self.shrmem_r.read()
except sysv_ipc.ExistentialError:
print("No shared memory Now, python ends gracefully :)")
logger.info("No shared memory Now, python ends gracefully :)")
sys.exit(0)
memory_value = memory_value.decode('unicode_escape')
i = memory_value.find('\0')
if i != -1:
memory_value = memory_value[:i]
readstate = np.fromstring(memory_value, dtype=float, sep=' ')
try:
rid = readstate[0]
except :
rid = self.prev_rid
sleep(0.01)
continue
try:
s0 = readstate[1:]
except :
print("s0 waring")
sleep(0.01)
continue
if rid != self.prev_rid:
succeed = True
break
else:
wwwwww=""
error_cnt=error_cnt+1
if error_cnt > 24000:
error_cnt=0
print("After 3 min, We didn't get any state from the server. Actor "+str(self.config.task)+" is going down down down ...\n")
sys.exit(0)
sleep(0.01)
error_cnt=0
if succeed == False:
raise ValueError('read Nothing new from shrmem for a long time')
reward=0
state=np.zeros(1)
w=s0
if len(s0) == (self.params.dict['input_dim']):
d=s0[0]
thr=s0[1]
samples=s0[2]
delta_t=s0[3]
target_=s0[4]
cwnd=s0[5]
pacing_rate=s0[6]
loss_rate=s0[7]
srtt_ms=s0[8]
snd_ssthresh=s0[9]
packets_out=s0[10]
retrans_out=s0[11]
max_packets_out=s0[12]
mss=s0[13]
min_rtt=s0[14]
self.local_counter+=1
if self.use_normalizer==True:
if evaluation!=True:
self.normalizer.observe(s0)
s0 = self.normalizer.normalize(s0)
min_ = self.normalizer.stats()
else:
min_ = s0-s0
d_n=s0[0]-min_[0]
thr_n=s0[1]
thr_n_min=s0[1]-min_[1]
samples_n=s0[2]
samples_n_min=s0[2]-min_[2]
delta_t_n=s0[3]
delta_t_n_min=s0[3]-min_[3]
cwnd_n_min=s0[5]-min_[5]
pacing_rate_n_min=s0[6]-min_[6]
loss_rate_n_min=s0[7]-min_[7]
srtt_ms_min=s0[8]-min_[8]
snd_ssthresh_min=s0[9]-min_[9]
packets_out_min=s0[10]-min_[10]
retrans_out_min=s0[11]-min_[11]
max_packets_out_min=s0[12]-min_[12]
mss_min=mss-min_[13]
min_rtt_min=min_rtt-min_[14]
if self.use_normalizer==False:
thr_n=thr_n
thr_n_min=thr_n_min
samples_n_min=samples_n_min
cwnd_n_min=cwnd_n_min
loss_rate_n_min=loss_rate_n_min
d_n=d_n
if self.max_bw<thr_n_min:
self.max_bw=thr_n_min
if self.max_cwnd<cwnd_n_min:
self.max_cwnd=cwnd_n_min
if self.max_smp<samples_n_min:
self.max_smp=samples_n_min
if self.min_del>d_n:
self.min_del=d_n
################# Transfer all of the vars. to Rate/Max(Rate) space
#cwnd_bytes= cwnd_n_min*mss_min
#cwnd_n_min=(cwnd_bytes*1000)/srtt_ms_min
#snd_ssthresh_min=(snd_ssthresh_min*mss_min*1000)/srtt_ms_min
#packets_out_min=(packets_out_min*mss_min*1000)/srtt_ms_min
#retrans_out_min=(retrans_out_min*mss_min*1000)/srtt_ms_min
#max_packets_out_min=(max_packets_out_min*mss_min*1000)/srtt_ms_min
#inflight_bytes=(packets_out-samples)*mss_min*1000
if min_rtt_min*(self.params.dict['delay_margin_coef'])<srtt_ms_min:
delay_metric=(min_rtt_min*(self.params.dict['delay_margin_coef']))/srtt_ms_min
else:
delay_metric=1
reward = (thr_n_min-5*loss_rate_n_min)/self.max_bw*delay_metric
if self.max_bw!=0:
state[0]=thr_n_min/self.max_bw
tmp=pacing_rate_n_min/self.max_bw
if tmp>10:
tmp=10
state=np.append(state,[tmp])
state=np.append(state,[5*loss_rate_n_min/self.max_bw])
else:
state[0]=0
state=np.append(state,[0])
state=np.append(state,[0])
state=np.append(state,[samples/cwnd])
state=np.append(state,[delta_t_n])
state=np.append(state,[min_rtt_min/srtt_ms_min])
state=np.append(state,[delay_metric])
self.prev_rid = rid
return state, d, reward, True
else:
return state, 0.0, reward, False
def map_action(self, action):
out = math.pow(4, action)
out *= 100
out = int(out)
return out
def map_action_reverse(self,a):
out = math.log(a/100,4)
return out
def write_action(self, action):
modified_action = self.map_action(action)
msg = str(self.wid)+" "+str(modified_action)+"\0"
self.shrmem_w.write(msg)
self.wid = (self.wid + 1) % 1000
pass
def step(self, action, eval_=False):
s1, delay_, rew0, error_code = self.get_state(evaluation=eval_)
return s1, rew0, False, error_code
class Moving_Win():
def __init__(self,win_size):
self.queue_main = collections.deque(maxlen=win_size)
self.queue_aux = collections.deque(maxlen=win_size)
self.length = 0
self.avg = 0.0
self.size = win_size
self.total_samples=0
def push(self,sample_value,sample_num):
if self.length<self.size:
self.queue_main.append(sample_value)
self.queue_aux.append(sample_num)
self.length=self.length+1
self.avg=(self.avg*self.total_samples+sample_value*sample_num)
self.total_samples+=sample_num
if self.total_samples>0:
self.avg=self.avg/self.total_samples
else:
self.avg=0.0
else:
pop_value=self.queue_main.popleft()
pop_num=self.queue_aux.popleft()
self.queue_main.append(sample_value)
self.queue_aux.append(sample_num)
self.avg=(self.avg*self.total_samples+sample_value*sample_num-pop_value*pop_num)
self.total_samples=self.total_samples+(sample_num-pop_num)
if self.total_samples>0:
self.avg=self.avg/self.total_samples
else:
self.avg=0.0
def get_avg(self):
return self.avg
def get_length(self):
return self.length
class Normalizer():
def __init__(self, params, config):
self.params = params
self.config = config
self.n = 1e-5
num_inputs = self.params.dict['input_dim']
self.mean = np.zeros(num_inputs)
self.mean_diff = np.zeros(num_inputs)
self.var = np.zeros(num_inputs)
self.dim = num_inputs
self.min = np.zeros(num_inputs)
def observe(self, x):
self.n += 1
last_mean = np.copy(self.mean)
self.mean += (x-self.mean)/self.n
self.mean_diff += (x-last_mean)*(x-self.mean)
self.var = self.mean_diff/self.n
def normalize(self, inputs):
obs_std = np.sqrt(self.var)
a=np.zeros(self.dim)
if self.n > 2:
a=(inputs - self.mean)/obs_std
for i in range(0,self.dim):
if a[i] < self.min[i]:
self.min[i] = a[i]
return a
else:
return np.zeros(self.dim)
def normalize_delay(self,delay):
obs_std = math.sqrt(self.var[0])
if self.n > 2:
return (delay - self.mean[0])/obs_std
else:
return 0
def stats(self):
return self.min
def save_stats(self):
dic={}
dic['n']=self.n
dic['mean'] = self.mean.tolist()
dic['mean_diff'] = self.mean_diff.tolist()
dic['var'] = self.var.tolist()
dic['min'] = self.min.tolist()
import json
with open(os.path.join(self.params.dict['train_dir'], 'stats.json'), 'w') as fp:
json.dump(dic, fp)
print("--------save stats at{}--------".format(self.params.dict['train_dir']))
logger.info("--------save stats at{}--------".format(self.params.dict['train_dir']))
def load_stats(self, file='stats.json'):
import json
if os.path.isfile(os.path.join(self.params.dict['train_dir'], file)):
print("Stats exist!, load", self.config.task)
with open(os.path.join(self.params.dict['train_dir'], file), 'r') as fp:
history_stats = json.load(fp)
print(history_stats)
self.n = history_stats['n']
self.mean = np.asarray(history_stats['mean'])
self.mean_diff = np.asarray(history_stats['mean_diff'])
self.var = np.asarray(history_stats['var'])
self.min = np.asarray(history_stats['min'])
return True
else:
print("stats file is missing when loading")
return False
rl-module/log/empty
-1
View File
@@ -1 +0,0 @@
empty file
rl-module/mm-del-file
-193
View File
@@ -1,193 +0,0 @@
#!/usr/bin/perl -w
use warnings;
use strict;
use POSIX;
sub usage
{
die qq{Usage: $0 [filename]\n};
}
if ( scalar @ARGV > 1 ) {
usage;
}
my $first_timestamp = undef;
my $last_timestamp = undef;
my $base_timestamp = undef;
my $delay_pre = 0;
my $timestamp = 0;
my $timestamp_pre = 0;
my @delays;
my %signal_delay;
my $points;
my $filename = $ARGV[0];
#print "filename: $filename\n";
open(my $delay_fh, '>', "$filename.dat");
#print "filename: $filename\n";
open(my $jitter_fh, '>', "$filename-jitter.dat");
#die {$!};
LINE: while ( <> ) {
chomp;
if ( m{^# base timestamp: (\d+)} ) {
if ( defined $base_timestamp ) {
die "base timestamp multiply defined";
} else {
$base_timestamp = $1;
}
next LINE;
} elsif ( m{^#} ) {
next LINE;
}
# parse and validate line
my ( $timestamp, $event_type, $num_bytes, $delay ) = split /\s+/, $_;
if ( not defined $num_bytes ) {
die q{Format: timestamp event_type num_bytes [delay]};
}
if ( $timestamp !~ m{^\d+$} ) {
die qq{Invalid timestamp: $timestamp};
}
if ( $num_bytes !~ m{^\d+$} ) {
die qq{Invalid byte count: $num_bytes};
}
if ( not defined $base_timestamp ) {
die "logfile is missing base timestamp";
}
$timestamp -= $base_timestamp; # correct for startup time variation
if ( $timestamp > 0) {
# $timestamp -= 30000; # correct for startup time variation
if ( not defined $last_timestamp ) {
$last_timestamp = $first_timestamp = $timestamp;
}
$last_timestamp = max( $timestamp, $last_timestamp );
# process the event
if ( $event_type eq q{-} ) {
if ( not defined $delay ) {
die q{Departure format: timestamp - num_bytes delay};
}
push @delays, $delay;
$signal_delay{ $timestamp - $delay } = min( $delay, $signal_delay{ $timestamp - $delay } );
$points .= ($timestamp - $delay) / 1000.0 . " $delay\n";
my $tt=($timestamp - $delay) / 1000.0;
my $timestamp_tmp=$timestamp-$timestamp_pre;
if ( $timestamp_tmp > 0 ) {
my $jitter_t=($timestamp_tmp/2+$timestamp_pre)/1000;
my $jitter=($delay-$delay_pre); #/$timestamp_tmp;
$timestamp_pre=$timestamp;
$delay_pre=$delay;
print $delay_fh qq{$tt $delay\n};
print $jitter_fh qq{$jitter_t $jitter\n};
}
}
}
}
sub max {
my $maxval = - POSIX::DBL_MAX;
for ( @_ ) {
next unless defined $_;
if ( $_ > $maxval ) {
$maxval = $_;
}
}
return $maxval;
}
sub min {
my $minval = POSIX::DBL_MAX;
for ( @_ ) {
next unless defined $_;
if ( $_ < $minval ) {
$minval = $_;
}
}
return $minval;
}
# calculate statistics
if ( scalar @delays == 0 ) {
die q{Must have at least one departure event};
}
@delays = sort { $a <=> $b } @delays;
my $pp95 = $delays[ 0.95 * scalar @delays ];
my $pp50 = $delays[ 0.5 * scalar @delays ];
# measure signal delay every millisecond
# = minimum time for a message created at time t to get to receiver
my @signal_delay_samples = sort { $a <=> $b } keys %signal_delay;
for ( my $ts = $signal_delay_samples[ -1 ]; $ts >= $signal_delay_samples[ 0 ]; $ts-- ) {
if ( not defined $signal_delay{ $ts } ) {
$signal_delay{ $ts } = $signal_delay{ $ts + 1 } + 1;
}
}
my $signal_delay_points;
for ( my $ts = $signal_delay_samples[ 0 ]; $ts <= $signal_delay_samples[ -1 ]; $ts++ ) {
$signal_delay_points .= $ts / 1000.0 . " " . $signal_delay{ $ts } . "\n";
}
my $size_=@delays;
my $sum=0;
for ( my $i =0 ; $i < $size_; $i++ ) {
$sum=$sum+$delays[$i];
}
my $avg=$sum/$size_;
my @signal_delays = sort { $a <=> $b } values %signal_delay;
my $pp95s = $signal_delays[ 0.95 * scalar @signal_delays ];
printf STDERR qq{average per packet delay: %.0f ms\n}, $avg;
printf STDERR qq{95th percentile per-packet queueing delay: %.0f ms\n}, $pp95;
printf STDERR qq{95th percentile signal delay: %.0f ms\n}, $pp95s;
my $graph_max = $signal_delays[-1] * 1.2;
# make graph
open GNUPLOT, q{| gnuplot} or die;
my $lower_limit = min( 30, int $pp50 );
print GNUPLOT <<END;
set xlabel "time (s)"
set ylabel "delay (ms)
set logscale y
#set yrange [$lower_limit:$graph_max]
set ytics add ($lower_limit)
set grid xtics lc rgb "#bbbbbb" lw 1 lt 0
set key center outside top horizontal
set terminal svg size 1024,514 fixed enhanced font "Arial,12" solid mouse standalone name "Delay"
set output "/dev/stdout"
END
#printf GNUPLOT qq{plot [%f:%f] "-" using 1:2 title "Per-packet delay" with points lt ps 0.5 lc rgb "#00ff0000", %f with lines lw 3 lc rgb "#80ff0000" title "95th percentile per-packet delay (%.0f ms)", %f with lines lw 5 lc rgb "#8000ff00" title "average per-packet delay (%.0f ms)"\n}, $first_timestamp / 1000.0, $last_timestamp / 1000.0, $pp95, $pp95, $avg, $avg;
#printf GNUPLOT qq{plot [%f:%f] "-" using 1:2 title "Signal delay" with filledcurves above x1 lt 1 fs solid 0.01 noborder fc rgb "#000010", "-" using 1:2 title "Per-packet delay" with points lt 2 ps 0.5 lc rgb "#00ff0000", %f with lines lw 2 lc rgb "#800000ff" title "95th percentile signal delay (%.0f ms)", %f with lines lw 3 lc rgb "#80ff0000" title "95th percentile per-packet delay (%.0f ms)", %f with lines lw 5 lc rgb "#8000ff00" title "average per-packet delay (%.0f ms)"\n}, $first_timestamp/1000,$last_timestamp/1000 , $pp95s, $pp95s, $pp95, $pp95, $avg, $avg;
#printf GNUPLOT qq{plot [%f:%f] "-" using 1:2 title "Signal delay" with filledcurves above x1 lt 1 fs solid 0.01 noborder fc rgb "#000010", "-" using 1:2 title "Per-packet delay" with points lt 2 ps 1 lc rgb "#00ff0000", %f with lines lw 2 lc rgb "#800000ff" title "95th percentile signal delay (%.0f ms)", %f with lines lw 3 lc rgb "#80ff0000" title "95th percentile per-packet delay (%.0f ms)", %f with lines lw 5 lc rgb "#8000ff00" title "average per-packet delay (%.0f ms)"\n}, $first_timestamp / 1000.0, $last_timestamp / 1000.0, $pp95s, $pp95s, $pp95, $pp95, $avg, $avg;
#printf GNUPLOT qq{plot [%f:%f] "-" using 1:2 title "Signal delay" with filledcurves above x1 lt 1 fs solid 0.4 noborder fc rgb "#0000ff", "-" using 1:2 title "Per-packet delay" with points lt 3 ps 0.15 lc rgb "#00ff0000", %f with lines lw 2 lc rgb "#800000ff" title "95th percentile signal delay (%.0f ms)", %f with lines lw 3 lc rgb "#80ff0000" title "95th percentile per-packet delay (%.0f ms)", %f with lines lw 5 lc rgb "#8000ff00" title "average per-packet delay (%.0f ms)"\n}, $first_timestamp / 1000.0, $last_timestamp / 1000.0, $pp95s, $pp95s, $pp95, $pp95, $avg, $avg;
#print GNUPLOT $signal_delay_points;
#print GNUPLOT "e\n";
#print GNUPLOT $points;
close $delay_fh;
close $jitter_fh;
#close GNUPLOT or die qq{$!};
rl-module/mm-thr
-247
View File
@@ -1,247 +0,0 @@
#!/usr/bin/perl -w
use warnings;
use strict;
use POSIX;
sub usage
{
die qq{Usage: $0 MS_PER_BIN [filename]\n};
}
if ( scalar @ARGV < 1 or scalar @ARGV > 2 ) {
usage;
}
my $MS_PER_BIN = shift;
if ( $MS_PER_BIN !~ m{^\d+$} ) {
usage;
}
sub ms_to_bin {
return int( $_[0] / $MS_PER_BIN );
}
sub bin_to_seconds {
return sprintf q{%.3f}, $_[0] * $MS_PER_BIN / 1000.0;
}
my ( %capacity, %arrivals, %departures );
my $first_timestamp = undef;
my $last_timestamp = undef;
my $base_timestamp = undef;
my $capacity_sum;
my $arrival_sum;
my $departure_sum;
my @delays;
my %signal_delay;
LINE: while ( <> ) {
chomp;
if ( m{^# base timestamp: (\d+)} ) {
if ( defined $base_timestamp ) {
die "base timestamp multiply defined";
} else {
$base_timestamp = $1;
}
next LINE;
} elsif ( m{^#} ) {
next LINE;
}
# parse and validate line
my ( $timestamp, $event_type, $num_bytes, $delay ) = split /\s+/, $_;
if ( not defined $num_bytes ) {
die q{Format: timestamp event_type num_bytes [delay]};
}
if ( $timestamp !~ m{^\d+$} ) {
die qq{Invalid timestamp: $timestamp};
}
if ( $num_bytes !~ m{^\d+$} ) {
die qq{Invalid byte count: $num_bytes};
}
if ( not defined $base_timestamp ) {
die "logfile is missing base timestamp";
}
$timestamp -= $base_timestamp; # correct for startup time variation
if ( $timestamp > 0 ) {
# $timestamp -= 30000; # correct for startup time variation
if ( not defined $last_timestamp ) {
$last_timestamp = $first_timestamp = $timestamp;
}
$last_timestamp = max( $timestamp, $last_timestamp );
my $num_bits = $num_bytes * 8;
my $bin = ms_to_bin( $timestamp );
# process the event
if ( $event_type eq q{+} ) {
$arrivals{ $bin } += $num_bits;
$arrival_sum += $num_bits;
} elsif ( $event_type eq q{d} ) {
; #noothing for now! We have drop!
} elsif ( $event_type eq q{#} ) {
$capacity{ $bin } += $num_bits;
$capacity_sum += $num_bits;
} elsif ( $event_type eq q{-} ) {
if ( not defined $delay ) {
die q{Departure format: timestamp - num_bytes delay};
}
$departures{ $bin } += $num_bits;
if ( $delay < 0 ) {
die qq{Invalid delay: $delay};
}
if ( $timestamp - $delay < 0 ) {
die qq{Invalid timestamp and delay: ts=$timestamp, delay=$delay};
}
push @delays, $delay;
$departure_sum += $num_bits;
$signal_delay{ $timestamp - $delay } = min( $delay,
(defined $signal_delay{ $timestamp - $delay })
? $signal_delay{ $timestamp - $delay }
: POSIX::DBL_MAX );
} else {
die qq{Unknown event type: $event_type};
}
}
}
sub min {
my $minval = POSIX::DBL_MAX;
for ( @_ ) {
if ( $_ < $minval ) {
$minval = $_;
}
}
return $minval;
}
sub max {
my $maxval = - POSIX::DBL_MAX;
for ( @_ ) {
if ( $_ > $maxval ) {
$maxval = $_;
}
}
return $maxval;
}
if ( not defined $first_timestamp ) {
die q{Must have at least one event};
}
# calculate statistics
my $duration = ($last_timestamp - $first_timestamp) / 1000.0;
my $average_capacity = ($capacity_sum / $duration) / 1000000.0;
my $average_ingress = ($arrival_sum / $duration) / 1000000.0;
my $average_throughput = ($departure_sum / $duration) / 1000000.0;
if ( scalar @delays == 0 ) {
die q{Must have at least one departure event};
}
@delays = sort { $a <=> $b } @delays;
my $pp95 = $delays[ 0.95 * scalar @delays ];
my $pp99 = $delays[ 0.99 * scalar @delays ];
my $pp50 = $delays[ 0.50 * scalar @delays ];
# measure signal delay every millisecond
# = minimum time for a message created at time t to get to receiver
my @signal_delay_samples = sort { $a <=> $b } keys %signal_delay;
for ( my $ts = $signal_delay_samples[ -1 ]; $ts >= $signal_delay_samples[ 0 ]; $ts-- ) {
if ( not defined $signal_delay{ $ts } ) {
$signal_delay{ $ts } = $signal_delay{ $ts + 1 } + 1;
}
}
my @signal_delays = sort { $a <=> $b } values %signal_delay;
my $pp95s = $signal_delays[ 0.95 * scalar @signal_delays ];
my $size_=@delays;
my $sum=0;
for ( my $i =0 ; $i < $size_; $i++ ) {
$sum=$sum+$delays[$i];
}
my $avg=$sum/$size_;
printf STDERR qq{Average capacity: %.2f Mbits/s\n}, $average_capacity;
printf STDERR qq{Average throughput: %.2f Mbits/s (%.1f%% utilization)\n}, $average_throughput, 100.0 * $average_throughput / $average_capacity;
printf STDERR qq{95th percentile per-packet queueing delay: %.0f ms\n}, $pp95;
printf STDERR qq{Average per packet delay: %.0f ms\n}, $avg;
# make graph
my $earliest_bin = min( keys %arrivals, keys %capacity, keys %departures );
my $latest_bin = max( keys %arrivals, keys %capacity, keys %departures );
if ( $earliest_bin == $latest_bin ) {
die q{MS_PER_BIN is too large for length of trace};
}
my $current_buffer_occupancy = 0;
sub default {
return defined $_[ 0 ] ? $_[ 0 ] : 0;
}
open GNUPLOT, q{| gnuplot} or die;
print GNUPLOT <<END;
set xlabel "time (s)"
set ylabel "throughput (Mbits/s)"
#set yrange [*:20]
set grid ytics lc rgb "#bbbbbb" lw 1 lt 0
#set grid xtics lc rgb "#bbbbbb" lw 1 lt 0
set key center outside top horizontal
set style fill solid 0.2 noborder
set terminal svg size 1024,512 fixed font "Arial,12" solid mouse standalone name "Throughput"
set output "/dev/stdout"
END
#printf GNUPLOT qq{plot [%f:%f] "-" using 1:2 title "Capacity (mean %.2f Mbits/s)" with filledcurves above x1 lw 0.5, "-" using 1:3 with lines lc rgb "#0020a0" lw 4 title "Traffic ingress (mean %.2f Mbits/s)", "-" using 1:4 with lines lc rgb "#ff6040" lw 2 title "Traffic egress (mean %.2f Mbits/s)"\n}, $first_timestamp / 1000.0, $last_timestamp / 1000.0, $average_capacity, $average_ingress, $average_throughput;
#printf GNUPLOT qq{set xtics 0,0.5,120\nset ytics 0,1,20\n};
printf GNUPLOT qq{plot [%f:%f] "-" using 1:2 title "Capacity (mean %.2f Mbits/s)" with filledcurves above x1 lw 0.2, "-" using 1:3 with lines lc rgb "#0020a0" lw 1.25 title "Traffic ingress (mean %.2f Mbits/s)", "-" using 1:4 with lines lc rgb "#ff6040" lw 1.25 title "Traffic egress (mean %.2f Mbits/s)"\n}, $first_timestamp/1000, $last_timestamp/1000,$average_capacity, $average_ingress, $average_throughput;
#printf GNUPLOT qq{plot [%f:%f] "-" using 1:2 title "Capacity (mean %.2f Mbits/s)" with filledcurves above x1 lw 0.2, "-" using 1:3 with lines lc rgb "#0020a0" lw 1.25 title "Traffic ingress (mean %.2f Mbits/s)", "-" using 1:4 with lines lc rgb "#ff6040" lw 1.25 title "Traffic egress (mean %.2f Mbits/s)"\n}, $first_timestamp / 1000.0, $last_timestamp / 1000.0, $average_capacity, $average_ingress, $average_throughput;
my $output;
for ( my $bin = $earliest_bin; $bin <= $latest_bin; $bin++ ) {
my $t = bin_to_seconds( $bin );
my ( $cap, $arr, $dep ) = map { (default $_) / ($MS_PER_BIN / 1000.0) / 1000000.0 } ( $capacity{ $bin }, $arrivals{ $bin }, $departures{ $bin } );
$current_buffer_occupancy += default $arrivals{ $bin };
$current_buffer_occupancy -= default $departures{ $bin };
$output .= qq{$t $cap $arr $dep $current_buffer_occupancy\n};
}
print GNUPLOT $output;
print GNUPLOT qq{\ne\n};
print GNUPLOT $output;
print GNUPLOT qq{\ne\n};
print GNUPLOT $output;
close GNUPLOT or die qq{$!};
rl-module/mm-thr-file
-217
View File
@@ -1,217 +0,0 @@
#!/usr/bin/perl -w
use warnings;
use strict;
use POSIX;
sub usage
{
die qq{Usage: $0 MS_PER_BIN [filename]\n};
}
if ( scalar @ARGV < 1 or scalar @ARGV > 2 ) {
usage;
}
my $MS_PER_BIN = shift;
if ( $MS_PER_BIN !~ m{^\d+$} ) {
usage;
}
sub ms_to_bin {
return int( $_[0] / $MS_PER_BIN );
}
sub bin_to_seconds {
return sprintf q{%.3f}, $_[0] * $MS_PER_BIN / 1000.0;
}
my ( %capacity, %arrivals, %departures );
my $first_timestamp = undef;
my $last_timestamp = undef;
my $base_timestamp = undef;
my $capacity_sum;
my $arrival_sum;
my $departure_sum;
my @delays;
my %signal_delay;
my $filename = $ARGV[0];
open(my $thr_fh, '>', "$filename-thr.dat");
LINE: while ( <> ) {
chomp;
if ( m{^# base timestamp: (\d+)} ) {
if ( defined $base_timestamp ) {
die "base timestamp multiply defined";
} else {
$base_timestamp = $1;
}
next LINE;
} elsif ( m{^#} ) {
next LINE;
}
# parse and validate line
my ( $timestamp, $event_type, $num_bytes, $delay ) = split /\s+/, $_;
if ( not defined $num_bytes ) {
die q{Format: timestamp event_type num_bytes [delay]};
}
if ( $timestamp !~ m{^\d+$} ) {
die qq{Invalid timestamp: $timestamp};
}
if ( $num_bytes !~ m{^\d+$} ) {
die qq{Invalid byte count: $num_bytes};
}
if ( not defined $base_timestamp ) {
die "logfile is missing base timestamp";
}
$timestamp -= $base_timestamp; # correct for startup time variation
if ( $timestamp > 0 ) {
# $timestamp -= 30000; # correct for startup time variation
if ( not defined $last_timestamp ) {
$last_timestamp = $first_timestamp = $timestamp;
}
$last_timestamp = max( $timestamp, $last_timestamp );
my $num_bits = $num_bytes * 8;
my $bin = ms_to_bin( $timestamp );
# process the event
if ( $event_type eq q{+} ) {
$arrivals{ $bin } += $num_bits;
$arrival_sum += $num_bits;
} elsif ( $event_type eq q{d} ) {
; #noothing for now! We have drop!
} elsif ( $event_type eq q{#} ) {
$capacity{ $bin } += $num_bits;
$capacity_sum += $num_bits;
} elsif ( $event_type eq q{-} ) {
if ( not defined $delay ) {
die q{Departure format: timestamp - num_bytes delay};
}
$departures{ $bin } += $num_bits;
if ( $delay < 0 ) {
die qq{Invalid delay: $delay};
}
if ( $timestamp - $delay < 0 ) {
die qq{Invalid timestamp and delay: ts=$timestamp, delay=$delay};
}
push @delays, $delay;
$departure_sum += $num_bits;
$signal_delay{ $timestamp - $delay } = min( $delay,
(defined $signal_delay{ $timestamp - $delay })
? $signal_delay{ $timestamp - $delay }
: POSIX::DBL_MAX );
} else {
die qq{Unknown event type: $event_type};
}
}
}
sub min {
my $minval = POSIX::DBL_MAX;
for ( @_ ) {
if ( $_ < $minval ) {
$minval = $_;
}
}
return $minval;
}
sub max {
my $maxval = - POSIX::DBL_MAX;
for ( @_ ) {
if ( $_ > $maxval ) {
$maxval = $_;
}
}
return $maxval;
}
if ( not defined $first_timestamp ) {
die q{Must have at least one event};
}
# calculate statistics
my $duration = ($last_timestamp - $first_timestamp) / 1000.0;
my $average_capacity = ($capacity_sum / $duration) / 1000000.0;
my $average_ingress = ($arrival_sum / $duration) / 1000000.0;
my $average_throughput = ($departure_sum / $duration) / 1000000.0;
if ( scalar @delays == 0 ) {
die q{Must have at least one departure event};
}
@delays = sort { $a <=> $b } @delays;
my $pp95 = $delays[ 0.95 * scalar @delays ];
my $pp99 = $delays[ 0.99 * scalar @delays ];
my $pp50 = $delays[ 0.50 * scalar @delays ];
# measure signal delay every millisecond
# = minimum time for a message created at time t to get to receiver
my @signal_delay_samples = sort { $a <=> $b } keys %signal_delay;
for ( my $ts = $signal_delay_samples[ -1 ]; $ts >= $signal_delay_samples[ 0 ]; $ts-- ) {
if ( not defined $signal_delay{ $ts } ) {
$signal_delay{ $ts } = $signal_delay{ $ts + 1 } + 1;
}
}
my @signal_delays = sort { $a <=> $b } values %signal_delay;
my $pp95s = $signal_delays[ 0.95 * scalar @signal_delays ];
my $size_=@delays;
my $sum=0;
for ( my $i =0 ; $i < $size_; $i++ ) {
$sum=$sum+$delays[$i];
}
my $avg=$sum/$size_;
# make graph
my $earliest_bin = min( keys %arrivals, keys %capacity, keys %departures );
my $latest_bin = max( keys %arrivals, keys %capacity, keys %departures );
if ( $earliest_bin == $latest_bin ) {
die q{MS_PER_BIN is too large for length of trace};
}
my $current_buffer_occupancy = 0;
sub default {
return defined $_[ 0 ] ? $_[ 0 ] : 0;
}
for ( my $bin = $earliest_bin; $bin <= $latest_bin; $bin++ ) {
my $t = bin_to_seconds( $bin );
my ( $cap, $arr, $dep ) = map { (default $_) / ($MS_PER_BIN / 1000.0) / 1000000.0 } ( $capacity{ $bin }, $arrivals{ $bin }, $departures{ $bin } );
$current_buffer_occupancy += default $arrivals{ $bin };
$current_buffer_occupancy -= default $departures{ $bin };
print $thr_fh qq{$t $cap $arr $dep $current_buffer_occupancy\n};
}
close $thr_fh or die qq{$!};
rl-module/orca-server-mahimahi
BIN
View File
Binary file not shown.
rl-module/params.json
-120
View File
@@ -1,120 +0,0 @@
{
"TARGET_MARGIN": 1,
"device": "/cpu",
"PER": false,
"CDQ": true,
"single_actor_eval": false,
"LOSS_TYPE": "MSE",
"use_hard_target": false,
"hard_target": 500,
"MVWIN": 2,
"batch_size": 512,
"use_TCP": true,
"use_normalizer": false,
"h1_shape": 256,
"h2_shape": 256,
"recurrent": true,
"rec_dim": 10,
"noise_type": 3,
"noise_exp": 50000,
"stddev": 0.2,
"update_delay": 5,
"max_eps_steps": 64,
"max_epochs": 1000000,
"eval_frequency": 1000,
"state_dim": 7,
"input_dim": 15,
"action_dim": 1,
"delay_margin_coef": 1.25,
"remote": true,
"dequeue_length": 100,
"lr_a": 0.0001,
"lr_c": 0.001,
"tau": 0.001,
"gamma": 0.995,
"memsize": 2553600,
"tb_interval": 1,
"ckptdir": null,
"scheme": null,
"num_actors": 1,
"logdir": "train_dir",
"learner_ip": "127.0.0.1:33330",
"actor_ip": [
"127.0.0.1:33331",
"127.0.0.1:33332",
"127.0.0.1:33333",
"127.0.0.1:33334",
"127.0.0.1:33335",
"127.0.0.1:33336",
"127.0.0.1:33337",
"127.0.0.1:33338",
"127.0.0.1:33339",
"127.0.0.1:33340",
"127.0.0.1:33341",
"127.0.0.1:33342",
"127.0.0.1:33343",
"127.0.0.1:33344",
"127.0.0.1:33345",
"127.0.0.1:33346",
"127.0.0.1:33347",
"127.0.0.1:33348",
"127.0.0.1:33349",
"127.0.0.1:33350",
"127.0.0.1:33351",
"127.0.0.1:33352",
"127.0.0.1:33353",
"127.0.0.1:33354",
"127.0.0.1:33355",
"127.0.0.1:33356",
"127.0.0.1:33357",
"127.0.0.1:33358",
"127.0.0.1:33359",
"127.0.0.1:33360",
"127.0.0.1:33361",
"127.0.0.1:33362",
"127.0.0.1:33363",
"127.0.0.1:33364",
"127.0.0.1:33365",
"127.0.0.1:33366",
"127.0.0.1:33367",
"127.0.0.1:33368",
"127.0.0.1:33369",
"127.0.0.1:33370",
"127.0.0.1:33381",
"127.0.0.1:33382",
"127.0.0.1:33383",
"127.0.0.1:33384",
"127.0.0.1:33385",
"127.0.0.1:33386",
"127.0.0.1:33387",
"127.0.0.1:33388",
"127.0.0.1:33389",
"127.0.0.1:33380",
"127.0.0.1:33391",
"127.0.0.1:33392",
"127.0.0.1:33393",
"127.0.0.1:33394",
"127.0.0.1:33395",
"127.0.0.1:33396",
"127.0.0.1:33397",
"127.0.0.1:33398",
"127.0.0.1:33399",
"127.0.0.1:33400",
"127.0.0.1:33401",
"127.0.0.1:33402",
"127.0.0.1:33403",
"127.0.0.1:33404",
"127.0.0.1:33405",
"127.0.0.1:33406",
"127.0.0.1:33407",
"127.0.0.1:33408",
"127.0.0.1:33409",
"127.0.0.1:33410"
]
}
rl-module/train_dir/learner0/checkpoint
-2
View File
@@ -1,2 +0,0 @@
model_checkpoint_path: "model.ckpt-1283529"
all_model_checkpoint_paths: "model.ckpt-1283529"
rl-module/train_dir/learner0/model.ckpt-1283529.data-00000-of-00001
BIN
View File
Binary file not shown.
rl-module/train_dir/learner0/model.ckpt-1283529.index
BIN
View File
Binary file not shown.
rl-module/train_dir/learner0/model.ckpt-1283529.meta
BIN
View File
Binary file not shown.
rl-module/utils.py
-269
View File
@@ -1,269 +0,0 @@
'''
MIT License
Copyright (c) Chen-Yu Yen - Soheil Abbasloo 2020
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
'''
import numpy as np
import logging
import os
import json
import random
def configure_logging(path_to_log_directory):
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
if not os.path.exists(path_to_log_directory):
os.makedirs(path_to_log_directory)
handler = logging.FileHandler(filename=os.path.join(path_to_log_directory, "result.log"))
handler.setLevel(logging.INFO)
formatter = logging.Formatter('%(asctime)s - %(message)s')
handler.setFormatter(formatter)
logger.addHandler(handler)
return logger
logger = configure_logging("./rl_logging")
class G_Noise(object):
def __init__(self, mu , sigma, explore=40000,theta=0.1,mu2=0.0,mode="exp",eps=1.0,step=0.3):
self.epsilon = eps
self.mu = mu
self.explore = explore
self.sigma = sigma
self.mu2 = mu2
self.theta = theta
self.noise = 0
self.cnt = 0
self.step = step
self.mode = mode
def show(self):
return self.noise
def __call__(self,point):
if self.explore!=None:
if self.mode=="exp":
if self.epsilon <= 0:
self.noise=np.zeros_like(self.mu)
else:
self.epsilon -= 1/self.explore
noise = self.epsilon * (self.sigma * np.random.randn(1))
self.noise = noise
else:
self.cnt += 1
if self.cnt >=self.explore:
self.sigma -= self.step*self.sigma
self.cnt = 0
if self.sigma <= 0.1:
self.segma = 0.1
noise = self.sigma*np.random.randn(1)
self.noise = noise
else:
noise = (self.sigma * np.random.randn(1))
self.noise = noise
return self.noise
def reset(self):
pass
class OU_Noise:
def __init__(self, mu, sigma, theta=.15, dt=0.01, x0=None,exp=None):
self.theta = theta
self.mu = mu
self.sigma = sigma
self.dt = dt
self.x0 = x0
self.eps = 1.0
self.exp = exp
self.reset()
def show(self):
return self.x_prev
def __call__(self,point):
if self.exp!=None:
self.dt -= 1/self.exp
if self.dt<=0.01:
self.dt=0.01
self.sigma -= 1/self.exp
if self.sigma<=0.3:
self.sigma=0.3
x = self.x_prev + self.theta * (self.mu - self.x_prev) * self.dt + self.sigma * np.sqrt(self.dt) * np.random.normal(size=self.mu.shape)
self.x_prev = x
return x
def reset(self):
self.x_prev = self.x0 if self.x0 is not None else np.zeros_like(self.mu)
class ReplayBuffer:
def __init__(self, size, s_dim, a_dim, batch_size):
self.size = size
self.s0_buf = np.zeros((size, s_dim), dtype=np.float32)
self.a_buf = np.zeros((size, a_dim), dtype=np.float32)
self.reward_buf = np.zeros((size,1), dtype=np.float32)
self.s1_buf = np.zeros((size, s_dim), dtype=np.float32)
self.terminal_buf = np.zeros((size,1), dtype=np.float32)
self.ptr = 0
self.full = False
self.batch_size = batch_size
self.length_buf = 0
def peek_buffer(self):
return [self.s0_buf, self.a_buf, self.reward_buf, self.s1_buf, self.terminal_buf]
def store(self, s0, a, r, s1, terminal):
self.s0_buf[self.ptr] = s0
self.a_buf[self.ptr] = a
self.reward_buf[self.ptr] = r
self.s1_buf[self.ptr] = s1
self.terminal_buf[self.ptr] = terminal
self.ptr += 1
# Buffer Full
if self.ptr == self.size:
self.ptr = 0
self.full = True
self.length_buf = self.size
if self.full == False:
self.length_buf = self.ptr
def store_many(self, s0, a, r, s1, terminal, length):
if self.ptr + length >= self.size:
firstpart = self.size-self.ptr
secondpart = length - firstpart
self.s0_buf[self.ptr:] = s0[:firstpart]
self.s0_buf[:secondpart] = s0[firstpart:]
self.a_buf[self.ptr:] = a[:firstpart]
self.a_buf[:secondpart] = a[firstpart:]
self.reward_buf[self.ptr:] = r[:firstpart]
self.reward_buf[:secondpart] = r[firstpart:]
self.s1_buf[self.ptr:] = s1[:firstpart]
self.s1_buf[:secondpart] = s1[firstpart:]
self.terminal_buf[self.ptr:] = terminal[:firstpart]
self.terminal_buf[:secondpart] = terminal[firstpart:]
self.ptr= secondpart
self.full = True
else:
self.s0_buf[self.ptr: self.ptr+length] = s0
self.a_buf[self.ptr: self.ptr+length] = a
self.reward_buf[self.ptr: self.ptr+length] = r
self.s1_buf[self.ptr: self.ptr+length] = s1
self.terminal_buf[self.ptr: self.ptr+length] = terminal
self.ptr += length
if self.full:
self.length_buf = self.size
else:
self.length_buf = self.ptr
def sample(self):
import random
if self.batch_size < self.length_buf :
start_index = int(self.length_buf*random.random())
if start_index + self.batch_size >= self.length_buf:
arr1 = list(range(start_index, self.length_buf))
arr2 = list(range(0, self.batch_size- len(arr1)))
index = arr1 + arr2
else:
index = list(range(start_index, start_index+self.batch_size))
else:
index = list(range(0, self.length_buf))
s0 = self.s0_buf[index]
a = self.a_buf[index]
r = self.reward_buf[index]
s1 = self.s1_buf[index]
terminal = self.terminal_buf[index]
return [s0, a, r, s1, terminal]
def _encode_sample(self, idxes):
s0 = self.s0_buf[idxes]
a = self.a_buf[idxes]
r = self.reward_buf[idxes]
s1 = self.s1_buf[idxes]
terminal = self.terminal_buf[idxes]
return [s0, a, r, s1, terminal]
class Prioritized_ReplayBuffer(ReplayBuffer):
def __init__(self, size, s_dim, a_dim, batch_size, alpha=1):
pass
def store(self, *args, **kwargs):
pass
def _sample_proportional(self, batch_size):
pass
def sample(self, beta=0.5):
pass
def update_priorities(self, idxes, priorities):
pass
class Params():
def __init__(self, json_path):
self.update(json_path)
def save(self, json_path):
with open(json_path, 'w') as f:
json.dump(self.__dict__, f, indent=4)
def update(self, json_path):
with open(json_path) as f:
params = json.load(f)
self.__dict__.update(params)
@property
def dict(self):
return self.__dict__
setup.sh
-10
View File
@@ -1,10 +0,0 @@
sudo sysctl -q net.ipv4.tcp_wmem="4096 32768 4194304" #Doubling the default value from 16384 to 32768
sudo sysctl -w -q net.ipv4.tcp_low_latency=1
sudo sysctl -w -q net.ipv4.tcp_autocorking=0
sudo sysctl -w -q net.ipv4.tcp_no_metrics_save=1
sudo sysctl -w -q net.ipv4.ip_forward=1
#Mahimahi Issue: it couldn't make enough interfaces
#Solution: increase max of inotify
sudo sysctl -w -q fs.inotify.max_user_watches=524288
sudo sysctl -w -q fs.inotify.max_user_instances=524288
src/client.c
-267
View File
@@ -1,267 +0,0 @@
//============================================================================
// Author : Soheil Abbasloo
// Version : 1.0
//============================================================================
/*
MIT License
Copyright (c) Soheil Abbasloo 2020 (ab.soheil@gmail.com)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <pthread.h>
#include <errno.h>
#include <netinet/tcp.h>
#include <time.h>
#include <inttypes.h>
#include "common.h"
#define DBG 0
#define HASH_RANGE 5
#define TCP_BBR_EN_MAXDEL 33
#define TCP_BBR_EN_PRBRTT 34
#define TCP_BBR_TRGTDEL_MS 35
#define TCP_BBR_MINRTTWIN_SEC 36
#define TCP_BBR_PRBERTTMDE_MS 37
#define TCP_BBR_BWAUTO 38
#define TCP_BBR_BWVAL 39
#define TCP_BBR_CWNDRVGAIN 40
#define TCP_BBR_DEBUG 41
struct timeval tv_start; //Start time (after three way handshake)
struct timeval tv_end; //End time
//Print usage information
void usage();
int check_input(int data_size, int deadline, int is_agnostic)
{
return 1; // :D
}
uint64_t raw_timestamp( void )
{
struct timespec ts;
clock_gettime( CLOCK_REALTIME, &ts );
uint64_t us = ts.tv_nsec / 1000;
us += (uint64_t)ts.tv_sec * 1000000;
return us;
}
uint64_t initial_timestamp( void )
{
static uint64_t initial_value = raw_timestamp();
return initial_value;
}
uint64_t timestamp( void )
{
return raw_timestamp() - initial_timestamp();
}
int main(int argc, char **argv)
{
uint64_t start_timestamp=10;
FILE *filep;
char line[4096];
// usleep(500000);
int connected=0;
int i;
start_timestamp=start_timestamp*1000;
if(argc!=4)
{
usage();
for(int i=0;i<argc;i++)
{
DBGERROR("%s\n",argv[i]);
}
DBGERROR("ARGC:%d\n",argc);
return 0;
}
char *savePtr;
char* server=(char*)malloc(16*sizeof(char));; //IP address (e.g. 192.168.101.101)
char query[100]; //query=data_size+' '+deadline+' '+agnostic
int port; //TCP port number
int data_size; //Request data size (KB) We allocate 20bits for this in nfmark [0, 4GB]
int flowid;
char *dst_ip;
int sockfd; //Socket
struct sockaddr_in servaddr; //Server address (IP:Port)
struct sockaddr_in clientaddr; //client address (IP:Port)
int len; //Read length
char buf[BUFSIZ]; //Receive buffer
unsigned long fct; //Flow Completion Time
//Initialize ‘server’
memset(server,'\0',16);
//Initialize 'query'
memset(query,'\0',150);
//Get server address
server=argv[1];
//Get deadline information: char* to int
flowid=atoi(argv[2]);
//Get data_size: char* to int
//data_size=atoi(argv[3]);
//Following line won't impact the code in this version: Server always sends data (no size request will be handled in this project ;) )
data_size=2700;
//Get is_agnostic: char* to int
//Following line won't impact the code in this version: Server always sends data (We don't need interface bounding in this project ;) )
//dst_ip=argv[4];
// dst_ip="10.10.10.10";
//Get TCP port: char* to int
port=atoi(argv[3]);
char time_s [50];
char time_us[37];
int cx;
uint64_t t=start_timestamp;
// printf("%"PRIu64, t);
cx = snprintf ( time_s, 49, "%" PRIu64, t);
DBGPRINT(DBG,2,"time_s:%" PRIu64"\n",t);
//If the inputs are legal
if(check_input(data_size,0,0)==0)
{
DBGPRINT(0,0,"Illegal input!\n");
return 0;
}
else
{
strcat(query,argv[1]);
strcat(query," ");
strcat(query,argv[2]);
strcat(query," ");
// strcat(query,argv[3]);
strcat(query,"2700");
strcat(query," ");
strcat(query,"10.10.10.10");
// strcat(query,argv[4]);
strcat(query," ");
strcat(query,time_s);
strcat(query," ");
// strcat(query,argv[3]);
strcat(query,"2700");
}
//Init sockaddr_in
memset(&servaddr,0,sizeof(servaddr));
servaddr.sin_family=AF_INET;
//IP address
servaddr.sin_addr.s_addr=inet_addr(server);
//Port number
servaddr.sin_port=htons(port);
//Init sockaddr_in
// memset(&clientaddr,0,sizeof(clientaddr));
// clientaddr.sin_family=AF_INET;
//IP address
// clientaddr.sin_addr.s_addr=inet_addr(dst_ip);
//Port number
//Init socket
if((sockfd=socket(PF_INET,SOCK_STREAM,0))<0)
{
DBGMARK(0,0,"socket error:%s\n",strerror(errno));
return 0;
}
int reuse = 1;
// gettimeofday(&tv_start,NULL);
if (setsockopt(sockfd, IPPROTO_TCP, TCP_NODELAY, &reuse, sizeof(reuse)) < 0)
{
DBGMARK(0,0,"ERROR: set TCP_NODELAY option %s\n",strerror(errno));
close(sockfd);
return 0;
}
DBGPRINT(DBG,5,"%s ---\n",inet_ntoa(servaddr.sin_addr));
//bind it to correct interface
//We don't need this for this project! :D
/* if(bind(sockfd,(struct sockaddr *)&clientaddr,sizeof(struct sockaddr))<0)
{
DBGMARK(0,0,"bind error add:%s : %s",dst_ip,strerror(errno));
return 0;
}
*/
//Establish connection
for(i=0;i<120;i++)
{
if(connect(sockfd,(struct sockaddr *)&servaddr,sizeof(struct sockaddr))<0)
{
DBGPRINT(0,0,"Trying to connect to %s\n",server);
usleep(500000);
}
else
{
connected=1;
DBGPRINT(0,0,"Connected!\n");
break;
}
}
if(!connected)
{
DBGPRINT(0,0,"No success after 120 tries :(\n");
close(sockfd);
return 0;
}
DBGPRINT(DBG,5,"%s ---\n",inet_ntoa(servaddr.sin_addr));
//Send request
len=strlen(query);
while(len>0)
{
len-=send(sockfd,query,strlen(query),0);
}
//DBGPRINT(0,0,"Request is sent to server ...\n");
DBGMARK(DBG,1,"after sending request\n");
//Receive data
while(1)
{
len=recv(sockfd,buf,BUFSIZ,0);
if(len<=0)
{
//DBGPRINT(DBG,2,"len:%d\n",len);
break;
}
}
//Get end time after receiving all of the data
//Close connection
close(sockfd);
DBGMARK(DBG,1,"after receiving data\n\n");
return 0;
}
void usage()
{
DBGWARN("./client [server IP address] [flowid] [server port] \n");
}
src/common.h
-184
View File
@@ -1,184 +0,0 @@
//============================================================================
// Author : Soheil Abbasloo
// Version : 1.0
//============================================================================
/*
MIT License
Copyright (c) Soheil Abbasloo 2020 (ab.soheil@gmail.com)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
//#####################################################################################################
#ifndef COMMON_H_
#define COMMON_H_
//#####################################################################################################
//=========================================================================
#include <stdio.h>
#include <string.h>
//#include "Debug.h"
//#include "MemTrack.h"
//=========================================================================
//=========================================================================
//General Result Codes
enum ERes
{
ERES_SUCCESS = 0,
ERES_WARNING = -1,
ERES_ERROR = -2,
ERES_ERR_INVALID_PARAMETER = -3,
ERES_ERR_OPERATION_FAILED = -4,
ERES_ERR_INVALID_POINTER = -5,
ERES_ERR_NO_MEM = -6,
ERES_ERR_INVALID_OPERATION = -7,
ERES_ERR_NOT_IMPLEMENTED = -8,
};
//=========================================================================
//#define ERROR -1
/*
//ANSI Terminal Colors
#define COL_DGRY ""
#define COL_DBLU ""
#define COL_DGRN ""
#define COL_DCYN ""
#define COL_DRED ""
#define COL_DMAG ""
#define COL_DYEL ""
#define COL_DWHT ""
#define COL_LGRY ""
#define COL_LBLU ""
#define COL_LGRN ""
#define COL_LCYN ""
#define COL_LRED ""
#define COL_LMAG ""
#define COL_LYEL ""
#define COL_LWHT ""
//#define COL_LBLK ""
#define HIDCRSR "[?25l"
#define SHWCRSR "[?25h"
#define CLSSTR ""
#define ESCS ""
#define ESCC ''
#define ESCNUM 0x1B
#define CC_BS '\x08'
*/
//ANSI Terminal Colors
#define COL_DGRY ""
#define COL_DBLU ""
#define COL_DGRN ""
#define COL_DCYN ""
#define COL_DRED ""
#define COL_DMAG ""
#define COL_DYEL ""
#define COL_DWHT ""
#define COL_LGRY ""
#define COL_LBLU ""
#define COL_LGRN ""
#define COL_LCYN ""
#define COL_LRED ""
#define COL_LMAG ""
#define COL_LYEL ""
#define COL_LWHT ""
//#define COL_LBLK ""
#define HIDCRSR ""
#define SHWCRSR ""
#define CLSSTR ""
#define ESCS ""
#define ESCC ''
#define ESCNUM 0x1B
#define CC_BS '\x08'
#define MOVCRSR(_x,_y) "["#_y";"#_x"H"
//=========================================================================
// GCC Attributes
//=========================================================================
//for Functions
#define __GFA_INLINE __attribute__((always_inline)) //Force in-line
#define __GFA_NOINLINE __attribute__((noinline)) //Force no in-line
#define __GFA_FLATTEN __attribute__((flatten)) //in-line all calls from this function(if possible)
#define __GFA_ERROR(_msg_) __attribute__((error(_msg_))) //Generate Error
#define __GFA_WARN(_msg_) __attribute__((warning(_msg_))) //Generate Warning
#define __GFA_UNUSED __attribute__((unused)) //do not generate warning if not used
#define __GFA_USED __attribute__((used)) //mark as used
//for Types
#define __GTA_ALIGNE_BYTE __attribute__((aligned(8))) //Byte alignment
#define __GTA_ALIGNE __attribute__((aligned)) //Use Optimum alignment
#define __GTA_PACKED __attribute__((packed)) //Use minimum memory(for enums,union,...)
#define __GTA_UNUSED __attribute__((unused)) //do not generate warning if not used
//for Variables
#define __GVA_ALIGNE_BYTE __attribute__((aligned(8))) //Byte alignment
#define __GVA_ALIGNE __attribute__((aligned)) //Use Optimum alignment
#define __GVA_PACKED __attribute__((packed)) //Use minimum memory(for enums,union,...)
#define __GVA_UNUSED __attribute__((unused)) //do not generate warning if not used
#define __GVA_USED __attribute__((used)) //mark as used
//=========================================================================
//=========================================================================
#define VERSION(_major_, _minor_) ( ((_major_ & 0xFF)<<8) | (_minor_ & 0xFF) )
#define MAJOR_VERSION(_version_) ((_version_ >> 8) & 0xFF)
#define MINOR_VERSION(_version_) (_version_ & 0xFF)
#define CHECK_VAR_RANGE_B(_var, _low, _high) (((_var) >= (_low))&&((_var) <= (_high)))
#define CHECK_VAR_RANGE_NB(_var, _low, _high) (((_var) > (_low))&&((_var) < (_high)))
#define CHECK_VAR_RANGE_NOT_B(_var, _low, _high) (((_var) <= (_low))||((_var) >= (_high)))
#define CHECK_VAR_RANGE_NOT_NB(_var, _low, _high) (((_var) < (_low))||((_var) > (_high)))
#define CHECK_POINTER(_pointer) ((_pointer)?1:0)
#define SUCCESS(_res) ( (_res==ERES_SUCCESS)?true:false)
#define ERROR(_res) ( (_res!=ERES_SUCCESS)?true:false)
#ifdef PALCON_H_
#define PUTCH(ch) CConsole::PutChar(ch)
#define PRINT(fmt, arg...) CConsole::Print(fmt, ## arg )
#define PRINTXY(_x,_y, fmt, arg...) do{CConsole::Print("[%d;%dH",_y,_x); CConsole::Print(fmt, ## arg );}while(0)
#define PRINTCOL(col, fmt, arg...) CConsole::Print( col fmt COL_DWHT, ## arg )
#define DBGPRINTPRTY(curlvl, showlvl, fmt, arg...) do{if(curlvl>=showlvl)CConsole::Print(COL_LWHT "\r[%s] " COL_DWHT fmt COL_DWHT, __PRETTY_FUNCTION__ , ## arg );}while(0)
#define DBGPRINT(curlvl, showlvl, fmt, arg...) do{if(curlvl>=showlvl)CConsole::Print(COL_LWHT "\r[%s] " COL_DWHT fmt COL_DWHT, __FUNCTION__ , ## arg );}while(0)
#define DBGMARK(curlvl, showlvl, fmt, arg...) do{if(curlvl>=showlvl)CConsole::Print(COL_LBLU "\r[%s-%s-%d] " COL_DBLU fmt COL_DWHT, __FILE_NAME__, __FUNCTION__, __LINE__ , ## arg );}while(0)
#define DBGERROR(fmt, arg...) CConsole::Print(COL_LRED "\r[%s] " COL_DRED fmt COL_DWHT, __FUNCTION__ , ## arg )
#define DBGWARN(fmt, arg...) CConsole::Print(COL_LYEL "\r[%s] " COL_DYEL fmt COL_DWHT, __FUNCTION__ , ## arg )
#define DBGINFO(fmt, arg...) CConsole::Print(COL_LBLU "\r[%s] " COL_DGRN fmt COL_DWHT, __FUNCTION__ , ## arg )
#else
#define PUTCH(ch) do{fputc(ch, stdout);}while(0)
#define PRINT(fmt, arg...) do{fprintf(stdout, fmt, ## arg );fflush(stdout);}while(0)
#define PRINTXY(_x,_y,fmt,arg...) do{fprintf(stdout, "[%d;%dH",_y,_x); fprintf(stdout, fmt, ## arg );fflush(stdout);}while(0)
#define PRINTCOL(col, fmt, arg...) do{fprintf(stdout, col fmt COL_DWHT, ## arg );fflush(stdout);}while(0)
#define DBGPRINTPRTY(curlvl, showlvl,fmt, arg...) do{if(curlvl>=showlvl){fprintf(stdout, COL_LWHT "\r[%s] " COL_DWHT fmt COL_DWHT, __PRETTY_FUNCTION__ , ## arg );fflush(stdout);}}while(0)
#define DBGPRINT(curlvl, showlvl, fmt, arg...) do{if(curlvl>=showlvl){fprintf(stdout, COL_LWHT "\r[%s] " COL_DWHT fmt COL_DWHT, __FUNCTION__ , ## arg );fflush(stdout);}}while(0)
#define DBGMARK(curlvl, showlvl, fmt, arg...) do{if(curlvl>=showlvl){fprintf(stdout, COL_LBLU "\r[%s-%s-%d] " COL_DBLU fmt COL_DWHT, __FILE_NAME__, __FUNCTION__, __LINE__ , ## arg );fflush(stdout);}}while(0)
#define DBGERROR(fmt, arg...) do{fprintf(stdout, COL_LRED "\r[%s] " COL_DRED fmt COL_DWHT, __FUNCTION__ , ## arg );fflush(stdout);}while(0)
#define DBGWARN(fmt, arg...) do{fprintf(stdout, COL_LYEL "\r[%s] " COL_DYEL fmt COL_DWHT, __FUNCTION__ , ## arg );fflush(stdout);}while(0)
#define DBGINFO(fmt, arg...) do{fprintf(stdout, COL_LGRN "\r[%s] " COL_DGRN fmt COL_DWHT, __FUNCTION__ , ## arg );fflush(stdout);}while(0)
#endif
//=========================================================================
#define __FILE_NAME__ (strrchr(__FILE__,'/')?strrchr(__FILE__,'/')+1:__FILE__)
//=========================================================================
//#####################################################################################################
#endif /* COMMON_H_ */
src/define.h
-246
View File
@@ -1,246 +0,0 @@
//============================================================================
// Author : Soheil Abbasloo
// Version : 1.0
//============================================================================
/*
MIT License
Copyright (c) Soheil Abbasloo 2020 (ab.soheil@gmail.com)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#include "flow.h"
#include <pthread.h>
#include <sched.h>
#include <sys/types.h> // needed for socket(), uint8_t, uint16_t, uint32_t
pthread_mutex_t lockit;
#include <unistd.h>
#include <math.h>
#include <time.h>
#define __STDC_FORMAT_MACROS
#include <inttypes.h>
//Shared Memory ==> Communication with RL-Module -----*
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
//-------------------------*
#define OK_SIGNAL 99999
#define TARGET_MARGIN 100 //100% ==> Full Target!
#define TARGET_CHANGE_TIME 8 //Unit: minutes (Each TARGET_CHANGE_TIME minutes chagne the target)
#define TARGET_CHANGE_STEP 25
int shmid;
key_t key=123456;
char *shared_memory;
int shmid_rl;
key_t key_rl=12345;
char *shared_memory_rl;
int shmem_size=2048; //Shared Memory size: 2KBytes
int key1=0;
int key2=0;
//----------------------------------------------------*
typedef unsigned int u32;
typedef int s32;
typedef uint64_t u64;
typedef int64_t i64;
typedef uint8_t u8;
bool send_traffic=true;
u32 step_it=0;
u32 duration_steps=0;
unsigned int duration=0; //If not zero, it would be the total duration for sending out the traffic.
struct timeval tv_start,tv_start2; //Start time (after three way handshake)
struct timeval tv_stamped,tv_stamped2;
uint64_t start_of_client;
struct timeval tv_end,tv_end2; //End time
bool done=false;
bool check=false;
uint64_t setup_time; //Flow Completion Time
int actor_id;
char *scheme;
char *ssh;
char *ssh_cmd;
char *downlink;
char *uplink;
char *log_file;
char *path;
char *congestion;
int it=0;
bool got_message=0;
int codel=0;
int first_time=0;
int min_thr=2;
int qsize=100; //pkts
int flow_index=0;
int target_ratio=150;
u32 target=50; //50ms
u32 report_period=5;//5s
double mm_loss_rate=0;
#define FLOW_NUM 1
int sock[FLOW_NUM];
int sock_for_cnt[FLOW_NUM];
struct tcp_orca_info {
u32 min_rtt; /* min-filtered RTT in uSec */
u32 avg_urtt; /* averaged RTT in uSec from the previous info request till now*/
u32 cnt; /* number of RTT samples uSed for averaging */
unsigned long thr; /*Bytes per second*/
u32 thr_cnt;
u32 cwnd;
u32 pacing_rate;
u32 lost_bytes;
u32 srtt_us; /* smoothed round trip time << 3 in usecs */
u32 snd_ssthresh; /* Slow start size threshold*/
u32 packets_out; /* Packets which are "in flight"*/
u32 retrans_out; /* Retransmitted packets out*/
u32 max_packets_out; /* max packets_out in last window */
u32 mss;
void init()
{
min_rtt=0;
avg_urtt=0;
cnt=0;
thr=0;
thr_cnt=0;
cwnd=0;
pacing_rate=0;
lost_bytes=0;
srtt_us=0;
snd_ssthresh=0;
retrans_out=0;
max_packets_out=0;
mss=0;
}
tcp_orca_info& operator =(const tcp_orca_info& a){
this->min_rtt=a.min_rtt;
this->avg_urtt=a.avg_urtt;
this->cnt=a.cnt;
this->thr=a.thr;
this->thr_cnt=a.thr_cnt;
this->cwnd=a.cwnd;
this->pacing_rate=a.pacing_rate;
this->lost_bytes=a.lost_bytes;
this->snd_ssthresh=a.snd_ssthresh;
this->packets_out=a.packets_out;
this->retrans_out=a.retrans_out;
this->max_packets_out=a.max_packets_out;
this->mss=a.mss;
}
}orca_info;
struct sTrace
{
double time;
double bw;
double minRtt;
};
struct sInfo
{
sTrace *trace;
int sock;
int num_lines;
};
int delay_ms;
int client_port;
sTrace *trace;
#define DBGSERVER 0
#define TARGET_RATIO_MIN 100
#define TARGET_RATIO_MAX 1000
#define TCP_CWND_CLAMP 42
#define TCP_CWND 43
#define TCP_ORCA_ENABLE 44
#define TCP_CWND_CAP 45
#define TCP_ORCA_INFO 46 /* Get Congestion Control (optional) orca info */
#define TCP_CWND_MIN 47
uint64_t raw_timestamp( void )
{
struct timespec ts;
clock_gettime( CLOCK_REALTIME, &ts );
uint64_t us = ts.tv_nsec / 1000;
us += (uint64_t)ts.tv_sec * 1000000;
return us;
}
uint64_t timestamp_begin(bool set)
{
static uint64_t start;
if(set)
start = raw_timestamp();
return start;
}
uint64_t timestamp_end( void )
{
return raw_timestamp() - timestamp_begin(0);
}
uint64_t initial_timestamp( void )
{
static uint64_t initial_value = raw_timestamp();
return initial_value;
}
uint64_t timestamp( void )
{
return raw_timestamp() - initial_timestamp();
}
//Start server
void start_server(int flow_num, int client_port);
//thread functions
void* DataThread(void*);
void* CntThread(void*);
void* TimerThread(void*);
void* MonitorThread(void*);
//Print usage information
void usage();
int get_orca_info(int sk, struct tcp_orca_info *info)
{
int tcp_info_length = sizeof(*info);
return getsockopt( sk, SOL_TCP, TCP_ORCA_INFO, (void *)info, (socklen_t *)&tcp_info_length );
};
void handler(int sig) {
void *array[10];
size_t size;
DBGMARK(DBGSERVER,2,"=============================================================== Start\n");
size = backtrace(array, 20);
fprintf(stderr, "We got signal %d:\n", sig);
DBGMARK(DBGSERVER,2,"=============================================================== End\n");
shmdt(shared_memory);
shmctl(shmid, IPC_RMID, NULL);
shmdt(shared_memory_rl);
shmctl(shmid_rl, IPC_RMID, NULL);
exit(1);
}
src/flow.cc
-99
View File
@@ -1,99 +0,0 @@
//============================================================================
// Author : Soheil Abbasloo
// Version : 1.0
//============================================================================
/*
MIT License
Copyright (c) Soheil Abbasloo 2020 (ab.soheil@gmail.com)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#include "flow.h"
sFlowinfo& sFlowinfo::operator=(const sFlowinfo& other)
{
if(this != &other)
{
Copy(other);
}
return *this;
}
void sFlowinfo::Init()
{
this->dst_pod=0;
this->dst_edg=0;
this->dst_index=0;
this->src_pod=0;
this->src_edg=0;
this->src_index=0;
this->flowid=0;
this->sock=0;
this->size=0;
this->rem_size=0;
}
void sFlowinfo::Copy(sFlowinfo info)
{
this->dst_pod=info.dst_pod;
this->dst_edg=info.dst_edg;
this->dst_index=info.dst_index;
this->src_pod=info.src_pod;
this->src_edg=info.src_edg;
this->src_index=info.src_index;
this->flowid=info.flowid;
this->sock=info.sock;
this->size=info.size;
this->rem_size=info.rem_size;
}
cFlow::cFlow()
{
flowinfo.Init();
state =eInit;
ctr_sock=0;
qplus_enable=false;
startListening=false;
fin_msg_sent= false;
done = false;
}
cFlow::cFlow(sFlowinfo info)
{
flowinfo=info;
state =eInit;
ctr_sock=0;
qplus_enable=false;
fin_msg_sent= false;
startListening=false;
done = false;
}
src/flow.h
-100
View File
@@ -1,100 +0,0 @@
//============================================================================
// Author : Soheil Abbasloo
// Version : 1.0
//============================================================================
/*
MIT License
Copyright (c) Soheil Abbasloo 2020 (ab.soheil@gmail.com)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#ifndef FLOW_H_
#define FLOW_H_
#include "common.h"
#include "util.h"
using namespace std;
struct sFlowinfo
{
int sock;
int flowid;
int src_pod;
int src_edg;
int src_index;
int dst_pod;
int dst_edg;
int dst_index;
uint64_t size;
uint64_t rem_size;
sFlowinfo& operator=(const sFlowinfo& other);
void Copy(sFlowinfo);
void Init();
};
enum EPrt_State
{
eInit=0, //Check whether we need a Request
eRQ, //Waiting for the response
eSTOP, //Stop signal is received
eGO //Go signal is received
};
/**
* Server-Side Flows!
* Client has sent its request including data-size, {src,dst}
*/
class cFlow
{
public:
cFlow();
cFlow(sFlowinfo);
void SetFlow(sFlowinfo info){flowinfo=info;};
int SendData();
void SetCtr(sockaddr_in addr,int ctr_sock){ctr_addr=addr;this->ctr_sock=ctr_sock;};
void SetSrcDst(sockaddr_in src,sockaddr_in dst){src_addr=src;dst_addr=dst;};
public:
bool qplus_enable;
sockaddr_in ctr_addr;
sockaddr_in src_addr;
sockaddr_in dst_addr;
int ctr_sock;
pthread_mutex_t lock;
sFlowinfo flowinfo;
EPrt_State state;
bool done;
bool fin_msg_sent;
bool startListening;
};
#endif /* FLOW_H_ */
src/orca-server-mahimahi.cc
-688
View File
@@ -1,688 +0,0 @@
//============================================================================
// Author : Soheil Abbasloo
// Version : 1.0
//============================================================================
/*
MIT License
Copyright (c) Soheil Abbasloo 2020 (ab.soheil@gmail.com)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#include <cstdlib>
#include <sys/select.h>
#include "define.h"
//#define CHANGE_TARGET 1
#define MAX_CWND 10000
#define MIN_CWND 4
int main(int argc, char **argv)
{
DBGPRINT(DBGSERVER,4,"Main\n");
if(argc!=14)
{
DBGERROR("argc:%d\n",argc);
for(int i=0;i<argc;i++)
DBGERROR("argv[%d]:%s\n",i,argv[i]);
usage();
return 0;
}
srand(raw_timestamp());
signal(SIGSEGV, handler); // install our handler
signal(SIGTERM, handler); // install our handler
signal(SIGABRT, handler); // install our handler
signal(SIGFPE, handler); // install our handler
signal(SIGKILL,handler); // install our handler
int flow_num;
flow_num=FLOW_NUM;
client_port=atoi(argv[1]);
path=argv[2];
target=50;
target_ratio=1;
report_period=atoi(argv[3]);
first_time=atoi(argv[4]);
scheme=argv[5];
actor_id=atoi(argv[6]);
downlink=argv[7];
uplink=argv[8];
delay_ms=atoi(argv[9]);
log_file=argv[10];
duration=atoi(argv[11]);
qsize=atoi(argv[12]);
duration_steps=atoi(argv[13]);
start_server(flow_num, client_port);
DBGMARK(DBGSERVER,5,"DONE!\n");
shmdt(shared_memory);
shmctl(shmid, IPC_RMID, NULL);
shmdt(shared_memory_rl);
shmctl(shmid_rl, IPC_RMID, NULL);
return 0;
}
void usage()
{
DBGMARK(0,0,"./server [port] [path to ddpg.py] [Report Period: 20 msec] [First Time: 1=yes(learn), 0=no(continue learning), 2=evaluate] [actor id=0, 1, ...] [downlink] [uplink] [one-way delay]\n");
}
void start_server(int flow_num, int client_port)
{
cFlow *flows;
int num_lines=0;
sInfo *info;
info = new sInfo;
flows = new cFlow[flow_num];
if(flows==NULL)
{
DBGMARK(0,0,"flow generation failed\n");
return;
}
//threads
pthread_t data_thread;
pthread_t cnt_thread;
pthread_t timer_thread;
//Server address
struct sockaddr_in server_addr[FLOW_NUM];
//Client address
struct sockaddr_in client_addr[FLOW_NUM];
//Controller address
//struct sockaddr_in ctr_addr;
for(int i=0;i<FLOW_NUM;i++)
{
memset(&server_addr[i],0,sizeof(server_addr[i]));
//IP protocol
server_addr[i].sin_family=AF_INET;
//Listen on "0.0.0.0" (Any IP address of this host)
server_addr[i].sin_addr.s_addr=INADDR_ANY;
//Specify port number
server_addr[i].sin_port=htons(client_port+i);
//Init socket
if((sock[i]=socket(PF_INET,SOCK_STREAM,0))<0)
{
DBGMARK(0,0,"sockopt: %s\n",strerror(errno));
return;
}
int reuse = 1;
if (setsockopt(sock[i], SOL_SOCKET, SO_REUSEADDR, (const char*)&reuse, sizeof(reuse)) < 0)
perror("setsockopt(SO_REUSEADDR) failed");
//Bind socket on IP:Port
if(bind(sock[i],(struct sockaddr *)&server_addr[i],sizeof(struct sockaddr))<0)
{
DBGMARK(0,0,"bind error srv_ctr_ip: 000000: %s\n",strerror(errno));
close(sock[i]);
return;
}
if (scheme)
{
if (setsockopt(sock[i], IPPROTO_TCP, TCP_CONGESTION, scheme, strlen(scheme)) < 0)
{
DBGMARK(0,0,"TCP congestion doesn't exist: %s\n",strerror(errno));
return;
}
}
}
char container_cmd[500];
sprintf(container_cmd,"sudo -u `whoami` %s/client $MAHIMAHI_BASE 1 %d",path,client_port);
char cmd[1000];
char final_cmd[1000];
if (first_time==4 || first_time==2)
sprintf(cmd, "sudo -u `whoami` mm-delay %d mm-link %s/../traces/%s %s/../traces/%s --downlink-log=%s/log/down-%s --uplink-queue=droptail --uplink-queue-args=\"packets=%d\" --downlink-queue=droptail --downlink-queue-args=\"packets=%d\" -- sh -c \'%s\' &",delay_ms,path,uplink,path,downlink,path,log_file,qsize,qsize,container_cmd);
else
sprintf(cmd, "sudo -u `whoami` mm-delay %d mm-link %s/../traces/%s %s/../traces/%s --uplink-queue=droptail --uplink-queue-args=\"packets=%d\" --downlink-queue=droptail --downlink-queue-args=\"packets=%d\" -- sh -c \'%s\' &",delay_ms,path,uplink,path,downlink,qsize,qsize,container_cmd);
sprintf(final_cmd,"%s",cmd);
DBGPRINT(DBGSERVER,0,"%s\n",final_cmd);
info->trace=trace;
info->num_lines=num_lines;
/**
*Setup Shared Memory
*/
key=(key_t) (actor_id*10000+rand()%10000+1);
key_rl=(key_t) (actor_id*10000+rand()%10000+1);
// Setup shared memory, 11 is the size
if ((shmid = shmget(key, shmem_size, IPC_CREAT | 0666)) < 0)
{
printf("Error getting shared memory id");
return;
}
// Attached shared memory
if ((shared_memory = (char*)shmat(shmid, NULL, 0)) == (char *) -1)
{
printf("Error attaching shared memory id");
return;
}
// Setup shared memory, 11 is the size
if ((shmid_rl = shmget(key_rl, shmem_size, IPC_CREAT | 0666)) < 0)
{
printf("Error getting shared memory id");
return;
}
// Attached shared memory
if ((shared_memory_rl = (char*)shmat(shmid_rl, NULL, 0)) == (char *) -1)
{
printf("Error attaching shared memory id");
return;
}
if (first_time==1){
sprintf(cmd,"/home/`whoami`/venv/bin/python %s/d5.py --tb_interval=1 --base_path=%s --task=%d --job_name=actor --train_dir=%s --mem_r=%d --mem_w=%d &",path,path,actor_id,path,(int)key,(int)key_rl);
DBGPRINT(0,0,"Starting RL Module (Without load) ...\n%s",cmd);
}
else if (first_time==2 || first_time==4){
sprintf(cmd,"/home/`whoami`/venv/bin/python %s/d5.py --tb_interval=1 --base_path=%s --load --eval --task=%d --job_name=actor --train_dir=%s --mem_r=%d --mem_w=%d &",path,path,actor_id,path,(int)key,(int)key_rl);
DBGPRINT(0,0,"Starting RL Module (No learning) ...\n%s",cmd);
}
else
{
sprintf(cmd,"/home/`whoami`/venv/bin/python %s/d5.py --load --tb_interval=1 --base_path=%s --task=%d --job_name=actor --train_dir=%s --mem_r=%d --mem_w=%d &",path,path,actor_id,path,(int)key,(int)key_rl);
DBGPRINT(0,0,"Starting RL Module (With load) ...\n%s",cmd);
}
initial_timestamp();
system(cmd);
//Wait to get OK signal (alpha=OK_SIGNAL)
bool got_ready_signal_from_rl=false;
int signal;
char *num;
char*alpha;
char *save_ptr;
int signal_check_counter=0;
while(!got_ready_signal_from_rl)
{
//Get alpha from RL-Module
signal_check_counter++;
num=strtok_r(shared_memory_rl," ",&save_ptr);
alpha=strtok_r(NULL," ",&save_ptr);
if(num!=NULL && alpha!=NULL)
{
signal=atoi(alpha);
if(signal==OK_SIGNAL)
{
got_ready_signal_from_rl=true;
}
else{
usleep(1000);
}
}
else{
usleep(10000);
}
if (signal_check_counter>18000)
{
DBGERROR("After 3 minutes, no response (OK_Signal) from the Actor %d is received! We are going down down down ...\n",actor_id);
return;
}
}
DBGPRINT(0,0,"RL Module is Ready. Let's Start ...\n\n");
usleep(actor_id*10000+10000);
//Now its time to start the server-client app and tune C2TCP socket.
system(final_cmd);
//Start listen
int maxfdp=-1;
fd_set rset;
FD_ZERO(&rset);
//The maximum number of concurrent connections is 1
for(int i=0;i<FLOW_NUM;i++)
{
listen(sock[i],1);
//To be used in select() function
FD_SET(sock[i], &rset);
if(sock[i]>maxfdp)
maxfdp=sock[i];
}
//Timeout {1Hour} if something goes wrong! (Maybe mahimahi error...!)
maxfdp=maxfdp+1;
struct timeval timeout;
timeout.tv_sec = 60 * 60;
timeout.tv_usec = 0;
int rc = select(maxfdp, &rset, NULL, NULL, &timeout);
/**********************************************************/
/* Check to see if the select call failed. */
/**********************************************************/
if (rc < 0)
{
DBGERROR("=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=- select() failed =-=-=-=-=-=--=-=-=-=-=\n");
return;
}
/**********************************************************/
/* Check to see if the time out expired. */
/**********************************************************/
if (rc == 0)
{
DBGERROR("=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=- select() Timeout! =-=-=-=-=-=--=-=-=-=-=\n");
return;
}
int sin_size=sizeof(struct sockaddr_in);
while(flow_index<flow_num)
{
if (FD_ISSET(sock[flow_index], &rset))
{
int value=accept(sock[flow_index],(struct sockaddr *)&client_addr[flow_index],(socklen_t*)&sin_size);
if(value<0)
{
perror("accept error\n");
DBGMARK(0,0,"sockopt: %s\n",strerror(errno));
DBGMARK(0,0,"sock::%d, index:%d\n",sock[flow_index],flow_index);
close(sock[flow_index]);
return;
}
sock_for_cnt[flow_index]=value;
flows[flow_index].flowinfo.sock=value;
flows[flow_index].dst_addr=client_addr[flow_index];
if(pthread_create(&data_thread, NULL , DataThread, (void*)&flows[flow_index]) < 0)
{
perror("could not create thread\n");
close(sock[flow_index]);
return;
}
if (flow_index==0)
{
if(pthread_create(&cnt_thread, NULL , CntThread, (void*)info) < 0)
{
perror("could not create control thread\n");
close(sock[flow_index]);
return;
}
if(pthread_create(&timer_thread, NULL , TimerThread, (void*)info) < 0)
{
perror("could not create timer thread\n");
close(sock[flow_index]);
return;
}
}
DBGPRINT(0,0,"Server is Connected to the client...\n");
flow_index++;
}
}
pthread_join(data_thread, NULL);
}
void* TimerThread(void* information)
{
uint64_t start=timestamp();
unsigned int elapsed;
if ((duration!=0))
{
while(send_traffic)
{
sleep(1);
elapsed=(unsigned int)((timestamp()-start)/1000000); //unit s
if (elapsed>duration)
{
send_traffic=false;
}
}
}
return((void *)0);
}
void* CntThread(void* information)
{
/* struct sched_param param;
param.__sched_priority=sched_get_priority_max(SCHED_RR);
int policy=SCHED_RR;
int s = pthread_setschedparam(pthread_self(), policy, &param);
if (s!=0)
{
DBGPRINT(0,0,"Cannot set priority (%d) for the Main: %s\n",param.__sched_priority,strerror(errno));
}
s = pthread_getschedparam(pthread_self(),&policy,&param);
if (s!=0)
{
DBGPRINT(0,0,"Cannot get priority for the Data thread: %s\n",strerror(errno));
}
*/
int ret1;
double min_rtt_=0.0;
double pacing_rate=0.0;
double lost_bytes=0.0;
double lost_rate=0.0;
double srtt_ms=0.0;
double snd_ssthresh=0.0;
double packets_out=0.0;
double retrans_out=0.0;
double max_packets_out=0.0;
int reuse = 1;
int pre_id=9230;
int pre_id_tmp=0;
int msg_id=657;
bool got_alpha=false;
bool slow_start_passed=0;
for(int i=0;i<FLOW_NUM;i++)
{
if (setsockopt(sock_for_cnt[i], IPPROTO_TCP, TCP_NODELAY, &reuse, sizeof(reuse)) < 0)
{
DBGMARK(0,0,"ERROR: set TCP_NODELAY option %s\n",strerror(errno));
return((void *)0);
}
//Enable orca on this socket:
//TCP_ORCA_ENABLE
int enable_orca=2;
if (setsockopt(sock_for_cnt[i], IPPROTO_TCP, TCP_ORCA_ENABLE, &enable_orca, sizeof(enable_orca)) < 0)
{
DBGERROR("CHECK KERNEL VERSION (0514+) ;CANNOT ENABLE ORCA %s\n",strerror(errno));
return ((void* )0);
}
}
char message[1000];
char *num;
char*alpha;
char*save_ptr;
int got_no_zero=0;
uint64_t t0,t1;
t0=timestamp();
//Time to start the Logic
struct tcp_orca_info tcp_info_pre;
tcp_info_pre.init();
int get_info_error_counter=0;
int actor_is_dead_counter=0;
int tmp_step=0;
while(send_traffic)
{
for(int i=0;i<flow_index;i++)
{
got_no_zero=0;
usleep(report_period*1000);
while(!got_no_zero && send_traffic)
{
ret1= get_orca_info(sock_for_cnt[i],&orca_info);
if(ret1<0)
{
DBGMARK(0,0,"setsockopt: for index:%d flow_index:%d TCP_C2TCP ... %s (ret1:%d)\n",i,flow_index,strerror(errno),ret1);
return((void *)0);
}
if(orca_info.avg_urtt>0)
{
t1=timestamp();
double time_delta=(double)(t1-t0)/1000000.0;
double delay=(double)orca_info.avg_urtt/1000.0;
min_rtt_=(double)(orca_info.min_rtt/1000.0);
lost_bytes=(double)(orca_info.lost_bytes);
pacing_rate=(double)(orca_info.pacing_rate);
lost_rate=lost_bytes/time_delta; //Rate in MBps
srtt_ms=(double)((orca_info.srtt_us>>3)/1000.0);
snd_ssthresh=(double)(orca_info.snd_ssthresh);
packets_out=(double)(orca_info.packets_out);
retrans_out=(double)(orca_info.retrans_out);
max_packets_out=(double)(orca_info.max_packets_out);
report_period=20;
if (!slow_start_passed)
//Just for the first Time
slow_start_passed=(orca_info.snd_ssthresh<orca_info.cwnd)?1:0;
if(!slow_start_passed)
{
//got_no_zero=1;
tcp_info_pre=orca_info;
t0=timestamp();
target_ratio=1.1*orca_info.cwnd;
ret1 = setsockopt(sock_for_cnt[i], IPPROTO_TCP,TCP_CWND, &target_ratio,sizeof(target_ratio));
if(ret1<0)
{
DBGPRINT(0,0,"setsockopt: for index:%d flow_index:%d ... %s (ret1:%d)\n",i, flow_index,strerror(errno),ret1);
return((void *)0);
}
continue;
}
sprintf(message,"%d %.7f %.7f %.7f %.7f %.7f %.7f %.7f %.7f %.7f %.7f %.7f %.7f %.7f %.7f %.7f",
msg_id,delay,(double)orca_info.thr,(double)orca_info.cnt,(double)time_delta,
(double)target,(double)orca_info.cwnd, pacing_rate,lost_rate,srtt_ms,snd_ssthresh,packets_out
,retrans_out,max_packets_out,(double)orca_info.mss,min_rtt_);
memcpy(shared_memory,message,sizeof(message));
if ((duration_steps!=0))
{
step_it++;
if(step_it>duration_steps)
send_traffic=false;
}
msg_id=(msg_id+1)%1000;
DBGPRINT(DBGSERVER,1,"%s\n",message);
got_no_zero=1;
tcp_info_pre=orca_info;
t0=timestamp();
get_info_error_counter=0;
}
else
{
get_info_error_counter++;
if(get_info_error_counter>30000)
{
DBGMARK(0,0,"No valid state for 1 min. We (server of Actor %d) are going down down down ...\n",actor_id);
send_traffic=false;
}
usleep(report_period*100);
}
}
got_alpha=false;
int error_cnt=0;
int error2_cnt=0;
while(!got_alpha && send_traffic)
{
//Get alpha from RL-Module
num=strtok_r(shared_memory_rl," ",&save_ptr);
alpha=strtok_r(NULL," ",&save_ptr);
if(num!=NULL && alpha!=NULL)
{
pre_id_tmp=atoi(num);
target_ratio=atoi(alpha);
if(pre_id!=pre_id_tmp /*&& target_ratio!=OK_SIGNAL*/)
{
got_alpha=true;
pre_id=pre_id_tmp;
target_ratio=atoi(alpha)*orca_info.cwnd/100;
if (target_ratio<MIN_CWND)
target_ratio=MIN_CWND;
ret1 = setsockopt(sock_for_cnt[i], IPPROTO_TCP,TCP_CWND, &target_ratio, sizeof(target_ratio));
if(ret1<0)
{
DBGPRINT(0,0,"setsockopt: for index:%d flow_index:%d ... %s (ret1:%d)\n",i,flow_index,strerror(errno),ret1);
return((void *)0);
}
error_cnt=0;
}
else{
if (error_cnt>1000)
{
DBGPRINT(DBGSERVER,0,"still no new value id:%d prev_id:%d\n",pre_id_tmp,pre_id);
error_cnt=0;
}
error_cnt++;
usleep(1000);
}
error2_cnt=0;
}
else{
if (error2_cnt==50)
{
DBGPRINT(0,0,"got null values: (downlink:%s delay:%d qs:%d) Actor: %d iteration:%d\n",downlink,delay_ms,qsize,actor_id,step_it);
//FIXME:
//A Hack for now! Let's send a new state to get new action in case we have missed previous action. Why it happens?!
if((1+tmp_step)==(step_it))
{
actor_is_dead_counter++;
tmp_step=step_it;
if(actor_is_dead_counter>120)
{
DBGMARK(0,0,"No valid action for 1 min. We (server of actor %d) are going down down down ...\n",actor_id);
send_traffic=false;
}
}
else
{
actor_is_dead_counter=0;
tmp_step=step_it;
}
got_alpha=true;
error2_cnt=0;
}
else{
error2_cnt++;
usleep(10000);
}
}
}
}
}
shmdt(shared_memory);
shmctl(shmid, IPC_RMID, NULL);
shmdt(shared_memory_rl);
shmctl(shmid_rl, IPC_RMID, NULL);
return((void *)0);
}
void* DataThread(void* info)
{
/*
struct sched_param param;
param.__sched_priority=sched_get_priority_max(SCHED_RR);
int policy=SCHED_RR;
int s = pthread_setschedparam(pthread_self(), policy, &param);
if (s!=0)
{
DBGERROR("Cannot set priority (%d) for the Main: %s\n",param.__sched_priority,strerror(errno));
}
s = pthread_getschedparam(pthread_self(),&policy,&param);
if (s!=0)
{
DBGERROR("Cannot get priority for the Data thread: %s\n",strerror(errno));
}*/
//pthread_t send_msg_thread;
cFlow* flow = (cFlow*)info;
int sock_local = flow->flowinfo.sock;
char* src_ip;
char write_message[BUFSIZ+1];
char read_message[1024]={0};
int len;
char *savePtr;
char* dst_addr;
u64 loop;
u64 remaining_size;
memset(write_message,1,BUFSIZ);
write_message[BUFSIZ]='\0';
/**
* Get the RQ from client : {src_add} {flowid} {size} {dst_add}
*/
len=recv(sock_local,read_message,1024,0);
if(len<=0)
{
DBGMARK(DBGSERVER,1,"recv failed! \n");
close(sock_local);
return 0;
}
/**
* For Now: we send the src IP in the RQ to!
*/
src_ip=strtok_r(read_message," ",&savePtr);
if(src_ip==NULL)
{
//discard message:
DBGMARK(DBGSERVER,1,"id: %d discarding this message:%s \n",flow->flowinfo.flowid,savePtr);
close(sock_local);
return 0;
}
char * isstr = strtok_r(NULL," ",&savePtr);
if(isstr==NULL)
{
//discard message:
DBGMARK(DBGSERVER,1,"id: %d discarding this message:%s \n",flow->flowinfo.flowid,savePtr);
close(sock_local);
return 0;
}
flow->flowinfo.flowid=atoi(isstr);
char* size_=strtok_r(NULL," ",&savePtr);
flow->flowinfo.size=1024*atoi(size_);
DBGPRINT(DBGSERVER,4,"%s\n",size_);
dst_addr=strtok_r(NULL," ",&savePtr);
if(dst_addr==NULL)
{
//discard message:
DBGMARK(DBGSERVER,1,"id: %d discarding this message:%s \n",flow->flowinfo.flowid,savePtr);
close(sock_local);
return 0;
}
char* time_s_=strtok_r(NULL," ",&savePtr);
char *endptr;
start_of_client=strtoimax(time_s_,&endptr,10);
got_message=1;
DBGPRINT(DBGSERVER,2,"Got message: %" PRIu64 " us\n",timestamp());
flow->flowinfo.rem_size=flow->flowinfo.size;
DBGPRINT(DBGSERVER,2,"time_rcv:%" PRIu64 " get:%s\n",start_of_client,time_s_);
//Get detailed address
strtok_r(src_ip,".",&savePtr);
if(dst_addr==NULL)
{
//discard message:
DBGMARK(DBGSERVER,1,"id: %d discarding this message:%s \n",flow->flowinfo.flowid,savePtr);
close(sock_local);
return 0;
}
//Calculate loops. In each loop, we can send BUFSIZ (8192) bytes of data
loop=flow->flowinfo.size/BUFSIZ*1024;
//Calculate remaining size to be sent
remaining_size=flow->flowinfo.size*1024-loop*BUFSIZ;
//Send data with 8192 bytes each loop
DBGPRINT(0,0,"Server is sending the traffic ...\n");
// for(u64 i=0;i<loop;i++)
while(send_traffic)
{
len=strlen(write_message);
while(len>0)
{
DBGMARK(DBGSERVER,5,"++++++\n");
len-=send(sock_local,write_message,strlen(write_message),0);
usleep(50);
DBGMARK(DBGSERVER,5,"------\n");
}
usleep(100);
}
flow->flowinfo.rem_size=0;
done=true;
DBGPRINT(DBGSERVER,1,"done=true\n");
close(sock_local);
DBGPRINT(DBGSERVER,1,"done\n");
return((void *)0);
}
src/util.h
-64
View File
@@ -1,64 +0,0 @@
//============================================================================
// Author : Soheil Abbasloo
// Version : 1.0
//============================================================================
/*
MIT License
Copyright (c) Soheil Abbasloo 2020 (ab.soheil@gmail.com)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
*/
#ifndef UTIL_H_
#define UTIL_H_
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/time.h>
#include <netinet/tcp.h>
#include <unistd.h>
#include <pthread.h>
#include <string.h>
#include <errno.h>
#include <execinfo.h>
#include <signal.h>
#define TCP_QPLUS_STATE 19
#define TCP_QPLUS_CWND 20
#define TCP_QPLUS_ISBIG 21
#define NUM_CTR 1
#define PORT_CTR 20120
#define IPOPT_MEASUREMENT 0x40
#define IPOPT_TIMESTAMP (4 |IPOPT_MEASUREMENT)
#define IPOPT_TS_TSONLY 0 /* timestamps only */
#define IPOPT_TS_TSANDADDR 1 /* timestamps and addresses */
#define IPOPT_TS_PRESPEC 3 /* specified modules only */
#endif /* UTIL_H_ */
stylesheets/github-light.css
+130
View File
@@ -0,0 +1,130 @@
/*!
* GitHub Light v0.4.1
* Copyright (c) 2012 - 2017 GitHub, Inc.
* Licensed under MIT (https://github.com/primer/github-syntax-theme-generator/blob/master/LICENSE)
*/
.pl-c /* comment, punctuation.definition.comment, string.comment */ {
color: #6a737d;
}
.pl-c1 /* constant, entity.name.constant, variable.other.constant, variable.language, support, meta.property-name, support.constant, support.variable, meta.module-reference, markup.raw, meta.diff.header, meta.output */,
.pl-s .pl-v /* string variable */ {
color: #005cc5;
}
.pl-e /* entity */,
.pl-en /* entity.name */ {
color: #6f42c1;
}
.pl-smi /* variable.parameter.function, storage.modifier.package, storage.modifier.import, storage.type.java, variable.other */,
.pl-s .pl-s1 /* string source */ {
color: #24292e;
}
.pl-ent /* entity.name.tag, markup.quote */ {
color: #22863a;
}
.pl-k /* keyword, storage, storage.type */ {
color: #d73a49;
}
.pl-s /* string */,
.pl-pds /* punctuation.definition.string, source.regexp, string.regexp.character-class */,
.pl-s .pl-pse .pl-s1 /* string punctuation.section.embedded source */,
.pl-sr /* string.regexp */,
.pl-sr .pl-cce /* string.regexp constant.character.escape */,
.pl-sr .pl-sre /* string.regexp source.ruby.embedded */,
.pl-sr .pl-sra /* string.regexp string.regexp.arbitrary-repitition */ {
color: #032f62;
}
.pl-v /* variable */,
.pl-smw /* sublimelinter.mark.warning */ {
color: #e36209;
}
.pl-bu /* invalid.broken, invalid.deprecated, invalid.unimplemented, message.error, brackethighlighter.unmatched, sublimelinter.mark.error */ {
color: #b31d28;
}
.pl-ii /* invalid.illegal */ {
color: #fafbfc;
background-color: #b31d28;
}
.pl-c2 /* carriage-return */ {
color: #fafbfc;
background-color: #d73a49;
}
.pl-c2::before /* carriage-return */ {
content: "^M";
}
.pl-sr .pl-cce /* string.regexp constant.character.escape */ {
font-weight: bold;
color: #22863a;
}
.pl-ml /* markup.list */ {
color: #735c0f;
}
.pl-mh /* markup.heading */,
.pl-mh .pl-en /* markup.heading entity.name */,
.pl-ms /* meta.separator */ {
font-weight: bold;
color: #005cc5;
}
.pl-mi /* markup.italic */ {
font-style: italic;
color: #24292e;
}
.pl-mb /* markup.bold */ {
font-weight: bold;
color: #24292e;
}
.pl-md /* markup.deleted, meta.diff.header.from-file, punctuation.definition.deleted */ {
color: #b31d28;
background-color: #ffeef0;
}
.pl-mi1 /* markup.inserted, meta.diff.header.to-file, punctuation.definition.inserted */ {
color: #22863a;
background-color: #f0fff4;
}
.pl-mc /* markup.changed, punctuation.definition.changed */ {
color: #e36209;
background-color: #ffebda;
}
.pl-mi2 /* markup.ignored, markup.untracked */ {
color: #f6f8fa;
background-color: #005cc5;
}
.pl-mdr /* meta.diff.range */ {
font-weight: bold;
color: #6f42c1;
}
.pl-ba /* brackethighlighter.tag, brackethighlighter.curly, brackethighlighter.round, brackethighlighter.square, brackethighlighter.angle, brackethighlighter.quote */ {
color: #586069;
}
.pl-sg /* sublimelinter.gutter-mark */ {
color: #959da5;
}
.pl-corl /* constant.other.reference.link, string.other.link */ {
text-decoration: underline;
color: #032f62;
}
stylesheets/normalize.css
Vendored
+424
View File
@@ -0,0 +1,424 @@
/*! normalize.css v3.0.2 | MIT License | git.io/normalize */
/**
* 1. Set default font family to sans-serif.
* 2. Prevent iOS text size adjust after orientation change, without disabling
* user zoom.
*/
html {
font-family: sans-serif; /* 1 */
-ms-text-size-adjust: 100%; /* 2 */
-webkit-text-size-adjust: 100%; /* 2 */
}
/**
* Remove default margin.
*/
body {
margin: 0;
}
/* HTML5 display definitions
========================================================================== */
/**
* Correct `block` display not defined for any HTML5 element in IE 8/9.
* Correct `block` display not defined for `details` or `summary` in IE 10/11
* and Firefox.
* Correct `block` display not defined for `main` in IE 11.
*/
article,
aside,
details,
figcaption,
figure,
footer,
header,
hgroup,
main,
menu,
nav,
section,
summary {
display: block;
}
/**
* 1. Correct `inline-block` display not defined in IE 8/9.
* 2. Normalize vertical alignment of `progress` in Chrome, Firefox, and Opera.
*/
audio,
canvas,
progress,
video {
display: inline-block; /* 1 */
vertical-align: baseline; /* 2 */
}
/**
* Prevent modern browsers from displaying `audio` without controls.
* Remove excess height in iOS 5 devices.
*/
audio:not([controls]) {
display: none;
height: 0;
}
/**
* Address `[hidden]` styling not present in IE 8/9/10.
* Hide the `template` element in IE 8/9/11, Safari, and Firefox < 22.
*/
[hidden],
template {
display: none;
}
/* Links
========================================================================== */
/**
* Remove the gray background color from active links in IE 10.
*/
a {
background-color: transparent;
}
/**
* Improve readability when focused and also mouse hovered in all browsers.
*/
a:active,
a:hover {
outline: 0;
}
/* Text-level semantics
========================================================================== */
/**
* Address styling not present in IE 8/9/10/11, Safari, and Chrome.
*/
abbr[title] {
border-bottom: 1px dotted;
}
/**
* Address style set to `bolder` in Firefox 4+, Safari, and Chrome.
*/
b,
strong {
font-weight: bold;
}
/**
* Address styling not present in Safari and Chrome.
*/
dfn {
font-style: italic;
}
/**
* Address variable `h1` font-size and margin within `section` and `article`
* contexts in Firefox 4+, Safari, and Chrome.
*/
h1 {
font-size: 2em;
margin: 0.67em 0;
}
/**
* Address styling not present in IE 8/9.
*/
mark {
background: #ff0;
color: #000;
}
/**
* Address inconsistent and variable font size in all browsers.
*/
small {
font-size: 80%;
}
/**
* Prevent `sub` and `sup` affecting `line-height` in all browsers.
*/
sub,
sup {
font-size: 75%;
line-height: 0;
position: relative;
vertical-align: baseline;
}
sup {
top: -0.5em;
}
sub {
bottom: -0.25em;
}
/* Embedded content
========================================================================== */
/**
* Remove border when inside `a` element in IE 8/9/10.
*/
img {
border: 0;
}
/**
* Correct overflow not hidden in IE 9/10/11.
*/
svg:not(:root) {
overflow: hidden;
}
/* Grouping content
========================================================================== */
/**
* Address margin not present in IE 8/9 and Safari.
*/
figure {
margin: 1em 40px;
}
/**
* Address differences between Firefox and other browsers.
*/
hr {
box-sizing: content-box;
height: 0;
}
/**
* Contain overflow in all browsers.
*/
pre {
overflow: auto;
}
/**
* Address odd `em`-unit font size rendering in all browsers.
*/
code,
kbd,
pre,
samp {
font-family: monospace, monospace;
font-size: 1em;
}
/* Forms
========================================================================== */
/**
* Known limitation: by default, Chrome and Safari on OS X allow very limited
* styling of `select`, unless a `border` property is set.
*/
/**
* 1. Correct color not being inherited.
* Known issue: affects color of disabled elements.
* 2. Correct font properties not being inherited.
* 3. Address margins set differently in Firefox 4+, Safari, and Chrome.
*/
button,
input,
optgroup,
select,
textarea {
color: inherit; /* 1 */
font: inherit; /* 2 */
margin: 0; /* 3 */
}
/**
* Address `overflow` set to `hidden` in IE 8/9/10/11.
*/
button {
overflow: visible;
}
/**
* Address inconsistent `text-transform` inheritance for `button` and `select`.
* All other form control elements do not inherit `text-transform` values.
* Correct `button` style inheritance in Firefox, IE 8/9/10/11, and Opera.
* Correct `select` style inheritance in Firefox.
*/
button,
select {
text-transform: none;
}
/**
* 1. Avoid the WebKit bug in Android 4.0.* where (2) destroys native `audio`
* and `video` controls.
* 2. Correct inability to style clickable `input` types in iOS.
* 3. Improve usability and consistency of cursor style between image-type
* `input` and others.
*/
button,
html input[type="button"], /* 1 */
input[type="reset"],
input[type="submit"] {
-webkit-appearance: button; /* 2 */
cursor: pointer; /* 3 */
}
/**
* Re-set default cursor for disabled elements.
*/
button[disabled],
html input[disabled] {
cursor: default;
}
/**
* Remove inner padding and border in Firefox 4+.
*/
button::-moz-focus-inner,
input::-moz-focus-inner {
border: 0;
padding: 0;
}
/**
* Address Firefox 4+ setting `line-height` on `input` using `!important` in
* the UA stylesheet.
*/
input {
line-height: normal;
}
/**
* It's recommended that you don't attempt to style these elements.
* Firefox's implementation doesn't respect box-sizing, padding, or width.
*
* 1. Address box sizing set to `content-box` in IE 8/9/10.
* 2. Remove excess padding in IE 8/9/10.
*/
input[type="checkbox"],
input[type="radio"] {
box-sizing: border-box; /* 1 */
padding: 0; /* 2 */
}
/**
* Fix the cursor style for Chrome's increment/decrement buttons. For certain
* `font-size` values of the `input`, it causes the cursor style of the
* decrement button to change from `default` to `text`.
*/
input[type="number"]::-webkit-inner-spin-button,
input[type="number"]::-webkit-outer-spin-button {
height: auto;
}
/**
* 1. Address `appearance` set to `searchfield` in Safari and Chrome.
* 2. Address `box-sizing` set to `border-box` in Safari and Chrome
* (include `-moz` to future-proof).
*/
input[type="search"] {
-webkit-appearance: textfield; /* 1 */ /* 2 */
box-sizing: content-box;
}
/**
* Remove inner padding and search cancel button in Safari and Chrome on OS X.
* Safari (but not Chrome) clips the cancel button when the search input has
* padding (and `textfield` appearance).
*/
input[type="search"]::-webkit-search-cancel-button,
input[type="search"]::-webkit-search-decoration {
-webkit-appearance: none;
}
/**
* Define consistent border, margin, and padding.
*/
fieldset {
border: 1px solid #c0c0c0;
margin: 0 2px;
padding: 0.35em 0.625em 0.75em;
}
/**
* 1. Correct `color` not being inherited in IE 8/9/10/11.
* 2. Remove padding so people aren't caught out if they zero out fieldsets.
*/
legend {
border: 0; /* 1 */
padding: 0; /* 2 */
}
/**
* Remove default vertical scrollbar in IE 8/9/10/11.
*/
textarea {
overflow: auto;
}
/**
* Don't inherit the `font-weight` (applied by a rule above).
* NOTE: the default cannot safely be changed in Chrome and Safari on OS X.
*/
optgroup {
font-weight: bold;
}
/* Tables
========================================================================== */
/**
* Remove most spacing between table cells.
*/
table {
border-collapse: collapse;
border-spacing: 0;
}
td,
th {
padding: 0;
}
stylesheets/stylesheet.css
+245
View File
@@ -0,0 +1,245 @@
* {
box-sizing: border-box; }
body {
padding: 0;
margin: 0;
font-family: "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;
font-size: 16px;
line-height: 1.5;
color: #606c71; }
a {
color: #1e6bb8;
text-decoration: none; }
a:hover {
text-decoration: underline; }
.btn {
display: inline-block;
margin-bottom: 1rem;
color: rgba(255, 255, 255, 0.7);
background-color: rgba(255, 255, 255, 0.08);
border-color: rgba(255, 255, 255, 0.2);
border-style: solid;
border-width: 1px;
border-radius: 0.3rem;
transition: color 0.2s, background-color 0.2s, border-color 0.2s; }
.btn + .btn {
margin-left: 1rem; }
.btn:hover {
color: rgba(255, 255, 255, 0.8);
text-decoration: none;
background-color: rgba(255, 255, 255, 0.2);
border-color: rgba(255, 255, 255, 0.3); }
@media screen and (min-width: 64em) {
.btn {
padding: 0.75rem 1rem; } }
@media screen and (min-width: 42em) and (max-width: 64em) {
.btn {
padding: 0.6rem 0.9rem;
font-size: 0.9rem; } }
@media screen and (max-width: 42em) {
.btn {
display: block;
width: 100%;
padding: 0.75rem;
font-size: 0.9rem; }
.btn + .btn {
margin-top: 1rem;
margin-left: 0; } }
.page-header {
color: #fff;
text-align: center;
background-color: #159957;
background-image: linear-gradient(120deg, #155799, #159957); }
@media screen and (min-width: 64em) {
.page-header {
padding: 5rem 6rem; } }
@media screen and (min-width: 42em) and (max-width: 64em) {
.page-header {
padding: 3rem 4rem; } }
@media screen and (max-width: 42em) {
.page-header {
padding: 2rem 1rem; } }
.project-name {
margin-top: 0;
margin-bottom: 0.1rem; }
@media screen and (min-width: 64em) {
.project-name {
font-size: 3.25rem; } }
@media screen and (min-width: 42em) and (max-width: 64em) {
.project-name {
font-size: 2.25rem; } }
@media screen and (max-width: 42em) {
.project-name {
font-size: 1.75rem; } }
.project-tagline {
margin-bottom: 2rem;
font-weight: normal;
opacity: 0.7; }
@media screen and (min-width: 64em) {
.project-tagline {
font-size: 1.25rem; } }
@media screen and (min-width: 42em) and (max-width: 64em) {
.project-tagline {
font-size: 1.15rem; } }
@media screen and (max-width: 42em) {
.project-tagline {
font-size: 1rem; } }
.main-content :first-child {
margin-top: 0; }
.main-content img {
max-width: 100%; }
.main-content h1, .main-content h2, .main-content h3, .main-content h4, .main-content h5, .main-content h6 {
margin-top: 2rem;
margin-bottom: 1rem;
font-weight: normal;
color: #159957; }
.main-content p {
margin-bottom: 1em; }
.main-content code {
padding: 2px 4px;
font-family: Consolas, "Liberation Mono", Menlo, Courier, monospace;
font-size: 0.9rem;
color: #383e41;
background-color: #f3f6fa;
border-radius: 0.3rem; }
.main-content pre {
padding: 0.8rem;
margin-top: 0;
margin-bottom: 1rem;
font: 1rem Consolas, "Liberation Mono", Menlo, Courier, monospace;
color: #567482;
word-wrap: normal;
background-color: #f3f6fa;
border: solid 1px #dce6f0;
border-radius: 0.3rem; }
.main-content pre > code {
padding: 0;
margin: 0;
font-size: 0.9rem;
color: #567482;
word-break: normal;
white-space: pre;
background: transparent;
border: 0; }
.main-content .highlight {
margin-bottom: 1rem; }
.main-content .highlight pre {
margin-bottom: 0;
word-break: normal; }
.main-content .highlight pre, .main-content pre {
padding: 0.8rem;
overflow: auto;
font-size: 0.9rem;
line-height: 1.45;
border-radius: 0.3rem; }
.main-content pre code, .main-content pre tt {
display: inline;
max-width: initial;
padding: 0;
margin: 0;
overflow: initial;
line-height: inherit;
word-wrap: normal;
background-color: transparent;
border: 0; }
.main-content pre code:before, .main-content pre code:after, .main-content pre tt:before, .main-content pre tt:after {
content: normal; }
.main-content ul, .main-content ol {
margin-top: 0; }
.main-content blockquote {
padding: 0 1rem;
margin-left: 0;
color: #819198;
border-left: 0.3rem solid #dce6f0; }
.main-content blockquote > :first-child {
margin-top: 0; }
.main-content blockquote > :last-child {
margin-bottom: 0; }
.main-content table {
display: block;
width: 100%;
overflow: auto;
word-break: normal;
word-break: keep-all; }
.main-content table th {
font-weight: bold; }
.main-content table th, .main-content table td {
padding: 0.5rem 1rem;
border: 1px solid #e9ebec; }
.main-content dl {
padding: 0; }
.main-content dl dt {
padding: 0;
margin-top: 1rem;
font-size: 1rem;
font-weight: bold; }
.main-content dl dd {
padding: 0;
margin-bottom: 1rem; }
.main-content hr {
height: 2px;
padding: 0;
margin: 1rem 0;
background-color: #eff0f1;
border: 0; }
@media screen and (min-width: 64em) {
.main-content {
max-width: 64rem;
padding: 2rem 6rem;
margin: 0 auto;
font-size: 1.1rem; } }
@media screen and (min-width: 42em) and (max-width: 64em) {
.main-content {
padding: 2rem 4rem;
font-size: 1.1rem; } }
@media screen and (max-width: 42em) {
.main-content {
padding: 2rem 1rem;
font-size: 1rem; } }
.site-footer {
padding-top: 2rem;
margin-top: 2rem;
border-top: solid 1px #eff0f1; }
.site-footer-owner {
display: block;
font-weight: bold; }
.site-footer-credits {
color: #819198; }
@media screen and (min-width: 64em) {
.site-footer {
font-size: 1rem; } }
@media screen and (min-width: 42em) and (max-width: 64em) {
.site-footer {
font-size: 1rem; } }
@media screen and (max-width: 42em) {
.site-footer {
font-size: 0.9rem; } }
traces/step-10s-3-level
-33333
View File
File diff suppressed because it is too large. Load diff
traces/wired12
-1
View File
@@ -1 +0,0 @@
1
traces/wired192
-16
View File
@@ -1,16 +0,0 @@
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
traces/wired24
-2
View File
@@ -1,2 +0,0 @@
1
1
traces/wired300
-25
View File
@@ -1,25 +0,0 @@
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
traces/wired48
-6000000
View File
File diff suppressed because it is too large. Load diff
traces/wired6
-750001
View File
File diff suppressed because it is too large. Load diff
traces/wired96
-12000008
View File
File diff suppressed because it is too large. Load diff