其他
多方计算实验中不同网络环境的模拟方法
tc来实现分别控制带宽(bandwidth)和网络延迟(latency/RTT), 本文简单介绍了模拟LAN/WAN环境的方法, 以供初学者食用.安装软件包
tc命令, 在Ubuntu下需要安装如下软件包:sudo apt-get update && sudo apt-get install -y iproute2 iperf3 iputils-ping
其中iproute2为tc命令软件包, iperf3为检查带宽的工具包, iputils-ping为检查网络延迟命令ping的工具包.
流量控制脚本
如下throttle.sh脚本出自Cheetah[1], 可用于模拟LAN和WAN的网络环境, 改变本地网络的带宽和网络延迟, 但需要管理员权限才能运行. 实际使用时, 可根据脚本中的提示修改相关参数.
#!/bin/bash
######
# Taken from https://github.com/emp-toolkit/emp-readme/blob/master/scripts/throttle.sh
######
## replace DEV=lo with your card (e.g., eth0)
DEV=lo
if [ "$1" == "del" ]
then
sudo tc qdisc del dev $DEV root
fi
if [ "$1" == "lan" ]
then
sudo tc qdisc del dev $DEV root
## about 3Gbps
sudo tc qdisc add dev $DEV root handle 1: tbf rate 3000mbit burst 100000 limit 10000
## about 0.3ms ping latency
sudo tc qdisc add dev $DEV parent 1:1 handle 10: netem delay 0.15msec
fi
if [ "$1" == "wan" ]
then
sudo tc qdisc del dev $DEV root
## about 400Mbps
sudo tc qdisc add dev $DEV root handle 1: tbf rate 400mbit burst 100000 limit 10000
## about 40ms ping latency
sudo tc qdisc add dev $DEV parent 1:1 handle 10: netem delay 20msec
fi
脚本使用方法:
# 授权, 只需执行一次
chmod 755 throttle.sh
# 模拟LAN
./throttle.sh lan
# 模拟WAN
./throttle.sh wan
# 删除配置
./throttle.sh del
为了检查带宽限制是否有效, 打开两个终端, 分别运行如下命令, 然后观察终端2最后产生的统计信息, 网络延迟则可以通过运行ping 127.0.0.1, 几秒后按Ctrl+C终止运行来查看效果.
# 终端1
iperf3 -s -i 1
# 终端2
iperf3 -c 127.0.0.1 -i 1 -t 10
在不设置任何限制时, 可能的输出情况如下:
# 终端1
root@0cc092f52a00:~# iperf3 -s -i 1
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 127.0.0.1, port 53288
[ 5] local 127.0.0.1 port 5201 connected to 127.0.0.1 port 53292
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 4.26 GBytes 36.6 Gbits/sec
[ 5] 1.00-2.00 sec 4.46 GBytes 38.3 Gbits/sec
[ 5] 2.00-3.00 sec 4.49 GBytes 38.6 Gbits/sec
[ 5] 3.00-4.00 sec 4.42 GBytes 38.0 Gbits/sec
[ 5] 4.00-5.00 sec 4.48 GBytes 38.5 Gbits/sec
[ 5] 5.00-6.00 sec 4.55 GBytes 39.1 Gbits/sec
[ 5] 6.00-7.00 sec 4.56 GBytes 39.1 Gbits/sec
[ 5] 7.00-8.00 sec 4.46 GBytes 38.3 Gbits/sec
[ 5] 8.00-9.00 sec 4.53 GBytes 38.9 Gbits/sec
[ 5] 9.00-10.00 sec 4.53 GBytes 38.9 Gbits/sec
[ 5] 10.00-10.04 sec 182 MBytes 37.3 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-10.04 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-10.04 sec 44.9 GBytes 38.4 Gbits/sec receiver
# 终端2
root@0cc092f52a00:~# iperf3 -c 127.0.0.1 -i 1 -t 10
Connecting to host 127.0.0.1, port 5201
[ 4] local 127.0.0.1 port 42508 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 4.13 GBytes 35.5 Gbits/sec 0 1.44 MBytes
[ 4] 1.00-2.00 sec 4.49 GBytes 38.6 Gbits/sec 0 1.44 MBytes
[ 4] 2.00-3.00 sec 4.41 GBytes 37.9 Gbits/sec 0 1.44 MBytes
[ 4] 3.00-4.00 sec 4.56 GBytes 39.2 Gbits/sec 0 1.44 MBytes
[ 4] 4.00-5.00 sec 4.44 GBytes 38.2 Gbits/sec 0 1.44 MBytes
[ 4] 5.00-6.00 sec 4.45 GBytes 38.2 Gbits/sec 0 1.44 MBytes
[ 4] 6.00-7.00 sec 4.54 GBytes 39.0 Gbits/sec 0 1.44 MBytes
[ 4] 7.00-8.00 sec 4.42 GBytes 38.0 Gbits/sec 0 1.44 MBytes
[ 4] 8.00-9.00 sec 4.29 GBytes 36.9 Gbits/sec 0 1.44 MBytes
[ 4] 9.00-10.00 sec 4.30 GBytes 36.9 Gbits/sec 0 1.44 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 44.0 GBytes 37.8 Gbits/sec 0 sender
[ 4] 0.00-10.00 sec 44.0 GBytes 37.8 Gbits/sec receiver
iperf Done.
# 查看网络延迟
root@0cc092f52a00:~# ping 127.0.0.1
PING 127.0.0.1 (127.0.0.1) 56(84) bytes of data.
64 bytes from 127.0.0.1: icmp_seq=1 ttl=64 time=0.043 ms
64 bytes from 127.0.0.1: icmp_seq=2 ttl=64 time=0.040 ms
64 bytes from 127.0.0.1: icmp_seq=3 ttl=64 time=0.045 ms
64 bytes from 127.0.0.1: icmp_seq=4 ttl=64 time=0.044 ms
64 bytes from 127.0.0.1: icmp_seq=5 ttl=64 time=0.047 ms
64 bytes from 127.0.0.1: icmp_seq=6 ttl=64 time=0.047 ms
64 bytes from 127.0.0.1: icmp_seq=7 ttl=64 time=0.044 ms
^C
--- 127.0.0.1 ping statistics ---
7 packets transmitted, 7 received, 0% packet loss, time 6122ms
rtt min/avg/max/mdev = 0.040/0.044/0.047/0.002 ms
可以看出本地带宽在不受限制时大约为37.8GB/s, 而网络延迟的平均值为0.044ms. 受限情况自测, 这里不再展示.
3注意事项
在Docker中使用: Docker容器(Ubuntu)默认 root用户, 因此使用脚本时需要去掉脚本中的sudo, 此时仍可能提示RTNETLINK answers: Operation not permitted, 这是因为容器没有网络管理权限, 需要在docker run镜像生成容器时加入额外的参数--cap-add=NET_ADMIN开启权限, 例如:
docker run --cap-add=NET_ADMIN -it --rm imagename bash
在WSL中使用: 由于Windows的原因, WSL中无法使用上述脚本进行网络限制.
参考资料及链接：
[1] Cheetah:
https://github.com/Alibaba-Gemini-Lab/OpenCheetah/blob/main/scripts/throttle.sh
[2] STACK OVERFLOW：
https://stackoverflow.com/questions/27708376/why-am-i-getting-an-rtnetlink-operation-not-permitted-when-using-pipework-with-d
往期推荐
横向联邦学习下隐私保护安全聚合：问题，方法，与展望
SPDZ学习笔记-part1
SPDZ学习笔记-part2