hosting&docker host net & docker nat net apache服务器性能测试

测试目的
对比hosting，docker host和docker nat三种模式下apache服务器的性能差异。补充测试采用linux bridge 和 ovs bridige桥接时的性能，桥接工具使用的是pipework （https://github.com/jpetazzo/ pipeworkovs）。 三种部署模式下，httpd的性能均没有做优化，配置为安装完成的默认配置。

测试环境
paralles部署vm
centos7，4 cpu,4g 内存
httpd的docker镜像
测试软件：
apache benchmaker工具ab
docker 1.9

httpd镜像使用的dockefile

From centos:latestRUN yum makecache fast && \    yum -y update && \    yum -y install httpdCOPY ./start.sh /start.shRUN chmod +x /start.shCMD "/start.sh"

测试数据
hosting

fangfenghuadeMacBook-Pro:bin fangfenghua$ ab -c 100 -n 1000000 -k http://192.168.2.67:80/This is ApacheBench, Version 2.3 <$Revision: 1663405 $>Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/Licensed to The Apache Software Foundation, http://www.apache.org/Benchmarking 192.168.2.67 (be patient)Completed 100000 requestsCompleted 200000 requestsCompleted 300000 requestsCompleted 400000 requestsCompleted 500000 requestsCompleted 600000 requestsCompleted 700000 requestsCompleted 800000 requestsCompleted 900000 requestsCompleted 1000000 requestsFinished 1000000 requestsServer Software:        Apache/2.4.6Server Hostname:        192.168.2.67Server Port:            80Document Path:          /Document Length:        4897 bytesConcurrency Level:      100Time taken for tests:   53.555 secondsComplete requests:      1000000Failed requests:        0Non-2xx responses:      1000000Keep-Alive requests:    990156Total transferred:      5203566861 bytesHTML transferred:       4897000000 bytesRequests per second:    18672.41 [#/sec] (mean)Time per request:       5.355 [ms] (mean)Time per request:       0.054 [ms] (mean, across all concurrent requests)Transfer rate:          94885.88 [Kbytes/sec] receivedConnection Times (ms)              min  mean[+/-sd] median   maxConnect:        0    0   0.3      0     128Processing:     0    5  29.7      1    1601Waiting:        0    5  29.7      0    1601Total:          0    5  29.7      1    1601Percentage of the requests served within a certain time (ms)  50%      1  66%      1  75%      1  80%      2  90%      4  95%     15  98%     63  99%    125 100%   1601 (longest request)

Docker网络模式，hosts

fangfenghuadeMacBook-Pro:bin fangfenghua$ ab -c 100 -n 1000000 -k http://192.168.2.67:80/This is ApacheBench, Version 2.3 <$Revision: 1663405 $>Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/Licensed to The Apache Software Foundation, http://www.apache.org/Benchmarking 192.168.2.67 (be patient)Completed 100000 requestsCompleted 200000 requestsCompleted 300000 requestsCompleted 400000 requestsCompleted 500000 requestsCompleted 600000 requestsCompleted 700000 requestsCompleted 800000 requestsCompleted 900000 requestsCompleted 1000000 requestsFinished 1000000 requestsServer Software:        Apache/2.4.6Server Hostname:        192.168.2.67Server Port:            80Document Path:          /Document Length:        4897 bytesConcurrency Level:      100Time taken for tests:   56.680 secondsComplete requests:      1000000Failed requests:        0Non-2xx responses:      1000000Keep-Alive requests:    990155Total transferred:      5203566903 bytesHTML transferred:       4897000000 bytesRequests per second:    17642.98 [#/sec] (mean)Time per request:       5.668 [ms] (mean)Time per request:       0.057 [ms] (mean, across all concurrent requests)Transfer rate:          89654.70 [Kbytes/sec] receivedConnection Times (ms)              min  mean[+/-sd] median   maxConnect:        0    0   0.3      0     138Processing:     0    6  30.9      1    1961Waiting:        0    6  30.8      1    1961Total:          0    6  30.9      1    1961Percentage of the requests served within a certain time (ms)  50%      1  66%      1  75%      1  80%      2  90%      4  95%     17  98%     69  99%    126 100%   1961 (longest request)

Docker网络模式，nat

fangfenghuadeMacBook-Pro:bin fangfenghua$ ab -c 100 -n 1000000 -k http://192.168.2.67:90/This is ApacheBench, Version 2.3 <$Revision: 1663405 $>Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/Licensed to The Apache Software Foundation, http://www.apache.org/Benchmarking 192.168.2.67 (be patient)Completed 100000 requestsCompleted 200000 requestsCompleted 300000 requestsCompleted 400000 requestsCompleted 500000 requestsCompleted 600000 requestsCompleted 700000 requestsCompleted 800000 requestsCompleted 900000 requestsCompleted 1000000 requestsFinished 1000000 requestsServer Software:        Apache/2.4.6Server Hostname:        192.168.2.67Server Port:            90Document Path:          /Document Length:        4897 bytesConcurrency Level:      100Time taken for tests:   65.848 secondsComplete requests:      1000000Failed requests:        0Non-2xx responses:      1000000Keep-Alive requests:    990156Total transferred:      5203566907 bytesHTML transferred:       4897000000 bytesRequests per second:    15186.59 [#/sec] (mean)Time per request:       6.585 [ms] (mean)Time per request:       0.066 [ms] (mean, across all concurrent requests)Transfer rate:          77172.31 [Kbytes/sec] receivedConnection Times (ms)              min  mean[+/-sd] median   maxConnect:        0    0   0.2      0     105Processing:     0    7  39.1      1    2593Waiting:        0    7  39.1      1    2593Total:          0    7  39.1      1    2593Percentage of the requests served within a certain time (ms)  50%      1  66%      2  75%      3  80%      3  90%      7  95%     18  98%     58  99%    122 100%   2593 (longest request)

下面利用pipework工具，为容器指定网络。分别测试采用linux bridge和ovs bridge两种桥接方式的性能。
linux bridge

fangfenghuadeMacBook-Pro:bin fangfenghua$ ab -c 100 -n 1000000 -k http://192.168.2.67:80/This is ApacheBench, Version 2.3 <$Revision: 1663405 $>Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/Licensed to The Apache Software Foundation, http://www.apache.org/Benchmarking 192.168.2.67 (be patient)Completed 100000 requestsCompleted 200000 requestsCompleted 300000 requestsCompleted 400000 requestsCompleted 500000 requestsCompleted 600000 requestsCompleted 700000 requestsCompleted 800000 requestsCompleted 900000 requestsCompleted 1000000 requestsFinished 1000000 requestsServer Software:        Apache/2.4.6Server Hostname:        192.168.2.67Server Port:            80Document Path:          /Document Length:        4897 bytesConcurrency Level:      100Time taken for tests:   71.656 secondsComplete requests:      1000000Failed requests:        0Non-2xx responses:      1000000Keep-Alive requests:    990155Total transferred:      5203566881 bytesHTML transferred:       4897000000 bytesRequests per second:    13955.59 [#/sec] (mean)Time per request:       7.166 [ms] (mean)Time per request:       0.072 [ms] (mean, across all concurrent requests)Transfer rate:          70916.85 [Kbytes/sec] receivedConnection Times (ms)              min  mean[+/-sd] median   maxConnect:        0    0   0.0      0       5Processing:     0    7  37.7      1    2200Waiting:        0    7  37.7      1    2200Total:          0    7  37.7      1    2200Percentage of the requests served within a certain time (ms)  50%      1  66%      2  75%      3  80%      4  90%      7  95%     23  98%     74  99%    136 100%   2200 (longest request)

ovs bridge

fangfenghuadeMacBook-Pro:bin fangfenghua$ ab -c 100 -n 1000000 -k http://192.168.2.67:80/This is ApacheBench, Version 2.3 <$Revision: 1663405 $>Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/Licensed to The Apache Software Foundation, http://www.apache.org/Benchmarking 192.168.2.67 (be patient)Completed 100000 requestsCompleted 200000 requestsCompleted 300000 requestsCompleted 400000 requestsCompleted 500000 requestsCompleted 600000 requestsCompleted 700000 requestsCompleted 800000 requestsCompleted 900000 requestsCompleted 1000000 requestsFinished 1000000 requestsServer Software:        Apache/2.4.6Server Hostname:        192.168.2.67Server Port:            80Document Path:          /Document Length:        4897 bytesConcurrency Level:      100Time taken for tests:   66.925 secondsComplete requests:      1000000Failed requests:        0Non-2xx responses:      1000000Keep-Alive requests:    990152Total transferred:      5203566750 bytesHTML transferred:       4897000000 bytesRequests per second:    14942.03 [#/sec] (mean)Time per request:       6.693 [ms] (mean)Time per request:       0.067 [ms] (mean, across all concurrent requests)Transfer rate:          75929.52 [Kbytes/sec] receivedConnection Times (ms)              min  mean[+/-sd] median   maxConnect:        0    0   0.5      0     117Processing:     0    7  44.5      1    3577Waiting:        0    7  44.4      1    2173Total:          0    7  44.6      1    3577Percentage of the requests served within a certain time (ms)  50%      1  66%      1  75%      1  80%      2  90%      4  95%     13  98%     64  99%    205 100%   3577 (longest request)

汇总分析


上图中，横轴从左到右依此是hosting、docker net host、docker host nat、pipenetowork linux－br、pipenetwork ovs－br。
从汇总的数据来看，hosting和docker host模式的网络性能差异较小，基本可以忽略性能行书。但是nat模式下，rps，tps，tr均损失了17％左右。性能损失较大。从上述测试数据看，当前的nat模式的网络模式较为不可取。尤其对网络性能敏感的场景。
与docker原生支持的两种方式比较，linux bridge的性能损失最大。比docker默认的nat模式的性能损失还要大。ovs－brdige的性能几乎与docker的nat网络模式性能相当。