MongoDB GridFS 数据读取效率 benchmark

GridFS 是 MongoDB 下的一个子模块，利用这一模块，可以实现使用MongoDB存储文件，MongoDB 二进制包的bin目录下的mongofile 命令即可模拟实现该功能。

原文链接：http://lgone.com/html/y2010/815.html

数据来源：
http://www.coffeepowered.net/2010/02/17/serving-files-out-of-gridfs/
http://www.coffeepowered.net/2010/02/24/serving-files-out-of-gridfs-part-2/

另外一份benchmark数据：http://www.ypass.net/solaris/nginx-gridfs-benchmarks/rawresults.php

目前使用GridFS有以下三种方式：

1. 使用脚本读取，这时文件数据会全部读取到脚本中，再调用脚本客户端的输出功能输出给页面。毫无疑问，这种使用方式是效率非常低下的。
2. 使用gridfs-fuse使GridFS中存的数据可以通过标准磁盘IO方式进行访问，可以说是真正实现了一个文件系统。
3. 使用nginx-gridfs利用Nginx直接读取gridfs中的数据进行发送，这个类似于Nginx的sendfile机制。

文章作者就这几种方式做了测试，得出以下的benchmark数据。

1.1 通过Apache直接读取普通文件

[chris@polaris conf]# ab -n 50000 -c 10 http://advice/images/embed/alliance-60.pngServer Software:        Apache/2.2.13Server Hostname:        adviceServer Port:            80Document Path:          /images/embed/normal_alliance-60.pngDocument Length:        31596 bytesConcurrency Level:      10Time taken for tests:   1.904 secondsComplete requests:      5000Failed requests:        0Write errors:           0Total transferred:      159463760 bytesHTML transferred:       158043192 bytesRequests per second:    2625.37 [#/sec] (mean)Time per request:       3.809 [ms] (mean)Time per request:       0.381 [ms] (mean, across all concurrent requests)Transfer rate:          81767.87 [Kbytes/sec] receivedConnection Times (ms)              min  mean[+/-sd] median   maxConnect:        0    1   0.4      1       4Processing:     1    3   0.5      3       6Waiting:        0    1   0.4      1       4Total:          2    4   0.4      4       8Percentage of the requests served within a certain time (ms)  50%      4  66%      4  75%      4  80%      4  90%      4  95%      4  98%      5  99%      5 100%      8 (longest request)

结果：每秒2625次的请求处理，81.77M的吞吐率。

1.2 通过Nginx直接读取普通文件

[chris@polaris conf]# ab -n 50000 -c 10 http://advice:81/images/embed/normal_alliance-60.pngServer Software:        nginx/0.8.33Server Hostname:        adviceServer Port:            81Document Path:          /images/embed/normal_alliance-60.pngDocument Length:        31596 bytesConcurrency Level:      10Time taken for tests:   7.623 secondsComplete requests:      50000Failed requests:        0Write errors:           0Total transferred:      1590513618 bytesHTML transferred:       1579863192 bytesRequests per second:    6559.31 [#/sec] (mean)Time per request:       1.525 [ms] (mean)Time per request:       0.152 [ms] (mean, across all concurrent requests)Transfer rate:          203763.10 [Kbytes/sec] receivedConnection Times (ms)              min  mean[+/-sd] median   maxConnect:        0    0   0.2      0       9Processing:     1    1   0.4      1      11Waiting:        0    0   0.1      0       9Total:          1    1   0.5      1      12Percentage of the requests served within a certain time (ms)  50%      1  66%      1  75%      1  80%      2  90%      2  95%      2  98%      3  99%      3 100%     12 (longest request)

结果：每秒6559次的请求处理，203.763M的吞吐率。

2.通过gridfs-fuse读取gridfs中的数据

[chris@polaris gridfs-fuse]# ab -n 5000 -c 25 http://advice:81/images/gfs/uploads/user/avatar/4b8347a698db740b30000057/thumb_adrine-big.pngServer Software:        nginx/0.8.33Server Hostname:        adviceServer Port:            81Document Path:          /images/gfs/uploads/user/avatar/4b8347a698db740b30000057/thumb_adrine-big.pngDocument Length:        14332 bytesConcurrency Level:      25Time taken for tests:   5.029 secondsComplete requests:      5000Failed requests:        0Write errors:           0Total transferred:      72725000 bytesHTML transferred:       71660000 bytesRequests per second:    994.22 [#/sec] (mean)Time per request:       25.145 [ms] (mean)Time per request:       1.006 [ms] (mean, across all concurrent requests)Transfer rate:          14121.93 [Kbytes/sec] receivedConnection Times (ms)              min  mean[+/-sd] median   maxConnect:        0    0   0.1      0       1Processing:    16   25   1.4     25      52Waiting:        2   24   1.4     24      52Total:         17   25   1.4     25      53Percentage of the requests served within a certain time (ms)  50%     25  66%     25  75%     25  80%     25  90%     25  95%     26  98%     27  99%     32 100%     53 (longest request)

结果：每秒994次的请求处理，14.121M的吞吐率。

3 通过nginx-gridfs模块读取gridfs中的数据

[chris@polaris conf]# ab -n 5000 -c 10 http://advice:81/images/gfs/uploads/user/avatar/4b7b2c0e98db7475fc000003/normal_alliance-60.pngServer Software:        nginx/0.8.33Server Hostname:        adviceServer Port:            81Document Path:          /images/gfs/uploads/user/avatar/4b7b2c0e98db7475fc000003/normal_alliance-60.pngDocument Length:        31596 bytesConcurrency Level:      10Time taken for tests:   4.613 secondsComplete requests:      5000Failed requests:        0Write errors:           0Total transferred:      158580000 bytesHTML transferred:       157980000 bytesRequests per second:    1083.88 [#/sec] (mean)Time per request:       9.226 [ms] (mean)Time per request:       0.923 [ms] (mean, across all concurrent requests)Transfer rate:          33570.65 [Kbytes/sec] receivedConnection Times (ms)              min  mean[+/-sd] median   maxConnect:        0    0   0.0      0       1Processing:     1    9   4.7      9     103Waiting:        1    9   4.7      9     102Total:          2    9   4.7      9     103Percentage of the requests served within a certain time (ms)  50%      9  66%      9  75%      9  80%      9  90%      9  95%      9  98%      9  99%     11 100%    103 (longest request)

结果：每秒1083次的请求处理，33.57M吞吐率。

4 通过Rails客户端读取数据再发送

[chris@polaris nginx-gridfs]$ ab -n 250 -c 4  http://advice/images/gfs/uploads/user/avatar/4b7b2c0e98db7475fc000003/normal_alliance-60.pngServer Software:        Apache/2.2.13Server Hostname:        adviceServer Port:            80Document Path:          /images/gfs/uploads/user/avatar/4b7b2c0e98db7475fc000003/normal_alliance-60.pngDocument Length:        31596 bytesConcurrency Level:      4Time taken for tests:   4.646 secondsComplete requests:      250Failed requests:        0Write errors:           0Total transferred:      7960000 bytesHTML transferred:       7899000 bytesRequests per second:    53.81 [#/sec] (mean)Time per request:       74.338 [ms] (mean)Time per request:       18.585 [ms] (mean, across all concurrent requests)Transfer rate:          1673.10 [Kbytes/sec] receivedConnection Times (ms)              min  mean[+/-sd] median   maxConnect:        0    0   0.1      0       1Processing:    15   74  75.6     34     287Waiting:        0   72  75.8     30     276Total:         15   74  75.6     34     288Percentage of the requests served within a certain time (ms)  50%     34  66%     39  75%    139  80%    192  90%    201  95%    210  98%    239  99%    245 100%    288 (longest request)

结果：每秒53的请求处理，1.673M的吞吐率（效率确实不可能高）

结论：

SolutionRequests/second% Apache FS% Nginx FS%gridfs-fuse% Nginx GridFS% Apache RubyFilesystem via Apache2625.37-40.03%264.06%242.22%4,878.96%Filesystem via Nginx6559.31249.84%-657.74%605.17%12,189.76%nginx via gridfs-fuse994.2237.86%15.15%-91.74%1847.65%GridFS via nginx module1083.8841.28%16.52%109.02%-2014.27%Rails metal handler via Passenger53.812.05%0.82%5.41%4.96%-