TFS文件系统 安装

TFS（Taobao FileSystem）是一个高可扩展、高可用、高性能、面向互联网服务的分布式文件系统，其设计目标是支持海量的非结构化数据的存储；TFS使用C++语言开发，需要运行在64bit Linux OS上，TFS为淘宝提供海量小文件存储，通常文件大小不超过1M，满足了淘宝对小文件存储的需求，被广泛地应用 在淘宝各项应用中。它采用了HA架构和平滑扩容，保证了整个文件系统的可用性和扩展性。同时扁平化的数据组织结构，可将文件名映射到文件的物理地址，简化了文件的访问流程，一定程度上为TFS提供了良好的读写性能。本文介绍如何在Linux环境编译安装TFS及相关使用说明：

一、安装依赖的软件包:
1、automake TFS基于automake工具构建:

1

yum install automake.noarch

2、libtool automake需要使用libtool:

1

yuminstalllibtool

3、realine 用于命令行编辑的库：

1

yum install readline-devel

4、libz-devel 用于数据压缩/解压缩：

1

yuminstallzlib-devel

5、uuid-devel 用于生成全局唯一ID：

1

2

yum install e2fsprogs-devel

yum install libuuid-devel

6、tcmalloc google的内存管理库(由玩googl被封，那就暂且跳过吧，可选)

二、安装tb-common-utils
TFS使用tb-common-utils软件包，tb-common-utils包含淘宝使用的基础系统库tbsys和网络库tbnet两个组件；安装tb-common-utils前需要设置环境变量TBLIB_ROOT，tbsys和tbnet将会被安装TBLIB_ROOT对应的路径（必须是绝对路径）下，TFS会在这个路径下查找tbsys、tbnet头文件和库。
设置TBLIB_ROOT环境变量：
1、在~/.bash_profile文件中加入，export TBLIB_ROOT=path_to_tbutil , 然后执行source：

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

~/.bash_profile

 

vi~/.bash_profile

# .bash_profile

 

# Get the aliases and functions

if[-f~/.bashrc];then

.~/.bashrc

fi

 

# User specific environment and startup programs

 

PATH=$PATH:$HOME/bin

 

exportPATH

exportTBLIB_ROOT=/usr/local/tb-common-utils

 

# source ~/.bash_profile

2、下载源码：

1

# svn co -r 18 http://code.taobao.org/svn/tb-common-utils/trunk tb-common-utils

注意： 这里不要checkout最新版本，version18以后的修改导致部分接口不能前向兼容

1

2

3

4

5

6

7

8

9

# cd /usr/local/tb-common-utils/

# sh build.sh

 

checkingforC++compilerdefaultoutputfilename...

configure:error:in`/usr/local/tb-common-utils/tbnet':

configure: error: C++ compiler cannot create executables

See `config.log'formoredetails.

make:***Notargetsspecifiedandnomakefilefound.  Stop.

make:***Noruletomaketarget`install'.  Stop.

这个错误是因为缺少gcc-c++：

1

yum install gcc-c++

如果一切顺利，tb-common-utils已经安装成功到$TBLIB_ROOT路径下；

三、安装TFS
TFS开源用户大都只使用TFS的基本功能，所以这个版本我们默认只编译TFS的nameserver，dataserver，client和tool，以去除对mysql的依赖，需要使用到rcserver（全局资源管理服务），metaserver(TFS自定义文件名服务）的用户请自行编译安装这两个服务。
下载源码并安装：

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

# svn co http://code.taobao.org/svn/tfs/branches/dev_for_outer_users tfs

# cd tfs/

# sh build.sh init

+aclocal

+libtoolize--force--copy

libtoolize:puttingauxiliaryfilesin`.'.

libtoolize: copying file `./ltmain.sh'

libtoolize:Consideradding`AC_CONFIG_MACRO_DIR([m4])' to configure.ac and

libtoolize: rerunning libtoolize, to keep the correct libtool macros in-tree.

libtoolize: Consider adding `-I m4'toACLOCAL_AMFLAGSinMakefile.am.

libtoolize:`AC_PROG_RANLIB' is rendered obsolete by `LT_INIT'

+autoconf--force

+automake--foreign--copy--add-missing

configure.ac:16:installing`./config.guess'

configure.ac:16: installing `./config.sub'

configure.ac:8:installing`./install-sh'

configure.ac:8: installing `./missing'

src/adminserver/Makefile.am:installing`./depcomp'

# ./configure --prefix=/usr/local/taobaoFS --with-release

 

checking for tc_cfree in -ltcmalloc... no

configure: error: in `/usr/local/tfs/tfs':

configure:error:tcmalloclinkfailed(--without-tcmalloctodisable)

See`config.log'formoredetails.

注意，这里报错了，是因为之前我们没有安装tcmalloc，所以这里要改一下编译条件：

1

2

3

# ./configure --prefix=/usr/local/taobaoFS --with-release --without-tcmalloc

# make

# make install

–prefix 指定tfs安装路径，默认会被安装到~/tfs_bin目录
–with-release 指定按release版本的参数进行编译，如果不指定这个参数，则会按开发版本比较严格的参数编译，包含-Werror参数，所有的警告都会被当错误，在高版本gcc下会导致项目编译不过，很多开源用户反馈的编译问题都跟这个有关，因为gcc高版本对代码的检查越来越严格，淘宝内部使用的gcc版本是gcc-4.1.2。

四、配置文件：
ns.conf:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

# cd /usr/local/taobaoFS/conf/

# cat ns.conf | grep -v ^# | grep -v ^$

[public]

log_size=1073741824

log_num=8

log_level=info

task_max_queue_size=10240

port=8108

work_dir=/usr/local/taobaoFS

dev_name=eth0

thread_count=32

ip_addr=192.168.1.63

[nameserver]

safe_mode_time=360

ip_addr_list=192.168.1.63|192.168.0.2

group_mask=255.255.255.255

max_write_timeout=3

cluster_id=1

block_max_use_ratio=98

block_max_size=75497472

max_replication=2

min_replication=2

replicate_ratio=50

max_write_filecount=64

use_capacity_ratio=96

heart_interval=4

object_dead_max_time=300

object_clear_max_time=180

heart_thread_count=4

heart_max_queue_size=2048

report_block_thread_count=6

report_block_max_queue_size=32

report_block_hour_range=2~4

report_block_time_interval=1

repl_wait_time=180

compact_delete_ratio=  10  

compact_max_load=200

compact_hour_range=1~10

dump_stat_info_interval=60000000

balance_percent=0.05

add_primary_block_count=3

task_percent_sec_size=200

oplog_sync_max_slots_num=1024

oplog_sync_thread_num=1

group_count=1

group_seq  =0

discard_newblk_safe_mode_time=360

choose_target_server_random_max_num=128

ds.conf:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

# cat ds.conf | grep -v ^# | grep -v ^$

[public]

log_size=1073741824

log_num = 8

log_level=info

task_max_queue_size = 10240

port = 8200

work_dir=/usr/local/taobaoFS

dev_name= eth0

thread_count = 32

ip_addr = 192.168.1.64

[dataserver]

ip_addr = 192.168.1.63

ip_addr_list = 192.168.1.63|192.168.0.2

port = 8108

heart_interval = 2

check_interval = 2

replicate_threadcount = 2

block_max_size = 75497472

dump_visit_stat_interval = 60

backup_type = 1

mount_name = /data/disk

mount_maxsize = 768959044

base_filesystem_type = 1

superblock_reserve = 0

avg_file_size = 15360

mainblock_size = 75497472

extblock_size = 4194304

block_ratio = 0.5

hash_slot_ratio = 0.5

第三台服务器的ds配置如下：

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

# cat ds.conf | grep -v ^# | grep -v ^$

[public]

log_size=1073741824

log_num=8

log_level=info

task_max_queue_size=10240

port=8200

work_dir=/usr/local/taobaoFS

dev_name=eth0

thread_count=32

ip_addr=192.168.1.66

[dataserver]

ip_addr=192.168.1.63

ip_addr_list=192.168.1.63|192.168.0.2

port=8108

heart_interval=2

check_interval=2

replicate_threadcount=2

block_max_size=75497472

dump_visit_stat_interval=60

backup_type=1

mount_name=/data/disk

mount_maxsize=768959044

base_filesystem_type=1

superblock_reserve=0

avg_file_size=15360

mainblock_size=75497472

extblock_size=4194304

block_ratio=0.5

hash_slot_ratio=0.5

五、启动说胆：
1、运行TFS
启动nameserver
执行scripts目录下的tfs：

1

2

# ./tfs start_ns

nameserver is up SUCCESSFULLY pid: 24744

2、启动DS
现有TFS可以在一台服务器上启动多个DataServer进程。一般每个DataServer进程负责一个磁盘。
将数据盘格式化成EXT4文件系统，并挂载到/data/tfs1至/data/tfs(i)，其中i为磁盘号。
启动步骤：
A、存储区预分配。执行scripts下的stfs format n (n为挂载点的序号，具体用法见stfs的Usage)。例如stfs format 2,4-6 则会对/data/tfs2,
/data/tfs4,/data/tfs5,/data/tfs6，进行预分配。运行完后会在生成/data/tfs2, /data/tfs4,/data/tfs5,/data/tfs6下预先创建主块，扩展块及相应的统计信息。

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

# ./stfs clear 1

cleards1SUCCESSFULLY

[2014-08-0710:59:11]INFO  blockfile_manager.cpp:111[140398366021408]clearblockfilesystemend.mount_point:/data/disk1,ret:1

# ./stfs format 1

formatds1SUCCESSFULLY

mountname:/data/disk1maxmountsize:768959044basefstype:1superblockreserveoffset:0mainblocksize:75497472extendblocksize:4194304blockratio:0.5filesystemversion:1avginnerfilesize:15360hashslotratio:0.5

[2014-08-0712:16:15]INFO  blockfile_manager.cpp:1091[140579583227680]superblockmountpoint:/data/disk1.

[2014-08-0712:16:15]INFO  blockfile_manager.cpp:1171[140579583227680]calblockcount.availdataspace:783334178816,mainblockcount:9338,extblockcount:18676

tagTAOBAO

mounttime1407384975

mountdesc/data/disk1

maxusespace787414061056

basefilesystem1

superblockreserve0

bitmapstartoffset324

avginnerfilesize15360

blocktyperatio0.5

mainblockcount9338

mainblocksize75497472

extendblockcount18676

extendblocksize4194304

usedmainblockcount0

usedextendblockcount0

hashslotratio0.5

hashslotsize2730

firstmmapsize122880

mmapsizestep4096

maxmmapsize3686400

version1

[2014-08-0712:16:15]INFO  blockfile_manager.cpp:1213[140579583227680]calbitmapcount.itemcount:28015,slotcount:3502

B、运行data server。有两种方法：
通过adminserver来启动dataserver（推荐）: 执行scripts下的./tfs admin_ds
直接启动dataserver，执行scripts下的./tfs start_ds 2,4-6， 则会启动dataserver2，dataserver4，dataserver5，dataserver6

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

# ./tfs start_ds 1

dataserver 1 is up SUCCESSFULLY pid: 2212

# ./tfs start_ds 2

dataserver 2 is up SUCCESSFULLY pid: 28715

 

# netstat -lantp | grep dataserver

tcp        0      0 0.0.0.0:8200                0.0.0.0:*                   LISTEN      2960/dataserver    

tcp        0      0 0.0.0.0:8201                0.0.0.0:*                   LISTEN      2960/dataserver    

tcp        0      0 192.168.1.64:36866          192.168.1.63:8108           ESTABLISHED 2960/dataserver    

tcp        0      1 192.168.1.64:38846          192.168.0.2:8108            SYN_SENT    2960/dataserver  

 

# netstat -lantp | grep dataserver

tcp        0      0 0.0.0.0:8203                0.0.0.0:*                   LISTEN      29636/dataserver    

tcp        0      0 0.0.0.0:8202                0.0.0.0:*                   LISTEN      29636/dataserver    

tcp        0      0 192.168.1.66:56412          192.168.1.63:8108           ESTABLISHED 29636/dataserver    

tcp        0      1 192.168.1.66:49068          192.168.0.2:8108            SYN_SENT    29636/dataserver

程序基本上启动起来了

第六、相关操作

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

# /usr/local/taobaoFS/bin/ssm -s 192.168.1.63:8108

show>server-b

SERVER_ADDR          CNTBLOCK

192.168.1.64:8200    194  2846  2847  2848  2849  2850  2851  2852  2853  2854  2855

                          2856  2857  2858  2859  2860  2861  2862  2863  2864  2865

                          2866  2867  2868  2869  2870  2871  2872  2873  2874  2875

                          2876  2877  2878  2879  2880  2881  2882  2883  2884  2885

                          2886  2887  2888  2889  2890  2891  2892  2893  2894  2895

                          2896  2897  2898  2899  2900  2901  2902  2903  2904  2905

                          2906  2907  2908  2909  2910  2911  2912  2913  2914  2915

                          2916  2917  2918  2919  2920  2921  2922  2923  2924  2925

                          2926  2927  2928  2929  2930  2931  2932  2933  2934  2935

                          2936  2937  2938  2939  2940  2941  2942  2943  2944  2945

                          2946  2947  2948  2949  2950  2951  2952  2953  2954  2955

                          2956  2957  2958  2959  2960  2961  2962  2963  2964  2965

                          2966  2967  2968  2969  2970  2971  2972  2973  2974  2975

                          2976  2977  2978  2979  2980  2981  2982  2983  2984  2985

                          2986  2987  2988  2989  2990  2991  2992  2993  2994  2995

                          2996  2997  2998  2999  3000  3001  3002  3003  3004  3005

                          3006  3007  3008  3009  3010  3011  3012  3013  3014  3015

                          3016  3017  3018  3019  3020  3021  3022  3023  3024  3025

                          3026  3027  3028  3029  3030  3031  3032  3033  3034  3035

                          3036  3037  3038  3139

192.168.1.66:8202    194  2846  2847  2848  2849  2850  2851  2852  2853  2854  2855

                          2856  2857  2858  2859  2860  2861  2862  2863  2864  2865

                          2866  2867  2868  2869  2870  2871  2872  2873  2874  2875

                          2876  2877  2878  2879  2880  2881  2882  2883  2884  2885

                          2886  2887  2888  2889  2890  2891  2892  2893  2894  2895

                          2896  2897  2898  2899  2900  2901  2902  2903  2904  2905

                          2906  2907  2908  2909  2910  2911  2912  2913  2914  2915

                          2916  2917  2918  2919  2920  2921  2922  2923  2924  2925

                          2926  2927  2928  2929  2930  2931  2932  2933  2934  2935

                          2936  2937  2938  2939  2940  2941  2942  2943  2944  2945

                          2946  2947  2948  2949  2950  2951  2952  2953  2954  2955

                          2956  2957  2958  2959  2960  2961  2962  2963  2964  2965

                          2966  2967  2968  2969  2970  2971  2972  2973  2974  2975

                          2976  2977  2978  2979  2980  2981  2982  2983  2984  2985

                          2986  2987  2988  2989  2990  2991  2992  2993  2994  2995

                          2996  2997  2998  2999  3000  3001  3002  3003  3004  3005

                          3006  3007  3008  3009  3010  3011  3012  3013  3014  3015

                          3016  3017  3018  3019  3020  3021  3022  3023  3024  3025

                          3026  3027  3028  3029  3030  3031  3032  3033  3034  3035

                          3036  3037  3038  3139

show>server-w

SERVER_ADDR          CNTWRITABLEBLOCK

192.168.1.64:8200    90  2849  2850  2852  2856  2857  2859  2860  2862  2865  2866

                          2871  2878  2879  2884  2886  2887  2891  2895  2899  2900

                          2901  2905  2906  2907  2911  2912  2913  2914  2915  2917

                          2920  2921  2925  2927  2929  2930  2932  2934  2935  2936

                          2937  2938  2942  2944  2945  2946  2948  2950  2951  2955

                          2957  2958  2959  2960  2961  2964  2967  2968  2978  2979

                          2980  2982  2984  2985  2987  2988  2990  2993  2994  3000

                          3004  3005  3006  3007  3010  3012  3013  3014  3015  3016

                          3018  3020  3022  3025  3027  3029  3030  3035  3038  3139

192.168.1.66:8202    103  2846  2847  2848  2851  2853  2854  2855  2858  2861  2863

                          2864  2867  2868  2869  2870  2872  2874  2875  2876  2877

                          2880  2881  2882  2883  2885  2888  2889  2890  2892  2893

                          2894  2896  2897  2898  2902  2903  2904  2908  2909  2910

                          2916  2918  2919  2922  2923  2924  2926  2928  2931  2933

                          2939  2940  2941  2943  2947  2949  2952  2953  2954  2956

                          2962  2963  2965  2966  2969  2970  2971  2972  2973  2974

                          2975  2976  2977  2981  2983  2986  2989  2991  2992  2995

                          2996  2997  2998  2999  3001  3002  3003  3008  3009  3011

                          3017  3019  3021  3023  3024  3026  3028  3031  3032  3033

                          3034  3036  3037

show>server-m

SERVER_ADDR          CNTMASTERBLOCK

192.168.1.64:8200    64  2849  2850  2852  2856  2857  2859  2860  2862  2865  2866

                          2871  2878  2879  2884  2886  2887  2891  2895  2899  2900

                          2901  2905  2906  2907  2911  2912  2913  2914  2915  2917

                          2920  2921  2925  2927  2929  2930  2932  2934  2935  2936

                          2937  2990  2993  2994  3000  3004  3005  3006  3007  3010

                          3012  3013  3014  3015  3016  3018  3020  3022  3025  3027

                          3029  3030  3035  3038

192.168.1.66:8202    64  2846  2847  2848  2851  2853  2854  2855  2858  2861  2863

                          2864  2867  2868  2869  2870  2872  2874  2875  2876  2877

                          2880  2881  2882  2883  2885  2969  2970  2971  2972  2973

                          2974  2975  2976  2977  2981  2983  2986  2989  2991  2992

                          2995  2996  2997  2998  2999  3001  3002  3003  3008  3009

                          3011  3017  3019  3021  3023  3024  3026  3028  3031  3032

                          3033  3034  3036  3037

show>machine-a

  SERVER_IP    NUMSUCAP  /TCAP=  UR  BLKCNT  LOADTOTAL_WRITE  TOTAL_READ  LAST_WRITE  LAST_READ  MAX_WRITE  MAX_READ

------------------------------------------------------------  ----------  ----------  ---------  --------  ---------

  192.168.1.64    115.16G729.53G  2%    194  10  4.1K    0    0    0    0    0    0    0  0    0  0    0

  192.168.1.66    115.16G729.53G  2%    194  10  4.1K    0    3    0    0    0    0    0  0    0  0    0

Total:2          230.31G  1.42T  2%    388  10  8.3K    0    3    0    0    0    0    0

 

show>machine-p

  SERVER_IP    NUMSUCAP  /TCAP=  UR  BLKCNT  LOADLAST_WRITE  LAST_READ  MAX_WRITE  MAX_READSTARTUP_TIME

-----------------------------------------------------------  ---------  ---------  --------------------

  192.168.1.64    115.16G729.53G  2%    194  10    0    0    0    0    0    0    0    02014-08-0718:57:11

  192.168.1.66    115.16G729.53G  2%    194  10    0    0    0    0    0    0    0    02014-08-0719:00:06

Total:2          230.31G  1.42T  2%    388  10      0    0    0    0

存数据至tfs：

1

2

3

4

5

6

7

8

# /usr/local/taobaoFS/bin/tfstool -s 192.168.1.63:8108 -i "put /root/haproxy.cfg"

中间省略N行...

[2014-08-07 14:49:38] DEBUG tfs_file.cpp:803 [139796598765408] do response success. index: 0, phase: 3, ret: 0, blockid: 2866, fileid: 1, offset: 0, size: 0, crc: -1605709817, inneroffset: 0, filenumber: 4612152245717303297, status: 4, rserver: 192.168.1.64:8200, wserver: 192.168.1.64:8200.

put /root/haproxy.cfg => T1YybTByJT1RCvBVdK success.

[2014-08-07 14:49:39] INFO  transport.cpp:460 [139796598765408] DELIOC, IOCount:1, IOC:0x26076b0

[2014-08-07 14:49:39] DEBUG socket.cpp:122 [139796598765408] 1?? fd=4, addr=192.168.1.63:8108

[2014-08-07 14:49:39] INFO  transport.cpp:460 [139796598765408] DELIOC, IOCount:0, IOC:0x2609460

[2014-08-07 14:49:39] DEBUG socket.cpp:122 [139796598765408] 1?? fd=6, addr=192.168.1.64:8200

从tfs取数据：

1

2

3

4

5

6

7

8

# /usr/local/taobaoFS/bin/tfstool -s 192.168.1.63:8108 -i "get T1YybTByJT1RCvBVdK /root/test/haproxy.cfg"

中间省略N行...

[2014-08-0715:27:27]DEBUGtfs_file.cpp:225[139699370776416]filereadreachend,offset:0,size:1048576

fetchT1YybTByJT1RCvBVdK=>/root/test/haproxy.cfgsuccess.

[2014-08-0715:27:28]INFO  transport.cpp:460[139699370776416]DELIOC,IOCount:1,IOC:0xb6e6b0

[2014-08-0715:27:28]DEBUGsocket.cpp:122[139699370776416]1??fd=4,addr=192.168.1.63:8108

[2014-08-0715:27:28]INFO  transport.cpp:460[139699370776416]DELIOC,IOCount:0,IOC:0xb70680

[2014-08-0715:27:28]DEBUGsocket.cpp:122[139699370776416]1??fd=6,addr=192.168.1.66:8202

存放大文件至tfs:

1

2

3

4

5

6

7

8

9

# /usr/local/taobaoFS/bin/tfstool -s 192.168.1.63:8108 -i "putL /root/CentOS-6.5-x86_64-minimal-freewaf-1.2.1-release_21152.iso"

中间省略N行...

put /root/CentOS-6.5-x86_64-minimal-freewaf-1.2.1-release_21152.iso => L1YybTByZT1RCvBVdK success.

[2014-08-07 17:03:50] INFO  transport.cpp:460 [140184226105184] DELIOC, IOCount:2, IOC:0x194c6b0

[2014-08-07 17:03:50] DEBUG socket.cpp:122 [140184226105184] 1?? fd=4, addr=192.168.1.63:8108

[2014-08-07 17:03:50] INFO  transport.cpp:460 [140184226105184] DELIOC, IOCount:1, IOC:0x194e970

[2014-08-07 17:03:50] DEBUG socket.cpp:122 [140184226105184] 1?? fd=8, addr=192.168.1.66:8202

[2014-08-07 17:03:50] INFO  transport.cpp:460 [140184226105184] DELIOC, IOCount:0, IOC:0x194f930

[2014-08-07 17:03:50] DEBUG socket.cpp:122 [140184226105184] 1?? fd=9, addr=192.168.1.64:8200

取大文件：

1

2

3

4

5

6

7

8

9

# /usr/local/taobaoFS/bin/tfstool -s 192.168.1.63:8108 -i "get L1YybTByZT1RCvBVdK /data/CentOS-6.5-x86_64-minimal-freewaf-1.2.1-release_21152.iso"

[2014-08-0717:08:14]DEBUGtfs_file.cpp:201[139878759466848]filereadreachend,offset:645128192,size:0

fetchL1YybTByZT1RCvBVdK=>/data/CentOS-6.5-x86_64-minimal-freewaf-1.2.1-release_21152.isosuccess.

[2014-08-0717:08:15]INFO  transport.cpp:460[139878759466848]DELIOC,IOCount:2,IOC:0xc5a6b0

[2014-08-0717:08:15]DEBUGsocket.cpp:122[139878759466848]1??fd=4,addr=192.168.1.63:8108

[2014-08-0717:08:15]INFO  transport.cpp:460[139878759466848]DELIOC,IOCount:1,IOC:0xc5c7b0

[2014-08-0717:08:15]DEBUGsocket.cpp:122[139878759466848]1??fd=6,addr=192.168.1.64:8200

[2014-08-0717:08:15]INFO  transport.cpp:460[139878759466848]DELIOC,IOCount:0,IOC:0xc63790

[2014-08-0717:08:15]DEBUGsocket.cpp:122[139878759466848]1??fd=7,addr=192.168.1.66:8202

对比结果：

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

[root@IP-1-63~]# ll -h /root/

total828M

-rw-------.1rootroot1.1KJul3110:48anaconda-ks.cfg

-rw-r--r--.1rootroot3.5MJul3114:16Atlas-2.1.el6.x86_64.rpm

-rw-r--r--  1rootroot616MAug  313:27CentOS-6.5-x86_64-minimal-freewaf-1.2.1-release_21152.iso

-rw-r--r--.1rootroot3.1KJul3113:53haproxy.cfg

-rw-r--r--.1rootroot8.4KJul3110:48install.log

-rw-r--r--.1rootroot3.4KJul3110:47install.log.syslog

-rw-r--r--  1rootroot  51MJun1906:05Ipart.apk

-rw-r--r--  1rootroot137MMar2214:47lmnp.tar.gz

-rw-r--r--  1rootroot  22MNov13  2013Percona-Server-5.5.31-rel30.3.tar.gz

drwxr-xr-x  2rootroot4.0KAug  715:27test

 

[root@IP-1-63~]# ll -h test/

total3.5M

-rw-r--r--1rootroot3.5MAug  715:26Atlas-2.1.el6.x86_64.rpm

-rw-r--r--1rootroot3.1KAug  715:27haproxy.cfg

 

[root@IP-1-63~]# ll  -h /data/

total616M

-rw-r--r--  1rootroot616MAug  717:08CentOS-6.5-x86_64-minimal-freewaf-1.2.1-release_21152.iso

drwx------.2rootroot  16KJul3110:45lost+found

对比结果大小相同

原文：http://www.0550go.com/cluster/tfs/tfs-file-system.html