文件对比

文件差异、相似度对比，经常用到，在此做一个总结

Google: Linux md5 compare sub directories

md5sum

find./rips-0.54/ -type f -print0 | xargs -0 md5sum
md5sum -cfile.md5
md5sum -c file.md5 | grep FAILED$ >failed_hashes
md5sum -c file.md5 | grep -v OK$ >failed_hashes

md5deep

http://sourceforge.net/projects/md5deep/

ssdeep

$ cd/data/src/
$ wgethttp://downloads.sourceforge.net/project/ssdeep/ssdeep-2.10/ssdeep-2.10.tar.gz
$tar xzvf ssdeep-2.9.tar.gz
$ cd ssdeep-2.9/
$ ./configure
$make
$ sudo make install

$ ssdeep-h
ssdeep version 2.10 by Jesse Kornblum
Copyright (C) 2013Facebook

Usage: ssdeep [-m file] [-k file] [-dpgvrsblcxa][-t val] [-h|-V] [FILES]
-m - Match FILES against known hashesin file
-k - Match signatures in FILES against signatures infile
-d - Directory mode, compare all files in a directory
-p- Pretty matching mode. Similar to -d but includes all matches
-g- Cluster matches together
-v - Verbose mode. Displays filenameas its being processed
-r - Recursive mode
-s - Silent mode;all errors are supressed
-b - Uses only the bare name of files;all path information omitted
-l - Uses relative paths forfilenames
-c - Prints output in CSV format
-x - CompareFILES as signature files
-a - Display all matches, regardlessof score
-t - Only displays matches above the giventhreshold
-h - Display this help message
-V - Displayversion number and exit

diff

diff -q -r./rips-0.54/ ./rips-0.541/

rsync

rsync -lrthvcn--delete /home/source/dir /home/destination/dir

GREP

root@gnu:~#cat 11.txt
1
2
3
a
c
root@gnu:~# cat 22.txt
a
b
c
2
3
root@gnu:~# grep -Fxvf 11.txt 22.txt
b
root@gnu:~# grep -Fxvf 22.txt 11.txt
1

comm -13<(sort file1) <(sort file2) > file
diff file1 file2 >file3
join -v 2 <(sort file1) <(sort file2) > file3

参考

http://www.linuxquestions.org/questions/linux-software-2/how-to-create-md5sum-for-a-directory-689242/
http://stackoverflow.com/questions/9473552/how-to-compare-two-directories-subdirectories-while-ignoring-certain-directory-n
http://www.aldeid.com/wiki/Ssdeep