shell命令之uniq

uniq，顾名思义是去重，但需注意，它统计的是相邻两行是否重复，因此很多时候会配合sort。
常用参数如下：

-u   只显示不重复的行-c   每行开头显示重复行出现的次数，重复行显示一次-d   只显示重复行-i   对大小写不敏感-f   跳过n个域-s   跳过n个字符

举例开始，文件file.txt如下：

this is a test  this is a test  this is a test  i am tank  i love tank  i love tank  this is a test  whom have a try  WhoM have a try  you  have a try  i want to abroad  those are good men  we are good men  

1、不加参数的uniq

uniq file.txtthis is a testi am tanki love tankthis is a testwhom have a tryWhoM have a tryyou  have a tryi want to abroadthose are good menwe are good men

2、显示不重复的行

uniq -u Desktop/testi am tankthis is a testwhom have a tryWhoM have a tryyou  have a tryi want to abroadthose are good menwe are good men

3、统计行的次数

uniq -c file.txt   3 this is a test   1 i am tank   2 i love tank   1 this is a test   1 whom have a try   1 WhoM have a try   1 you  have a try   1 i want to abroad   1 those are good men   1 we are good men

4、忽略大小写，并统计重复行次数

uniq -i -c file.txt   3 this is a test   1 i am tank   2 i love tank   1 this is a test   2 whom have a try    ／／注意这里和上面例子的区别   1 you  have a try   1 i want to abroad   1 those are good men   1 we are good men

5、显示重复行

uniq -d file.txtthis is a testi love tank

6、跳过n个域

uniq -f 1 -c file.txt   3 this is a test   1 i am tank   2 i love tank   1 this is a test   2 whom have a try   1 you  have a try   1 i want to abroad   2 those are good men

这里跳过第一个域，所以1至3行跳过this，4至6行跳过i，当whom和whoM被跳过时，这两行被当作一样的，至于接下来的一行，跳过you后，还剩2个空格，和上面两行并不一样，这里需要注意。

7、跳过n个字符

uniq -s 4 -c file.txt   3 this is a test   1 i am tank   2 i love tank   1 this is a test   3 whom have a try   1 i want to abroad   1 those are good men   1 we are good men

这里跳过前4个字符，所以和上面的例子类似，但是此时whoM have a try 和 you have a try就是一样的了。