python相似性检测的安装包

安装python-Levenshtein模块

pip install python-Levenshtein

使用python-Levenshtein模块

import Levenshtein

算法说明

1). Levenshtein.hamming(str1, str2)
计算汉明距离。要求str1和str2必须长度一致。是描述两个等长字串之间对应 位置上不同字符的个数。

2). Levenshtein.distance(str1, str2)
计算编辑距离（也称为 Levenshtein距离）。是描述由一个字串转化成另一个字串最少的操作次数，在其中的操作包括插入、删除、替换。
算法实现参考动态规划整理。

3). Levenshtein.ratio(str1, str2)
计算莱文斯坦比。计算公式r = (sum - ldist) / sum, 其中sum是指str1 和 str2 字串的长度总和，ldist是 类编辑距离
注意 ：这里的类编辑距离不是2中所说的编辑距离，2中三种操作中每个操作+1，而在此处，删除、插入依然+1，但是替换+2
这样设计的目的：ratio('a', 'c')，sum=2, 按2中计算为（2-1）/2 = 0.5,’a','c'没有重合，显然不合算，但是替换操作+2，就可以解决这个问题。

4). Levenshtein.jaro(s1 , s2 )
计算jaro距离，

其中的 m 为s1 , s2的匹配长度，当某位置的认为匹配当该位置字符相同，或者在不超过

t是调换次数的一半

5.) Levenshtein.jaro_winkler(s 1 , s 2 )
计算 Jaro–Winkler距离：

import Levenshtein 报错：ImportError: No module named Levenshtein
于是去： python-Levenshtein 下载源码进行安装（在 http://www.lfd.uci.edu/~gohlke/pythonlibs/#python-levenshtein其实也有编译好的exe），第一次安装的时候报错：error: Unable to find vcvarsall.bat ，但其实我是装了VS2010的，所以执行如下步骤正常安装：
1.设置环境变量，执行：
SET VS90COMNTOOLS=%VS100COMNTOOLS%
2.再去安装：
setup.py install
就可以正常，编译，安装了。
$ python>>> import Levenshtein>>> help(Levenshtein.ratio)ratio(...)    Compute similarity of two strings.    ratio(string1, string2)    The similarity is a number between 0 and 1, it's usually equal or    somewhat higher than difflib.SequenceMatcher.ratio(), becuase it's    based on real minimal edit distance.    Examples:    >>> ratio('Hello world!', 'Holly grail!')    0.58333333333333337    >>> ratio('Brian', 'Jesus')    0.0>>> help(Levenshtein.distance)distance(...)    Compute absolute Levenshtein distance of two strings.    distance(string1, string2)    Examples (it's hard to spell Levenshtein correctly):    >>> distance('Levenshtein', 'Lenvinsten')    4    >>> distance('Levenshtein', 'Levensthein')    2    >>> distance('Levenshtein', 'Levenshten')    1    >>> distance('Levenshtein', 'Levenshtein')    0

difflib 库
>>> import difflib>>> difflib.SequenceMatcher(None, 'abcde', 'abcde').ratio()1.0>>> difflib.SequenceMatcher(None, 'abcde', 'zbcde').ratio()0.80000000000000004>>> difflib.SequenceMatcher(None, 'abcde', 'zyzzy').ratio()0.0
 
FuzzyWuzzy
git clone git://github.com/seatgeek/fuzzywuzzy.git fuzzywuzzycd fuzzywuzzypython setup.py install>>> from fuzzywuzzy import fuzz>>> from fuzzywuzzy import processSimple Ratio>>> fuzz.ratio("this is a test", "this is a test!")    96Partial Ratio>>> fuzz.partial_ratio("this is a test", "this is a test!")    100Token Sort Ratio>>> fuzz.ratio("fuzzy wuzzy was a bear", "wuzzy fuzzy was a bear")    90>>> fuzz.token_sort_ratio("fuzzy wuzzy was a bear", "wuzzy fuzzy was a bear")    100Token Set Ratio>>> fuzz.token_sort_ratio("fuzzy was a bear", "fuzzy fuzzy was a bear")    84>>> fuzz.token_set_ratio("fuzzy was a bear", "fuzzy fuzzy was a bear")    100
gitclone git://github.com/seatgeek/fuzzywuzzy.git fuzzywuzzycdfuzzywuzzypythonsetup.pyinstall >>> fromfuzzywuzzyimportfuzz>>> fromfuzzywuzzyimportprocess SimpleRatio>>> fuzz.ratio("this is a test", "this is a test!")    96 PartialRatio>>> fuzz.partial_ratio("this is a test", "this is a test!")    100 TokenSortRatio>>> fuzz.ratio("fuzzy wuzzy was a bear", "wuzzy fuzzy was a bear")    90>>> fuzz.token_sort_ratio("fuzzy wuzzy was a bear", "wuzzy fuzzy was a bear")    100 TokenSetRatio>>> fuzz.token_sort_ratio("fuzzy was a bear", "fuzzy fuzzy was a bear")    84>>> fuzz.token_set_ratio("fuzzy was a bear", "fuzzy fuzzy was a bear")    100
google-diff-match-patch
import diff match patch textA = "the cat in the red hat" textB = "the feline in the blue hat"
dmp = diff match patch.diff match patch()  #create a diff match patch object diffs = dmp.diff main(textA, textB)   # All 'diff' jobs start with invoking diff main()
d value = dmp.diff levenshtein(diffs) print d_value
maxLenth = max(len(textA), len(textB)) print float(d_value)/float(maxLenth)
similarity = (1 - float(d_value)/float(maxLenth)) * 100 print similarity
importdiff_match_patchtextA = "the cat in the red hat"textB = "the feline in the blue hat" dmp = diff_match_patch.diff_match_patch()  #create a diff_match_patch objectdiffs = dmp.diff_main(textA, textB)   # All 'diff' jobs start with invoking diff_main() d_value = dmp.diff_levenshtein(diffs)printd_value maxLenth = max(len(textA), len(textB))printfloat(d_value)/float(maxLenth) similarity = (1 - float(d_value)/float(maxLenth)) * 100printsimilarity