字符串相似度算法 levenshtein distance 编辑距离算法

来源：互联网发布：mac飞秋怎么改ip 编辑：程序博客网时间：2024/04/30 00:04

参考：http://www.merriampark.com/ld.htm#WHATIS
http://en.wikipedia.org/wiki/Levenshtein_distance

题目：一个字符串可以通过增加一个字符，删除一个字符，替换一个字符得到另外一个字符串，假设，我们把从字符串A转换成字符串B，前面3种操作所执行的最少次数称为AB相似度
如   abc adc   度为 1
   ababababa babababab 度为 2
   abcd acdb 度为2

这个题目中，我们并不在乎两个字符串变得相等之后的字符是怎样的，如果第一个字符是相等的，则计算剩下的就可以了。如果两个字符串的第一个字符不相等，可以进行以下三操作：

1、一步之后，再将A[2,...,lenA]和B[1,...,lenB]变成相同的字符串
2、一步之后，再将A[1,...,lenA]和B[2,...,lenB]变成相同的字符串
3、一步之后，再将A[2,...,lenA]和B[2,...,lenB]变成相同的字符串

这样就可以有以下递归程序：

int caculateStringDistance(string firstStr, int firstBegin,int firstEnd, string secondStr, int secondBegin, int secondEnd)
{

            if (firstBegin > firstEnd)
            {
                if (secondBegin > secondEnd)
                    return 0;
                else
                    return secondEnd - secondBegin + 1;
            }
            if (secondBegin > secondEnd)
            {
               if (firstBegin > firstEnd)
                    return 0;
               else
                  return firstEnd - firstBegin + 1;
            }
           if (firstStr[firstBegin] == secondStr[secondBegin])
           {
                return caculateStringDistance(firstStr, firstBegin + 1,
                        firstEnd, secondStr, secondBegin + 1, secondEnd);
           }
           else
           {
                int oneValue = caculateStringDistance(firstStr, firstBegin + 1,
                        firstEnd, secondStr, secondBegin + 2, secondEnd);
                int twoValue = caculateStringDistance(firstStr, firstBegin + 2,
                        firstEnd, secondStr, secondBegin + 1, secondEnd);
                int threeValue = caculateStringDistance(firstStr,
                        firstBegin + 2, firstEnd, secondStr, secondBegin + 2,secondEnd);
                return min(oneValue, twoValue, threeValue) + 1;
            }
}

字符串相似度算法可以使用 Levenshtein Distance算法(中文翻译：编辑距离算法) 这算法是由俄国科学家Levenshtein提出的。其步骤

StepDescription1Set n to be the length of s.
Set m to be the length of t.
If n = 0, return m and exit.
If m = 0, return n and exit.
Construct a matrix containing 0..m rows and 0..n columns.2Initialize the first row to 0..n.
Initialize the first column to 0..m.3Examine each character of s (i from 1 to n).4Examine each character of t (j from 1 to m).5If s[i] equals t[j], the cost is 0.
If s[i] doesn't equal t[j], the cost is 1.6Set cell d[i,j] of the matrix equal to the minimum of:
a. The cell immediately above plus 1: d[i-1,j] + 1.
b. The cell immediately to the left plus 1: d[i,j-1] + 1.
c. The cell diagonally above and to the left plus the cost: d[i-1,j-1] + cost.7After the iteration steps (3, 4, 5, 6) are complete, the distance is found in cell d[n,m].

    * Java
    * C++
    * Visual Basic
    * Python

Java代码

Java
public class Distance {
//****************************
// Get minimum of three values
//****************************
private int Minimum (int a, int b, int c) {
int mi;
mi = a;
if (b < mi) {
mi = b;
}
if (c < mi) {
mi = c;
}
return mi;
}
//*****************************
// Compute Levenshtein distance
//*****************************
public int LD (String s, String t) {
int d[][]; // matrix
int n; // length of s
int m; // length of t
int i; // iterates through s
int j; // iterates through t
char s_i; // ith character of s
char t_j; // jth character of t
int cost; // cost
// Step 1
n = s.length ();
m = t.length ();
if (n == 0) {
return m;
}
if (m == 0) {
return n;
}
d = new int[n+1][m+1];
// Step 2
for (i = 0; i <= n; i++) {
d[i][0] = i;
}
for (j = 0; j <= m; j++) {
d[0][j] = j;
}
// Step 3
for (i = 1; i <= n; i++) {
s_i = s.charAt (i - 1);
// Step 4
for (j = 1; j <= m; j++) {
t_j = t.charAt (j - 1);
// Step 5
if (s_i == t_j) {
cost = 0;
}
else {
cost = 1;
}
// Step 6
d[i][j] = Minimum (d[i-1][j]+1, d[i][j-1]+1, d[i-1][j-1] + cost);
}
}
// Step 7
return d[n][m];
}
}
C++
In C++, the size of an array must be a constant, and this code fragment causes an error at compile time:
int sz = 5;
int arr[sz];
This limitation makes the following C++ code slightly more complicated than it would be if the matrix could simply be declared as a two-dimensional array, with a size determined at run-time.
In C++ it's more idiomatic to use the System Template Library's vector class, as Anders Sewerin Johansen has done in an alternative C++ implementation.
Here is the definition of the class (distance.h):
class Distance
{
public:
int LD (char const *s, char const *t);
private:
int Minimum (int a, int b, int c);
int *GetCellPointer (int *pOrigin, int col, int row, int nCols);
int GetAt (int *pOrigin, int col, int row, int nCols);
void PutAt (int *pOrigin, int col, int row, int nCols, int x);
};
Here is the implementation of the class (distance.cpp):
#include "distance.h"
#include <string.h>
#include <malloc.h>
//****************************
// Get minimum of three values
//****************************
int Distance::Minimum (int a, int b, int c)
{
int mi;
mi = a;
if (b < mi) {
mi = b;
}
if (c < mi) {
mi = c;
}
return mi;
}
//**************************************************
// Get a pointer to the specified cell of the matrix
//**************************************************
int *Distance::GetCellPointer (int *pOrigin, int col, int row, int nCols)
{
return pOrigin + col + (row * (nCols + 1));
}
//*****************************************************
// Get the contents of the specified cell in the matrix
//*****************************************************
int Distance::GetAt (int *pOrigin, int col, int row, int nCols)
{
int *pCell;
pCell = GetCellPointer (pOrigin, col, row, nCols);
return *pCell;
}
//*******************************************************
// Fill the specified cell in the matrix with the value x
//*******************************************************
void Distance::PutAt (int *pOrigin, int col, int row, int nCols, int x)
{
int *pCell;
pCell = GetCellPointer (pOrigin, col, row, nCols);
*pCell = x;
}
//*****************************
// Compute Levenshtein distance
//*****************************
int Distance::LD (char const *s, char const *t)
{
int *d; // pointer to matrix
int n; // length of s
int m; // length of t
int i; // iterates through s
int j; // iterates through t
char s_i; // ith character of s
char t_j; // jth character of t
int cost; // cost
int result; // result
int cell; // contents of target cell
int above; // contents of cell immediately above
int left; // contents of cell immediately to left
int diag; // contents of cell immediately above and to left
int sz; // number of cells in matrix
// Step 1
n = strlen (s);
m = strlen (t);
if (n == 0) {
return m;
}
if (m == 0) {
return n;
}
sz = (n+1) * (m+1) * sizeof (int);
d = (int *) malloc (sz);
// Step 2
for (i = 0; i <= n; i++) {
PutAt (d, i, 0, n, i);
}
for (j = 0; j <= m; j++) {
PutAt (d, 0, j, n, j);
}
// Step 3
for (i = 1; i <= n; i++) {
s_i = s[i-1];
// Step 4
for (j = 1; j <= m; j++) {
t_j = t[j-1];
// Step 5
if (s_i == t_j) {
cost = 0;
}
else {
cost = 1;
}
// Step 6
above = GetAt (d,i-1,j, n);
left = GetAt (d,i, j-1, n);
diag = GetAt (d, i-1,j-1, n);
cell = Minimum (above + 1, left + 1, diag + cost);
PutAt (d, i, j, n, cell);
}
}
// Step 7
result = GetAt (d, n, m, n);
free (d);
return result;
}
Visual Basic
'*******************************
'*** Get minimum of three values
'*******************************
Private Function Minimum(ByVal a As Integer, _
ByVal b As Integer, _
ByVal c As Integer) As Integer
Dim mi As Integer
mi = a
If b < mi Then
mi = b
End If
If c < mi Then
mi = c
End If
Minimum = mi
End Function
'********************************
'*** Compute Levenshtein Distance
'********************************
Public Function LD(ByVal s As String, ByVal t As String) As Integer
Dim d() As Integer ' matrix
Dim m As Integer ' length of t
Dim n As Integer ' length of s
Dim i As Integer ' iterates through s
Dim j As Integer ' iterates through t
Dim s_i As String ' ith character of s
Dim t_j As String ' jth character of t
Dim cost As Integer ' cost
' Step 1
n = Len(s)
m = Len(t)
If n = 0 Then
LD = m
Exit Function
End If
If m = 0 Then
LD = n
Exit Function
End If
ReDim d(0 To n, 0 To m) As Integer
' Step 2
For i = 0 To n
d(i, 0) = i
Next i
For j = 0 To m
d(0, j) = j
Next j
' Step 3
For i = 1 To n
s_i = Mid$(s, i, 1)
' Step 4
For j = 1 To m
t_j = Mid$(t, j, 1)
' Step 5
If s_i = t_j Then
cost = 0
Else
cost = 1
End If
' Step 6
d(i, j) = Minimum(d(i - 1, j) + 1, d(i, j - 1) + 1, d(i - 1, j - 1) + cost)
Next j
Next i
' Step 7
LD = d(n, m)
Erase d
End Function

Javapublic class Distance {  //****************************  // Get minimum of three values  //****************************  private int Minimum (int a, int b, int c) {  int mi;    mi = a;    if (b < mi) {      mi = b;    }    if (c < mi) {      mi = c;    }    return mi;  }  //*****************************  // Compute Levenshtein distance  //*****************************  public int LD (String s, String t) {  int d[][]; // matrix  int n; // length of s  int m; // length of t  int i; // iterates through s  int j; // iterates through t  char s_i; // ith character of s  char t_j; // jth character of t  int cost; // cost    // Step 1    n = s.length ();    m = t.length ();    if (n == 0) {      return m;    }    if (m == 0) {      return n;    }    d = new int[n+1][m+1];    // Step 2    for (i = 0; i <= n; i++) {      d[i][0] = i;    }    for (j = 0; j <= m; j++) {      d[0][j] = j;    }    // Step 3    for (i = 1; i <= n; i++) {      s_i = s.charAt (i - 1);      // Step 4      for (j = 1; j <= m; j++) {        t_j = t.charAt (j - 1);        // Step 5        if (s_i == t_j) {          cost = 0;        }        else {          cost = 1;        }        // Step 6        d[i][j] = Minimum (d[i-1][j]+1, d[i][j-1]+1, d[i-1][j-1] + cost);      }    }    // Step 7    return d[n][m];  }}C++In C++, the size of an array must be a constant, and this code fragment causes an error at compile time:int sz = 5;int arr[sz];This limitation makes the following C++ code slightly more complicated than it would be if the matrix could simply be declared as a two-dimensional array, with a size determined at run-time.In C++ it's more idiomatic to use the System Template Library's vector class, as Anders Sewerin Johansen has done in an alternative C++ implementation.Here is the definition of the class (distance.h):class Distance{  public:    int LD (char const *s, char const *t);  private:    int Minimum (int a, int b, int c);    int *GetCellPointer (int *pOrigin, int col, int row, int nCols);    int GetAt (int *pOrigin, int col, int row, int nCols);    void PutAt (int *pOrigin, int col, int row, int nCols, int x);}; Here is the implementation of the class (distance.cpp):#include "distance.h"#include <string.h>#include <malloc.h>//****************************// Get minimum of three values//****************************int Distance::Minimum (int a, int b, int c){int mi;  mi = a;  if (b < mi) {    mi = b;  }  if (c < mi) {    mi = c;  }  return mi;}//**************************************************// Get a pointer to the specified cell of the matrix//************************************************** int *Distance::GetCellPointer (int *pOrigin, int col, int row, int nCols){  return pOrigin + col + (row * (nCols + 1));}//*****************************************************// Get the contents of the specified cell in the matrix //*****************************************************int Distance::GetAt (int *pOrigin, int col, int row, int nCols){int *pCell;  pCell = GetCellPointer (pOrigin, col, row, nCols);  return *pCell;}//*******************************************************// Fill the specified cell in the matrix with the value x//*******************************************************void Distance::PutAt (int *pOrigin, int col, int row, int nCols, int x){int *pCell;  pCell = GetCellPointer (pOrigin, col, row, nCols);  *pCell = x;}//*****************************// Compute Levenshtein distance//*****************************int Distance::LD (char const *s, char const *t){int *d; // pointer to matrixint n; // length of sint m; // length of tint i; // iterates through sint j; // iterates through tchar s_i; // ith character of schar t_j; // jth character of tint cost; // costint result; // resultint cell; // contents of target cellint above; // contents of cell immediately aboveint left; // contents of cell immediately to leftint diag; // contents of cell immediately above and to leftint sz; // number of cells in matrix  // Step 1  n = strlen (s);  m = strlen (t);  if (n == 0) {    return m;  }  if (m == 0) {    return n;  }  sz = (n+1) * (m+1) * sizeof (int);  d = (int *) malloc (sz);  // Step 2  for (i = 0; i <= n; i++) {    PutAt (d, i, 0, n, i);  }  for (j = 0; j <= m; j++) {    PutAt (d, 0, j, n, j);  }  // Step 3  for (i = 1; i <= n; i++) {    s_i = s[i-1];    // Step 4    for (j = 1; j <= m; j++) {      t_j = t[j-1];      // Step 5      if (s_i == t_j) {        cost = 0;      }      else {        cost = 1;      }      // Step 6       above = GetAt (d,i-1,j, n);      left = GetAt (d,i, j-1, n);      diag = GetAt (d, i-1,j-1, n);      cell = Minimum (above + 1, left + 1, diag + cost);      PutAt (d, i, j, n, cell);    }  }  // Step 7  result = GetAt (d, n, m, n);  free (d);  return result;}Visual Basic'*******************************'*** Get minimum of three values'*******************************Private Function Minimum(ByVal a As Integer, _                         ByVal b As Integer, _                         ByVal c As Integer) As IntegerDim mi As Integer                            mi = a  If b < mi Then    mi = b  End If  If c < mi Then    mi = c  End If    Minimum = mi                          End Function'********************************'*** Compute Levenshtein Distance'********************************Public Function LD(ByVal s As String, ByVal t As String) As IntegerDim d() As Integer ' matrixDim m As Integer ' length of tDim n As Integer ' length of sDim i As Integer ' iterates through sDim j As Integer ' iterates through tDim s_i As String ' ith character of sDim t_j As String ' jth character of tDim cost As Integer ' cost    ' Step 1    n = Len(s)  m = Len(t)  If n = 0 Then    LD = m    Exit Function  End If   If m = 0 Then    LD = n    Exit Function  End If   ReDim d(0 To n, 0 To m) As Integer    ' Step 2    For i = 0 To n    d(i, 0) = i  Next i    For j = 0 To m    d(0, j) = j  Next j  ' Step 3  For i = 1 To n        s_i = Mid$(s, i, 1)        ' Step 4        For j = 1 To m            t_j = Mid$(t, j, 1)            ' Step 5            If s_i = t_j Then        cost = 0      Else        cost = 1      End If            ' Step 6            d(i, j) = Minimum(d(i - 1, j) + 1, d(i, j - 1) + 1, d(i - 1, j - 1) + cost)        Next j      Next i    ' Step 7    LD = d(n, m)  Erase dEnd Function

Python代码

#!/user/bin/env python
# -*- coding: utf-8 -*-
class arithmetic():
def __init__(self):
pass
''''' 【编辑距离算法】【levenshtein distance】【字符串相似度算法】 '''
def levenshtein(self,first,second):
if len(first) > len(second):
first,second = second,first
if len(first) == 0:
return len(second)
if len(second) == 0:
return len(first)
first_length = len(first) + 1
second_length = len(second) + 1
distance_matrix = [range(second_length) for x in range(first_length)]
#print distance_matrix
for i in range(1,first_length):
for j in range(1,second_length):
deletion = distance_matrix[i-1][j] + 1
insertion = distance_matrix[i][j-1] + 1
substitution = distance_matrix[i-1][j-1]
if first[i-1] != second[j-1]:
substitution += 1
distance_matrix[i][j] = min(insertion,deletion,substitution)
print distance_matrix
return distance_matrix[first_length-1][second_length-1]
if __name__ == "__main__":
arith = arithmetic()
print arith.levenshtein('GUMBOsdafsadfdsafsafsadfasfadsfasdfasdfs','GAMBOL00000000000dfasfasfdafsafasfasdfdsa'