hash函数的简单介绍

来源：互联网发布：阿里云logo矢量编辑：程序博客网时间：2024/05/16 08:41

HASH函数

应用Hash函数

作者：冲处宇宙

时间：2007.1.25

计算理论中，没有Hash函数的说法，只有单向函数的说法。所谓的单向函数，是一个复杂的定义，大家可以去看计算理论或者密码学方面的数据。用“人类”的语言描述单向函数就是：如果某个函数在给定输入的时候，很容易计算出其结果来；而当给定结果的时候，很难计算出输入来，这就是单项函数。各种加密函数都可以被认为是单向函数的逼近。Hash函数（或者成为散列函数）也可以看成是单向函数的一个逼近。即它接近于满足单向函数的定义。

Hash函数还有另外的含义。实际中的Hash函数是指把一个大范围映射到一个小范围。把大范围映射到一个小范围的目的往往是为了节省空间，使得数据容易保存。除此以外，Hash函数往往应用于查找上。所以，在考虑使用Hash函数之前，需要明白它的几个限制：

1. Hash的主要原理就是把大范围映射到小范围；所以，你输入的实际值的个数必须和小范围相当或者比它更小。不然冲突就会很多。
2. 由于Hash逼近单向函数；所以，你可以用它来对数据进行加密。
3. 不同的应用对Hash函数有着不同的要求；比如，用于加密的Hash函数主要考虑它和单项函数的差距，而用于查找的Hash函数主要考虑它映射到小范围的冲突率。

应用于加密的Hash函数已经探讨过太多了，在作者的博客里面有更详细的介绍。所以，本文只探讨用于查找的Hash函数。

Hash函数应用的主要对象是数组（比如，字符串），而其目标一般是一个int类型。以下我们都按照这种方式来说明。

一般的说，Hash函数可以简单的划分为如下几类：
1. 加法Hash；
2. 位运算Hash；
3. 乘法Hash；
4. 除法Hash；
5. 查表Hash；
6. 混合Hash；
下面详细的介绍以上各种方式在实际中的运用。
一加法Hash

所谓的加法Hash就是把输入元素一个一个的加起来构成最后的结果。标准的加法Hash的构造如下：

static int additiveHash(String key, int prime)
{
int hash, i;
for (hash = key.length(), i = 0; i < key.length(); i++)
hash += key.charAt(i);
return (hash % prime);
}
这里的prime是任意的质数，看得出，结果的值域为[0,prime-1]。
二位运算Hash

这类型Hash函数通过利用各种位运算（常见的是移位和异或）来充分的混合输入元素。比如，标准的旋转Hash的构造如下：

static int rotatingHash(String key, int prime)
{
int hash, i;
for (hash=key.length(), i=0; i<key.length(); ++i)
hash = (hash<<4)^(hash>>28)^key.charAt(i);
return (hash % prime);
}

先移位，然后再进行各种位运算是这种类型Hash函数的主要特点。比如，以上的那段计算hash的代码还可以有如下几种变形：
1. hash = (hash<<5)^(hash>>27)^key.charAt(i);
2. hash += key.charAt(i);
hash += (hash << 10);
hash ^= (hash >> 6);
3. if((i&1) == 0)
{
hash ^= (hash<<7) ^ key.charAt(i) ^ (hash>>3);
}
else
{
hash ^= ~((hash<<11) ^ key.charAt(i) ^ (hash >>5));
}
4. hash += (hash<<5) + key.charAt(i);
5. hash = key.charAt(i) + (hash<<6) + (hash>>16) – hash;
6. hash ^= ((hash<<5) + key.charAt(i) + (hash>>2));
三乘法Hash

这种类型的Hash函数利用了乘法的不相关性（乘法的这种性质，最有名的莫过于平方取头尾的随机数生成算法，虽然这种算法效果并不好）。比如，

static int bernstein(String key)
{
int hash = 0;
int i;
for (i=0; i<key.length(); ++i) hash = 33*hash + key.charAt(i);
return hash;
}

jdk5.0里面的String类的hashCode()方法也使用乘法Hash。不过，它使用的乘数是31。推荐的乘数还有：131, 1313, 13131, 131313等等。

使用这种方式的著名Hash函数还有：
// 32位FNV算法
int M_SHIFT = 0;
public int FNVHash(byte[] data)
{
int hash = (int)2166136261L;
for(byte b : data)
hash = (hash * 16777619) ^ b;
if (M_SHIFT == 0)
return hash;
return (hash ^ (hash >> M_SHIFT)) & M_MASK;
}

以及改进的FNV算法：
public static int FNVHash1(String data)
{
final int p = 16777619;
int hash = (int)2166136261L;
for(int i=0;i<data.length();i++)
hash = (hash ^ data.charAt(i)) * p;
hash += hash << 13;
hash ^= hash >> 7;
hash += hash << 3;
hash ^= hash >> 17;
hash += hash << 5;
return hash;
}

除了乘以一个固定的数，常见的还有乘以一个不断改变的数，比如：
static int RSHash(String str)
{
int b = 378551;
int a = 63689;
int hash = 0;

for(int i = 0; i < str.length(); i++)
{
hash = hash * a + str.charAt(i);
a = a * b;
}
return (hash & 0x7FFFFFFF);
}

虽然Adler32算法的应用没有CRC32广泛，不过，它可能是乘法Hash里面最有名的一个了。关于它的介绍，大家可以去看RFC 1950规范。
四除法Hash

除法和乘法一样，同样具有表面上看起来的不相关性。不过，因为除法太慢，这种方式几乎找不到真正的应用。需要注意的是，我们在前面看到的hash的结果除以一个prime的目的只是为了保证结果的范围。如果你不需要它限制一个范围的话，可以使用如下的代码替代”hash%prime”： hash = hash ^ (hash>>10) ^ (hash>>20)。
五查表Hash

查表Hash最有名的例子莫过于CRC系列算法。虽然CRC系列算法本身并不是查表，但是，查表是它的一种最快的实现方式。下面是CRC32的实现：

static int crctab[256] = {
0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419, 0x706af48f, 0xe963a535, 0x9e6495a3, 0x0edb8832, 0x79dcb8a4, 0xe0d5e91e, 0x97d2d988, 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, 0x90bf1d91, 0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de, 0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7, 0x136c9856, 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9, 0xfa0f3d63, 0x8d080df5, 0x3b6e20c8, 0x4c69105e, 0xd56041e4, 0xa2677172, 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b, 0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940, 0x32d86ce3, 0x45df5c75, 0xdcd60dcf, 0xabd13d59, 0x26d930ac, 0x51de003a, 0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423, 0xcfba9599, 0xb8bda50f, 0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924, 0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d, 0x76dc4190, 0x01db7106, 0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f, 0x9fbfe4a5, 0xe8b8d433, 0x7807c9a2, 0x0f00f934, 0x9609a88e, 0xe10e9818, 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01, 0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e, 0x6c0695ed, 0x1b01a57b, 0x8208f4c1, 0xf50fc457, 0x65b0d9c6, 0x12b7e950, 0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, 0xfbd44c65, 0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2, 0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb, 0x4369e96a, 0x346ed9fc, 0xad678846, 0xda60b8d0, 0x44042d73, 0x33031de5, 0xaa0a4c5f, 0xdd0d7cc9, 0x5005713c, 0x270241aa, 0xbe0b1010, 0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f, 0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17, 0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad, 0xedb88320, 0x9abfb3b6, 0x03b6e20c, 0x74b1d29a, 0xead54739, 0x9dd277af, 0x04db2615, 0x73dc1683, 0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8, 0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1, 0xf00f9344, 0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb,
0x196c3671, 0x6e6b06e7, 0xfed41b76, 0x89d32be0, 0x10da7a5a, 0x67dd4acc, 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5, 0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252, 0xd1bb67f1, 0xa6bc5767, 0x3fb506dd, 0x48b2364b, 0xd80d2bda, 0xaf0a1b4c, 0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55, 0x316e8eef, 0x4669be79, 0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236, 0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f, 0xc5ba3bbe, 0xb2bd0b28, 0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31, 0x2cd99e8b, 0x5bdeae1d, 0x9b64c2b0, 0xec63f226, 0x756aa39c, 0x026d930a, 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713, 0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38, 0x92d28e9b, 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21, 0x86d3d2d4, 0xf1d4e242, 0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1, 0x18b74777, 0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c, 0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45, 0xa00ae278, 0xd70dd2ee, 0x4e048354, 0x3903b3c2, 0xa7672661, 0xd06016f7, 0x4969474d, 0x3e6e77db, 0xaed16a4a, 0xd9d65adc, 0x40df0b66, 0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9, 0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605, 0xcdd70693, 0x54de5729, 0x23d967bf, 0xb3667a2e, 0xc4614ab8, 0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, 0x2d02ef8d
};
int crc32(String key, int hash)
{
int i;
for (hash=key.length(), i=0; i<key.length(); ++i)
hash = (hash >> 8) ^ crctab[(hash & 0xff) ^ k.charAt(i)];
return hash;
}

查表Hash中有名的例子有：Universal Hashing和Zobrist Hashing。他们的表格都是随机生成的。
六混合Hash

混合Hash算法利用了以上各种方式。各种常见的Hash算法，比如MD5、Tiger都属于这个范围。它们一般很少在面向查找的Hash函数里面使用。
七对Hash算法的评价

http://www.burtleburtle.net/bob/hash/doobs.html 这个页面提供了对几种流行Hash算法的评价。我们对Hash函数的建议如下：

1. 字符串的Hash。最简单可以使用基本的乘法Hash，当乘数为33时，对于英文单词有很好的散列效果（小于6个的小写形式可以保证没有冲突）。复杂一点可以使用FNV算法（及其改进形式），它对于比较长的字符串，在速度和效果上都不错。

2. 长数组的Hash。可以使用http://burtleburtle.net/bob/c/lookup3.c这种算法，它一次运算多个字节，速度还算不错。
八后记

本文简略的介绍了一番实际应用中的用于查找的Hash算法。Hash算法除了应用于这个方面以外，另外一个著名的应用是巨型字符串匹配（这时的Hash算法叫做：rolling hash，因为它必须可以滚动的计算）。设计一个真正好的Hash算法并不是一件容易的事情。做为应用来说，选择一个适合的算法是最重要的。

附录 I：（http://www.burtleburtle.net/bob/hash/doobs.html）

Abstract

I offer you a new hash function for hash table lookup that is faster and more thorough than the one you are using now. I also give you a way to verify that it is more thorough.

All the text in this color wasn't in the 1997 Dr Dobbs article. The code given here are all public domain.

The Hash

Over the past two years I've built a general hash function for hash table lookup. Most of the two dozen old hashes I've replaced have had owners who wouldn't accept a new hash unless it was a plug-in replacement for their old hash, and was demonstrably better than the old hash.

These old hashes defined my requirements:

The keys are unaligned variable-length byte arrays.
Sometimes keys are several such arrays.
Sometimes a set of independent hash functions were required.
Average key lengths ranged from 8 bytes to 200 bytes.
Keys might be character strings, numbers, bit-arrays, or weirder things.
Table sizes could be anything, including powers of 2.
The hash must be faster than the old one.
The hash must do a good job.

Without further ado, here's the fastest hash I've been able to design that meets all the requirements. The comments describe how to use it.

Update: I'm leaving the old hash in the text below, but it's obsolete, I have faster and more thorough hashes now. http://burtleburtle.net/bob/c/lookup3.c (2006) is about 2 cycles/byte, works well on 32-bit platforms, and can produce a 32 or 64 bit hash. SpookyHash (2011) is specific to 64-bit platforms, is about 1/3 cycle per byte, and produces a 32, 64, or 128 bit hash.

typedef  unsigned long  int  ub4;   /* unsigned 4-byte quantities */typedef  unsigned       char ub1;   /* unsigned 1-byte quantities */#define hashsize(n) ((ub4)1<<(n))#define hashmask(n) (hashsize(n)-1)/*--------------------------------------------------------------------mix -- mix 3 32-bit values reversibly.For every delta with one or two bits set, and the deltas of all three  high bits or all three low bits, whether the original value of a,b,c  is almost all zero or is uniformly distributed,* If mix() is run forward or backward, at least 32 bits in a,b,c  have at least 1/4 probability of changing.* If mix() is run forward, every bit of c will change between 1/3 and  2/3 of the time.  (Well, 22/100 and 78/100 for some 2-bit deltas.)mix() was built out of 36 single-cycle latency instructions in a   structure that could supported 2x parallelism, like so:      a -= b;       a -= c; x = (c>>13);      b -= c; a ^= x;      b -= a; x = (a<<8);      c -= a; b ^= x;      c -= b; x = (b>>13);      ...  Unfortunately, superscalar Pentiums and Sparcs can't take advantage   of that parallelism.  They've also turned some of those single-cycle  latency instructions into multi-cycle latency instructions.  Still,  this is the fastest good hash I could find.  There were about 2^^68  to choose from.  I only looked at a billion or so.--------------------------------------------------------------------*/#define mix(a,b,c) \{ \  a -= b; a -= c; a ^= (c>>13); \  b -= c; b -= a; b ^= (a<<8); \  c -= a; c -= b; c ^= (b>>13); \  a -= b; a -= c; a ^= (c>>12);  \  b -= c; b -= a; b ^= (a<<16); \  c -= a; c -= b; c ^= (b>>5); \  a -= b; a -= c; a ^= (c>>3);  \  b -= c; b -= a; b ^= (a<<10); \  c -= a; c -= b; c ^= (b>>15); \}/*--------------------------------------------------------------------hash() -- hash a variable-length key into a 32-bit value  k       : the key (the unaligned variable-length array of bytes)  len     : the length of the key, counting by bytes  initval : can be any 4-byte valueReturns a 32-bit value.  Every bit of the key affects every bit ofthe return value.  Every 1-bit and 2-bit delta achieves avalanche.About 6*len+35 instructions.The best hash table sizes are powers of 2.  There is no need to domod a prime (mod is sooo slow!).  If you need less than 32 bits,use a bitmask.  For example, if you need only 10 bits, do  h = (h & hashmask(10));In which case, the hash table should have hashsize(10) elements.If you are hashing n strings (ub1 **)k, do it like this:  for (i=0, h=0; i<n; ++i) h = hash( k[i], len[i], h);By Bob Jenkins, 1996.  bob_jenkins@burtleburtle.net.  You may use thiscode any way you wish, private, educational, or commercial.  It's free.See http://burtleburtle.net/bob/hash/evahash.htmlUse for hash table lookup, or anything where one collision in 2^^32 isacceptable.  Do NOT use for cryptographic purposes.--------------------------------------------------------------------*/ub4 hash( k, length, initval)register ub1 *k;        /* the key */register ub4  length;   /* the length of the key */register ub4  initval;  /* the previous hash, or an arbitrary value */{   register ub4 a,b,c,len;   /* Set up the internal state */   len = length;   a = b = 0x9e3779b9;  /* the golden ratio; an arbitrary value */   c = initval;         /* the previous hash value */   /*---------------------------------------- handle most of the key */   while (len >= 12)   {      a += (k[0] +((ub4)k[1]<<8) +((ub4)k[2]<<16) +((ub4)k[3]<<24));      b += (k[4] +((ub4)k[5]<<8) +((ub4)k[6]<<16) +((ub4)k[7]<<24));      c += (k[8] +((ub4)k[9]<<8) +((ub4)k[10]<<16)+((ub4)k[11]<<24));      mix(a,b,c);      k += 12; len -= 12;   }   /*------------------------------------- handle the last 11 bytes */   c += length;   switch(len)              /* all the case statements fall through */   {   case 11: c+=((ub4)k[10]<<24);   case 10: c+=((ub4)k[9]<<16);   case 9 : c+=((ub4)k[8]<<8);      /* the first byte of c is reserved for the length */   case 8 : b+=((ub4)k[7]<<24);   case 7 : b+=((ub4)k[6]<<16);   case 6 : b+=((ub4)k[5]<<8);   case 5 : b+=k[4];   case 4 : a+=((ub4)k[3]<<24);   case 3 : a+=((ub4)k[2]<<16);   case 2 : a+=((ub4)k[1]<<8);   case 1 : a+=k[0];     /* case 0: nothing left to add */   }   mix(a,b,c);   /*-------------------------------------------- report the result */   return c;}

Most hashes can be modeled like this:

  initialize(internal state)  for (each text block)  {    combine(internal state, text block);    mix(internal state);  }  return postprocess(internal state);

In the new hash, mix() takes 3n of the 6n+35 instructions needed to hash n bytes. Blocks of text are combined with the internal state (a,b,c) by addition. This combining step is the rest of the hash function, consuming the remaining 3n instructions. The only postprocessing is to choose c out of (a,b,c) to be the result.

Three tricks promote speed:

Mixing is done on three 4-byte registers rather than on a 1-byte quantity.
Combining is done on 12-byte blocks, reducing the loop overhead.
The final switch statement combines a variable-length block with the registers a,b,c without a loop.

The golden ratio really is an arbitrary value. Its purpose is to avoid mapping all zeros to all zeros.

The Hash Must Do a Good Job

The most interesting requirement was that the hash must be better than its competition. What does it mean for a hash to be good for hash table lookup?

A good hash function distributes hash values uniformly. If you don't know the keys before choosing the function, the best you can do is map an equal number of possible keys to each hash value. If keys were distributed uniformly, an excellent hash would be to choose the first few bytes of the key and use that as the hash value. Unfortunately, real keys aren't uniformly distributed. Choosing the first few bytes works quite poorly in practice.

The real requirement then is that a good hash function should distribute hash values uniformly for the keys that users actually use.

How do we test that? Let's look at some typical user data. (Since I work at Oracle, I'll use Oracle's standard example: the EMP table.) The EMP table. Is this data uniformly distributed?

EMPNOENAMEJOBMGRHIREDATESALCOMMDEPTNO7369SMITHCLERK790217-DEC-80800 207499ALLENSALESMAN769820-FEB-811600300307521WARDSALESMAN769822-FEB-811250500307566JONESMANAGER783902-APR-812975 207654MARTINSALESMAN789828-SEP-8112501400307698BLAKEMANAGER753901-MAY-812850 307782CLARKMANAGER756609-JUN-812450 107788SCOTTANALYST769819-APR-873000 207839KINGPRESIDENT 17-NOV-815000 107844TURNERSALESMAN769808-SEP-811500 307876ADAMSCLERK778823-MAY-8711000207900JAMESCLERK769803-DEC-81950 307902FORDANALYST756603-DEC-813000 207934MILLERCLERK778223-JAN-821300 10

Consider each horizontal row to be a key. Some patterns appear.

Keys often differ in only a few bits. For example, all the keys are ASCII, so the high bit of every byte is zero.
Keys often consist of substrings arranged in different orders. For example, the MGR of some keys is the EMPNO of others.
Length matters. The only difference between zero and no value at all may be the length of the value. Also, "aa aaa" and "aaa aa" should hash to different values.
Some keys are mostly zero, with only a few bits set. (That pattern doesn't appear in this example, but it's a common pattern.)

Some patterns are easy to handle. If the length is included in the data being hashed, then lengths are not a problem. If the hash does not treat text blocks commutatively, then substrings are not a problem. Strings that are mostly zeros can be tested by listing all strings with only one bit set and checking if that set of strings produces too many collisions.

The remaining pattern is that keys often differ in only a few bits. If a hash allows small sets of input bits to cancel each other out, and the user keys differ in only those bits, then all keys will map to the same handful of hash values.

A common weakness

Usually, when a small set of input bits cancel each other out, it is because those input bits affect only a smaller set of bits in the internal state.

Consider this hash function:

  for (hash=0, i=0; i<hash; ++i)    hash = ((hash<<5)^(hash>>27))^key[i];  return (hash % prime);

This function maps the strings "EXXXXXB" and "AXXXXXC" to the same value. These keys differ in bit 3 of the first byte and bit 1 of the seventh byte. After the seventh bit is combined, any further postprocessing will do no good because the internal states are already the same.

Any time n input bits can only affect m output bits, and n > m, then the 2ⁿ keys that differ in those input bits can only produce 2^m distinct hash values. The same is true if n input bits can only affect m bits of the internal state -- later mixing may make the 2^m results look uniformly distributed, but there will still be only 2^mresults.

The function above has many sets of 2 bits that affect only 1 bit of the internal state. If there are n input bits, there are (n choose 2)=(n*n/2 - n/2) pairs of input bits, only a few of which match weaknesses in the function above. It is a common pattern for keys to differ in only a few bits. If those bits match one of a hash's weaknesses, which is a rare but not negligible event, the hash will do extremely bad. In most cases, though, it will do just fine. (This allows a function to slip through sanity checks, like hashing an English dictionary uniformly, while still frequently bombing on user data.) A weakness

In hashes built of repeated combine-mix steps, this is what usually causes this weakness:

A small number of bits y of one input block are combined, affecting only y bits of the internal state. So far so good.
The mixing step causes those y bits of the internal state to affect only z bits of the internal state.
The next combining step overwrites those bits with z more input bits, cancelling out the first y input bits.

When z is smaller than the number of bits in the output, then y+z input bits have affected only z bits of the internal state, causing 2^y+zpossible keys to produce at most 2^z hash values.

The same thing can happen in reverse:

Uncombine this block, causing y block bits to unaffect y bits of the internal state.
Unmix the internal state, leaving x bits unaffected by the y bits from this block.
Unmixing the previous block unaffects those x bits, cancelling out this block's y bits.

If x is less than the number of bits in the output, then the 2^x+y keys differing in only those x+y input bits can produce at most 2^x hash values.

(If the mixing function is not a permutation of the internal state, it is not reversible. Instead, it loses information about the earlier blocks every time it is applied, so keys differing only in the first few input blocks are more likely to collide. The mixing function ought to be a permutation.)

It is easy to test whether this weakness exists: if the mixing step causes any bit of the internal state to affect fewer bits of the internal state than there are output bits, the weakness exists. This test should be run on the reverse of the mixing function as well. It can also be run with all sets of 2 internal state bits, or all sets of 3.

Another way this weakness can happen is if any bit in the final input block does not affect every bit of the output. (The user might choose to use only the unaffected output bit, then that's 1 input bit that affects 0 output bits.)

A Survey of Hash Functions

We now have a new hash function and some theory for evaluating hash functions. Let's see how various hash functions stack up.

Additive Hash

ub4 additive(char *key, ub4 len, ub4 prime){  ub4 hash, i;  for (hash=len, i=0; i<len; ++i)     hash += key[i];  return (hash % prime);}

This takes 5n+3 instructions. There is no mixing step. The combining step handles one byte at a time. Input bytes commute. The table length must be prime, and can't be much bigger than one byte because the value of variable hash is never much bigger than one byte.

Rotating Hash

ub4 rotating(char *key, ub4 len, ub4 prime){  ub4 hash, i;  for (hash=len, i=0; i<len; ++i)    hash = (hash<<4)^(hash>>28)^key[i];  return (hash % prime);}

This takes 8n+3 instructions. This is the same as the additive hash, except it has a mixing step (a circular shift by 4) and the combining step is exclusive-or instead of addition. The table size is a prime, but the prime can be any size. On machines with a rotate (such as the Intel x86 line) this is 6n+2 instructions. I have seen the (hash % prime) replaced with

  hash = (hash ^ (hash>>10) ^ (hash>>20)) & mask;

eliminating the % and allowing the table size to be a power of 2, making this 6n+6 instructions. % can be very slow, for example it is 230 times slower than addition on a Sparc 20.

One-at-a-Time Hash

ub4 one_at_a_time(char *key, ub4 len){  ub4   hash, i;  for (hash=0, i=0; i<len; ++i)  {    hash += key[i];    hash += (hash << 10);    hash ^= (hash >> 6);  }  hash += (hash << 3);  hash ^= (hash >> 11);  hash += (hash << 15);  return (hash & mask);}

This is similar to the rotating hash, but it actually mixes the internal state. It takes 9n+9 instructions and produces a full 4-byte result. Preliminary analysis suggests there are no funnels.

This hash was not in the original Dr. Dobb's article. I implemented it to fill a set of requirements posed by Colin Plumb. Colin ended up using an even simpler (and weaker) hash that was sufficient for his purpose.

Bernstein's hash

ub4 bernstein(ub1 *key, ub4 len, ub4 level){  ub4 hash = level;  ub4 i;  for (i=0; i<len; ++i) hash = 33*hash + key[i];  return hash;}

If your keys are lowercase English words, this will fit 6 characters into a 32-bit hash with no collisions (you'd have to compare all 32 bits). If your keys are mixed case English words, 65*hash+key[i] fits 5 characters into a 32-bit hash with no collisions. That means this type of hash can produce (for the right type of keys) fewer collisions than a hash that gives a more truly random distribution. If your platform doesn't have fast multiplies, no sweat, 33*hash = hash+(hash<<5) and most compilers will figure that out for you.

On the down side, if you don't have short text keys, this hash has a easily detectable flaws. For example, there's a 3-into-2 funnel that 0x0021 and 0x0100 both have the same hash (hex 0x21, decimal 33) (you saw that one coming, yes?).

FNV Hash

I need to fill this in. See it on wikipedia. It's faster than lookup3 on Intel (because Intel has fast multiplication), but slower on most other platforms. Preliminary tests suggested it has decent distributions. It's public domain.

I have three complaints against it. First, it's specific about how to reduce the size if you don't use all the bits, it's not just a mask. Increasing the result size by one bit gives you a completely different hash. If you use a hash table that grows by increasing the result size by one bit, one old bucket maps across the entire new table, not to just two new buckets. If your algorithm has a sliding pointer for which buckets have been split, that just won't work with FNV. Second, it's linear. That means that widely separated things can cancel each other out at least as easily as nearby things. Third, since multiplication only affects higher bits, the lowest 7 bits of the state are never affected by the 8th bit of all the input bytes.

On the plus side, very nearby things never cancel each other out at all. This makes FNV a good choice for hashing very short keys (like single English words). FNV is more robust than Bernstein's hash. It can handle any byte values, not just ASCII characters.

ub4 fnv(){  /* I need to fill this in */}

Goulburn Hash

The Goulburn hash is like my one-at-a-time, but more thorough and slower for all lengths beyond 0, asymptotically over 2x slower. It has two tables, g_table0 and g_table1, of respectively 256 and 128 4-byte integers.

For large hash tables (which is where being more thorough ought to buy you something), it does worse, because its internal operations not reversible. Specifically h^=rotate(h,3) and h^=rotate(h,14), which each cause an even number of bits to be set. I hashed the 2³² 32-bit integers with lookup3, one-at-a-time, and goulburn, and they produced 2,696,784,567, and 1,667,635,157, and 897,563,758 values respectively. The expected number for a random mapping would be 2,714,937,129 values.

u4 goulburn( const unsigned char *cp, size_t len, uint32_t last_value){  register u4 h = last_value;  int u;  for( u=0; u<len; ++u ) {    h += g_table0[ cp[u] ];    h ^= (h << 3) ^ (h >> 29);    h += g_table1[ h >> 25 ];    h ^= (h << 14) ^ (h >> 18);    h += 1783936964UL;  }  return h;}

MurmurHash

I need to fill this in too. This is faster than any of my hash, and is more nonlinear than a rotating hash or FNV. I can see it's weaker than my lookup3, but I don't by how much, I haven't tested it.

http://murmurhash.googlepages.com/.

Cessu

The Cessu hash (search for msse2 in his blog) uses SSE2, allowing it to be faster and more thorough (at first glance) than what I can do 32 bits at a time. I haven't done more than a first glance at it.

Pearson's Hash

char pearson(char *key, ub4 len, char tab[256]){  char hash;  ub4  i;  for (hash=len, i=0; i<len; ++i)     hash=tab[hash^key[i]];  return (hash);}

This preinitializes tab[] to an arbitrary permutation of 0 .. 255. It takes 6n+2 instructions, but produces only a 1-byte result. Larger results can be made by running it several times with different initial hash values.

CRC Hashing

ub4 crc(char *key, ub4 len, ub4 mask, ub4 tab[256]){  ub4 hash, i;  for (hash=len, i=0; i<len; ++i)    hash = (hash >> 8) ^ tab[(hash & 0xff) ^ key[i]];  return (hash & mask);}

This takes 9n+3 instructions. tab is initialized to simulate a maximal-length Linear Feedback Shift Register (LFSR) which shifts out the low-order bit and XORs with a polynomial if that bit was 1. I used a 32-bit state with a polynomial of 0xedb88320 for the tests. Keys that differ in only four consecutive bytes will not collide.

A sample implementation, complete with table, is here.

You could also implement it like

ub4 crc(char *key, ub4 len, ub4 mask, ub4 tab[256]){  ub4 hash, i;  for (hash=len, i=0; i<len; ++i)    hash = (hash << 8) ^ tab[(hash >> 24) ^ key[i]];  return (hash & mask);}

but, since shifts are sometimes slow, the other way might be faster. If you did it that way you'd have to reverse the bits of the generating polynomial because bits shift out the top instead of the bottom.

Generalized CRC Hashing

This is exactly the same code as CRC hashing except it fills tab[] with each of the 4 bytes forming a random permutation of 0..255. Unlike a true CRC hash, its mixing is nonlinear. Keys that differ in only one byte will not collide. The top byte has to be a permutation of 0..255 so no information is lost when the low byte is shifted out. The other bytes are permutations of 0..255 only to make hold the guarantee that keys differing in one byte will not collide.

A sample implementation, complete with table, is here.

Universal Hashing

ub4 universal(char *key, ub4 len, ub4 mask, ub4 tab[MAXBITS]){  ub4 hash, i;  for (hash=len, i=0; i<(len<<3); i+=8)  {    register char k = key[i>>3];    if (k&0x01) hash ^= tab[i+0];    if (k&0x02) hash ^= tab[i+1];    if (k&0x04) hash ^= tab[i+2];    if (k&0x08) hash ^= tab[i+3];    if (k&0x10) hash ^= tab[i+4];    if (k&0x20) hash ^= tab[i+5];    if (k&0x40) hash ^= tab[i+6];    if (k&0x80) hash ^= tab[i+7];  }  return (hash & mask);}

This takes 52n+3 instructions. The size of tab[] is the maximum number of input bits. Values in tab[] are chosen at random. Universal hashing can be implemented faster by a Zobrist hash with carefully chosen table values.

Zobrist Hashing

ub4 zobrist( char *key, ub4 len, ub4 mask, ub4 tab[MAXBYTES][256]){  ub4 hash, i;  for (hash=len, i=0; i<len; ++i)    hash ^= tab[i][key[i]];  return (hash & mask);}

This takes 10n+3 instructions. The size of tab[][256] is the maximum number of input bytes. Values of tab[][256] are chosen at random. This can implement universal hashing, but is more general than universal hashing.

Zobrist hashes are especially favored for chess, checkers, othello, and other situations where you have the hash for one state and you want to compute the hash for a closely related state. You xor to the old hash the table values that you're removing from the state, then xor the table values that you're adding. For chess, for example, that's 2 xors to get the hash for the next position given the hash of the current position.

Paul Hsieh's hash

This is kind of a cross between that big hash at the start of this article and my one-at-a-time hash. Paul's timed it and it was than that big hash. It has a 4-byte internal state that it does light nonlinear mixing after every combine. That's good. It combines 2-byte blocks with its 4-byte state, which is something I'd never tried. (FNV and CRC and one-at-a-time combine 1-byte blocks with the 4-byte state. Their input blocks are all smaller than their state, and they mix their state after each input block, which makes it impossible for consecutive input blocks to cancel.)

On the down side, it has funnels of 3 bits into 2, for example hex 01 00 00 00 00 00 00 00 and 00 00 20 00 01 00 00 00 both hash to 0xc754ae23.

#include "pstdint.h" /* Replace with  if appropriate */#undef get16bits#if (defined(__GNUC__) && defined(__i386__)) || defined(__WATCOMC__) \  || defined(_MSC_VER) || defined (__BORLANDC__) || defined (__TURBOC__)#define get16bits(d) (*((const uint16_t *) (d)))#endif#if !defined (get16bits)#define get16bits(d) ((((const uint8_t *)(d))[1] << UINT32_C(8))\                      +((const uint8_t *)(d))[0])#endifuint32_t SuperFastHash (const char * data, int len) {uint32_t hash = len, tmp;int rem;    if (len <= 0 || data == NULL) return 0;    rem = len & 3;    len >>= 2;    /* Main loop */    for (;len > 0; len--) {        hash  += get16bits (data);        tmp    = (get16bits (data+2) << 11) ^ hash;        hash   = (hash << 16) ^ tmp;        data  += 2*sizeof (uint16_t);        hash  += hash >> 11;    }    /* Handle end cases */    switch (rem) {        case 3: hash += get16bits (data);                hash ^= hash << 16;                hash ^= data[sizeof (uint16_t)] << 18;                hash += hash >> 11;                break;        case 2: hash += get16bits (data);                hash ^= hash << 11;                hash += hash >> 17;                break;        case 1: hash += *data;                hash ^= hash << 10;                hash += hash >> 1;    }    /* Force "avalanching" of final 127 bits */    hash ^= hash << 3;    hash += hash >> 5;    hash ^= hash << 4;    hash += hash >> 17;    hash ^= hash << 25;    hash += hash >> 6;    return hash;}

My Hash

This takes 6n+35 instructions. It's the big one at the beginning of the article. It's implemented along with a self-test at http://burtleburtle.net/bob/c/lookup2.c.

lookup3.c

A hash I wrote nine years later designed along the same lines as "My Hash", see http://burtleburtle.net/bob/c/lookup3.c. It takes 2n instructions per byte for mixing instead of 3n. When fitting bytes into registers (the other 3n instructions), it takes advantage of alignment when it can (a trick learned from Paul Hsieh's hash). It doesn't bother to reserve a byte for the length. That allows zero-length strings to require no mixing. More generally, the length that requires additional mixes is now 13-25-37 instead of 12-24-36.

One theoretical insight was that the last mix doesn't need to do well in reverse (though it has to affect all output bits). And the middle mixing steps don't have to affect all output bits (affecting some 32 bits is enough), though it does have to do well in reverse. So it uses different mixes for those two cases. "My Hash" (lookup2.c) had a single mixing operation that had to satisfy both sets of requirements, which is why it was slower.

On a Pentium 4 with gcc 3.4.?, Paul's hash was usually faster than lookup3.c. On a Pentium 4 with gcc 3.2.?, they were about the same speed. On a Pentium 4 with icc -O2, lookup3.c was a little faster than Paul's hash. I don't know how it would play out on other chips and other compilers. lookup3.c is slower than the additive hash pretty much forever, but it's faster than the rotating hash for keys longer than 5 bytes.

lookup3.c does a much more thorough job of mixing than any of my previous hashes (lookup2.c, lookup.c, One-at-a-time). All my previous hashes did a more thorough job of mixing than Paul Hsieh's hash. Paul's hash does a good enough job of mixing for most practical purposes.

The most evil set of keys I know of are sets of keys that are all the same length, with all bytes zero, except with a few bits set. This is tested by frog.c.. To be even more evil, I had my hashes return b and c instead of just c, yielding a 64-bit hash value. Both lookup.c and lookup2.c start seeing collisions after 2⁵³ frog.c keypairs. Paul Hsieh's hash sees collisions after 2¹⁷ keypairs, even if we take two hashes with different seeds. lookup3.c is the only one of the batch that passes this test. It gets its first collision somewhere beyond 2⁶³ keypairs, which is exactly what you'd expect from a completely random mapping to 64-bit values.

MD4

This takes 9.5n+230 instructions. MD4 is a hash designed for cryptography by Ron Rivest. It takes 420 instructions to hash a block of 64 aligned bytes. I combined that with my hash's method of putting unaligned bytes into registers, adding 3n instructions. MD4 is overkill for hash table lookup.

The table below compares all these hash functions.

NAME: is the name of the hash.
SIZE-1000: is the smallest reasonable hash table size greater than 1000.
SPEED: is the speed of the hash, measured in instructions required to produce a hash value for a table with SIZE-1000 buckets. It is assumed the machine has a rotate instruction. These aren't very accurate measures ... I should really just do timings on a Pentium 4 or such.
INLINE: This is the speed assuming the hash is inlined in a loop that has to walk through all the characters anyways, such as a tokenizer. Such a loop doesn't always exist, and even when it does inlining isn't always possible. Some hashes (my new hash and MD4) work on blocks larger than a character. Inlining a hash removes 3n+1 instructions of loop overhead. It also removes the n instructions needed to get the characters out of the key array and into a register. It also means the length isn't known. Inlining offers other advantages. It allows the string to be converted to uppercase, and/or to unicode, before the hash is performed without the expense of an extra loop or a temporary buffer.
FUNNEL-15: is the largest set of input bits affecting the smallest set of internal state bits when mapping 15-byte keys into a 1-byte result.
FUNNEL-100: is the largest set of input bits affecting the smallest set of internal state bits when mapping 100-byte keys into a 32-bit result.
COLLIDE-32: is the number of collisions found when a dictionary of 38,470 English words was hashed into a 32-bit result. (The expected number of collisions is 0.2 .)
COLLIDE-1000: is a chi² measure of how well the hash did at mapping the 38470-word dictionary into the SIZE-1000 table. (A chi² measure greater than +3 is significantly worse than a random mapping; less than -3 is significantly better than a random mapping; in between is just random fluctuations.)

Comparison of several hash functionsNAMESIZE-1000SPEEDINLINEFUNNEL-15FUNNEL-100COLLIDE-32COLLIDE-1000Additive10095n+3n+215 into 2100 into 237006+806.02Rotating10096n+32n+24 into 125 into 124+1.24One-at-a-Time10249n+95n+8nonenone0-0.05Bernstein10247n+33n+23 into 23 into 24+1.69FNV1024??????Pearson102412n+54n+3nonenone0+1.65CRC10249n+35n+22 into 111 into 100+0.07Generalized10249n+35n+2nonenone0-1.83Universal102452n+348n+24 into 350 into 280+0.20Zobrist102410n+36n+2nonenone1-0.03Paul Hsieh's10245n+17N/A3 into 23 into 21+1.12My Hash10246n+35N/Anonenone0+0.33lookup3.c10245n+20N/Anonenone0-0.08MD410249.5n+230N/Anonenone1+0.73

From the measurements we can conclude that the Additive and Rotating hash (and maybe Bernstein) were noticably bad for 32-bit results, and only the Additive hash was noticably bad for 10-bit results. If inlining is possible, the Rotating hash was the fastest acceptable hash, followed by Bernstein, Pearson or the Generalized CRC (if table lookup is OK) or Bernstein or One-at-a-Time (if table lookup is not OK). If inlining is not possible, it's a draw between lookup3 and Paul Hsieh's hash. Note that, for many applications, the Rotating hash is noticably bad and should not be used, and the Bernstein hash is marginal. Table lengths should always be a power of 2 because that's faster than prime lengths and all acceptable hashes allow it.

The COLLIDE-1000 numbers should be ignored, unless the numbers are bigger than 3 or less than -3. For example, generalized CRC produced +.8, -.8, or -1.8 for three different tables I tried. It's just noise. A different set of keys would give unrelated random fluctuations.

Conclusion

A common weakness in hash function is for a small set of input bits to cancel each other out. There is an efficient test to detect most such weaknesses, and many functions pass this test. I gave code for the fastest such function I could find. Hash functions without this weakness work equally well on all classes of keys.

Testimonials:

A fractal mountain and image of same

附录II（http://burtleburtle.net/bob/c/lookup3.c）

/*-------------------------------------------------------------------------------lookup3.c, by Bob Jenkins, May 2006, Public Domain.These are functions for producing 32-bit hashes for hash table lookup.hashword(), hashlittle(), hashlittle2(), hashbig(), mix(), and final() are externally useful functions.  Routines to test the hash are included if SELF_TEST is defined.  You can use this free for any purpose.  It's inthe public domain.  It has no warranty.You probably want to use hashlittle().  hashlittle() and hashbig()hash byte arrays.  hashlittle() is is faster than hashbig() onlittle-endian machines.  Intel and AMD are little-endian machines.On second thought, you probably want hashlittle2(), which is identical tohashlittle() except it returns two 32-bit hashes for the price of one.  You could implement hashbig2() if you wanted but I haven't bothered here.If you want to find a hash of, say, exactly 7 integers, do  a = i1;  b = i2;  c = i3;  mix(a,b,c);  a += i4; b += i5; c += i6;  mix(a,b,c);  a += i7;  final(a,b,c);then use c as the hash value.  If you have a variable length array of4-byte integers to hash, use hashword().  If you have a byte array (likea character string), use hashlittle().  If you have several byte arrays, ora mix of things, see the comments above hashlittle().  Why is this so big?  I read 12 bytes at a time into 3 4-byte integers, then mix those integers.  This is fast (you can do a lot more thoroughmixing with 12*3 instructions on 3 integers than you can with 3 instructionson 1 byte), but shoehorning those bytes into integers efficiently is messy.-------------------------------------------------------------------------------*/#define SELF_TEST 1#include <stdio.h>      /* defines printf for tests */#include <time.h>       /* defines time_t for timings in the test */#include <stdint.h>     /* defines uint32_t etc */#include <sys/param.h>  /* attempt to define endianness */#ifdef linux# include <endian.h>    /* attempt to define endianness */#endif/* * My best guess at if you are big-endian or little-endian.  This may * need adjustment. */#if (defined(__BYTE_ORDER) && defined(__LITTLE_ENDIAN) && \     __BYTE_ORDER == __LITTLE_ENDIAN) || \    (defined(i386) || defined(__i386__) || defined(__i486__) || \     defined(__i586__) || defined(__i686__) || defined(vax) || defined(MIPSEL))# define HASH_LITTLE_ENDIAN 1# define HASH_BIG_ENDIAN 0#elif (defined(__BYTE_ORDER) && defined(__BIG_ENDIAN) && \       __BYTE_ORDER == __BIG_ENDIAN) || \      (defined(sparc) || defined(POWERPC) || defined(mc68000) || defined(sel))# define HASH_LITTLE_ENDIAN 0# define HASH_BIG_ENDIAN 1#else# define HASH_LITTLE_ENDIAN 0# define HASH_BIG_ENDIAN 0#endif#define hashsize(n) ((uint32_t)1<<(n))#define hashmask(n) (hashsize(n)-1)#define rot(x,k) (((x)<<(k)) | ((x)>>(32-(k))))/*-------------------------------------------------------------------------------mix -- mix 3 32-bit values reversibly.This is reversible, so any information in (a,b,c) before mix() isstill in (a,b,c) after mix().If four pairs of (a,b,c) inputs are run through mix(), or throughmix() in reverse, there are at least 32 bits of the output thatare sometimes the same for one pair and different for another pair.This was tested for:* pairs that differed by one bit, by two bits, in any combination  of top bits of (a,b,c), or in any combination of bottom bits of  (a,b,c).* "differ" is defined as +, -, ^, or ~^.  For + and -, I transformed  the output delta to a Gray code (a^(a>>1)) so a string of 1's (as  is commonly produced by subtraction) look like a single 1-bit  difference.* the base values were pseudorandom, all zero but one bit set, or   all zero plus a counter that starts at zero.Some k values for my "a-=c; a^=rot(c,k); c+=b;" arrangement thatsatisfy this are    4  6  8 16 19  4    9 15  3 18 27 15   14  9  3  7 17  3Well, "9 15 3 18 27 15" didn't quite get 32 bits diffingfor "differ" defined as + with a one-bit base and a two-bit delta.  Iused http://burtleburtle.net/bob/hash/avalanche.html to choose the operations, constants, and arrangements of the variables.This does not achieve avalanche.  There are input bits of (a,b,c)that fail to affect some output bits of (a,b,c), especially of a.  Themost thoroughly mixed value is c, but it doesn't really even achieveavalanche in c.This allows some parallelism.  Read-after-writes are good at doublingthe number of bits affected, so the goal of mixing pulls in the oppositedirection as the goal of parallelism.  I did what I could.  Rotatesseem to cost as much as shifts on every machine I could lay my handson, and rotates are much kinder to the top and bottom bits, so I usedrotates.-------------------------------------------------------------------------------*/#define mix(a,b,c) \{ \  a -= c;  a ^= rot(c, 4);  c += b; \  b -= a;  b ^= rot(a, 6);  a += c; \  c -= b;  c ^= rot(b, 8);  b += a; \  a -= c;  a ^= rot(c,16);  c += b; \  b -= a;  b ^= rot(a,19);  a += c; \  c -= b;  c ^= rot(b, 4);  b += a; \}/*-------------------------------------------------------------------------------final -- final mixing of 3 32-bit values (a,b,c) into cPairs of (a,b,c) values differing in only a few bits will usuallyproduce values of c that look totally different.  This was tested for* pairs that differed by one bit, by two bits, in any combination  of top bits of (a,b,c), or in any combination of bottom bits of  (a,b,c).* "differ" is defined as +, -, ^, or ~^.  For + and -, I transformed  the output delta to a Gray code (a^(a>>1)) so a string of 1's (as  is commonly produced by subtraction) look like a single 1-bit  difference.* the base values were pseudorandom, all zero but one bit set, or   all zero plus a counter that starts at zero.These constants passed: 14 11 25 16 4 14 24 12 14 25 16 4 14 24and these came close:  4  8 15 26 3 22 24 10  8 15 26 3 22 24 11  8 15 26 3 22 24-------------------------------------------------------------------------------*/#define final(a,b,c) \{ \  c ^= b; c -= rot(b,14); \  a ^= c; a -= rot(c,11); \  b ^= a; b -= rot(a,25); \  c ^= b; c -= rot(b,16); \  a ^= c; a -= rot(c,4);  \  b ^= a; b -= rot(a,14); \  c ^= b; c -= rot(b,24); \}/*-------------------------------------------------------------------- This works on all machines.  To be useful, it requires -- that the key be an array of uint32_t's, and -- that the length be the number of uint32_t's in the key The function hashword() is identical to hashlittle() on little-endian machines, and identical to hashbig() on big-endian machines, except that the length has to be measured in uint32_ts rather than in bytes.  hashlittle() is more complicated than hashword() only because hashlittle() has to dance around fitting the key bytes into registers.--------------------------------------------------------------------*/uint32_t hashword(const uint32_t *k,                   /* the key, an array of uint32_t values */size_t          length,               /* the length of the key, in uint32_ts */uint32_t        initval)         /* the previous hash, or an arbitrary value */{  uint32_t a,b,c;  /* Set up the internal state */  a = b = c = 0xdeadbeef + (((uint32_t)length)<<2) + initval;  /*------------------------------------------------- handle most of the key */  while (length > 3)  {    a += k[0];    b += k[1];    c += k[2];    mix(a,b,c);    length -= 3;    k += 3;  }  /*------------------------------------------- handle the last 3 uint32_t's */  switch(length)                     /* all the case statements fall through */  {   case 3 : c+=k[2];  case 2 : b+=k[1];  case 1 : a+=k[0];    final(a,b,c);  case 0:     /* case 0: nothing left to add */    break;  }  /*------------------------------------------------------ report the result */  return c;}/*--------------------------------------------------------------------hashword2() -- same as hashword(), but take two seeds and return two32-bit values.  pc and pb must both be nonnull, and *pc and *pb mustboth be initialized with seeds.  If you pass in (*pb)==0, the output (*pc) will be the same as the return value from hashword().--------------------------------------------------------------------*/void hashword2 (const uint32_t *k,                   /* the key, an array of uint32_t values */size_t          length,               /* the length of the key, in uint32_ts */uint32_t       *pc,                      /* IN: seed OUT: primary hash value */uint32_t       *pb)               /* IN: more seed OUT: secondary hash value */{  uint32_t a,b,c;  /* Set up the internal state */  a = b = c = 0xdeadbeef + ((uint32_t)(length<<2)) + *pc;  c += *pb;  /*------------------------------------------------- handle most of the key */  while (length > 3)  {    a += k[0];    b += k[1];    c += k[2];    mix(a,b,c);    length -= 3;    k += 3;  }  /*------------------------------------------- handle the last 3 uint32_t's */  switch(length)                     /* all the case statements fall through */  {   case 3 : c+=k[2];  case 2 : b+=k[1];  case 1 : a+=k[0];    final(a,b,c);  case 0:     /* case 0: nothing left to add */    break;  }  /*------------------------------------------------------ report the result */  *pc=c; *pb=b;}/*-------------------------------------------------------------------------------hashlittle() -- hash a variable-length key into a 32-bit value  k       : the key (the unaligned variable-length array of bytes)  length  : the length of the key, counting by bytes  initval : can be any 4-byte valueReturns a 32-bit value.  Every bit of the key affects every bit ofthe return value.  Two keys differing by one or two bits will havetotally different hash values.The best hash table sizes are powers of 2.  There is no need to domod a prime (mod is sooo slow!).  If you need less than 32 bits,use a bitmask.  For example, if you need only 10 bits, do  h = (h & hashmask(10));In which case, the hash table should have hashsize(10) elements.If you are hashing n strings (uint8_t **)k, do it like this:  for (i=0, h=0; i<n; ++i) h = hashlittle( k[i], len[i], h);By Bob Jenkins, 2006.  bob_jenkins@burtleburtle.net.  You may use thiscode any way you wish, private, educational, or commercial.  It's free.Use for hash table lookup, or anything where one collision in 2^^32 isacceptable.  Do NOT use for cryptographic purposes.-------------------------------------------------------------------------------*/uint32_t hashlittle( const void *key, size_t length, uint32_t initval){  uint32_t a,b,c;                                          /* internal state */  union { const void *ptr; size_t i; } u;     /* needed for Mac Powerbook G4 */  /* Set up the internal state */  a = b = c = 0xdeadbeef + ((uint32_t)length) + initval;  u.ptr = key;  if (HASH_LITTLE_ENDIAN && ((u.i & 0x3) == 0)) {    const uint32_t *k = (const uint32_t *)key;         /* read 32-bit chunks */    const uint8_t  *k8;    /*------ all but last block: aligned reads and affect 32 bits of (a,b,c) */    while (length > 12)    {      a += k[0];      b += k[1];      c += k[2];      mix(a,b,c);      length -= 12;      k += 3;    }    /*----------------------------- handle the last (probably partial) block */    /*      * "k[2]&0xffffff" actually reads beyond the end of the string, but     * then masks off the part it's not allowed to read.  Because the     * string is aligned, the masked-off tail is in the same word as the     * rest of the string.  Every machine with memory protection I've seen     * does it on word boundaries, so is OK with this.  But VALGRIND will     * still catch it and complain.  The masking trick does make the hash     * noticably faster for short strings (like English words).     */#ifndef VALGRIND    switch(length)    {    case 12: c+=k[2]; b+=k[1]; a+=k[0]; break;    case 11: c+=k[2]&0xffffff; b+=k[1]; a+=k[0]; break;    case 10: c+=k[2]&0xffff; b+=k[1]; a+=k[0]; break;    case 9 : c+=k[2]&0xff; b+=k[1]; a+=k[0]; break;    case 8 : b+=k[1]; a+=k[0]; break;    case 7 : b+=k[1]&0xffffff; a+=k[0]; break;    case 6 : b+=k[1]&0xffff; a+=k[0]; break;    case 5 : b+=k[1]&0xff; a+=k[0]; break;    case 4 : a+=k[0]; break;    case 3 : a+=k[0]&0xffffff; break;    case 2 : a+=k[0]&0xffff; break;    case 1 : a+=k[0]&0xff; break;    case 0 : return c;              /* zero length strings require no mixing */    }#else /* make valgrind happy */    k8 = (const uint8_t *)k;    switch(length)    {    case 12: c+=k[2]; b+=k[1]; a+=k[0]; break;    case 11: c+=((uint32_t)k8[10])<<16;  /* fall through */    case 10: c+=((uint32_t)k8[9])<<8;    /* fall through */    case 9 : c+=k8[8];                   /* fall through */    case 8 : b+=k[1]; a+=k[0]; break;    case 7 : b+=((uint32_t)k8[6])<<16;   /* fall through */    case 6 : b+=((uint32_t)k8[5])<<8;    /* fall through */    case 5 : b+=k8[4];                   /* fall through */    case 4 : a+=k[0]; break;    case 3 : a+=((uint32_t)k8[2])<<16;   /* fall through */    case 2 : a+=((uint32_t)k8[1])<<8;    /* fall through */    case 1 : a+=k8[0]; break;    case 0 : return c;    }#endif /* !valgrind */  } else if (HASH_LITTLE_ENDIAN && ((u.i & 0x1) == 0)) {    const uint16_t *k = (const uint16_t *)key;         /* read 16-bit chunks */    const uint8_t  *k8;    /*--------------- all but last block: aligned reads and different mixing */    while (length > 12)    {      a += k[0] + (((uint32_t)k[1])<<16);      b += k[2] + (((uint32_t)k[3])<<16);      c += k[4] + (((uint32_t)k[5])<<16);      mix(a,b,c);      length -= 12;      k += 6;    }    /*----------------------------- handle the last (probably partial) block */    k8 = (const uint8_t *)k;    switch(length)    {    case 12: c+=k[4]+(((uint32_t)k[5])<<16);             b+=k[2]+(((uint32_t)k[3])<<16);             a+=k[0]+(((uint32_t)k[1])<<16);             break;    case 11: c+=((uint32_t)k8[10])<<16;     /* fall through */    case 10: c+=k[4];             b+=k[2]+(((uint32_t)k[3])<<16);             a+=k[0]+(((uint32_t)k[1])<<16);             break;    case 9 : c+=k8[8];                      /* fall through */    case 8 : b+=k[2]+(((uint32_t)k[3])<<16);             a+=k[0]+(((uint32_t)k[1])<<16);             break;    case 7 : b+=((uint32_t)k8[6])<<16;      /* fall through */    case 6 : b+=k[2];             a+=k[0]+(((uint32_t)k[1])<<16);             break;    case 5 : b+=k8[4];                      /* fall through */    case 4 : a+=k[0]+(((uint32_t)k[1])<<16);             break;    case 3 : a+=((uint32_t)k8[2])<<16;      /* fall through */    case 2 : a+=k[0];             break;    case 1 : a+=k8[0];             break;    case 0 : return c;                     /* zero length requires no mixing */    }  } else {                        /* need to read the key one byte at a time */    const uint8_t *k = (const uint8_t *)key;    /*--------------- all but the last block: affect some 32 bits of (a,b,c) */    while (length > 12)    {      a += k[0];      a += ((uint32_t)k[1])<<8;      a += ((uint32_t)k[2])<<16;      a += ((uint32_t)k[3])<<24;      b += k[4];      b += ((uint32_t)k[5])<<8;      b += ((uint32_t)k[6])<<16;      b += ((uint32_t)k[7])<<24;      c += k[8];      c += ((uint32_t)k[9])<<8;      c += ((uint32_t)k[10])<<16;      c += ((uint32_t)k[11])<<24;      mix(a,b,c);      length -= 12;      k += 12;    }    /*-------------------------------- last block: affect all 32 bits of (c) */    switch(length)                   /* all the case statements fall through */    {    case 12: c+=((uint32_t)k[11])<<24;    case 11: c+=((uint32_t)k[10])<<16;    case 10: c+=((uint32_t)k[9])<<8;    case 9 : c+=k[8];    case 8 : b+=((uint32_t)k[7])<<24;    case 7 : b+=((uint32_t)k[6])<<16;    case 6 : b+=((uint32_t)k[5])<<8;    case 5 : b+=k[4];    case 4 : a+=((uint32_t)k[3])<<24;    case 3 : a+=((uint32_t)k[2])<<16;    case 2 : a+=((uint32_t)k[1])<<8;    case 1 : a+=k[0];             break;    case 0 : return c;    }  }  final(a,b,c);  return c;}/* * hashlittle2: return 2 32-bit hash values * * This is identical to hashlittle(), except it returns two 32-bit hash * values instead of just one.  This is good enough for hash table * lookup with 2^^64 buckets, or if you want a second hash if you're not * happy with the first, or if you want a probably-unique 64-bit ID for * the key.  *pc is better mixed than *pb, so use *pc first.  If you want * a 64-bit value do something like "*pc + (((uint64_t)*pb)<<32)". */void hashlittle2(   const void *key,       /* the key to hash */  size_t      length,    /* length of the key */  uint32_t   *pc,        /* IN: primary initval, OUT: primary hash */  uint32_t   *pb)        /* IN: secondary initval, OUT: secondary hash */{  uint32_t a,b,c;                                          /* internal state */  union { const void *ptr; size_t i; } u;     /* needed for Mac Powerbook G4 */  /* Set up the internal state */  a = b = c = 0xdeadbeef + ((uint32_t)length) + *pc;  c += *pb;  u.ptr = key;  if (HASH_LITTLE_ENDIAN && ((u.i & 0x3) == 0)) {    const uint32_t *k = (const uint32_t *)key;         /* read 32-bit chunks */    const uint8_t  *k8;    /*------ all but last block: aligned reads and affect 32 bits of (a,b,c) */    while (length > 12)    {      a += k[0];      b += k[1];      c += k[2];      mix(a,b,c);      length -= 12;      k += 3;    }    /*----------------------------- handle the last (probably partial) block */    /*      * "k[2]&0xffffff" actually reads beyond the end of the string, but     * then masks off the part it's not allowed to read.  Because the     * string is aligned, the masked-off tail is in the same word as the     * rest of the string.  Every machine with memory protection I've seen     * does it on word boundaries, so is OK with this.  But VALGRIND will     * still catch it and complain.  The masking trick does make the hash     * noticably faster for short strings (like English words).     */#ifndef VALGRIND    switch(length)    {    case 12: c+=k[2]; b+=k[1]; a+=k[0]; break;    case 11: c+=k[2]&0xffffff; b+=k[1]; a+=k[0]; break;    case 10: c+=k[2]&0xffff; b+=k[1]; a+=k[0]; break;    case 9 : c+=k[2]&0xff; b+=k[1]; a+=k[0]; break;    case 8 : b+=k[1]; a+=k[0]; break;    case 7 : b+=k[1]&0xffffff; a+=k[0]; break;    case 6 : b+=k[1]&0xffff; a+=k[0]; break;    case 5 : b+=k[1]&0xff; a+=k[0]; break;    case 4 : a+=k[0]; break;    case 3 : a+=k[0]&0xffffff; break;    case 2 : a+=k[0]&0xffff; break;    case 1 : a+=k[0]&0xff; break;    case 0 : *pc=c; *pb=b; return;  /* zero length strings require no mixing */    }#else /* make valgrind happy */    k8 = (const uint8_t *)k;    switch(length)    {    case 12: c+=k[2]; b+=k[1]; a+=k[0]; break;    case 11: c+=((uint32_t)k8[10])<<16;  /* fall through */    case 10: c+=((uint32_t)k8[9])<<8;    /* fall through */    case 9 : c+=k8[8];                   /* fall through */    case 8 : b+=k[1]; a+=k[0]; break;    case 7 : b+=((uint32_t)k8[6])<<16;   /* fall through */    case 6 : b+=((uint32_t)k8[5])<<8;    /* fall through */    case 5 : b+=k8[4];                   /* fall through */    case 4 : a+=k[0]; break;    case 3 : a+=((uint32_t)k8[2])<<16;   /* fall through */    case 2 : a+=((uint32_t)k8[1])<<8;    /* fall through */    case 1 : a+=k8[0]; break;    case 0 : *pc=c; *pb=b; return;  /* zero length strings require no mixing */    }#endif /* !valgrind */  } else if (HASH_LITTLE_ENDIAN && ((u.i & 0x1) == 0)) {    const uint16_t *k = (const uint16_t *)key;         /* read 16-bit chunks */    const uint8_t  *k8;    /*--------------- all but last block: aligned reads and different mixing */    while (length > 12)    {      a += k[0] + (((uint32_t)k[1])<<16);      b += k[2] + (((uint32_t)k[3])<<16);      c += k[4] + (((uint32_t)k[5])<<16);      mix(a,b,c);      length -= 12;      k += 6;    }    /*----------------------------- handle the last (probably partial) block */    k8 = (const uint8_t *)k;    switch(length)    {    case 12: c+=k[4]+(((uint32_t)k[5])<<16);             b+=k[2]+(((uint32_t)k[3])<<16);             a+=k[0]+(((uint32_t)k[1])<<16);             break;    case 11: c+=((uint32_t)k8[10])<<16;     /* fall through */    case 10: c+=k[4];             b+=k[2]+(((uint32_t)k[3])<<16);             a+=k[0]+(((uint32_t)k[1])<<16);             break;    case 9 : c+=k8[8];                      /* fall through */    case 8 : b+=k[2]+(((uint32_t)k[3])<<16);             a+=k[0]+(((uint32_t)k[1])<<16);             break;    case 7 : b+=((uint32_t)k8[6])<<16;      /* fall through */    case 6 : b+=k[2];             a+=k[0]+(((uint32_t)k[1])<<16);             break;    case 5 : b+=k8[4];                      /* fall through */    case 4 : a+=k[0]+(((uint32_t)k[1])<<16);             break;    case 3 : a+=((uint32_t)k8[2])<<16;      /* fall through */    case 2 : a+=k[0];             break;    case 1 : a+=k8[0];             break;    case 0 : *pc=c; *pb=b; return;  /* zero length strings require no mixing */    }  } else {                        /* need to read the key one byte at a time */    const uint8_t *k = (const uint8_t *)key;    /*--------------- all but the last block: affect some 32 bits of (a,b,c) */    while (length > 12)    {      a += k[0];      a += ((uint32_t)k[1])<<8;      a += ((uint32_t)k[2])<<16;      a += ((uint32_t)k[3])<<24;      b += k[4];      b += ((uint32_t)k[5])<<8;      b += ((uint32_t)k[6])<<16;      b += ((uint32_t)k[7])<<24;      c += k[8];      c += ((uint32_t)k[9])<<8;      c += ((uint32_t)k[10])<<16;      c += ((uint32_t)k[11])<<24;      mix(a,b,c);      length -= 12;      k += 12;    }    /*-------------------------------- last block: affect all 32 bits of (c) */    switch(length)                   /* all the case statements fall through */    {    case 12: c+=((uint32_t)k[11])<<24;    case 11: c+=((uint32_t)k[10])<<16;    case 10: c+=((uint32_t)k[9])<<8;    case 9 : c+=k[8];    case 8 : b+=((uint32_t)k[7])<<24;    case 7 : b+=((uint32_t)k[6])<<16;    case 6 : b+=((uint32_t)k[5])<<8;    case 5 : b+=k[4];    case 4 : a+=((uint32_t)k[3])<<24;    case 3 : a+=((uint32_t)k[2])<<16;    case 2 : a+=((uint32_t)k[1])<<8;    case 1 : a+=k[0];             break;    case 0 : *pc=c; *pb=b; return;  /* zero length strings require no mixing */    }  }  final(a,b,c);  *pc=c; *pb=b;}/* * hashbig(): * This is the same as hashword() on big-endian machines.  It is different * from hashlittle() on all machines.  hashbig() takes advantage of * big-endian byte ordering.  */uint32_t hashbig( const void *key, size_t length, uint32_t initval){  uint32_t a,b,c;  union { const void *ptr; size_t i; } u; /* to cast key to (size_t) happily */  /* Set up the internal state */  a = b = c = 0xdeadbeef + ((uint32_t)length) + initval;  u.ptr = key;  if (HASH_BIG_ENDIAN && ((u.i & 0x3) == 0)) {    const uint32_t *k = (const uint32_t *)key;         /* read 32-bit chunks */    const uint8_t  *k8;    /*------ all but last block: aligned reads and affect 32 bits of (a,b,c) */    while (length > 12)    {      a += k[0];      b += k[1];      c += k[2];      mix(a,b,c);      length -= 12;      k += 3;    }    /*----------------------------- handle the last (probably partial) block */    /*      * "k[2]<<8" actually reads beyond the end of the string, but     * then shifts out the part it's not allowed to read.  Because the     * string is aligned, the illegal read is in the same word as the     * rest of the string.  Every machine with memory protection I've seen     * does it on word boundaries, so is OK with this.  But VALGRIND will     * still catch it and complain.  The masking trick does make the hash     * noticably faster for short strings (like English words).     */#ifndef VALGRIND    switch(length)    {    case 12: c+=k[2]; b+=k[1]; a+=k[0]; break;    case 11: c+=k[2]&0xffffff00; b+=k[1]; a+=k[0]; break;    case 10: c+=k[2]&0xffff0000; b+=k[1]; a+=k[0]; break;    case 9 : c+=k[2]&0xff000000; b+=k[1]; a+=k[0]; break;    case 8 : b+=k[1]; a+=k[0]; break;    case 7 : b+=k[1]&0xffffff00; a+=k[0]; break;    case 6 : b+=k[1]&0xffff0000; a+=k[0]; break;    case 5 : b+=k[1]&0xff000000; a+=k[0]; break;    case 4 : a+=k[0]; break;    case 3 : a+=k[0]&0xffffff00; break;    case 2 : a+=k[0]&0xffff0000; break;    case 1 : a+=k[0]&0xff000000; break;    case 0 : return c;              /* zero length strings require no mixing */    }#else  /* make valgrind happy */    k8 = (const uint8_t *)k;    switch(length)                   /* all the case statements fall through */    {    case 12: c+=k[2]; b+=k[1]; a+=k[0]; break;    case 11: c+=((uint32_t)k8[10])<<8;  /* fall through */    case 10: c+=((uint32_t)k8[9])<<16;  /* fall through */    case 9 : c+=((uint32_t)k8[8])<<24;  /* fall through */    case 8 : b+=k[1]; a+=k[0]; break;    case 7 : b+=((uint32_t)k8[6])<<8;   /* fall through */    case 6 : b+=((uint32_t)k8[5])<<16;  /* fall through */    case 5 : b+=((uint32_t)k8[4])<<24;  /* fall through */    case 4 : a+=k[0]; break;    case 3 : a+=((uint32_t)k8[2])<<8;   /* fall through */    case 2 : a+=((uint32_t)k8[1])<<16;  /* fall through */    case 1 : a+=((uint32_t)k8[0])<<24; break;    case 0 : return c;    }#endif /* !VALGRIND */  } else {                        /* need to read the key one byte at a time */    const uint8_t *k = (const uint8_t *)key;    /*--------------- all but the last block: affect some 32 bits of (a,b,c) */    while (length > 12)    {      a += ((uint32_t)k[0])<<24;      a += ((uint32_t)k[1])<<16;      a += ((uint32_t)k[2])<<8;      a += ((uint32_t)k[3]);      b += ((uint32_t)k[4])<<24;      b += ((uint32_t)k[5])<<16;      b += ((uint32_t)k[6])<<8;      b += ((uint32_t)k[7]);      c += ((uint32_t)k[8])<<24;      c += ((uint32_t)k[9])<<16;      c += ((uint32_t)k[10])<<8;      c += ((uint32_t)k[11]);      mix(a,b,c);      length -= 12;      k += 12;    }    /*-------------------------------- last block: affect all 32 bits of (c) */    switch(length)                   /* all the case statements fall through */    {    case 12: c+=k[11];    case 11: c+=((uint32_t)k[10])<<8;    case 10: c+=((uint32_t)k[9])<<16;    case 9 : c+=((uint32_t)k[8])<<24;    case 8 : b+=k[7];    case 7 : b+=((uint32_t)k[6])<<8;    case 6 : b+=((uint32_t)k[5])<<16;    case 5 : b+=((uint32_t)k[4])<<24;    case 4 : a+=k[3];    case 3 : a+=((uint32_t)k[2])<<8;    case 2 : a+=((uint32_t)k[1])<<16;    case 1 : a+=((uint32_t)k[0])<<24;             break;    case 0 : return c;    }  }  final(a,b,c);  return c;}#ifdef SELF_TEST/* used for timings */void driver1(){  uint8_t buf[256];  uint32_t i;  uint32_t h=0;  time_t a,z;  time(&a);  for (i=0; i<256; ++i) buf[i] = 'x';  for (i=0; i<1; ++i)   {    h = hashlittle(&buf[0],1,h);  }  time(&z);  if (z-a > 0) printf("time %d %.8x\n", z-a, h);}/* check that every input bit changes every output bit half the time */#define HASHSTATE 1#define HASHLEN   1#define MAXPAIR 60#define MAXLEN  70void driver2(){  uint8_t qa[MAXLEN+1], qb[MAXLEN+2], *a = &qa[0], *b = &qb[1];  uint32_t c[HASHSTATE], d[HASHSTATE], i=0, j=0, k, l, m=0, z;  uint32_t e[HASHSTATE],f[HASHSTATE],g[HASHSTATE],h[HASHSTATE];  uint32_t x[HASHSTATE],y[HASHSTATE];  uint32_t hlen;  printf("No more than %d trials should ever be needed \n",MAXPAIR/2);  for (hlen=0; hlen < MAXLEN; ++hlen)  {    z=0;    for (i=0; i<hlen; ++i)  /*----------------------- for each input byte, */    {      for (j=0; j<8; ++j)   /*------------------------ for each input bit, */      {for (m=1; m<8; ++m) /*------------ for serveral possible initvals, */{  for (l=0; l<HASHSTATE; ++l)    e[l]=f[l]=g[l]=h[l]=x[l]=y[l]=~((uint32_t)0);        /*---- check that every output bit is affected by that input bit */  for (k=0; k<MAXPAIR; k+=2)  {     uint32_t finished=1;    /* keys have one bit different */    for (l=0; l<hlen+1; ++l) {a[l] = b[l] = (uint8_t)0;}    /* have a and b be two keys differing in only one bit */    a[i] ^= (k<<j);    a[i] ^= (k>>(8-j));     c[0] = hashlittle(a, hlen, m);    b[i] ^= ((k+1)<<j);    b[i] ^= ((k+1)>>(8-j));     d[0] = hashlittle(b, hlen, m);    /* check every bit is 1, 0, set, and not set at least once */    for (l=0; l<HASHSTATE; ++l)    {      e[l] &= (c[l]^d[l]);      f[l] &= ~(c[l]^d[l]);      g[l] &= c[l];      h[l] &= ~c[l];      x[l] &= d[l];      y[l] &= ~d[l];      if (e[l]|f[l]|g[l]|h[l]|x[l]|y[l]) finished=0;    }    if (finished) break;  }  if (k>z) z=k;  if (k==MAXPAIR)   {     printf("Some bit didn't change: ");     printf("%.8x %.8x %.8x %.8x %.8x %.8x  ",            e[0],f[0],g[0],h[0],x[0],y[0]);     printf("i %d j %d m %d len %d\n", i, j, m, hlen);  }  if (z==MAXPAIR) goto done;}      }    }   done:    if (z < MAXPAIR)    {      printf("Mix success  %2d bytes  %2d initvals  ",i,m);      printf("required  %d  trials\n", z/2);    }  }  printf("\n");}/* Check for reading beyond the end of the buffer and alignment problems */void driver3(){  uint8_t buf[MAXLEN+20], *b;  uint32_t len;  uint8_t q[] = "This is the time for all good men to come to the aid of their country...";  uint32_t h;  uint8_t qq[] = "xThis is the time for all good men to come to the aid of their country...";  uint32_t i;  uint8_t qqq[] = "xxThis is the time for all good men to come to the aid of their country...";  uint32_t j;  uint8_t qqqq[] = "xxxThis is the time for all good men to come to the aid of their country...";  uint32_t ref,x,y;  uint8_t *p;  printf("Endianness.  These lines should all be the same (for values filled in):\n");  printf("%.8x                            %.8x                            %.8x\n",         hashword((const uint32_t *)q, (sizeof(q)-1)/4, 13),         hashword((const uint32_t *)q, (sizeof(q)-5)/4, 13),         hashword((const uint32_t *)q, (sizeof(q)-9)/4, 13));  p = q;  printf("%.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x\n",         hashlittle(p, sizeof(q)-1, 13), hashlittle(p, sizeof(q)-2, 13),         hashlittle(p, sizeof(q)-3, 13), hashlittle(p, sizeof(q)-4, 13),         hashlittle(p, sizeof(q)-5, 13), hashlittle(p, sizeof(q)-6, 13),         hashlittle(p, sizeof(q)-7, 13), hashlittle(p, sizeof(q)-8, 13),         hashlittle(p, sizeof(q)-9, 13), hashlittle(p, sizeof(q)-10, 13),         hashlittle(p, sizeof(q)-11, 13), hashlittle(p, sizeof(q)-12, 13));  p = &qq[1];  printf("%.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x\n",         hashlittle(p, sizeof(q)-1, 13), hashlittle(p, sizeof(q)-2, 13),         hashlittle(p, sizeof(q)-3, 13), hashlittle(p, sizeof(q)-4, 13),         hashlittle(p, sizeof(q)-5, 13), hashlittle(p, sizeof(q)-6, 13),         hashlittle(p, sizeof(q)-7, 13), hashlittle(p, sizeof(q)-8, 13),         hashlittle(p, sizeof(q)-9, 13), hashlittle(p, sizeof(q)-10, 13),         hashlittle(p, sizeof(q)-11, 13), hashlittle(p, sizeof(q)-12, 13));  p = &qqq[2];  printf("%.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x\n",         hashlittle(p, sizeof(q)-1, 13), hashlittle(p, sizeof(q)-2, 13),         hashlittle(p, sizeof(q)-3, 13), hashlittle(p, sizeof(q)-4, 13),         hashlittle(p, sizeof(q)-5, 13), hashlittle(p, sizeof(q)-6, 13),         hashlittle(p, sizeof(q)-7, 13), hashlittle(p, sizeof(q)-8, 13),         hashlittle(p, sizeof(q)-9, 13), hashlittle(p, sizeof(q)-10, 13),         hashlittle(p, sizeof(q)-11, 13), hashlittle(p, sizeof(q)-12, 13));  p = &qqqq[3];  printf("%.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x %.8x\n",         hashlittle(p, sizeof(q)-1, 13), hashlittle(p, sizeof(q)-2, 13),         hashlittle(p, sizeof(q)-3, 13), hashlittle(p, sizeof(q)-4, 13),         hashlittle(p, sizeof(q)-5, 13), hashlittle(p, sizeof(q)-6, 13),         hashlittle(p, sizeof(q)-7, 13), hashlittle(p, sizeof(q)-8, 13),         hashlittle(p, sizeof(q)-9, 13), hashlittle(p, sizeof(q)-10, 13),         hashlittle(p, sizeof(q)-11, 13), hashlittle(p, sizeof(q)-12, 13));  printf("\n");  /* check that hashlittle2 and hashlittle produce the same results */  i=47; j=0;  hashlittle2(q, sizeof(q), &i, &j);  if (hashlittle(q, sizeof(q), 47) != i)    printf("hashlittle2 and hashlittle mismatch\n");  /* check that hashword2 and hashword produce the same results */  len = 0xdeadbeef;  i=47, j=0;  hashword2(&len, 1, &i, &j);  if (hashword(&len, 1, 47) != i)    printf("hashword2 and hashword mismatch %x %x\n",    i, hashword(&len, 1, 47));  /* check hashlittle doesn't read before or after the ends of the string */  for (h=0, b=buf+1; h<8; ++h, ++b)  {    for (i=0; i<MAXLEN; ++i)    {      len = i;      for (j=0; j<i; ++j) *(b+j)=0;      /* these should all be equal */      ref = hashlittle(b, len, (uint32_t)1);      *(b+i)=(uint8_t)~0;      *(b-1)=(uint8_t)~0;      x = hashlittle(b, len, (uint32_t)1);      y = hashlittle(b, len, (uint32_t)1);      if ((ref != x) || (ref != y))       {printf("alignment error: %.8x %.8x %.8x %d %d\n",ref,x,y,               h, i);      }    }  }}/* check for problems with nulls */ void driver4(){  uint8_t buf[1];  uint32_t h,i,state[HASHSTATE];  buf[0] = ~0;  for (i=0; i<HASHSTATE; ++i) state[i] = 1;  printf("These should all be different\n");  for (i=0, h=0; i<8; ++i)  {    h = hashlittle(buf, 0, h);    printf("%2ld  0-byte strings, hash is  %.8x\n", i, h);  }}void driver5(){  uint32_t b,c;  b=0, c=0, hashlittle2("", 0, &c, &b);  printf("hash is %.8lx %.8lx\n", c, b);   /* deadbeef deadbeef */  b=0xdeadbeef, c=0, hashlittle2("", 0, &c, &b);  printf("hash is %.8lx %.8lx\n", c, b);   /* bd5b7dde deadbeef */  b=0xdeadbeef, c=0xdeadbeef, hashlittle2("", 0, &c, &b);  printf("hash is %.8lx %.8lx\n", c, b);   /* 9c093ccd bd5b7dde */  b=0, c=0, hashlittle2("Four score and seven years ago", 30, &c, &b);  printf("hash is %.8lx %.8lx\n", c, b);   /* 17770551 ce7226e6 */  b=1, c=0, hashlittle2("Four score and seven years ago", 30, &c, &b);  printf("hash is %.8lx %.8lx\n", c, b);   /* e3607cae bd371de4 */  b=0, c=1, hashlittle2("Four score and seven years ago", 30, &c, &b);  printf("hash is %.8lx %.8lx\n", c, b);   /* cd628161 6cbea4b3 */  c = hashlittle("Four score and seven years ago", 30, 0);  printf("hash is %.8lx\n", c);   /* 17770551 */  c = hashlittle("Four score and seven years ago", 30, 1);  printf("hash is %.8lx\n", c);   /* cd628161 */}int main(){  driver1();   /* test that the key is hashed: used for timings */  driver2();   /* test that whole key is hashed thoroughly */  driver3();   /* test that nothing but the key is hashed */  driver4();   /* test hashing multiple buffers (all buffers are null) */  driver5();   /* test the hash against known vectors */  return 1;}#endif  /* SELF_TEST */

0 0