hash function比较(一)

来源:互联网 发布:知天下事是什么动物 编辑:程序博客网 时间:2024/06/14 04:54

由于工作需要,针对千万级别的数据,使用stl::map着实存在着效率问题,最后使用boost::unordered_map替代前者,发现效率上有很大的提升,但是还是无法达到我们的需求;

stl::map  底层算法:B+tree 实现

boost::unordered_map 底层算法:hash 实现

所以可能要针对不同的数据类型编写hash function来优化查找和插入的效率,自己编写,着实没有这个实力,只有在google上来寻找先辈们的精妙算法来借鉴:

常用字符串哈希函数有BKDRHash,APHash,DJBHash,JSHash,RSHash,SDBMHash,PJWHash,ELFHash等等;

有前辈做了评测:如下


ash函数数据1数据2数据3数据4数据1得分数据2得分数据3得分数据4得分平均分BKDRHash20477448196.5510090.9582.0592.64APHash23475449396.5588.4610051.2886.28DJBHash22497547496.5592.31010083.43JSHash14476150610084.6296.8317.9581.94RSHash10486150510010051.5820.5175.96SDBMHash32484950493.192.3157.0123.0872.41PJWHash302648785130043.89021.95ELFHash302648785130043.89021.95


其中数据1为100000个字母和数字组成的随机串哈希冲突个数。数据2为100000个有意义的英文句子哈希冲突个数。数据3为数据1的哈希值与1000003(大素数)求模后存储到线性表中冲突的个数。数据4为数据1的哈希值与10000019(更大素数)求模后存储到线性表中冲突的个数。

各种hash function的C 语言实现:

unsigned int SDBMHash(char *str){unsigned int hash = 0; while (*str){// equivalent to: hash = 65599*hash + (*str++);hash = (*str++) + (hash << 6) + (hash << 16) - hash;} return (hash & 0x7FFFFFFF);} // RS Hash Functionunsigned int RSHash(char *str){unsigned int b = 378551;unsigned int a = 63689;unsigned int hash = 0; while (*str){hash = hash * a + (*str++);a *= b;} return (hash & 0x7FFFFFFF);} // JS Hash Functionunsigned int JSHash(char *str){unsigned int hash = 1315423911; while (*str){hash ^= ((hash << 5) + (*str++) + (hash >> 2));} return (hash & 0x7FFFFFFF);} // P. J. Weinberger Hash Functionunsigned int PJWHash(char *str){unsigned int BitsInUnignedInt = (unsigned int)(sizeof(unsigned int) * 8);unsigned int ThreeQuarters= (unsigned int)((BitsInUnignedInt  * 3) / 4);unsigned int OneEighth= (unsigned int)(BitsInUnignedInt / 8);unsigned int HighBits = (unsigned int)(0xFFFFFFFF) << (BitsInUnignedInt - OneEighth);unsigned int hash = 0;unsigned int test = 0; while (*str){hash = (hash << OneEighth) + (*str++);if ((test = hash & HighBits) != 0){hash = ((hash ^ (test >> ThreeQuarters)) & (~HighBits));}} return (hash & 0x7FFFFFFF);} // ELF Hash Functionunsigned int ELFHash(char *str){unsigned int hash = 0;unsigned int x= 0; while (*str){hash = (hash << 4) + (*str++);if ((x = hash & 0xF0000000L) != 0){hash ^= (x >> 24);hash &= ~x;}} return (hash & 0x7FFFFFFF);} // BKDR Hash Functionunsigned int BKDRHash(char *str){unsigned int seed = 131; // 31 131 1313 13131 131313 etc..unsigned int hash = 0; while (*str){hash = hash * seed + (*str++);} return (hash & 0x7FFFFFFF);} // DJB Hash Functionunsigned int DJBHash(char *str){unsigned int hash = 5381; while (*str){hash += (hash << 5) + (*str++);} return (hash & 0x7FFFFFFF);} // AP Hash Functionunsigned int APHash(char *str){unsigned int hash = 0;int i; for (i=0; *str; i++){if ((i & 1) == 0){hash ^= ((hash << 7) ^ (*str++) ^ (hash >> 3));}else{hash ^= (~((hash << 11) ^ (*str++) ^ (hash >> 5)));}} return (hash & 0x7FFFFFFF);}

文章资料来自:http://www.byvoid.com/blog/string-hash-compare/


原创粉丝点击