MIT Introduction to Algorithms Record-8

Weakness of hashing: For any choice of hash function, it always exist bad set of keys that all hash to same slot.
Idea: Choose hash function at random,independently from the keys.

---

Universal Hashing

Definition1: Let U be a universal of keys, and let H be a finite collection of hash functions,mapping U to {0,1,...,m−1}.

So H is universal:

if ∀x,y∈U,where x≠y,then {h∈H,h(x)=h(y)}=|H|m

.
I.e. if h is chosen randomly from H, the probability of collision between x and y is 1x.

Theorem1:
Choose h randomly from H, suppose hashing n keys into m slots in Table T, then for a given key x,its expect number of collisions with x is:

E[#collisions with x]<nm

Proof:
Let cx be random variable, the total number collisions of keys in T with x, and let

cxy={1,0,if h(x)=h(y)otherwise

Note: E[cxy]=1m and Cx=∑y∈T−{x}cxy ,and y is the element not equal with x in Table T.

So:

E[Cx]=E[∑y∈T−{x}cxy]=∑y∈T−{x}E[cxy]=∑y∈T−{x}1m=n−1m

---

Constructing a universal hash function

Let m be prime, decompose key k into r+1 digits, so

k=⟨k0,k1,...,kr⟩ where 0≤ki≤m−1

in here we treating k as an r+1 base m number.
Now we’re going to pick an a at random which we’re also going to look at as a base m number, so pick a=⟨a0,a1,...,ar⟩, each ai is chosen randomly from {0,1,...,m−1}, so the hash function as below:

Definition2:

ha(k)=(∑i=0raiki)%m

We want to know how big is the set of hash function here? how many different hash functions do I have in this set?
Conclusion:

|H|=mr+1

Explanation:
Because it have m choices for each ai (0≤ai≤m−1), and for a vector has r+1 elements from a=⟨a0,a1,...,ar⟩.

Theorem2: H is universal.

Proof:
Let x=⟨x0,x1,...,xr⟩ , y=⟨y0,y1,...,yr⟩ be distinct keys and they differ m at least one digit. They could differ in any one of these digits, in here we assume they differ in position 0.

Question: How many hash functions in universal ha(x)∈H do x and y collide?

It must have ha(x)=ha(y) if they collide.

⇒(∑i=0raixi)%m≡(∑i=0raiyi)%m

⇒(∑i=0raixi) ≡(∑i=0raiyi)(%m)//do module m after everything is being done. 

⇒∑i=0rai(xi−yi)≡0(%m)

⇒a0(x0−y0)+∑i=1rai(xi−yi)≡0(%m)

⇒a0(x0−y0)=−∑i=1rai(xi−yi)(%m)

since x0≠y0, so ∃(x0−y0)−1 to make following formula is true according with Number Theory Fact.

a0=(−∑i=1rai(xi−yi))(x0−y0)−1

Number Theory Fact:

let m be prime, for any z∈Zm (Zm are intergers module m), so for any z is not congruent to 0, there exists a unique z inverse in Zm, such that if I multiply z times the inverse, it produces something congruent to one mod m.
I.e.

$z≢0,∃ unique z−1∈Zm ⇒$zz−1≡1 (mod m)

.

Conclusion:
Thus, for any choice of a1,a2,...,ar exactly 1 of the m choices for a0 cause x and y to collide, and no collision for other m−1 choices for a0.
So the number of ha’s that cause x, y to collide:

⇒m∗m...∗m∗1=mr=|H|m

because a1 has m choices, and m choices for a2 …, but only 1 choice for a0 if want to cause collision.

---

Perfect Hashing

Situation: Given n keys construct a static hash table of size m=O(n), such that search takes O(1) time in the worst case.

Idea: Use a 2-level scheme with universal hashing at both levels. And the idea is that we’re going to do it in such a way that we have no collisions at level 2 and we’ll take any collides at level 1.
If ni items hash to level-1 slot i, then use mi=n2i slots in level-2 table si.

---

Level-2 Analysis

Theorem: Hash n keys into m=n2 slots using random h in universal H, we can get

E[#collisions]<12

Proof: Probability 2 given keys collide under h is 1m=1n2.

E[#collisions]=(2n)∗1n2

⇒=n(n−1)21n2<12

Note: (2n)=C2n.

Markov inequality

For randomly variable x≥0, Pr{x≥t} ≤ E[x]t.

Proof:

E[x]=∑x=0∞x∗Pr{X=x} ≥∑x=t∞x∗Pr{X=x}

⇒≥∑x=t∞t∗Pr{X=x}=t∗Pr{X≥t}

Corollary:

Pr{no collisions} ≥ 12

Now we can use the Markov inequality theorem to prove that the corollary is correct.

Proof:

Pr{≥1 collisions}≤E[#collisions]1<12

⇒Pr{no collisions}=1−Pr{≥1 collisions}≥12

Conclusion: So we can know that to find a good level-2 hash function, just test a few at random, and we’ll find one quickly since at least half will work.

Analysis of storage

For level-1 choose m=n, and let ni be the random variable for the number of keys that hash to slot i in table T, use mi=n2i slots in each level-2 table si, so

E[total storage]=n+E[∑i=0m−1θ(n2i)]=θ(n)

Note: we can get E[∑m−1i=0θ(n2i)]=θ(n) by bucket sort analysis.