Bayesian Networks Independencies

Independence

1. For events α,β, P⊨α⊥β if:
   
   • P(α,β)=P(α)P(β)
   • P(α|β)=P(α)
   • P(β|α)=P(β)
2. For random variables X,Y,P⊨X⊥Y if:
   
   • P(X,Y)=P(X)P(Y)
   • P(X|Y)=P(X)
   • P(Y|X)=P(Y)

Conditional Independence

For (sets of) random variables X,Y,Z, P⊨(X⊥Y|Z) if:
- P(X,Y|Z)=P(X|Z)P(Y|Z)
- P(X|Y,Z)=P(X|Z)
- P(Y|X,Z)=P(Y|Z)
- P(X,Y|Z)∝ϕ1(X,Z)ϕ2(Y,Z)

d-seperated

X and Y are d-seperated in G given Z if there is no active trail in G between X and Y given Z, d-sepG(X,Y|Z)

Factorization ⇒ Independence: BNs

• Theorem: If P factorized over G, and d-sepG(X,Y|Z), then P satisfies (X⊥Y|Z)
• Any node is d-seperated from its non-descendants given its parents.
• If P factorizes over G, then in P, any variable is independent of its non-descendants given its parents.
• I-maps
  
  • d-separation in G⇒P satisfies corresponding independence statement
    
    I(G)={(X,Y|Z):d-sepG(X,Y|Z)}
  • Definition: If P satisfies I(G), we say that G is an I-map (independency map) of P.
• Theorem: If P factorized over G, then G is an I-map of P.

Independence ⇒ Factorization

• Theorem: If G is an I-map for P, then P factorized over G.

Summary

• Two equivalent views of graph structure
  
  • Factorization: G allows P to be represented
  • I-map: Independencies encoded in G hold in P
• If P factorizes over a graph G, we can read from the graph independencies that must hold in P (an independency map)