Stanford University, Spring 2016, STATS 205

Exchangeablility

  • The idea is that before we observe 20 flips of a coin
  • and if we think of 17 heads and 3 tails as a possible outcome
  • then exchangeablility means that we don't think of the positions that the 3 heads can occupy as being special

Exchangeablility

  • An infinite sequence \(X_1,\dots,X_n,\dots\) of random variables is said to be exchangeable if for all \(n = 2,3,\dots\), \[X_1,\dots,X_n \overset{d}{=} X_{\pi(1)},\dots,X_{\pi(n)} \text{ for all } \pi \in S(n)\] where \(S(n)\) is the group of permutations of \(\{1,\dots,n\}\)
  • For example, for a binary sequence, we may have: \[p(1,1,0,0,0,1,1,0) = p(1,0,1,0,1,0,0,1)\]
  • If \(X_1,\dots,X_n,\dots\) are independent and identically distributed, they are exchangeable, but not conversely

Polya's Urn

  • Consider an urn with \(b\) black balls and \(w\) white balls
  • Draw a ball at random and note its colour
  • Replace the ball together with \(a\) balls of the same colour
  • Repeat the procedure ad infinitum
  • Let \(X_i =1\) if the \(i\)th draw yields a black ball and \(X_i = 0\) otherwise
  • The sequence \(X_1,\dots,X_n,\dots\) is exchangeable: \[ \begin{align} p(1,1,0,1) & = \frac{b}{b+w} \frac{b+a}{b+w+a} \frac{w}{b+w+2a} \frac{b+2a}{b+w+3a} \\ & = \frac{b}{b+w} \frac{w}{b+w+a} \frac{b+a}{b+w+2a} \frac{b+2a}{b+w+3a} = p(1,0,1,1) \end{align} \]
  • \(X_1,\dots,X_n,\dots\) are not independent