A Proof of the Rule of Succession In this section we shall show an explicit proof for Laplace's "rule of succession," which states: "given that you find and flip a biased coin $n$ times, resulting in $s$ heads, the probability that the next flip will turn up heads is $\frac{s+1}{n+2}$". More formally: Let each $X_i$ for $i\in\mathbb{Z}$ be an independent Bernoulli variable with identical success parameter $p$, where $p$ itself has a uniform distribution $p\sim U(0,1)$. Let $X = \sum_{i=1}^{n} X_i$. Then: $$\displaystyle P\left(X_{n+1} = 1\ |\ X = s\right) = \frac{s+1}{n+2}$$ Bayes' Theorem on conditional probabilities states: \begin{equation} P(X_{n+1}=1\ |\ X=s) = \dfrac{P(X_{n+1} = 1\ \&\ X = s)}{P(X = s)}\end{equation} Note that for any fixed $p$, the probability of a binomial random variable $X\sim B(n,p)$ equalling $s = 0,1,2,\dots, n$, is $\displaystyle \binom{n}{s}p^s(1-p)^{n-s}$ (since we require $s$ successes with probability $p$ each, and $n-s$ failures with probability $1-p$ each, and there are $\binom{n}{s} =$ "$n$ choose $s$" ways to order the successes and failures amongst $n$ trials). We then condition on $p$ to obtain: \begin{equation}P(X = s) = \int_0^1\binom{n}{s}p^s(1-p)^{n-s} dp\end{equation} (Note we have implictly assumed the uniformity of $p$ in this integral by using a density function of $1$ - each $p$ in the interval $[0,1]$ contributes equally in weight to the overall probability.) Similarly for any fixed $p$ the event $ (X_{n+1} = 1\ \&\ X = s)$ occurs with probability $\displaystyle p\cdot\binom{n}{s}p^s(1-p)^{n-s}$ as we require both the $(n+1)$-th trial to be a success as well as $X=s$. Then similarly: \begin{equation} P(X_{n+1} = 1\ \&\ X = s) = \int_0^1\binom{n}{s}p^{s+1}(1-p)^{n-s}dp\end{equation} Both (2) and (3) are special cases of Euler's beta integral, defined for real $x,y>0$ as: $$B(x,y) = \int_0^1 p^{x-1}(1-p)^{y-1} dp$$ so that (1) becomes: \begin{equation} P(X_{n+1}=1\ |\ X=s) = \frac{\binom{n}{s}B(s+2,n-s+1)}{\binom{n}{s}B(s+1,n-s+1)} =\frac{B(s+2,n-s+1)}{B(s+1,n-s+1)} \end{equation} Observe first that for any integer $k\geq 1$: \begin{equation}B(k,1) = B(1,k) = \int_0^1 p^{k-1}dp = \frac{1}{k}\end{equation} and then that by integration by parts, we have for integers $x,y\geq1$: \begin{align} B(x,y)& = \int_0^1 p^{x-1}(1-p)^{y-1} dp\notag\\ &=\frac{1}{x}p^x(1-p)^{y-1}\bigg|_0^1 - \left[\int_0^1 \left(\frac{1}{x}p^x\right)(-(y-1))(1-p)^{y-2}dp\right]\notag\\ &= 0 + \frac{y-1}{x}\int_0^1 p^x(1-p)^{y-2}dp \notag\\ &=\frac{y-1}{x} B(x+1,y-1) \end{align} We then argue via induction from (5) and (6) that for integers $x,y\geq 1$: \begin{align} B(x,y) &= \frac{y-1}{x}B(x+1,y-1) \notag\\ &=\frac{(y-1)(y-2)}{x(x+1)}B(x+2,y-2) \notag \\ &= \dots \dots\dots \notag\\ &=\frac{(y-1)(y-2)\dots(1)}{x(x+1)\dots(x+y-2)}B(x+y-1,1) \notag\\ & =\frac{(y-1)!(x-1)!}{(x+y-1)!} \end{align} Hence finally continuing from (4), we obtain: \begin{align} P(X_{n+1}=1\ |\ X=s) &= \frac{(s+1)!(n-s)!}{(n+2)!}\frac{(n+1)!}{s!(n-s)!} \notag\\ &= \frac{s+1}{n+2} \end{align} as desired.