{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Probability \n",
"\n",
"Probability allows us to talk about uncertainty, in certain terms. Once, we are able to quantify uncertainties, we can deterministically make deductions about the future. The language of statistics also allows us to talk about uncertainty in uncertain but tractable terms that we can reason about."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Random Variable"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"A **random variable** is a mathematical formalization of an abstract quantity that has some degree of uncertainty associated with the values it may take on. \n",
"The set of all possible values that a random variable can take on is called its **range**. \n",
"\n",
"A random variable is very much similar to a variable in computer programming. **In the context of `pandas`, a random variable is a column or feature in a `DataFrame`.**\n",
"\n",
"Just as numerical features in a `DataFrame` can be either discrete or continuous, random variables can also be either discrete or continuous. The two types require different mathematical formalizations as we will see later.\n",
"\n",
"Random variables are usually denoted by capital letters, such as $X$ or $Y$. The values that a random variable can take on are denoted by lower case letters, such as $x$ or $y$.\n",
"\n",
"It is important to note that in the real world, it is _often impossible to obtain the range_ of a random variable. Since most real-world datasets are **samples**, **`df['X'].unique()` does not necessarily give us the range of $X$**.\n",
"\n",
"It is also important to remember that **$x$ is a single value** but **$X$ is a collection of values** (i.e. `pd.Series`). "
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"In the example below, $C$ (coin) and $D$ (dice) are two random variables. "
]
},
{
"cell_type": "code",
"execution_count": 68,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
C
\n",
"
D
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
T
\n",
"
1
\n",
"
\n",
"
\n",
"
1
\n",
"
T
\n",
"
3
\n",
"
\n",
"
\n",
"
2
\n",
"
T
\n",
"
3
\n",
"
\n",
"
\n",
"
3
\n",
"
H
\n",
"
2
\n",
"
\n",
"
\n",
"
4
\n",
"
T
\n",
"
1
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" C D\n",
"0 T 1\n",
"1 T 3\n",
"2 T 3\n",
"3 H 2\n",
"4 T 1"
]
},
"execution_count": 68,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import pandas as pd \n",
"\n",
"data = pd.read_csv('../data/experiment.csv')\n",
"data.head()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"The ranges of $C$ and $D$ are $\\{H, T\\}$ and $\\{1, 2, 3, 4, 5, 6\\}$ respectively. It is worth repeating for emphasis that the ranges of the two variables is independent of observed data, since the observed data is a limited sample."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Experiment, Outcome $\\omega$ and Sample Space $\\Omega$\n",
"\n",
"An **outcome**, denoted by $\\omega$, is the set of values that one or more random variables take on as a result of an **experiment**.\n",
"\n",
"An **experiment** is a process that yields outcomes out of set of all possible outcomes. \n",
"\n",
"The **sample space**, denoted by $\\Omega$, is the set of all possible outcomes. \n",
"\n",
"The important operative word here is _\"possible\"_. The sample space is _not_ the set of all _observed_ outcomes, the set of all possible outcomes.\n",
"\n",
"If an experiment involves two random variables say $X$ and $Y$ which _can_ take on $n$ possible values (i.e. $~\\text{range}_X = \\{x_1, x_2, \\ldots, x_n\\})$ and $m$ possible values (i.e. $~\\text{range}_Y = \\{y_1, y_2, \\ldots, y_m\\}$) respectively, then the sample space $\\Omega$ is the set of all possible combinations of $x_i$ and $y_j$ and is of size $n \\times m$. \n",
"\n",
" \n",
"
\n",
" \n",
"\n",
"In other words, the sample space is the **cross product of the ranges of all random variables** involved in the experiment.\n",
"\n",
"In our example, the experiment is the act of tossing a coin and rolling a dice. \n",
"\n",
"Each row in the data is an outcome $w_i$ from the set of all possible outcomes $\\Omega$. \n",
"\n",
"$C$ variable can take on two ($n=2$) values: $\\{H, T\\}$ and $D$ variable can take on six $m=6$ value: $\\{1, 2, 3, 4, 5, 6\\}$. This means that the sample space $\\Omega$ is of size $n \\times m = 2 \\times 6 = 12$.\n",
"\n",
"However, the observed outcomes are only 11, as shown below. "
]
},
{
"cell_type": "code",
"execution_count": 73,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
C
\n",
"
D
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
H
\n",
"
1
\n",
"
\n",
"
\n",
"
1
\n",
"
H
\n",
"
2
\n",
"
\n",
"
\n",
"
2
\n",
"
H
\n",
"
3
\n",
"
\n",
"
\n",
"
3
\n",
"
H
\n",
"
4
\n",
"
\n",
"
\n",
"
4
\n",
"
H
\n",
"
5
\n",
"
\n",
"
\n",
"
5
\n",
"
H
\n",
"
6
\n",
"
\n",
"
\n",
"
6
\n",
"
T
\n",
"
1
\n",
"
\n",
"
\n",
"
7
\n",
"
T
\n",
"
2
\n",
"
\n",
"
\n",
"
8
\n",
"
T
\n",
"
3
\n",
"
\n",
"
\n",
"
9
\n",
"
T
\n",
"
4
\n",
"
\n",
"
\n",
"
10
\n",
"
T
\n",
"
5
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" C D\n",
"0 H 1\n",
"1 H 2\n",
"2 H 3\n",
"3 H 4\n",
"4 H 5\n",
"5 H 6\n",
"6 T 1\n",
"7 T 2\n",
"8 T 3\n",
"9 T 4\n",
"10 T 5"
]
},
"execution_count": 73,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"data.groupby(['C', 'D']).count().reset_index()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"This means that the sample space $\\Omega$ is not the set of all observed outcomes. This is despite the fact that many observed outcomes are observed more than once. The missing outcome, that is never observed, is $w_{12} = (T, 6)$."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Probability Model $P(X)$"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Probability model is a function that assigns a probability score $P(\\omega_i)$ to each possible outcome $\\omega_i$ for every $\\omega_i \\in \\Omega$ such that \n",
"\n",
"\n",
"\n",
"$$ 0 \\lt P(\\omega_i) \\lt 1 ~~~\\text{and}~~~ \\sum_{\\omega \\in \\Omega} P(\\omega_i) = 1$$\n",
"\n",
"For example, if we have a random variable $D$ for rolling a die, the probability model assigns a probability to each number that we can roll. The probability model is usually denoted by $P(\\omega_i)$ or $P(D=d)$\n",
"\n",
"\n",
"$\\omega$ | $D$ | $P(D=d)$ |\n",
":-------:|:----:|:-----:|\n",
"$\\omega_1$ | $1$ | $P(D=1)$ |\n",
"$\\omega_2$ | $2$ | $P(D=2)$ |\n",
"$\\omega_3$ | $3$ | $P(D=3)$ |\n",
"$\\omega_4$ | $4$ | $P(D=4)$ |\n",
"$\\omega_5$ | $5$ | $P(D=5)$ |\n",
"$\\omega_6$ | $6$ | $P(D=6)$ |\n",
"\n",
"such that $0 \\leq P(D=d) \\leq 1$ and and $\\sum_{d \\in D} P(d=D) = 1$."
]
},
{
"cell_type": "code",
"execution_count": 80,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
D
\n",
"
P(D)
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
1
\n",
"
0.166667
\n",
"
\n",
"
\n",
"
1
\n",
"
2
\n",
"
0.166667
\n",
"
\n",
"
\n",
"
2
\n",
"
3
\n",
"
0.166667
\n",
"
\n",
"
\n",
"
3
\n",
"
4
\n",
"
0.166667
\n",
"
\n",
"
\n",
"
4
\n",
"
5
\n",
"
0.166667
\n",
"
\n",
"
\n",
"
5
\n",
"
6
\n",
"
0.166667
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" D P(D)\n",
"0 1 0.166667\n",
"1 2 0.166667\n",
"2 3 0.166667\n",
"3 4 0.166667\n",
"4 5 0.166667\n",
"5 6 0.166667"
]
},
"execution_count": 80,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"fair_die = pd.read_csv('../data/fair_die.csv')\n",
"fair_die"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"The code cell above shows the probability model for the random variable $D$ for a fair die in our examples, where each number has a probability of $\\frac{1}{6}$."
]
},
{
"cell_type": "code",
"execution_count": 350,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAFwCAYAAACGt6HXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAeRElEQVR4nO3dcbwdZX3n8c/XIEWRiC2xQBIFaqpNu7Xyiohrq67aFtA2dtftQqsobc2ygkKrdanrrrq1trtVa90iKQsoVJR2UXdTTUVbpdau0kSgKiDbNEUTgeUqAoJWiPz2j5l0D4dzb869ZO5DLp/363VeOTPzPDO/Offc753zzMxJqgpJ0uJ7WOsCJOmhygCWpEYMYElqxACWpEYMYElqxACWpEYM4AaSVJInLLDvDUmeN8uyn0hy/aS2SV6X5LyFVbzHmo7o92m/fvrPkrx0L6171n3aS+u/Jsmz99b6ptxmkrw7yTeS/M1eWN/Lknx6ZPqf3l9JNib5jw90Gw9Uku9P8qkk30zytina35nkqAewvTcmeW///HH9+pYtdH1D2a91AfuKJDcA3w98F7gL2Ay8sqrubFnXqKr6K+CJsyx7y+7nSY4A/gF4eFXtGqCO46dpl6SANVW1bY51zbpP85XkPcDOqnr9yPp/eG+se55+HPhJYFVV3TXkhqrq1CHXPw8bgK8By2uKmw+q6lF7a8NV9RVgr61vb/IIeH5+pn9jHA08FXj9eIPdR4F64Jbwa/l44IZpw3eJvA6PB66dJnz35MF4JLtQBvACVNVXgT8DfgT+6SPfaUn+Dvi7ft7Lk2xLcmuSTUkOH1vNCUm2J/lakt9N8rC+3w8k+USSr/fLLk5y8Fjfpya5tv8I++4kB/R9n51k56SaRz+SAZ/q/72t/2j2rL7OfzbS/rFJvp1kxYR1LUvy1r6+7cDzx5ZfnuRX+udPSPKXSW7v2/9xP393DX/b1/Bvdtef5N8nuRl49yz7NNv+3+ej+MjP5glJNgC/CLy2396f9stHh2m+J8k7ktzYP96R5HtGX9skr05yS5Kbkpwy6bXu2x/e/9xv7d8HL+/n/zJwHvD0vo43Tej7siR/neT3ktwKvDHJo5NclGQmyZeTvH73e2YuSd6T5M3T7EOS70vyp0nuSLIlyZvHX8+xdf+PJDf3P9tPJZn4aaL/5PHSkdf+eUmOSfKZJLf1dfxBkv1H+owOo7wnyTlJNie5C/gXE7ZxZP8++2aSjwOHjCwbHyJ7dJLz++1+td/PJqFuAC9AktXACcBVI7NfCDwNWJvkOcBvAz8PHAZ8GbhkbDU/B6yjO5peD/zS7tX3fQ8HfghYDbxxrO8vAj8N/ADwg0w4Et+DZ/b/HlxVj6qqv+zre/FIm5OAP6+qmQn9Xw68AHhKvw8vmmNbvwl8DHgMsAr4bwBVtbuGJ/c1/HE/fSjwvXRHTBtmWee897+qzgUuBv5rv72fmdDsPwDHAj8GPBk4ZmzdhwKPBlYCvwycneQxs2zy/cBOup/ji4C3JHluVZ0PnAp8pq/jDbP0fxqwHXgs8Ft0r9ujgaOAZwEnA7P+AZjDXPtwNt3w2qF0gbmncfw/A9b0NV5J9/reT1W9jPu+9n9ON5T3q3RB+XTgucAr5tjWL9C9DgcBk/4ovA/4XL++39xD7RcCu4An0L2Hfwr4lTnaD6eqfEzxAG4A7gRuowvUdwGP6JcV8JyRtufTvdl2Tz8KuAc4YqT9cSPLXwH8xSzbfSFw1Vgdp45MnwD8ff/82XRjnKNtn9c/fyPw3v75EX0N+420fRqwA3hYP70V+PlZavrEWA0/Nbo+4HLgV/rnFwHn0o13jq+ngCeMTD8buBs4YGze+D7Ntv8vAz492zaA9wBvnvBz3f0a/T1wwsiyn6YbKthdx7fHXrNbgGMn7NdquoA5aGTebwPvma3Osf4vA74yMr0M+A6wdmTevwUun7S+2fZ5rn3ot3EP8MSRZW+eq86xmg/ut/voWZbf77UfW34m8KE59uGiOfo+ji5QDxyZ9z4mvN/pzuN8h/53t19+EvDJafZzbz88Ap6fF1bVwVX1+Kp6RVV9e2TZjpHnh9OFNADVnaj7Ot1Rx6T2X+777P7of0n/0egO4L2MfJyaq+8DUVVX0B39PCvJk+iODjbN0vzwCTXM5rV0R/V/k+6Kg1+aoy3ATFX94x7a7PX9793n5zZh3V+v+560/BaTT+4cDtxaVd8cW9fKCW1nM7qPhwD7T6htPuvbbbZ9WEEXUKPbHX1+H+mGoX4nyd/379MbRmrdoyQ/mOTD/RDGHcBb9tB31lroXu9v1H3H1Gd7Tz4eeDhwUz/8cRvwh3RH8YvOAN57Rk8u3Ej3gwYgyYHA9wFfHWmzeuT54/o+0B0pFfCjVbWcblggY9uare9Cah11Yb+9lwCXzhGEN02oYfKGqm6uqpdX1eF0R23vytyX4E1zkma2/b8LeOTuBUkOnee67/NzY2Gv7e71fG+Sg8bW9dVZ2k8yWuvX6I5Ox2ubz/r2ZIbuKHLVyLzVs7SFbkhgPfA8uiGNI/r54+/V2ZwDfInuKpjlwOv20Heun91NwGP637PdZntP7qA7Aj6kP5g6uKqWV5urYQzggbwPOCXJj/Uncd4CXFFVN4y0+fUkj+nHk88Ado+BHkQ/1JFkJfDrE9Z/WpJVSb6X7o37xxPazGUGuJduPHHUH9GNTb+YbuhgNn8CvKqv4THAWbM1TPKvk+z+pf4G3S/Sd/vp/zuhhmnMtv9/C/xw/7ofwP3Hzve0vfcDr0+yIskhwH+i+wQyL1W1A/jfwG8nOSDJj9KNt04cI51ifd+le81/K8lBSR4P/NpCatvDNj5Id8Lvkf2noJPn6HIQXZB9ne6P3lvmaDtb/zuAO/tt/bv5V92pqi/TDZm9Kcn+SX4cmDTGT1XdRHdO4m1Jlid5WLoT389a6PYfCAN4AFX1F8B/BD5A99f5B4ATx5r9L7qTBlcDH6EbNwZ4E92Judv7+R+csIn30b2JtvePN8+zvm/RndD46/5j2LH9/J10J1MK+Ks5VvHfgcvoAu/KWWrc7anAFUnupBvSOKOq/qFf9kbgwr6Gn5/HLkzc/6r6P8B/Bv6c7mqU8ZM159OdJL0tyf+csN430/0ifx74Qr9v83ptR5xEd1R4I/Ah4A1V9fEFrgvglXRH+Nvp9ut9wAUPYH2TnE53NHsz3R/j99OF7CQX0X3M/ypwLfDZeW7rNXRH0d+kez/N9yBi3C/Qnce4FXgDcx9AnEw3pHMt3UHBpXQnyxdd+kFoCYAkFwA31sjNCnpoSvJfgEOraq/c1aj7WwoXeGsvSXeH3L+kuzRHDzH9UMD+dEf/T6UbNmlzedZDhEMQAiDJbwJfBH53ZIhADy0H0Q0n3UU35vw2uqEyDcQhCElqxCNgSWpkSY0BH3fccfXRj360dRmSNG7iNc5L6gj4a1/7WusSJGlqSyqAJWlfYgBLUiMGsCQ1YgBLUiMGsCQ1YgBLUiMGsCQ1YgBLUiMGsCQ1YgBLUiMGsCQ1YgBLUiMGsCQ1sqS+jnLcR665oXUJc3r+Dx8xVbsH8364Dw8OS2EfYLr9WAr7sJtHwJLUiAEsSY0YwJLUiAEsSY0YwJLUiAEsSY0YwJLUiAEsSY0MGsBJjktyfZJtSc6asPxJST6T5DtJXjO27OAklyb5UpLrkjx9yFolabENdidckmXA2cBPAjuBLUk2VdW1I81uBV4FvHDCKn4f+GhVvSjJ/sAjh6pVkloY8gj4GGBbVW2vqruBS4D1ow2q6paq2gLcMzo/yXLgmcD5fbu7q+q2AWuVpEU3ZACvBHaMTO/s503jKGAGeHeSq5Kcl+TASQ2TbEiyNcnWmZmZB1axJC2iIQM4E+bVlH33A44GzqmqpwB3AfcbQwaoqnOral1VrVuxYsXCKpWkBoYM4J3A6pHpVcCN8+i7s6qu6KcvpQtkSVoyhgzgLcCaJEf2J9FOBDZN07GqbgZ2JHliP+u5wLVzdJGkfc5gV0FU1a4kpwOXAcuAC6rqmiSn9ss3JjkU2AosB+5NciawtqruAF4JXNyH93bglKFqlaQWBv1C9qraDGwem7dx5PnNdEMTk/peDawbsj5Jask74SSpEQNYkhoxgCWpEQNYkhoxgCWpEQNYkhoxgCWpEQNYkhoxgCWpEQNYkhoxgCWpEQNYkhoxgCWpEQNYkhoxgCWpEQNYkhoxgCWpEQNYkhoxgCWpEQNYkhoxgCWpEQNYkhoxgCWpEQNYkhoxgCWpEQNYkhoZNICTHJfk+iTbkpw1YfmTknwmyXeSvGbC8mVJrkry4SHrlKQWBgvgJMuAs4HjgbXASUnWjjW7FXgV8NZZVnMGcN1QNUpSS0MeAR8DbKuq7VV1N3AJsH60QVXdUlVbgHvGOydZBTwfOG/AGiWpmSEDeCWwY2R6Zz9vWu8AXgvcO1ejJBuSbE2ydWZmZt5FSlIrQwZwJsyrqTomLwBuqarP7altVZ1bVeuqat2KFSvmW6MkNTNkAO8EVo9MrwJunLLvM4CfTXID3dDFc5K8d++WJ0ltDRnAW4A1SY5Msj9wIrBpmo5V9RtVtaqqjuj7faKqXjxcqZK0+PYbasVVtSvJ6cBlwDLggqq6Jsmp/fKNSQ4FtgLLgXuTnAmsrao7hqpLkh4sBgtggKraDGwem7dx5PnNdEMTc63jcuDyAcqTpKa8E06SGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJakRA1iSGjGAJamRQQM4yXFJrk+yLclZE5Y/KclnknwnyWtG5q9O8skk1yW5JskZQ9YpSS3sN9SKkywDzgZ+EtgJbEmyqaquHWl2K/Aq4IVj3XcBr66qK5McBHwuycfH+krSPm3II+BjgG1Vtb2q7gYuAdaPNqiqW6pqC3DP2PybqurK/vk3geuAlQPWKkmLbsgAXgnsGJneyQJCNMkRwFOAK2ZZviHJ1iRbZ2ZmFlKnJDUxZABnwrya1wqSRwEfAM6sqjsmtamqc6tqXVWtW7FixQLKlKQ2hgzgncDqkelVwI3Tdk7ycLrwvbiqPriXa5Ok5oYM4C3AmiRHJtkfOBHYNE3HJAHOB66rqrcPWKMkNTPYVRBVtSvJ6cBlwDLggqq6Jsmp/fKNSQ4FtgLLgXuTnAmsBX4UeAnwhSRX96t8XVVtHqpeSVpsgwUwQB+Ym8fmbRx5fjPd0MS4TzN5DFmSlgzvhJOkRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWrEAJakRgxgSWpk0ABOclyS65NsS3LWhOVPSvKZJN9J8pr59JWkfd1gAZxkGXA2cDywFjgpydqxZrcCrwLeuoC+krRPG/II+BhgW1Vtr6q7gUuA9aMNquqWqtoC3DPfvpK0rxsygFcCO0amd/bz9mrfJBuSbE2ydWZmZkGFSlILQwZwJsyrvd23qs6tqnVVtW7FihVTFydJrQ0ZwDuB1SPTq4AbF6GvJO0ThgzgLcCaJEcm2R84Edi0CH0laZ+w31ArrqpdSU4HLgOWARdU1TVJTu2Xb0xyKLAVWA7cm+RMYG1V3TGp71C1SlILgwUwQFVtBjaPzds48vxmuuGFqfpK0lLinXCS1IgBLEmNGMCS1IgBLEmNGMCS1IgBLEmNGMCS1IgBLEmNGMCS1MhUd8IlOQB4AfATwOHAt4EvAh/xFmFJWpg9BnCSNwI/A1wOXAHcAhwA/CDwO304v7qqPj9cmZK09ExzBLylqt44y7K3J3ks8Li9V5IkPTTsMYCr6iN7WH4L3VGxJGkepjoJl+SlSa5Mclf/2Jrk5KGLk6SlbJox4JOBM4FfA66k+++CjgZ+NwlVddGgFUrSEjXNEfArgJ+rqk9W1e1VdVtVfQL4V/0ySdICTBPAy6vqhvGZ/bzle7sgSXqomCaAv73AZZKkOUxzGdoPJZl0jW+Ao/ZyPZL0kDFVAA9ehSQ9BE0TwF+pqpqrQZLsqY0k6b6mGQP+ZJJXJrnP3W5J9k/ynCQXAi8dpjxJWrqmOQI+Dvgl4P1JjgK+QfddEMuAjwG/V1VXD1ahJC1R09yK/I/Au4B3JXk4cAjw7aq6beDaJGlJm+ZOuAOAU4EnAJ8HLqiqXUMXJklL3TRjwBcC64AvACcAbxu0Ikl6iJgmgNdW1Yur6g+BF9F9KftUkhyX5Pok25KcNWF5kryzX/75JEePLPvVJNck+WKS9/dH4pK0ZEwTwPfsfjKfoYcky4CzgeOBtcBJSdaONTseWNM/NgDn9H1XAq8C1lXVj9Cd8Dtx2m1L0r5gmqsgnpzkjv55gEf00wGqqmb7PohjgG1VtR0gySXAeuDakTbrgYv6a4g/m+TgJIeN1PaIJPcAjwRunM+OSdKD3R6PgKtqWVUt7x8HVdV+I8/n+jKelcCOkemd/bw9tqmqrwJvBb4C3ATcXlUfm7SRJBv67yfeOjMzs6fdkaQHjSH/V+RMmDd+t9zENkkeQ3d0fCTdfwJ6YJIXT9pIVZ1bVeuqat2KFSseUMGStJiGDOCdwOqR6VXcfxhhtjbPA/6hqmaq6h7gg8A/H7BWSVp0QwbwFmBNkiOT7E93Em3TWJtNwMn91RDH0g013EQ39HBskkcmCfBc4LoBa5WkRTfNSbgFqapdSU4HLqO7iuGCqromyan98o3AZrpri7cB3wJO6ZddkeRSuv8CaRdwFXDuULVKUguDBTBAVW2mC9nReRtHnhdw2ix93wC8Ycj6JKmlIYcgJElzMIAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqZFBAzjJcUmuT7ItyVkTlifJO/vln09y9Miyg5NcmuRLSa5L8vQha5WkxTZYACdZBpwNHA+sBU5Ksnas2fHAmv6xAThnZNnvAx+tqicBTwauG6pWSWphyCPgY4BtVbW9qu4GLgHWj7VZD1xUnc8CByc5LMly4JnA+QBVdXdV3TZgrZK06IYM4JXAjpHpnf28adocBcwA705yVZLzkhw4aSNJNiTZmmTrzMzM3qtekgY2ZABnwryass1+wNHAOVX1FOAu4H5jyABVdW5VrauqdStWrHgg9UrSohoygHcCq0emVwE3TtlmJ7Czqq7o519KF8iStGQMGcBbgDVJjkyyP3AisGmszSbg5P5qiGOB26vqpqq6GdiR5Il9u+cC1w5YqyQtuv2GWnFV7UpyOnAZsAy4oKquSXJqv3wjsBk4AdgGfAs4ZWQVrwQu7sN7+9gySdrnDRbAAFW1mS5kR+dtHHlewGmz9L0aWDdkfZLUknfCSVIjBrAkNWIAS1IjBrAkNWIAS1IjBrAkNWIAS1IjBrAkNWIAS1IjBrAkNWIAS1IjBrAkNWIAS1IjBrAkNWIAS1IjBrAkNWIAS1IjBrAkNWIAS1IjBrAkNWIAS1IjBrAkNWIAS1IjBrAkNWIAS1IjBrAkNTJoACc5Lsn1SbYlOWvC8iR5Z7/880mOHlu+LMlVST48ZJ2S1MJgAZxkGXA2cDywFjgpydqxZscDa/rHBuCcseVnANcNVaMktTTkEfAxwLaq2l5VdwOXAOvH2qwHLqrOZ4GDkxwGkGQV8HzgvAFrlKRmhgzglcCOkemd/bxp27wDeC1w70D1SVJTQwZwJsyradokeQFwS1V9bo8bSTYk2Zpk68zMzELqlKQmhgzgncDqkelVwI1TtnkG8LNJbqAbunhOkvdO2khVnVtV66pq3YoVK/ZW7ZI0uCEDeAuwJsmRSfYHTgQ2jbXZBJzcXw1xLHB7Vd1UVb9RVauq6oi+3yeq6sUD1ipJi26/oVZcVbuSnA5cBiwDLqiqa5Kc2i/fCGwGTgC2Ad8CThmqHkl6sBksgAGqajNdyI7O2zjyvIDT9rCOy4HLByhPkpryTjhJasQAlqRGDGBJasQAlqRGDGBJasQAlqRGDGBJasQAlqRGDGBJasQAlqRGDGBJasQAlqRGDGBJasQAlqRGDGBJasQAlqRGDGBJasQAlqRGDGBJasQAlqRGDGBJasQAlqRGDGBJasQAlqRGDGBJasQAlqRGBg3gJMcluT7JtiRnTVieJO/sl38+ydH9/NVJPpnkuiTXJDljyDolqYXBAjjJMuBs4HhgLXBSkrVjzY4H1vSPDcA5/fxdwKur6oeAY4HTJvSVpH3akEfAxwDbqmp7Vd0NXAKsH2uzHrioOp8FDk5yWFXdVFVXAlTVN4HrgJUD1ipJi27IAF4J7BiZ3sn9Q3SPbZIcATwFuGLvlyhJ7QwZwJkwr+bTJsmjgA8AZ1bVHRM3kmxIsjXJ1pmZmQUXK0mLbcgA3gmsHpleBdw4bZskD6cL34ur6oOzbaSqzq2qdVW1bsWKFXulcElaDEMG8BZgTZIjk+wPnAhsGmuzCTi5vxriWOD2qropSYDzgeuq6u0D1ihJzew31IqraleS04HLgGXABVV1TZJT++Ubgc3ACcA24FvAKX33ZwAvAb6Q5Op+3uuqavNQ9UrSYhssgAH6wNw8Nm/jyPMCTpvQ79NMHh+WpCXDO+EkqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqREDWJIaMYAlqZFBAzjJcUmuT7ItyVkTlifJO/vln09y9LR9JWlfN1gAJ1kGnA0cD6wFTkqydqzZ8cCa/rEBOGcefSVpnzbkEfAxwLaq2l5VdwOXAOvH2qwHLqrOZ4GDkxw2ZV9J2qftN+C6VwI7RqZ3Ak+bos3KKfsCkGQD3dEzwJ1Jrn8ANe/JIcDXBlz/YlgK+wBLYz/chweHxdiHj1bVceMzhwzgTJhXU7aZpm83s+pc4Nz5lbYwSbZW1brF2NZQlsI+wNLYD/fhwaHlPgwZwDuB1SPTq4Abp2yz/xR9JWmfNuQY8BZgTZIjk+wPnAhsGmuzCTi5vxriWOD2qrppyr6StE8b7Ai4qnYlOR24DFgGXFBV1yQ5tV++EdgMnABsA74FnDJX36FqnYdFGeoY2FLYB1ga++E+PDg024dUTRxalSQNzDvhJKkRA1iSGjGAp5DkgiS3JPli61oWKsnqJJ9Mcl2Sa5Kc0bqm+UpyQJK/SfK3/T68qXVNC5VkWZKrkny4dS0LkeSGJF9IcnWSra3rWagkBye5NMmX+t+Npy/q9h0D3rMkzwTupLtr70da17MQ/R2Gh1XVlUkOAj4HvLCqrm1c2tSSBDiwqu5M8nDg08AZ/V2U+5QkvwasA5ZX1Qta1zNfSW4A1lXVPn0TRpILgb+qqvP6K64eWVW3Ldb2PQKeQlV9Cri1dR0PRFXdVFVX9s+/CVxHd8fhPqO/Zf3OfvLh/WOfO4JIsgp4PnBe61oeypIsB54JnA9QVXcvZviCAfyQlOQI4CnAFY1Lmbf+o/vVwC3Ax6tqn9sH4B3Aa4F7G9fxQBTwsSSf678OYF90FDADvLsfDjovyYGLWYAB/BCT5FHAB4Azq+qO1vXMV1V9t6p+jO7uyGOS7FNDQkleANxSVZ9rXcsD9IyqOpruGwtP64fp9jX7AUcD51TVU4C7gEX96lsD+CGkHzf9AHBxVX2wdT0PRP9R8XLgfl9w8iD3DOBn+zHUS4DnJHlv25Lmr6pu7P+9BfgQ3TcY7mt2AjtHPkVdShfIi8YAfojoT2CdD1xXVW9vXc9CJFmR5OD++SOA5wFfalrUPFXVb1TVqqo6gu4W+09U1YsblzUvSQ7sT+TSf2T/KWCfu0Koqm4GdiR5Yj/rucCinpQe8st4lowk7weeDRySZCfwhqo6v21V8/YM4CXAF/oxVIDXVdXmdiXN22HAhf0X9j8M+JOq2icv49rHfT/woe5vOvsB76uqj7YtacFeCVzcXwGxnf7rEBaLl6FJUiMOQUhSIwawJDViAEtSIwawJDViAEtSI16GJvWSfBf4At13TOwCLgTeUVX78i3DehAzgKX/79v9bc4keSzwPuDRwBtaFqWly+uApV6SO6vqUSPTR9H9B7GHlL8oGoBjwNIsqmo73e/IY1vXoqXJAJbmltYFaOkygKVZ9EMQ36X77mFprzOApQmSrAA2An/g+K+G4kk4qTfhMrQ/At7uZWgaigEsSY04BCFJjRjAktSIASxJjRjAktSIASxJjRjAktSIASxJjfw/ATOP3j2dAQsAAAAASUVORK5CYII=",
"text/plain": [
"
"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"import seaborn as sns \n",
"from matplotlib import pyplot as plt\n",
"\n",
"axs = sns.catplot(data=fair_die, kind='bar', x=\"D\", y=\"P(D)\", color=\"lightblue\");\n",
"axs.set(title=\"Probability distribution of rolling a fair die\");"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"``` {admonition} A word of caution on mathematical notation and dimensionality: \n",
"\n",
"Uppercase letters ($X, Y ...$) often refer to a random variable. Lowercase letters ($x, y ...$) often refer to a particular outcome of a random variable.\n",
"\n",
"The following refer to a probability value (`int`, `float` etc.):\n",
"* $P(X = x)$ \n",
" * also written in shorthand as $P(x)$\n",
"* $P(X = x ∧ Y = y)$ \n",
" * also written in shorthand as $P(x, y)$\n",
"\n",
"The following refer to a collection of values (`pd.Series`, `pd.DataFrame` etc.):\n",
"\n",
"* $P(X)$\n",
"* $P(X ∧ Y)$\n",
" * also written as P(X, Y)\n",
"* $P(X = x, Y)$\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Probability of an Event $P(\\phi)$\n",
"\n",
"An event $\\phi$ is a set of possible worlds $\\{\\omega_i, \\omega_j, ... \\omega_n\\}$. In other words, an event $\\phi$ is a subset of $\\Omega$ i.e. $\\phi \\subset \\Omega$\n",
"\n",
"If we continue with the example of rolling a die, we can define an event $\\phi$ as the set of all possible worlds where the die rolls an even number. From the table above, we can see that there are three possible worlds where the die rolls an even number. \n",
"\n",
"Therefore, the event $\\phi$ is the set $\\{\\omega_2, \\omega_4, \\omega_6\\}$ or $\\{D=2, D=4, D=6\\}$.\n",
"\n",
"\n",
"\n",
"$P (\\phi) = \\sum_{\\omega \\in \\phi} P(\\omega)$ is the sum of probabilities of the set of possible worlds defining $\\phi$\n",
"\n",
"$P (\\phi_1) = P(\\text{Die rolls an even number}) = P(\\omega_2) +P(\\omega_4) + P(\\omega_6) = 0.167 + 0.167 + 0.167 \\approx 0.5 $\n"
]
},
{
"cell_type": "code",
"execution_count": 97,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0.5"
]
},
"execution_count": 97,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"event_condition = fair_die['D'].apply(lambda x: x % 2 == 0)\n",
"\n",
"event = fair_die[event_condition]\n",
"\n",
"P_event = event['P(D)'].sum()\n",
"\n",
"round(P_event, 2)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Joint Probability $P(A, B)$"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Joint probability is the probability of two events occurring together.The joint probability is usually denoted by $P(A, B)$, which is shorthand for $P(A \\wedge B)$ read as _Probability of $A$ AND $B$._\n",
"\n",
"Note that $P(A, B) = P(B, A)$ since $A \\wedge B = B \\wedge A$.\n",
"\n",
"For example, if we are rolling two dice, the joint probability is the probability of rolling a 1 on the first die and a 2 on the second die. \n",
"\n",
"In Data Science, we rarely know the true joint probability. Instead, we estimate the joint probability from data. We will talk more about this when we talk about Statistics. \n"
]
},
{
"cell_type": "code",
"execution_count": 355,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
C
\n",
"
D
\n",
"
P(C, D)
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
H
\n",
"
1
\n",
"
0.24
\n",
"
\n",
"
\n",
"
1
\n",
"
H
\n",
"
2
\n",
"
0.13
\n",
"
\n",
"
\n",
"
2
\n",
"
H
\n",
"
3
\n",
"
0.09
\n",
"
\n",
"
\n",
"
3
\n",
"
H
\n",
"
4
\n",
"
0.01
\n",
"
\n",
"
\n",
"
4
\n",
"
H
\n",
"
5
\n",
"
0.03
\n",
"
\n",
"
\n",
"
5
\n",
"
H
\n",
"
6
\n",
"
0.01
\n",
"
\n",
"
\n",
"
6
\n",
"
T
\n",
"
1
\n",
"
0.19
\n",
"
\n",
"
\n",
"
7
\n",
"
T
\n",
"
2
\n",
"
0.09
\n",
"
\n",
"
\n",
"
8
\n",
"
T
\n",
"
3
\n",
"
0.13
\n",
"
\n",
"
\n",
"
9
\n",
"
T
\n",
"
4
\n",
"
0.04
\n",
"
\n",
"
\n",
"
10
\n",
"
T
\n",
"
5
\n",
"
0.00
\n",
"
\n",
"
\n",
"
11
\n",
"
T
\n",
"
6
\n",
"
0.04
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" C D P(C, D)\n",
"0 H 1 0.24\n",
"1 H 2 0.13\n",
"2 H 3 0.09\n",
"3 H 4 0.01\n",
"4 H 5 0.03\n",
"5 H 6 0.01\n",
"6 T 1 0.19\n",
"7 T 2 0.09\n",
"8 T 3 0.13\n",
"9 T 4 0.04\n",
"10 T 5 0.00\n",
"11 T 6 0.04"
]
},
"execution_count": 355,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"joint_probs = pd.read_csv('../data/experiment_probs.csv')\n",
"joint_probs"
]
},
{
"cell_type": "code",
"execution_count": 267,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1.0"
]
},
"execution_count": 267,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"joint_probs['P(C, D)'].sum()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that sum of joint probabilities is 1 i.e. $\\sum P(C, D) = 1$ at the end of the day, since the sum of all probabilities is 1. \n",
"\n",
"The following three are all true at the same time: \n",
"\n",
"1. $\\sum_{C, D} P(C, D) = 1$ where $P(C, D)$ is a probability table with 12 rows and 3 columns: $C, D, P(C, D)$.\n",
"\n",
"\n",
"2. $\\sum_{C} P(C) = 1$ where $P(C)$ is a probability table with 2 rows (${H, T}$) and 2 columns: $C, P(C)$.\n",
"\n",
"\n",
"3. $\\sum_{D} P(D) = 1$ where $P(D)$ is a probability table with 6 rows (${1, 2, 3, 4, 5, 6}$) and 2 columns: $D, P(D)$.\n"
]
},
{
"cell_type": "code",
"execution_count": 268,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
C
\n",
"
D
\n",
"
P(C, D)
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
H
\n",
"
1
\n",
"
0.24
\n",
"
\n",
"
\n",
"
1
\n",
"
H
\n",
"
2
\n",
"
0.13
\n",
"
\n",
"
\n",
"
2
\n",
"
H
\n",
"
3
\n",
"
0.09
\n",
"
\n",
"
\n",
"
3
\n",
"
H
\n",
"
4
\n",
"
0.01
\n",
"
\n",
"
\n",
"
4
\n",
"
H
\n",
"
5
\n",
"
0.03
\n",
"
\n",
"
\n",
"
5
\n",
"
H
\n",
"
6
\n",
"
0.01
\n",
"
\n",
"
\n",
"
6
\n",
"
T
\n",
"
1
\n",
"
0.19
\n",
"
\n",
"
\n",
"
7
\n",
"
T
\n",
"
2
\n",
"
0.09
\n",
"
\n",
"
\n",
"
8
\n",
"
T
\n",
"
3
\n",
"
0.13
\n",
"
\n",
"
\n",
"
9
\n",
"
T
\n",
"
4
\n",
"
0.04
\n",
"
\n",
"
\n",
"
10
\n",
"
T
\n",
"
5
\n",
"
0.00
\n",
"
\n",
"
\n",
"
11
\n",
"
T
\n",
"
6
\n",
"
0.04
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" C D P(C, D)\n",
"0 H 1 0.24\n",
"1 H 2 0.13\n",
"2 H 3 0.09\n",
"3 H 4 0.01\n",
"4 H 5 0.03\n",
"5 H 6 0.01\n",
"6 T 1 0.19\n",
"7 T 2 0.09\n",
"8 T 3 0.13\n",
"9 T 4 0.04\n",
"10 T 5 0.00\n",
"11 T 6 0.04"
]
},
"execution_count": 268,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"joint_probs"
]
},
{
"cell_type": "code",
"execution_count": 377,
"metadata": {},
"outputs": [],
"source": [
"joint_probs[\"CD_vals\"] = joint_probs.apply(lambda x: \"C=%s and D=%s\" % (x['C'], x['D']), axis=1)"
]
},
{
"cell_type": "code",
"execution_count": 378,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
C
\n",
"
D
\n",
"
P(C, D)
\n",
"
CD_vals
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
H
\n",
"
1
\n",
"
0.24
\n",
"
C=H and D=1
\n",
"
\n",
"
\n",
"
1
\n",
"
H
\n",
"
2
\n",
"
0.13
\n",
"
C=H and D=2
\n",
"
\n",
"
\n",
"
2
\n",
"
H
\n",
"
3
\n",
"
0.09
\n",
"
C=H and D=3
\n",
"
\n",
"
\n",
"
3
\n",
"
H
\n",
"
4
\n",
"
0.01
\n",
"
C=H and D=4
\n",
"
\n",
"
\n",
"
4
\n",
"
H
\n",
"
5
\n",
"
0.03
\n",
"
C=H and D=5
\n",
"
\n",
"
\n",
"
5
\n",
"
H
\n",
"
6
\n",
"
0.01
\n",
"
C=H and D=6
\n",
"
\n",
"
\n",
"
6
\n",
"
T
\n",
"
1
\n",
"
0.19
\n",
"
C=T and D=1
\n",
"
\n",
"
\n",
"
7
\n",
"
T
\n",
"
2
\n",
"
0.09
\n",
"
C=T and D=2
\n",
"
\n",
"
\n",
"
8
\n",
"
T
\n",
"
3
\n",
"
0.13
\n",
"
C=T and D=3
\n",
"
\n",
"
\n",
"
9
\n",
"
T
\n",
"
4
\n",
"
0.04
\n",
"
C=T and D=4
\n",
"
\n",
"
\n",
"
10
\n",
"
T
\n",
"
5
\n",
"
0.00
\n",
"
C=T and D=5
\n",
"
\n",
"
\n",
"
11
\n",
"
T
\n",
"
6
\n",
"
0.04
\n",
"
C=T and D=6
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" C D P(C, D) CD_vals\n",
"0 H 1 0.24 C=H and D=1\n",
"1 H 2 0.13 C=H and D=2\n",
"2 H 3 0.09 C=H and D=3\n",
"3 H 4 0.01 C=H and D=4\n",
"4 H 5 0.03 C=H and D=5\n",
"5 H 6 0.01 C=H and D=6\n",
"6 T 1 0.19 C=T and D=1\n",
"7 T 2 0.09 C=T and D=2\n",
"8 T 3 0.13 C=T and D=3\n",
"9 T 4 0.04 C=T and D=4\n",
"10 T 5 0.00 C=T and D=5\n",
"11 T 6 0.04 C=T and D=6"
]
},
"execution_count": 378,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"joint_probs"
]
},
{
"cell_type": "code",
"execution_count": 351,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAFiCAYAAADsnE/CAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAArrUlEQVR4nO3dd7ycZZ3+8c9FCEW6JNQQAgJCkKKEosAuRXpT10WQoiBm2Z+IWNZlLQhiX3WFFWVREUQpioAIQUCRIkUSkBZqDBFi6BADUiPf3x/3PTCZ3HPOnJN5zjkzc71fr/Oamadd9zMzZ75PmeceRQRmZmaNFhvuBpiZ2cjkAmFmZkUuEGZmVuQCYWZmRS4QZmZW5AJhZmZFLhAdTtJzktYd7naUSLpa0hGDnPcMSV/qY/xr610/raTtJd03uBa31K5Zkt6Z739G0g/buOziOrVp2adK+ny7ljeA3H+X9Fhet5WHOr8vko6X9NPhbsdI5gIxQrX64RoRy0bEzBaXGZLWW/TWDb9m6x0R10XEm2uP6z/QK2jDVyKi39eoiteyn7wPSvpDw7KPjIgTF3XZA2zHaODbwK553Z4qTLNE/qB+QNLf8+t1uqQJQ9nWQrt2kPRqLmzPSZot6eeSthzOdg01FwgbFCV+/7SBpMWHuw0VWRVYCpjexzTnA/sC7wdWADYDbgF2rrx1/ZsTEcsCywHbAPcC10kaCW0bEv4H7wCSPixphqSnJV0saY26ca/tFeTDEqdIulTSs5L+KOlNedy1eZbb8xbR+wo5H5R0vaT/lfQ3SffW/zPkLeEvS7oeeB5YV9I7JE3N00+V9I6Gxb5J0s15/K8kvbFueb+Q9Gged62kjRvmHSPpyrwu10hau7TeDeuwg6TZ+f5ZwHjg13mdP52fm482zHOHpHc1ee4PkfQXSU9J+mzDuNcOUUhaStJP83Rz83OxqqQvA9sD381t+G5d+z8i6QHggSbrVFx/SRPytIvXteVqSUdI2gg4FXh7zpubxy9wyKqF99SReav+mfyeUpPnZ0lJ35E0J/99Jw/bAKgd6psr6arCvO8EdgH2i4ipETE/Iv4WEadExI+a5B0r6c/5Oblb0rvrxn1Q0h8kfTO3+0FJe9SNXyc/j89KuhIYU8poFMnsiDgO+CHw9Vbm6wYuECOcpJ2ArwL7A6sDfwHO7WOWA4ETgJWAGcCXASLin/L4zfLu/nlN5t8amEn65/kCcEH9hzpwCDCZtFX1LHApcDKwMulwwqVa8FjzocDhwBrA/DxtzWXA+sAqwK3AzxrachBwYm7LbYXxfYqIQ4CHgH3yOn8DOBM4uDaNpM2ANYEpjfNLmgh8P6/zGnkdxzWJ+wBpC3itPN2RwAsR8VngOuCo3Iaj6uZ5F+n5nthkmQNe/4i4J2ffmPNWLKxXK++pvYEtSVv0+wO7NYn8LGnrevM87VbA5yLifqBW8FeMiJ0K874TuDkiHu5vver8mVRwVyC9z38qafW68VuTCtMY4BvAj+qK29mkvZMxpOf1AwPIrbkAeJukZQYxb8dxgRj5DgJOj4hbI+Il4L9IW4cTmkx/QUTcHBHzSR8omw8w73HgOxHxSi4i9wF71Y0/IyKm5+XvCjwQEWflrb9zSLvh+9RNf1ZE3BURfwc+D+wvaRRARJweEc/m9Toe2EzSCnXzXhoR1+bxn83rvdYA16fRr4D1Ja2fHx8CnBcRLxemfS9wSV0bPg+82mS5r5AKw3oR8Y+IuCUi5vXTlq9GxNMR8UKT8VWsP7T2nvpaRMyNiIeA39P8fXQQ8MWIeDwiniB9aB/SYjtWBh4ZSMMj4hcRMSciXs3vzwdIRanmLxHxg4j4B2ljYHVgVUnjSQXv8xHxUkRcC/x6INnZHEDAioOYt+O4QIx8a5C28ACIiOeAp0hbvSWP1t1/Hlh2gHl/jQV7cPxLbkNN/dbeAm2rm76+bQ83jBtNOnQyStLX8uGCecCsPM2Y0rx5vZ9uaMuA5Q/EnwMHK51DORA4q8nkazS04e+k577kLOBy4Nx8qOUbSidp+9LflnPb1z9r5T3V6vuo8T3Q+H7py1OkD/CWSTpU0m35MN5c4C0s+J55rd0R8Xy+u2xu0zP5Naxv60CtCQQwdxDzdhwXiJFvDlB/7H0Z0pbXXyvKW7PhePP43Iaa+uKxQNvqpq9v21oN414BniSdlNyPdJhhBWBCnkaleSUtC7yxoS2tKHVXfCZpy3dn4PmIuLHJvI80tOENpOd+4ZC0x3VCREwE3kE6RHNoH23oa3hNs/Wvfci9oW7a1Qaw3Ha+pxrfA43vl778FthKUrPDdgvI52B+ABwFrJwPn93Fgu+ZZh4BVmo4NDS+xXbWezdwa0Oh6VouECPf2cBhkjaXtCTwFeCPETFrEMt6DOjvmolVgKMljZb0r8BGFI7PZ1OADSS9X9LiSie+JwKX1E1zsKSJ+cP1i8D5efd/OeAl0lbkG/J6NdpT0naSliAdM/7jAI9XQ2Gdc0F4FfgWzfceIH3DZu+6NnyRJv8zknaUtEk+fDaPVAj/0awNLSqufz6U81fScztK0uHAm+rmewwYl+craed76hzgc5LGShoDHAe0dG1BRPwWuBK4UNIW+T20XD5BfnhhlmVIxe8JAEmHkfYgWsn6CzANOEHpq7XbseCh0KaUrCnpC8ARwGdama8buECMbBERvyMd+/4laSvoTcABg1ze8cCZefd8/ybT/JF04vhJ0gnu95a+v54b9xRpS/mTpA/6TwN7R8STdZOdBZxB2vVfCjg6D/8JaRf/r8DdwE2FiLNJJ8qfBrYgbfUP1FdJH2BzJX2qbvhPgE3o48MsIqYDH8nteAR4BpjdZPLVSAVlHnAPcE3dsk8C3pu/WXNyk/lL+lr/DwP/QXreNwZuqBt3FemrpY9Kqn8tauvVzvfUl0gfvHcAd5K+bDCQC/zeS9rQOA/4G2mPYBJp76Kx3XeTivqNpCK4CXD9ALLeTzqJ/TTpef1JP9OvIek54Dlgas7bISKuGEBmR5N/MGhkknQr6eTfRUOY+UHgiIjYbqgyh4ukQ4HJvbCuZoPlPYgRSOl6gI2APw13W7pRPtz1/4DThrstZiOZC8QII+nrwBXAf+bjptZGknYjHcN+jHQIx8ya8CEmMzMr8h6EmZkVuUBYkQr9/QxBZtu6uJZ0maR+u1KQNF6pz6JR7chtNy1Cb7TqoxfZxvWun1bSQZKaflNHFXepbiOHC8QIkT+M71RdD6mSviTpjBbnX6RurRd1/sLy+uxaXIUuqdspIvaIiDNbmO6h3GfRP/qbdjiKZlX6Wu+I+FlE7Fp73PhaRkOX6p1sUTdKJO0v6QZJz0u6uo1NGxFcIEaWNRj899Gtw3RDoTGeBr4DfG2Y21EJF4iR5RukKz2LHxyS9pU0PV/0dbVS187Fbq3z8G3y1s1cSbdL2qHJcovzZwdJekjSk6rr7lrSVpJuzMt+RNJ3a1fuqp+uxdWkS+psJRW6K8/zbajU/fXTku7r42K/xkMmi0n6nFK33Y9L+olyp4CNewV5vhOVuj1/VtIVSlcIA9TWa25u99sLucdLOl/SeXn+W5V6jK2NnyXpPyXdAfxd6erh4utaZ0ulrq2fkfRjSUvlZa0k6RJJT+Rxl2jhbiuK3a33tTdUv3dXei1V16V6nmYNSb/M7XhQ0tF147aSNE3SPKVflvt2k9drTG7/3Pz6Xqe8N62GPRgt+AuCOyj9mM+n82v7iKR3SdpT0v15WcUrnyVNJl18+Om8br/OwzfKr8Pc/LrsW5of0tXgEfFzBt4FTGeICP+NgD9SFwLrk7ojPiIP+xKp91SADUh98OxC6vDu06TuvJfI42cB76xb3pqkq2z3JG0I7JIfj22S3zj/hNymHwBLk7pyfgnYKI/fgtTN8+J52nuAYxrWZ70+1veDwB8ahp1B2iLbKi/3Z8C5edwypM7rDsvj3ka62nvjJsu/uu55PDw/V+uSOm67gNTLbP16Ll4335/z8710fvy10rRNco8ndbPx3vw6fQp4EBhd9zzfRupnaekWX9e78vRvJF05/KU8bmXgX0hdlSwH/AK4qOE5+CupO4plSFdO/7SP9T6i9No0vpbADsDsfH8x0nv2OGCJ/BzPBHbL428EDsn3lwW2afK8fZW00TA6/23P69+ybMw/o+452IHUjfxxeb4Pk77GfHZ+TjYGXgTWbZL72rLy49H5+f9MXp+dSN3av7mf/98jgKuH+3Ok3X/egxhZgtQFwnFKfeTUex+p++crI+IV4JukD5jGH+ipORiYEhFTInWNfCWpS4Q9B9imEyLihYi4HbidVCiI1J31TZG6+Z4F/B/wzwNcdkmz7sr3BmZFxI9z5q2kD7z3trDMg4BvR8TMSD2X/hdwQGnrOftxRNwfqRvunzPwLtNviYjz8+v0bVIXI9vUjT85Up9KL9Da6/rdPP3TpO5PDoTU1UlE/DIino+IZ/O4xtegaXfrbbIlaaPjixHxcqSfTP0Brx8qfQVYT9KYiHguIkpdqtSmWx1YO1LHh9dF/uRtwSvAl/Pzdy6pd9eTInUlP53U7cimLS5rG1Ih+1pen6tIfYsd2OL8XcUFYoSJiCmkH7mZ3DCqsYvmV0lb1M26/V4b+Ne8m1zrGnk7Bti9Mk26fZa0QT4k8KhSd91focVf6BpMHml9tm5Yn4NYsBfTZkpdUi9O+knMgbShVfXddL9K6r+ppS7Tm7yujV2mrwHpinBJ/5cPnc0jHQJbsaEAFLtbH+D69GVtUp9F9a/LZ3j9uf0QaS/pXqVf2du7yXL+m7TlfoWkmZKOHUAbnorXT7bXflvjsbrxL9D6a7gG8HB+HWoau7DvGS4QI9PnSD8QU9+dc2MXzSIddqh10dy4tfUwaetxxbq/ZSKi2cm0gV4x+X3SjwOtHxHLkz4UWul2ebB5DwPXNKzPshHx7y3MW+qSej4Lfoi0otU213fTvRjpV+ha6jK98LousDwW7E77k8Cbga3za1D71cBil+ks2N16uzwMPNjwuiwXEXsCRMQDEXEgqZfgrwPnq/BrbHlr/5MRsS6pl9VP6PWfu32e5l2bL6rG13QOsJYW/L31xi7se4YLxAgUEVeTesas/x7/z4G9JO2s9EM0nySdE6j14tnYpfRPgX0k7abUJfRS+YRes773B9ol9XKknkufk7Qh0PhB3d/y+uuSutElpK7FD1Hqiny0pC0LJ3RLzgE+rvSbxMuS9nbOy4exBuIJUjfh/T1PW0h6Tz6EdQzpdWp2aKW/1xXgI5LG5RPMnyH1fArpNXiBdNL8jaQeShs16259IPp6LW8G5imdeF86v9feImlLAEkHSxqbt8jn5nkWype0t6T1coGcl6epTXcb8P687N1pz6HMmsZ1+yPpnNCn83tsB1LBKv7Mb+1/i7RHulj+P+vvh6I6hgvEyPU50klJACLiPtJ5hf8lbQHuQ/qt5dpPZS7QrXWk303Yj/SB8gRpS+8/aP6aN+sWu5lPkbpPfpZ0zLnxN66Pp++uxfvskrpRPsa+K+nY9hzSYaCvA43nakpOJ3U7fi3phPGLwEdbmK+xDc+TjvNfn9drmyaT/op0buEZ0s9vvicfHy8ts7/XFdIJ1ytIJ39n8np32t8hna94klSAflOIaNbd+kAcT5PXMhebfUjnaR7Mbfkh6UegAHYHpit1m30ScEBEvFjIWJ/UxfdzpBPb38sbSgAfyxlzSYcVLxrEOjTzI2BiXreL8vO+L7BHXpfvAYdGxL1N5j+EVKS/Tzqx/gLp/6EruC8m60pKX8/8YUT01+d/u3OPJ33j5uChzDWrgvcgrOvkwynrkrZozWyQXCCsq0hahXQ45Rqgsq48zHqBDzGZmVlRpXsQknZX6hJhRul7zUq9Rt6R/27Qwl0S3CnpNknTqmynmZktrLI9iHyxzv2kLgRmk370+8BIPzxem+YdwD0R8YykPYDjI2LrPG4WMCki2vmdbTMza1GVvUluBczIl94j6VzS1y5fKxARUf9d75tIFxQN2pgxY2LChAmLsggzs55yyy23PBkRY0vjqiwQa7LgZf6zga37mP5DwGV1j4N02X0A/xcR/f7A/IQJE5g2zUejzMxaJekvzcZVWSBK3S4Uj2dJ2pFUILarG7xtRMzJ30q5UtK9EXFtYd7J5H6Lxo8fv+itNjMzoNqT1LNZsB+Yxv5oAJC0KenKy/0i4qna8IiYk28fBy4kHbJaSEScFhGTImLS2LHFvSQzMxuEKgvEVGD93P/NEqQuEi6un0DSeFLf/IdExP11w5eRtFztPqmLhbsqbKuZmTWo7BBTRMyXdBRwOTAKOD0ipks6Mo8/lfQjHysD30t9dDE/IiaRugq+MA9bHDg7Ikr9zJiZWUW66kK5SZMmhU9Sm5m1TtItecN8Ie5qw8zMilwgzMysyAXCzMyKqrwOYthcOn1WZcvea+MJlS3bzGwk8R6EmZkVuUCYmVmRC4SZmRW5QJiZWZELhJmZFblAmJlZkQuEmZkVuUCYmVmRC4SZmRW5QJiZWZELhJmZFblAmJlZkQuEmZkVuUCYmVmRC4SZmRW5QJiZWZELhJmZFblAmJlZkQuEmZkVuUCYmVmRC4SZmRW5QJiZWZELhJmZFblAmJlZkQuEmZkVuUCYmVmRC4SZmRW5QJiZWZELhJmZFblAmJlZUaUFQtLuku6TNEPSsYXxB0m6I//dIGmzVuc1M7NqVVYgJI0CTgH2ACYCB0qa2DDZg8A/R8SmwInAaQOY18zMKlTlHsRWwIyImBkRLwPnAvvVTxARN0TEM/nhTcC4Vuc1M7NqVVkg1gQerns8Ow9r5kPAZYOc18zM2mzxCpetwrAoTijtSCoQ2w1i3snAZIDx48cPvJVmZlZU5R7EbGCtusfjgDmNE0naFPghsF9EPDWQeQEi4rSImBQRk8aOHduWhpuZWbUFYiqwvqR1JC0BHABcXD+BpPHABcAhEXH/QOY1M7NqVXaIKSLmSzoKuBwYBZweEdMlHZnHnwocB6wMfE8SwPy8N1Cct6q2mpnZwqo8B0FETAGmNAw7te7+EcARrc5rZmZDx1dSm5lZkQuEmZkVuUCYmVlRpecgzGxhl06fVcly99p4QiXLtd7lPQgzMytygTAzsyIXCDMzK3KBMDOzIhcIMzMrcoEwM7MiFwgzMytygTAzsyIXCDMzK3KBMDOzIhcIMzMrcoEwM7MiFwgzMytygTAzsyIXCDMzK3KBMDOzIhcIMzMrcoEwM7MiFwgzMytygTAzsyIXCDMzK3KBMDOzIhcIMzMrcoEwM7MiFwgzMytygTAzsyIXCDMzK3KBMDOzIhcIMzMrcoEwM7OiSguEpN0l3SdphqRjC+M3lHSjpJckfaph3CxJd0q6TdK0KttpZmYLW7yqBUsaBZwC7ALMBqZKujgi7q6b7GngaOBdTRazY0Q8WVUbzcysuSr3ILYCZkTEzIh4GTgX2K9+goh4PCKmAq9U2A4zMxuEKgvEmsDDdY9n52GtCuAKSbdImtzWlpmZWb8qO8QEqDAsBjD/thExR9IqwJWS7o2IaxcKScVjMsD48eMH11IzM1tIlXsQs4G16h6PA+a0OnNEzMm3jwMXkg5ZlaY7LSImRcSksWPHLkJzzcysXpUFYiqwvqR1JC0BHABc3MqMkpaRtFztPrArcFdlLTUzs4VUdogpIuZLOgq4HBgFnB4R0yUdmcefKmk1YBqwPPCqpGOAicAY4EJJtTaeHRG/qaqtZma2sCrPQRARU4ApDcNOrbv/KOnQU6N5wGZVts3MzPrWb4GQNI50eGh7YA3gBdLhnkuByyLi1UpbaGZmw6LPAiHpx6Svpl4CfB14HFgK2ADYHfispGNL3y4yM7PO1t8exLcionRy+C7ggnzy2d8tNTPrQn0WiPriIGlsHvZE3fiXgRmVtc7MzIZNn19zVXK8pCeBe4H7JT0h6bihaZ6ZmQ2X/q6DOAbYFtgyIlaOiJWArYFtJX286saZmdnw6a9AHAocGBEP1gZExEzg4DzOzMy6VH8FYnSpu+18HmJ0NU0yM7ORoL8C8fIgx5mZWYfr72uum0maVxgu0vUQZmbWpfr7muuooWqImZmNLJX+JrWZmXWuSjvr6xWXTp9V2bL32nhCZcs2q0JV/w/+Xxh63oMwM7MiFwgzMysaVIGQ9FtJl0nau90NMjOzkWGw5yAOBVYHtmljW8zMbATp7/cgxgJjI+LuhlErAQ9FxC2VtczMzIZVf4eY/hcYWxg+Djip/c0xM7ORor8CsUlEXNM4MCIuBzatpklmZjYS9NtZ3yDHmZlZh+uvQDwgac/GgZL2AGZW0yQzMxsJ+vsW08eBSyTtD9ROSE8C3g74K65mZl2szz2IiLgf2AS4BpiQ/64BNs3jzMysS/X3NVdFxEvAj/uZJtreMjMzG1b9nYP4vaSPShpfP1DSEpJ2knQm8IHqmmdmZsOlv3MQuwOHA+dIWhd4BliaVFiuAP4nIm6rtIVmZjYs+vvBoBeB7wHfkzQaGAO8EBFzh6BtZmY2jPo7B7EUcCSwHnAHcHpEzB+KhpmZ2fDq7xzEmaSvtd4J7Al8q/IWmZnZiNDfOYiJEbEJgKQfATdX3yQzMxsJ+tuDeKV2x4eWzMx6S397EJtJmpfvC1g6PxYQEbF8pa0zM7Nh09+3mEYNVUPMzGxkqfQ3qSXtLuk+STMkHVsYv6GkGyW9JOlTA5nXzMyqVVmBkDQKOAXYA5gIHChpYsNkTwNHA98cxLxmZlahKvcgtgJmRMTMiHgZOBfYr36CiHg8IqZSdzK81XnNzKxaVRaINYGH6x7PzsOqntfMzNqgygKhwrBWe31teV5JkyVNkzTtiSeeaLlxZmbWtyoLxGxgrbrH44A57Z43Ik6LiEkRMWns2LGDaqiZmS2sygIxFVhf0jqSlgAOAC4egnnNzKwN+rtQbtAiYr6ko4DLgVGkjv6mSzoyjz9V0mrANGB54FVJx5C695hXmreqtpqZ2cIqKxAAETEFmNIw7NS6+4+SDh+1NK+ZmQ2dSi+UMzOzzuUCYWZmRS4QZmZW5AJhZmZFlZ6ktmpcOn1WJcvda+MJlSzXhpffLzZY3oMwM7MiFwgzMytygTAzsyIXCDMzK3KBMDOzIhcIMzMrcoEwM7MiFwgzMytygTAzsyIXCDMzK3KBMDOzIhcIMzMrcoEwM7MiFwgzMytygTAzsyIXCDMzK3KBMDOzIhcIMzMrcoEwM7MiFwgzMytygTAzsyIXCDMzK3KBMDOzIhcIMzMrcoEwM7MiFwgzMytygTAzsyIXCDMzK3KBMDOzokoLhKTdJd0naYakYwvjJenkPP4OSW+rGzdL0p2SbpM0rcp2mpnZwhavasGSRgGnALsAs4Gpki6OiLvrJtsDWD//bQ18P9/W7BgRT1bVRjMza67KPYitgBkRMTMiXgbOBfZrmGY/4CeR3ASsKGn1CttkZmYtqrJArAk8XPd4dh7W6jQBXCHpFkmTK2ulmZkVVXaICVBhWAxgmm0jYo6kVYArJd0bEdcuFJKKx2SA8ePHL0p7zcysTpV7ELOBteoejwPmtDpNRNRuHwcuJB2yWkhEnBYRkyJi0tixY9vUdDMzq7JATAXWl7SOpCWAA4CLG6a5GDg0f5tpG+BvEfGIpGUkLQcgaRlgV+CuCttqZmYNKjvEFBHzJR0FXA6MAk6PiOmSjszjTwWmAHsCM4DngcPy7KsCF0qqtfHsiPhNVW01M7OFVXkOgoiYQioC9cNOrbsfwEcK880ENquybWZm1jdfSW1mZkUuEGZmVuQCYWZmRS4QZmZWVOlJajOzKl06fVZly95r4wmVLbtVw71+3oMwM7MiFwgzMytygTAzsyIXCDMzK3KBMDOzIhcIMzMrcoEwM7MiFwgzMyvyhXI24lR1cdBIuPDJrJN4D8LMzIpcIMzMrMgFwszMilwgzMysyAXCzMyKXCDMzKzIBcLMzIpcIMzMrMgFwszMilwgzMysyAXCzMyKXCDMzKzIBcLMzIpcIMzMrMgFwszMilwgzMysyD8YZP3q9h/w6fb1Mxss70GYmVmRC4SZmRW5QJiZWZELhJmZFVVaICTtLuk+STMkHVsYL0kn5/F3SHpbq/OamVm1KisQkkYBpwB7ABOBAyVNbJhsD2D9/DcZ+P4A5jUzswpVuQexFTAjImZGxMvAucB+DdPsB/wkkpuAFSWt3uK8ZmZWoSqvg1gTeLju8Wxg6xamWbPFeQGQNJm09wHwnKT7BtHWMcCTg5hvMIYyy3nOc17v5A02a+1mI6osECoMixanaWXeNDDiNOC0gTWtoRHStIiYtCjLGIlZznOe83onr4qsKgvEbGCtusfjgDktTrNEC/OamVmFqjwHMRVYX9I6kpYADgAubpjmYuDQ/G2mbYC/RcQjLc5rZmYVqmwPIiLmSzoKuBwYBZweEdMlHZnHnwpMAfYEZgDPA4f1NW9VbWURD1GN4CznOc95vZPX9ixFFA/tm5lZj/OV1GZmVuQCYWZmRS4QZmZW5AKRSTqsgmVuKGlnScs2DN+93Vl5uVtJ2jLfnyjpE5L2rCKrSf5PhjBru7x+u1a0/K0lLZ/vLy3pBEm/lvR1SStUkWnVqu/rrZtIWl7SFpJWaveyXSBed0I7FybpaOBXwEeBuyTVdxXylXZm5bwvACcD35f0VeC7wLLAsZI+W0HexQ1/vwbeU3tcQd7Ndfc/TFq/5YAvVNSZ4+mkb9YBnASsAHw9D/txBXlNSbqzgmWuJelcSddJ+oyk0XXjLqogb0NJl0m6VNKbJJ0haa6kmyVtVEHe2xr+tgAulvTWKgqFpMPr7o+T9Lu8fjdI2qDNWT+VNCbf3w2YTnpv3ibpX9ua1UvfYpJ0R7NRwAYRsWQbs+4E3h4Rz0maAJwPnBURJ0n6U0S8tV1ZdXmbA0sCjwLjImKepKWBP0bEpm3OuxW4G/ghr1/9fg7pmhUi4po25732nEmaCuwZEU9IWga4KSI2aXPePRGxUb5/a0TU9zR8W0Rs3ua89zQbBZwaEWPbnHcl8EvgJuBDwBbAPhHxVEXvz2uB/yZttHwN+E/gPGBv4JiI2LnNea+S1u2lusHb5GERETu1Oe+194iknwO/A35A6kPuqHaun6Q7a+93STcA74+IWblo/C4iNmtXVq/9JvWqwG7AMw3DBdzQ5qxREfEcQH7xdgDOl7Q25a5EFtX8iPgH8LykP0fEvJz9Qv5nabdJwMeAzwL/ERG3SXqh3YWhzmJ5F3ox0obNEwAR8XdJ8yvIu0vSYRHxY+B2SZMiYlreGnylgrzzgJ9R7lJmqQryxuZrkQA+Kulg4FpJ+zZpw6JaLiJ+DSDpxIg4Nw//taS27r1n+5P23v87Iqbk3AcjYscKshptEBH75/sXSjquzctfTNLy+X/8VeAhgIh4UlJbP9N7rUBcAiwbEbc1jpB0dZuzHpW0eS0r70nsTTp00dat3exlSW+IiOdJW4MA5OPlbS8QEfEq8D+SfpFvH6Pa99MKwC2k4hqSVouIR/P5nSoK7hHASZI+R+oA7UZJD5M6kTyigrw7gG9GxF2NIyS9s4K80ZKWiogXASLip5IeJV2cukwFeaPq7n+7YdwS7Q6LiPMl/QY4Uen84ieppvDVjJN0Mum9OFbS6IiobUiM7mO+wTgB+L2kU4DrgV9I+hWwE/Cbdgb11CGmoSRpHGmr/tHCuG0j4vo25y0ZES8Vho8BVo+Ith/HbsjZC9g2Ij5TZU4h9w3AqhHxYEXLXw5Yl1T8ZkfEYxXlbA/8JSIeKoybFBHT2pz3ceDWxj0+SW8FvhERu7Q579+An9X2quuGr0c6BHNMO/MaMt5KKkobR8QqFWV8oGHQxRHxjKTVgKPb/X+Rn7cPAxuQ35vARRFxeVtzXCDMrNtJEukw17zhbksn6dlvMUn6Q/1tt2Q5ryvyDq6/dd6i50XaEt53qPLqbzs5q2cLBPCGfFvF8dbhzHJe5+d9ouHWec4blqxeLhBmI10VJ9+d1515lWS5QJiZWZELhJmZFfVygej43T/ndW2e2YjQywXi4w233ZLlvM7Puz/f3uc85w1nlq+DMDOzol7raqN2xePHgDfnQfcAJ0dE27uqHsos53VF3uLAHsCGdXm/iYgq+ppyXgfnDVlWRPTMH3Ao8CdgR1LfPiuS+i+5BTi0U7Oc1xV5a5AOE1wN/A/wHeCaPGwN5zlvWLLa/USN5D9SV78TCsMnkLqM7sgs53VF3hmkbq8bhx8NnOk85w1LVrufqJH8B9w9mHEjPct5XZF3bx/j7nOe84Yjq9e+xfTCIMeN9CzndXfe832Mc17v5Q1ZVq+dpN6oya/KidSlc6dmOa/z81Zo8qtyApZ3nvOGI6vnCkSXZjmv8/OuAfZpMu5a5zlvOLJ8HYSZmRX12jkIMzNrkQuEmZkVuUCYmVlRT52klnQn0PSkS0Rs2olZzuuKvNK3UurzLnCe84Y6q6cKBLB3vv1Ivj0r3x5E+7+rPJRZzuv8vNq3UlYB3gFclR/vSOpSoa0faM7r6Lyhy2r3FYWd8Adc38qwTstyXlfkXQKsXvd4deAC5zlvOLJ69RzEMpK2qz2Q9A6q+0H6ocxyXufnTYiIR+oePwZs4DznDUdWrx1iqvkQcLqkFfLjucDhXZDlvM7Pu1rS5cA5pHMgBwC/d57zhiOrpy+Uk7Q86Tn4WzdlOa+z8/JJyO3zw2sj4kLnOW84snqyQEhaEvgXUtfNr+1FRcQXOznLeZ2fZzaS9Oo5iF8B+wHzgb/X/XV6lvM6PE/SeyQ9IOlvkuZJelbSPOc5bziyenUP4q6IeEu3ZTmvK/JmAPtExD3Oc95wZ/XqHsQNkjbpwizndX7eY0P1Yea8js+rPKtX9yDuBtYDHgReIvWjHtHmq2OHOst5XZF3ErAacFHOA9p/5a/zOj9vKLJ69Wuue3RplvM6P2950pXau9YNC9p/5a/zOj+v8qye3IOokbQKsFTtcUQ81A1Zzuv8PLORoCf3ICTtC3wLWAN4HFgbuAfYuJOznNcVeUuRLs7bmAULUiUX5zmvc/OGIqtXT1KfCGwD3B8R6wA7A9d3QZbzOj/vLNJx5d1IPy05DnjWec4blqwqOpEa6X/AtHx7O7BYvn9zp2c5ryvy/pRv78i3o4GrnOe84cjqyUNMwFxJy5J+4Ptnkh4nXQjV6VnO6/y8V+py3wI8SrqK23nOG/qsqirpSP4j9ca5GOkczAeAo4GVOz3LeV2RdwSwEvBPwEzSeY9/c57zhiOrp7/FZGZmzfXqSWozM+uHC4SZmRW5QJiNAJK2cZ7zRlxWL52DkHQn6VL0hUbR5v51hjLLeV2Rd2tEvK2dy3Red+YNZVavfc1173wr4FJgzy7Jcl7n55mNOD21B1GvWyu+8zozT9Jc0rUWRRGxr/OcN9RZvbYHYTZSPUHq88l5zhsxWT1VICTVbwUuLemtpEMIAETErZ2Y5bzOzwOejYhr2rxM53Vn3pBl9dQhJkm/72N0RMROnZjlvK7IuyAi3tPOZTqvO/OGNKuXCoRZJ5C0YUTcW7t1nvOGK6tnr4OQtGH9bbdkOa/z84CzG26d57xhyerZAkEXvYjO67q8GvU/ifOcV11WLxeImo5/EZ3XtXlmw8oFwszMilwgzEauof4GifM6N6+SLBeILngRndd1eWq4dZ7zhiWrlwtE17yIzuu6vO0bbp3nvGHJ6tnrICQtGxHP1W67Jct5nZknaT1g1Yi4vmH49sCciPiz85w31Fk9tQchaT1J2wLU/tHzP/72kt7UqVnO6/w84DvAs4XhL+RxznPekGf1VIGgS19E53VF3oSIuKNxYERMAyY4z3nDkdVrBaIrX0TndUXeUn2MW9p5zhuOrF4rEF35IjqvK/KmSvpw40BJHwJucZ7zhiOrp05SSzoHuCoiftAw/EPArhHxvk7Mcl5X5K0KXAi8zOv/5JOAJYB3R8SjznPekGf1WIHoyhfReZ2fV5e7I/CW/HB6RFxVRY7zOj9vKLJ6qkDUdNuL6LzuyTMbSXqyQJiZWf967SS1mZm1yAXCzMyKXCCs60haTdK5kv4s6W5JUyRtUEFOsdsNSf+QdJuk6ZJul/QJSYv8vyZpgqS7FnU5Zq1afLgbYNZOkkT69tGZEXFAHrY5sCpw/xA144WI2Dxnr0L6JboVgC8MUb5ZW3gPwrrNjsArEXFqbUBE3BYR1zVOKOkiSbfkLf3JdcOfk/TlvPV/U/7KK5LWkXSjpKmSTmylMRHxODAZOCoXr/r88yTtWff4DEn/kvcUrpN0a/57R6HtH5T03brHl0jaId/fNbfzVkm/kLRsHv61vEd1h6RvttJ+620uENZt3kLrV5MeHhFbkK5vOFrSynn4MsBNEbEZcC1Qu2r1JOD7EbEl0PJ1EBExk/S/tkrDqHOB9wFIWgLYGZgCPA7sEhFvy+NPbjVL0hjgc8A78/zTgE9IeiPwbmDjiNgU+FKry7Te5QJhvexoSbcDNwFrAevn4S8Dl+T7t/B630vbAufk+2cNMKv0WxKXATtJWhLYA7g2Il4ARgM/kHQn8Atg4gBytsnTXy/pNuADwNrAPOBF4IeS3gM8P8D2Ww/yOQjrNtOB9/Y3UT4c807g7RHxvKSreb3/pVfi9QuE/sGC/ycDvnBI0rp5OY/XD4+IF3PubqQ9hVrx+TjwGLAZaSPuxcJi57PgBl6t7QKujIgDC+3YirSXcgBwFLDTQNfFeov3IKzbXAUsWd+ZmaQtJf1zw3QrAM/k4rAhacu7P9eTPlwBDmqlMZLGAqcC343yVannAoeRfhHs8rq2PRIRrwKHAKMK880CNpe0mKS1gK3y8JuAbZV+VAZJb5C0QT4PsUJETAGOATZvpf3W21wgrKvkD+F3A7vkr7lOB44H5jRM+htgcUl3ACeSPlj78zHgI5Kmkj7Em1m69jVX4LfAFcAJTaa9Avgn4LcR8XIe9j3gA5JuAjYA/l6Y73rgQeBO4JvArQAR8QTwQeCcvG43ARsCywGX5GHXkPZSzPrkrjbMzKzIexBmZlbkAmFmZkUuEGZmVuQCYWZmRS4QZmZW5AJhZmZFLhBmZlbkAmFmZkX/H/tqWdI3vD9PAAAAAElFTkSuQmCC",
"text/plain": [
"
"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"axs = sns.barplot(data=joint_probs, x=\"CD_vals\", y=\"P(C, D)\", color=\"lightblue\");\n",
"axs.set(title=\"Joint probability distribution of C and D\\n Note that the joint probabilities sum to 1\", \\\n",
" xlabel=\"C and D values\", \\\n",
" ylabel=\"P(C, D)\");\n",
"plt.xticks(rotation=90);"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"## Marginal Probability $P(A)$\n",
"\n",
"Because most data sets are multi-dimensional i.e. involving multiple random variables, we can sometimes find ourselves in a situation where we want to know the joint probability $P(A, B)$ of two random variables $A$ and $B$ but we don't know $P(A)$ or $P(B)$. In such cases, we compute the **marginal probability** of one variable from joint probability over multiple random variables. \n",
"\n",
"Marginalizing is the process of summing over one or more variables (say B) to get the probability of another variable (say A). This summing takes place over the joint probability table.\n",
"\n",
"$$ P(A) = \\sum_{b \\in \\Omega_B} P(A, B=b) $$\n"
]
},
{
"cell_type": "code",
"execution_count": 212,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"C\n",
"H 0.51\n",
"T 0.49\n",
"Name: P(C), dtype: float64"
]
},
"execution_count": 212,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"P_C = joint_probs.groupby('C').sum()['P(C, D)']\n",
"P_C.name = 'P(C)'\n",
"P_D = joint_probs.groupby('D').sum()['P(C, D)']\n",
"P_D.name = 'P(D)'\n",
"P_C"
]
},
{
"cell_type": "code",
"execution_count": 224,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(D\n",
" 1 0.43\n",
" 2 0.22\n",
" 3 0.22\n",
" 4 0.05\n",
" 5 0.03\n",
" 6 0.05\n",
" Name: P(D), dtype: float64,\n",
" 1.0)"
]
},
"execution_count": 224,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"P_D, P_D.sum()"
]
},
{
"cell_type": "code",
"execution_count": 376,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
C
\n",
"
D
\n",
"
P(C, D)
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
H
\n",
"
1
\n",
"
0.24
\n",
"
\n",
"
\n",
"
1
\n",
"
H
\n",
"
2
\n",
"
0.13
\n",
"
\n",
"
\n",
"
2
\n",
"
H
\n",
"
3
\n",
"
0.09
\n",
"
\n",
"
\n",
"
3
\n",
"
H
\n",
"
4
\n",
"
0.01
\n",
"
\n",
"
\n",
"
4
\n",
"
H
\n",
"
5
\n",
"
0.03
\n",
"
\n",
"
\n",
"
5
\n",
"
H
\n",
"
6
\n",
"
0.01
\n",
"
\n",
"
\n",
"
6
\n",
"
T
\n",
"
1
\n",
"
0.19
\n",
"
\n",
"
\n",
"
7
\n",
"
T
\n",
"
2
\n",
"
0.09
\n",
"
\n",
"
\n",
"
8
\n",
"
T
\n",
"
3
\n",
"
0.13
\n",
"
\n",
"
\n",
"
9
\n",
"
T
\n",
"
4
\n",
"
0.04
\n",
"
\n",
"
\n",
"
10
\n",
"
T
\n",
"
5
\n",
"
0.00
\n",
"
\n",
"
\n",
"
11
\n",
"
T
\n",
"
6
\n",
"
0.04
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" C D P(C, D)\n",
"0 H 1 0.24\n",
"1 H 2 0.13\n",
"2 H 3 0.09\n",
"3 H 4 0.01\n",
"4 H 5 0.03\n",
"5 H 6 0.01\n",
"6 T 1 0.19\n",
"7 T 2 0.09\n",
"8 T 3 0.13\n",
"9 T 4 0.04\n",
"10 T 5 0.00\n",
"11 T 6 0.04"
]
},
"execution_count": 376,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"joint_probs"
]
},
{
"cell_type": "code",
"execution_count": 368,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
C
\n",
"
D
\n",
"
P(C, D)
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
H
\n",
"
1
\n",
"
0.24
\n",
"
\n",
"
\n",
"
1
\n",
"
H
\n",
"
2
\n",
"
0.13
\n",
"
\n",
"
\n",
"
2
\n",
"
H
\n",
"
3
\n",
"
0.09
\n",
"
\n",
"
\n",
"
3
\n",
"
H
\n",
"
4
\n",
"
0.01
\n",
"
\n",
"
\n",
"
4
\n",
"
H
\n",
"
5
\n",
"
0.03
\n",
"
\n",
"
\n",
"
5
\n",
"
H
\n",
"
6
\n",
"
0.01
\n",
"
\n",
"
\n",
"
6
\n",
"
T
\n",
"
1
\n",
"
0.19
\n",
"
\n",
"
\n",
"
7
\n",
"
T
\n",
"
2
\n",
"
0.09
\n",
"
\n",
"
\n",
"
8
\n",
"
T
\n",
"
3
\n",
"
0.13
\n",
"
\n",
"
\n",
"
9
\n",
"
T
\n",
"
4
\n",
"
0.04
\n",
"
\n",
"
\n",
"
10
\n",
"
T
\n",
"
5
\n",
"
0.00
\n",
"
\n",
"
\n",
"
11
\n",
"
T
\n",
"
6
\n",
"
0.04
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" C D P(C, D)\n",
"0 H 1 0.24\n",
"1 H 2 0.13\n",
"2 H 3 0.09\n",
"3 H 4 0.01\n",
"4 H 5 0.03\n",
"5 H 6 0.01\n",
"6 T 1 0.19\n",
"7 T 2 0.09\n",
"8 T 3 0.13\n",
"9 T 4 0.04\n",
"10 T 5 0.00\n",
"11 T 6 0.04"
]
},
"execution_count": 368,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"joint_probs"
]
},
{
"cell_type": "code",
"execution_count": 403,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA2oAAAGPCAYAAAAtPSMTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAzZklEQVR4nO3debgkZXn38e9PBtzYVMYNkHFBEBUQR9SoccUgoigxBkWNK5I3qHljohgXXF6NxiVgxKCJKK6ogIqCIXHBfWFARNkUFcPINuxrBOR+/6g60tOcpftMn+nqc76f6zpXd1c9dddT1d3Pqbuep6pTVUiSJEmSuuM2466AJEmSJGltJmqSJEmS1DEmapIkSZLUMSZqkiRJktQxJmqSJEmS1DEmapIkSZLUMSZq0jpI8vIkBw9R/pgkuy9glfrXd26SJ81z2R2SrBpymfcl2X+Act9L8pD51GuxSPKxJP9viPL7JvmvAcu+MMl3h4h92yRnJbnroMusb0lOT/K4EcRZa98kuSbJfdY1bt86Tkzy0hnmJclHk1ye5MejXO+AdTssyRsHLDvyfTMqs+3jAZad9bvXu929ZZM8JsnZ86vxvOr5mSTPWI/ru1uSM5PcdpYyA7dbSbZL8pMkVyd55ehqKi0dJmrSLNpE5/r2H/dF7QHWxu28jYA3AO/uKb9Rkjcn+WWSa9vlD0+yoi3yTuDt631DptH+w72h3bbLkvx3ku17irwNeE9P+al9cXWSK5J8P8n+SXrbkXcDr2/3zUzrfRpwdVX9ZOQbtfZ6Ksn9FnId61NVfaqqnjyKWP0HuVX1e+Bw4LWjiL8QquqBVXXiAsTduKp+Peq4s3g0sBuwVVXtuh7XC0BV7V9Vbxuw7MD7ZjF932ba7qr6TlVtN/V6HU+ErWj32TXt37lJDuyZvyOwE/Clnmn3SPKRJBe07fBZSd6S5I4DrO+FSf7Qs77ftP/P7t+zfRcB3wT2m882TeM1wIlVtUlVvX9d9pe0VJmoSXN7WlVtDOwCPIwmOQPYCzirqn7XU/Yo4OnAc4HNaP7Rngw8EaCqfgxsmmTleqr7XP653batgIuBj0FzQAA8HvhiX/mnVdUmwDY0SedrgY9MzayqC4CzaPbBTPYHPjGa6i+MJMvGXYde66k+nwb+araz6Qula/t7gW0DnFtV1043c4nti2m1vY5L5fhk87YNfg7wptwy4uLlwKeqqgCS3Bn4AXB74JFtO7wbsDlw3wHX9YN2XZsBTwKuB05O8qCeMp9q1z0K2wCnjyiWtCQtlYZQWmdtQvZVYOqf2lOAb03Nb88U7gbsVVUnVdVNVXVlVR1aVR/pCXUi8NTp1pHkTkm+kmRNOzTqK0m26pl/YpK3pRk6eHWS/0qyRc/85yf5bZJLk7x+iG27juZAfWrbdgNOqar/naH8lVV1LPCXNAf3vf/oZ9u+jYAnsPZ+2yDJPyb5VbtNJyfZup23fdvTd1mSs5M8u2e5jyU5NMlx7XI/SnLfdt6322I/bc8e/2U7fc8kp/b0CO7YE+/cJK9Nchpw7XQHzEkOSXJekqvaej6mZ96bk3wuycfb+pzem5AneUiSU9p5nwVuN90+asu+sH2P/yXJZcCbc+she09u98mVST6Y5FvpGwqW5D3t5+g3SZ7STns78BjgA+2++QBAVa0GLgceMUOd3pzk80k+2W7Dz5LcP8nrklzc7pcn95R/UZphVFcn+XWSl/fMe1yS1e3+vhD4aJLbJzmire+ZSV6TZHXf+/OkAff1gT2fpzOSPHOWfV1J7pfknrmlt+GaJNclqZ5yL27rdXmSE5Js0zNvtzS9G1e2+zMzrOslwH8Aj2zX8ZYZ9sVtkxyc5Pz27+C0CXRP+de0+/2CJM9IskeSX6T5rvzjLNu71tC1JC9Lck673LFJ7tm/b3qWG+r71rfeqc/0v7b76awkT+yZf2KStyf5HnAdcJ8kf5LkpLb8SUn+pC/sfZP8uJ3/pTTJzFS8zye5sJ337SQP7Ft2izRty9Vpvju97+e0vYNT+759/gngXsCX221+TbtvXtG3zGkZYPhiVf2AJqmZ9v8L8HfA1cDzqurcdpnzqupVVXXaXPH71vWHqvpVVf2fdh1v7pn9I5p9v820C/fJDG1qkm/QnOybamc+Q9/+GqbO0lJloiYNKE3ysAcwNWTvwUDv9QpPAn5cVefNEepMmp626dwG+CjNmch70Zzx/EBfmecCLwLuCmwE/H1bvx2AfwOeD9wTuAtNT9mc0gzn3JeZt21abQ/hapoD/ymzbd+2wM1tUjDl72jOJu8BbAq8GLguzXCe/6ZJIO/alvlg3wHXc4C3AHcCzqEdVlpVf9rO36kdxvTZJLvQDO97Oc2++RBwbNbuQXoOTZK5eVXdNE39TwJ2Bu7c1uvzSXoTrqcDR9Kc5T6W9r1Lk6B+kaYn8c7A54E/n2EfTXk48Ot229caLpsmOT8KeF27LWcD/QexD2+nbwH8M/CRJKmq1wPfAQ5o980BPcvM9t4BPK3dhjvRfFZOoPnMbgm8lWafTrkY2JPmPX0R8C/tezDl7jT7YhuaoVYHASuA+9CcKHjeLPWAGfZ161c0n8nNaD4fn0zTSzyjqjq/3R8bt70OX2jj0x5o/yOwN7CcZv99pp23BXA0TU/7Fu26HzXDOj5C06P8g3Y9B82wL15PkzDvTPN+7MotPflT5W9Hs9/fBPw7zf56aLvdb8oA15YleQLwT8CzgXsAv53a5hkM/H2bYfmpz/QWNO/3Mb3JFU3btR+wCU1SchzwfprP+PuA45Lcpaf8C2jai3sCN7Vlp3yVpr25K3AKTU9Rr31phndvAZw6zfxZVdXzgf+hHXFRVf8MHEHP5zbJTjTv0fGzxUrjUcADgZ+0bd+9ufX/l2Oq6uZh6jmAY+hpv9t27xxmbwcAmK1NraonsHY78xxuvb8kzcFETZrbF5NcAXyX5uzjO9rpm9McTEy5C3DBAPGubpe9laq6tKqOrqrrqupqmgOhx/YV+2hV/aKqrgc+R3MwB/As4CtV9e32mqM3AnP9U//7dtvOATYGXthO35y1t20259McZE6ZcftmiPtS4A1VdXY1flpVl9Ic5J9bVR9teydPoTkgflbPssdU1Y/bg4tPccu+mM7LgA9V1Y/aM8pHAL9n7R6k97dnqa+fLkBVfbJ9j26qqvcCtwW26yny3ao6vqr+QJPQTB3sPALYEDi4qm6sqqNokr7ZnF9V/9quq78+ewCnV9Ux7ba/H7iwr8xvq+rf27ocQXMgfrc51jnbewfwnao6oV3n52mSlndW1Y00B/grkmwOUFXHtWftq6q+BfwXayf0NwMHVdXv2+17NvCOqrq8TeR7D7qnM9O+pqo+3yZeN7dJwy9pkp2BJHktsD1NEgDNgeg/VdWZ7ba/A9i57XXYAzijqo5q98PB3Pq9mEv/vtgXeGtVXVxVa2iSo+f3lL8ReHvPft8COKSqrq6q02l6ZnZkbvsCh1fVKW2b8Tqa3r4VM5Qf5vs2nYu55TvwWZpEpLf3/WNVdXob/8nAL6vqE+134DM0w6qf1lP+E1X182qGkb4ReHaSDQCq6vB2f/yepsdopySb9Sx7XE9b+fp2u7cecnv6fQnYNsm27evnA5+tqhtmWeYS4DKantYDq+rr3PIdnM//l2H1t99T69381kVvZZA2VdI6MFGT5vaMqtq8qrapqv/Tc9B8Oc2Z3ymX0hwMz2UT4IrpZiS5Q5IPpRm+eBXwbWDzqYOPVu9B4HU0CRY0Z5X/2JvXHrxcOkdd3tNu292r6ulV9at2ev+2zWZLmgONKTNu3wxxt6bphei3DfDwdkjNFW1CuS9Nb8KUmfbFdLYBXt0Xb2ua/TZl1t7QJK9OM/ztynb5zWgOkmeqz+3SDKG8J/C7qqqe+b+dbV1z1KX/vS6ans1eF/bMv659Otv+gdnfO4CLep5fD1zSJkpTr/+4jiRPSfLDNEPqrqBJaHr31Zpae2jtWtvEHO8FM+9rkrygZzjWFTTDybaYJsatpBki+iqa7/3UNm0DHNIT7zKa4Y1b9te7fS/mqnu/6fZF7+fjt6z9Ob10mv3e/97M9V7faj1VdQ1Nm7HlDOWH+b5NZ7rvwEzfv/59MFW+t27n9c3bkGZI4wZJ3plm+OtVwLltmS2mW7bd7sv66jK0Nun7HPC8NNfYPYe5r8fdoqruVFUPqKqpkxNXtI/z+f8yrP72e2q9V9y66K0M0qZKWgcmatL8nQbcv+f114Bd03NN2QweAPx0hnmvpumheXhVbQpMDSma9pqXPhfQ/JNsFkjuQHMWdj76t21aSR5G84++91bws23fL5vF0n+wNd3F8OcB32oTyam/javqrwfagunjvb0v3h3aM/VTaqaF01yP9lqanp87VdXmwJUM/t5smaS37L3mWGbGurTxeq9dDAMOc50j9mzv3cDa4aRH09w19G7tvjqetfdVfx3W2iZ6PstDrnsbmqGABwB3adf9cwZ4n5JsR9P7+OxaewjzecDL+z47t6+q73Pr713mUff+fXE+zUHwlHu100ZtrfW0Q+7uAvxuxiXWzXTfgd7t6t0P/ftgqnxv3bbum3cjTQ/Vc2lu9vQkmpMpK9oymW7Zduj3nRl+H0/3PTqC5oTSE4Hrqrn2bLigzUm2X3Hr/y/PzOhvsvJMmiGKwB9vZnM/BmsHBmlTe83WpkmahomaNH/H0zMssaq+RnNN1ReSPDTJsiSbpLmF/Yt7lnsszfUT09mE5mz4Fe21GwfNUG46RwF7Jnl0e03UW5n/d/y/gV36rr/6oySbJtmTZtjVJ6vqZz2zZ9y+dqjW11h7OOd/AG9Lsm17rcaO7XUoXwHun+YGKRu2fw9L8oABt+Eimuudpvw7sH+Sh7fruWOSpyYZtOdwE5rrYNYAy5K8ieb6q0H8oF32le3nYm+GGIo3jeOAB6e5icQy4G9Yu6dxLv37hjZ5vjPww3Wo15SNaIaFrgFuanup5vppgc8Br0tzQ50taRKt+bgjzQHhGmhuasItN2iYUZJNaYauvaGq+n+D7rC2bg9sy26W5C/aeccBD0yyd/tevJLh3ovpfAZ4Q5Llaa6BexPwyXWMOZ1PAy9KsnObXL8D+FG1N6sY0q0+U9O4K813YMN2/z2Ama/fOp7m+//c9jvzl8AONO3ClOel+b3HO9C0d0e1PY2b0AzBuxS4A7cMV++1R09b+Taa7R62J/RW29wmZjcD72Xd7m671v8Xmmv0NgWOaE9GkGTLNL9dOXUDjxOTvHmuwG2P472T/CvwOJqhtVN2pRlyPlePPwzfpg7yGZHUw0RNmr8vA9un5y5pNNdPHQ98lqa35efASprkZKoH6tpqbsIxnYNpbr98Cc0B838OWpn22pS/oTn4uoBmmGH/cLhBY10EfIPmrHSvLye5muZM6utpDh5eNDUzzQ0bduDWt/Xv9SHWvt7mfTQH6f8FXEVzu//bV3ON3pOBfWjOdF8IvIsmARjEm2kOaq5I8uyqWkVzTcUHaPbNOdxyTd4gTqBJQH9BM8zqfxlwiFt7jcre7foup7lb5jFDrLs/3iXAX9DcJORSmn2+iubgdBCHAM9KcwfDqeFWzwWOaIdvrZP2vXslzft6eRv72DkWeyvN5/U3NN+Xoxh8e3rXfQbNQfIPaA4MHwx8b4BFd6HpzX5feu7+2Mb8As1n78h2KN3Pae7K1/tevJPmvdh2wPXN5v/RvJ+nAT+juRnGwD+OPqj2eqg30vR+XkDTs73PPMO9mZ7v2wxlfkSzfy6huf72WdVcjzpd3aauU301zX59DbBnu7+nfILmJ0UupLm5ytSPKn+c5jv6O+AMpj/58GmaE2GX0dyEZd9BNrLPP9Ek1Fck+fue6R+n+dytS3L9YWDfqR7IqrqM5oZBNwI/atvhr9P8nzmnXWZrZv/sPbL9TF9Fc3feTYGH9Z1o25fmxMSc5tGmzrS/JM0gaw8XlzSMJPsBO1TV3w5Y/mjgI1U1613AuiDNXSSPAHatARuKJO8FflVVH5yj3HeBV9QC/+j1UtEOh1oN7FtV35zH8relGer0p1V18ajrNx9J/hrYp6r6b6ajdZDk48A5VfXW9bzeFwIvrapHr8/1jkOSFwD7reu2Jvk08Lmq+uIAZbcCPl9Vj1yH9d2V5oZZD6kZfppF0vploiZJEyjJn9H0UFwP/ANNb+p9aoY7VnZd2xt7H5qesG1phhR+oKoOHme9FpN2aOb3aPbrev3R+aWSqLXDML8BfLCqPj7u+kiabA59lKTJ9EiaGw5cQnPL8mdMapLW2ohmWOzVNAe6XwJm7ZnV0C6kuZvf0WOux6LUnjxZQzPk9tNjro6kRcAeNUmSJEnqGHvUJEmSJKljTNQkSZIkqWNM1CRJkiSpY0zUJEmSJKljTNQkSZIkqWNM1CRJkiSpY0zUNLQkL09y8Hpe54+TPHB9rlPS4jJs25XkmCS7L2CVJGlex1UeFy0NJmq6lSTnJrk+yTVJLkry0SQbt/M2At4AvLun/EZJ3pzkl0mubZc/PMmKAda1Ikm165pa31eS7NZX9D3AW0e4mZIWoZnar3m2Xe8E3j6GzZC0yAzaNnlcpF4maprJ06pqY2AX4GE0jQjAXsBZVfW7nrJHAU8HngtsBuwEnAw8cYj1bd6ubyfgv4EvJHlhz/xjgccnucc8tkXS0jJd+zV021VVPwY2TbJyPdZd0uI1aNsEHhcJWDbuCqjbqup3Sb4KPKid9BTgW1PzkzwJ2A24f1Wd106+Ejh0nuu7EDgkyYbAu5J8vKpurqr/TXIy8GTgiHlujqQlpK/9uivza7tOBJ4KrFrwCktaEmZrm6Yp63HREmaPmmaVZGtgD+An7aQHA2f3FHkS8OOeA51ROYam8dquZ9qZNGeWJGlOfe3XfNsu2x1JIzVH2zQTj4uWIBM1zeSLSa4Avktzpucd7fTNgat7yt0FuGAB1n9++3jnnmlXt+uXpNlM135tzvzaLtsdSaMySNs0E4+LliCHPmomz6iqr00z/XJgk57XlwL3X4D1b9k+XtYzbRPgigVYl6TF5VbtV5L5tl22O5JGZZC2aSYeFy1B9qhpWKex9sHN14Bdk2w14vU8E7iYtYcDPAD46YjXI2lpmG/bZbsjaSH1t00z8bhoCTJR07COBx479aI9MzR1N6KHJlmWZJMk+yd5MUB7++sTBwme5G5JDgAOAl5XVTe3028LPLRdlyQNa+i2q/VY4Kvrua6Slo612qZ+HhctbSZqGtaXge2T3LNn2rNoGprP0tw17efASpoz1gBbA9+bI+4VSa4FfkZzge1fVNXhPfOfDpxYVedPu7QkzW7otivJw4Br29v0S9JCmK5tAo+LBKSqxl0HTZgk+wE7VNXfDlj+VOCJVXXpOqzzR8BLqurn840haWmbR9t1NPCRqjp+QSsmaUkbtm1ql/G4aAkwUZMkSZKkjnHooyRJkiR1jImaJEmSJHWMiZokSZIkdYyJmiRJkiR1zLJxrXiLLbaoFStWjGv1khbAySeffElVLR93PdaV7ZO0+CyG9sm2SVp8ZmubxpaorVixglWrVo1r9ZIWQJLfjrsOo2D7JC0+i6F9sm2SFp/Z2iaHPkqSJElSx5ioSZIkSVLHmKhJkiRJUseYqEmSJElSx5ioSZIkSVLHmKhJkiRJUseYqEmSJElSx5ioSZIkSVLHmKhJkiRJUseYqEmSJElSx5ioSZIkSVLHLBt3BSRpkq048LhxV0Gtc9/51HFXQZKkkRmoRy3J7knOTnJOkgOnmf+4JFcmObX9e9PoqypJkiRJS8OcPWpJNgAOBXYDVgMnJTm2qs7oK/qdqtpzAeooSZIkSUvKID1quwLnVNWvq+oG4Ehgr4WtliRJkiQtXYMkalsC5/W8Xt1O6/fIJD9N8tUkDxxJ7SRJkiRpCRokUcs006rv9SnANlW1E/CvwBenDZTsl2RVklVr1qwZqqKStJBsnyR1kW2TtHQNkqitBrbueb0VcH5vgaq6qqquaZ8fD2yYZIv+QFX14apaWVUrly9fvg7VlqTRsn2S1EW2TdLSNUiidhKwbZJ7J9kI2Ac4trdAkrsnSft81zbupaOurCRJkiQtBXPe9bGqbkpyAHACsAFweFWdnmT/dv5hwLOAv05yE3A9sE9V9Q+PlCRJkiQNYKAfvG6HMx7fN+2wnucfAD4w2qpJkiRJ0tI0UKImSZJgxYHHjbsK6nHuO5867ipI0oIZ5Bo1SZIkSdJ6ZKImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR2zbNwVkCRJ0mglbxnbuqsOGtu6pcXEHjVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqmIEStSS7Jzk7yTlJDpyl3MOS/CHJs0ZXRUmSJElaWuZM1JJsABwKPAXYAXhOkh1mKPcu4IRRV1KSJEmSlpJBetR2Bc6pql9X1Q3AkcBe05R7BXA0cPEI6ydJkiRJS84gidqWwHk9r1e30/4oyZbAM4HDZguUZL8kq5KsWrNmzbB1laQFY/skqYtsm6Sla5BELdNMq77XBwOvrao/zBaoqj5cVSurauXy5csHrKIkLTzbJ0ldZNskLV3LBiizGti65/VWwPl9ZVYCRyYB2ALYI8lNVfXFUVRSkiRJkpaSQRK1k4Btk9wb+B2wD/Dc3gJVde+p50k+BnzFJE2SJEmS5mfORK2qbkpyAM3dHDcADq+q05Ps386f9bo0SZIkSdJwBulRo6qOB47vmzZtglZVL1z3akmSJEnS0jXQD15LkiRJktYfEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqmIEStSS7Jzk7yTlJDpxm/l5JTktyapJVSR49+qpKkiRJ0tKwbK4CSTYADgV2A1YDJyU5tqrO6Cn2deDYqqokOwKfA7ZfiApLkiRJ0mI3SI/arsA5VfXrqroBOBLYq7dAVV1TVdW+vCNQSJIkSZLmZZBEbUvgvJ7Xq9tpa0nyzCRnAccBL54uUJL92qGRq9asWTOf+krSgrB9ktRFtk3S0jVIopZppt2qx6yqvlBV2wPPAN42XaCq+nBVrayqlcuXLx+qopK0kGyfJHWRbZO0dA2SqK0Gtu55vRVw/kyFq+rbwH2TbLGOdZMkSZKkJWmQRO0kYNsk906yEbAPcGxvgST3S5L2+S7ARsClo66sJEmSJC0Fc971sapuSnIAcAKwAXB4VZ2eZP92/mHAnwMvSHIjcD3wlz03F5EkSZIkDWHORA2gqo4Hju+bdljP83cB7xpt1SRJkiRpaRroB68lSZIkSeuPiZokSZIkdYyJmiRJkiR1jImaJEmSJHWMiZokSZIkdYyJmiRJkiR1jImaJEmSJHWMiZokSZIkdYyJmiRJkiR1jImaJEmSJHWMiZokSZIkdYyJmiRJkiR1jImaJEmSJHWMiZokSZIkdYyJmiRJkiR1jImaJEmSJHWMiZokSZIkdYyJmiRJkiR1jImaJEmSJHWMiZokSZIkdYyJmiRJkiR1jImaJEmSJHWMiZokSZIkdYyJmiRJkiR1jImaJEmSJHXMsnFXYBDHnH3BOi2/93b3GFFNJEmSJGnh2aMmSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkds2zcFZAkSdLSkbxlbOuuOmjGeV2tl5Yue9QkSZIkqWNM1CRJkiSpY0zUJEmSJKljTNQkSZIkqWNM1CRJkiSpYwZK1JLsnuTsJOckOXCa+fsmOa39+36SnUZfVUmSJElaGuZM1JJsABwKPAXYAXhOkh36iv0GeGxV7Qi8DfjwqCsqSZIkSUvFID1quwLnVNWvq+oG4Ehgr94CVfX9qrq8fflDYKvRVlOSJEmSlo5BErUtgfN6Xq9up83kJcBX16VSkiRJkrSULRugTKaZVtMWTB5Pk6g9eob5+wH7AdzrXvcasIqStPBsnyR1kW2TtHQN0qO2Gti65/VWwPn9hZLsCPwHsFdVXTpdoKr6cFWtrKqVy5cvn099JWlB2D5J6iLbJmnpGiRROwnYNsm9k2wE7AMc21sgyb2AY4DnV9UvRl9NSZIkSVo65hz6WFU3JTkAOAHYADi8qk5Psn87/zDgTcBdgA8mAbipqlYuXLUlSZIkafEa5Bo1qup44Pi+aYf1PH8p8NLRVk2SJEmSlqaBEjVpFI45+4J5L7v3dvcYYU0kSZKkbhvkGjVJkiRJ0npkoiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHWOiJkmSJEkdY6ImSZIkSR1joiZJkiRJHbNs3BVY3445+4J1Wn7v7e4xopqoK9blM+HnQZIkSQvBHjVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqGBM1SZIkSeoYEzVJkiRJ6hgTNUmSJEnqmIEStSS7Jzk7yTlJDpxm/vZJfpDk90n+fvTVlCRJkqSlY9lcBZJsABwK7AasBk5KcmxVndFT7DLglcAzFqKSkiRJkrSUDNKjtitwTlX9uqpuAI4E9uotUFUXV9VJwI0LUEdJkiRJWlIGSdS2BM7reb26nSZJkiRJWgCDJGqZZlrNZ2VJ9kuyKsmqNWvWzCeEJC0I2ydJXWTbJC1dgyRqq4Gte15vBZw/n5VV1YeramVVrVy+fPl8QkjSgrB9ktRFtk3S0jVIonYSsG2SeyfZCNgHOHZhqyVJkiRJS9ecd32sqpuSHACcAGwAHF5VpyfZv51/WJK7A6uATYGbk/wtsENVXbVwVZckSZKkxWnORA2gqo4Hju+bdljP8wtphkRKkiRJktbRQImapnfM2RfMe9m9t7vHCGuy9LjvJUmStJgNco2aJEmSJGk9MlGTJEmSpI4xUZMkSZKkjjFRkyRJkqSOMVGTJEmSpI4xUZMkSZKkjjFRkyRJkqSOMVGTJEmSpI4xUZMkSZKkjjFRkyRJkqSOMVGTJEmSpI4xUZMkSZKkjjFRkyRJkqSOMVGTJEmSpI4xUZMkSZKkjjFRkyRJkqSOMVGTJEmSpI4xUZMkSZKkjjFRkyRJkqSOWTbuCkiSJEmaWfKWsa276qCxrXtdLIZ9Zo+aJEmSJHWMiZokSZIkdYyJmiRJkiR1jImaJEmSJHWMNxORRuiYsy+Y97J7b3ePEdZkbV2tlyRJkqZnj5okSZIkdYyJmiRJkiR1jImaJEmSJHWMiZokSZIkdYw3E1mEvHGEJEmS1ofkLWNZb9VBY1nv+mSPmiRJkiR1jImaJEmSJHWMiZokSZIkdYyJmiRJkiR1jImaJEmSJHWMiZokSZIkdYyJmiRJkiR1jImaJEmSJHWMiZokSZIkdcyycVdAjWPOvmDey+693T1GWJO1dbVekiRJ0mJmj5okSZIkdcxAiVqS3ZOcneScJAdOMz9J3t/OPy3JLqOvqiRJkiQtDXMmakk2AA4FngLsADwnyQ59xZ4CbNv+7Qf824jrKUmSJElLxiA9arsC51TVr6vqBuBIYK++MnsBH6/GD4HNk3iBkiRJkiTNQ6pq9gLJs4Ddq+ql7evnAw+vqgN6ynwFeGdVfbd9/XXgtVW1qi/WfjQ9bgDbAWePaDu2AC4xlrGMNfZY21TV8hHEWe8WsH2aFKP8PKn7luL7PZHt0xjbpq5+RrpaL+hu3bpaL7BuMEvbNMhdHzPNtP7sbpAyVNWHgQ8PsM6hJFlVVSuNZSxjjT/WpFqo9mlS+BlYWny/J8e42qaufka6Wi/obt26Wi+wbnMZZOjjamDrntdbAefPo4wkSZIkaQCDJGonAdsmuXeSjYB9gGP7yhwLvKC9++MjgCurav4/wCVJkiRJS9icQx+r6qYkBwAnABsAh1fV6Un2b+cfBhwP7AGcA1wHvGjhqjytUQ4JMJaxjKWlyM/A0uL7rbl09TPS1XpBd+vW1XqBdZvVnDcTkSRJkiStXwP94LUkSZIkaf0xUZMkSZKkjjFRk6QlKsk1fa9fmOQD46qPFlaSuyQ5tf27MMnvel5vNO76qRuSHJ7k4iQ/H3ddeiXZOsk3k5yZ5PQkrxp3nQCS3C7Jj5P8tK3XW8Zdp35JNkjyk/Z3jzsjyblJfta2QavmXmL9SLJ5kqOSnNV+3h45rrosmkQtydA3MEmyfZInJtm4b/ru84i1a5KHtc93SPJ3SfYYNs4MsT8+ojiPbuv15Hks+/Akm7bPb5/kLUm+nORdSTYbRf0WuyS7jLsO/ZJsmuShSe407rpIWlhVdWlV7VxVOwOHAf8y9bqqbhhz9dQdHwOGPg5aD24CXl1VDwAeAfxNkh3GXCeA3wNPqKqdgJ2B3ds7oHfJq4Azx12JGTy+bYO69FtqhwD/WVXbAzsxxn23aBI1YKgzGEleCXwJeAXw8yR79cx+x5CxDgLeD/xbkn8CPgBsDByY5PVDxjq27+/LwN5Tr4eM9eOe5y9r67UJcFCSA4eJBRxOc0dPaD7AmwHvaqd9dMhYM0rysyHLb53kyCTfSfKPSTbsmffFIWNtn+SrSY5Lct8kH0tyRXum7AFDxtql7++hwLFJHjJswpbkxT3Pt0ry9bZe309y/yFjfTLJFu3zPwNOp3kfT03yF8PEkiQtPlX1beCycdejX1VdUFWntM+vpjl43nK8tYJqTI1O2LD968yd+pJsBTwV+I9x12UStJ0Sfwp8BKCqbqiqK8ZVnzlvz98lSU6baRZwtyHDvQx4aFVdk2QFcFSSFVV1SBtvGM+iOYtyW+BCYKuquirJu4EfAW8fItZWwBk0X6hq67ISeO+QdYKmsZiyH7BbVa1J8h7gh8A7h4h1m6q6qX2+sqqmko3vJjl1mEol2XumWcDdh4lFk0AeTbM9LwG+leRpVXUpsM2QsT4MvJsmyf4G8Fqan5rYkybJfeIQsVa1dfp9z7S7AO+jeV+fMESsA2i2k3b5zwG7AXsB/zZkvXaqqkva5wcBj6mqc9vk7evA54eIpcl3+77v75259e9kSlKntMdtD6E5xhq7JBsAJwP3Aw6tqk7Uq3Uw8BqaE/VdU8B/JSngQ1U19tvhA/cB1gAfTbITzfv6qqq6dhyVmahEjSYZ+zPg8r7pAb4/ZKwNps6AtAeqj6NJ1rZh+ETtpqr6A3Bdkl9V1VVt3OuT3DxkrJU0XdSvB/6hqk5Ncn1VfWvIOAC3aYe03YbmpxjWtPW6NslNsy96Kz9P8qKq+ijw0yQrq2pV26Nz45CxPgt8iunPON1uyFjL29/yA3hFkucB307y9Bniz2aTqvoyQJK3VdWR7fQvz2PM+bNpemvfXVXHtzF/U1WPHzJOv/tX1bPb519I8qYhl79Nkk3bz+jNwP8AVNUlSSatPdC6u74dBgc016jRtEGS1Ent5SpHA387dbw1bu0x4M5JNqf53/ygqhr7NX5J9gQurqqT2+PcrnlUVZ2f5K7Afyc5q+3RHadlwC7AK6rqR0kOAQ4E3jiuykySrwAbV9Wp/TOSnDhkrAuT7DwVq+1Z25Om5+LBQ8a6Ickdquo64KE9ddqM5mB4YFV1M/AvST7fPl7E/N+nzWjOBASoJHevqgvbRm7YZPSlwCFJ3gBcAvwgyXnAee28YZwGvGe6RizJk4aMtWGS21XV/wJU1SeTXEjzA+13HDLWBj3P39c3b6gL7avqqCT/CbwtzfWTr2b+QyG2SvJ+mvdseZINq2oqOd5wluWm8xbgm0kOBb4HfD7Jl2h6+P5znvWTJGnBtZc3HA18qqqOGXd9+lXVFe3x6O7A2BM14FHA09PcM+F2wKZJPllVzxtzvQCoqvPbx4uTfAHYFRh3orYaWN3TK3oUTaI2FhOVqFXVS2aZ99whw72A5sLU3hg3AS9I8qEhY/1pVf2+jdGbmG0I/NWQsabqshr4iyRPBeZ1xqiqVsww62bgmUPGuhJ4YZJNaLqFl9F8kC+aR9X+lpm3aah60QwRfTjwxx7Hqvpae73VPw8Z69AkG1fVNVX1wamJSe4HfG3IWLQ9tv83yUOAI2iGVM7HP/Q8X9XGuTzJ3RlymFpVfS7JKTRDf+9P8z4+EvhMVZ0wz/pJkrSgkoTmuqEzq6r/ZOrYJFkO3NgmabcHnkRz7ffYVdXrgNcBtD1qf9+VJC3JHWkuq7m6ff5k4K1jrhZth8Z5SbarqrNpLi85Y1z1SVVnrneUFq32H8wmXRmmIUFze/6q2rjn9QtprkE9YHy10vqQ5M3ANVX1nnHXRd2S5DPA44AtgIuAg6rqI2OtFM2dq4HvAD/jltFK/zh1ecG4JNmR5mTsBjSXmnyuqsaecPTrSdT2HHNVAEhyH+AL7ctlwKeraph7OiyYJDvTdARsBPwaeFFV9V92tX7qMqmJWpLvVtWjpx6NNbGxntcOV3xeVX3SWJMZS5IkSaM1ybfnv0P7OOx1SMbqVqy/63s01mTGkiRJ0ghNcqKmxWXYm5sYq5uxJEmSNAImapIkSZLUMSZqkiRJktQxk5yodXXol7EkSZIkrZNJTtT+b9+jsSYz1i/ax7ONNdGxJEmSNEITe3t+SZIkSVqslo27AsNK8lfAq4Dt2klnAu+vqo8ba6JiLQOeAmzfE+s/q+omY01OLEmSJC2MiepRS/ICmuF2fwecQnOt1C7Au4FDhkkYjDXWWPcEvglcAPykjfUQ4O7A46vqfGN1P5YkSZIWzqQlaj8E9qmqc/umrwCOrKpHGGsiYn0MOLWqDu6b/krgoVX1V8bqfixJkiQtnElL1M6oqh2GnWeszsU6q6q2n2He2VW13XTzjNWtWJIkSVo4k3bXx+vnOc9YkxPrOmNNTCxJkiQtkEm7mcgDkpw2zfQA9zHWxMTaLMneM8Ta1FgTE0uSJEkLZOISNWMtiljfAp42w7xvG2tiYkmSJGmBTNQ1apIkSZK0FEzaNWqSJEmStOiZqEmSJElSx5ioSZIkSVLHTNTNRJL8DJjxorqq2tFYExFrursO9sY6xljdjyVJkqSFM1GJGrBn+/g37eMn2sd9Gf43oIw1vlhTdx28K/AnwDfa148HTgSGSRaMNb5YkiRJWiATedfHJN+rqkfNNc1YnY/1FeBlVXVB+/oewKFVNWuvj7G6FUuSJEmjN6nXqN0xyaOnXiT5E+COxpq4WCumEoXWRcD9jTVxsSRJkjRikzb0ccpLgMOTbNa+vgJ4sbEmLtaJSU4APkNz/ds+wDeNNXGxJEmSNGITOfRxSpJNabbhSmNNZqz25haPaV9+u6q+YKzJiyVJkqTRmshELcltgT8HVtDTK1hVbzXW5MSSJEmSNL1JvUbtS8BewE3AtT1/xpqgWEn2TvLLJFcmuSrJ1UmuMtZkxZIkSdLoTWqP2s+r6kHGmvhY5wBPq6ozjTW5sSRJkjR6k9qj9v0kDzbWxMe6aISJgrHGF0uSJEkjNqk9amcA9wN+A/weCFBVtaOxJirWIcDdgS+2saAJNvSPLhtrfLEkSZI0epN6e/6nGGtRxNoUuA54cs+0AuaTLBhrfLEkSZI0YhPZozYlyV2B2029rqr/MdbkxZIkSZK0tonsUUvydOC9wD2Bi4FtgDOBBxpromLdjuYHtB/I2knf0D+gbazxxZIkSdLoTerNRN4GPAL4RVXdG3gi8D1jTVysT9BcJ/VnwLeArYCrjTVxsSRJkjRik5qo3VhVlwK3SXKbqvomsLOxJi7W/arqjcC1VXUE8FRgvneUNNb4YkmSJGnEJnLoI3BFko2BbwOfSnIxzQ8wG2uyYt3YE/NBwIXACmNNXCxJkiSN2ETeTCTJHYHraXoE9wU2Az7V9vQYa3JivRQ4mqYn52PAxsAbq+pDxpqcWJIkSRq9iUzUJEmSJGkxm9Rr1CRJkiRp0TJRkyRJkqSOMVHTepfkEcaa/FiSJElaOBN1jVqSnwHTVThAVdWOxpqIWKdU1S6DljdWN2NJkiRp4Uza7fn3bB8DHAfsYayJjCVJkiRpFhPVo9arq70Mxhpo+StofodtWlX1dGN1P5YkSZIWzqT1qGlxWAO811gTH0uSJEkLZKIStSS9vTi3T/IQmqF4AFTVKcbqfizg6qr61hDljdXNWJIkSVogEzX0Mck3Z5ldVfUEY01ErGOqau9Byxurm7EkSZK0cCYqUdPik2T7qjpr6tFYkxlLkiRJozWxv6OWZPveR2NNZizg032PxprMWJIkSRqhiU3U6O4Bq7HmJ3MXMdYExJIkSdIITHKiNqWrB6zGkiRJkjQviyFRkyRJkqRFxURNXTHKu9oYa3yxJEmSNAKLIVHr6gGrsQaTvkdjTWYsSZIkjdAkJ2pdPWA11nAe0/dorMmMJUmSpBGa2N9RS7JxVV0z9WisyYmV5H7A3arqe33THwOcX1W/Mlb3Y0mSJGnhTFSPWpL7JXkUwFSC0CYMj0lyX2NNRizgYODqaaZf384z1mTEkiRJ0gKZqESN7h6wGms4K6rqtP6JVbUKWGGsiYklSZKkBTJpiVpXD1iNNZzbzTLv9saamFiSJElaIJOWqHX1gNVYwzkpycv6JyZ5CXCysSYmliRJkhbIRN1MJMlngG9U1b/3TX8J8OSq+ktjTUSsuwFfAG7gluRgJbAR8MyqutBY3Y8lSZKkhTNpiVonD1iNNb8D/CSPBx7Uvjy9qr4xbAxjjT+WJEmSRm+iErUpXT1gNZYkSZKkUZjIRE2SJEmSFrNJu5mIJEmSJC16JmqSJEmS1DEmapIkSZLUMSZqkiRJktQxJmqSJEmS1DH/HyiQrKf9Rh5TAAAAAElFTkSuQmCC",
"text/plain": [
"
"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"fig, axs = plt.subplots(1, 3, figsize=(15, 5), sharey=True)\n",
"\n",
"axs[0].bar(joint_probs[\"CD_vals\"], joint_probs[\"P(C, D)\"], color=\"lightblue\");\n",
"axs[1].bar(P_C.index, P_C);\n",
"axs[2].bar(P_D.index, P_D, color=\"navy\");\n",
"\n",
"axs[0].tick_params('x', labelrotation=90)\n",
"\n",
"axs[0].set_title(\"P(C, D)\");\n",
"axs[1].set_title(\"P(C)\");\n",
"axs[2].set_title(\"P(D)\");\n",
"\n",
"fig.suptitle(\"P(C) and P(D) (center and right) marginalized from joint probability P(C, D) left\");"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"As we look at new concepts in probability, it is important to stay mindful of i) what the probability sums to ii) what are the dimensions of the table that represents the probability.\n",
"\n",
"You can see from the cell below that the dimensions of marginal probability table is the length of the range of the variable.\n",
"\n",
"You can see from the code below that both the computed marginal probabilities in add up to 1. "
]
},
{
"cell_type": "code",
"execution_count": 230,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(1.0, 1.0)"
]
},
"execution_count": 230,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"P_C.sum().round(3), P_D.sum().round(3)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Independent Random Variables\n",
"\n",
"Random variables can be either independent or dependent. If two random variables are independent, then the value of one random variable does not affect the value of the other random variable. \n",
"\n",
"For example, if we are rolling two dice, we can use two random variables to represent the numbers that we roll. The two random variables are independent because the value of one die does not affect the value of the other die. If two random variables are dependent, then the value of one random variable does affect the value of the other random variable. For example, if we are measuring the temperature and the humidity, we can use two random variables to represent the temperature and the humidity. The two random variables are dependent because the temperature affects the humidity and the humidity affects the temperature.\n",
"\n",
"More formally, **two random variables $X$ and $Y$ are independent if and only if $P(X, Y) = P(X) \\cdot P(Y)$**."
]
},
{
"cell_type": "code",
"execution_count": 214,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
C
\n",
"
D
\n",
"
P(C, D)
\n",
"
P(C)
\n",
"
P(D)
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
H
\n",
"
1
\n",
"
0.24
\n",
"
0.51
\n",
"
0.43
\n",
"
\n",
"
\n",
"
1
\n",
"
T
\n",
"
1
\n",
"
0.19
\n",
"
0.49
\n",
"
0.43
\n",
"
\n",
"
\n",
"
2
\n",
"
H
\n",
"
2
\n",
"
0.13
\n",
"
0.51
\n",
"
0.22
\n",
"
\n",
"
\n",
"
3
\n",
"
T
\n",
"
2
\n",
"
0.09
\n",
"
0.49
\n",
"
0.22
\n",
"
\n",
"
\n",
"
4
\n",
"
H
\n",
"
3
\n",
"
0.09
\n",
"
0.51
\n",
"
0.22
\n",
"
\n",
"
\n",
"
5
\n",
"
T
\n",
"
3
\n",
"
0.13
\n",
"
0.49
\n",
"
0.22
\n",
"
\n",
"
\n",
"
6
\n",
"
H
\n",
"
4
\n",
"
0.01
\n",
"
0.51
\n",
"
0.05
\n",
"
\n",
"
\n",
"
7
\n",
"
T
\n",
"
4
\n",
"
0.04
\n",
"
0.49
\n",
"
0.05
\n",
"
\n",
"
\n",
"
8
\n",
"
H
\n",
"
5
\n",
"
0.03
\n",
"
0.51
\n",
"
0.03
\n",
"
\n",
"
\n",
"
9
\n",
"
T
\n",
"
5
\n",
"
0.00
\n",
"
0.49
\n",
"
0.03
\n",
"
\n",
"
\n",
"
10
\n",
"
H
\n",
"
6
\n",
"
0.01
\n",
"
0.51
\n",
"
0.05
\n",
"
\n",
"
\n",
"
11
\n",
"
T
\n",
"
6
\n",
"
0.04
\n",
"
0.49
\n",
"
0.05
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" C D P(C, D) P(C) P(D)\n",
"0 H 1 0.24 0.51 0.43\n",
"1 T 1 0.19 0.49 0.43\n",
"2 H 2 0.13 0.51 0.22\n",
"3 T 2 0.09 0.49 0.22\n",
"4 H 3 0.09 0.51 0.22\n",
"5 T 3 0.13 0.49 0.22\n",
"6 H 4 0.01 0.51 0.05\n",
"7 T 4 0.04 0.49 0.05\n",
"8 H 5 0.03 0.51 0.03\n",
"9 T 5 0.00 0.49 0.03\n",
"10 H 6 0.01 0.51 0.05\n",
"11 T 6 0.04 0.49 0.05"
]
},
"execution_count": 214,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"P_C.name = \"P(C)\"\n",
"P_D.name = \"P(D)\"\n",
"merged = pd.merge(joint_probs, P_C, on='C')\n",
"merged = pd.merge(merged, P_D, on='D')\n",
"merged"
]
},
{
"cell_type": "code",
"execution_count": 217,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
C
\n",
"
D
\n",
"
P(C, D)
\n",
"
P(C)
\n",
"
P(D)
\n",
"
P(C) P(D)
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
H
\n",
"
1
\n",
"
0.24
\n",
"
0.51
\n",
"
0.43
\n",
"
0.2193
\n",
"
\n",
"
\n",
"
1
\n",
"
T
\n",
"
1
\n",
"
0.19
\n",
"
0.49
\n",
"
0.43
\n",
"
0.2107
\n",
"
\n",
"
\n",
"
2
\n",
"
H
\n",
"
2
\n",
"
0.13
\n",
"
0.51
\n",
"
0.22
\n",
"
0.1122
\n",
"
\n",
"
\n",
"
3
\n",
"
T
\n",
"
2
\n",
"
0.09
\n",
"
0.49
\n",
"
0.22
\n",
"
0.1078
\n",
"
\n",
"
\n",
"
4
\n",
"
H
\n",
"
3
\n",
"
0.09
\n",
"
0.51
\n",
"
0.22
\n",
"
0.1122
\n",
"
\n",
"
\n",
"
5
\n",
"
T
\n",
"
3
\n",
"
0.13
\n",
"
0.49
\n",
"
0.22
\n",
"
0.1078
\n",
"
\n",
"
\n",
"
6
\n",
"
H
\n",
"
4
\n",
"
0.01
\n",
"
0.51
\n",
"
0.05
\n",
"
0.0255
\n",
"
\n",
"
\n",
"
7
\n",
"
T
\n",
"
4
\n",
"
0.04
\n",
"
0.49
\n",
"
0.05
\n",
"
0.0245
\n",
"
\n",
"
\n",
"
8
\n",
"
H
\n",
"
5
\n",
"
0.03
\n",
"
0.51
\n",
"
0.03
\n",
"
0.0153
\n",
"
\n",
"
\n",
"
9
\n",
"
T
\n",
"
5
\n",
"
0.00
\n",
"
0.49
\n",
"
0.03
\n",
"
0.0147
\n",
"
\n",
"
\n",
"
10
\n",
"
H
\n",
"
6
\n",
"
0.01
\n",
"
0.51
\n",
"
0.05
\n",
"
0.0255
\n",
"
\n",
"
\n",
"
11
\n",
"
T
\n",
"
6
\n",
"
0.04
\n",
"
0.49
\n",
"
0.05
\n",
"
0.0245
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" C D P(C, D) P(C) P(D) P(C) P(D)\n",
"0 H 1 0.24 0.51 0.43 0.2193\n",
"1 T 1 0.19 0.49 0.43 0.2107\n",
"2 H 2 0.13 0.51 0.22 0.1122\n",
"3 T 2 0.09 0.49 0.22 0.1078\n",
"4 H 3 0.09 0.51 0.22 0.1122\n",
"5 T 3 0.13 0.49 0.22 0.1078\n",
"6 H 4 0.01 0.51 0.05 0.0255\n",
"7 T 4 0.04 0.49 0.05 0.0245\n",
"8 H 5 0.03 0.51 0.03 0.0153\n",
"9 T 5 0.00 0.49 0.03 0.0147\n",
"10 H 6 0.01 0.51 0.05 0.0255\n",
"11 T 6 0.04 0.49 0.05 0.0245"
]
},
"execution_count": 217,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"merged['P(C) P(D)'] = merged['P(C)'] * merged['P(D)']\n",
"merged"
]
},
{
"cell_type": "code",
"execution_count": 218,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"sns.barplot(data=merged, x=\"CD_vals\", y=\"P(C, D)\", color=\"navy\", alpha=0.5, label=\"$P(C, D)$\");\n",
"sns.barplot(data=merged, x=\"CD_vals\", y=\"P(C) P(D)\", color=\"orange\", alpha=0.5, label=\"$P(C)\\cdot P(D)$\");\n",
"plt.xticks(rotation=90);\n",
"plt.ylabel(\"Probability\");\n",
"plt.legend();\n",
"plt.title(\"$C$ and $D$ are NOT independent since $P(C, D) \\\\neq P(C) \\cdot P(D)$\");"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Conditional Probability $P(A | B)$\n",
"\n",
"Conditional probability is the probability of one event occurring given that another event has occurred. \n",
"\n",
"The conditional probability is usually denoted by $P(A | B)$ and is defined as:\n",
"\n",
"$$ P(A | B) = \\frac{P(A, B)}{P(B)} $$\n",
"\n",
"The denominator is the marginal probability of $B$.\n",
"\n",
"\n",
"\n",
" \n",
"\n",
"For example, if we are flipping two coins, the conditional probability of flipping heads in the second toss, knowing the first toss was tails is: \n",
"\n",
"| Possible world | $\\text{Coin}_1$ | $\\text{Coin}_2$ | $P(\\omega)$ |\n",
"|:----------------:|:-------------:|:-------------:|:-------------:|\n",
"| $\\omega_1$ | H | H | 0.25 |\n",
"| $\\omega_2$ | H | T | 0.25 |\n",
"| $\\omega_3$ | T | H | 0.25 |\n",
"| $\\omega_4$ | T | T | 0.25 |\n",
"\n",
"$$ P(\\text{Coin}_2 = H | \\text{Coin}_1 = T) = \\frac{P(\\text{Coin}_2 = H, \\text{Coin}_1 = T)}{P(\\text{Coin}_1 = T)} = \\frac{0.25}{0.5} = 0.5 $$"
]
},
{
"cell_type": "code",
"execution_count": 219,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
"
\n",
"
\n",
"
C
\n",
"
D
\n",
"
P(C, D)
\n",
"
P(C)
\n",
"
P(D)
\n",
"
P(C) P(D)
\n",
"
\n",
" \n",
" \n",
"
\n",
"
0
\n",
"
H
\n",
"
1
\n",
"
0.24
\n",
"
0.51
\n",
"
0.43
\n",
"
0.2193
\n",
"
\n",
"
\n",
"
1
\n",
"
T
\n",
"
1
\n",
"
0.19
\n",
"
0.49
\n",
"
0.43
\n",
"
0.2107
\n",
"
\n",
"
\n",
"
2
\n",
"
H
\n",
"
2
\n",
"
0.13
\n",
"
0.51
\n",
"
0.22
\n",
"
0.1122
\n",
"
\n",
"
\n",
"
3
\n",
"
T
\n",
"
2
\n",
"
0.09
\n",
"
0.49
\n",
"
0.22
\n",
"
0.1078
\n",
"
\n",
"
\n",
"
4
\n",
"
H
\n",
"
3
\n",
"
0.09
\n",
"
0.51
\n",
"
0.22
\n",
"
0.1122
\n",
"
\n",
"
\n",
"
5
\n",
"
T
\n",
"
3
\n",
"
0.13
\n",
"
0.49
\n",
"
0.22
\n",
"
0.1078
\n",
"
\n",
"
\n",
"
6
\n",
"
H
\n",
"
4
\n",
"
0.01
\n",
"
0.51
\n",
"
0.05
\n",
"
0.0255
\n",
"
\n",
"
\n",
"
7
\n",
"
T
\n",
"
4
\n",
"
0.04
\n",
"
0.49
\n",
"
0.05
\n",
"
0.0245
\n",
"
\n",
"
\n",
"
8
\n",
"
H
\n",
"
5
\n",
"
0.03
\n",
"
0.51
\n",
"
0.03
\n",
"
0.0153
\n",
"
\n",
"
\n",
"
9
\n",
"
T
\n",
"
5
\n",
"
0.00
\n",
"
0.49
\n",
"
0.03
\n",
"
0.0147
\n",
"
\n",
"
\n",
"
10
\n",
"
H
\n",
"
6
\n",
"
0.01
\n",
"
0.51
\n",
"
0.05
\n",
"
0.0255
\n",
"
\n",
"
\n",
"
11
\n",
"
T
\n",
"
6
\n",
"
0.04
\n",
"
0.49
\n",
"
0.05
\n",
"
0.0245
\n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" C D P(C, D) P(C) P(D) P(C) P(D)\n",
"0 H 1 0.24 0.51 0.43 0.2193\n",
"1 T 1 0.19 0.49 0.43 0.2107\n",
"2 H 2 0.13 0.51 0.22 0.1122\n",
"3 T 2 0.09 0.49 0.22 0.1078\n",
"4 H 3 0.09 0.51 0.22 0.1122\n",
"5 T 3 0.13 0.49 0.22 0.1078\n",
"6 H 4 0.01 0.51 0.05 0.0255\n",
"7 T 4 0.04 0.49 0.05 0.0245\n",
"8 H 5 0.03 0.51 0.03 0.0153\n",
"9 T 5 0.00 0.49 0.03 0.0147\n",
"10 H 6 0.01 0.51 0.05 0.0255\n",
"11 T 6 0.04 0.49 0.05 0.0245"
]
},
"execution_count": 219,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"merged"
]
},
{
"cell_type": "code",
"execution_count": 308,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"0 0.470588\n",
"1 0.387755\n",
"2 0.254902\n",
"3 0.183673\n",
"4 0.176471\n",
"5 0.265306\n",
"6 0.019608\n",
"7 0.081633\n",
"8 0.058824\n",
"9 0.000000\n",
"10 0.019608\n",
"11 0.081633\n",
"Name: P(D | C), dtype: float64"
]
},
"execution_count": 308,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"merged['P(D | C)'] = merged['P(C, D)'] / merged['P(C)']\n",
"merged['P(D | C)']"
]
},
{
"cell_type": "code",
"execution_count": 312,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAFwCAYAAACraUwoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAbS0lEQVR4nO3de/RndV3v8eerAQS5KTBeuCkmWhhGMlFmJZWdBrUDLtJAA62IqOjk0hByGSKalygPucLmcJC4JWQoLk5NYonX1GIwZAS1iEaYBmRQUFAIBt7nj71Hvnz5XT6/3/z2/C7zfKz1W/Pdl+9nv/f39tr7sy+TqkKSpOl833wXIElaHAwMSVITA0OS1MTAkCQ1MTAkSU0MDElSEwNjCyV5TZLPjAzfm+QZU8x/Q5LDB67p6UkqyXZDLqdf1uFJ1s/yuVPWmeSNSc6baN4kf5/k1bOvfMq6ZvSezrDtSddpDtrev6912Vy0N4PlPjnJp5Lck+RPt9IyB/8e6bGWbGAkeWWSNf0X6Lb+B+Ynh15uVe1SVTf3NVyQ5G1j059TVZ8Yuo6loKreXlUnTDLtiKq6EB77Az9AHd97TyfTGpxTrdNMJVmX5EUjbd/S1/rQXLQ/AycCdwK7VdXrxyf234MH+kC5J8mXkrwjye6zXeB8fY+SPCvJ3yS5M8m3klyf5HVbO6Tny5IMjCSvA84G3g48GdgfeC9w5DyWtShtjb2UbcUSfi2fBtxYU18F/MdVtSuwHPhV4MeBf0qy89YocC4k+X7gn4FbgYOranfg5cAKYNf5rG2rqaol9QfsDtwLvHyKeR5HFygb+r+zgcf10w4H1gOvB+4AbgN+deS5ewJXAt8G/gV4K/CZkekFPJNuq+tB4IG+nv/XT18HvGgO6ngJ8K99HbcCZ4xMe3pfx3aTrP864A+AG4G7gL8Edhxb7qnA7cDFjXW+kW4rcx3wqhnWeWLf7m3A60emnwFcMtE6AZ8ATgB+ELgfeKh/ne8GfhT4+uj6A0cD103yejS9p/3jF/ev2z3AfwG/D+wM3Ac83NdwL7B3X//lwCV92ydMsk6Trf8FwNtGhg8H1vePL+6Xd1+/vDdM8Brt3a/XN4GbgN8Ye20/AFzUr8sNwIopvjM/AVwDfKv/9ydGahz9nL9oguc+aj36cbv263vyJMvbCbiQ7vP55X791o99hl/Ur+N9wB4j036E7rO4fT/8a30bdwFXAU8be29PAv69n34OkElqugT4u/n+jZvPv3kvYM5XCFYCm5jkx7Kf50zg88CT6LZ4Pgu8tZ92eP/8M4Ht6X4gvgs8sZ9+Wf9F2xn4Ibofjcl+XCb6oqzjkcDYkjoOBw6m20t8Lt0P5FH9tKczfWB8CdgP2AP4p811jiz3XXRBsVNjne/u538h8B3g2TOo89L+9TwY2Djy+pzBNIHRP37N6HvQj7sROGJk+ApGfozH5p3Je3ob8FP94ycCzxtZz/Vj7Z5B92N6VL/+O02yTpOt/wVMEhjjn6VJXqNP0u1Z7wgc0rf9cyO13U/3uVoGvAP4/CSvzx50P6bHAdsBx/bDe072OR97/oTT6cLqryd5zjv7+p8I7AtcP9m6A1fz6DA8C1jVPz6KLix/sK/9TcBnx97bvwWeQNcTsRFYOUlNtzOy0bYt/i3FLqk9gTuratMU87wKOLOq7qiqjcBb6L4Mmz3YT3+wqlbTbTk9u++nPBo4vaq+U1VfotsKmq1Z1QFQVZ+oqrVV9XBVXU/3o/PCGSz7z6vq1qr6JvBHdD8Cmz0MvLmq/ruq7muoE+AP+/k/Cfwd8IoZ1PmW/vVcS7e3cyxb7kLgVwCS7AH8AvD+8Zlm8Z4+CByUZLeququqvjBNHZ+rqg/363/fJPPM+fon2Q/4SeDUqrq/qq4DzuPR79tnqmp1dcc8LgZ+eJLmXgL8e1VdXFWbqupS4CvAL25hmRvowmgirwDe3r/G64H3TNHO++lfsyQBjuGR9/o3gXdU1Zf734S3A4ckedrI899ZVXdX1S3Ax+nCdSJ70m0wbLOWYmB8A9hrmv7ivYGvjQx/rR/3vTbGAue7wC50W9fb0XWtjD53tmZbB0l+LMnHk2xM8i263eq9ZrDs8XUYXe7Gqrp/BnXeVVXfmWh6Y51T1TJblwC/mGQXuh+fT1fVRF/2mb6nR9NtlX8tySeTPH+aOm6dZvr4PHO1/nsD36yqe8ba3mdk+PaRx98FdpzkezP+/k/U1mzsQ9ddNpG9efTrMtXreDnw/CR7Az9Nt9fw6X7a04A/S3J3krv75YWpX4ddJlnON4CnTlHHkrcUA+NzdLvaR00xzwa6D9Jm+/fjprORrvtlv7HnTmaqg4BbUgd0W1BXAvtVd/BtFd0XodX4Oowud7zu6ep84tjBy9HpLXVOVUuLx7zOVfVfdJ+Fl9FtVV88yXNn9J5W1TVVdSRd99yH6bqyJqxhmvGjJlv/7wCPH5n2lBm0vQHYI8nowdj96brbZmr8/d+StgDog/xFPPLDPu42uq6ozfabZD6q6m7go3QbBq8ELq2qza/NrcBvVtUTRv52qqrPzqLsf6TbYNhmLbnAqKpvAacD5yQ5Ksnjk2yf5Igkf9zPdinwpiTLk+zVz39JQ9sPAR8CzujbPQh49RRP+Tow1fn7s6qjtyvdFuT9SQ6j+6LMxO8k2bfvrnkj8NdbWOdbkuyQ5KeAlwJ/M4M6/7B/PZ9DdwbNVLVM5OvAvkl2GBt/Ed3B0oPpjmE8xkze0379XpVk96p6kO5A9uZTWL8O7DnLU0UnW//rgBcn2SPJU4DXjj1v0s9XVd1Kd6zpHUl2TPJc4NeBv5pFfauBZ/Wnqm+X5JeBg+j6/mckyeOSHEoXtptPuJjIB4A/SPLEJPsAJ0/T9PuB4+l+0Ee7Hlf17TynX/7uSV4+07p7bwZ+IslZ/ftBkmcmuSTJE2bZ5qKy5AIDoKreDbyO7gDXRrqtjJPpPqQAbwPW0B1IWwt8oR/X4mS6Xdbb6Q7mTfaBB3gfXX/33Uk+PMH0Lanjt4Ezk9xD9wP+gWnmH/d+uq2ym/u/qZY7XZ230335N9D9IJ1UVV+ZQZ2fpDsw+THgT6rqozNcl6vpzvK5PcmdI+OvoNsyvmKsy2zcTN7T44B1Sb5N1732KwD9+l4K3Ny/3zPpVpps/S8Gvkh3gPejPDZI30EX5Hcn+f0J2j2W7kD4BrrX4s1V9Q8zqAuAqvoG3UbA6+m6Zd4AvLSq7pzyiY/2hv4z8E26IL+W7kyryd6XM+nOvvtPui37y4H/nqL9K4EDga9X1RdHar+C7gSOy/r37EvAETOo+3uq6j+A59O9pjf0XawfpPtu3DPFU5eMPLLnpm1FknV0Zxj943zXMrQk/0HXJbHk13UpS/JbwDFVNZMTOzTHluQehgSQ5Gi6fv6r57sWzUySpyZ5QZLvS/Jsur2bCbsVtfUs1StPtY1L8gm6fvbjqurheS5HM7cD8H+AA+guxryM7poSzSO7pCRJTeySkiQ1WZRdUitXrqyPfOQj812GJM3ETK6TWpAW5R7GnXfO5Gw+SdJcWJSBIUna+gwMSVITA0OS1MTAkCQ1MTAkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTRbl7c0ncugpF81pe9eedfyctidJi517GJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaDBoYSVYm+WqSm5KcNsV8P5rkoSS/NGQ9kqTZGywwkiwDzgGOAA4Cjk1y0CTzvQu4aqhaJElbbsg9jMOAm6rq5qp6ALgMOHKC+X4X+CBwx4C1SJK20JCBsQ9w68jw+n7c9yTZB3gZsGq6xpKcmGRNkjUbN26c00IlSdMbMjAywbgaGz4bOLWqHpqusao6t6pWVNWK5cuXz0V9kqQZ2G7AttcD+40M7wtsGJtnBXBZEoC9gBcn2VRVHx6wLknSLAwZGNcAByY5APgv4BjglaMzVNUBmx8nuQD4W8NCkhamwQKjqjYlOZnu7KdlwPlVdUOSk/rp0x63kCQtHEPuYVBVq4HVY+MmDIqqes2QtUiStoxXekuSmhgYkqQmg3ZJLWa3nHnwnLa3/+lr57Q9Sdra3MOQJDUxMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTQYNjCQrk3w1yU1JTptg+pFJrk9yXZI1SX5yyHokSbO33VANJ1kGnAP8PLAeuCbJlVV148hsHwOurKpK8lzgA8APDFWTJGn2htzDOAy4qapurqoHgMuAI0dnqKp7q6r6wZ2BQpK0IA0ZGPsAt44Mr+/HPUqSlyX5CvB3wK9N1liSE/tuqzUbN26c82IlSVMbMjAywbjH7EFU1RVV9QPAUcBbJ2usqs6tqhVVtWL58uVzV6UkqcmQgbEe2G9keF9gw2QzV9WngO9PsteANUmSZmnIwLgGODDJAUl2AI4BrhydIckzk6R//DxgB+AbA9YkSZqlwc6SqqpNSU4GrgKWAedX1Q1JTuqnrwKOBo5P8iBwH/DLIwfBJUkLyGCBAVBVq4HVY+NWjTx+F/CuIWuQJM0Nr/SWJDUxMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU0MDElSEwNDktTEwJAkNTEwJElNmm8NkuSJwN5093xaV1UPD1aVJGnBmTIwkuwO/A5wLN2dZDcCOwJPTvJ54L1V9fHBq5Qkzbvp9jAuBy4Cfqqq7h6dkORQ4Lgkz6iq9w1UnyRpgZgyMKrq56eYdi1w7ZxXJElakKY86J3kF5L80gTjX5Vk0jCRJC09050l9RbgkxOM/xhw5tyXI0lqleQpSS5L8h9JbkyyOsmzhlredMcwHl9VG8dHVtXtSXYeqCbpMW458+A5b3P/09fOeZvS1tL/99ZXABdW1TH9uEOAJwP/NsQypwuMHZNsV1WbxgrdHthpiIIkSU1+Bnhw7H8xvW7IBU7XJfUh4P+O7k30j1f10yRJ8+OH2MonHk0XGG8Cvg58Lcm1Sa4F1tFdj/GmgWuTJC0g051Wuwk4LclbgGf2o2+qqvsGr0ySNJUbgMecxTqkpntJVdV9VbW2/zMsJGn+XQ08LslvbB6R5EeTvHCoBTbfS0pbx6GnXDSn7V171vFz2p6khaGqKsnLgLOTnAbcT3fI4LVDLdPAkKRFqqo2AK/YWsvz9uaSpCbT3a32P4GaapZ++tlV9Z65LEyStLBMd5bUAVurEEnSwmaXlCSpiYEhSWpiYEiSmhgYkrQNSXLv2PBrkvx5y3OnvQ4jyauB3wOe3Y/6MvCeqprbK8wkaRtz6CkXTXUW6oxde9bxmcv2xk13Wu3xdFcNvg74At1ptM8DzkqCoSFJ247p9jB+G3hZVa0bGXd1kqOBywADQ5IWl52SXDcyvAdwZcsTpwuM3cbCAoCqWpdkt+byJEkLxX1VdcjmgSSvAVa0PHG6g95T3ZnWu9ZK0jZkuj2MH0xy/QTjAzxjgHokSQvUtIGxVaqQJC140wXGLVU15WlfSTLdPJKkxxr6NNiJVNUuY8MXABe0PHe6YxgfT/K7SfYfHZlkhyQ/m+RC4NUzqFWStEhNt4exEvg14NIkzwDuAnYElgEfBf53VV03aIWSpAVhutub3w+8F3hvku2BvehOybp7K9QmSVpAprvSe0fgJOCZwPXA+VW1aWsUJklaWKY7hnEh3QUda4EXA386eEWSpAVpumMYB1XVwQBJ3gf8y/AlSZIWoukC48HND6pqU7LVzwCTJM2RJHsCH+sHnwI8BGzshw+rqgemev50gfHDSb69eVl0N636dv+4qsr7SUnSLN1y5sFzeg3b/qevnXKrvqq+ARwCkOQM4N6q+pPW9qc7S2pZa0OSpKXN/3FPktTEwJAkNTEwJElNDAxJUhMDQ5LUZLrTardIkpXAn9HdrPC8qnrn2PRXAaf2g/cCv1VVXxyyJklaKKY7DXZIVXXGTJ8z2B5GkmXAOcARwEHAsUkOGpvtP4EXVtVzgbcC5w5VjyRpywzZJXUYcFNV3dxfPXgZcOToDFX12aq6qx/8PLDvgPVIkrbAkIGxD3DryPD6ftxkfh34+8kmJjkxyZokazZu3DjZbJKkgQwZGBP1zU14GXySn6ELjFMnmg5QVedW1YqqWrF8+fI5KlGS1GrIg97rgf1GhvcFNozPlOS5wHnAEf19TiRJC9CQexjXAAcmOSDJDsAxwJWjM/T/V/iHgOOq6t8GrEWStIUG28Pob4d+MnAV3Wm151fVDUlO6qevAk4H9qT7L2ABNlXViqFqkiTN3qDXYVTVamD12LhVI49PAE4YsgZJ0tzwSm9JUhMDQ5LUxMCQJDUxMCRJTQwMSVKTQc+S0vy75cyD57S9/U9fO6ftSVo83MOQJDUxMCRJTQwMSVITj2FoEIeectGctnfFrnPanKRZcA9DktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTQYNjCQrk3w1yU1JTptg+g8k+VyS/07y+0PWIknaMtsN1XCSZcA5wM8D64FrklxZVTeOzPZN4H8BRw1VhyRpbgy5h3EYcFNV3VxVDwCXAUeOzlBVd1TVNcCDA9YhSZoDQwbGPsCtI8Pr+3GSpEVoyMDIBONq1o0lJyZZk2TNxo0bt6AsSdJsDBkY64H9Rob3BTbMtrGqOreqVlTViuXLl29xcZKkmRkyMK4BDkxyQJIdgGOAKwdcniRpQIOdJVVVm5KcDFwFLAPOr6obkpzUT1+V5CnAGmA34OEkrwUOqqpvD1WXJGl2BgsMgKpaDaweG7dq5PHtdF1VkqQFziu9JUlNDAxJUhMDQ5LUxMCQJDUxMCRJTQwMSVKTQU+rlfRYt5x58Jy2t//pa+e0vW2N70c79zAkSU0MDElSEwNDktTEwJAkNTEwJElNDAxJUhMDQ5LUxMCQJDUxMCRJTQwMSVITA0OS1MTAkCQ1MTAkSU28W620jTj0lIvmtL1rzzp+TtvTwucehiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmhgYkqQmBoYkqYmBIUlqYmBIkpoYGJKkJgaGJKmJgSFJamJgSJKaGBiSpCbbzXcB0kJ36CkXzWl7V+w6p81tc3w/5o97GJKkJgaGJKmJgSFJamJgSJKaGBiSpCYGhiSpiYEhSWpiYEiSmgwaGElWJvlqkpuSnDbB9CR5Tz/9+iTPG7IeSdLsDRYYSZYB5wBHAAcBxyY5aGy2I4AD+78Tgb8Yqh5J0pYZcg/jMOCmqrq5qh4ALgOOHJvnSOCi6nweeEKSpw5YkyRpllJVwzSc/BKwsqpO6IePA36sqk4emedvgXdW1Wf64Y8Bp1bVmgnaO5FuLwTg2cBXByn8EXsBdw68jK3B9Vh4lsq6uB4zc2dVrdwKyxnMkDcfzATjxtOpZZ5uZNW5wLlbWlSrJGuqasXWWt5QXI+FZ6msi+ux7RmyS2o9sN/I8L7AhlnMI0laAIYMjGuAA5MckGQH4BjgyrF5rgSO78+W+nHgW1V124A1SZJmabAuqaralORk4CpgGXB+Vd2Q5KR++ipgNfBi4Cbgu8CvDlXPLGy17q+BuR4Lz1JZF9djGzPYQW9J0tLild6SpCYGhiSpiYExJsn5Se5I8qX5rmVLJNkvyceTfDnJDUl+b75rmo0kOyb5lyRf7NfjLfNd05ZIsizJv/bXIC1aSdYlWZvkuiSPuW5qsUjyhCSXJ/lK/115/nzXtJB5DGNMkp8G7qW7Av2H5rue2eqvmH9qVX0hya7AtcBRVXXjPJc2I0kC7FxV9ybZHvgM8Hv9nQEWnSSvA1YAu1XVS+e7ntlKsg5YUVWL+sK9JBcCn66q8/qzOR9fVXfPc1kLlnsYY6rqU8A357uOLVVVt1XVF/rH9wBfBvaZ36pmrr9tzL394Pb936LcykmyL/AS4Lz5rkWQZDfgp4H3AVTVA4bF1AyMbUCSpwM/AvzzPJcyK303znXAHcA/VNWiXA/gbOANwMPzXMdcKOCjSa7tb9uzGD0D2Aj8Zd9NeF6Snee7qIXMwFjikuwCfBB4bVV9e77rmY2qeqiqDqG7E8BhSRZdV2GSlwJ3VNW1813LHHlBVT2P7o7Tv9N35S422wHPA/6iqn4E+A7wmP+GQY8wMJawvs//g8BfVdWH5rueLdV3F3wCWIw3cHsB8D/7vv/LgJ9Ncsn8ljR7VbWh//cO4Aq6u1MvNuuB9SN7rJfTBYgmYWAsUf3B4vcBX66qd893PbOVZHmSJ/SPdwJeBHxlXouahar6g6rat6qeTnebnKur6lfmuaxZSbJzfyIFfRfO/wAW3VmFVXU7cGuSZ/ejfg5YVCeFbG1D3q12UUpyKXA4sFeS9cCbq+p981vVrLwAOA5Y2/f/A7yxqlbPX0mz8lTgwv4/5Po+4ANVtahPSV0Cngxc0W2TsB3w/qr6yPyWNGu/C/xVf4bUzSys2xMtOJ5WK0lqYpeUJKmJgSFJamJgSJKaGBiSpCYGhiSpiafVapuX5CFgLd19qjYBFwJnV9VSuIWHNGcMDAnu6289QpInAe8HdgfePJ9FSQuN12Fom5fk3qraZWT4GcA1wF7lF0T6Ho9hSGOq6ma678aT5rsWaSExMKSJZb4LkBYaA0Ma03dJPUT3/29I6hkY0ogky4FVwJ97/EJ6NA96a5s3wWm1FwPv9rRa6dEMDElSE7ukJElNDAxJUhMDQ5LUxMCQJDUxMCRJTQwMSVITA0OS1OT/AyuIwCqh3NYSAAAAAElFTkSuQmCC",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"merged[['C', 'D', 'P(D | C)']]\n",
"\n",
"axs = sns.catplot(data=merged, x=\"D\", y=\"P(D | C)\", hue=\"C\", kind=\"bar\");\n",
"axs.set(title=\"Conditional probability distribution of D given C\\nNote that the blue bars add up to 1\");"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that the sum of conditional probabilites, unlike joint probability, is not 1. "
]
},
{
"cell_type": "code",
"execution_count": 410,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2.0"
]
},
"execution_count": 410,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"merged[\"P(D | C)\"].sum()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"This is because \n",
"\n",
"$$ \\sum_C \\sum_D P(D|C) = \\sum_D P(D|C=\\text{Heads}) + \\sum_D P(D|C=\\text{Tails}) $$\n",
"\n",
"And $\\sum_D P(D|C=\\text{Heads})$ and $\\sum_D P(D|C=\\text{Tails})$ are individually probability distributions that each sum to 1, over different values of $D$. \n",
"\n",
"In other words, in the plot above, the blue bars add up to 1 and the orange bars add up to 1. "
]
},
{
"cell_type": "code",
"execution_count": 415,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(1.0, 1.0)"
]
},
"execution_count": 415,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"heads = merged[merged[\"C\"] == \"H\"]\n",
"tails = merged[merged[\"C\"] == \"T\"]\n",
"\n",
"heads[\"P(D | C)\"].sum(), tails[\"P(D | C)\"].sum()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Product Rule $P(A, B)$\n",
"\n",
"Rearranging the definition of conditional probability, we get the product rule:\n",
"\n",
"$$ P(A, B) = P(A | B) \\cdot P(B) $$\n",
"\n",
"Similarly, we can also write:\n",
"\n",
"$$ P(A, B) = P(B | A) \\cdot P(A)$$\n",
"\n",
"In summary, \n",
"\n",
"$$ P(A, B) = P(A | B) \\cdot P(B) = P(B | A) \\cdot P(A)$$\n",
"\n",
"## Chain Rule $P(A, B, C)$\n",
"\n",
"The chain rule is a generalization of the product rule to more than two events.\n",
"\n",
"$ P(A, B, C) = P(A | B, C) \\cdot P(B, C) $\n",
"\n",
"$P(A, B, C) = P(A | B, C) \\cdot P(B | C) \\cdot P(C)$\n",
"\n",
"since $P(B, C) = P(B | C) \\cdot P(C)$ as per the product rule.\n",
"\n",
"**Chain rule essentially allows expressing the joint probability of multiple random variables as a product of conditional probabilities.** This is useful because conditional probabilities are often easier to estimate from data than joint probabilities.\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Inclusion-Exclusion Principle $P(A \\vee B)$\n",
"\n",
"Inclusion-Exclusion Principle is a way of calculating the probability of two events occurring i.e. $ P(A=a ~\\text{OR}~ B=b) $ denoted generally as $P(A = a \\vee B = b)$.\n",
"\n",
"It is defined as:\n",
"\n",
"$$ P(A = a \\vee B = b) = P(A = a) + P(B = b) - P(A = a \\wedge B = b) $$\n",
"\n",
"\n",
"\n",
"For example, if we are rolling two dice, the Inclusion-Exclusion Principle can be used to calculate the probability of rolling a 1 on the first die or a 2 on the second die.\n",
"\n",
"$P(\\text{Coin}_1=H \\vee \\text{Coin}_2=T) $\n",
"\n",
"$ = P(\\text{Coin}_2=H) + P(\\text{Coin}_1=T) - P(\\text{Coin}_2=H ∧ \\text{Coin}_1=T)$\n",
"\n",
"$ = 0.5 + 0.5 - 0.25 $\n",
"\n",
"$ = 0.75$ \n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Bayes Theorem $P(A|B)$"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Bayes theorem is a way of calculating conditional probability. For example, if we are rolling two dice, Bayes theorem can be used to calculate the probability of rolling a 1 on the first die given that we rolled a 2 on the second die.\n",
"\n",
"$$ P(A | B) = \\frac{P(B | A) \\cdot P(A)}{P(B)} $$\n",
"\n",
"$P(A|B)$ in the context of Bayes theorem is called the **Posterior** probability. \n",
"\n",
"$P(B|A)$ is called the **Likelihood**. \n",
"\n",
"$P(A)$ is called the **Prior** probability. \n",
"\n",
"$P(B)$ is called the **Evidence**, also known as _Marginal Likelihood_."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"$$ P(\\text{Posterior}) = \\frac{P(\\text{Likelihood})\\cdot P(\\text{Prior})}{P(\\text{Evidence})}$$\n",
"\n",
"\n",
"\n",
"
"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
""
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Bayes Theorem allows a formal method of updating prior beliefs with new evidence and is the foundation of Bayesian Statistics. We will talk more about this when we talk about Statistics. "
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"In machine learning, the task is often to find $P(Y | X_1 = x_1, X_2 = x_2, \\ldots X_D = x_D)$ i.e. the probability of an unknown Y, given some values for $D$ features ($X_1, X_2 \\ldots X_D$). Bayes theorem allows us to calculate this probability from the data. \n",
"\n",
"Let's assume we are interested in predicting if a person is a football player ($Y_F=1$) or not ($Y_F=0$), given their height ($X_H$) and weight ($X_W$).\n",
"\n",
"Say, we observe a person who is 7 feet tall and weighs 200 pounds. We can use Bayes theorem to calculate the probability of this person being a football player using the following equation:\n",
"\n",
"$P(Y | X_H = 7, X_W = 200) = \\frac{P(X_H = 7, X_W = 200 | Y_F) \\cdot P(Y_F)}{P(X_H = 7, X_W = 200)}$"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that here $P(X_H = 7, X_W = 200 | Y_F)$ is the **Likelihood** probability of observing someone who is 7 feet tall and weighs 200 pounds, knowing if they are a football player. \n",
"\n",
"$P(Y_F)$ is the **Prior** probability of a person being a football player out of the entire population. \n",
"\n",
"$P(X_H = 7, X_W = 200)$ is the probability of the **Evidence** i.e. probability of observing _anyone_ who is 7 feet tall and weighs 200 pounds in the entire population."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "base",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.12"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}
![](\"../assets/conditionalprob.png\")
\n",
"\n",
"![](\"../assets/inclusion_exclusion.png\")
\n",
"\n",
"For example, if we are rolling two dice, the Inclusion-Exclusion Principle can be used to calculate the probability of rolling a 1 on the first die or a 2 on the second die.\n",
"\n",
"$P(\\text{Coin}_1=H \\vee \\text{Coin}_2=T) $\n",
"\n",
"$ = P(\\text{Coin}_2=H) + P(\\text{Coin}_1=T) - P(\\text{Coin}_2=H ∧ \\text{Coin}_1=T)$\n",
"\n",
"$ = 0.5 + 0.5 - 0.25 $\n",
"\n",
"$ = 0.75$ \n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Bayes Theorem $P(A|B)$"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Bayes theorem is a way of calculating conditional probability. For example, if we are rolling two dice, Bayes theorem can be used to calculate the probability of rolling a 1 on the first die given that we rolled a 2 on the second die.\n",
"\n",
"$$ P(A | B) = \\frac{P(B | A) \\cdot P(A)}{P(B)} $$\n",
"\n",
"$P(A|B)$ in the context of Bayes theorem is called the **Posterior** probability. \n",
"\n",
"$P(B|A)$ is called the **Likelihood**. \n",
"\n",
"$P(A)$ is called the **Prior** probability. \n",
"\n",
"$P(B)$ is called the **Evidence**, also known as _Marginal Likelihood_."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"$$ P(\\text{Posterior}) = \\frac{P(\\text{Likelihood})\\cdot P(\\text{Prior})}{P(\\text{Evidence})}$$\n",
"\n",
"\n",
"\n",
"![](\"https://www.analyticsvidhya.com/wp-content/uploads/2016/06/12.jpg\")