477 lines
15 KiB
Plaintext
477 lines
15 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "c88902ee",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Úkol: BI-PST\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "91241ee8",
|
|
"metadata": {},
|
|
"source": [
|
|
"Spolupracovali:\n",
|
|
" * Ondřej Hladůvka (reprezentant)\n",
|
|
" * Tomáš Kaňka"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "334be38a",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"#import csv\n",
|
|
"#import math\n",
|
|
"import numpy as np\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"import pandas as pd\n",
|
|
"#np.set_printoptions(precision=3)\n",
|
|
"#from sympy import *\n",
|
|
"from scipy import stats \n",
|
|
"from scipy.stats import norm, uniform, expon\n",
|
|
"#from scipy.optimize import minimize"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "7c90184c-5f76-4277-b0ad-aeec2ac37d30",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"K = 28\n",
|
|
"L = 8\n",
|
|
"M = (((K + L) * 47) % 11) + 1\n",
|
|
"print(\"M =\",M)\n",
|
|
"print(\"dataset ex0221\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "129f8693",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Úloha č. 1"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "503f77b6-1c9d-4406-8b30-ec3b792267e7",
|
|
"metadata": {},
|
|
"source": [
|
|
"(1b) Načtěte datový soubor a rozdělte sledovanou proměnnou na příslušné dvě pozorované skupiny.\n",
|
|
"Stručně popište data a zkoumaný problém. Pro každou skupinu zvlášť odhadněte střední hodnotu, rozptyl a medián příslušného rozdělení."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "b80d5cec-db0c-42e7-a3b9-296803242269",
|
|
"metadata": {
|
|
"scrolled": true
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# načtení dat\n",
|
|
"df = pd.read_csv(\"/data.csv\")\n",
|
|
"df = df.drop(df.columns[0], axis=1)\n",
|
|
"survived = df[df['Status'] == 'survived']['Weight'].values\n",
|
|
"perished = df[df['Status'] == 'perished']['Weight'].values\n",
|
|
"# informace\n",
|
|
"print(\"Prvních 5 řádků:\")\n",
|
|
"print(df.head())\n",
|
|
"print(\"Info\")\n",
|
|
"print(\"Počet řádků:\", df.shape[0])\n",
|
|
"print(\"Datové typy:\")\n",
|
|
"display(df.info())\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "41e43831",
|
|
"metadata": {},
|
|
"source": [
|
|
"Tento datový soubor zkoumá, zda hmotnost dospělých samců vrabců hraje roli v jejich přežití během extrémních klimatických podmínek. Cílem je zjistit, zda vrabci, kteří přežili, měli významně jinou průměrnou hmotnost ve srovnání se vrabci, kteří zahynuli. \n",
|
|
"\n",
|
|
"Tabulak má dva sloupce weight a status, kde\n",
|
|
"**weight** udává hmotnost dospělých samců vrabců v gramech a\n",
|
|
"**status** udává, zda vrabec přežil nebo zahynul během zimní bouře. Může mít hodnoty \"survived\" (přežil) nebo \"perished\" (zahynul).\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "717d3775",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Zobrazení názvů sloupců\n",
|
|
"print(df.columns)\n",
|
|
"\n",
|
|
"groups = df.groupby('Status')\n",
|
|
"summary_stats = groups['Weight'].agg(['mean', 'var', 'median']).reset_index()\n",
|
|
"\n",
|
|
"# Zobrazení výsledků\n",
|
|
"print(summary_stats)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "b370bf22",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Úloha č. 2"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "29eef015-c103-4e7f-89e8-2b59526f4b1e",
|
|
"metadata": {},
|
|
"source": [
|
|
"(1b) Pro každou skupinu zvlášť odhadněte hustotu a distribuční funkci pomocí histogramu a empirické distribuční funkce."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "9c8f12d3",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Funkce pro výpočet hustoty a CDF\n",
|
|
"def calculate_density_cdf(group):\n",
|
|
" sorted_weights = group.sort_values('Weight')\n",
|
|
" \n",
|
|
" # Výpočet CDF\n",
|
|
" cdf = pd.Series(range(1, len(sorted_weights) + 1)) / len(sorted_weights)\n",
|
|
" \n",
|
|
" # Přidání CDF k seřazeným váhám\n",
|
|
" results = pd.DataFrame({'Weight': sorted_weights['Weight'].values, 'CDF': cdf.values})\n",
|
|
" \n",
|
|
" return results\n",
|
|
"\n",
|
|
"# Vytvoření hustoty a CDF pro každou skupinu\n",
|
|
"groups = df.groupby('Status')\n",
|
|
"density_cdf_results = {}\n",
|
|
"\n",
|
|
"for name, group in groups:\n",
|
|
" density_cdf_results[name] = calculate_density_cdf(group)\n",
|
|
"\n",
|
|
"# Zobrazení výsledků\n",
|
|
"for name, results in density_cdf_results.items():\n",
|
|
" plt.figure(figsize=(10, 6))\n",
|
|
" plt.hist(df[df['Status'] == name]['Weight'], bins=20, density=True, alpha=0.5, label=f'Hustota {name}')\n",
|
|
" \n",
|
|
" # CDF\n",
|
|
" plt.plot(results['Weight'], results['CDF'], label=f'CDF {name}', color='red')\n",
|
|
" \n",
|
|
" # Nastavení g\n",
|
|
" plt.title(f'Hustota a CDF pro skupinu: {name}')\n",
|
|
" plt.xlabel('Hmotnost')\n",
|
|
" plt.ylabel('Hustota / Kumulativní pravděpodobnost (CDF)')\n",
|
|
" plt.legend()\n",
|
|
" plt.grid()\n",
|
|
" plt.show()\n",
|
|
" print(f\"\\n--- Výsledky pro skupinu: {name} ---\")\n",
|
|
" print(results)\n",
|
|
"\n",
|
|
"\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "f98f6ad6",
|
|
"metadata": {},
|
|
"source": [
|
|
"Vytvořili jsme funkci calculate_density_cdf, která seřadila data podle hmotnosti a spočítala kumulativní distribuční funkci. Tato funkce vrátila DataFrame se seřazenými hmotnostmi a odpovídajícími hodnotami CDF."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "9f9b7946",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Úloha č. 3"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "626d8bce-65af-4659-96e9-c1fbe4ccb3cc",
|
|
"metadata": {},
|
|
"source": [
|
|
"(3b) Pro každou skupinu zvlášť najděte nejbližší rozdělení: \n",
|
|
"Odhadněte parametry normálního, exponenciálního a rovnoměrného rozdělení.\n",
|
|
"Zaneste příslušné hustoty s odhadnutými parametry do grafů histogramu. Diskutujte, které z rozdělení odpovídá pozorovaným datům nejlépe."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "542a742d-2c96-4d06-bf05-49823752ed6d",
|
|
"metadata": {},
|
|
"source": [
|
|
"Odhady získáme pomocí momentové metody"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "b1995dc6-a79c-4859-a720-5dca78d63e44",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def show_distribution(data):\n",
|
|
" fig = plt.figure(figsize = (10, 14))\n",
|
|
" gs = fig.add_gridspec(4,2, figure = fig)\n",
|
|
" sp1 = fig.add_subplot(gs[0, 0])\n",
|
|
" sp2 = fig.add_subplot(gs[0, 1])\n",
|
|
" sp3 = fig.add_subplot(gs[1, 0])\n",
|
|
" sp4 = fig.add_subplot(gs[1, 1])\n",
|
|
" sp5 = fig.add_subplot(gs[ 2:, 0:])\n",
|
|
" \n",
|
|
" sp1.set_title('Data Histogram')\n",
|
|
" sp1.hist(data, bins=10, density=True, alpha=0.5)\n",
|
|
" sp2.hist(data, bins=10, density=True, alpha=0.5)\n",
|
|
" sp3.hist(data, bins=10, density=True, alpha=0.5)\n",
|
|
" sp4.hist(data, bins=10, density=True, alpha=0.5)\n",
|
|
" sp5.hist(data, bins=10, density=True, alpha=0.5)\n",
|
|
" \n",
|
|
" mu, std = norm.fit(data)\n",
|
|
" xmin, xmax = sp1.get_xlim()\n",
|
|
" x = np.linspace(xmin, xmax, 100)\n",
|
|
" p_norm = norm.pdf(x, mu, std)\n",
|
|
" sp2.plot(x, p_norm, 'red')\n",
|
|
" sp2.fill_between(x, p_norm, alpha=0.2, color='red')\n",
|
|
" sp2.set_title(f'Normal fit ($\\mu={mu:.2f}, \\sigma={std:.2f}$)')\n",
|
|
"\n",
|
|
" loc, scale = expon.fit(data)\n",
|
|
" p_exp = expon.pdf(x, loc, scale)\n",
|
|
" split0 = next((i for i, x in enumerate(p_exp) if x != 0), len(p_exp))\n",
|
|
" sp3.plot(x[split0:], p_exp[split0:], 'green')\n",
|
|
" sp3.fill_between(x[split0:], p_exp[split0:], alpha=0.2, color='green')\n",
|
|
" sp3.set_title(f'Exponential Distribution Fit (scale={scale:.2f})')\n",
|
|
"\n",
|
|
" loc_uni, scale_uni = uniform.fit(data)\n",
|
|
" p_uni = uniform.pdf(x, loc_uni, scale_uni)\n",
|
|
" split1 = next((i for i, x in enumerate(p_uni) if x != 0), len(p_uni))\n",
|
|
" split2 = -next((i for i, x in enumerate(reversed(p_uni)) if x != 0), len(p_uni))\n",
|
|
" sp4.plot(x[split1:split2], p_uni[split1:split2], 'purple')\n",
|
|
" sp4.plot(x[:split1], p_uni[:split1], 'purple')\n",
|
|
" sp4.plot(x[split2:], p_uni[split2:], 'purple')\n",
|
|
" sp4.fill_between(x[split1:split2], p_uni[split1:split2], alpha=0.1, color='purple')\n",
|
|
" sp4.set_title(f'Uniform Distribution Fit (range=({loc_uni:.2f}, {loc_uni+scale_uni:.2f}))')\n",
|
|
" \n",
|
|
" sp5.plot(x, p_norm, 'red', label=f'Normal fit: $\\mu={mu:.2f}, \\sigma={std:.2f}$')\n",
|
|
" sp5.plot(x[split0:], p_exp[split0:], 'green', label=f'Exponential fit: scale={scale:.2f}')\n",
|
|
" sp5.plot(x[split1:split2], p_uni[split1:split2], 'purple')\n",
|
|
" sp5.plot(x[:split1], p_uni[:split1], 'purple')\n",
|
|
" sp5.plot(x[split2:], p_uni[split2:], 'purple', label=f'Uniform fit: range=({loc_uni:.2f}, {loc_uni+scale_uni:.2f})')\n",
|
|
" #sp5.plot(x, p_uni, 'blue', label=f'Uniform fit: range=({loc_uni:.2f}, {loc_uni+scale_uni:.2f})')\n",
|
|
" sp5.set_title('Combined Distributions')\n",
|
|
" sp5.legend()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "2121c37e-edac-446d-8ed5-1f1fab942637",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"show_distribution(survived)\n",
|
|
"show_distribution(perished)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "424b6dc3",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Úloha č. 4"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "6226456c-fdf3-4537-830c-05f8ee7022c5",
|
|
"metadata": {},
|
|
"source": [
|
|
"(1b) Pro každou skupinu zvlášť vygenerujte náhodný výběr o 100 hodnotách z rozdělení, \n",
|
|
"které jste zvolili jako nejbližší, s parametry odhadnutými v předchozím bodě.\n",
|
|
"Porovnejte histogram simulovaných hodnot s pozorovanými daty."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "85df4788-b10c-48cd-aa08-10398d9c124a",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def showRandomData(mean, std, dataset, title):\n",
|
|
" data = np.random.normal(mean, std, 100)\n",
|
|
" plt.figure(figsize=(10, 6))\n",
|
|
" x = np.linspace(data.min() - 5, data.max() + 5, 100)\n",
|
|
" p_norm = norm.pdf(x, mean, std)\n",
|
|
" plt.plot(x, p_norm, 'black', linewidth = 2)\n",
|
|
" plt.hist(data, bins = 10, color = \"blue\", alpha=0.5 , density = True)\n",
|
|
" #plt.set_title(\"Náhodné hodnoty z normálního rozdělení\")\n",
|
|
" plt.hist(dataset, bins = 10, color = \"green\", alpha=0.5, density = True)\n",
|
|
" plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "c4466cb2-863a-4c61-8338-0f6a25014c6c",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"mean_survived, std_survived = norm.fit(survived)\n",
|
|
"showRandomData(mean_survived, std_survived, survived, \"survived\")\n",
|
|
"mean_perished, std_perished = norm.fit(perished)\n",
|
|
"showRandomData(mean_perished, std_perished, survived, \"perished\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "d43bff73",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Úloha č. 5"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1c5f7d31-ca21-42b4-9a23-1111bbf599b9",
|
|
"metadata": {},
|
|
"source": [
|
|
"(1b) Pro každou skupinu zvlášť spočítejte oboustranný 95% konfidenční interval pro střední hodnotu."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "dec7a5cc-4c6b-4213-b3b8-8b58813f73cb",
|
|
"metadata": {},
|
|
"source": [
|
|
"Dle Studentova rozdělení:\n",
|
|
"$$\n",
|
|
"\\begin{align}\n",
|
|
"\\left( \\overset{\\_}{X_{n}} - t_{\\frac{\\alpha}{2},n-1}\\frac{s}{\\sqrt{n}}, \\overset{\\_}{X_{n}} + t_{\\frac{\\alpha}{2},n-1}\\frac{s}{\\sqrt{n}} \\right)\n",
|
|
"\\end{align}\n",
|
|
"$$"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "69ad2d59-bdc0-4cc7-b51c-7453f2126157",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def conf_interval(data, name):\n",
|
|
" conf = stats.t.interval(confidence = 0.95, df = len(data) - 1, loc = np.mean(data), scale = stats.sem(data))\n",
|
|
" print(f\"Oboustranný konfidenční interval 95% střední hodnoty skupiny \\\"{name}\\\":\", conf)\n",
|
|
" return conf"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "16ee6a87-8b6a-472c-9689-9d3877d31084",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"def plot_conf_interval(title, list, interval):\n",
|
|
" fig, ax = plt.subplots(figsize = (6, 4))\n",
|
|
" ax.hist(list, bins = 10, color = \"blue\", alpha=0.5)\n",
|
|
" ax.set_title(f\"Oboustranný 95% interval skupiny \\\"{title}\\\"\")\n",
|
|
" ax.axvline(x = interval[0], color = \"red\")\n",
|
|
" ax.axvline(x = interval[1], color = \"red\")\n",
|
|
" plt.show()"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "b5fd12a4-8517-4c5e-9f18-eaf627c56afa",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"conf_survived = conf_interval(survived, \"survived\")\n",
|
|
"conf_perished = conf_interval(perished, \"perished\")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"id": "f17915e9-aef8-430d-8096-c13f078e25eb",
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"plot_conf_interval(\"survived\", survived, conf_survived)\n",
|
|
"plot_conf_interval(\"perished\", perished, conf_perished)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "1c7cf77b",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Úloha č. 6"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "53a61e4f-fc67-4237-ab38-f5f9fb7767c5",
|
|
"metadata": {},
|
|
"source": [
|
|
"(1b) Pro každou skupinu zvlášť otestujte na hladině významnosti 5 % hypotézu,\n",
|
|
"zda je střední hodnota rovná hodnotě K (parametr úlohy), proti oboustranné alternativě.\n",
|
|
"Můžete použít buď výsledek z předešlého bodu, nebo výstup z příslušné vestavěné funkce vašeho softwaru."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "617cf82f",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Úloha č. 7"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"id": "7007c195-97a3-4cd4-8427-dcc8417eedf8",
|
|
"metadata": {},
|
|
"source": [
|
|
"(2b) Na hladině významnosti 5 % otestujte, jestli mají pozorované skupiny stejnou střední hodnotu.\n",
|
|
"Typ testu a alternativy stanovte tak, aby vaše volba nejlépe korespondovala s povahou zkoumaného problému."
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3 (ipykernel)",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.10.15"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|