{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# How to Handle Missing Data in Machine Learning: UCI Adult Hands-On Notebook\n",
        "\n",
        "This notebook accompanies the article [How to Handle Missing Data in Machine Learning: A Hands-On Example](https://viscioletti.com/posts/2026/07/24/hands-on-missing-data-machine-learning-uci-adult/).\n",
        "\n",
        "We use the UCI Adult Census Income dataset to compare practical missing-data strategies in a supervised machine learning workflow.\n",
        "\n",
        "The companion CSV is bundled as `uci_adult.csv`. Keep it in the same folder as this notebook when running locally; Notebook Studio mounts it automatically for the example URL."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## What We Will Compare\n",
        "\n",
        "- Complete case analysis: drop rows with missing values.\n",
        "- Mode imputation: fill categorical missing values with the most frequent category.\n",
        "- Missing as category: preserve missingness as an explicit categorical value.\n",
        "- Mode imputation plus indicators: fill values and add flags for missingness.\n",
        "- Tree-based model with missing categories: use a model family that is robust for tabular data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "\n",
        "from sklearn.compose import ColumnTransformer\n",
        "from sklearn.ensemble import HistGradientBoostingClassifier\n",
        "from sklearn.impute import SimpleImputer\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.metrics import accuracy_score, f1_score, roc_auc_score\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.pipeline import Pipeline\n",
        "from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder, StandardScaler\n",
        "\n",
        "RANDOM_STATE = 42"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Load The Dataset\n",
        "\n",
        "The Adult dataset uses `?` to represent missing values. We convert those markers to proper `NaN` values while loading the CSV."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "columns = [\n",
        "    \"age\",\n",
        "    \"workclass\",\n",
        "    \"fnlwgt\",\n",
        "    \"education\",\n",
        "    \"education_num\",\n",
        "    \"marital_status\",\n",
        "    \"occupation\",\n",
        "    \"relationship\",\n",
        "    \"race\",\n",
        "    \"sex\",\n",
        "    \"capital_gain\",\n",
        "    \"capital_loss\",\n",
        "    \"hours_per_week\",\n",
        "    \"native_country\",\n",
        "    \"income\",\n",
        "]\n",
        "\n",
        "data_path = \"uci_adult.csv\"\n",
        "\n",
        "df = pd.read_csv(\n",
        "    data_path,\n",
        "    names=columns,\n",
        "    skipinitialspace=True,\n",
        "    na_values=\"?\",\n",
        ")\n",
        "\n",
        "df.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Audit Missingness\n",
        "\n",
        "Before choosing an imputation method, inspect where missing values appear and how common they are."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "missing = (\n",
        "    df.isna()\n",
        "      .sum()\n",
        "      .loc[lambda s: s > 0]\n",
        "      .sort_values(ascending=False)\n",
        "      .to_frame(\"missing_rows\")\n",
        ")\n",
        "\n",
        "missing[\"missing_pct\"] = (missing[\"missing_rows\"] / len(df)).round(4)\n",
        "missing"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "missing_by_target = (\n",
        "    df.assign(income_above_50k=df[\"income\"].eq(\">50K\"))\n",
        "      .groupby(\"income_above_50k\")[[\"workclass\", \"occupation\", \"native_country\"]]\n",
        "      .apply(lambda frame: frame.isna().mean())\n",
        "      .round(4)\n",
        ")\n",
        "\n",
        "missing_by_target"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Prepare A Train/Test Split\n",
        "\n",
        "Fit imputers only on the training data, then apply them to the test data. That keeps the evaluation honest."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "X = df.drop(columns=\"income\")\n",
        "y = df[\"income\"].str.replace(\".\", \"\", regex=False).eq(\">50K\").astype(int)\n",
        "\n",
        "numeric_features = X.select_dtypes(include=\"number\").columns.tolist()\n",
        "categorical_features = X.select_dtypes(exclude=\"number\").columns.tolist()\n",
        "\n",
        "X_train, X_test, y_train, y_test = train_test_split(\n",
        "    X,\n",
        "    y,\n",
        "    test_size=0.2,\n",
        "    random_state=RANDOM_STATE,\n",
        "    stratify=y,\n",
        ")\n",
        "\n",
        "X_train.shape, X_test.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "def evaluate_model(name, pipeline, X_train, X_test, y_train, y_test):\n",
        "    pipeline.fit(X_train, y_train)\n",
        "\n",
        "    predictions = pipeline.predict(X_test)\n",
        "    probabilities = pipeline.predict_proba(X_test)[:, 1]\n",
        "\n",
        "    return {\n",
        "        \"strategy\": name,\n",
        "        \"train_rows\": len(X_train),\n",
        "        \"test_rows\": len(X_test),\n",
        "        \"accuracy\": accuracy_score(y_test, predictions),\n",
        "        \"f1\": f1_score(y_test, predictions),\n",
        "        \"roc_auc\": roc_auc_score(y_test, probabilities),\n",
        "    }"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Strategy 1: Complete Case Analysis\n",
        "\n",
        "Drop every row with at least one missing value. This is simple, but it changes the training sample and can discard useful information."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "train_complete = X_train.notna().all(axis=1)\n",
        "test_complete = X_test.notna().all(axis=1)\n",
        "\n",
        "X_train_complete = X_train.loc[train_complete]\n",
        "y_train_complete = y_train.loc[train_complete]\n",
        "X_test_complete = X_test.loc[test_complete]\n",
        "y_test_complete = y_test.loc[test_complete]\n",
        "\n",
        "complete_case_preprocess = ColumnTransformer(\n",
        "    transformers=[\n",
        "        (\"num\", StandardScaler(), numeric_features),\n",
        "        (\"cat\", OneHotEncoder(handle_unknown=\"ignore\"), categorical_features),\n",
        "    ]\n",
        ")\n",
        "\n",
        "complete_case_model = Pipeline(\n",
        "    steps=[\n",
        "        (\"preprocess\", complete_case_preprocess),\n",
        "        (\"model\", LogisticRegression(max_iter=1000)),\n",
        "    ]\n",
        ")\n",
        "\n",
        "complete_case_result = evaluate_model(\n",
        "    \"complete_case\",\n",
        "    complete_case_model,\n",
        "    X_train_complete,\n",
        "    X_test_complete,\n",
        "    y_train_complete,\n",
        "    y_test_complete,\n",
        ")\n",
        "\n",
        "complete_case_result"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Strategy 2: Mode Imputation\n",
        "\n",
        "Fill categorical missing values with the most frequent category observed in the training data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "mode_preprocess = ColumnTransformer(\n",
        "    transformers=[\n",
        "        (\"num\", StandardScaler(), numeric_features),\n",
        "        (\n",
        "            \"cat\",\n",
        "            Pipeline(\n",
        "                steps=[\n",
        "                    (\"imputer\", SimpleImputer(strategy=\"most_frequent\")),\n",
        "                    (\"encoder\", OneHotEncoder(handle_unknown=\"ignore\")),\n",
        "                ]\n",
        "            ),\n",
        "            categorical_features,\n",
        "        ),\n",
        "    ]\n",
        ")\n",
        "\n",
        "mode_model = Pipeline(\n",
        "    steps=[\n",
        "        (\"preprocess\", mode_preprocess),\n",
        "        (\"model\", LogisticRegression(max_iter=1000)),\n",
        "    ]\n",
        ")\n",
        "\n",
        "mode_result = evaluate_model(\"mode_imputation\", mode_model, X_train, X_test, y_train, y_test)\n",
        "mode_result"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Strategy 3: Missing As Category\n",
        "\n",
        "For categorical fields, missingness can be meaningful. This strategy keeps it as a value called `Missing`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "missing_category_preprocess = ColumnTransformer(\n",
        "    transformers=[\n",
        "        (\"num\", StandardScaler(), numeric_features),\n",
        "        (\n",
        "            \"cat\",\n",
        "            Pipeline(\n",
        "                steps=[\n",
        "                    (\"imputer\", SimpleImputer(strategy=\"constant\", fill_value=\"Missing\")),\n",
        "                    (\"encoder\", OneHotEncoder(handle_unknown=\"ignore\")),\n",
        "                ]\n",
        "            ),\n",
        "            categorical_features,\n",
        "        ),\n",
        "    ]\n",
        ")\n",
        "\n",
        "missing_category_model = Pipeline(\n",
        "    steps=[\n",
        "        (\"preprocess\", missing_category_preprocess),\n",
        "        (\"model\", LogisticRegression(max_iter=1000)),\n",
        "    ]\n",
        ")\n",
        "\n",
        "missing_category_result = evaluate_model(\n",
        "    \"missing_as_category\",\n",
        "    missing_category_model,\n",
        "    X_train,\n",
        "    X_test,\n",
        "    y_train,\n",
        "    y_test,\n",
        ")\n",
        "\n",
        "missing_category_result"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Strategy 4: Mode Imputation Plus Missingness Indicators\n",
        "\n",
        "This strategy fills missing values and also adds flags showing whether each value was missing originally."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "indicator_preprocess = ColumnTransformer(\n",
        "    transformers=[\n",
        "        (\"num\", StandardScaler(), numeric_features),\n",
        "        (\n",
        "            \"cat\",\n",
        "            Pipeline(\n",
        "                steps=[\n",
        "                    (\"imputer\", SimpleImputer(strategy=\"most_frequent\", add_indicator=True)),\n",
        "                    (\"encoder\", OneHotEncoder(handle_unknown=\"ignore\")),\n",
        "                ]\n",
        "            ),\n",
        "            categorical_features,\n",
        "        ),\n",
        "    ]\n",
        ")\n",
        "\n",
        "indicator_model = Pipeline(\n",
        "    steps=[\n",
        "        (\"preprocess\", indicator_preprocess),\n",
        "        (\"model\", LogisticRegression(max_iter=1000)),\n",
        "    ]\n",
        ")\n",
        "\n",
        "indicator_result = evaluate_model(\n",
        "    \"mode_plus_indicators\",\n",
        "    indicator_model,\n",
        "    X_train,\n",
        "    X_test,\n",
        "    y_train,\n",
        "    y_test,\n",
        ")\n",
        "\n",
        "indicator_result"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Strategy 5: Tree-Based Model With Missing Categories\n",
        "\n",
        "For this dataset, the missing columns are categorical. We still need to encode them, but a tree-based model is often a strong practical choice for mixed tabular data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "tree_preprocess = ColumnTransformer(\n",
        "    transformers=[\n",
        "        (\"num\", \"passthrough\", numeric_features),\n",
        "        (\n",
        "            \"cat\",\n",
        "            Pipeline(\n",
        "                steps=[\n",
        "                    (\"imputer\", SimpleImputer(strategy=\"constant\", fill_value=\"Missing\")),\n",
        "                    (\n",
        "                        \"encoder\",\n",
        "                        OrdinalEncoder(\n",
        "                            handle_unknown=\"use_encoded_value\",\n",
        "                            unknown_value=-1,\n",
        "                        ),\n",
        "                    ),\n",
        "                ]\n",
        "            ),\n",
        "            categorical_features,\n",
        "        ),\n",
        "    ]\n",
        ")\n",
        "\n",
        "tree_model = Pipeline(\n",
        "    steps=[\n",
        "        (\"preprocess\", tree_preprocess),\n",
        "        (\"model\", HistGradientBoostingClassifier(random_state=RANDOM_STATE)),\n",
        "    ]\n",
        ")\n",
        "\n",
        "tree_result = evaluate_model(\n",
        "    \"tree_model_missing_category\",\n",
        "    tree_model,\n",
        "    X_train,\n",
        "    X_test,\n",
        "    y_train,\n",
        "    y_test,\n",
        ")\n",
        "\n",
        "tree_result"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Compare Results\n",
        "\n",
        "The complete-case result uses fewer rows, so compare it with care. The other strategies use the same train/test rows."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "results = pd.DataFrame(\n",
        "    [\n",
        "        complete_case_result,\n",
        "        mode_result,\n",
        "        missing_category_result,\n",
        "        indicator_result,\n",
        "        tree_result,\n",
        "    ]\n",
        ").sort_values(\"roc_auc\", ascending=False)\n",
        "\n",
        "results"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Takeaway\n",
        "\n",
        "For categorical missing values, `Missing` as an explicit category and missingness indicators are often stronger first choices than forcing numeric-style imputers into the workflow.\n",
        "\n",
        "KNN and MICE are useful tools, especially for numeric missing values, but the data type matters. On this dataset, the missing fields are categorical, so preserving missingness as categorical information is usually the cleaner starting point."
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}