How to Handle Missing Data in Machine Learning: A Hands-On Example

In Part 1 of this series, I covered the main ways to handle missing data: deletion, simple imputation, KNN imputation, multiple imputation, and algorithms that can deal with missing values internally.

In Part 2, I showed those ideas on the Titanic dataset. Titanic is useful because everyone understands the problem quickly, but it is also a bit too small and familiar. For a stronger hands-on example, we want something closer to a real modelling workflow.

So in this article we will use the UCI Adult Census Income dataset, also available from the UCI archive files. For the downloadable notebook and Notebook Studio example, I bundle a copy of the data as uci_adult.csv so the browser runtime does not need to make an HTTPS request from inside Python. It is a good missing-data case study because:

• the target is clear: predict whether income is above 50K;
• the data mixes numeric and categorical variables;
• the missing values are real dataset markers, not artificially injected blanks;
• the missingness appears in important categorical fields such as workclass, occupation, and native-country.

The goal is not to find the best possible income model. The goal is to make a clean, repeatable comparison of missing-data strategies.

Prefer to run it yourself? Download the notebook and execute the workflow locally.

Download .ipynb

Open in Notebook Studio

Download dataset

Load The Dataset

The Adult dataset uses ? to mark missing values. We convert those markers to proper NaN values as soon as we load the data.

import numpy as np
import pandas as pd

columns = [
    "age",
    "workclass",
    "fnlwgt",
    "education",
    "education_num",
    "marital_status",
    "occupation",
    "relationship",
    "race",
    "sex",
    "capital_gain",
    "capital_loss",
    "hours_per_week",
    "native_country",
    "income",
]

data_path = "uci_adult.csv"

df = pd.read_csv(
    data_path,
    names=columns,
    skipinitialspace=True,
    na_values="?",
)

df.head()

The first rule of missing data: do not rush to fill it. First, measure it.

missing = (
    df.isna()
      .sum()
      .loc[lambda s: s > 0]
      .sort_values(ascending=False)
      .to_frame("missing_rows")
)

missing["missing_pct"] = (missing["missing_rows"] / len(df)).round(4)
missing

You should see missing values in three categorical columns:

• occupation
• workclass
• native_country

That matters. If the missing data were numeric, mean, median, KNN, or iterative imputation would be the obvious candidates. Here we need to think about categorical missingness, where the choice is usually between deletion, most-frequent imputation, and treating missing as its own category.

Build A Baseline Split

Before comparing strategies, split the data. Any imputer must be fitted on the training data only, then applied to the test data. That keeps information from the test set out of the training process.

from sklearn.model_selection import train_test_split

X = df.drop(columns="income")
y = df["income"].str.replace(".", "", regex=False).eq(">50K").astype(int)

numeric_features = X.select_dtypes(include="number").columns.tolist()
categorical_features = X.select_dtypes(exclude="number").columns.tolist()

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42,
    stratify=y,
)

We will use the same model for every strategy so that the comparison focuses on missing-data handling, not model choice.

from sklearn.compose import ColumnTransformer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, f1_score, roc_auc_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

def evaluate_model(name, pipeline, X_train, X_test, y_train, y_test):
    pipeline.fit(X_train, y_train)

    predictions = pipeline.predict(X_test)
    probabilities = pipeline.predict_proba(X_test)[:, 1]

    return {
        "strategy": name,
        "accuracy": accuracy_score(y_test, predictions),
        "f1": f1_score(y_test, predictions),
        "roc_auc": roc_auc_score(y_test, probabilities),
    }

Strategy 1: Complete Case Analysis

Complete case analysis removes every row with at least one missing value. It is simple and sometimes defensible, but only when the missing rows are few and plausibly random.

train_complete = X_train.notna().all(axis=1)
test_complete = X_test.notna().all(axis=1)

X_train_complete = X_train.loc[train_complete]
y_train_complete = y_train.loc[train_complete]

X_test_complete = X_test.loc[test_complete]
y_test_complete = y_test.loc[test_complete]

complete_case_preprocess = ColumnTransformer(
    transformers=[
        ("num", StandardScaler(), numeric_features),
        ("cat", OneHotEncoder(handle_unknown="ignore"), categorical_features),
    ]
)

complete_case_model = Pipeline(
    steps=[
        ("preprocess", complete_case_preprocess),
        ("model", LogisticRegression(max_iter=1000)),
    ]
)

complete_case_result = evaluate_model(
    "complete_case",
    complete_case_model,
    X_train_complete,
    X_test_complete,
    y_train_complete,
    y_test_complete,
)

The advantage is clarity. The cost is data loss. On this dataset, dropping all incomplete rows removes thousands of training examples. That may be acceptable for a quick baseline, but it is rarely where I would stop.

Strategy 2: Most-Frequent Imputation

For categorical columns, mode imputation replaces missing values with the most common category in the training data.

from sklearn.impute import SimpleImputer

mode_preprocess = ColumnTransformer(
    transformers=[
        ("num", StandardScaler(), numeric_features),
        (
            "cat",
            Pipeline(
                steps=[
                    ("imputer", SimpleImputer(strategy="most_frequent")),
                    ("encoder", OneHotEncoder(handle_unknown="ignore")),
                ]
            ),
            categorical_features,
        ),
    ]
)

mode_model = Pipeline(
    steps=[
        ("preprocess", mode_preprocess),
        ("model", LogisticRegression(max_iter=1000)),
    ]
)

mode_result = evaluate_model(
    "mode_imputation",
    mode_model,
    X_train,
    X_test,
    y_train,
    y_test,
)

This is often a good baseline. It keeps all rows and it is easy to explain. The trade-off is that it can hide the fact that a value was missing in the first place. If missingness is informative, mode imputation throws that signal away.

Strategy 3: Treat Missing As A Category

For categorical features, missing can be a valid state. Someone’s occupation being unavailable may carry information that is different from the most common occupation.

missing_category_preprocess = ColumnTransformer(
    transformers=[
        ("num", StandardScaler(), numeric_features),
        (
            "cat",
            Pipeline(
                steps=[
                    ("imputer", SimpleImputer(strategy="constant", fill_value="Missing")),
                    ("encoder", OneHotEncoder(handle_unknown="ignore")),
                ]
            ),
            categorical_features,
        ),
    ]
)

missing_category_model = Pipeline(
    steps=[
        ("preprocess", missing_category_preprocess),
        ("model", LogisticRegression(max_iter=1000)),
    ]
)

missing_category_result = evaluate_model(
    "missing_as_category",
    missing_category_model,
    X_train,
    X_test,
    y_train,
    y_test,
)

This is usually my preferred first serious attempt for categorical missing values. It is simple, leakage-safe, and preserves the possibility that missingness itself matters.

Strategy 4: Add Missingness Indicators

Another option is to impute the missing values and add binary flags that tell the model whether each value was missing.

indicator_preprocess = ColumnTransformer(
    transformers=[
        ("num", StandardScaler(), numeric_features),
        (
            "cat",
            Pipeline(
                steps=[
                    (
                        "imputer",
                        SimpleImputer(
                            strategy="most_frequent",
                            add_indicator=True,
                        ),
                    ),
                    ("encoder", OneHotEncoder(handle_unknown="ignore")),
                ]
            ),
            categorical_features,
        ),
    ]
)

indicator_model = Pipeline(
    steps=[
        ("preprocess", indicator_preprocess),
        ("model", LogisticRegression(max_iter=1000)),
    ]
)

indicator_result = evaluate_model(
    "mode_plus_indicators",
    indicator_model,
    X_train,
    X_test,
    y_train,
    y_test,
)

This gives the model two pieces of information:

• the imputed feature value;
• whether that value was originally missing.

It is a useful middle ground when you want a conventional imputation strategy but do not want to discard the missingness signal.

Strategy 5: Use A Model That Handles Missing Values

Some models can handle missing values in numeric features internally. For this dataset, the missing columns are categorical, so we still need to encode categories. One practical approach is to use an ordinal encoder that keeps missing values as a distinct code, then train a tree-based model.

from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.preprocessing import OrdinalEncoder

tree_preprocess = ColumnTransformer(
    transformers=[
        ("num", "passthrough", numeric_features),
        (
            "cat",
            Pipeline(
                steps=[
                    ("imputer", SimpleImputer(strategy="constant", fill_value="Missing")),
                    (
                        "encoder",
                        OrdinalEncoder(
                            handle_unknown="use_encoded_value",
                            unknown_value=-1,
                        ),
                    ),
                ]
            ),
            categorical_features,
        ),
    ]
)

tree_model = Pipeline(
    steps=[
        ("preprocess", tree_preprocess),
        ("model", HistGradientBoostingClassifier(random_state=42)),
    ]
)

tree_result = evaluate_model(
    "tree_model_missing_category",
    tree_model,
    X_train,
    X_test,
    y_train,
    y_test,
)

This is not a pure “do nothing” approach, because categorical values still need to become numbers. But it is closer to how many production systems work: choose a model family that is robust to messy tabular data, then encode missing categorical values deliberately.

Compare The Results

Now collect the results in one table.

results = pd.DataFrame(
    [
        complete_case_result,
        mode_result,
        missing_category_result,
        indicator_result,
        tree_result,
    ]
).sort_values("roc_auc", ascending=False)

results

Do not treat the top row as a universal rule. The point is to compare strategies under the same split, target, features, and metric. On this dataset, I would pay attention to three things:

• Does complete case analysis lose too many rows? If yes, the clean simplicity is not worth the information loss.
• Does “Missing” as a category perform as well as or better than mode imputation? If yes, missingness is probably carrying signal.
• Does the tree-based model improve ROC AUC or F1? If yes, the missing-data decision may be interacting with broader model choice.

Where KNN And MICE Fit

In Part 1 I included KNN and MICE because they are important techniques. In Part 2 I used them on Titanic’s numeric age column.

For the Adult dataset, they are less natural as first choices because the missing values are categorical. You can force categorical variables into numeric codes and run KNN or iterative imputation, but that often creates fake distances and fake orderings. For example, there is no real numeric distance between Private, Self-emp-not-inc, and State-gov work classes.

That is the lesson: the best missing-data method depends on the type of missing feature, not only on the name of the technique.

Practical Recommendation

For a real tabular machine learning project, I would use this order:

1. Audit missingness by column and by target.
2. Build a complete-case baseline, but do not fall in love with it.
3. For categorical missing values, compare mode imputation against “Missing” as a category.
4. Add missingness indicators when you suspect the absence of a value is informative.
5. Re-run the comparison with the model family you actually plan to use.

Missing data is not a preprocessing nuisance. It is part of the signal, part of the bias risk, and part of the modelling decision.

That is why the right question is not “which imputer should I always use?” The right question is: what does missingness mean in this dataset, and which modelling choice preserves that meaning best?

Related Reading

• How to Handle Missing Data in Machine Learning (Part 1)
• Missing Data in Machine Learning: Imputation Examples in Python
• Signal vs Noise: Machine Learning Metrics Made Interactive
• Download the companion notebook
• Download the companion dataset
• Open the companion notebook in Notebook Studio

Let’s Connect

You can also find me on:

• X/Twitter: @feddernico
• Medium: @federico.viscioletti
• Substack: https://feddernico.substack.com/