{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# LambdaRank-style Learning to Rank\n",
        "\n",
        "This Notebook Studio example accompanies the article [LambdaRank Explained: How It Improves Learning to Rank](/posts/2026/07/27/lambdarank-explained-how-it-improves-learning-to-rank/).\n",
        "\n",
        "The goal is to make the core idea concrete: ranking models compare candidate items within the same query, and evaluation should reward putting the most relevant items near the top."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 1. Load a small ranking dataset\n",
        "\n",
        "Each row is a candidate search result. The `query_id` column tells us which candidates belong to the same query. Relevance is graded from `0` to `3`, where `3` is the most relevant.\n",
        "\n",
        "This is a small LETOR-style sample bundled with the notebook so it can run fully in the browser. The feature columns are:\n",
        "\n",
        "- `item`: the candidate result we might show to the user.\n",
        "- `topicality`: how closely the candidate appears to match the query intent.\n",
        "- `authority`: a rough proxy for trust, quality, or source strength.\n",
        "- `freshness`: how current or recently updated the candidate is. This feature is intentionally noisy: fresh content is not always the most relevant content.\n",
        "- `click_prior`: a rough prior from past engagement.\n",
        "- `content_depth`: a rough proxy for how complete or substantial the content is.\n",
        "\n",
        "In a real ranking system, these features might come from embeddings, metadata, click logs, content quality signals, or business rules. Here they are compact and readable so the ranking mechanics stay visible.\n",
        "\n",
        "Notebook Studio should mount `lambdarank_search_relevance.csv` automatically. The loading cell below also includes a browser fallback that fetches the CSV from `/downloads/` if the mounted file is not present."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import math\n",
        "from pathlib import Path\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "\n",
        "data_path = Path(\"lambdarank_search_relevance.csv\")\n",
        "\n",
        "if data_path.exists():\n",
        "    data = pd.read_csv(data_path)\n",
        "else:\n",
        "    from js import window\n",
        "    from pyodide.http import open_url\n",
        "\n",
        "    dataset_url = f\"{window.location.origin}/downloads/lambdarank_search_relevance.csv\"\n",
        "    data = pd.read_csv(open_url(dataset_url))\n",
        "\n",
        "data.head(12)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 2. Compare a baseline with an XGBoost ranking model\n",
        "\n",
        "Pyodide includes XGBoost, so we can train a small `XGBRanker` directly in Notebook Studio. The first run may take a little longer because the browser has to load XGBoost and scikit-learn.\n",
        "\n",
        "Before training the model, we also create a deliberately simple baseline that ranks candidates by `freshness` only. This gives us something imperfect to compare against.\n",
        "\n",
        "We train on queries 1-8 and evaluate on queries 9-12. Splitting by query is important: a ranking model should prove it can rank new query groups, not just remember the rows it already saw.\n",
        "\n",
        "The important detail for XGBoost is `qid`: it tells the model which candidates belong to the same query. Ranking losses compare candidates inside each query group."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import sklearn  # Required by XGBoost's scikit-learn wrapper in Pyodide.\n",
        "import xgboost as xgb\n",
        "\n",
        "train = data[data[\"query_id\"] <= 8].copy()\n",
        "test = data[data[\"query_id\"] > 8].copy()\n",
        "\n",
        "baseline_ranked = test.copy()\n",
        "baseline_ranked[\"score\"] = baseline_ranked[\"freshness\"]\n",
        "baseline_ranked = (\n",
        "    baseline_ranked.sort_values([\"query_id\", \"score\"], ascending=[True, False])\n",
        "                   .assign(rank=lambda df: df.groupby(\"query_id\").cumcount() + 1)\n",
        ")\n",
        "\n",
        "feature_cols = [\"topicality\", \"authority\", \"freshness\", \"click_prior\", \"content_depth\"]\n",
        "X_train = train[feature_cols]\n",
        "y_train = train[\"relevance\"]\n",
        "qid_train = train[\"query_id\"]\n",
        "X_test = test[feature_cols]\n",
        "\n",
        "ranker = xgb.XGBRanker(\n",
        "    objective=\"rank:ndcg\",\n",
        "    tree_method=\"hist\",\n",
        "    n_estimators=60,\n",
        "    learning_rate=0.1,\n",
        "    max_depth=2,\n",
        "    random_state=42,\n",
        ")\n",
        "\n",
        "ranker.fit(X_train, y_train, qid=qid_train)\n",
        "\n",
        "xgboost_ranked = test.copy()\n",
        "xgboost_ranked[\"score\"] = ranker.predict(X_test)\n",
        "xgboost_ranked = (\n",
        "    xgboost_ranked.sort_values([\"query_id\", \"score\"], ascending=[True, False])\n",
        "                  .assign(rank=lambda df: df.groupby(\"query_id\").cumcount() + 1)\n",
        ")\n",
        "\n",
        "ranked = xgboost_ranked\n",
        "\n",
        "pd.concat(\n",
        "    [\n",
        "        baseline_ranked.assign(model=\"freshness baseline\"),\n",
        "        xgboost_ranked.assign(model=\"XGBoost rank:ndcg\"),\n",
        "    ],\n",
        "    ignore_index=True,\n",
        ")[[\"model\", \"query_id\", \"rank\", \"item\", \"score\", \"relevance\"]]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 3. Implement DCG and NDCG\n",
        "\n",
        "NDCG rewards relevant results near the top. The discount term means that relevance at rank 1 matters more than the same relevance at rank 10."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "def dcg(relevances):\n",
        "    \"\"\"Compute Discounted Cumulative Gain for relevance scores in ranked order.\"\"\"\n",
        "    return sum(((2 ** rel) - 1) / math.log2(position + 2) for position, rel in enumerate(relevances))\n",
        "\n",
        "def ndcg(relevances):\n",
        "    \"\"\"Compute Normalized DCG by comparing a ranking with the ideal relevance order.\"\"\"\n",
        "    actual = dcg(relevances)\n",
        "    ideal = dcg(sorted(relevances, reverse=True))\n",
        "    return actual / ideal if ideal else 0.0\n",
        "\n",
        "scores = []\n",
        "for model_name, frame in [\n",
        "    (\"freshness baseline\", baseline_ranked),\n",
        "    (\"XGBoost rank:ndcg\", xgboost_ranked),\n",
        "]:\n",
        "    for query_id, group in frame.groupby(\"query_id\"):\n",
        "        relevances = group.sort_values(\"rank\")[\"relevance\"].tolist()\n",
        "        scores.append({\"query_id\": query_id, \"model\": model_name, \"ndcg\": ndcg(relevances)})\n",
        "\n",
        "ndcg_scores = (\n",
        "    pd.DataFrame(scores)\n",
        "      .pivot(index=\"query_id\", columns=\"model\", values=\"ndcg\")\n",
        "      .reset_index()\n",
        ")\n",
        "ndcg_scores[\"improvement\"] = ndcg_scores[\"XGBoost rank:ndcg\"] - ndcg_scores[\"freshness baseline\"]\n",
        "ndcg_scores"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 4. Why LambdaRank cares about swaps\n",
        "\n",
        "LambdaRank asks how much the ranking metric would change if two items swapped position. A swap near the top of the list often changes NDCG more than a swap near the bottom.\n",
        "\n",
        "Here we use the imperfect freshness baseline so the effect is easier to see."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "def delta_ndcg_for_swap(group, rank_a, rank_b):\n",
        "    \"\"\"Return the NDCG change caused by swapping two 1-based rank positions.\"\"\"\n",
        "    ordered = group.sort_values(\"rank\").reset_index(drop=True)\n",
        "    before = ndcg(ordered[\"relevance\"].tolist())\n",
        "    swapped = ordered.copy()\n",
        "    i, j = rank_a - 1, rank_b - 1\n",
        "    swapped.iloc[[i, j]] = swapped.iloc[[j, i]].to_numpy()\n",
        "    after = ndcg(swapped[\"relevance\"].tolist())\n",
        "    return after - before\n",
        "\n",
        "rows = []\n",
        "for query_id, group in baseline_ranked.groupby(\"query_id\"):\n",
        "    rows.append({\n",
        "        \"query_id\": query_id,\n",
        "        \"swap\": \"rank 1 vs rank 2\",\n",
        "        \"delta_ndcg\": delta_ndcg_for_swap(group, 1, 2),\n",
        "    })\n",
        "    rows.append({\n",
        "        \"query_id\": query_id,\n",
        "        \"swap\": \"rank 3 vs rank 4\",\n",
        "        \"delta_ndcg\": delta_ndcg_for_swap(group, 3, 4),\n",
        "    })\n",
        "\n",
        "pd.DataFrame(rows)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 5. Visualise relevance by rank\n",
        "\n",
        "This chart shows the freshness-only baseline on held-out queries. Because freshness is a noisy signal, high relevance is not always concentrated on the left side of each chart. That imperfection is the point: it gives the ranking model room to improve."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "fig, axes = plt.subplots(2, 2, figsize=(10, 6), sharey=True)\n",
        "axes = axes.ravel()\n",
        "\n",
        "for ax, (query_id, group) in zip(axes, baseline_ranked.groupby(\"query_id\")):\n",
        "    ordered = group.sort_values(\"rank\")\n",
        "    ax.bar(ordered[\"rank\"], ordered[\"relevance\"], color=\"#3b82f6\")\n",
        "    ax.set_title(f\"Query {query_id}\")\n",
        "    ax.set_xlabel(\"Rank\")\n",
        "    ax.set_xticks(ordered[\"rank\"])\n",
        "\n",
        "axes[0].set_ylabel(\"Relevance\")\n",
        "fig.suptitle(\"Freshness baseline: relevance by ranked position on held-out queries\")\n",
        "plt.tight_layout()\n",
        "fig"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Takeaway\n",
        "\n",
        "The concept behind LambdaRank-style training is: compare candidates inside each query group and give stronger learning signals to swaps that improve the top of the ranked list.\n",
        "\n",
        "In this example, XGBoost's `rank:ndcg` objective gives us a browser-runnable LambdaMART-style ranking model. For larger real-world ranking problems, you would usually add more query groups, stronger features, and proper train/test splitting by query."
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "pygments_lexer": "ipython3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}