Implement Bootstrap and Jackknife Estimates on a DataFrame

Problem Statement

At Pinterest, many key user engagement metrics (such as the ratio of Pin saves to impressions, or click-through rates) are complex ratio statistics where analytical closed-form standard errors are difficult or impossible to derive. In these scenarios, non-parametric resampling methods like the Bootstrap and the Jackknife are widely used to quantify metric variance and construct confidence intervals.

Implement two statistical functions that operate on a pandas DataFrame:

1. bootstrap_ci(df, stat_fn, n_boot=1000, ci=0.95, random_state=None): Computes a two-sided empirical percentile confidence interval for the statistic returned by stat_fn(df).
2. jackknife_variance(df, stat_fn): Computes the leave-one-out jackknife variance estimate of stat_fn(df).

Mathematical Definitions

• Bootstrap Percentile Interval: Draw n_boot samples of size n with replacement from df (where n = len(df)). Evaluate stat_fn on each sample. For confidence level ci, the lower quantile is alpha / 2 and the upper quantile is 1 - alpha / 2, where alpha = 1 - ci.
• Jackknife Variance: Given n observations, compute stat_i = stat_fn(df_{-i}) for each i in 0, ..., n - 1, where df_{-i} denotes df with row i omitted. The jackknife variance estimate is: Var_jack = ((n - 1) / n) * sum((stat_i - stat_mean)**2) where stat_mean = (1 / n) * sum(stat_i).

Function Signatures

import pandas as pd
from typing import Callable, Tuple

def bootstrap_ci(
    df: pd.DataFrame,
    stat_fn: Callable[[pd.DataFrame], float],
    n_boot: int = 1000,
    ci: float = 0.95,
    random_state: int | None = None
) -> Tuple[float, float]:
    ...

def jackknife_variance(
    df: pd.DataFrame,
    stat_fn: Callable[[pd.DataFrame], float]
) -> float:
    ...

Constraints

• df contains n rows where 2 <= n <= 10_000.
• stat_fn accepts a DataFrame and returns a scalar float.
• Ensure sampling in bootstrap is done positionally to prevent row index alignment artifacts.
• Output should be rounded or comparable to within standard numerical precision tolerances (1e-4).

Example

import pandas as pd
import numpy as np

data = pd.DataFrame({
    "saves": [10, 20, 15, 30, 25],
    "impressions": [100, 200, 150, 250, 200]
})

def save_rate(d: pd.DataFrame) -> float:
    return d["saves"].sum() / d["impressions"].sum()

# Jackknife variance
j_var = jackknife_variance(data, save_rate)
# j_var is approximately 0.000015

# Bootstrap confidence interval
ci_low, ci_high = bootstrap_ci(data, save_rate, n_boot=2000, ci=0.90, random_state=42)
# ci_low, ci_high will bracket the point estimate (~0.111) reasonably

Edge Cases

• Handling n < 2: raise ValueError for jackknife_variance.
• Preserving reproducibility when random_state is provided.
• Non-contiguous or duplicate indices in input df must not break leave-one-out drops or bootstrap resampling.