Walmart — Demand Forecasting — A/B Test & Causal Inference Questions

Role context: Senior Data Scientist, Demand Forecasting (US Stores & eCommerce) · Est. study time: 80 min · 7 questions

Experimentation in this domain

Forecasting at Walmart breaks the standard supervised-learning setup in four specific ways, and this section is about each of them.

You never see the label. You observe sales, not demand. A stockout produces sales of zero with demand above zero, so your target variable is censored exactly when you got it wrong. Every model on this site is trained on the outcome it is predicting. This one is not.

Your prediction changes the label. The forecast becomes the order, the order caps the sales, and the sales become tomorrow's training data. That is closer to a bandit problem than to supervised learning, and almost nobody frames it that way in an interview.

The mean is the wrong answer. Over-forecast a banana and you throw it away; under-forecast and you lose the sale and maybe the trip. Costs are unequal and item-specific, so the right point forecast is a quantile, chosen by the newsvendor ratio, and the right loss is the pinball loss. MSE optimizes the conditional mean, which is right for nothing here.

The evaluation is where careers go to die. Random k-fold on a time series leaks the future. One-step-ahead accuracy is not your decision horizon. MAPE is undefined on a grid that is mostly zeros. Each of those produces a number that looks great and predicts nothing.

Two more things shape everything below. Scale forces global models: 500 million store-item forecasts means you cannot fit and monitor per-series models, so cold start is a constant condition. And retail has an advantage most domains lack: you can genuinely randomize stores, so a real experiment is available if you can afford the power.

The set covers the measurement core (loss functions, censoring, backtesting, metric selection), then weights toward store-randomized experiments, hierarchical reconciliation, and the feedback loop that makes this domain unlike any other.