Riot Games — League of Legends — A/B Test & Causal Inference

Role context: Insights Analyst, League of Legends PC · Est. study time: 50 min · 5 questions

How these map to the role

These are the causal problems of a live, competitive, team game: experiments where the meta shifts under you, matches span both arms, champion win rates need real significance, and metrics must resist gaming. Each answer is a coaching walkthrough: a Sample answer (clarify → approach → simulated back-and-forth → clear call), a Deep dive with the math, then a Grading rubric.

The recurring instinct: player experience and fairness are the goal and the guardrail, and a competitive team game breaks naive A/B assumptions (interference, meta effects) — so design around them.

Questions (5)

How do you test a balance change when it shifts the whole meta?Experiment designHardIs a champion really overpowered, or is it noise?Test selectionHardA/B testing matchmaking when a match spans both armsInterferenceHardTest a shop change without a misleading novelty spikeExperiment designMediumPick a ranked metric players can't gameMetric designHard