Slack — Go-to-Market — A/B Test & Causal Inference Questions

Role context: Senior Data Scientist, Slack Go-to-Market (Sales Strategy & Programs) · Est. study time: 75 min · 7 questions

Experimentation in this domain

Go-to-market experiments look different from product A/B tests because the treatment usually runs through a person.

• Reps are the unit, or the problem. If you randomize accounts within a rep's book, the rep learns the new play and uses it everywhere. If you randomize reps, you have a few hundred units with very different skill. Either way, outcomes cluster by rep.
• People don't always comply. Reps ignore recommendations, customer success managers prioritize their own way. Intent-to-treat is the honest default; the effect on those who actually used the program needs an instrument or a compliance adjustment.
• Outcomes are slow, lumpy and heavy-tailed. Bookings take a quarter, renewals a year, and a few big accounts dominate the dollars. Validated leading indicators, binary outcomes, capping and pre-period adjustment do the heavy lifting.
• Self-serve tests are ordinary A/B tests, with the usual traps: novelty, and upgrades that cancel a month later.
• Much of what matters can't be randomized. Customer success coverage follows a threshold, price changes hit customers at their renewal dates. Regression discontinuity and renewal-cohort designs fill the gap.
• Targeting is a causal question. The accounts most likely to buy aren't the ones a call changes most.

This GTM role weights experiment design inside a sales org (rep clustering, compliance), power with heavy-tailed revenue, stratified tests that avoid Simpson's paradox, uplift targeting, and quasi-experiments on thresholds and pricing. It skips network interference and large-scale multiple testing, which rarely decide go-to-market calls.