Expedia Group — Customer Data Platform — A/B Test & Causal Inference Questions

Role context: Senior Data Scientist, Customer Data Platform (traveler profiles, identity, loyalty data) · Est. study time: 75 min · 7 questions

Experimentation in this domain

Expedia Group runs experiments at large scale on one unified technology stack, with monitoring that can stop a harmful test within minutes. What makes experiments around the customer data platform different is that the platform decides who a traveler is.

• The unit is a person, and people are fragmented. One traveler can have several cookies, an app install and accounts on three brands. Randomizing a cookie can put the same person in both arms. Randomizing a profile works only as well as the identity graph.
• Identity changes during a test. Profiles merge and split as travelers sign in. If assignment follows the current profile, travelers can switch arms mid-test.
• Outcomes are rare and slow. A few bookings a year, booked weeks ahead, some cancelled. Read net bookings over a full booking window, and expect small effects.
• The group, not the brand. Personalization can move trips between Expedia, Hotels.com and Vrbo. Decisions use group-level outcomes.
• Who can be measured depends on consent. Changes in consent change the measured population, which can fake an effect.
• Much of the value is long-term and non-random. Loyalty membership and engagement features are chosen by travelers, so their effects need careful quasi-experiments, and marketing's cumulative value needs long-running holdouts.

This platform role weights identity-aware experiment design, holdout sizing for marketing, group-level decisions, quasi-experiments on loyalty and consent, and measuring the identity graph itself. It skips sequential monitoring and multiple-testing questions, which the experimentation platform already handles.