The watch-time lift is fading over the days-in window — real or novelty? — Google YouTube Data Scientist Interview

Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.

1

Clarify the question first

"Is the daily lift decaying toward a positive floor or toward zero? That's the difference between a real, smaller effect and pure novelty."
"Is this measured on the same users over time (days-in), not confounded by new entrants?"
"Can I run it long enough to see it stabilize?"

2

Lay out your approach

Name it: novelty effect. Anything new gets a temporary bump — viewers try the different thing — and it fades.
Read the stabilized effect, not the peak. The honest number is where the days-in curve levels off.
Decide by the floor. Plateau above zero = a real, smaller effect; decay to zero = pure novelty (don't ship).
Extend if it's still falling, and watch guardrails over the same window.

Then check in: "So I'd read the stabilized effect off the days-in curve, not the +6% peak, and decide by whether it plateaus above zero. Want me to show how I'd estimate the floor?"

3

Simulate the back-and-forth

Interviewer

Day one was +6%. Why not report that?

You

Because day one is inflated by novelty — the change is new, so viewers engage with it for being new, and that fades. Reporting +6% overstates the durable effect. I'd read where the days-in curve settles.

Interviewer

Suppose it slides from +6% toward +2% and flattens.

You

Then the real effect is about +2%, the plateau. The decay from 6 to 2 is novelty wearing off; the 2 that remains is durable. I'd report ~+2% and flag the novelty component so no one anchors on day one.

Interviewer

What if it's heading to zero?

You

Then it was pure novelty — a temporary bump with no lasting effect, and I wouldn't ship. The tell is whether it's asymptoting above zero or continuing down to it. If it's still falling at two weeks, I'd extend rather than guess the floor.

Interviewer

Does anything else confound the curve?

You

New users entering mid-experiment can, which is why the days-in view aligns users by time-in-experiment rather than calendar date. And if a model retrains during the window, that can add steps to the curve, so I'd align the read with retrain dates and use a stable stretch.

4

Land a clear conclusion

You

So: this is a novelty effect, and the honest number is the stabilized lift off the days-in curve, not the +6% peak. If it plateaus above zero — say +2% — that's the real effect, reported with the novelty flagged; if it decays to zero, it was pure novelty and I wouldn't ship. If it's still falling at two weeks, extend rather than guess, and read guardrails over the same window.

Key summary — what really matters.

A day-1 peak that fades is a novelty effect.
Report the stabilized effect, not the peak.
Decide by the floor: plateau above zero = real; decay to zero = pure novelty (don't ship).
Use days-in (time-in-experiment), not calendar time, to avoid new-entrant confounding.
Extend if still falling; align with retrain dates.

Dimension	What interviewers look for	Below the bar ❌	Above the bar ✅
Framing & estimand	Whether you pin down the causal quantity, the population, and the decision before any math.	You jump to a test or a number without saying what you're estimating or why.	You name the estimand, the unit, the population, and the decision it drives.
Design validity	Whether the design actually identifies the effect — randomization unit, interference, selection/confounding.	You analyze a biased or diluted comparison as if it were clean.	You pick the unit that contains interference, name the threat and its direction, and control it.
Statistical rigor	Whether the stats hold up — power and sample size, variance, the right test, and the assumptions behind them.	You misuse a test, ignore power or variance, or lean on an unchecked assumption.	You size the test, reduce or adjust variance, choose the fitting test, and state what would break it.
Decision & trade-offs	Whether you turn the result into a call — guardrails, cost, and what would flip it.	You stop at "significant / not," or ship on one metric while a guardrail breaks.	You land a ship/hold call, weigh the trade-offs, and name the number that changes it.
Communication	Whether you reason clearly, lead the conversation, and translate the stats for a stakeholder.	You dump jargon, or can't explain the method in plain words.	You reason out loud in order, check in, and give a plain recommendation.

Interview tips

Strong-answer signal: naming the novelty effect, reading the stabilized plateau off the days-in curve, and deciding by whether it asymptotes above zero. Common trap: reporting the day-1 peak, or averaging the whole window and overstating the durable effect.