Meta — Facebook — Product Sense & Metrics
Role context: Data Scientist, Product Analytics (Facebook) · Est. study time: 40 min · 4 practice questions
How to prepare for this role
Meta's Data Scientist, Product Analytics is a general, product-agnostic role judged on problem framing, metric design, experimentation, and communication — not ML modeling. You're hired into the analytics community and matched to a team after the loop, so study products like Facebook as representative practice, not as a syllabus.
The loop is four rounds: Analytical Reasoning (structure an ambiguous product question, pick data, design a measurement), Analytical Execution (hypotheses plus applied statistics, usually with SQL), AI-Native DS Technical Skills (using AI tools in the analysis workflow without losing rigor), and Behavioral. Across all of them the interviewer is really testing one instinct: before you solve a problem, can you tell whether it's a problem worth solving? They will hand you a vague prompt ("engagement dropped in Brazil"), then add constraints mid-answer (seasonality, a holiday, a competitor, network effects) and watch whether your reasoning stays coherent.
Where to spend your prep time:
- Product sense and metrics (this article) — Facebook's surfaces, why it ranks on Meaningful Social Interactions, and how an engagement metric can backfire.
- Root-cause and product cases — practice decomposing a metric move by segment before diagnosing. See the Product Case section.
- Experimentation and statistics — A/B design, network interference, reading results honestly. See the A/B Test & Causal Inference section.
- SQL and communication — be fast in SQL, and rehearse the recommendation out loud: what you'd tell the PM, and why.
The through-line: find the right problem, define the right metric, defend it against gaming, and land a clear decision. That matters more than any method.
What Facebook actually is
Facebook is Meta's original social network and still one of the largest products on earth. The identity to carry into every answer: a personal and community network that connects people to friends, family, groups, and local commerce, monetized almost entirely by advertising. The loop is simple to state: people connect and post, Feed ranking surfaces the most relevant content, interactions (comments, reshares, reactions) bring people back, and that attention creates the ad inventory that funds everything.
The single most important thing to understand about Facebook's measurement history is Meaningful Social Interactions (MSI). Around 2018, Feed ranking shifted away from raw time spent toward interactions between people — comments, reshares, replies, reactions — weighted by how close the two people are (friends and family and groups over passive publisher content). The stated reasoning was that Facebook had been overvaluing time on the platform and undervaluing genuine interaction.
MSI is also Facebook's most famous lesson in metric design. Rewarding comments, reshares, and strong reactions (including "angry") ended up promoting divisive, outrage-driving content, because that is what those signals reward. It is a clean, real example of the core idea a Product DS lives by: a proxy metric optimizes for exactly what it measures, so an engagement metric must travel with an integrity guardrail.
The engagement loop
At each step a DS asks: where do people drop, which interactions are meaningful versus merely reactive, and which content builds a durable return habit versus a short outrage spike. The bottom branch matters as much as the top: a comment driven by anger still counts as an interaction, which is exactly how a well-intentioned metric can go wrong.
The sides of the market
The sides — people, advertisers, and inside Marketplace the buyers and sellers — mean most decisions carry a cross-side tension. More ad load lifts revenue but costs the person's experience; ranking that maximizes raw engagement can amplify low-quality content that erodes long-term trust; a Marketplace change that helps buyers can starve seller liquidity. Naming that trade-off is half of a strong answer.
Metrics by layer
North Star (engagement quality):
- Meaningful Social Interactions (MSI) — interactions between people, weighted by relationship. The Feed objective, chosen over time-spent to reward genuine connection.
Reach and activity:
- DAU and MAU, and the DAU/MAU stickiness ratio — how many days a monthly user actually shows up.
- Content-creation rate — the share of people who post or comment, the supply that feeds the network.
Communities and commerce:
- Groups — active vs passive members, contributions per member, group joins and retention.
- Marketplace — listings, buyer-seller conversations, message-to-seller rate, transactions (a two-sided liquidity problem).
Monetization:
- Ad revenue, ARPU, impressions, ad load, price per ad — the ads that fund almost all of it.
Integrity guardrails:
- Prevalence of violating content, reports, misinformation exposure, "see less" — the signals that keep the engagement metric honest.
The senior point in one line: an engagement metric like MSI can be gamed by outrage and low-quality content, so it never travels alone — always pair it with an integrity guardrail.
The data you'd look at daily
Most days this role works with logs of what Feed showed and what people did about it. Three tables do most of the work (example columns and rows):
1. Feed impression log — one row per post shown to one person (billions of rows a day):
| viewer_country | surface | author_tie | dwell_ms | reaction | comment | reshare | reported | exp_layer |
|---|---|---|---|---|---|---|---|---|
| US | feed | close_friend | 6,400 | like | 1 | 0 | 0 | msiV3 |
| US | feed | page | 1,100 | — | 0 | 0 | 0 | msiV3 |
| US | groups | group_member | 22,000 | love | 2 | 1 | 0 | msiV3 |
| PH | feed | page | 3,800 | angry | 4 | 2 | 1 | control |
author_tie is the column that makes this Facebook rather than any feed. MSI weights an interaction by how close the two people are, so a comment from a close friend and a comment on a publisher post are not the same event, and a table that collapses them cannot measure the thing being optimized.
The last row is the failure mode in a single line: four comments and two reshares is a strong MSI contribution, and the angry reaction plus the report say it is the kind of engagement nobody wants. Both facts come from the same row, which is why the integrity columns live next to the engagement ones rather than in a separate system.
2. Experiment scorecard — one row per experiment × metric. This is what you read to make a ship call:
| metric | control | treatment | delta | p_value | srm | guardrail |
|---|---|---|---|---|---|---|
| MSI / DAP | 8.42 | 8.79 | +4.4% | <0.001 | ok | — |
| DAU | 2.09B | 2.09B | +0.1% | 0.62 | ok | — |
| content-creation rate | 11.2% | 11.6% | +3.6% | 0.008 | ok | — |
| violating-content prevalence | 0.061% | 0.074% | +21% | 0.002 | ok | breach |
| reports / 10k impressions | 3.10 | 3.68 | +19% | <0.001 | ok | breach |
| "see less" / 10k impressions | 1.90 | 2.31 | +22% | 0.001 | ok | breach |
| ad revenue / DAP | $0.0412 | $0.0409 | −0.7% | 0.29 | ok | — |
A 4.4% MSI lift is a large win by Feed standards, and this scorecard is still a hold. Three integrity rows moved together in the wrong direction, which is the signature of a ranking change that found engagement in divisive content. The number the ranker optimizes went up because the product got worse.
3. Daily metric aggregate — one row per day × slice; what the dashboards roll up from:
| slice | dap | msi_per_dap | creation_rate | stickiness | prevalence | reports_10k | arpu |
|---|---|---|---|---|---|---|---|
| US+CA · all | 265M | 9.81 | 13.4% | 0.78 | 0.052% | 2.60 | $0.187 |
| US+CA · new (<30d) | 8.2M | 4.10 | 6.1% | 0.41 | 0.058% | 3.90 | $0.021 |
| Asia-Pacific · all | 1.32B | 7.90 | 9.8% | 0.71 | 0.068% | 3.40 | $0.019 |
| Global · Groups | 1.80B | 12.40 | 21.0% | 0.74 | 0.081% | 4.10 | $0.008 |
Reading the global row alone would hide every interesting thing here. New people interact at less than half the rate of established ones and report twice as often, because they have not built a friend graph yet. Groups is the most engaged surface and the one with the highest violating-content prevalence, which is the trade-off in one line. And ARPU differs by roughly 10x between regions, so a global average is dominated by wherever the ads sell.
What the dashboards look like at Facebook
Two views matter most each day.
The health dashboard — checked every morning by the team and leadership:
| Metric | Today | WoW | vs target | Status |
|---|---|---|---|---|
| MSI per daily person (North Star) | 8.42 | +0.9% | on track | OK |
| DAP | 2.09B | +0.3% | on track | OK |
| DAU/MAU stickiness | 0.74 | flat | on track | OK |
| Content-creation rate | 11.2% | −0.4pp | below | Watch |
| Violating-content prevalence | 0.061% | +0.006pp | below | Alert |
| Reports / 10k impressions | 3.10 | +0.21 | below | Watch |
| Ad revenue / person | $0.0412 | +0.6% | on track | OK |
Your job is to explain every Watch or Alert cell. The pairing to look at first is creation falling while reports rise: people posting less and flagging more is usually the same underlying story about what the Feed is currently rewarding.
A trend tile sits alongside it, and on this product it is always a pair:
Beyond those two views, a Facebook Product DS reads the product the way it actually runs:
- Engagement with an integrity guardrail. The primary read is an MSI-style engagement metric, always shown next to integrity signals (violating-content prevalence, reports) so a hollow or harmful lift is visible.
- Decomposed, not blended. DAU and engagement are split by country, cohort (new vs existing), surface (Feed / Groups / Marketplace / Reels), and device, because a flat global number usually hides offsetting segment moves.
- Experiments that respect the network. Because a change to one person's Feed spills over to their friends, social experiments use cluster randomization, not naive user-level splits (see the A/B section).
- Read over time. Effects are watched over a days-in window to separate a novelty spike from durable change, and against seasonality and external events.
Every read ends in a decision for a PM or leader: ship, hold, iterate, or investigate.
Quick check
What does Feed ranking optimize for, and why does that choice matter?
A ranking change ships and MSI rises 4%. What is the first thing you check before calling it a win?
Practice Questions
What would you track to know Facebook Groups is healthy?
Metrics frameworkMediumYou're the DS for Facebook Groups. Leadership asks for the metric set that tells you whether Groups is healthy. What do you track, and why each?
Before you reveal: say your answer out loud, as if you were in the real interview — get your reasoning across clearly first. There is no single correct answer: reading what the interviewer is really after and defending your own thinking is what makes an answer strong.
Anchor on active participation, not raw membership, and cover both sides of a community (creators of activity and consumers of it):
- Primary (participation): contributions per active member — posts, comments, reactions inside groups. A healthy group is one where people participate, not just belong. Raw membership is a vanity number: someone can join 40 groups and engage with none.
- Reach and growth: group joins, active groups, and the share of a user's sessions that touch a group. This tells you Groups is growing as a habit, not just a directory.
- Retention: do members stay active in a group after 30/90 days, and do groups themselves stay alive (a group with no posts is dead weight).
- Health / quality guardrail: reports, removed content, and admin load. Groups can concentrate harmful content, so a participation lift with rising reports is not a clean win.
One-line framing: track contributions per active member as the North Star for Groups, joins and active-group counts for growth, retention for durability, and reports as the guardrail.
Why did Facebook rank Feed on Meaningful Social Interactions instead of time spent?
Metric designMediumFacebook moved News Feed ranking from optimizing time spent to optimizing Meaningful Social Interactions. Why make that change, and what's the catch that a DS has to watch for?
Because time spent rewards passive consumption, and Facebook's value is people connecting with people. Optimizing raw time-on-platform pushes toward whatever keeps eyeballs stuck — autoplaying video, passive scrolling of publisher content — even when it crowds out the interactions between friends and family that make the product worth returning to. MSI re-centered ranking on comments, reshares, replies, and reactions, weighted by relationship, to reward genuine connection over passive minutes.
The catch, and the part a strong candidate raises unprompted: an interaction metric can be gamed, and it was. Weighting comments, reshares, and strong reactions rewarded engagement-bait and divisive, outrage-driving content, because anger and argument generate exactly those signals. So the metric that was meant to improve well-being ended up amplifying low-quality content. The lesson is the general one: a proxy optimizes for what it literally measures, so an engagement objective must travel with integrity guardrails (violating-content prevalence, reports, "see less") and ideally distinguish a positive interaction from a merely reactive one. Pick the metric that reflects real value, then guard it against being gamed.
Design a metric for a 'successful' Marketplace transaction
Metric designHardFacebook Marketplace wants a single metric for whether a listing led to a successful outcome. There's no clean "purchase" event like an e-commerce checkout — most deals happen in chat or in person. Design one.
The hard part is that the real outcome (a completed sale) is mostly unobservable — Marketplace is a C2C surface where money often changes hands off-platform, so you can't just count checkouts. Build the metric in layers of increasing intent:
- Contact: did the listing generate a buyer-seller conversation (message-to-seller)? This is the first observable signal of real interest.
- Serious intent: did the conversation go beyond one message — questions about price, availability, a meetup arranged? A back-and-forth is a much stronger proxy than a single "is this available?"
- Outcome proxy: the listing gets marked sold / taken down shortly after an active conversation, plus (where available) a post-transaction survey or a rating. No single one is perfect, so combine them.
So the metric is something like the rate of listings that reach a serious buyer-seller conversation and then resolve as sold, validated against survey-confirmed completions on a sample. Then state the failure modes: people mark items sold without using Marketplace, or never mark them sold at all, so the proxy needs to be calibrated against a labeled sample rather than trusted raw. The trap to avoid is stopping at "messages sent," which counts curiosity and spam as success. And because the true outcome is delayed and noisy, you read an early proxy (serious conversation) and validate it against the confirmed-sale signal over time.
Engagement is up but so is divisive content — which do you optimize?
Trade-offHardA Feed ranking change lifts Meaningful Social Interactions, but part of the lift is coming from heated, divisive threads. Leadership likes the engagement number. As the DS, which do you optimize, and how do you measure the trade-off?
Name the tension first: engagement (MSI) and integrity are in genuine conflict here, because outrage is one of the cheapest ways to manufacture comments and reshares. Optimizing MSI alone will quietly select for the content that divides people, which is a long-term loss even when the short-term number looks great.
What I'd optimize and how I'd protect the other side:
- Objective: meaningful engagement, but defined to reward positive interactions (replies between friends, helpful comments) more than reactive ones. If the metric can't distinguish, that's the first fix.
- Constraint (guardrail with a threshold): treat integrity as a hard guardrail — violating-content prevalence, reports, and "see less" must not rise. Frame it as "improve engagement subject to integrity not getting worse," not "maximize engagement."
- Measure the exchange rate: quantify how much of the MSI lift is coming from content that also raises reports or "see less." If, say, the top 5% most-divisive threads drive a third of the lift but most of the report increase, that's a bad trade you can cut without losing much engagement.
- Second-order effect: watch downstream retention and sentiment, not just next-day clicks. Outrage engagement can raise today's number and lower tomorrow's trust.
The recommendation is a net read: ship only if the engagement gain survives once you hold integrity flat, ideally by suppressing the divisive tail rather than riding it. Don't let a rising engagement number end the conversation — MSI's own history is the cautionary tale.