Pinterest — Trust & Safety — Product Sense & Metrics

Role context: Data Scientist, Trust & Safety (prevalence measurement) · Est. study time: 45 min · 5 practice questions

How to prepare for this role

This is a measurement science role. It is not product data science, not growth, and only incidentally machine learning. The work is to design the foundations for measuring the prevalence of unsafe content across the platform. You are being hired to build the instrument, not to use it.

The day-to-day is: design sampling frameworks that find rare violations without labeling the whole internet, build pipelines that assemble what a user asked for and what the system showed them into one labelable record, turn written safety policies into LLM prompts, calibrate how good the labelers are, and publish a daily number with a confidence interval that leadership and regulators both read. And one duty that is easy to miss: advocating for decision quality before a metric reaches executive leadership. Saying "this number isn't ready, and here's the arithmetic" is part of the job.

Where to spend your prep time

• Sampling theory. This is the rare data science role where it is the main subject, not a footnote.
• Why a rare event breaks your intuition about precision, recall, and agreement rates.
• Estimators under unequal sampling probabilities, and what reweighting buys you.
• Label quality as a statistical object: sensitivity, false positive rate, drift.
• Explaining a confidence interval to an executive who wants a single number.

What Is Trust & Safety at Pinterest

Pinterest is a visual search and discovery platform serving over 500 million monthly active users on their journey from inspiration to action. It makes money from ads. People search, browse, and save images (Pins) that mostly link out to the web.

For this role, the framing that matters is: Pinterest sells inspiration, and inspiration is a promise about how it feels to be here. Trust and Safety is not a cost centre bolted onto the product. It is the thing being sold. A platform people do not feel safe on has no product left.

The work starts as a measurement problem, not an enforcement problem. You cannot manage what you cannot measure, and violations are rare enough that measuring them naively is either wildly imprecise or unaffordable. That is why sampling, not moderation, is the first thing this role owns.

The stakes here are unusually literal. The policy areas in scope include adult content, self-harm, harassment, and misinformation. This is not a platform where a bad recommendation costs a click. And the audience for your number is not only your team: Pinterest publishes transparency reports and, as a large platform in the EU, produces Digital Services Act risk assessments. The number this role produces is a public and regulatory artifact.

Now the part that defines the hardest work here, and that most candidates read past. The pipelines this role builds aggregate Pinner-generated queries, system responses, and recommended Pin images into a single labelable record, treating complex multi-component interactions as distinct measurement units.

Read that again. The unit of measurement is not a Pin. It is the interaction: what the Pinner asked for, what the system said back, and what it recommended. That is much harder than labeling images, because the harm often lives in the pairing rather than the item. An image can be completely innocuous on its own and harmful as the response to a particular query. Label Pins alone and you miss that entire class of harm by construction.

The product is a mature platform with a new measurement surface. Pinterest has moderated images for years with classifiers and human review queues. What is new is measuring the safety of generated and recommended experiences as a unit, every day, at 500-million-user scale, using large language models as the labelers.