Plaid — Fraud (Plaid Protect) — Product Case Questions

Role context: Data Scientist, Fraud Data team (Plaid Protect) · Est. study time: 60 min · 5 questions

How to approach product cases here

Every case follows one chain: understand what the customer and Plaid need, turn it into a data problem, pick the metric or method, name the bias and the trade-off, then make a call someone can act on.

Four facts about Plaid Protect sit under almost every case:

• Performance only means something at an operating point. Customers step up or block a fixed share of users. Recall, precision and dollars prevented should all be stated at that share.
• Only incremental fraud counts. Customers already have fraud tools. Protect's value is the fraud those tools miss.
• Labels are late, partial, and owned by customers. Blocked users never get an outcome. Fraud takes 30 to 90 days to show up. Every customer labels a little differently.
• There's an adversary. Fraudsters adapt, attacks come in waves, and fraud moves to wherever the defenses are weakest.

The metrics that matter: incremental fraud dollars prevented at the operating point, recall and precision at that point, good users stepped up and lost, score stability, and label coverage.

Traps specific to this domain:

• Believing a number computed on labels that haven't matured.
• Reading a drop in losses after a launch as the product's effect, when the customer bought it in the middle of an attack wave.
• Judging a model on the average across customers while one vertical gets worse.
• Counting a rule's catches without subtracting what other rules and the model already caught.

This role weights Measure success (precision and recall flavored, twice), plus one Diagnose, one Launch or not, and one Measure impact, since fraud products are judged on proving value without a clean experiment. It skips Forecasting, which this team touches less.