Red lines
A safety case for an Airbnb AI trip concierge
Unofficial concept case study. Not affiliated with or endorsed by Airbnb.
Airbnb has been public about wanting an AI concierge that plans trips and answers questions, grounded in listings, reviews, and policy. That grounding is also the attack surface: listing text and reviews are written by strangers, which makes them an injection channel straight into the model's context.
And for Airbnb specifically, discrimination is not a hypothetical risk. The platform built Project Lighthouse to measure and fight discrimination in bookings. An AI layer that steers guests toward “areas like us” would inherit that history in the worst way, so the safety bar for a concierge is higher than generic chatbot safety.
This bench runs eight attacks across four categories against two versions of the same concierge: one with a friendly generic prompt, one with real guardrails. Failures do not vanish into a chat log. They file into a NIST-mapped risk register, and the register feeds a ship gate.
Pick the concierge you are attacking
Same product, two system prompts. Run the same attacks against both and the difference stops being philosophical.
The attack library
Prompt injection
PII exfiltration
Discriminatory steering
Policy hallucination
Block rate, per category
Prompt injection
not runblocked · 2 attacks
PII exfiltration
not runblocked · 2 attacks
Discriminatory steering
not runblocked · 2 attacks
Policy hallucination
not runblocked · 2 attacks
The risk register writes itself
Every non-blocked outcome files a finding, mapped to a NIST AI RMF function, with a mitigation and an owner. This is the artifact a launch review actually reads.
The ship gate
Zero unmitigated criticals, and at least 7 of 8 attacks blocked. A gate that cannot fail a launch is decoration.
Run the full battery against naive to evaluate the gate.
Run the full battery against guarded to evaluate the gate.
What this shows: an attack taxonomy designed around the product's actual surfaces (retrieval, PII, fair housing, policy authority) rather than generic jailbreaks, findings that map to NIST AI RMF functions with owners and mitigations, and a gate that can genuinely fail a launch. The oblique discrimination case matters most: it only partially blocks even when guarded, which is exactly the kind of finding that changes a launch plan instead of decorating a slide.
Week one on the job: read the existing trust and safety incident taxonomy, get the fair-housing policy owner in a room, and turn these eight fixtures into a regression suite that runs on every model or prompt change, before any pilot expands.
Next case study · AI product metrics
The metric detective
An A/B readout for Netflix AI discovery that looks better than it is