How to Get a Yes on Growth Work You Can't Attribute

In 2008–2009, Airbnb’s founders were scrapping for every listing and every guest. The company’s big break came from a guerrilla tactic that is now part startup lore: the founders wrote scripts that automatically posted Airbnb listings to Craigslist, tapping into an enormous pool of demand the platform hadn’t won through conventional marketing. That move—described in Leigh Gallagher’s book The Airbnb Story and in multiple interviews with the founders—was equal parts creative engineering, opportunism, and brutal focus on a theory of change: get listings in front of millions of Craigslist users -> generate inquiries -> convert to bookings -> scale supply and trust. It worked, fast. What makes the Airbnb example relevant for anyone who’s ever tried to win approval for “growth work you can’t attribute” is this: the tactic produced clear outcomes, but it also created attribution ambiguity. Some bookings came directly via the Craigslist posts, some came via organic discovery later, and some users entered the funnel through channels Airbnb didn’t control or couldn’t instrument. Yet the founders convinced investors and early hires to double down. Why? They framed a crisp causal story, picked decision-ready metrics, and layered evidence in ways that made the logic compelling—even when strict last-click attribution failed. I’ll use Airbnb’s scrape-as-growth origin as a running touchstone here, paired with other well-documented cases, to show how you can win a “yes” for growth experiments that are inherently hard to attribute. You’ll get concrete methods, real-world examples, and a practical playbook you can tailor to your organization.

Start with a crisp theory of change and decision metric

Whenever I think about the power of a clear theory of change, I return to two canonical examples: Hotmail (the 1996 "PS: Get your free email at Hotmail" viral tag) and Dropbox (the 2008 referral program). Both are simple, documented, and instructive.

• Hotmail's logic: Founders Sabeer Bhatia and Jack Smith added a one-line signature to outgoing emails. Hypothesis: recipients who see the free email tagline will click and sign up. Decision metric: new signups per email sent (and viral coefficient). Result: exponential user growth that made the service a household name within months.
• Dropbox's referral program: Rather than optimize expensive ad channels, Dropbox tested giving both referrer and referee storage space. Hypothesis: incentivized sharing increases signups and retention. Decision metric: referral-driven signups and long-term retention of referred users. Result: millions of signups and a clear, measurable viral loop.

These are meal-simple hypotheses with direct causal chains: stimulus -> user action -> business outcome. That simplicity is their superpower—and it's exactly what you need when attribution is messy.

Here’s how to translate that into your proposal:

1. Write a one-sentence hypothesis: I will do X to cause Y within Z timeframe.
2. Map the causal chain: Identify immediate user behavior you expect to change, intermediate engagement signals, and the ultimate business outcome.
3. Pick one decision-ready metric: Choose the metric that will determine whether to continue, stop or scale. Make it simple and operational (e.g., % increase in activated users within 14 days), and list 1–3 proxies in case the primary metric is noisy.
4. Define the minimum detectable effect (MDE): Be explicit about the smallest lift that would change decisions. Small lifts may still be valuable; declare that threshold upfront.

My reaction: In too many proposals I’ve read, teams bury the metric in slides and float fluffy outcome statements. The Hotmail and Dropbox stories remind me that the people who win are those who can say, with conviction and math, “Here’s what success looks like, and here’s how we’ll know.” That clarity converts uncertainty into a decision problem—and that’s what stakeholders can sign up for.

Design for causality: proxy metrics, holdouts and advanced inference

When attribution paths are broken, you must design experiments that isolate causality. The gold standard is randomized control; the practical repertoire includes holdouts, geo experiments, synthetic controls, uplift models and Bayesian inference. These are not academic toys—they have real history in big firms.

Consider Microsoft and Ron Kohavi’s work on online controlled experiments. Kohavi’s papers and talks from Microsoft and later at Airbnb describe how randomized experiments (A/B tests) are used to detect subtle effects in complex systems. The team’s emphasis on trustable experimentation is now industry standard for tech companies that care about causal claims.

Another instructive public example is Facebook’s conversion lift and holdout studies. Advertisers who can’t track conversions due to cookie restrictions are often shown lift results from randomized holdouts: a fraction of users are prevented from seeing an ad and used as a control group to estimate incremental conversions. Facebook’s public documentation and measurement products demonstrate the practical value of randomized holdouts when attribution channels are broken.

Here’s how to operationalize causality in your proposals:

• Prefer randomization where possible: Use user-level or session-level randomization for feature rollouts, messaging variants, and experiments with immediate touchpoints.
• Use geographic or cohort holdouts: If you can’t randomize individuals (e.g., store listing changes that affect everyone), hold out entire regions or cohorts so you have a control baseline.
• Apply synthetic control for business-level interventions: When a true control isn’t available, build a synthetic control from historical and peer data (Abadie, Diamond & Hainmueller’s approach). This method has been used in policy and increasingly in business to infer impact when randomized trials can’t be run.
• Employ uplift models: For targeted spend, predict who is persuadable and randomize within high- and low-uplift segments to estimate incremental effect.
• Use Bayesian approaches: Bayesian inference naturally encodes prior uncertainty and lets you update beliefs as data arrives—handy for showing decision-makers probabilities instead of single-point estimates.

My reaction: The technical toolset is not the hard part—stakeholder alignment is. When I see teams lean on these methods, the conversation shifts from “Did it cause X?” to “How confident are we, and what’s our next controlled step?” That shift matters more than the p-value.

Layer evidence: quantitative + qualitative triangulation

Rarely will a single test convince a skeptical exec. The companies that succeed at un-attributable growth build convergent evidence from multiple sources. Booking.com and Airbnb are textbook cases here.

Booking.com

Airbnb

Practical ways to triangulate:

1. Cohort LTV and funnel decomposition: Track cohorts over time to see whether short-term spikes translate into durable value. Break funnels into micro-conversions to spot where gains originate.
2. Store listing and CRO experiments with holdouts: Run A/B tests on app stores or landing pages, but keep a control subset of users (or regions) so you can estimate incremental lift beyond general upward trends.
3. Search visibility and keyword ranking shifts: Use tools (Search Console, third-party SEO tools) to correlate visibility deltas with traffic and conversions; pair this with timestamped site changes for attribution windows.
4. Qualitative research and customer interviews: Use moderated interviews and diaries to understand causality claims—why did users choose you after seeing X? Booking.com’s UX labs and Airbnb’s host interviews are instructive models.
5. Brand-lift and perception surveys: When direct conversion tracking is missing, run lightweight surveys to estimate brand awareness and intent shifts; Facebook and Google’s brand-lift products formalize this approach.

My reaction: I’ve watched teams get seduced by a single uplift in conversion and then be blindsided when retention reveals the change was superficial. Convergent evidence mitigates that risk—numbers alone can be misleading, and qualitative insights often explain the “why” behind the numbers in ways that convince stakeholders.

Translate uncertainty into a persuasive stakeholder narrative

Data without narrative is rarely persuasive. The Obama 2012 campaign provides a powerful model of turning probabilistic insights into funded action: the campaign’s data team ran countless experiments and used predictive models, but they always packaged findings as clear decision frameworks for field organizers and fundraisers. They communicated risk and reward, prioritized tests with clear thresholds, and created governance to stop or scale initiatives quickly.

Here are techniques to turn uncertainty into buy-in:

• Risk/reward framing: Don’t just show expected lift—illustrate upside, downside, and the probability of each. Use scenarios (best-case, base-case, worst-case) with estimated financials.
• Pre-mortems: Adopt the pre-mortem exercise (from psychologist Gary Klein) to surface failure modes upfront. This builds stakeholder trust because you’ve considered what could go wrong and how to mitigate it.
• Visualize uncertainty: Use bands, probability distributions, and decision trees rather than single numbers. Decision-makers understand ranges more than point estimates.
• Governance and milestone-based funding: Propose staged commitments: small initial investment, predefined checkpoints, and go/no-go criteria. This converts a risky big ask into a sequence of manageable bets.
• Offer fallbacks: If the experiment underperforms, present an alternative plan or rollback approach to reduce perceived risk.

My reaction: Stakeholders are human. They fear regret. When you combine transparent modeling with pre-mortems and staged funding, you remove the fear factor. It’s the difference between asking for a leap of faith and offering a structured expedition with a map, contingency plans, and pulleys.

Operational playbooks for SEO, ASO and CRO where attribution fails

When channels are inherently noisy or opaque—search engines, app stores, and conversion funnels—you need channel-specific operational playbooks that bake in defensibility.

Let’s look at three channels and some real-world inspirations:

• SEO: Companies such as HubSpot and The New York Times have shown that content and technical SEO investments pay off over time, but attribution is delayed and confounded by seasonality and algorithm updates. Practical tactics:
  • Create time-stamped experiments (e.g., publish X pieces targeted at Y keywords vs. control topics); measure organic traffic and conversions by cohort.
  • Monitor crawl logs and SERP visibility shifts; use a set of diagnostic pages as a control bucket.
  • Instrument UTM and landing page variants for new content to link on-site behavior to organic referrals.
• ASO (App Store Optimization): Google Play and Apple App Store now offer store listing experiments and product page optimization. Developers like Peak and many indie studios have documented how iterative artwork and text changes can move conversion rates.
  • Run A/B tests on alternate creatives and screenshots for a subset of traffic; hold out a region as control where you keep the old listing.
  • Track keyword visibility and correlate with organic installs using store analytics and third-party tools.
  • Assign incremental KPIs (e.g., “organic installs per 1k impressions of store listing”) and instrument events post-install to approximate LTV.
• CRO (Conversion Rate Optimization): Booking.com and Expedia are well-known for disciplined CRO. They combine:
  • rigorous A/B testing with statistical safeguards
  • session recordings and heatmaps to generate hypotheses
  • holdout segmentation so that sitewide trends don’t contaminate test results
  Operationally:
  • Set up templated experiment briefs that include hypothesis, target segment, MDE, and time horizon.
  • Prioritize experiments by impact x ease, and always maintain an explicit control cohort.
  • Instrument downstream events (e.g., cancellations, returns) to ensure you’re not optimizing a vanity metric.

Additional tooling and instrumentation:

1. Logging and event taxonomy: Standardize event names, user IDs, and timestamps so you can stitch signals across channels.
2. Analytics warehouse: Push raw events to a warehouse to enable custom uplift and synthetic-control analyses later.
3. Dashboards with provenance: Always show what data source and filters produce a metric—transparency builds trust.

My reaction: When attribution is fuzzy, operational rigor replaces attribution with defensibility. Booking.com didn’t get to be experimental by accident—they treated experiments as production systems. That’s the level of discipline you need if you’re asking others to fund work that won’t yield clean last-click proofs.

I’ve laid out practical choices and real company cases because I believe that getting buy-in for hard-to-attribute growth is less about persuading people you’re right and more about making your proposal robust to doubt. - Start with a crisp, testable theory of change and a single decision metric that stakeholders can agree on. - Design experiments that prioritize causality—randomization, holdouts, synthetic controls and uplift models are not optional if you want credible claims. - Build a mosaic of evidence: cohort LTV, funnel decomposition, store-listing holdouts, qualitative interviews, and brand-lift surveys make a stronger case together than any one signal alone. - Translate probabilistic findings into governance and narrative: pre-mortems, scenario planning, staged funding and clear go/no-go criteria will make decision-makers comfortable betting. - For channels where attribution is messy, adopt operational playbooks and instrumentation that make incremental measurement defensible. The stories of Hotmail, Dropbox, Airbnb, Booking.com and the Obama campaign are not magic—they are examples of disciplined thinking, clever experimentation, and the humility to present uncertainty honestly. My personal take is this: when you can show that your work has a clear causal logic, measurable rules for success and a slowing mechanism for failure, you’re not asking for blind faith—you’re asking to run a controlled, accountable experiment. That is a pitch most leaders are willing to back. I’d love to hear where you see the biggest friction in your organization: is it at the hypothesis stage, the measurement stage, or the governance stage? Which of these playbook elements would make it easier for you to get a yes?