Andrew Chen

JUNE 25, 2019

As the number of homes on Airbnb scaled from around 100,000 in 2012 to over 6 million today, I led teams tackling everything from supply growth, to guest booking conversion, to marketplace quality. Once you reach scale, fraud becomes a real issue. Question: What’s the simplest way you can test a referrals offering? Here’s Why.

Retention 48

Retention User Experience Travel Marketing 48

Harvard Business Review

NOVEMBER 6, 2018

These data scientists design, define, and implement metrics, run and interpret experiments, create dashboards, draw causal inferences, and generate recommendations from modeling and measurement. If you reach this scale, a fully distributed model can make sense, but very few companies work this way.). Decision scientist: Humans.

Engineering 15

Engineering Workflows User Experience Software 15

Harvard Business Review

OCTOBER 31, 2018

Scaling Your Team’s Data Skills. A/B testing. One of the more common experiments companies use these days is the A/B test ( which is a type of randomized controlled experiment). At their most basic, these tests are a way to compare two versions of something to figure out which performs better.

A/B Testing 8

A/B Testing Manager Metrics Assets 8

Harvard Business Review

NOVEMBER 27, 2018

They work towards clear goals and they are focused on and accountable for achieving certain performance metrics. But when there is compelling evidence and big potential, the data scientist moves on to more rigorous methods like randomized controlled trials or A/B Testing, which can provide evidence of causal impact.

A/B Testing 14

A/B Testing Engineering Metrics Implementation 14

Andrew Chen

NOVEMBER 1, 2018

The reason for this is that the steps for starting and scaling a new startup share many of the same skills as investing in a new startup: 1) First, we seek to understand the existing state of customer growth – including growth loops, the quality of acquisition, engagement, churn, and monetization. There was a lot to learn.

Metrics 111

Metrics Scaling Marketing Accounts 111

Harvard Business Review

MARCH 8, 2018

To this end, they will need to move from pilot programs to large-scale efforts routinely offered across the care spectrum. These are all very different strategic needs requiring different implementation approaches and performance metrics. The challenge is huge.

Staffing 8

Staffing A/B Testing Hospitality Rollout 8

Harvard Business Review

AUGUST 21, 2015

Plugging those numbers into the formula, you’d get a price elasticity of demand of: Note that the negative is traditionally ignored and the absolute value of the number is used to interpret the price elasticity metric, as it’s the magnitude of distance from zero that matters and not whether it’s positive or negative.

Marketing 8

Marketing A/B Testing Metrics Manager 8

Harvard Business Review

NOVEMBER 2, 2012

Definitely the scale. In your work, do you only focus on the metrics side, or do you decide what sort of messaging would go out in targeted ads? So we'll do a lot of A/B testing with online ads. What are some of the key differences are between Obama's 2008 campaign and what you're seeing in this one?

A/B Testing 10

A/B Testing Marketing Education Operations 10

Andrew Chen

SEPTEMBER 4, 2018

And we cover everything from the basics of growth and defining key metrics to know, to the nuances of paid vs organic marketing and the role of network effects and more. And then we layered in payment integration and each time we did that the total growth of the company would actually accelerate which is very hard to do at scale. .

Marketing 75

Marketing Metrics Retention eCommerce 75

Andrew Chen

AUGUST 10, 2023

You can ask questions, gather data, and assess the stall to zero in on the problems that are driving the metrics downwards. There’s equivalent numbers for net revenue retention, session lengths, and lots of other metrics too. A marketplace company might look a different set of metrics.

Retention 80

Retention Metrics Scaling Marketing 80

Harvard Business Review

JUNE 30, 2015

We can benchmark how each client on the network performs against key metrics, identify high performers, and then, with a little extra research, figure out what those organizations are doing right. “Portal adoption” is in fact one of 10 “sentinel metrics” we benchmark and focus on in support of client performance.

Benchmarking 15

Benchmarking Workflows A/B Testing B2B 15

High Scalability

NOVEMBER 25, 2019

He and his colleagues spend their productive hours scaling large distributed file systems. In this article, I will share the current architecture and some of the lessons we learned scaling it along with some of the things we are looking to improve upon in the near future. How does your system evolve to meet new scaling challenges?

Scaling 21

Scaling Engineering A/B Testing Manager 21