Andrew Chen

DECEMBER 10, 2018

It has tracking codes so you can see how well it circulates, and you A/B test the whole thing to make sure it’s highly optimized to be viral and spread. The important, core concept here is simple: When there are new technologies and platforms hitting scale… … and products tap in pre-existing consumer motivations.

Media 100

Media Advertising Scaling Marketing 100

Harvard Business Review

JUNE 28, 2017

One of the most common methods, particularly in online settings, is A/B testing. What Is A/B Testing? A/B testing, at its most basic, is a way to compare two versions of something to figure out which performs better. A/B testing, in its current form, came into existence in the 1990s.

A/B Testing 8

A/B Testing Metrics Software Sales 8

Get Lighthouse

DECEMBER 15, 2022

So they let the data do the talking and ran an A/B test. Treating employees like teammates instead of family can be scary if you've never thought that way, but if you're scaling your business or making significant changes, you may have no choice. In the case of Netflix, that included admitting when you're wrong, even publicly.

Manager 98

Manager A/B Testing Engineering Recruitment 98

Andrew Chen

JUNE 25, 2019

As the number of homes on Airbnb scaled from around 100,000 in 2012 to over 6 million today, I led teams tackling everything from supply growth, to guest booking conversion, to marketplace quality. Once you reach scale, fraud becomes a real issue. Question: What’s the simplest way you can test a referrals offering?

Retention 48

Retention User Experience Travel Marketing 48

Andrew Chen

DECEMBER 10, 2018

It has tracking codes so you can see how well it circulates, and you A/B test the whole thing to make sure it’s highly optimized to be viral and spread. The important, core concept here is simple: When there are new technologies and platforms hitting scale… … and products tap in pre-existing consumer motivations.

Media 40

Media Advertising Scaling Marketing 40

Harvard Business Review

AUGUST 21, 2012

Leaders will have to role model the use of these technologies, explain how to use them to drive value, observe success stories and help them to scale up to the rest of the enterprise. gamification), A/B testing , and mobile deployment can be applied in the enterprise, just as they are used in the consumer space.

Media 18

Media Workflows A/B Testing B2B 18

Harvard Business Review

AUGUST 5, 2014

Doing an A/B test comparing whether people like a new product design more than an old one? The person who runs that test might come back providing average ratings for each design showing that B’s average rating is significantly better than A’s. Well, “average” usually implies testing based on means. How can they?

A/B Testing 14

A/B Testing Software Scaling Sales 14

Harvard Business Review

AUGUST 12, 2015

My guess is that I was part of a Maps traffic A/B test, because of the lack of other people commenting on this interface change and the fact that I didn’t need to download an update in order to change it. ” I was surprised I got a response at all, but even more amazed at the personalization of the response.

User Experience 14

User Experience A/B Testing Scaling Sales 14

Harvard Business Review

JUNE 28, 2016

APIs are a technology that allows firms to interact and share information with other firms at an unprecedented scale. Provides a low-cost, real-time network to test ideas. A/B testing allows for rapid comparison of alternative features and functions. Crowdfunding sites are another avenue to test product ideas.

A/B Testing 13

A/B Testing Travel Engineering Human Resources 13

Harvard Business Review

NOVEMBER 6, 2018

Depending on the engineering background of these data scientists, these work products are either deployed directly to the production system, or if they are prototypes they are handed off to software engineers to help implement, optimize and scale them. The elusive full stack data scientists do exist, though they are hard to find. Who to hire.

Engineering 15

Engineering Workflows User Experience Software 15

Harvard Business Review

OCTOBER 31, 2018

Scaling Your Team’s Data Skills. A/B testing. One of the more common experiments companies use these days is the A/B test ( which is a type of randomized controlled experiment). At their most basic, these tests are a way to compare two versions of something to figure out which performs better.

A/B Testing 8

A/B Testing Manager Metrics Assets 8

Harvard Business Review

DECEMBER 9, 2015

It’s been several years now since this experience debuted and yet few have replicated a similar experience at this scale, despite its powerful appeal: the giver feels empowered for giving the item, and the taker feels the thrill of getting a surprise from a friend or stranger. This isn’t groundbreaking thinking.

Marketing 13

Marketing A/B Testing Advertising Scaling 13

Harvard Business Review

DECEMBER 29, 2016

We then tested our conclusions in a large-scale field experiment with the Booth School of Business’s alumni fundraising campaign. Ideally, this would involve A/B testing that varies the default amount, along with a no-default control group. How can you determine the best default for a specific campaign?

A/B Testing 12

A/B Testing Nonprofit Compensation Scaling 12

Harvard Business Review

APRIL 30, 2013

We A/B test new product features. We test our content, including data visualizations. Should engagement be represented by a two-color scale, or one? In our final visualization, we moved to a two-color scale to highlight relative differences in engagement between cities. Ten failures for one success.

A/B Testing 10

A/B Testing Scaling Advertising 10

Harvard Business Review

JULY 15, 2014

To test new software before release, managers of large projects use elaborate test procedures run by teams of “quality assurance” professionals. This traditional testing takes weeks or months, and still misses errors. Continuous integration and automated testing is important for all modern, large scale software development.

Software 9

Software A/B Testing Scaling Operations 9

Harvard Business Review

NOVEMBER 1, 2018

To be sure, these are not the only tools you’ll need — for example, I haven’t included A/B testing, understanding variation, or visualization here. Scaling Your Team’s Data Skills. Nor is my intent to make people experts. Insight Center. Sponsored by Splunk. Help your employees be more data-savvy.

Sales 15

Sales Sales A/B Testing Manager 15

Harvard Business Review

NOVEMBER 27, 2018

But when there is compelling evidence and big potential, the data scientist moves on to more rigorous methods like randomized controlled trials or A/B Testing, which can provide evidence of causal impact. This way the data scientist focuses more on driving business value through testing and learning, and less on technology.

A/B Testing 14

A/B Testing Engineering Metrics Implementation 14

Harvard Business Review

MARCH 30, 2016

If this sorting of participants sounds like A/B testing , that’s because they’re similar. A/B can be a randomized controlled experiment, assuming you’ve controlled factors and randomized subjects, but not all randomized controlled experiments are A/B tests. So let’s put it all together.

A/B Testing 13

A/B Testing Accounts Manager Assets 13

Harvard Business Review

NOVEMBER 23, 2018

AI-supported A/B testing in fast cycles can determine best price points at a SKU level to achieve margin and volume goals. And successes can be scaled up quickly. Iterative machine learning. Start small, fail fast. The vast number of digital sales tools can be overwhelming.

Sales 14

Sales Sales B2B Accounts 14

Harvard Business Review

MARCH 8, 2018

To this end, they will need to move from pilot programs to large-scale efforts routinely offered across the care spectrum. When formal research trials aren’t practical, providers will need to embrace A/B testing and learn to more rapidly evaluate the effectiveness and scalability of these technologies.

Staffing 8

Staffing A/B Testing Rollout Hospitality 8

Harvard Business Review

NOVEMBER 2, 2012

Definitely the scale. There's a combination of features and attributes that make the candidate or the product appealing to them, and being able to just test different ways of presenting an ad and seeing how people respond is very valuable. So we'll do a lot of A/B testing with online ads.

A/B Testing 11

A/B Testing Marketing Education Operations 11

Harvard Business Review

AUGUST 21, 2015

Rather, they send out questionnaires, run focus groups, or perform small-scale experiments in certain markets, to give them a sense of what would happen if they changed their price. And to be truly certain, you’d have to change your price multiple times to see what would happen at each price point.

Marketing 8

Marketing A/B Testing Metrics Manager 8

Andrew Chen

NOVEMBER 1, 2018

The reason for this is that the steps for starting and scaling a new startup share many of the same skills as investing in a new startup: 1) First, we seek to understand the existing state of customer growth – including growth loops, the quality of acquisition, engagement, churn, and monetization.

Metrics 111

Metrics Scaling Marketing Accounts 111

Andrew Chen

AUGUST 10, 2023

But unfortunately, new customer growth channels tend to be fairly linear — most marketing channels don’t scale up as the user base scales up, and even the channels that do, like viral marketing, eventually saturate and slow down. Finding and scaling marketing channels is typically pretty easy.

Retention 80

Retention Metrics Scaling Marketing 80

Andrew Chen

SEPTEMBER 4, 2018

And then we layered in payment integration and each time we did that the total growth of the company would actually accelerate which is very hard to do at scale. . And then there’s this dynamics on how does it scale over time, CAC tends to go up, LTV tends to go down. ” “How is it gonna scale?”

Marketing 75

Marketing Metrics Retention eCommerce 75

Harvard Business Review

JUNE 30, 2015

Second, we performed a host of A/B tests to refine interactions on behalf of our customers to engage their patients, including better office workflows to enroll patients in the portal and direct-to-patient campaigns to encourage adoption. Practices of all sizes can be successful at patient engagement.

Benchmarking 15

Benchmarking Workflows A/B Testing B2B 15

Harvard Business Review

MARCH 2, 2016

The speed and scale on which this is occurring helps us recognize that we are not in a cyclical downturn as corporations attempt to compensate for the disruptive impact of digital technology. Simply stated, it’s harder to make money by working or creating value when the scales tip too far in favor of investors and shareholders.

Assets 8

Assets Balance Sheet Operations A/B Testing 8

High Scalability

NOVEMBER 25, 2019

He and his colleagues spend their productive hours scaling large distributed file systems. In this article, I will share the current architecture and some of the lessons we learned scaling it along with some of the things we are looking to improve upon in the near future. How does your system evolve to meet new scaling challenges?

Scaling 21

Scaling Engineering A/B Testing Manager 21