Andrew Chen

DECEMBER 26, 2018

Conservation of Intent: The hidden reason why A/B tests aren’t as effective as they look. Everyone’s had the frustrating experience of running an A/B test, seeing a big lift, closing it out, and expecting the top level metrics to move a lot. But they don’t.

Metrics 80

Metrics A/B Testing Transportation Marketing 80

Andrew Chen

AUGUST 8, 2022

These larger headline numbers always tested much better on A/B tests, whether in email marketing or banner form, and while it might feel like the reward becomes unattainable, it’s possible to create a second or third or fourth tier to go along with the big headline number. The Payback.

A/B Testing 94

A/B Testing B2B Onboarding ROI 94

UVA Darden

JANUARY 12, 2023

The star analyst is now a data scientist, a private equity associate is tasked with reinventing a company’s IT instead of engineering its finances, and the marketer is now a “growth hacker” running A/B tests.

A/B Testing 111

A/B Testing Digital Transformation Engineering Onboarding 111

Harvard Business Review

APRIL 18, 2023

Testing your approach with a randomized sample of employees can help you find out what works and what doesn’t.

A/B Testing 20

A/B Testing Inclusion Diversity 20

Harvard Business Review

FEBRUARY 5, 2014

The secret to getting value from data is testing, and if you’re looking to grow your online business, implementing well-executed, consistent A/B testing is a necessity. A/B Testing in Action. A/B testing is a sure way to get to the bottom of a decision without relying on anyone’s gut instinct.

A/B Testing 13

A/B Testing Benefits Metrics User Experience 13

Andrew Chen

AUGUST 23, 2021

People solve this with ideas like wait list hacks, A/B tested invite flows, contact importers, “people you may know” algos. Without the right users there at the beginning, a user-generated content product would inherently be too shallow on content. Low engagement would lead to more low engagement.

A/B Testing 96

A/B Testing eCommerce Media Travel 96

Harvard Business Review

JUNE 23, 2021

It can be a valuable tool to prevent customer churn — if used correctly.

A/B Testing 19

A/B Testing Marketing 19

Andrew Chen

FEBRUARY 26, 2024

I had a superpower in my domain knowledge of CAC, A/B testing, funnel optimization, lead gen, etc, and began to merge all of this thinking with consumer products. I had previously worked in online ads, with customers from WSJ, CBS, MySpace, etc, and had even gotten a patent filed on ad targeting (yes, US7747676B1).

A/B Testing 123

A/B Testing Retention Metrics Engineering 123

Vantage Circle

MARCH 13, 2019

One way of achieving this is through A/B testing. A/B testing is basically a way of testing two variants of the same variable by testing the subject's response to both the variants and statistically determining the more effective of the two. Final Touches.

Advertising 40

Advertising A/B Testing Media Implementation 40

Alison Green

OCTOBER 19, 2022

The other option would be for you and your fellow team members to push back on your PI’s preferred strategy, possibly suggesting some A/B testing so you get real data on what does and doesn’t work … but it doesn’t sound like you want to take that on, which is completely reasonable. Putting video game playing on your resume.

Media 105

Media Insurance A/B Testing Benefits 105

Andrew Chen

OCTOBER 17, 2017

Optimization : A/B testing growth levers – from email/SMS/push copy – to when/how to reach out. Use cases : Understanding use cases and how to up-sell and cross-sell the stickiest ones. Offers/promotions : Using offers and content throughout the calendar cycle to engage.

Workflows 0

Workflows Marketing Retention A/B Testing 0

Andrew Chen

JUNE 21, 2017

He’ll ask about the sample sizes on your A/B tests. As it’s been widely reported, TK cares about the details. He’ll look at your graphs and ask about week-to-week deviations. I’ve worked on projects where he wants to give final approval for the final mockups of flows.

Transportation 110

Transportation A/B Testing Scaling Energy 110

AIHR

OCTOBER 12, 2021

Before sending out any messages to potential candidates, it is a good idea to do A/B testing internally. You can compare your response rate to the previous performance but also to benchmarks. The average response rate for an initial sourcing email should be between 30-50%.

Metrics 98

Metrics Recruitment A/B Testing Benchmarking 98

SME Strategy

MARCH 6, 2023

For instance, in Marketing, data is being used to calculate ROI on marketing campaigns, or come up with new pricing strategies based on A/B testing of campaigns which helps marketing and managers bring in more revenue, and stay ahead of the competition.

Strategic Planning 96

Strategic Planning Manager Implementation Software 96

Harvard Business Review

JUNE 28, 2017

One of the most common methods, particularly in online settings, is A/B testing. What Is A/B Testing? A/B testing, at its most basic, is a way to compare two versions of something to figure out which performs better. A/B testing, in its current form, came into existence in the 1990s.

A/B Testing 8

A/B Testing Metrics Software Sales 8

Zenefits

AUGUST 25, 2022

A/B testing. A/B testing is a great way to test marketing campaigns and ensure that your organization takes advantage of the most effective means of attracting customers or users. Let’s dig into 10 of the top trends that go along with modern growth marketing efforts.

Marketing 52

Marketing A/B Testing Media Onboarding 52

AIHR

JULY 20, 2021

You may want to use A-B testing here, where you communicate the training program to employees using different messages to measure the effectiveness of different images and email subject headers used when presenting the course. This is simply to measure how many students enroll in the course.

Metrics 96

Metrics ROI Sales Sales 96

Harvard Business Review

OCTOBER 3, 2024

A/B testing by Instacart suggests that telling customers about stocking times can increase spending and decrease the proportion of items replaced or refunded.

A/B Testing 24

A/B Testing Operations 24

AIHR

JULY 31, 2023

A/B testing everything will be incredibly valuable. It can also tell you if certain questions in your application process deter candidates from filling them out or completing them so you can eliminate these questions. You may also discover that images and videos highlighting company culture could encourage applications.

Recruitment 98

Recruitment Onboarding Diversity Human Resources 98

AIHR

NOVEMBER 5, 2018

This was measured through A/B testing. The company was a large Dutch FMCG retailer that used people analytics to analyze the effects of training. The analysts found that training shop personnel had a positive impact on the shop’s financial performance. In the first year alone, the ROI of their training program was 400%.

Staffing 102

Staffing Benchmarking Transportation A/B Testing 102

Andrew Chen

DECEMBER 26, 2018

Conservation of Intent: The hidden reason why A/B tests aren’t as effective as they look. Everyone’s had the frustrating experience of running an A/B test, seeing a big lift, closing it out, and expecting the top level metrics to move a lot. But they don’t.

Metrics 40

Metrics A/B Testing Transportation Marketing 40

AIHR

MARCH 29, 2022

They found that training shop personnel positively impacted the shop’s financial performance, which they measured through A/B testing. A large Dutch FMCG retailer used people analytics to analyze the effectiveness of training. The ROI of their training program was 400% in the first year. Preventing turnover.

Benefits 111

Benefits Recruitment Retention Compensation 111

Andrew Chen

JANUARY 23, 2017

This is the realm of A/B testing, funnel optimization, viral loop construction, building viral content, and creating sticky products. Execute thoughtfully and iteratively, so that you are learning about your product and platform. If you are prepared, and have mastered the tactics of the trade, then this is when you can shine.

Media 100

Media A/B Testing Marketing Advertising 100

15Five

JANUARY 25, 2024

Here’s a quick example of an OKR for a marketing department: Objective : Increase conversion rates across all channels Key results : Increase the conversion rate of key landing pages from 10% to 20% Improve email conversion rate from 2%-4% Conduct A/B testing on product pages with the goal of running two statistically significant improvements Benefits: (..)

Implementation 52

Implementation Scaling Metrics A/B Testing 52

Get Lighthouse

DECEMBER 15, 2022

So they let the data do the talking and ran an A/B test. In the case of Netflix, that included admitting when you're wrong, even publicly. In a heated debate between the CEO, Reed Hastings, and another executive, they could not come to an agreement.

Manager 98

Manager A/B Testing Engineering Recruitment 98

Andrew Chen

AUGUST 10, 2023

I use tons of A/B testing and experiments in messaging to make this happen. Should the user be “forced” to activate in any way, by adding required signup steps? … and on and on For new user flows, I try to get more users that hit the landing page to ultimately become activated users.

Retention 80

Retention Metrics Scaling Marketing 80

Andrew Chen

JUNE 25, 2019

The biggest gains in optimization don’t come from brute-force A/B tests, but from trying to understand the real barriers to people using your product. Question: Is this a channel your competition hasn’t tried yet? Tactic #17: Optimize conversion. All conversion optimization should start with user research.

Retention 111

Retention User Experience Travel Marketing 111

Andrew Chen

DECEMBER 10, 2018

It has tracking codes so you can see how well it circulates, and you A/B test the whole thing to make sure it’s highly optimized to be viral and spread. Of course, we’ve improved the whole thing based on the latest tech. It integrates into Facebook Messenger and your email addressbook.

Media 100

Media Advertising Scaling Marketing 100

Vantage Circle

JANUARY 31, 2023

So they ran an A/B test, and the results were in Tom’s favor. This value is best exemplified in the story Patty McCord recalls in her book “ Powerful.” ” In the story, she recalls how CEO Reed Hastings was once involved in a heated debate with another executive Tom.

Diversity 104

Diversity Inclusion Accounts Operations 104

Andrew Chen

NOVEMBER 1, 2018

Of course, as an investor you can’t run A/B tests or analyze results directly, but you can form hypotheses, ideate, and apply the same type of thinking. 2) Then, to identify potential upside based learnings from within the company as well as across benchmarks from across industry.

Metrics 111

Metrics Scaling Marketing Accounts 111

Andrew Chen

NOVEMBER 1, 2021

There were projects to help with A/B testing (Morpheus!), There were teams working referral programs, landing pages, adtech integrations, funnel optimizations, activation sequences over email, retention and loyalty programs, as well as all the underlying infrastructure.

Scaling 82

Scaling Marketing Recruitment B2B 82

Harvard Business Review

DECEMBER 30, 2016

Traditional operations research process optimization, for example, seldom embraces A/B testing or incorporates customer recommendation engines. ” When fulfillment processes become platforms, opportunities to segment customers, A/B test bundled offers and make targeted recommendations explode.

A/B Testing 15

A/B Testing User Experience Software Compensation 15

Harvard Business Review

JANUARY 30, 2015

Reverse causation investigations have the opposite structure from A/B tests , in which we vary known causes, and observe how the variations affect an outcome. Note: Good A/B test construction can help you see most likely causes; bad A/B test construction creates its own set of problems.).

A/B Testing 13

A/B Testing Metrics Marketing Sales 13

Walk Me

AUGUST 24, 2022

Netflix is one example of such a company, using A/B tests and sharing data collection findings with all aspects of the organization to use continuous change to foster a learning culture. Organizational knowledge is used more efficiently in this way. Trans-organizational Change.

Organizational Development 52

Organizational Development Organizational Change Human Resources Change Management 52

Harvard Business Review

OCTOBER 16, 2012

A/B testing is very much part of the culture of Silicon Valley web start-ups, but hasn't yet made its way to corporate America. For the CDO, this would mean ensuring that processes are being instrumented and that all necessary data is being captured and stored. This, however, is only the first step.

A/B Testing 16

A/B Testing Sales Sales Banking 16

Harvard Business Review

MARCH 10, 2014

Cartoons can make A/B testing easier — and more fun. Draw several different cartoons and captions to test out which ones get to the heart of the customer need and convey value in the most meaningful way. This means we understand, remember and retains images (and their meaning) better than words.

A/B Testing 15

A/B Testing Strategic Planning Advertising Benefits 15

Harvard Business Review

AUGUST 21, 2012

gamification), A/B testing , and mobile deployment can be applied in the enterprise, just as they are used in the consumer space. How do you create applications that are as compelling to corporate employees as they are to those same people in their personal lives? Techniques such as self-reinforcing behavior loops (e.g.

Media 18

Media Workflows A/B Testing B2B 18