Harvard Business Review

FEBRUARY 5, 2014

The secret to getting value from data is testing, and if you’re looking to grow your online business, implementing well-executed, consistent A/B testing is a necessity. A/B Testing in Action. A/B testing is a sure way to get to the bottom of a decision without relying on anyone’s gut instinct.

A/B Testing 13

A/B Testing Benefits Metrics User Experience 13

Vantage Circle

MARCH 13, 2019

Analyze your campaign as to maximizing what works and chucking out what doesn’t and acting accordingly because these metrics have deep implication on the final conversion rate. One way of achieving this is through A/B testing. In A/B testing, you take your ad and modify it to create a second version of the original.

Advertising 40

Advertising A/B Testing Media Implementation 40

15Five

JANUARY 25, 2024

The objective is what the business or employee wants to achieve, and the key results are measurable metrics that show how well the business or individual is accomplishing the objectives. Implementation: Companies can implement forced ranking by establishing clear performance criteria and using objective metrics to evaluate employees.

Implementation 52

Implementation Scaling Metrics A/B Testing 52

AIHR

NOVEMBER 5, 2018

This was measured through A/B testing. Business performance would increase if these three metrics would go up. The company was a large Dutch FMCG retailer that used people analytics to analyze the effects of training. The analysts found that training shop personnel had a positive impact on the shop’s financial performance.

Staffing 102

Staffing Benchmarking Transportation A/B Testing 102

Harvard Business Review

JUNE 28, 2017

One of the most common methods, particularly in online settings, is A/B testing. What Is A/B Testing? A/B testing, at its most basic, is a way to compare two versions of something to figure out which performs better. A/B testing, in its current form, came into existence in the 1990s.

A/B Testing 8

A/B Testing Metrics Software Sales 8

Andrew Chen

JANUARY 23, 2017

We can do so much more these days by integrating into platforms, and by tracking, and our sophistication in metrics like viral factor and cohort analysis, but the same general trust is there. This is the realm of A/B testing, funnel optimization, viral loop construction, building viral content, and creating sticky products.

Media 100

Media A/B Testing Marketing Advertising 100

Andrew Chen

DECEMBER 10, 2018

Another note is that this is closely related to, and complimentary, to this deck: The red flags and magic numbers that investors look for in your startup’s metrics. If the below deck is the macro view of how I’m looking at markets, industries, and technologies, then the metrics deck gives my POV on how to diligence each company.

Media 100

Media Advertising Scaling Marketing 100

Get Lighthouse

DECEMBER 15, 2022

Is it publishing your revenue metrics so the whole world can see it? So they let the data do the talking and ran an A/B test. Instead, they lean into engaging their people to help solve the company's biggest challenges. A word like "Transparency" can feel loaded these days.

Manager 98

Manager A/B Testing Engineering Recruitment 98

Walk Me

JULY 18, 2022

Supply managers with metrics of change progress. Consistency and metrics are two of the most potent tools in change management. Goals must be aware of resource limitations such as budget and physical tools used and clear a timeline including metrics or Key Performance Indicators (KPI) to record progress. How they communicate.

Change Management 59

Change Management Manager Benchmarking Strategic Planning 59

Andrew Chen

NOVEMBER 1, 2021

Roadmap : Testing a series of more scalable growth channels, showing that if one network can be built, then multiple networks might be launched as well. Typically organic “pull” from the market combined with a scalable channel with reasonable metrics (CAC/LTV, or viral factor, or otherwise) going into in-app acquisition funnels.

Scaling 82

Scaling Marketing Recruitment B2B 82

Andrew Chen

JUNE 25, 2019

The biggest gains in optimization don’t come from brute-force A/B tests, but from trying to understand the real barriers to people using your product. Retention metrics roundup of articles and links by Andrew Chen. Question: Is this a channel your competition hasn’t tried yet? Tactic #17: Optimize conversion. Here’s Why.

Retention 111

Retention User Experience Travel Marketing 111

Harvard Business Review

JANUARY 30, 2015

Only a patient reader who persists to the last paragraph will see that Sharp ultimately admits that New England’s spectacular performance on the metric could be explained in any number of ways, including legitimate ones like perfecting ball security techniques or practicing prevention. above the NFL team average).

A/B Testing 13

A/B Testing Metrics Marketing Sales 13

Walk Me

AUGUST 24, 2022

This action involves ongoing improvement to reduce costs and increase revenue to allow more significant profits, all directed by qualitative and quantitative metrics and analysis. Profit management impacts organizational development, operations, and stakeholder value management. Organizational knowledge is used more efficiently in this way.

Organizational Development 52

Organizational Development Organizational Change Human Resources Change Management 52

Andrew Chen

DECEMBER 10, 2018

Another note is that this is closely related to, and complimentary, to this deck: The red flags and magic numbers that investors look for in your startup’s metrics. If the below deck is the macro view of how I’m looking at markets, industries, and technologies, then the metrics deck gives my POV on how to diligence each company.

Media 40

Media Advertising Scaling Marketing 40

Harvard Business Review

OCTOBER 31, 2018

A/B testing. One of the more common experiments companies use these days is the A/B test ( which is a type of randomized controlled experiment). At their most basic, these tests are a way to compare two versions of something to figure out which performs better. You can learn more about A/B tests here.

A/B Testing 8

A/B Testing Manager Metrics Assets 8

Harvard Business Review

JULY 17, 2013

New school marketers, typically those closer to social marketing channels targeted at Millennials, are telling me they''re having a hard time getting their projects funded internally because old school marketers in the C-suite don''t understand the new context and metrics driving social marketing.

Marketing 13

Marketing A/B Testing Assets Advertising 13

Andrew Chen

JUNE 25, 2019

The biggest gains in optimization don’t come from brute-force A/B tests, but from trying to understand the real barriers to people using your product. Retention metrics roundup of articles and links by Andrew Chen. Question: Is this a channel your competition hasn’t tried yet? Tactic #17: Optimize conversion. Here’s Why.

Retention 48

Retention User Experience Travel Marketing 48

Harvard Business Review

NOVEMBER 6, 2018

These data scientists design, define, and implement metrics, run and interpret experiments, create dashboards, draw causal inferences, and generate recommendations from modeling and measurement. Data science for machines: here the consumers of the output are computers which consume data in the form of training data, models, and algorithms.

Engineering 15

Engineering Workflows User Experience Software 15

Harvard Business Review

FEBRUARY 19, 2016

These include everything from analytics tools like Adobe Analytics and Google Analytics, to A/B testing tools like Oracle’s Maxymiser and Optimizely. This task is accomplished through the selection of key performance indicators, and the development of reports on these metrics for consumption across the organization.

Manager 15

Manager A/B Testing Engineering SaaS 15

Harvard Business Review

NOVEMBER 27, 2018

They work towards clear goals and they are focused on and accountable for achieving certain performance metrics. But when there is compelling evidence and big potential, the data scientist moves on to more rigorous methods like randomized controlled trials or A/B Testing, which can provide evidence of causal impact.

A/B Testing 14

A/B Testing Engineering Metrics Implementation 14

Andrew Chen

NOVEMBER 1, 2018

Of course, as an investor you can’t run A/B tests or analyze results directly, but you can form hypotheses, ideate, and apply the same type of thinking. Thinking about new user experience, engagement metrics, and other important concepts. The problem is that the Growth Accounting Framework provides for lagging metrics.

Metrics 111

Metrics Scaling Marketing Accounts 111

Andrew Chen

DECEMBER 26, 2018

The red flags and magic numbers that investors look for in your startup’s metrics – 80 slide deck included! The deck unpacks a lot of different topics: How “growth accounting” metrics are great, but are lagging indicators. DAU/MAU is an important metric to measure engagement, but here’s where it fails.

Metrics 80

Metrics A/B Testing Transportation Marketing 80

Harvard Business Review

MARCH 8, 2018

These are all very different strategic needs requiring different implementation approaches and performance metrics. When formal research trials aren’t practical, providers will need to embrace A/B testing and learn to more rapidly evaluate the effectiveness and scalability of these technologies.

Staffing 8

Staffing A/B Testing Rollout Hospitality 8

Harvard Business Review

AUGUST 21, 2015

Plugging those numbers into the formula, you’d get a price elasticity of demand of: Note that the negative is traditionally ignored and the absolute value of the number is used to interpret the price elasticity metric, as it’s the magnitude of distance from zero that matters and not whether it’s positive or negative.

Marketing 8

Marketing A/B Testing Metrics Manager 8

Harvard Business Review

NOVEMBER 2, 2012

In your work, do you only focus on the metrics side, or do you decide what sort of messaging would go out in targeted ads? There's a combination of features and attributes that make the candidate or the product appealing to them, and being able to just test different ways of presenting an ad and seeing how people respond is very valuable.

A/B Testing 11

A/B Testing Marketing Education Operations 11

Andrew Chen

DECEMBER 26, 2018

The red flags and magic numbers that investors look for in your startup’s metrics – 80 slide deck included! The deck unpacks a lot of different topics: How “growth accounting” metrics are great, but are lagging indicators. DAU/MAU is an important metric to measure engagement, but here’s where it fails.

Metrics 40

Metrics A/B Testing Transportation Marketing 40

Andrew Chen

SEPTEMBER 4, 2018

And we cover everything from the basics of growth and defining key metrics to know, to the nuances of paid vs organic marketing and the role of network effects and more. Sometimes aren’t these metrics ways to figure that out or is this all when you have product-market fit… like is there a pre- and a post- difference between these?

Marketing 75

Marketing Metrics Retention eCommerce 75

Andrew Chen

AUGUST 10, 2023

You can ask questions, gather data, and assess the stall to zero in on the problems that are driving the metrics downwards. There’s equivalent numbers for net revenue retention, session lengths, and lots of other metrics too. A marketplace company might look a different set of metrics.

Retention 80

Retention Metrics Scaling Marketing 80

Harvard Business Review

JUNE 30, 2015

We can benchmark how each client on the network performs against key metrics, identify high performers, and then, with a little extra research, figure out what those organizations are doing right. “Portal adoption” is in fact one of 10 “sentinel metrics” we benchmark and focus on in support of client performance.

Benchmarking 15

Benchmarking Workflows A/B Testing B2B 15

Harvard Business Review

MAY 28, 2015

Citation metrics are widely used in faculty evaluations and routinely come up in tenure reviews. By far the dominant measure of impact is citations: how often a piece is cited in subsequent works. An advantage of this measure is that it is easily accessible: Google Scholar and Web of Knowledge are just a click away.

Manager 14

Manager A/B Testing Government Staffing 14

High Scalability

NOVEMBER 25, 2019

Combination of unit, functional, integration and performance tests. We also generate quite a bit of internal application metrics using a home grown framework. Do you do A/B testing? Yes, we use feature flags to do A/B testing. How you analyze performance? How Do You Decide What Features To Add/Keep?

Scaling 21

Scaling Engineering A/B Testing Manager 21

Andrew Chen

NOVEMBER 13, 2018

There’s a long laundry list of skills that are critical, but not often considered core to the product: adtech integrations, signup funnel A/B testing, optimizing notification delivery, testing price points, testing cohort curves, etc. Impact is how much the metric moves as a result of the change.

A/B Testing 110

A/B Testing Engineering User Experience Marketing 110