Home > Digital > A/B testing

A/B testing


A/B testing, also known as split testing, is essentially data-driven decision making. We use the data we gain from running tests to inform the decisions we make, or to validate our original hypothesis if it’s a success.

A/B testing takes the guesswork and subjectivity out of the decision making and uses data to validate the decisions, making it a valuable part of the UX and Conversion Rate Optimisation (CRO) process.

What is A/B testing?

At its simplest, A/B testing is taking two or more versions of a design (or anything which could add value to a digital product or service, i.e. copy, images, layout, forms, etc) and testing which version performs better, Design A or Design B.

The hypothesis should specify the key metric, such as conversions, click-through rates, bounce rates, session duration, page speed, sign-ups, or share rates. After the A/B test, we review the data to refine the designs. A/B testing is just a starting point, not the final solution.

Who we work with

How to A/B test

We start A/B testing with a hypothesis – ‘doing X should achieve Y’ – along with a control version (existing design or layout) and an alternative version(s).

Anything can be A/B tested, from completely redesigning and restructuring a sales funnel to something as simple as changing an image with the aim of improving click through rate.

First, we aim to gain an understanding of the product/service, work out the pain points or problems and look for areas of opportunity to improve it. Next, we create a hypothesis to test against, and design an alternative solution(s). That solution is tested against the existing one based on a percentage split (50/50, 33/33/33, 20/80, etc).

If the test performs better, great! If the test performs worse, we review and try to understand why, and then repeat the process until we’re all happy with the results.

Looking to A/B test something?

Get in touch

A/B testing engagement on social media

As well as things like webpages, email designs and CTA formatting, we can A/B test things like heading styles and designs on social posts. Testing which gets more engagement, so you can continue using the most successful format going forward.

Testing things like this means that design decisions aren’t based on opinions. Say the marketing manager prefers one design and the CEO prefers another. The decision will be based on actual results, rather than personal preference. See our example below, which we A/B tested for our own podcast promotion posts.

A/B testing forms

We regularly design multiple versions of forms for our clients to see which performs better. The tests can either be carried out by us, or your own development team. See an example below of a form which was A/B tested for our client, DMFA. The one on the right (or below, if you’re reading this on mobile) achieved better results, so that one is now live on their site. As you can see, things like backgrounds, buttons, and visual differences can all be changed to compare one version against another.

Caroline Hazard

Marketing Manager Adidas
We really thank you for everything. Everyone loved the content you produced for our global event - it's also being used as a sell-in tool internally so it will be really helpful to the sales team. Our VP of Marketing was really impressed so let’s keep up the great work.

Joanna Pawlowicz

Marketing Manager Softbank
Dawn Creative team is a dream to work with! We partnered up on a rebranding project for our [DMFA] website, and could not be happier with the end results. The support we've received was truly exceptional - from the initial branding workshops, development phase, all the way through to the launch. It is great to see how our vision came to life, and how the team managed to take a very simple concept, transforming it into a whole brand identity that aligns with our vision & mission.

Emma Casson

Head of Product Marketing Apadmi
I just wanted to drop you all a note to say how delighted I am with the showreel!! It's absolutely brilliant! Thank you so much for all of your hard work and for pulling it all together so quickly. The pitch also went really well and I'm sure the video will have played a part in that.

Neil Richardson

Head of Business Development and Strategy Rick Shiels Media
Dawn Creative surpassed our expectations with their creativity and originality. They took the time to understand our company values, mission, and target audience, and developed a brand identity that truly resonates with our core beliefs. The innovative concepts, thoughtful designs, and attention to detail were evident in every aspect of the brand development process.

Andy Atkinson

Marketing Director Mutual Vision
We worked with Dawn to completely overhaul our company messaging, branding and website and we couldn’t be happier with the result! The people at Dawn led us through the process from start to finish and worked in a very collaborative way which made the project a real joy. We look forward to working with you on future projects.

Emma Parven

Senior Digital Marketing Manager Zen Internet
We wanted a complete web refresh with additional functionality with a planned live date within 6 weeks. I had just started working with Dawn (have worked with multiple agencies) and was sceptical at to whether they'd hit the deadline but they absolutely did. Was very impressed with the service, speed and delivery. Great job!

Don't just take our word for it

Brand, Digital
& Motion

We believe every brand is a sleeping giant just waiting to be awoken. To rise up and show the world what it's made of. It's our job to chip away at the stone and realise its potential.

You need a website that does it all. That inspires and informs, builds confidence and trust. That enhances experiences and turns casual browsers into lifetime customers.

How your brand moves says things words can’t. Every movement either raises hairs or raises eyebrows. Delights or deters. It's important to make every movement count.

Say hello

0161 711 0910


Find us

73a Hulme Hall Road,
Cheadle Hulme, Stockport,
Greater Manchester,

Thank you.

We’ll be in touch soon.