AB testing in PushPushGo application

AB tests are one of the most useful tools in a marketer's arsenal. Their main goal is to enable the testing of different versions of campaigns, messages, and ads, to choose the one with the best results in a small group of receivers. 

The AB tests can be also used in push campaigns. Let's look at the AB testing process in the PushPushGo application. We’ve recently introduced some changes to make the process easier and better and more comprehensible. 

AB testing in PushPushGo app

By conducting an AB test of your web push campaign you can verify which content, title, image, or any other settings perform better and give you a better CTR result. 

Our application allows you to create between 2 to 10 variants of your message and test it on a predefined group of your subscribers. While creating the variants, you'll be able to set up

• title

• content

• images

To set the most engaging combinations and boost your CTR rates. 

You can also apply other push settings, like icons, require interaction features, action buttons, etc. The app also uses options you define in the Project Settings menu. 

Choose a test group

After preparing test variants of your push notifications, you have to select testing groups. It's important to choose a representative portion of your subscribers to receive the most reliable results. In the PushPushGo application, you can select receivers for each message either by typing the number of subscribers or using a slider. 

The test audience has to be at least 1% and up to 50% of the targeted group. You need at least one subscriber for each variant to make the test work. 

To facilitate the split, we prepared a helpful pie chart representing the portions of your base.

Example:

You have 100 subscribers, and you decide to have two variants in the AB test. 

If you choose a 20% split part, it means that from all of your 100 subscribers, 20% will receive test variants: which equals 20 subscribers. 

These 20 subscribers will be split into two groups, one for each of 2 variants, so each variant will be sent to 10 subscribers.

Timing your messages

After you've prepared your test variants and selected test groups, the test is ready for sending. There are two options: you can send the AB test right away or schedule the time of sending. Additionally, if there's something you'd like to change or edit before the start, your test can be saved as a draft and wait for a better time. 

If you choose the "Send now" option: 

• your test will be sent immediately, 

• the test will last for 1 hour

• the winner will be selected automatically

• the winning push will be displayed for default 72 hours or a period previously defined. 

When choosing "Schedule AB test" you'll need to provide: 

• start date

• end date (between 1-72 hours from start date)

• winner selection mode

• time and expiration date for the winner option

And the winner is...

When it comes to picking the winner variant, you can choose between automatic and manual choices. 

If you opt for the former, your work is done as soon as the test is sent. The app will choose the winner automatically, based on the CTR results, and send it to the defined base. The only exception is the case of two variants receiving very similar test results. If the difference between the winner and the following variant is 1% or less in CTR, you'll be notified by email. In this situation, you'll have to manually pick your winner to proceed with the campaign. 

Test your web push campaigns

You can find information and statistics on all your AB tests in the Campaigns section of the PushPushGo app. The list is clear and easy to navigate. Thanks to the layout and the search options you can easily track the results and gain new insights into your campaigns. 

You can find more detailed information on our AB tests in PushPushGo gitbook. 

If you have any additional questions, contact us at [email protected].