What is an A/B test and how can I create one on the Tagalys dashboard?

Simultaneously publish multiple merchandising strategies by creating A/B tests for your collections on the Tagalys dashboard

Merchants can simultaneously publish multiple merchandising strategies by creating multiple variants for their collections on the Tagalys dashboard. Users can determine the duration of the test and detailed reports will be available for each variant using which merchants can compare, analyze and make informed decisions.

Note: This feature is currently only available to customers who are on Shopify. 

Follow the steps mentioned below to create an A/B test:

Step 1: Navigate to the A/B testing menu under 'Collections':


Step 2: Click on 'New' at the top right. Users can name the test from the pop-up box and select the required Collection | Category. Once this is done, click on 'Create & Configure'.


Users can also create an A/B test directly from the required collection's merchandising edit page by clicking on the icon highlighted below:


Note: By default, An A/B test will have 2 variants, users can create a maximum of 3 variants (including the current version). 
Step 3: Users can configure the test based on their requirements using the following options:

• • Adding a third variant: Users can add a third variant by clicking the '+Variant' button.
    
  • Merchandising each variant: Users can merchandise each variant based on their preferences by clicking on the edit icon next to the required variant.
    
  • Splitting traffic: Users can configure the percentage of traffic that will be directed to each variant once the A/B test is live. You can either push the toggle up and down to adjust the percentage or enter the exact percentage. We recommend directing a minimum of 10% of the traffic towards each variant. 
    
    Note: There can be a traffic split deviation of ±2% for every 10,000 visitors

Step 4: Scheduling the A/B test:

• • Once all the configuration is done, users can schedule the test by clicking on the 'Create Schedule' button. 
    
  • From the pop-up box, select the start and end date of the test and click on 'Save'. The earliest an A/B test can be live is one hour from the time of scheduling. 
    
  • Once it is scheduled, you will notice the test status will have changed from 'Unscheduled' to 'Scheduled'. 
    

Note: In order for a user to view and edit an A/B test, they will require edit access to collections and to view the reports of the test they will require reporting access. In case you face any issues with regard to access/permissions, contact your store admin. 

An A/B test's status can be one of the following:

• • Unscheduled: This indicates that an A/B test is yet to be scheduled.
  • Scheduled: This indicates that an A/B test has been scheduled to go live at a future date.
  • Preparing: The variants will be created on the Shopify platform approximately 30 minutes before the A/B test is scheduled to go live.
  • Ready to Go-live: This indicates that the variants are created and the set-up for the A/B test is completed and is ready to go live. 
  • Live: This indicates that an A/B test is currently ongoing. Please note that once a test is live, users will not be able to edit/configure the test at all, for example, making merchandising changes to a variant or editing the traffic split to each variant. The only actionable options once a test is live will be:
    
    • To view the merchandising of the variants.
    • To end the test.
    • In order to make changes to an A/B test that is already live, users will need to end the test, make the required changes and schedule it once again.
  • Completed.

• • Interrupted: To know the reasons why an A/B test can be interrupted, click here.

Once an A/B test has been completed, users can view the reports of each of the variants from the test by clicking on the 'View Report' icon from the A/B testing window. 


Users can also select the required date range for which they want to view the reports by clicking on the calendar icon at the top right.

Once the reports have been analyzed and a decision has been made as to which variant should be the primary live version moving forward, users have the option to set the required variant live, directly from the A/B test window.