Skip to content
Let's Talk

Why A/B testing is crucial for app store optimisation (ASO)

With over 2 million apps available today, it’s becoming increasingly difficult to hit a high number of views and downloads. If your app hasn’t been featured by Apple or Google Play’s editors, the only way to reach users is through organic discovery.


App Store Optimisation (ASO) is fast becoming a hot topic among brands with their own app, or with plans to launch one in future. It is the process of optimising mobile apps to rank higher in the app store’s search results. The higher your app ranks, the more visible you will become to your potential customers.

In short, ASO helps marketers achieve their goals by driving traffic to your app’s store page. Even if your app is ranked first for all of its major keywords, strategically converting views into downloads can be difficult. A/B testing helps make this possible.


What is A/B Testing?

A/B testing (also known as split/bucket testing) is the process of comparing two versions of a web page or app against one another, to determine which performs better.

It’s essentially an experiment - two or more variants of a page are shown to users at random. They act naturally, unaware of the testing, allowing you to identify which performs better for a given conversion goal. Well-constructed A/B tests have been known to increase conversions by 30%.

Although A/B testing predominantly focuses on improving conversion rates, its benefits far surpass conversion optimisation. It provides valuable insights into marketing activity, evaluates audience segments and engagement/user experience, and measures the efficiency of new traffic channels. In addition, publishers are able to find out whether they need to change their product positioning.

Planning A/B testing is integral to achieving accurate results. Testing eliminates the guesswork from your optimisation strategy. As a result, you derive data-informed decisions that shift business conversations from ‘we think’ to ‘we know’.

The Importance of the App Store within the User Journey

The app store is ultimately where page views are converted into downloads.

Visitors of the crowded app store decide whether or not to download an app a couple of seconds after browsing the app store page. This offers a very brief time period for you to tell the app’s ‘story’ and explain its benefits.

When publishers advertise apps, a large sum of money is wasted if the user goes from the campaign link to the app store page without proceeding to download - the advertiser still pays for the click.

In order to optimise media expenditure and the app marketing budget, marketers should test app store creatives prior to launching a killer content marketing campaign. A good conversion rate could help your app become one of the store’s high flyers!

Why it’s Important to Accommodate your Target Persona

A target persona for your app is vital. They’re your ideal prospects and customers that you’re trying to attract with your marketing efforts. Your app would not be successful without them!

A/B testing helps to determine what your users want. Optimisation of the icon, screenshots and video preview will give you a better chance of higher conversions, a greater volume of organic downloads and a better return on your investment in user acquisition.

Lead generation

To help put things into perspective, imagine a user scrolling through dozens of apps on the store’s search page. The icon is often the first element of your product that users see; as such, you want it to be innovative and attractive. An A/B test can help to determine which icon your target persona would prefer, aiding your conversation rate.

Once you understand their buyer behaviour and learn about their needs, you’ll determine how best to engage them with your app!

The Differences Between the Apple App Store and Google Play Store

The Apple App and Google Play stores have the same purpose - to allow users to search, download and install an app. Although many of the elements on both platforms are the same, aspects of their layouts differ.

Here are some examples:

  • The Google Play app description generates  twice as many taps on the ‘Read More’ button as Apple's App Store. This button is also more prominent in Google Play. The newer iPhone sizes allocate two lines for each app description, before citing ‘Read More’.

  • According to CitrusBits, in Google Play, the image gallery is situated beneath the fold, making users 27% less likely to scroll through. On the other hand, the iOS image gallery is the most prominent part of the screen, ensuring all visitors interact with gallery items.


  • CitrusBits claim visitors are 20% more likely to play videos in Google Play without finishing. In Apple’s App Store, an estimated 30% of visitors are expected to complete the video, due to Apple’s enforcement of a 15-30sec video length time.


  • Apple’s App Store does not offer a push app installation system - iPad and iPhone users have to do it manually for both their devices. iOS users have to install the entire instant and beta testing app for a quick check-in. On the other hand, Google users are provided instant push app installation.


  • The categorisation system of Google is more advanced than that of the App Store. However, Apple’s store is based on the simple phrase ‘There is an app for that’. Recently, Apple has started to replicate Google’s store as a result.


  • The searching app on Google is very user-friendly. Apple have now incorporated Google+ into its search system, allowing users to read reviews of the application. This makes it easy for users to download or skip any app after reading the reviews and recommendations of people they already know.


  • Google Play is developer-friendly and includes A/B testing. However, being developer-friendly does not guarantee better quality. Google’s A/B testing does lack certain features that are important for marketers, such as only running experiments on published apps, only being able to run one at a time and no metrics to see how visitors interact with the page.

In light of all this, here’s a few things to remember:

  • The first two lines in your description must be engaging.
  • Your image gallery needs to be creative and intriguing to stand out from the crowd.
  • All video content must be engaging and user-friendly.
  • Your ASO has to be accurate to make you more visible to potential customers.
  • Don’t rely solely on Google’s A/B testing.

It’s important to tailor your approach to each platform to maximise your reach. Though Apple and Google’s stores have their own characteristics, you must ensure you account for the differences between the two when testing users' priorities.

The Apple App Store is image-focused, whilst Google Play focuses on both video and featured graphics. As a developer, you need to explore the requirements of your target persona with A/B testing for each store.

How to Conduct Research for ASO A/B Tests

A/B Test

Leading companies no longer A/B test random changes. They’d rather use research and create strong hypotheses based on the collected data. A solid hypothesis is a cornerstone of any A/B testing success.

Here’s a list of core research activities that can help prepare you for this very important phase:

  • Compare your screenshots, featured graphic, videos and icon designs with the styles of other apps that appear in your keyword search results. Remember, your app must stand out from the crowd to gain downloads.


  • Focus on the keywords with the highest volume and for which your app ranks best for - these will drive the most organic downloads.


  • Compare the messaging in your captions, videos and descriptions with that of your competitors. You should be creating messages that give you the competitive advantage.


  • Think of any statements you could make that your competitors do not. Analyse the advantages of your competitors and how you can overcome them to convince users to download your app.


  • Read your app’s reviews to identify the language and key features for users and integrate them into your messaging, marking any selling points that don’t appeal to them.


  • Try to focus on features considered most useful by users and inspect the most recent reviews for better results.


  • Review your app usage data to identify which features or configurations are most used or most purchased. Use these insights to increase your app listing’s appeal to new users.

Tracking Metrics

Special A/B testing software, such as SplitMetrics, will track visitor behaviour. Ultimately conversion is the most important marketing metric, though others are also effective. Other metrics that may help to improve your ASO strategy include:

  • Bounce rate
  • Direct installs vs. explored installs
  • Average session time

Providing you find a clear winner, you’ll be able to implement the results in the app store immediately. Tracking the conversion changes after the upload of the optimised product page elements is a given.

If you’re consistent with your A/B testing activity, the results will speak for themselves. However, your conversion won’t change overnight - keep tracking your metrics for success!

Inbound Marketing Technology Companies