What can e-commerce sites do to optimize the customer experience without getting lost in bazillion single tests over time?

This is a complex topic. Testing is not only about a single item or a static process variant of what is already in place. A testing-strategy is nothing like once-and-done, it has to be an ongoing process. Websites with millions of visits have to serve millions of intentions from all kind of perspectives. Some things can be tested rather simply. For example, how to layout a product page with text, using different images, colours and call-to-actions. But this comes only at the end. Before purchasing, a customer has to arrive at a relevant product detail page which meets the customer’s expectation: with the right content at the right time. This puts product overview pages – or any other product listings – into primary scope.

Unique shops vs strong competition

Some shops are providing certain products exclusively, or are unique for specific products ranges. Think of fan merchandise sellers, luxury brand distributors or other aggregators who own a market, or other specialized shops or online-services. Here – and probably only here -, a customer would rather tolerate some usability issues before giving up or changing its mind.

However, a sub-optimal strategy for product listing, recommendations and up-sales can leave a lot of money on the table and consequently has a negative impact on customer retention and reputation. Compensating for a lack of customer loyalty through additional traffic channels leads to persistently high – additional – costs. Relying only on a unique market position can be very risky.

Then there are shops which are in a strong competitive field because the same products and services are available elsewhere, too. These online shops have to put even more attention to a smooth and really good user experience. They can’t afford to lose a customer’s attention for even just a few seconds. They have to provide relevant products right away.

What is relevant and for whom?

Regardless, if being a unique shop or being challenged by many competitors: What are relevant products for a single customer? This cannot be averaged or determined by more-or-less static overviews.

Customers are different and not all alike, and even the same customer can re-visit a shop in a new situation, e.g. mobile vs desktop, at home or in the tube, workdays or weekends, before or after purchase, etc. A new situation can change everything in terms of what is relevant for this very visit.

Displaying best-seller lists or manually sorted top-lists for everyone ignores all non-average customers (so, pretty much everyone) and can only inspire some few customers by chance.

However, such and similar product listing “strategies” are not uncommon. This must not be the case. Changing more-or-less static product lists into a dynamic and individual customer experience can be done by using only the existing website and e-commerce assets.

Dynamic testing with relevance for everyone

There’s no point in trying out multiple static approaches for a huge amount of user segments which are manually or half-manually derived from certain data points, e.g. visit frequency, landing pages, basket values, etc. – it will always be inaccurate and it’ll require to rebuild the model endlessly which will make it overcomplex very soon.

On the flipside, segmenting customers only by backend data points like e.g. purchase history, lifetime value, order items, shipping address, etc. ignores the customer behavior and patterns before the customer makes a decision.

The process of (half)manually segmenting customers along single data points or along some persona-definitions, as described above, is old fashioned, too slow, not accurate enough and oversimplifying the issue.

What’s needed is identifying patterns which are actionable, which means they are significantly different from other patterns. Identifying those is math, not art or marketing (as marketing is often considered half and half science and art).

With some amount of historical data (web- and backend-data, and optionally additional data sources), a performant real-time cluster and an API-connection for the e-commerce site individual dynamic personalization is possible.

Now, real performance tests to increase uplifts can begin on a large scale.

Testing individual strategies for millions – without millions of test scenarios

Does that sound like a paradox? It is not. Instead of testing a static scenario for a pre-defined type of customer better make data-driven decisions:

The historical web- and purchase-data can be clustered to identify statistically significant segments of any kind. This dataset is getting updated day by day so that any new customer-behavior – and thus new significances – are always kept into account.

Then, each website visit is mapped in real-time against the historical data in order to find peer-groups with similar data patterns (“statistical siblings” aka “users like you”). The computing process with ODOSCOPE’s real-time cluster takes only 20 milliseconds. While a page is rendering the decision engine provides relevant personalization based on statistical significance. The most likely conversion drivers are displayed to a user in real time and fully data-driven.

Running multiple tests at once

The data governance is now sorted and the data mapping is updated with every day’s new data batch. These pre-settings allow quite some powerful data-driven experiments because every visit’s meta-data will be connected in real-time to its own historical peer group which allows one test-setting but with different results per individual visit.

Starting with a relatively simple – but most promising – test will gain uplifts almost (and often) immediately. It is recommended to start with a 50/50 test-vs-control split-test. The situation-based personalization will provide the most relevant information on top which results in higher conversions. Once this fully automated test does not only show a positive trend but significantly better results than the control group, the test can be either enlarged to an 80/20 or 90/10 split-test (why leaving money on the table?), or more tests can be added in parallel. For example, 25% control, 25% test-A (which was proven to perform better), and two more testing variants at each 25%.

The more experience is collected the more the tests will be fine-tuned and adjusted accordingly, always fully automated and data-driven. However, seasonality and marketing or sales campaigns are impacting the customer behavior, too, and will provide additional learnings and allow to even improve the running scenarios. The data will show what’s on and what to do.

Testing to improve business goals directly

Such testing strategies can now directly improve business goals, and still be fully individualized in order to provide a great personal customer experience. Running a test is measured by uplifts of business goals, not only behavioral KPIs for example:

  • more sales from paid channels
  • higher basket values
  • faster sale of last stocks
  • more email-up-sales
  • effective sample order rates
  • etc.

Modern testing strategies are automated, fully data driven and based on owned 1st party data.

ODOSCOPE can provide all of this. It is transparent, fully configurable, based on statistical significance, and it comes with an extremely fast and flexible dashboard for any analysis and report.

Ready for uplift? Let’s talk!

Let's talk