A/B testing has become an important feature for many e-commerce sites, allowing changes to be tested with less risk. Kodmyran Commerce has support both built-in and the ability to integrate with external tools.
Kodmyran Commerce contains several primitives for doing A/B testing without the use of external tools, however we recommend using Google Analytics Experiments for tracking the results.
The built-in AB tests are limited to only pure A/B tests (one variation), and there is no visual design tool unlike some of the external tools. Instead the built-in method allows A/B testing of non-graphical changes, such as connecting freight and payment options to the test. You can also code for whatever scenario you wish in the template. Of course you can code any graphical change you want in this case, but it has to be written as code.
The built-in functionality works by first checking whether the user has already been subjected to the test (cookie based), and if so the result is immediately known based upon the cookie and the user is presented with the same test/variation. If the user does not have a cookie, indicating that this is a new test, the platform will create a random number between 0 and 100 and store this in the users cookie.
For each test you set a threshold, a number between 0 and 100. If the number generated in the cookie is above the threshold the corresponding test is taken, otherwise it is not. This means the threshold can be changed after the test has started if so desired.
If you need significant changes between your A and B versions you can install an AB test selection module, this module can control which template is shown to a particuliar user based upon advanced code. It is typically used to choose between the original template and a new variation.
To report the result of the A/B test, and to be able to slice and dice the results you can enable Google Analytics Experiments integration, you do this by enabling Google Analytics as per usual recommendations, and then turning on Google Analytics Experiments support under the A/B testing tab in the administration portal. Lastly you need to add a Google Analytics Experiments ID to the proper A/B test to ensure the result can be accredited to the proper test.
During development of the A/B test you will likely have to force a particular outcome in order to debug your new functionality. This can be done by instructing the platform that you wish to take a test to either 100% or 0% using the control commands:
E.g.:
http://www.myshop.com/?cmd=settestperm&testid=1
In addition to the internal tools you can use external tools such as Visual Website Optimizer (VWO), Optimizely and Google Optimize. Normally one can use the recommended installation procedures provided by each tool, but special precautions must be taken inside the cashier pages which frequently turn out to be hard to A/B test using external tools. This is due to their very dynamic nature and that some tools have very strict requirements on external libraries such as jQuery.
In general whenever using an external tools pay extra attention to verifying that the cashier still operates properly, you will have to test every combination of payment methods, freight methods, countries etc.