Skip to main content

A/B testing

JAICP provides the A/B testing tool with which you can create A/B tests in your script.

tip
A/B testing is a research method in which different script versions are shown to different clients. This way, you can determine which version leaves the greatest impact and leads to more conversions.

Compare received results and optimize the script on the insights.

Creating experiment​

To create an experiment, go to Analytics > A/B testing. Click Create experiment and fill out the following fields:

  • Code — the experiment name. It will be used in the script.
  • Experiment description — the experiment description.
  • A−E branches — the branch description.

Click Create experiment — the experiment has been activated. Now go to Editor and call the experiment from the script.

Calling experiment from the script​

tip
Use the $analytics.joinExperiment method to call the experiment.

The method accepts a string with the experiment code. The method randomly returns a branch letter.

$analytics.joinExperiment("Code"); // => "B"
caution
If the client re-enters the experiment within the same session, the method will return alreadyJoined.

Re-entering experiment​

If within the same session the client re-enters the state with an active experiment, $analytics.joinExperiment will return the alreadyJoined string.

$analytics.joinExperiment("Code"); // => "alreadyJoined"
tip
Provide a check for re-entering the experiment in the appropriate state.

Finishing experiment​

To finish the experiment, go to Analytics > A/B testing and click the Finish button next to the experiment you need.

tip
In the state with the experiment, consider the case when the experiment is finished.

Viewing results​

When the client enters one of the experiment branches, $analytics.joinExperiment automatically calls the $analytics.setSessionData method, so the experiment result will appear in the session report.

The column name will conform to the experiment code, while the column value will conform to the branch letter.

How to use​

In the example below, we check which of the experiment options A or B will lead to the highest conversion.

state: Partnership
    intent: /partnership
    a: Join the JAICP partnership program and earn interest on creating bots.
    script:
        // Calling the “Partnership” experiment.
        var branch = $analytics.joinExperiment("Partnership");
        // Script option that the client will follow if they get into branch A.
        if (branch == 'A') {
            $reactions.answer("Write your email to receive detailed information about the program.");
        // Script option that the client will follow if they get into branch B.
        } else if (branch == 'B') {
            $response.replies = $response.replies || [];
            $response.replies.push ({
                "type": "inlineButtons",
                "buttons": [{
                    "text": "Learn more about the partnership program",
                    "url": "https://just-ai.com/partnerskaya-programma"

                }]
            });
        // Script option that the client will follow when they re-enter the experiment or when the experiment is finished.
        } else {
            $reactions.answer("Write your request at client@just-ai.com.");
        }