∧pex

Running Experiments

This guide walks you through the full experiment lifecycle: creating an experiment, setting up variants, activating it, reading results, and promoting the winner.

Before You Start

Make sure you have:

  • The tracking snippet installed on your site
  • At least one goal configured
  • A clear hypothesis about what change you expect to improve your metric

Tip

The best experiments start with a recorded belief. Even if you don't use the MCP server, writing down "I think X will improve Y" before testing keeps your team honest about assumptions.

Creating an Experiment

Navigate to Experiments

Go to Experiments in the dashboard sidebar and click New Experiment.

Name your experiment

Give it a descriptive name that includes the page and the change. Good: "Pricing page — social proof badges." Bad: "Test 1."

Set the target URL

Enter the URL of the page you're testing. For section-level targeting, you can append an anchor (e.g. https://example.com/pricing#hero).

Define your variants

Control: The current page as-is. No changes needed — this is the baseline.

Variant B: Describe or implement the change you want to test. For snippet-mode experiments, this can be a text or DOM change applied at runtime. For SDK-mode experiments, you'll implement the change in code using useApexVariant.

Choose your experiment mode

  • Snippet mode — Changes are applied via the DOM at runtime. No code changes needed. Best for text changes, headline tests, and simple visual changes.
  • SDK mode — Changes are implemented in your codebase using the useApexVariant hook. Best for structural changes, new components, or complex logic.

Configure traffic split

Set what percentage of visitors see each variant:

  • 50/50 (recommended) — Equal split, fastest to reach statistical significance
  • 80/20 — Conservative, lower risk but takes longer to get results
  • Adaptive — Apex automatically shifts traffic toward the winning variant over time

Link a goal

Select the conversion goal that defines success for this experiment. This is what Apex measures to determine the winner.

Link a belief (optional)

If you've recorded a belief about why this change should work, link it. When the experiment completes, the belief's confidence will be automatically updated.

Activating the Experiment

After creating the experiment, it starts in Draft status. Review the summary, then click Activate to start splitting traffic.

Warning

Once activated, visitors are assigned to variants immediately. Make sure your variant is ready — for SDK-mode experiments, the code must be deployed before activating.

Reading Results

While the experiment runs, the results page shows:

  • Visitors per variant — How many unique visitors saw each version
  • Conversions per variant — How many completed the goal
  • Conversion rate — Per-variant conversion rate
  • Lift — Percentage improvement of Variant B over Control
  • Confidence — Statistical confidence that the difference is real, not noise
  • Days running — How long the experiment has been active

Wait for confidence to reach 95% or higher before making a decision. Calling experiments early leads to false positives.

Promoting the Winner

Verify confidence is sufficient

Ensure the experiment has reached at least 95% confidence. Below this, the result may not be reliable.

Choose the winner

Click Promote Winner and select either Control or Variant B.

Clean up (SDK mode only)

For SDK-mode experiments, remove the useApexVariant conditional from your code. Keep only the winning variant's content and deploy.

Record the outcome

If the experiment was linked to a belief, Apex automatically updates the belief's confidence. Confirmed beliefs go up; contradicted beliefs go down.

Other Actions

  • Pause — Stop splitting traffic without losing data. All visitors see the control. Resume anytime with Activate.
  • Archive — Remove from active views while preserving all data and belief graph connections. Archived experiments still contribute to intelligence scoring.

Best Practices

  • One change per experiment. Testing multiple changes at once makes it impossible to know which one caused the result.
  • Don't peek too early. Statistical significance takes time. Checking results hourly and reacting leads to false conclusions.
  • Run for at least 7 days. Even if confidence is high after 2 days, day-of-week effects can skew results.
  • Document your learnings. The belief system is most valuable when you close the loop: believe, predict, test, update.

Next Steps

PreviousDashboardNextSetting Up Goals