Skip to main content

A/B testing overview

A/B Tests compares complete videos through a test-specific embed and reports the observed performance of each variant.

How a test works​

A test contains 2–9 videos, exactly one control, one goal, and a traffic allocation totaling 100%. Supported goals are CTA clicks, conversions, retention, watch time, and lead capture. Each test has its own embed; individual video embeds remain unchanged.

Where to find it​

Open A/B Tests from the main navigation. The workspace shows three KPI cards—Active tests, Lift this month, and Videos promoted—plus Running, Drafts, and Completed tabs, goal filters, CSV export, and New test.

Current lifecycle statuses are Draft, Running, Paused · serving control, and Completed. The UI reports sessions, observed goal rate, lift versus control, average retention, traffic share, and a leader. It does not report confidence, significance, p-values, or underpowered/ready statuses.

Create, embed, and launch​

  1. Click New test.
  2. In Setup, name the test, choose 2–9 videos, select the control and goal, and use an equal or manual split totaling 100%.
  3. In Promotion, optionally enable Auto-promote winner and choose a delay from 1 to 336 hours.
  4. In Review & save, review the configuration and click Save test. This creates a draft; it does not launch.
  5. Open the draft, copy its test embed, place it where the video should appear, and verify the control renders.
  6. Click Launch from Drafts to begin splitting traffic.

Manage a test​

A running test can be paused, resumed, have its traffic percentages edited, or have a selected video promoted. Pausing serves the control while preserving statistics. Video membership is locked after launch; duplicate the test to create an editable draft.

Results​

Open A/B Test Analytics for a running, paused, or completed test. The route is /tests/[id]/analytics/. Choose Today, 7d, 30d, 90d, 12m, or custom dates. Results include retention, the variant table, and available country, device, operating-system, browser, and source dimensions. Previous-period comparison is intentionally unavailable for experiment analytics.

Best practices​

  • Put the test embed on the target page before launch.
  • Treat observed lift as descriptive; the product does not expose statistical significance.
  • Change live traffic deliberately because recorded observations are not reset.
  • Promote only after reviewing sessions, rates, lift, retention, and the goal you selected.

Frequently asked questions​

How many variants can a test have?

A current test compares 2–9 complete videos, with exactly one control and traffic totaling 100%.

Does the wizard launch the test?

No. Review & save creates a draft. Copy and place the test embed, then click Launch from Drafts.

How do I decide what to promote?

Review sessions, goal rate, lift versus control, retention, and traffic share. The current UI identifies an observed leader but does not expose confidence or significance.