--- id: "tools/experiments-v1/configuring-experiments-v1" title: "Configuring Experiments" description: "Before setting up an experiment, make sure you've created the Offerings you want to test. This may include:" permalink: "/docs/tools/experiments-v1/configuring-experiments-v1" slug: "configuring-experiments-v1" version: "current" original_source: "docs/tools/experiments-v1/configuring-experiments-v1.md" --- > **AI agents:** This is the Markdown version of a RevenueCat documentation page. For the complete documentation index, see [llms.txt](https://www.revenuecat.com/docs/llms.txt). Before setting up an experiment, make sure you've created the Offerings you want to test. This may include: - Creating new Products on the stores - Setting Offering Metadata, and/or - Creating a RevenueCat Paywall You should also [test the Offerings you've chosen](https://www.revenuecat.com/docs/dashboard-and-metrics/customer-profile#current-offering) on any platform your app supports. ## Setting up a new experiment First, navigate to your Project. Then, click on **Experiments** in the project sidebar. You have two options for creating a new experiment: ### Creating a new experiment from scratch Select **+ New** to create a new experiment from scratch. :::info[Set up offering(s) for your experiment] If you've not setup multiple offerings yet, you'll be prompted to do so now, since you'll need at least 2 available offerings to run an experiment. For **App Store** apps, we recommend setting up new products to test as a new **Subscription Group** in App Store Connect so that customers who are offered those products through Experiments will see only that same set of products to select from their subscription settings. ::: ### Duplicating an existing experiment You can also create a new experiment by duplicating the configuration of a previous experiment. This is useful when you want to run a similar test with slight modifications or test the same configuration on a different audience. To duplicate an experiment: 1. Find the experiment you want to duplicate in your experiments list 2. Click on the **context menu** (three dots) next to the experiment 3. Select **Duplicate** from the menu 4. A new experiment will be created with the same configuration as the original 5. You can then modify any settings as needed before starting the duplicated experiment :::tip[When to use duplicate] Duplicating is particularly useful when you want to: - Re-run a successful experiment on a different audience or time period - Test slight variations of a previous experiment configuration - Quickly set up similar experiments without manually configuring all settings again ::: Regardless of which method you choose, you'll need to configure the experiment settings as described below. ## Fields To create your experiment, you must first enter: - **Experiment name**: A descriptive name for your test - **Experiment type** (optional): Choose from preset types (Introductory offer, Free trial offer, Paywall design, Price point, Subscription duration, Subscription ordering, or Other) to get relevant default metric suggestions - **Notes** (optional): Add markdown-formatted notes to document your hypothesis and track insights - **Variant A (Control)**: The Offering(s) for your control group (baseline) - **Variant B (Treatment)**: The Offering(s) for your first treatment group ### Adding more variants for multivariate testing You can add up to 2 additional treatment variants: - **Variant C (Treatment)**: Optional second treatment variant - **Variant D (Treatment)**: Optional third treatment variant Multivariate experiments allow you to test multiple variations against your control simultaneously, helping you identify the best-performing option more efficiently than running sequential A/B tests. ## Using Placements in Experiments Placements allow you to define a unique Offering to serve at each paywall location in your app, so that you can do things like: 1. Show a unique paywall design at the end of onboarding vs. in settings 2. Offer unique prices on the paywall triggered when a customer attempts to access a gated feature vs. the paywall that's triggered after a certain number of app opens If you've not setup Placements yet, [start here](https://www.revenuecat.com/docs/tools/targeting/placements). With Experiments, you can create A/B tests that serve unique Offerings at each Placement to your Control group and Treatment group. ![Adding Placements to Experiments](https://www.revenuecat.com/docs_images/experiments/v1/experiments-placements.png) With Placements, your customers will be served an array of Offerings depending on the paywall location they visit, and with Experiments you can A/B test that experience by changing any element of that array. :::tip[Isolating the impact of each paywall change] If you're looking to isolate the impact of changing just one paywall location, then modify the Offering being served at that Placement in the Treatment group and keep all other Placements the same. But be sure to specify your desired Offering for each Placement that your app uses, even if the Offering for a given Placement should be the same on both the Control and Treatment. ::: ## Enrollment ### Enrollment type Choose which customers to enroll in your experiment: - **New customers**: Enroll customers the moment they open the app for the first time. This is the default behavior and doesn't require any minimum SDK version. - **New and existing customers**: Enroll new and existing customers that qualify for the experiment on any app open. This allows you to test changes on your entire qualifying user base, not just first-time users. :::warning[SDK version requirement for existing customer enrollment] Enrolling new and existing customers requires [specific SDK versions](#required-sdk-versions-for-existing-customer-experiments). This rule is automatically applied and can't be edited when selecting this enrollment type. ::: ![Enrollment type selection](https://www.revenuecat.com/docs_images/experiments/v1/experiments-enrollment-type.png) ### Enrollment criteria You can optionally customize the audience who will be enrolled through your experiment. Select from any of the available dimensions to filter which customers are enrolled. | Dimension | Description | | :--------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Country | Which countries are eligible to have their customers enrolled in the experiment. | | App | Which of your RevenueCat apps the experiment will be made available to. | | App version | Which app version(s) of the specified apps must a customer be on to be enrolled in the experiment. | | RevenueCat SDK version | Which RevenueCat SDK version(s) of the specified SDK flavor must a customer be on to be enrolled in the experiment. *(NOTE: This is most likely to be used in conjunction with features like RevenueCat Paywalls which are only available in certain SDK versions)* | | Platform | Which platform(s) that were first used by the customer are eligible for the experiment (e.g. iOS, watchOS, etc). | ### Customers to enroll You can modify the % of customers to enroll (minimum 10%) based on how much of your audience you want to expose to the test. Keep in mind that enrolled customers will be split evenly between all variants. For example, an A/B test (2 variants) that enrolls 10% of customers would yield 5% in the Control group and 5% in the Treatment group. A 4-variant multivariate test enrolling 20% of customers would yield 5% in each variant. Once done, select **CREATE EXPERIMENT** to complete the process. ### Customers created via the REST API When the experiment enrollment setting is "New customers only", RevenueCat enrolls customers in experiments when they are first created **from the SDK**. Creating a customer directly through our REST API creates the customer before the SDK runs and without SDK metadata. As a result, the customer already exists and is no longer treated as “new,” so they are never enrolled. To ensure customers are enrolled in experiments, **let the SDK create customers first.** Use our REST API only after the SDK has confirmed the customer exists in RevenueCat through `configure` or `logIn` calls on the device. :::info[Enrollment timing for all customer experiments] If your experiment's enrollment criteria is set to **New and existing customers** (not just new customers), eligible existing customers will be enrolled in the experiment when the SDK next requests offerings. This requires [supported SDK versions](#required-sdk-versions-for-existing-customer-experiments). ::: ### Required SDK versions for existing customer experiments Experiments targeting new and existing customers require that your app uses a supported SDK version. Customers on older SDK versions won't be enrolled. This requirement ensures your app can report [paywall views](https://www.revenuecat.com/docs/getting-started/tracking-custom-paywall-impressions), which power the [Paywall view filter](https://www.revenuecat.com/docs/tools/experiments-v1/experiments-results-v1#paywall-view-filter) on your results. | SDK | Minimum Version | | :----------- | :-------------- | | iOS | 5.66.0+ | | Android | 9.26.1+ | | Flutter | 9.15.0+ | | React Native | 9.12.0+ | | KMP | 2.9.0+ | | Capacitor | 12.3.0+ | | Unity | 8.8.0+ | :::tip[Custom paywalls require manual paywall view tracking] If you're using a custom paywall (not RevenueCat Paywalls), you must call `trackCustomPaywallImpression` when your paywall is displayed. Without this call, enrolled customers will appear as not having viewed a paywall. For experiments that enroll new and existing customers, they will also be excluded from the default results view. See [Tracking Custom Paywall Impressions](https://www.revenuecat.com/docs/getting-started/tracking-custom-paywall-impressions) for implementation details. ::: ## Starting an experiment When viewing a new experiment, you can start, edit, or delete the experiment. - **Start**: Starts the experiment. Customer enrollment and data collection begin immediately and results appear in real time. - **Edit**: Change the name, enrollment criteria, or Offerings in an experiment before it's been started. After it's been started, only the percent of customers to enroll can be edited. - **Delete**: Deletes the experiment. Test users are still placed into experiment Offering variants. Sandbox and TestFlight paywall views and purchases aren't included in results. See [Sandbox and TestFlight](https://www.revenuecat.com/docs/tools/experiments-v1/experiments-results-v1#sandbox-and-testflight). To check that a paywall can display the Offerings in your experiment, use an [Offering Override](https://www.revenuecat.com/docs/dashboard-and-metrics/customer-profile#current-offering) for a specific customer instead of relying on experiment results. ## Pausing an experiment Once an experiment is running, you can pause it to stop enrolling customers while continuing to collect data from existing participants. This is useful when you want to: - Evaluate the long-term impact of experiment exposure on already enrolled customers - Stop exposing additional customers to test variants while maintaining consistent behavior for existing participants - Temporarily halt enrollment while analyzing preliminary results When paused: - No additional customers will be enrolled in the experiment - Customers already enrolled will continue to see their assigned variant - Data collection continues for all enrolled customers - Results will continue to update with data from existing participants for up to 400 days ## Resuming a paused experiment You can resume a paused experiment to start enrolling customers again by clicking the **Resume** button. Before resuming an experiment, we'll check if it conflicts with any currently active experiments. If resuming would cause audience overlap, you'll see an error message and will need to either pause or stop the conflicting active experiment(s) to avoid overlap. ## Stopping an experiment Once an experiment is running or paused, you can permanently stop it by clicking the **Stop** button. This is appropriate when you want to: - End the experiment after a clear winner has emerged and you're ready to implement the results - Stop a poorly performing experiment to prevent further negative impact - Clear the way for a new experiment with an overlapping audience - Conclude an experiment that has run its full planned duration :::warning[Stopping vs. Pausing] **Pausing** is reversible - you can resume the experiment later. **Stopping** is permanent - the experiment can't be restarted. Consider pausing instead of stopping if you might want to resume enrollment in the future. ::: When an experiment is stopped: - No additional customers will be enrolled. - Customers who were enrolled will begin receiving the Default Offering on their next paywall view. - Results will continue to refresh for 400 days after the experiment has ended. Renewals from customers who enrolled while the experiment was running can still appear; new subscriptions and one-time purchases started after the experiment ended aren't included. ## Rolling out a winner Once you've identified a winning variant from your experiment results, you can roll it out to all your users. RevenueCat provides several options for applying your experiment results: ### Rollout options When you mark a variant as the winner, you can choose from these rollout strategies: 1. **Set as default offering**: The winning variant's offering becomes your project's default offering, served to all customers who aren't targeted by specific rules 2. **Create targeting rule**: Create a new targeting rule that serves the winning offering to a specific audience (e.g., specific countries, platforms, or custom attributes) 3. **Mark winner only**: Record which variant won without immediately changing your offering configuration - useful for tracking insights and planning future rollouts ### How to roll out a winner 1. Navigate to your experiment's results page 2. Review the performance data to identify the winning variant 3. Click **Review and mark as winner** 4. Select your preferred rollout option 5. Confirm the rollout | Winner recommendation | Rollout options | | :-----------------------------------------------------------------------------: | :---------------------------------------------------------------------------: | | ![Winner recommendation](https://www.revenuecat.com/docs_images/experiments/v1/winner-recommendation.png) | ![Winner rollout modal](https://www.revenuecat.com/docs_images/experiments/v1/winner-rollout-modal.png) | :::info[Experiment data after rollout] After rolling out a winner, your experiment results will continue to get updated for 400 days, allowing you to track long-term performance and learn from your test. However, any customers enrolled in the experiment will get served their default offering once you stop the experiment. To keep collecting data for the experiment with the offering assignment of the experiment, pause the experiment instead. ::: ## Running multiple tests simultaneously You can use Experiments to run multiple test simultaneously as long as: 1. All audiences being enrolled in running tests are mutually exclusive (e.g. either two tests have exactly the same audience, or have fully unique audiences) 2. A given audience has no more than 100% of its customers enrolled in experiments If a test that you've created doesn't meet the above criteria, we'll alert you to that in the Dashboard and you'll be prevented from starting the test, as seen below. ![Experiments conflict](https://www.revenuecat.com/docs_images/experiments/v1/experiments-conflict.png) ### Examples of valid tests to run simultaneously **Scenario #1 -- Multiple tests on unique audiences** 1. Test A is running against 100% of customers for your App Store app 2. Test B, targeting 100% of customers for your Play Store app, can also be run since its targeted audience is mutually exclusive with Test A, and no more than 100% of each audience's customers are being enrolled in running experiments **Scenario #2 -- Multiple tests on identical audiences** 1. Test A is running against 20% of customers in Brazil 2. Test B, targeting 40% of customers in Brazil, can also be run since its targeted audience is identical with Test A, and no more than 100% of that audience is being enrolled in running experiments ### Examples of invalid tests to run simultaneously **Scenario #3 -- Multiple tests on partially overlapping audiences** 1. Test A is running against 100% of customers for your App Store app 2. Test B, targeting 100% of customers in Brazil, **can't** be run because there is partial overlap between the audience of Test A and the audience of Test B (customers using your App Store app in Brazil). 1. To run Test B, either Test A will need to be paused, or the audience of Test B will need to be modified to exclude customers from the App Store app. **Scenario #4 -- Multiple tests on >100% of an identical audience** 1. Test A is running against 20% of customers in Brazil 2. Test B, targeting 100% of customers in Brazil, **can't** be run because the targeted audience would have > 100% of customers enrolled in experiments, which isn't possible. 1. To run Test B, either Test A will need to be paused, or the enrollment percentage of Test B OR Test A will need to be modified so that the total doesn't exceed 100%. :::info[Editing running experiments] When an experiment is running, only the percent of customers to enroll can be edited. This is because editing the audience being targeted would change the nature of the test, rendering its results invalid. ::: ## Limitations and edge cases These constraints cover what you can edit after an experiment starts, plus enrollment and platform limits that affect setup. For sandbox and TestFlight behavior in results, and for unexpected products or attribution (including aliasing and transfers), see [Limitations and edge cases](https://www.revenuecat.com/docs/tools/experiments-v1/experiments-results-v1#limitations-and-edge-cases) on the Experiments Results page. ### Configuration after start Once an experiment has started (or while it is paused), most configuration is locked so results stay statistically valid: | Change | Allowed? | | --- | --- | | Percent of customers to enroll | Yes, while the experiment is running. | | Offerings / products on a variant | No. Stop and create a new experiment. | | Add a variant | No. Stop and create a new experiment. | | Enrollment audience (apps, countries, new vs existing, etc.) | No. Only the enrollment percentage can change. | | Edit configuration while paused | No. Stop and create a new experiment to change setup. | | Resume after pause | Yes. You can pause and resume an experiment an unlimited number of times. | | Restart after stop | No. Duplicate the experiment to continue testing with the same Offerings. | ### Platforms and versions You can't target different app versions for each app in a single experiment. Create separate experiments per app (and target by app version) if you need that split. ### Enrollment: customers created via the REST API Customers created through the Developer API before the SDK aren't enrolled in experiments, which can make enrollment counts look lower than trials or paywall views in other tools. See [Customers created via the REST API](#customers-created-via-the-rest-api). ## FAQ | Question | Answer | | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Can I run multiple experiments simultaneously? | Yes, as long as they meet the criteria in [Running multiple tests simultaneously](#running-multiple-tests-simultaneously). | | Can I add multiple Treatment groups to a single test? | Yes, experiments support up to 4 variants total: 1 Control (Variant A) and up to 3 Treatment variants (B, C, D). | | Can I enroll existing customers in an experiment? | Yes. Choose **New and existing customers** when creating the experiment. This requires the [minimum SDK versions](#required-sdk-versions-for-existing-customer-experiments). Existing customers that qualify enroll on their next app open. | | What's the difference between pausing and stopping an experiment? | Pausing temporarily stops new enrollment; existing participants keep their variant and you can resume later. Stopping permanently ends enrollment; existing participants see the Default Offering on their next paywall view and the experiment can't be restarted. Both continue collecting data for up to 400 days. See [Pausing](#pausing-an-experiment) and [Stopping](#stopping-an-experiment). | | Can I pause an experiment multiple times? | Yes. You can pause and resume as needed. Pausing only affects future enrollments; already enrolled customers keep their variant and continue to be tracked. | | How do paused experiments affect the audience overlap checks? | Paused experiments don't count toward audience overlap. When you resume, RevenueCat checks for conflicts with currently active experiments and blocks resume if there's an overlap. | | Can I duplicate an experiment? | Yes. Use the context menu on the experiments list. The duplicate starts as a new draft with the same configuration. | | What happens after I pick a winner? | You can set the winning Offering as default, create a targeting rule for specific audiences, or mark a winner without rolling out yet. See [Rolling out a winner](#rolling-out-a-winner). | | How many variants should I use in my experiment? | Start with 2 variants (A/B) for most cases. Use 3–4 when you have multiple distinct hypotheses. More variants need more customers to reach significance. | | What experiment type should I choose? | Choose the preset that best matches what you're testing. Presets suggest default metrics (for example "Price point" → revenue metrics). You can customize metrics after selecting a type. | | What's the difference between "New customers" and "New and existing customers"? | **New customers** enrolls only customers seen for the first time after the experiment starts. **New and existing customers** also enrolls existing customers and requires [supported SDK versions](#required-sdk-versions-for-existing-customer-experiments). | | Do I need to do anything in my app for existing customer experiments? | RevenueCat Paywalls track paywall views automatically. For a custom paywall, call `trackCustomPaywallImpression` when the paywall is displayed — see [Tracking Custom Paywall Impressions](https://www.revenuecat.com/docs/getting-started/tracking-custom-paywall-impressions). Without that call, enrolled customers won't be marked as having viewed a paywall and are excluded from the default results view. |