Frequently Asked Questions

Q. What is flicker / flash of original content, and how does SiteSpect avoid it?

A. Flicker, or flash of original content, is the experience of a page loading certain content (for example a 20% off banner), and then quickly replaces it with different content (like a 10% off banner). It looks like the page is “flickering.” It happens when sites use JavaScript tags to replace content, such as when they’re doing an A/B test. The site loads with the original code, and then the JavaScript fires telling the browser to display the new content. This can be jarring, and can also reveal information that is irrelevant for the site visitor in question. It has also been shown to decrease conversions. You can learn more about flicker here.

SiteSpect avoids flicker because we don’t use JavaScript tags to change content in the browser, and we don’t need to wait for the page to be loaded and rendered to make a change. Instead, content is changed in the flow of traffic before it hits the browser, meaning the browser only ever receives the correct content, eliminating the risk of flicker.

Q. Who can use SiteSpect? Do I need to be a developer?

A. SiteSpect can be used by any team or job function, including marketing or other less-technical users. We have a Visual Editor that allows you to make visual changes without any code or technical knowledge. We have other tools that enable developers to use coding as well. We also have an in-house, full-service customer success team that can consult and build experiences for your site either on an as-needed basis, or can work as an extension of your own team.

Q. Can I use SiteSpect on multiple domains?

A. Absolutely. When you create an A/B test or other experience you can release it across all of your domains or you can pick and choose. Many SiteSpect customers maintain multiple domains and experiment across all of them.

Q. Can I use SiteSpect on my SPA site?

A. Yes, SiteSpect fully supports SPAs, including but not limited to Angular, React, Ember.JS, Vue.JS, and Backbone. Our Visual Editor is also compatible with SPAs, in addition to developer friendly tools.

Q. How does SiteSpect safeguard against RegEx (regular expressions) changes affecting more than intended, since RegEx can be so powerful?

A. Any campaign using RegEx built for you by a SiteSpect Solutions Developer will go through SiteSpect’s rigorous QA process. Clients building their own campaigns with RegEx can use SiteSpect’s preview capability as well as our advanced QA tools as part of their own QA processes. We also approach development with RegEx with a trend toward specific matches. Finally, we have a disablement functionality that you can use to disable a campaign in seconds, and can intentionally trigger if something in production gets changed.

Q. What is multi-armed bandit testing and how does SiteSpect support it?

A. The “Multi-armed bandit” is a probability theory that informs how we assign traffic to campaigns using Auto Optimization. Traditionally when A/B testing, you will split traffic evenly between your variations. SiteSpect Auto Optimization uses multi-armed bandits to dynamically distribute more traffic to variations that are performing well and less to variations that are underperforming. This allows you to produce faster results and capitalize on your wins sooner.
You can learn more about Auto Optimization here

Q. Does SiteSpect support XPath anywhere?

A. Yes, you can use JavaScript to find and identify elements in the DOM based on XPath, using our Client-Side Solutions or with JavaScript injection.

Q: In regards to the Data Outliers portion of Enhanced Analytics, how are outliers displayed?

A: When the Enhanced Analytics add-on is enabled, outliers are displayed within the same format of other campaign results by either choosing to include, exclude, or average the outlier data points into the total data set. “Toggling” these 3 options on/off is very easy.

Q. Can I build multi-page forms in SiteSpect?

A. Yes. SiteSpect allows you to build multi-page forms using web technologies such as HTML, CSS, JavaScript and form fields to do this. We expect an endpoint to receive the final form submission.

Q. How do I create a new test in an SPA framework?

A. You would do this using Client Side Factors. You can find a step by step guide here

Q. Does SiteSpect support HTTP/2?

A. SiteSpect does not currently support HTTP/2. However, our partner Section does, and we offer deployment via Section that covers HTTP/2.

Q. Could Enhanced Analytics be used as a standalone offering (without the ability to run any A/B tests or personalization campaigns in SiteSpect)?

A. No, Enhanced Analytics is designed as an add-on to SiteSpect’s other products. Some aspects of RUM (real user monitoring) can be monitored across traffic, but the core components of Enhanced Analytics (RUM and Outliers) apply to testing, personalization, or recommendation campaigns.

Q. Can I release a winning Variation to 100% of traffic through SiteSpect?

A. Absolutely, and we have several ways to accomplish this depending on your needs.

  1. When a Variation has won, you can enter the Campaign Builder and simply select “Promote to all Traffic.” This will end your Campaign, and automatically create a new Campaign that contains only the winning Variation and serves it to 100% of traffic.
  2. You can also recreate your winning Variations using Global Variations that apply to all traffic. Global Variations are commonly used for analytics integrations, and to deploy EventTrack and other tracking code. They can also be used to push changes to all traffic.

Q. What is the best way to exclude myself from a campaign?

A. There are several options to exclude yourself from a campaign.

  • You can create an IP exclusion audience for your IP address. Keep in mind that your IP address will change as you move to different networks (home, office, coffee, shop, etc), or within the same network depending on how your ISP manages its IP releases.
  • You can create a URL parameter, for example add ?user=name to the end of the URL, and create an audience to match that. When you visit the site using this URL you will be excluded from the campaign.
  • You can use a cookie audience and create a custom cookie when you enter the campaign.

Q. Is it possible to see the history of user logins in SiteSpect history?

SiteSpect history shows the last login for a given user. It also shows granular user changes for Campaigns, Metrics and other components.Our professional services team can provide additional logging capabilities upon request.

Q. Does SiteSpect support Single Sign On (SSO)for SiteSpect Cloud and for on-premise SiteSpect customers?

A. SiteSpect supports SSO for SiteSpect Cloud clients who do not use Okta. SiteSpect fully supports SSO for on-premise users with our own SAML provider.

Q. Is it possible to disable notifications by Campaign?

A. No, you can disable notifications by site only.

Q. Do clients typically give QA teams SiteSpect logins?

A. Most clients don’t require QA teams to have SiteSpect control panel logins. One of the great things about SiteSpect is the shared preview link functionality that does not require a login. The user who generates the preview link has the option to show or not show the diagnostic overlay.

Q. Is it possible to show a Personalization to a customer only once without dev work?

A. Yes, you can do this by using a metric as a trigger.

Q. What is the difference between client-side and server-side A/B testing?

A. Client-side testing refers to any experience that occurs in the browser. Another way to think of this is any change the user will see. For example, changes to copy, images, layout, or CTAs are all client-side changes. Server-side testing refers to parts of the experience that happen before the browser. For example, changes to algorithms, site platforms, or, URLs such as redirects or rewrites are all server-side changes.

Q. Do you offer multivariate testing?

A. Yes. When you create a new campaign in SiteSpect you can select “Multivariate Test.” When you define your factors you can have SiteSpect automatically generate a “full-factorial,” where we create every combination of each of your variations to test against each other. Or, you can select which specific experiences you would like to test. You test up to 511 concurrent Variation groups. For more full documentation click here.

Q. How many Variation Groups can I include in a single campaign?

A. You can include up to 50 Variation Groups in one campaign, and up to 15 Variations within each group. In the rare circumstance where you need more, we’ve had clients use linked Variations to create a Campaign with over 150 stacked Variations.

Q. Can I create an A/B test if I don’t know code?

A. Yes. SiteSpect Visual Editor allows you to design and create campaigns with our drag and drop interface. We also offer an in-house professional services team who are available to work with you as part of your team to design and build A/B tests.

Q. Can I A/B test my personalization campaigns?

A. Absolutely! This one of the benefits of SiteSpect personalization. You can personalize any experience by setting your audience based on any number of identifiers, including geolocation, device type, browser type, visit history, IP addressed, or based on specific actions a user has taken on your site, and many more. This capability can apply to an A/B test, a Rollout campaign, and even a Recommendation campaign.

Q. How can I personalize an experience based on actions a user has taken on my site? For example, show a specific promotion to only users to have clicked on a certain product type.

A. You can do this by creating an audience based on a metric. If a user has completed a metric, such as a click on a specific link, they will be assigned to this audience. This gives you the ability to personalize an experience based on a fully custom criteria.

Q. Is SiteSpect Recommendations only for retailers?

A. Nope! Most of our Recommendations customers are retailers, but if you can categorize your content with a SKU and measure conversion for each item, then you can use SiteSpect Recommendations. For example, universities have used SiteSpect Recommendations for course suggestions, or insurance can use it to recommend plans.

Q. How much coding is involved if I want to use SiteSpect Recommendations?

A. No coding is required to use SiteSpect Recommendations. We offer pre-made algorithms that sync with your inventory, and when you want to insert a recommendations module on your page you can simply select your options from a dropdown menu. If you’d like, we offer the capability for you to custom code your recommendations algorithms and design, but it is absolutely not required.

Q. Can I combine my recommendations campaigns with my A/B testing and personalization campaigns?

A. Absolutely, that’s one of the major benefits of using SiteSpect. All of our products live within one interface, and you can A/B test and personalize your recommendations campaigns. You will access all of your campaigns from the same place, and can use them all simultaneously. You can even use SiteSpect to A/B test other recommendations tools against each other and against SiteSpect.

Q: If an audience excludes users from other campaigns, does that mean those users cannot be assigned to any overlays or personalizations?

A: Yes. For example, if experience is accepting 100% of traffic, all users who are eligible will be assigned to Experience A and then will be ineligible for any other campaign, including overlays and personalizations. Users who are already assigned to another campaign/experience are ineligible for experience A.

Q. Can a “feature flag” have metrics like conversions and revenue impact? Can you compare audiences that saw the feature flag versus those that didn’t?

A. Any campaign, including feature flags, and ones that compare a feature being turned on vs. turned off can have business, engagement, and performance metrics. This is a big plus for SiteSpect because with other feature flag tools the burden of keeping track of those metrics falls heavily on the developers. Not so with SiteSpect!

Q. Does SiteSpect work with sites built on Netx.js, with full server-side rendering?

A. Yes. This is essentially an SPA (powered by React) that pre-renders the initial requests. SiteSpect can process the initial request and do standard Find and Replace (removing flicker on initial page load) and then use Client Side Factors to manage that application once it’s loaded and running.

Q: Can we use the SiteSpect API to send data to Tableau, Power BI, or another BI or data tool? Can you only send raw data, or can you send data already cleaned up with segments?

A: Yes, clients can use the SiteSpect API to send data to any BI or data tool, and this data can be sent raw or you can apply segments before export. Several clients use the API to download active campaign performance matrixes that apply segments to each campaign. SiteSpect’s professional services team can help you build API scripts to send the data of your choice to your external data management tool. To do this, you need to make sure you have helpful custom variables in SiteSpect campaigns that capture useful unique IDs, such as customer ID, basket ID, session ID, etc., so that the exported data can easily be connected to data in your external tool.

Q. What types of factors, triggers, and/or techniques can you use to define category affinity audiences?

A. Anything in SiteSpect can be used to judge affinity. For example, user behavior metrics triggers, page source data islands, or cookies.

Q. If I would like to track validation metrics for multiple parts of my site, is it best practice to set up separate validation campaigns or to have one campaign tracking all of my metrics?

A. If it is a validation campaign, meaning that it makes no changes and just tracks a series of metrics for traffic validation or baseline data, it’s best practice to keep only one campaign. You only need a single variation group, and can turn off the non-control variation. When you view your performance matrix you can filter out the metrics you’re not interested in at the moment.

Q. Can I use RUM+ to isolate the analysis time for a Trigger?

A. "SiteSpect's ssanalysis time metric will show this, or you could use X-SiteSpect-Debug: 1 to look at pages one off and see each time broken down per variation (

Q. Can SiteSpect tie their optimization efforts to offline conversions?

A. Yes, our client EnergySage uses Engine API calls to track offline conversions. When a user submits a form with EnergySage, their SiteSpect cookie and a SiteSpect user id are picked up with the EngineAPI, which can then be stored in their CRM or other analytics system. An EnergySage employee then approves or denies the submission, which triggers an API call referencing the user id and SiteSpect cookie, which is captured as a metric in SiteSpect’s analytics.

Q. How do SiteSpect clients sync data across platforms, for example when they use different analytics platforms for the same site?

A. SiteSpect WATTS allows SiteSpect clients to push assignment data to any other platform by dynamically inserting information about website visitors into tracking tags used by all major web analytics programs. Doing this lets you automate how you sync your data across your analytics platforms so you have a consistent record of truth.

Best practice is to pass not only Campaign and Variation Group IDs but also SSGUID so those tools have our user IDs. That makes any comparison much easier. Beyond WATTS, clients can also capture IDs with custom variables in SiteSpect, including things like basket ID, session ID, or order ID. That way we have this available in our reports and data export. We can then provide clients with an API script to download all campaign data including those IDs to a data lake (AWS, Azure, other location) and it makes stitching SiteSpect data to other sources (beyond web analytics with WATTS) much easier.

Q. How does SiteSpect define a visit? How do you differentiate between visits and sessions, and how do you define page views?

A. Visits and sessions essentially measure the same thing. A visit and session is defined as one continuous site visit, regardless of the number of page views, requests, or impressions. As long as the user is active, the session can last any amount of time. After 30 minutes of inactivity the visit and session times out. You can create custom page view metrics in SiteSpect, allowing you to define a page view however you wish, for example using URL, page source, or content=html.

Q. Does SiteSpect use a Bayesian or Frequentist approach?

A. Natively, SiteSpect uses a Frequentist approach in its analytics and reporting, but customers can export raw data from SiteSpect and use any approach they wish with their stats tool of choice.

Q. Is it possible to set incoming cookies’ value to empty using a campaign or site variation?

A. Yes, an origin campaign can set a cookie value to empty.

Q. How does SiteSpect quantify their revenue impact for the customer? For example, can you determine the amount of additional revenue a campaign generated, and how would you do it?

A. There are many ways to quantify the revenue impact of a SiteSpect campaign. You can measure the revenue at checkout and compare the variation to the control, or you can quantify risk avoidance if you see sales drop off in one variation. For most clients we implement a series of metrics triggered on different URL and page content to capture conversion and revenue, and use report segmenting to identify winners and quantify revenue impact.

Q. Do clients ever do “operational tests,” where they will benchmark how long a UI / UX test will take to build internally versus externally?

A. Yes, several SiteSpect customers have done this. In these assessments the outcome was that StieSpect’s team could build the test either as quickly or quicker, and that they would get to the test sooner than an internal dev team would.

Q. What are some examples of tests you might run using Visual Editor to impact conversion rate?

A. The options are endless. You can use the Visual Editor to insert messaging that instills confidence, security, and trust near Add to Cart or Purchase buttons, you can change your Calls to Action, or introduce new or different featured items. For more ideas, you can browse our blogs and use cases.

Q. Will we need to keep up with the latest APIs, or is there a backwards capability guarantee that existing APIs will still be maintained despite any updates?

A. As a company we try to avoid breaking changes in both our admin and engine APIs. In the rare case a breaking change is introduced, there will be an advanced warning of at least 6 months.

Q. How do you set a custom variable using the Engine API?

A. You can set a custom variable with Engine API the same way as you would a standard metric, but the “CustomCaptureValue” would include the string / custom variable.

Q. Does SiteSpect work with my CDN?

A. Yes, SiteSpect works with all CDNs, including Akamai, Cloudflare, Fastly, StackPath, Amazon CloudFront, Microsoft Azure. We can integrate with whichever CDN you use.

Q. What are some low-cost heat mapping or recording tools that are popular with SiteSpect customers?

A. Many clients use HotJar and Crazy Egg along with SiteSpect. These tools are usually priced per page view.

Q. Do any customers do any type of weather integration?

A. Yes, we can integrate this as part of a professional service engagement. We do find some clients wish to use weather applications tied to state or zip codes and target those areas with different types of outerwear products during very specific weather conditions. Clients can also execute weather integration using the Admin API.