The ceiling of the average and evolution of CRO

Experimentation is evidence to determine whether something works. Intent decides who receives what, when (or whether they need it at all). “Does this work.” Think of this as 80% exploration, and 20% exploitation.

The first works towards the best single answer for everybody to prove a hypothesis. The second works out where a single answer was never going to be enough; scaling a proven hypothesis for true commercial gain. “For whom does it work best, when should it be shown, and do a proportion of users even need it at all?” Think of this as 20% exploration, and 80% exploitation. Where testing tells you whether an experience works, but Intent decides who receives it, when, and whether they need it at all.

In this blog post, we'll identify the differences between Intent and experimentation, so you can understand when to utilise each strategy to further your business goals.

Defining A/B testing

An A/B test answers one single bounded question.

Does this change (a treatment) move the metric we care about, across our traffic, better than the alternative does (a control)?

It's a hypothesis, and what comes out the other end is 80% knowledge (learnings) and 20% value (hopefully positive gain). A primary "does this prove or disprove our hypothesis" binary answer.

Our intent engine answers a different question, and it answers it again every few seconds throughout a users session:

Given what this visitor is doing right now, which of the experiences we already know works suits them at this moment in time? (If any)

What comes out of that is not just a single decision, but hundreds of them a second, all trying to move the agents primary goal, or reward. The open question stopped being does this work a long time ago. What's left is who needs it, and when.

For example: if a visitor starts showing signs of basket abandonment. You can put a returns message in front of them, or a discount, or a trust signal, because you already trust all three of those mechanics. What you don't know is which one that particular person needs, at what point, or whether they need any of them at all. That's the decision the agent is making.

There are several reasons why experimentation (read: split testing) might not be an appropriate mechanism; here are four reasons:

But first, a note on the word experimentation. Throughout this set, experimentation means the whole practice of using evidence to drive ideas: user research, usability testing, prototyping, quasi-experiments and switchbacks where a split isn't possible, and A/B tests where it is. The split test is one instrument inside it, usually the last step rather than the whole of it.

1. Averages can lie.

An A/B test reports an average treatment effect which can hide segments or audiences that have a detrimental or negative impact.

An A/B test reports an average treatment effect. That's the whole point of it. Randomising across your population is what buys you a causal claim in the first place. The price you pay is that the average swallows everything inside it. A headline +2% can be +12% for hesitant first-timers and -4% for loyal returners. Same test. Same green tick. Two completely different stories.

Statisticians call these heterogeneous treatment effects, and there's good evidence that acting on them beats a blanket rollout. In one worked simulation of a ranking change, treating only the 69% of users predicted to benefit delivered a 2.2% gain in revenue per user against shipping it to everybody. A CUNY study of financial aid nudges captured roughly 75% of the total benefit while treating half the population.

Experimentation is deliberately built to stop you trusting the segment breakdown. Pre-registration, power calculations, no post-hoc slicing. Those rules exist because sliced results are where false positives breed. Kohavi, Deng and Vermeer's A/B Testing Intuition Busters (KDD 2022) found that underpowered analyses inflate the effects you observe by 25 to 50%. At the power levels most mid-market programmes actually run at, over half of your significant results are false positives. So the question you have to answer to deploy well, for whom, is the question your test is worst at answering. That's a boundary, and a sensible one. Intent sits on the other side of it. It predicts for every visitor in advance, rather than carving up a sample after the fact.

‍Where Intent solves this problem: The experience is being delivered across hundreds of contexts, reallocating to where the experience "works" (where it's best seen and best felt), excluding it from where "it doesn't work." In theory, it only looks and focuses on the good, excluding the bad.

2. Testing assumes the trigger.

Every A/B test starts from a fixed, assumed rule (usually on page load). The trigger "when to show it" is often assumed, arbitrary, and based on a website proxy like "3 page views" or "PDP" rather than genuine user intent.

Nearly every A/B test starts from a fixed rule. Show this on the PDP after three pageviews. Fire this on exit intent. Serve this to returning visitors. Then it tests which creative performs best inside that rule.

The treatment gets examined. The trigger gets assumed.

There's a practical reason for that. The trigger space is far too big to write out by hand. Nobody can enumerate every combination of buying stage, purchase confidence, abandon risk, intent trend and product affinity, let alone power a test across them. So teams pick a proxy or an average and move on.

Proxies are usually where the damage happens. These are largely based on website attributes as opposed to personalised signals. Returning visitor stands in for higher intent. Mobile stands in for lower intent. Pageview count stands in for engagement, when a confused shopper racks up far more pageviews than a decisive one, and high intent visitors often convert inside three.

Our own Intent Gap research found 10% of visitors on checkout pages aren't ready to buy. And 34% never get past browsing, whatever page they happen to be on. There's a reason we say stages, not pages.

‍Where Intent solves this problem: Intent agentic delivery serves the individual at a moment in time. A user might need an experience on the 3rd page view after 30 seconds, a different user might need it after 50 seconds, a different user might need it after 9 page views and 1 second. The continuous prediction modelling gives a threshold to meet first, and then a mechanism to automate a decision second.

3. Statistical significance is a search for one answer.

The search for statistical significance can a) slow you down and b) inhibit your ability to personalise to different segments.

Significance exists to license a claim about a single population, and the claim is singular by design where B beats A: one answer, for everyone. That is exactly what costs time.

You need a sample big enough to speak for the whole population with a minimum detectable effect. If you want the claim to be about a segment instead (what some call "personalisation") you need that sample inside the segment too, so the wait multiplies. Most claim they don't have the traffic to personalise because of this.

Experimentation vendor's posterior is a belief about a variant. One distribution per arm, computed across everybody who saw it, resolving to a single winner for all your traffic. Better maths than a p-value, and the shape of the answer is identical. One number per variant, and a threshold you wait to cross before you act.

Our posterior at Made with Intent is a belief about a variant given a context. Where their question is whether B beats A, ours is whether B beats A for a visitor in this intent state, at this point in their session. The model predicts reward conditional on context, so what comes out is a policy; hundreds of answers, running at once.

Put it this way: instead of a human designing three segments and running three underpowered tests, the winner goes to the agent alongside its alternatives and the segment level optimisation runs continuously. Three mechanisms make that more traffic-efficient than a segmented test. Worth understanding rather than taking on trust.

1. Allocation is adaptive. Thompson Sampling draws plausible performance rates from each variant's posterior and routes traffic accordingly, so your exploration cost falls as confidence climbs rather than sitting at 50% for the duration. In practice agentic campaigns push 80 to 90% of traffic to the winner inside two to three weeks. A fixed-allocation A/B test takes around six. Optimised Control shrinks the control group automatically as confidence grows.

2. Without waiting for statistical significance. It borrows statistical strength across slices. This is the important one. Made With Intent runs a contextual bandit, not a multi-armed one. A multi-armed bandit only learns from the data each arm receives, so every segment needs its own volume, which is the same problem as a segmented A/B test in different clothing. A contextual bandit trains a model that predicts reward given context, so it can estimate performance for a combination it has never directly seen. Show it mobile traffic, low intent traffic and a particular page separately and it can predict for "mobile, low intent, that page" by generalising from how each component behaves. A hand-built segment can't borrow like that. Every cell starts at zero.

3. You're not assuming which segment to slice or review. The agent can build up to 500 contextual combinations and up to 75 moment-based triggers per campaign, and it trims its own feature selection to keep each slice sample-rich. Nobody has to guess which distinctions matter before there's evidence about which ones do.

Why we don't report per-segment significance

This surprises experimentation-native teams more than anything else in the product. It's a deliberate methodological choice.

Run an independent significance test on every intent combination and multiple comparisons swallow the results. At α = 0.05, roughly 14 independent tests give you about a 50% chance of at least one false positive. A hundred tests will throw up around five false positives by chance alone, and across hundreds of combinations you'd be manufacturing findings.

We trialled per-segment significance reporting and pulled it, because it surfaced spurious and inverse correlations. For example, users traverse through different stages of intent in their journey, at what point is a user in low intent and when did they see the treatment?

The per-segment decisions get justified differently. Is the model calibrated, meaning when it says 70% does it convert around 70% of the time (measured by expected calibration error)? Is it discriminating, meaning can it rank likely converters above unlikely ones (AUC around 0.84)? A well calibrated probability is a legitimate basis for acting on a slice. An underpowered significance test on that same slice is not.

This calls back to something we talked about earlier.The reason you shouldn't chase per-segment significance in your testing tool is the same reason we don't chase it in ours. Prediction is the right instrument for the for whom question, retrospective slicing isn't.

Why we don't report on single variant uplifts

You get one clean causal number, control against the agent's allocation, measured the way any experimentation lead would want it measured. You give up per-arm inference, and in exchange the arm is adaptive. That's the trade. An A/B test gives you clean inference on every arm and one winner for everybody. An agentic campaign gives you clean inference on one arm, and a different winner in every context.

Load five experiences into an agentic campaign and it looks like a five-arm test. It isn't. It's actually a two-arm experiment, the same shape as any A/B test you've run. The difference being:

• Control. A baseline you define.

• Variant. The agent's allocation across all five experiences.

A traditional A/B test asks which variant wins with a blanket rule. An agentic campaign asks a different question with an adaptive rule (it's a reason why we call them campaigns, not experiments)

Does allocating these experiences by intent beat applying a blanket rule?

We don't report on single variant uplifts eg: Variant B is better by 5% because:

1. There's no single number to give you. Experience B might be the winner for high intent, budget-conscious mobile visitors and the loser for low intent desktop. That's the entire point of running it this way. Collapsing it to one figure puts back exactly the average this campaign exists to get rid of.

2. The arms were never randomised against control. Traffic reaching experience B was chosen by the agent, on context and accumulated evidence. It isn't a random slice of your audience so there's no effective control. Comparing a deliberately selected group against everybody is confounded by construction, and nothing in the data separates the effect of the experience from the effect of the selection.

3. You'd be reading a state the system has already left. If experience B was struggling in a context, the agent would have noticed days ago and moved traffic away (hence: reallocation). The number in front of you is an average across a period in which allocation was actively changing, and the campaign has moved on since. Acting on it means acting on history the agent has already corrected for.

That distinction does three things, in order.

1. It changes what you can personalise. A per-variant posterior can only ever produce one answer for everyone, however good the statistics behind it are. Varying what people get requires probability conditional on the person. That's the whole game, and it matters far more than the frequentist argument ever did.

2. It changes when you can act. Certainty becomes a dial rather than a gate. At 62% probability, Thompson Sampling tilts the allocation and updates again tonight. Nobody declares anything. A stopping rule stops being necessary once allocation is continuous.

3. It changes how fast you get there. Nothing is waiting for a threshold, so allocation improves from the first night onward. There's no finish line to reach before the work starts paying.

Where Intent solves this problem: Our posterior at Made with Intent is a belief about a variant given a context. Where experimentation question is whether B beats A, ours is whether B beats A for a visitor in this intent state, at this point in their session. The model predicts reward conditional on context, so what comes out is a policy; hundreds of answers, running at once based on a series of probabilities using a contextual bandit.

4. Limited to a single answer for only one point in time.

An A/B test only tells you what was true of your traffic during the weeks the test ran.

An A/B test tells you what was true of your traffic during the weeks the test ran; a shelf life. Nothing about that answer refreshes itself so brands often end up "re-testing" the same thing a few years later, hoping there's no change from a previous positive test. The world moves on, results degrade as users become "used" to the treatment, competitors copy; not to mention that your purchase lifecycle varies by days, weeks, months, years.

For example: a Microsoft experiment on MSN.com saw replacing one button with another produce a 4.7% increase in overall clicks. The daily breakdown post launch showed the difference decreasing rapidly day over day as users learned the change, leading to the team shut the experiment down mid-way. The conclusion was that the observed treatment effects "are not always permanently stable, sometimes revealing increasing or decreasing patterns over time."

There are a few reasons why this happens.

1. Novelty and primacy wear off. A new element gets attention because it's new. Returning visitors are briefly worse off because it isn't what they knew (which is why tests are often split, somewhat arbitrarily, between new and returning users)

2. Buying patterns differ throughout the year. December traffic behaves nothing like March traffic, different intent, different price sensitivity, different tolerance for being interrupted (especially for retailers)

3. The population itself drifts. Mix of paid traffic allocation shifts, category mixes alter or a competitor changes their delivery proposition. Macro-economic factors alter the interaction effect.

Does a mature site slowly accumulate a layer of decisions that were correct once, are serving everybody by default, yet are answerable to nobody? It is entirely possible to have a well-governed testing function and a site full of expired answers at the same time.

‍Where Intent solves this problem: Continuous allocation doesn't have this failure mode, because it never declares anything. The posterior is a live belief rather than a verdict, continually reallocating and therefore responding to changes, not static. Allocation is re-scored against current behaviour and retrained nightly, and standing exploration keeps a small share of traffic asking whether the current answer is still the right one. When February stops behaving like November, the agent finds out because it never stopped looking.

Intent vs experimentation: In depth

We've done a quick reference table on the differences between Intent and experimentation. Take a look, send it to your colleagues. You're welcome.

Experimentation and Intent aren't solving the same problem, and treating them as substitutes is where most testing programmes stall. An A/B test proves whether something works for everybody, on average. But it can't tell you who needs a returns message versus a discount versus nothing at all, and it becomes redundant the moment the traffic that validated it moves on.

Intent picks up where that boundary sits, deciding who an experience is for, when, and whether they need it at all, re-scored every few seconds instead of declared once and left to expire. Most eCommerce teams already have the first instrument. The second is what turns that evidence into revenue, visitor by visitor, instead of one rollout for everybody.

Book a demo to see how Made With Intent can help you deliver more appropriate experiences to your customers.

Made With Intent is an on-site intent engine for eCommerce businesses. It reads buying intent in real time and decides which experiences go to which visitors, and when.

Dynamic Yield is an experience optimization and personalization platform. It helps businesses tailor digital customer journeys across websites, mobile apps, and email.

This post explains where the two platforms work together, and where you'd still use Dynamic Yield instead.

If you can't wait until the end, here's a TLDR;

Pick Dynamic Yield if:

• Your priority is consistent personalisation across web, mobile app, email, and in-store kiosk

• You've got the team and budget to do a weeks long enterprise deployment

• Recommendation capabilities matters more than first-page view intent coverage.

Made With Intent is for you if:

• You want to read buying intent in real-time

◦ Then, serve the right content, the right experience at the appropriate moment

• Want to benefit from a 30-60 minute install time

And when we're thinking about Made With Intent and Dynamic Yield working together, here's how we recommend thinking about it:

Made With Intent decides what experiences you should serve, to who and when, and Dynamic Yield delivers those experiences.

How Made With Intent is different to Dynamic Yield

There's three things that Made With Intent does different to Dynamic Yield. Let's dig into them below:

1. Made With Intent reads intent in real time

Dynamic Yield's intent workflow utilises two routes. First, Empathic Personalization classifies visitors into four inferred states, which are Curious, Interested, Focused and Satisfied.

Its Audience Hub lets teams hand-build "low / medium / high intent" audiences from rules. Dynamic Yield's Primary Audiences framework shows a leading golf retailer defining low intent as fewer than 12 page views per session and high intent as more than 24.

That's a useful starting estimate, but page views aren't intent. A hesitant shopper racks up more pages than a decisive one. A high-intent visitor often converts inside three.

The more common version isn't pageview counts — it's event proxies. Add to cart equals high intent. Wishlist equals consideration. But an add to cart is as often a price check, a size comparison or a shipping-cost probe as it is a purchase signal. The event tells you what happened. It doesn't tell you what it meant.

Now, this is pretty fundamental: Dynamic Yield's model relies on behaviours your shoppers have already exhibited. It's a snapshot backwards in time, and like many what we call "rules-based" personalisation tools, you're reacting to things that have already have happened. Unlike Made With Intent.

Made With Intent reads buying intent directly — multi-dimensional, second-by-second signals predicting where the visitor is in their decision right now.

We break downs shopper behaviour into six stages, and these are: Intent stage. Intent signals. Intent trends. Purchase confidence. Abandon risk. Shopper mindset.

All updated every three–five seconds during a live session, on a model trained across 150+ retailers and 50 billion+ events.

2. Allocation across hundreds of intent combinations and not post-test segmentation

It takes eCommerce teams considerable time to build and maintain audience rules in Dynamic Yield. For instance, coding things such as "low intent equals fewer than 12 page views, high intent equals more than 24," then QA-ing those cohorts and redesigning them after each test.

Made With Intent replaces that with continuous intent prediction and delivery. The model decides who sees what, in real time, across hundreds of intent combinations. The actual grunt work is handled by our agent. Your team focuses on strategy, creative, and proof.

Our agent retrains daily, so the experience keeps allocating toward the intent combinations where impact is actually felt. Always on, always learning, always improving on where it started.

3. First-pageview coverage — no fallback needed

Behavioural data takes time to accrue; for anonymous or first-time visitors, Dynamic Yield falls back to geo-based predictive targeting and contextual signals.

Made With Intent has no fallback by design. It doesn't need one. We've trained it across 50bn+ events, in a range of contexts, giving the model day-one predictive power on every visitor, anonymous, identified, first-time, returning.

Made With Intent and Dynamic Yield: In depth

There are lots of areas of cross-over between Made With Intent and Dynamic Yield. The way our technologies work is similar in principle, but different in its practical implementation.

Let's start first with how Dynamic Yield's prediction capabilities work:

How Dynamic Yield's prediction works

While we're talking about AdaptML, we have to say, it's a really sophisticated bit of engineering. It utilises recurrent neural networks and NLP models to get smarter and smarter as it consume more data its got on a specific retailer's visitors.

However, it predicts what you'd expect (affinity and relevance), not what's happening in a visitor's decision right now.

Our model is a driven by a single purpose: it reads buying intent. Sure, it's a narrower job, but it's the one that determines whether a visitor converts, hesitates, or leaves.

Made With Intent vs Dynamic Yield

When you should pick Dynamic Yield

Hey, we're not here to blindly sell you Made With Intent. Sometimes, our tool just isn't right for your business. So, unlike loads of other SaaS vendors, let us tell you when we wouldn't be a good fit:

• If you're trying to do cross-channel, Dynamic Yield is the right choice versus Made With Intent. We're web-first, and for multi-channel personalisation programmes Dynamic Yield is the right option.

• Recommendations: NextML and AffinityML are purpose built models for product and content recommendations. Made With Intent adds an intent decison later but it is not a replacement for NextML or AffinityML.

• Sophisticated email personalisation: Klaviyo-grade dynamic content across email and ad placements out of the box. If email is a key part of what you do, Dynamic Yield is a top choice.

• Security of a big vendor: Dynamic Yield is an eight-time Gartner Magic Quadrant Leader for personalisation engines. If you're enterprise org and need to run a formal RFP, this helps your procurement team make a decision.

• Strong experience builder. Sections, Page Contexts, and Selectors give users very fine control. But only after you've climbed the initial learning curve.

Where Made With Intent wins

Of course, this is an article designed to help you pick between Dynamic Yield and Made With Intent. From our table above, it's clear there's plenty of areas of overlap, but there's some stuff we do that, we don't mind saying, makes us a better choice. Have a read:

• Acts on the anonymous majority: Every visitor gets a multi-dimensional intent read from the first pageview. No prior data, no behavioural accrual, no geo fallback.

• Multi-dimensional intent prediction: The way we predict intent is comprehensive. We use six measures: stage, signals, trends, purchase confidence, abandon risk and shopper mindset. These are all updated every three-five seconds, unlike Dynamic Yield.

• Autonomously runs campaigns and makes decisions under directives: Agentic Campaigns can dynamically allocate experiences across hundreds of intent combinations during a campaign you'll run. Dynamic Yields's models are sophisticated. The targeting layer above them is still rule-driven. That's simply how the tool is built. But it means what experiences are served is decided before the campaign runs, and not during it.

• Cross-merchant model: 50 billion+ events across 150+ retailers. Day-one predictive power for our customers.

• Start within 30 minutes: Single 7kb GTM tag, no PII and we're ISO 27001 accredited.

So, it's time to make a decision: Let's pick one. Or both?

It's crunch time. We've given you all the facts, but it's time to wrap up and make a judgment call.

Pick Dynamic Yield if the you want consistent cross-channel personalisation — content and recommendations spanning web, mobile app, email, and kiosk. And if you have the technical resources to operate it.

Dynamic Yield's breadth and recommendation algorithm depth is class-leading, and trying to replicate the scale of that capability with Made With intent, plus some integrations would be a worse outcome for you.

Choose Made With Intent if you want a on-site decisioning, intent-driven experiences, and proven incrementality. Our tool delivers experiences natively, allocates them dynamically across hundreds of intent combinations, and proves causal lift on every one.

Now, something to think about. If you're an enterprise customer, you actually could consider both. And here's why:

• Dynamic Yield gives you the breadth and means of channels and mediums to serve personalised content to

• Made With Intent helps you accurately decide who, and when that content should be served

That's brought us to the end of the comparison blog post. If you're still unsure of the differences between Dynamic Yield and Made With Intent, the best thing for you to do is talk with one of our team.

You can book a demo here.

Disclaimer: This comparison is based on publicly available information from Dynamic Yield's documentation, marketing site, and customer reviews as of April 2026. Both products evolve continuously. If anything looks out of date, get in touch and we'll sort it.

Your A/B testing programme just found a winner. A new discount banner lifted conversion by a healthy margin, the test ran long enough to be statistically significant, and the result is sitting in a dashboard somewhere with a green checkmark next to it. Mission accomplished, right?

For most eCommerce teams, the default answer is the same: roll it out to 100% of visitors.

That decision gets made without a second thought, because it doesn't feel like a decision at all. It feels like the natural conclusion of a successful test. You've gone and validated your hypothesis. And when you've got a winner, there's a paper trail to an uplift.

It isn't. It's a choice, and for most brands it's the wrong one.

A/B testing tells you which experience wins across your traffic as a whole. It doesn't tell you which specific visitor should see that winner, or when. That's a different question, that's answered in an entirely different way. It's a decision that requires you needing to know something about the person you're serving an experience to in the first place.

Let's be clear — A/B testing definitely has a place

A/B testing tools do exactly what they're built to do, and they do it well. They replace opinion with evidence. A test that runs to statistical significance tells you, with confidence, that variant B outperforms variant A on your website or mobile app.

Made With Intent customers continue to use tools like Convert, VWO, and AB Tasty alongside our tool, because testing answers a question nothing else can: does this specific change move the number we care about?

Think about what a test actually proves. It proves that, across hundreds or thousands of visitors, variant B beat variant A. It says nothing about any single visitor in front of you right now. Statistical significance is for large volumes of people not a prediction about the individual standing at your checkout, which is a limitation CRO metrics share more broadly.

The difference between A/B testing tools and Made With Intent

Should 100% of people get your A/B test winner?

Once a test determines a winner, most teams roll that experience out broadly, often to every visitor who fits the original test's targeting. Many testing tools do offer segment targeting: device type, geography, new versus returning.

Some brands go further and layer in basic rules, like showing a winning banner only after 30 seconds on site, or only to visitors on their third page view. These rules feel like personalisation. They're really just a slightly more granular version of the same default: a fixed condition, checked once, applied broadly.

But look closely at how it gets set up. Teams choose segments once, at test design time, and then leave them alone. Even a well-built intent-based segmentation programme needs continuous application, not just at launch, or it drifts back into the same static-rule problem.

A rule that says "show this to returning visitors" doesn't change three minutes later when a returning visitor starts behaving like someone about to abandon their basket, or like someone who was going to buy anyway with or without an incentive.

The real problem is narrower and more difficult to solve. The targeting decision is frozen at the moment the test launches, while the person in front of it keeps changing their mind. (Their intent)

A discount that lifts conversion in aggregate can still be the wrong call for two specific people in your test group. A first-time visitor who's price-sensitive and hesitating needs it. A loyal customer who's already three products deep into checkout, on their fourth order this year, does not.

It's the same tension at the heart of whether discounting is a race to the bottom or a genuine growth lever. There's no problem with offering discounts, but it can be a blunt instrument or entirely inappropriate depending on where your customer is at in their buying journey.

How reading intent allows you to respond immediately to changing demands

A/B testing tools make their decision once, when the page loads or the session starts. Whatever segment a visitor falls into at that moment determines what they see for the rest of that test.

But purchase intent isn't fixed at page load. A visitor can arrive browsing casually, spend four minutes comparing two products, add one to their basket, then hesitate at the price for 30 seconds before starting to type a discount code into Google in another tab. That's not a static segment. That's a person, moving in real time, and most testing infrastructure was never built to react to this situation as it unfolds.

Some testing and CRO tools react to single behavioural triggers: an exit-intent popup fired on mouse movement toward the browser bar, a scroll-based banner, a basic engagement score. We think those are great, and definitely worth using. However, they're single-trigger rules, checked once a condition is met, and not a continuous read on a person as they use your website and apps.

What real-time behavioural data adds

Made With Intent's platform tracks over 900+ behavioural signals through each visitor's session, including scroll speed, click sequences, time spent, and product comparison patterns, updating its prediction of purchase intent continuously rather than once. When intent shifts, the response can shift with it.

None of this means every session tells a dramatic story. Plenty of visitors browse, leave, and never come close to converting in that session, and no amount of real-time signal turns a casual browser into a buyer. The claim isn't that every visitor is secretly ready to buy. It's that the visitors who are ready look different from the ones who aren't, and a rule set once at test launch can't tell the difference between them three minutes into the session.

What real-time intent-based deployment looks like

Picture the same winning discount from earlier. Instead of firing it to every visitor who matches a fixed segment, an intent-based system asks a narrower question: Is this specific visitor showing signals of hesitation or exit risk right now, or are they already moving smoothly toward checkout?

Made With Intent's Intent Scoring analyses those signals continuously and its Timing Engine decides the moment to act, rather than firing on a page-load rule or a fixed timer. A hesitant first-time visitor comparing prices across tabs might see the discount. A returning customer already at the payment page, showing no hesitation signals at all, doesn't need it, and doesn't get it. This is what discounting with intent looks like in practice: the same offer, reserved for the visitors who actually need it to convert.

The winning experience from your test doesn't change. Who receives it, and when, does.

Take a product page test that proved a scarcity message ("only 3 left") increased add-to-basket rate. Fired at everyone, it also lands on a loyal customer who's bought the same product line four times before and knows exactly what they want. To that visitor, it reads as pressure, not information. Deployed by intent, the same message is reserved for the visitors actually showing comparison and hesitation behaviour, the ones it was designed to nudge in the first place.

How an actual company used Made With Intent alongside A/B testing

Appliances Direct, a UK home appliances retailer, ran into a version of this problem. Their discounting was broad and largely untargeted; offers went out early in the visitor journey, without real-time context, regardless of whether a given shopper actually needed the incentive to convert.

Using Made With Intent, they began reserving discounts for visitors showing genuine exit and abandonment signals, rather than firing them to everyone who matched a static segment. They left everyone else to convert at full price, as they were always going to.

In the end, they made a 42% saving in margin previously given away to visitors who would have converted regardless, without a drop in conversions from the visitors who needed the discount to purchase.

But don't take our word for it. Take a look at our Appliances Direct case study and make up your mind yourself.

That figure lines up with a broader pattern. Made With Intent's own Intent Gap research found that 83% of shoppers have used a discount code despite being ready to pay full price.

Why new test winners are rare

That scarcity of new winners isn't unique to Appliances Direct. Research from Microsoft's own experimentation team, drawn from thousands of controlled experiments at Bing and elsewhere, found that only 10 to 20% of test ideas move the metric they were designed to move, and that a genuine breakthrough, the kind of result that reshapes a programme rather than nudging it, shows up in roughly one in 500 experiments. The findings come from Kohavi et al.'s Seven Rules of Thumb for Web Site Experimenters (2014). Most of what a mature testing programme finds after the early wins are used up is small, single-digit-percentage gains, not new step changes.

Testing tells you what works. Intent tells you for whom, right now.

None of this is an argument against testing. It's an argument for finishing the job testing starts.

Your A/B testing programme answers one question well: does this experience outperform the alternative? What it was never built to answer is a second, quieter question that gets decided by default instead of by design: who should see this winning experience, and at what moment in their session?

Testing finds your best experience. Making sure it reaches the right person, at the right moment, without giving away margin to people who didn't need it, is a different problem, and it needs a different kind of data to solve.

If you want to see what your existing winning experiences would look like deployed by real-time intent instead of a blanket rule, book a demo and we'll walk through it against your own traffic.