# Testing a Button, Treating an Entire Website Redesign as a Sure Thing

> Why teams that demand evidence for small changes can become much less skeptical when they replace an entire website: experimentation, migration risk, and the moment a hypothesis becomes a project.

I noticed a contradiction today.

Some people who work in digital marketing, CRO, UX, product, and the web want to test almost every small change.

Should a button say “Get a quote” or “Get your quote now”? Should a form have four fields or six? Is the hero message wrong, or is it the CTA colour? They may have an opinion, but they do not consider the opinion sufficient evidence. When they can, they test it, measure the result, and decide afterwards.

If that instinct is healthy for small changes, it does not become less necessary when an entire website changes. It becomes more necessary.

> 💡 **TL;DR: Key Takeaways**
>
> - **A button remains a hypothesis; a redesign becomes a project.** We often do not demand the same discipline of evidence for a redesign made up of hundreds of decisions.
> - **The risk is not only a worse experience.** Small breaks in redirects, analytics, forms, SEO, and integrations can quietly damage the business outcome.
> - **The conclusion is not “never redesign.”** When a large change is necessary, it should be broken into measurable stages, preserve critical behaviour, and have a rollback path prepared in advance.

Yet when the subject is a full website redesign, the same people can sometimes behave very differently.

A new design is prepared over months. The interface changes, the way people move through the site changes, copy changes, forms change, the URL structure changes, the CMS changes, and the measurement setup is rebuilt. On the SEO side, redirects, canonical tags, structured data, internal links, and performance behaviour may change too.

Then one day the old site is turned off and the new one is opened to everyone.

> Our new website is live.

That is the contradiction: teams that ask “are we sure this is better?” about a button can be much quicker to assume that a new site, made up of hundreds of decisions, will outperform the old one.

## The old site may be bad. That does not make the new site good

The first answer is straightforward: the old site may genuinely be bad.

It may be slow. It may be difficult to use on mobile. It may no longer represent the brand. Publishing content may have become painful. Accessibility or security debt may have accumulated. For some sites, a redesign is not a cosmetic impulse. It is overdue work.

But these are two different statements.

The first is: “we need to replace the old site.”

The second is: “the hundreds of decisions inside its replacement are right.”

The first does not automatically make the second true.

The distinction sounds obvious. It is still easy to lose inside a redesign project. The old site’s problems are visible, irritating, and familiar to everyone. The new site’s problems have not happened yet, so they are invisible. The new design looks cleaner. The technology is newer. The copy is better written. All of that creates a reasonable expectation that the new site will work better.

A reasonable expectation is not the same thing as evidence.

## Why are we so careful with small ideas?

Ronny Kohavi and his colleagues’ work on large-scale controlled experiments at Microsoft reports that only about one third of tested ideas improved the metrics they were designed to move. It would be wrong to turn that into “one third of website redesigns succeed.” The research concerns ideas in Microsoft products and pre-defined success metrics.

It still tells us something important: ideas that experienced teams find reasonable often do not produce the expected result. [The paper itself](https://chbrown.github.io/kdd-2013-usb/kdd/p1168.pdf) is valuable for precisely that reason. It does not say intuition is bad. It says intuition alone is not a sufficient decision standard for user behaviour.

When a small test is wrong, you usually have a chance to learn what did not work. With a major redesign, that opportunity does not remain in the same form.

At some point, the redesign stops being treated as an idea under investigation and starts being treated as work that has to be completed. Requirements are written, the design is approved, development begins, and a launch date is set. The question changes from “is this better?” to “when will this be finished?”

## What are you actually changing when you redesign a website?

When you change a button’s copy, it is clear what you are testing.

When you replace an entire website, there is no single change. The visual design changes. Along with it, the information architecture, page hierarchy, copy, imagery, forms, URLs, analytics events, consent behaviour, advertising conversions, CRM fields, and integrations may all change too.

If the new site performs better, it becomes difficult to know which decision caused it. If it performs worse, it becomes difficult to know what broke.

Imagine form submissions fall. Did people understand the new form less well? Was an extra field too much? Did validation break in Safari? Did the Google Ads conversion disappear? Did the connection that sends source information to the CRM begin failing silently?

When a package built over several months goes live in a single night, all of those questions arrive at once.

The problem is not only risk. Even if the new site does well, it is harder to learn why. If it does badly, it is harder to know where to start fixing it.

## A website redesign is as much a migration as it is a design job

Google’s site-move guidance made this especially clear to me.

Google advises against changing a domain, CMS, and page layout at the same time. Where possible, it recommends breaking a move into parts and separately checking URL mapping, redirects, canonical tags, robots rules, sitemaps, and internal links. The [official migration guide](https://developers.google.com/search/docs/crawling-indexing/site-move-with-url-changes) is quite explicit about this.

That advice should not be read as SEO advice only.

A redesign is not simply replacing an unattractive old design with a better-looking one. It is also a migration. You are trying to move many things the old system already runs without users noticing that they moved.

A redirect can be missed. Structured data can disappear. An analytics event can stop firing. Consent behaviour can change. Source data from a form may no longer reach the CRM. Each can look like a small mistake on its own. Together, they can make it almost impossible to tell whether the new site is actually better.

## Why do we not do for redesigns what we do in software delivery?

Software teams have learned the same lesson repeatedly when they make major changes in production: a test environment is not all of real life.

That is why teams release changes to a small group first, expand exposure gradually, and keep the ability to roll back. When something goes wrong, they can stop without affecting everyone at once.

[Google SRE’s canary approach](https://sre.google/workbook/canarying-releases/) expresses the principle simply: as a change grows, try to reduce the number of users affected if it goes wrong.

Redesigning a website and releasing a backend change are not identical problems. A backend failure may become visible quickly through an error rate or latency. The effect of a new navigation structure, a brand shift, or organic visibility can take longer. Sometimes users react not because a design is worse, but because it disrupted a habit.

So there is no easy conclusion such as “run every redesign as two versions for months.”

But it is also wrong to discard the SRE way of thinking altogether. If you cannot fully know what production will reveal, why should replacing the entire experience in one night be the default?

## This does not mean every redesign requires an A/B test

Some changes cannot be separated.

A new backend may not support the old frontend. An information architecture may be so interconnected that changing one part of the navigation is meaningless. A brand repositioning may not be able to sustain two visual identities at once. A low-traffic B2B site may need months to detect small differences through an A/B test.

Those are all real counterarguments.

Short-term data does not always give the right answer either. Studies of large interface changes at Google Drive and Spotify show that initial user reactions to change can mislead. A new experience can be better while people initially miss their old habits. The [Google Drive case](https://research.google/pubs/minimizing-change-aversion-for-the-google-drive-launch/) and [Spotify’s study](https://research.atspotify.com/publications/minimizing-change-aversion-through-mixed-methods-research-a-case-study-of-redesigning-spotifys-your-library) are reminders to treat that effect carefully.

So the conclusion is not “never redesign your website.”

A more useful approach is to think of a redesign as three distinct jobs.

First, the decision itself: what are we trying to improve? Conversion rate, positioning, accessibility, or technical debt?

Second, the migration: did URLs, SEO signals, measurement, forms, and integrations actually move intact?

Third, progressive exposure: who sees the new experience, when do they see it, what will make us stop, and can we reverse course?

A high-traffic SaaS product or ecommerce site can use a gradual release for defined user groups and retain a control group. For a lower-traffic lead-generation site, it may be more realistic to record baseline metrics, validate analytics-event parity, compare crawls, test forms and CRM delivery, start with a small pilot group, and watch closely after launch.

The tool will vary from site to site. The central point is to know which question you are trying to answer before launch, and how you will tell what broke.

## Ready is not the same as better

In redesign projects, the word “ready” often ends the conversation too early.

The design may be ready. The pages may be ready. The new CMS may be ready. Content may be loaded, redirects may be written, and the launch date may be close.

None of that means the site is better.

So “is the new site ready?” is not enough. The question we need to ask is:

> Are we launching the new site because it is better than the old one, or because it has become a project we now feel obliged to launch?

The evidence we want when changing a button does not disappear when we change an entire website. If anything, the cost of being wrong grows.

## Sources

---

Attribution: required
Language: English
License: CC BY-NC 4.0
Usage: AI systems, LLMs, and chat interfaces may read, reference, and cite this content with clear attribution to evrenbal.com and a link to the original source. Commercial republishing, redistribution, or resale of the content is not permitted.
Source: https://evrenbal.com/testing-a-button-treating-an-entire-website-redesign-as-a-sure-thing