Duplicate content causes more unnecessary worry than any other subject in search. People imagine a penalty, and there is not one. What happens instead is quieter and more expensive.
This article explains what really goes on.
There is no duplicate content penalty
Search engines do not punish a site for having two similar pages. They have said so repeatedly, and the behaviour matches the statement.
What actually happens is a choice. When several pages say much the same thing, the search engine picks one to show and sets the others aside. Nobody penalised your page. Something else won, which looks identical from the outside and is not the same problem.
The real cost is division. Those pages split whatever authority they earned instead of concentrating it on one, so the version that wins ends up weaker than a single page would have been. If split authority is a new idea, where a page's authority comes from covers it.
The four common causes of duplicate content
On a normal site, almost all of it comes from one of these.
- Several addresses for one page. With and without www, with and without a trailing slash, http and https, or a tracking parameter attached.
- Filters and sort orders. Common on shops and listings sites, where a handful of options can generate thousands of addresses.
- Borrowed descriptions. Product text supplied by a manufacturer and used by every shop that sells the item.
- Pages written twice. Two articles aimed at the same question, usually years apart, by people who did not know about each other.
Notice that three of the four are technical accidents rather than writing problems. That is why duplicate content is so often misdiagnosed as a content issue.
How to fix each cause
Each has a different answer, and using the wrong one wastes effort.
For several addresses, pick one as the real version, redirect the rest to it permanently, and declare the canonical address on the page. That single fix resolves the majority of cases on a brochure site.
For filters, declare the clean category page as canonical and stop the pointless combinations being crawled. Keep an indexable version only where people genuinely search for that combination, such as a colour or a size.
For borrowed descriptions, rewrite the products that matter and leave the rest. Rewriting a catalogue of thousands is unrealistic, and the long tail rarely earns the effort.
For pages written twice, merge them. Pick whichever address is stronger, move the best of both into it, and redirect the other. Keeping both means they carry on competing.
What the canonical tag actually does
A canonical declaration is a hint rather than an instruction. It tells a search engine which version you consider the real one, and the search engine usually agrees.
Usually, not always. Where the page you point at differs sharply from the one carrying the declaration, or where other signals contradict it, the search engine ignores the hint. So a canonical tag pointing at your home page from every product page tidies nothing up. The search engine disregards it, which is the correct outcome for a claim that is obviously untrue.
Duplicate content across different sites
Two situations come up often and they need separating.
Syndication, where you publish an article and another site republishes it with permission, is normal. Ask for a canonical declaration pointing back at your original, or at least a link. Without either, the larger site may well take the ranking.
Scraping, where somebody copies your pages without asking, is annoying and usually harmless. Search engines are generally good at working out who published first, particularly when your version is well linked. Chasing every scraper is rarely a good use of an afternoon.
How to find duplicate content on your own site
Two free checks catch most of it.
In Search Console, look at the pages excluded from the index and read the reasons. "Duplicate, Google chose a different canonical" tells you directly which pages are being set aside and which one won.
Then take a distinctive sentence from an important page and search for it in quotes. If several of your own addresses come back, you have found the problem in ten seconds.
Duplicate content created by a redesign
Migrations produce more of this than anything else, and it is worth checking straight after one. A new site often keeps the old pages alive at their old addresses while the new versions sit at new ones, so every page exists twice.
The symptom is distinctive. Traffic does not collapse, it wobbles, because the search engine keeps changing its mind about which version to show. Redirecting the old addresses to the new ones permanently resolves it, and the sooner that happens after launch the less confusion accumulates.
When similar pages are fine
Not every repetition is duplicate content worth acting on. A boilerplate footer, a shared description on a category page, and location pages that differ mainly by place name are all ordinary.
The question to ask is whether a reader arriving on each page would find something genuinely different. If yes, the pages are fine even though a tool flagged them. If no, they were probably one page all along.
Frequently asked questions
Is there a duplicate content penalty?
No. Search engines have said so repeatedly and the behaviour matches. What happens instead is that they pick one version to show and set the others aside, so your page is passed over rather than punished. From the outside those look identical.
What is the real cost of duplicate content?
Division. Whatever authority those pages earned gets split between them instead of concentrating on one, so the version that does get shown ends up weaker than a single page would have been. That is a quiet cost rather than a dramatic one.
What causes duplicate content on most sites?
Four things: one page reachable at several addresses, filters and sort orders generating thousands of variations, product descriptions supplied by a manufacturer, and two articles written years apart aimed at the same question. Three of the four are technical accidents.
What does a canonical tag do?
It tells a search engine which version of a page you consider the real one. It is a hint rather than an instruction, so a declaration pointing somewhere obviously unrelated gets ignored, which is the correct outcome for a claim that is not true.
Can I use the manufacturer description on my product pages?
You can, and your page is then identical to every other shop selling the item, so a search engine has to choose between them. Rewriting a whole catalogue is unrealistic, so rewrite the products that sell or carry margin and leave the long tail.
What should I do if somebody copies my content?
Usually nothing. Search engines are generally good at working out who published first, especially when your version is well linked. Chasing every scraper is rarely worth an afternoon. Syndication with permission is different and deserves a canonical pointing back to you.
How do I find duplicate content on my site?
Read the excluded pages report in Search Console, where "Duplicate, Google chose a different canonical" names the pages being set aside. Then search a distinctive sentence from an important page in quotes and see how many of your own addresses come back.
Are location pages duplicate content?
Only if they differ by nothing but a place name. The test is whether a reader arriving on each would find something genuinely different: local knowledge, different services, real examples. If the answer is no, those pages were probably always one page.