Shocked?? Yeah, I was too when a friend of mine, Bishal Adhikary told me about this WP bug. Here’s the scenario. If you are using /%category%/%postname%/ permalink structure at your WP blog, and let’s say the URL of one of your posts is yourdomainname/some-category-name/post-name. Now, the problem with using /%category%/%postname%/ permalink structure is that if you replace the category name in the real URL of your post (not the one given above) with something else (any word you can imagine), it still takes you to the exact same post without any redirections. That means if you are using /%category%/%postname%/ permalink at your blog and if you haven’t done anything to get rid of the scenario I explained to you earlier, there’s bad news for you, YOU HAVE GOT INFINTE DUPLICATE CONTENT AT YOUR BLOG.
I was amazed when Bishal showed me some of the blogs that I admire and that use /%category%/%postname%/ permalink still hadn’t done nothing to get rid of the duplicate content generated by this permalink structure. Say for example, labnol.org still has infinite duplicate content at the time I am writing this post. Labnol averages more than 2 million page views per month. Bishal is planning to write an email to labnol soon, so I guess labnol would deal with this problem soon and you might not see the same effect as I talked about.
When using the /%category%/%postname%/ permalink, problems don’t seem to arise until someone links to your post with a URL address that you didn’t intend to be linked with but which still take them to the intended content. What kind of URLs are those? Say for example, URLs with everything else being the same except for the category name on the URL. Google does not bother to crawl your pages with these different URLs (that take you to the same page) which is actually good but this applies only until someone links to you with a URL that you didn’t intend to be linked with. My posts have never been linked by someone with other URLs than my intended URLs, so despite having infinite URLs that lead to the same content for over a month at this blog, none of those URLs made to Google’s index. But things are not the same for everyone. This might actually be a dangerous scenario for some of you because as competitors are trying to outrank you and gain an edge over you trying everything possible, this WP vulnerability would make life easy for them for they can make Google think that you have placed duplicate content at your blog and after that it won’t take long for Google to come and get you.
So, what’s the solution?
The solution is not tough at all considering the scale of this problem. Go and get yourself Scott Yang’s Permanent Redirect WordPress Plugin. Upload the files and activate it and voila, it solves the problem. It adds a 301 redirection to all those “bad URLs” that you never intended to lead to your post, without you having to do any work other than of course, activating it. This plugin has one more advantage. As this plugin identifies a post taking into consideration the end part of the URL of the post (something that we are used to calling slugs in WP), if someone links to you with some misspellings in the URL but with a correctly spelled slug, you would still get a link back as this plugin would 301 redirect the misspelled URL to the correct post. So go ahead and install this plugin if you are using /%category%/%postname%/ permalink structure. Sit back and enjoy blogging until someone discovers some other deadly vulnerability in WP.
I still love WP though and would keep loving it till eternity. WP is like your wife, groom her more and it’s you who would ultimately enjoy it more than her.
[...] Hard to imagine that if you type domain.com/2008/your-post/ or domain.com/FAIL/your-post/ that Google may end up indexing both and therefore create the dreaded duplicate content scenario! But it’s true, try it on your own blog…and if you want more details on this, you can check out this blog post on permalink structure and duplicate content. [...]
Hi Ghatozkat,
Very interesting and useful article.
I note that you havent applied permalink migration plugin.
Can you please tell me the solution for this problem
Reply to this comment
Thanks for the post, I recently had all my articles double-indexed by Google both under category/postname and default-category/postname permalinks and was worried about the duplicate content issue.
I think that this plugin will take care of this issue as the second one now 301 redirects to the first one.
Thanks again.
Reply to this comment
thank this post I get to slove my problem
Reply to this comment