However all your suggestions are relevant and they’re all good too. I don’t use an xml sitemap because I haven’t felt the need. I may start using one in the future though. All the rest of your suggestions I do follow.

Maybe I should update this post one of these days or at least mention at the top how old it is and that much of it may no longer be an issue, especially with all the tools available today.

My first time here and have just spent 15 minutes reading various bits & pieces

Anyway I’m just a little curious & have to ask if you use All In One SEO plugin..? Because surely many of the issues mentioned here are dealt with by AIOSP.

On the subject of people not realising there is a problem, a simple check of their site map on Google Webmasters would very easily confirm if there were issues with a site e.g I currently have 200 pages submitted in my site map & around 170+ currently indexed (which is about the norm), but if anyone was to find the majority of their pages were not indexed then that is a sure sign of issues that need dealing with.

Finally just a few pointers to help beat duplicate content;
1. Always use excerpts on your home / index page
2. Always use excerpts / summary in your feeds, this also helps prevent content theft.
3. Always use a top quality SEO Plugin such as All In One SEO or Total SEO.
4. If you have any doubt about whether a page/post/article is original, then just copy all the text and feed it to Plagium.com it is free to use and will check the entire web for any duplicate of the text you submit.

Hope that helps

Steve (Taffy)

I’ve seen people suggesting robots.txt not being the best way to eliminate duplicate content too. There was a video by Matt Cutts a couple years back talking about the different ways Google might find urls on your site. None of the methods for blocking certain pages from getting indexed was 100% effective and Matt suggested trying a few to make sure the page didn’t get crawled.

Some methods would be robots.txt, meta noindex, meta canonical. I think you have to look at what each does in order to decide which is best to use and I think it’s ok to combine techniques.

robots.txt is a quick way to tell search engines not to crawl everything inside a directory for example, but if other sites are linking to those pages then they might still get crawled. Adding noindex should ensure the page doesn’t get indexed.

PR comes from links pointing into your page. It’s also important to note that Google only updates the PR we see in the toolbar every few months. They just updated over the weekend so you may be seeing PR for your blog now.

Google and WordPress have also changed things in regards to the feeds and the supplemental index since I wrote this post. While I still think it important to eliminate duplicate content from your site I don’t know that what I described here is the same problem it was a couple years ago.

The funny thing is I have gotten to #1 on a few Google searches yet still no PR value? I guess I have a lot to learn. (I mainly blog for the fun of it.)