All SEO Plugin v1.5 has new file readme.txt included I’ll tell you more about this file and what problems it fixes.
First I offer you to read this post about Duplicate content and README.TXT was made to fix this problem. This file tells GOOGLE and other search engines what they should index and what thay should not take into consideration.
Why this is good thing:
- Search engines index your website faster if they index only pages that is importand without all Duplicate garbige.
- You will not get any penalties for the Duplicate stuff from Big G!
- this is just right thing to have website without duplicate content.
ROBOT.XTX – solve Duplicate problem for:
- Index page
- Profile page
- Pages on other languages (if website has 2 or more languages)
here is the robot.txt file content
You are free to use it and get more from SEO!