I have taught my visitors before on how to properly implement their Robots.txt and just today AdSense just warned their Publisher to check their Robots.txt file in order not to block some Ads, They have explained that Google BOT is different from AdSense BOT so you better add the following codes in your robots.txt file, if you are using Robots.txt for your blog.
User-agent: Mediapartners-Google*
Disallow:
This only means that you are allowing crawl from Google Media Partners. This will also possibly increase relevant ads for your blog.
BTW I have added mine. You can check my Robots.txt file here don’t forget to check my Adsense Tips before, It still help me a lot to get a relevant ads. And I am sure that michigan SEO Consultants will agree with my findings.
Thanks for the tips Dex. 🙂
Fitzs last blog post..The Relationship Of Obsolescence And Frugality Part 1
@ Fitz,
Generally I believe your robots.txt is fine, One thing I have seen is your robots.txt file does not restrict your page Page..Since your page ie.. http://fitzvillafuerte.com/page/2 is in excerpt it might also be considered as duplicate pages. Fo my site i have restricted it using as follows :
Disallow: /page/*/
Your Category page is also in excerpt mode.. you can also add
Disallow: /category/*/
this is to restrict access to your category page.
Above mentioned suggestion is only applicable if you don’t want your category and “page” page to be indexed but if not your robots.txt is fine
Dexters last blog post..Better Check Your Robots.txt says Adsense
I have added that line in my robot.txt Thanks for the great tip. I hope my adsense earning will be greater. 😀
Husseins last blog post..What Motivates You To Blog?
I read this over at Inside Adsense blog just a while ago, I was wondering if I should do it because I usually see relevant ads for my pages. But you did it, so I would too. Shempre, idol kita sa SEO eh. Hehehe. If you have time, can you check my robot.txt file and see if it’s okay. Thank Dex. 😀
Fitzs last blog post..Things To Do When Dealing With Frustration