Better Check Your Robots.txt says Adsense

Posted by : Dexter Panganiban on June 11, 2008 | ; Filed Under Adsense, Site Updates

I have taught my visitors before on how to properly implement their Robots.txt and just today AdSense just warned their Publisher to check their Robots.txt file in order not to block some Ads, They have explained that Google BOT is different from AdSense BOT so you better add the following codes in your robots.txt file, if you are using Robots.txt for your blog.

User-agent: Mediapartners-Google*
Disallow:

This only means that you are allowing crawl from Google Media Partners. This will also possibly increase relevant ads for your blog.

BTW I have added mine. You can check my Robots.txt file here don’t forget to check my Adsense Tips before, It still help me a lot to get a relevant ads. And I am sure that michigan SEO Consultants will agree with my findings.

Robots.txt How important it is?
It is good that I am already using WordPress self hosted blog. There is another reason to be happy, I can alter my robots.txt . For blogs hosted in WordPress.com or Blogspot.com , I am afraid this is not applicable to you guys and gals. This post will aim to educate readers on how to implement ro...
Did You know that Google Also has it’s Own Robots.txt
Did you know that Google has it’s own Robots.txt, Robots.txt is a text file that prevents search engines from accessing and indexing some file. So check it out. Well upon checking Google Robots.txt, this is what I saw. User-agent: zombies Disallow: /brains User-agent: * Allow: /searchhi...
Having Problem with The SERP
I just don't know what happen to this blog. I am now experiencing a problem with Google SERP. Actually at first I found out that I have hundreds of errors in my subok ( keyword Site ) which is part of techathand folder and suddenly most of my keywords hardly gets to the top 3 position. The maximum...

Tags: Articles, robots.txt, SEO Consultants, Site Updates

About the author

Dexter is a person that love technology,new gadget, SEO, Social Media and Christianity. Follow him at twitter via @techathand and add him @ Google+ and contact us at admin@techathand.net

Interesting Comments

4 Responses to “Better Check Your Robots.txt says Adsense”

Fitz on June 11th, 2008 7:40 pm

I read this over at Inside Adsense blog just a while ago, I was wondering if I should do it because I usually see relevant ads for my pages. But you did it, so I would too. Shempre, idol kita sa SEO eh. Hehehe. If you have time, can you check my robot.txt file and see if it’s okay. Thank Dex.

Fitzs last blog post..Things To Do When Dealing With Frustration

Reply
Hussein on June 12th, 2008 7:11 am

I have added that line in my robot.txt Thanks for the great tip. I hope my adsense earning will be greater.

Husseins last blog post..What Motivates You To Blog?

Reply
Dexter on June 12th, 2008 10:18 am

@ Fitz,

Generally I believe your robots.txt is fine, One thing I have seen is your robots.txt file does not restrict your page Page..Since your page ie.. http://fitzvillafuerte.com/page/2 is in excerpt it might also be considered as duplicate pages. Fo my site i have restricted it using as follows :

Disallow: /page/*/

Your Category page is also in excerpt mode.. you can also add

Disallow: /category/*/

this is to restrict access to your category page.

Above mentioned suggestion is only applicable if you don’t want your category and “page” page to be indexed but if not your robots.txt is fine

Dexters last blog post..Better Check Your Robots.txt says Adsense

Reply
Fitz on June 13th, 2008 4:31 am

Thanks for the tips Dex.

Fitzs last blog post..The Relationship Of Obsolescence And Frugality Part 1

Reply