tips

Know when Googlebot visits your site/blog

10 Mar , 2010  

Hey Folks,

A webmaster is always curious to know about the visitors on his website such as the number of visits, pageviews etc. Another important thing which needs to be taken care of is when do the Googlebot or crawler visits your website. Albeit, this information can be collected from Google’s webmaster tools but in some cases we need to know instantly when the google bot visits  your website.

So for that insert the following PHP code on your webpage. You can insert this code on the index page and it can be inserted anywhere on the page, may be in the header or footer. This code simply detects the term “Googlebot” in the useragent supplied and then shoots an email to the specified email address. You can also extend that code to store the information to the database and can analyze afterwards.

<?php
if ( strpos( $_SERVER['HTTP_USER_AGENT'], 'Googlebot' ) !== false )
{
// paste your email address here
  $my_email = 'your_email_address@email.com';
// notify via email
  mail($my_email,'[Notification]Googlebot Visit', 'Googlebot has just visited your website WEBSITE_NAME: '.$_SERVER['REQUEST_URI']);
}
?>

Just a note, anyone can easily fake the useragent to pretend as Google bot and can send mass mail to your email address. So its a best practice to do some validation before you implement this script, you can try inserting the IP of the user in the database with corresponding timestamp.

This script is not limited to only Googlebot, you can replace the useragent “Googlebot” with Yahoo, MSN etc if you are interested in knowing when their crawler(s) visit.

Next time Googlebot visits your website, you will be pinged to welcome it.

Hope you learnt something new.

Stay Digified !!

Sachin Khosla.

, ,


9 Responses to “Know when Googlebot visits your site/blog”

  1. Patrick says:

    I think I just learned how to bomb your mailbox.

    As you said. Everyone can fake their User agent, so it might be wise to put some additional checks in that snippet. Testing the IP address for being in Google’s subnet may be a start.

  2. Yeah, and that varies from person to person how secure he can make the script. You are most welcome to *bomb* my mailbox :)

  3. Alap says:

    Excellent Post.

    Very Informative. Useful for SEO?

  4. ESN says:

    That’s a cool idea Scahin. How many of those emails have you received this week? I am interested to find out how ften google bot visits a blog.

  5. bmoc says:

    GoogleBot isn’t the only bot that visits your website, there are plenty of others crawlers – check here http://www.user-agents.org/

  6. yeah sort of, once can keep track of the bots visiting his/her website.

  7. Haven’t checked it yet myself.

  8. Thanks for the link bmoc, going to help visitors pretty much.

  9. Hangar 17 says:

    Good informative. If i need to track multiple bots then i need to write more codes right?… but its not secure script??… how do i check this???…

More in tips (88 of 132 articles)


Hey folks, Wonder what are the best SEO techniques ? Wonder what should be the font-size of your blog, title ...