WillyBot is the name of WillyFogg.com's indexing agent. This is the web-crawler that automatically searches and indexes products that are currently being sold on the Internet.

WillyBot uses a load spreading technique that reduces the load on the servers of the web site that is being indexed. The effect of this is that our crawler will not put any noticeable strain on these servers.

If you want to submit your shop for indexing - Submit Shop to index.

If you want to remove your shop from index - Remove Shop from index.

If you want to avoid your products from being indexed, use one of the following methods:

1. add willybot to your robots.txt (The Robot Exclusion Standard)
   
   User-agent: willybot
   Disallow: /
   
   
2. add META tag to your HTML pages
   
   <meta name="willybot" content="noindex,nofollow">
   
   or
   
   <meta name="robots" content="noindex,nofollow">
   
   
   

Do you have any questions for us? Contact us.