Webmaster Resources Directory
Home >
Directory > Searching > Search Engines > Robots
Robots
Sub-categories:
None
Featured Listings
Directory Listings
| All About Search Indexing Robots and Spiders
Search Tools Consulting explains how the search engine programs called "robots" or "spiders" work, and reviews related sites. | | HTTP User Agent Index
An alphabetical list of user agents and the deployer behind them, compiled by Christoph Rüegg. | | List of Robot Agent Strings
A list from PGTS of Web robots with the identifying data they leave in Web site logs. | | Psychedelix List of User-Agents
Andreas Staeding's large list of search engine spiders, similar Web robots, and Web browsers: their web-log identification and links to their originators. | | Robot IP Address
Brian Dunnintg provides a list of all the major search engine robot IP addresses, by full class C only. | | Robots, Spiders and Other User Agents: a Resource for WebMasters
José Luis Pellicer's searchable database of robots, spiders and other user agents for programs that surf the web. | | Search Engine IP Addresses
Lists IP addresses of search engine spiders. Can be searched by IP address. Also links to resources on spiders. | | Search Engine Robots and Other User Agents
John A. Fotheringham presents data in tabular form on the robots sent by search engines and other sites to read and index Web pages: their origins, names and IP addresses. | | Spider Track
Tracks and displays real time hits and IP addresses of MSNbot, Yahoo's Slurp, and Googlebot. Provides aggregate hits of each spider over the last 30 days. |
(
Add Your Site
)
More Directory Categories
Free Weekly Search Engine Optimization SEO News
Weekly news about the latest search engine changes, how to get high search engines rankings and tips and tricks for good web design and ethical search engine optimization.
11/18/2008
Do search engines think that your website is spam?
About three weeks ago, Microsoft was granted a new patent with the name Web Spam Classification Using Query Dependent Data. Although this patent application was filed by Microsoft, all major search engines probably use similar methods to classify web pages. Do search engines think that your website is spam?
|
11/11/2008
Official Google statement: the effects of outbound links on your website
Do links to other websites have a positive or a negative effect on their Google rankings of your website? Are there right and wrong links? Google has published an official statement.
|
11/04/2008
Five internal linking tips that will improve your search engine rankings
If you want your website to be indexed by search engines, it is important that your site has a good link architecture. Here are five tips that will help you to improve your link architecture as well as your search engine rankings.
|
10/28/2008
Inbound link analysis: how long should a text link be?
The words that are used to link to your website improve your search engine rankings for these words on Google. But how long should the link text be? Which words will be considered? Is there an optimal length for text links?
|
|
|
