Open Directory Project
home | feedback

Spider image

ODP RoboSpider

Help fixing Robozilla Reds


URL:
Domains to test:
Query:
Search target:
  

RoboSpider is a tool that helps you locate a site that has been marked red by Robozilla, or a site that has been submitted with a bad URL. The tool tests several variations that has proven good candidates for being "the right one", or for pointing you towards a functioning address: The server of the URL is tested with and without a "www" prefix, and the path is tested with file name and subdirectories removed in sequence. This is what you had to do yourself manually when trying to locate a missing link - now RoboSpider can do it for you by testing each of the selected URL variations with the DeMozulator or LinkSpider. If one of the variations does not return an error, you can simply click a link to continue your examination.

You can write a list of top level domains that RoboSpider will also test - eg. a submitted http://www.site.co.uk/page/ can also be tested as http://www.site.com/page/. Furthermore you can fill in an query string, eg. the title of the missing link, which you can send to Google (default) or other search engines as a search string. Currently AltaVista, Teoma and WiseNut are available, as well as a Dmoz lookup using UrlSpider. A new address may show up that you can use instead of the erroneous one.

To integrate RoboSpider with your ODP editing you can grab a Bookmarklet here. There are two variants:
  RoboSpider this link with no additional domains
  RoboSpider this link, testing also com, net and org domains
Each will invoke RoboSpider automatically from any ODP edit screen, filling out the URL and title for you. You can adapt the bookmarklet to your default needs by editing the dom variable in tha JavaScript source.

If the RoboSpider test could not help you locate the missing site, a group of volunteer editors offers personal help with Robozilla Reds.

RoboSpider is one of my Spider tools. Comments or suggestions can be posted to this thread for discussion.

Spider tools by wkn.

    Copyright © 2000-04 Werner Knudsen. Last update 2004-11-28.