- - - - - - - - - - - - - - - - - - - - - - - - - - - -
Name: Maxime
Subject: Re: parameters in URL

To crawl the net, you need to collect links from other pages, for example you 
need to HrefOnly .htm and .html extenstions:
Allow *tag=yus-op*
HrefOnly *.htm *.html
- - - - - - - - - - - - - - - - - - - - - - - - - - - -

Read the full topic here:
http://www.dataparksearch.org/cgi-bin/simpleforum.cgi?fid=02;topic_id=1165353979

Reply via email to