There are probably a few "spider" programs out there that are free. Why reinvent the wheel? http://www.freecode.com/viewsource/acmewebspider/Spider.java/
I'm not recommending the above link, but it was the first one that popped up on a google search for Java and Spider. Good Luck, Joe Hall -----Original Message----- From: Francisco Morales Arcia [mailto:[EMAIL PROTECTED]] Sent: Thursday, May 23, 2002 6:57 AM To: JDJList Subject: [jdjlist] RE: Get dirs and files under a URL I think you could something like: 1. Get yahoo's main page, 2. Repeat 3. With an HTML parser determine all the page links ("A HREF ... " tags) 4. Get the linked pages 5. For each page -> goto 2 It's something like a tree, and each pass represents a level, but be carefull the tree expands arithmetically. hope this helps > -----Original Message----- > From: Panagiotis Plevrakis [mailto:[EMAIL PROTECTED]] > Sent: Thursday, May 23, 2002 3:11 AM > To: JDJList > Subject: [jdjlist] Get dirs and files under a URL > > > Is there any way to get all directories and/or files under a > given URL? For > example, given the URL "www.yahoo.com", I would like to > create an array of > URLs filled with all directories and files of yahoo. > > Thanks, > Panos > > _________________________________________________________________ > Join the world's largest e-mail service with MSN Hotmail. > http://www.hotmail.com > > > To change your membership options, refer to: > http://www.sys-con.com/java/list.cfm > ---------------------------------------------------------------------------- ---------- This message and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. No confidentiality or privilege is waived or lost by any wrong transmission. If you have received this message in error, please immediately destroy it and kindly notify the sender by reply email. You must not, directly or indirectly, use, disclose, distribute, print, or copy any part of this message if you are not the intended recipient. Opinions, conclusions and other information in this message that do not relate to the official business of Ydilo Advanced Voice Solutions, S.A. shall be understood as neither given nor endorsed by it. ---------------------------------------------------------------------------- ---------- To change your membership options, refer to: http://www.sys-con.com/java/list.cfm To change your membership options, refer to: http://www.sys-con.com/java/list.cfm
