U can use dictionary or linked list ............ Better if U choose language like python or java.
Python have module named "Urllib2" and "httplib2" which implements all the functions for getiing ,posting and browsing data. INFORMAL ALGORITHM...... 1. Start with any arbitray link. LINK=[new link] 2.a>Get html content of the link . b>Parse the required Content and store it . c>Add the new link on the page to the LINK it it is not present there. 3.Repeat step 2 untill U want to crawl. On 2/5/12, Ravi Ranjan <ravi.cool2...@gmail.com> wrote: > what will the algorithm and the appropriate data structure to implement a > web crawler?? > > -- > You received this message because you are subscribed to the Google Groups > "Algorithm Geeks" group. > To post to this group, send email to algogeeks@googlegroups.com. > To unsubscribe from this group, send email to > algogeeks+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/algogeeks?hl=en. > > -- *Durgesh Kumar* Final Year, B.tech Information Technology HALDIA INSTITUTE OF TCHNOLOGY HALDIA -- You received this message because you are subscribed to the Google Groups "Algorithm Geeks" group. To post to this group, send email to algogeeks@googlegroups.com. To unsubscribe from this group, send email to algogeeks+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/algogeeks?hl=en.
