I'm researching "screen scraping" or "web crawling", basically creating a tool which could activate controls on a particular web page to simulate user activity. Specifically I have an external customer website I want to access which has various dropdowns that let you drill down selections until you reach a specific set of criteria, then retrieve the data which is retrieved using that criteria.
I have the ability and authorization to access the site manually as far as credentials are concerned, I'd just like to automate the process both to make things easier for me and to help me learn about how crawling is accomplished through .NET. If someone could please help me find some sample code or help me with some additional info I'd appreciate it greatly. Thanks!!
