Just want to add. I have had the guys from scraping hub work on a small project for me. A custom scrape of a large site. They quoted a reasonable fee and had the project delivered in just a few days. Couldn't be happier with the work they did for me
On Wednesday, March 12, 2014 6:54:22 AM UTC+11, Joseph Piscal wrote: > > Greetings Scrapy Enthusiasts, > > I have been searching for a reliable and experienced developer to develop > a web crawler/data scraper for some time. I work for a marketing company > and have a list of 200+ consumer product URLS (blogs, e-commerce stores, > home shopping networks, & big sites like amazon/pintrest) that I would be > interested in scraping. Information I desire is product image, product > price, product description, link to purchase, etc. I would like the > information presented to me in a web RSS feed type format (attached) for > ease of sifting through products quickly. After the base program is built I > would also be interested in making it more intelligent. Utilizing a keyword > or weighting system to filter out products I don't want to see. > > I spoke to one developer who claimed scraping blogs would be easy since > they mostly run on Wordpress, or at least provide an RSS feed so monitoring > would be simple. He also mentioned that it would be difficult to do the > e-commerce sites because almost all of them utilize a different platform. > So we would pretty much require custom code for every single site. Not sure > how true this is but it made sense. > > The goal is for me to see new data daily. After a URL has been scraped the > first time, I would assume all the products (data) would be stored in a > data base. So the next time that URL is scraped it only collects the NEW > products (data) for my viewing pleasure. > > I am looking for any and all advice or suggestions on this project. If you > are interested please feel free to respond to this post or contact me > directly: > > [email protected] <javascript:> > > Thank you in advance! > > > > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
