Thanks for the reply. I will look into it! On Friday, April 22, 2016 at 3:17:23 PM UTC+1, Tsouras wrote: > > Hello, > > in most cases Splash will be the bottleneck, not Scrapy. > > Probably you will need one instance of Scrapy and a cluster of Splash. > Or maybe you will need a cluster of "Scrapy+Splash" instances. > > A good start point for a cluster of Splash is to use aquarium > https://github.com/TeamHG-Memex/aquarium > > On Thursday, April 21, 2016 at 6:01:17 PM UTC+3, Alan Kavanagh wrote: >> >> Hi guys, >> >> I'm trying to determine whether using Scrapy-cluster and Splash is a >> viable option. Has anyone done this before? >> >> Thanks >> >
-- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
