Local filesystem exporter is blocking, the others use deferToThread to run the blocking code in a thread: https://github.com/scrapy/scrapy/blob/master/scrapy/contrib/feedexport.py#L46
On Mon, May 26, 2014 at 3:58 PM, Dimitris Kouzis - Loukas <[email protected] > wrote: > Hello, > > I'm lookign at scrapy/contrib/exporter/__init__.py > and scrapy/contrib/feedexport.py and I'm still not sure if/how storage is > handled twisted-style asynchronously ( fdesc.setNonBlocking(fd) ). > Shouldn't this also be non-blocking given that it writes to slow > disk/s3/ftp? > > Thanks, > Dimitris > > -- > You received this message because you are subscribed to the Google Groups > "scrapy-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/scrapy-users. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
