Thanks everyone. Turns out my spider file was fine, but strangely, my pipeline file had some weird code inserted in that was not fetching rows for which the spider was unable to download images (due to the pipeline.py bug). And since the spider was unable to get imagex for any urls in the file, it was not writing the data to csv. As soon as we removed the strange code in the pipeline file, things are running smoothly.
Thanks again. > On Nov 19, 2014, at 7:44 PM, Tuấn Lê <[email protected]> wrote: > > Do you return item? > > >> On Wednesday, November 5, 2014 10:59:59 PM UTC+7, ara wrote: >> Hi, >> >> Can anyone tell me how to fix this: >> >> I am using Scrapy 0.24.4, Python 2.7: things are fine when I scrape within >> Terminal. However, when I try to export it as CSV, it creates a CSV file, >> but completely empty. >> >> Here's what the terminal shows (note: I had 3 urls that I was scraping as a >> test: >> >> 2014-11-05 10:47:09-0500 [file_nov2014] INFO: Closing spider (finished) >> >> 2014-11-05 10:47:09-0500 [file_nov2014] INFO: Dumping Scrapy stats: >> >> {'downloader/request_bytes': 688, >> >> 'downloader/request_count': 3, >> >> 'downloader/request_method_count/GET': 3, >> >> 'downloader/response_bytes': 60910, >> >> 'downloader/response_count': 3, >> >> 'downloader/response_status_count/200': 3, >> >> 'file_count': 6, >> >> 'file_status_count/uptodate': 6, >> >> 'finish_reason': 'finished', >> >> 'finish_time': datetime.datetime(2014, 11, 5, 15, 47, 9, 188837), >> >> 'log_count/DEBUG': 11, >> >> 'log_count/ERROR': 3, >> >> 'log_count/INFO': 7, >> >> 'response_received_count': 3, >> >> 'scheduler/dequeued': 3, >> >> 'scheduler/dequeued/memory': 3, >> >> 'scheduler/enqueued': 3, >> >> 'scheduler/enqueued/memory': 3, >> >> 'start_time': datetime.datetime(2014, 11, 5, 15, 47, 7, 935771)} >> >> 2014-11-05 10:47:09-0500 [file_nov2014] INFO: Spider closed (finished) >> >> How do I get Scrapy to write to the CSv file? >> >> Thanks. >> -Ara > > -- > You received this message because you are subscribed to a topic in the Google > Groups "scrapy-users" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/scrapy-users/zHKoujfJvzs/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/scrapy-users. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
