Hi Tiago, I fixed this typo but still doesn't work :(
2014-06-10 19:28 GMT+08:00 Tiago Natel de Moura <[email protected]>: > Hi Jerry, > > Your problem is in line 11 of file pipeline.py. You're using > self.ids_seens when it should be self.ids_seen. > > Cheers! > > > 2014-06-10 8:07 GMT-03:00 Jerry Wu <[email protected]>: > >> Hello, >> >> I am a newbie to scrapy (and have little programming background). I want >> to learn scrapy fast and efficiently and believe start a project is the >> best way to learn. English is not my mother language so it sometimes makes >> me feel difficult. But I am trying my best to understand what I read on >> Tutorial and Stackoverflow. I hope I could be more pythonic and think like >> most scrapy users think. So if you have any suggestion, please feel free to >> let me know. If you come to Shanghai someday, I am very glad to buy you a >> cup of coffee and take you around. >> >> Here is the project I am working on. I want to scrape down name and >> address pair from the link: http://www.lawson.com.cn/store/ . Here is my >> roadmap: >> 1. get links which would be scraped later with rules: for example: >> http://www.lawson.com.cn/store/shanghai/west/west01/ >> 2. for each links I scrape, I call def parse_shop to deal with it. "店名" >> means the name and “地址” means address. I also use regular expression for >> the address. >> >> Above two steps are fine for me. However, when I export result to csv >> file, I found there are quite a few duplicates. I add a class in >> pipeline.py and activiate it according to tutorial >> <http://doc.scrapy.org/en/latest/topics/item-pipeline.html> but doesn't >> work. What I got is: exceptions.KeyError: 'id'. I have no idea what to >> do with it. >> >> My code is below. Any thoughts are welcomed. >> >> 1. spider: http://pastebin.com/6AjGNdFH >> 2. pipeline: http://pastebin.com/JfBpq0t7 >> 3. setting: http://pastebin.com/yeiAth3L >> >> -- >> You received this message because you are subscribed to the Google Groups >> "scrapy-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> >> To post to this group, send email to [email protected]. >> Visit this group at http://groups.google.com/group/scrapy-users. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to a topic in the > Google Groups "scrapy-users" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/scrapy-users/t8UGQDknMw8/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/scrapy-users. > For more options, visit https://groups.google.com/d/optout. > -- Best Regards. Jerry Wu *Life is short. Change is possible : )* -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
