By the way there's almost certainly a very nice elegant and cool way to do what you want to do without having the entire datastream in memory (sketches/streaming algorithms <https://www.mapr.com/blog/some-important-streaming-algorithms-you-should-know-about>, Chapter 4 here <http://www.mmds.org/>)
On Sunday, April 10, 2016 at 11:04:44 AM UTC+1, Salvad0r wrote: > > I would like to act on all items, in other word s collect all items and > wirte them once to a file adding a header wrapping all the items. A good > place for this seem to bee the pipelines, here only item by item is > handeled. > > > > I fond this solution: „How to access all scraped items in Scrapy item > pipeline?“ > https://stackoverflow.com/questions/12768247/how-to-access-all-scraped-items-in-scrapy-item-pipeline > > > But the way seems to be more complex than nessesary. Is there a smarter, > shorter, easier or more elegant way? THX! > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
