Or even better, use guava's Multiset there... On Saturday, September 28, 2013 8:51:56 PM UTC+2, Jozef Wagner wrote: > > Well it should be possible to implement a foldseq variant which takes a > reducible collection as an input. This would speed things, as you don't > create so much garbage with reducers. XML parser which produces reducible > collection will be a bit harder :). > > Anyway, I think the bottleneck in your code is at > https://github.com/paulbutcher/parallel-word-count/blob/master/src/wordcount/core.clj#L9Instead > of creating new persistent map for each word, you should use a > transient here. > > JW > > On Saturday, September 28, 2013 7:12:08 PM UTC+2, Paul Butcher wrote: >> >> On 28 Sep 2013, at 17:42, Jozef Wagner <jozef....@gmail.com> wrote: >> >> I mean that you should forgot about lazy sequences and sequences in >> general, if you want to have a cutting edge performance with reducers. >> Example of reducible slurp, https://gist.github.com/wagjo/6743885 , does >> not hold into the head. >> >> >> OK - I buy that logic. But I'm unsure whether it can be applied to the >> case that I'm interested in. >> >> Firstly, I'm dealing with an XML file, so I need to parse it. Right now >> I'm using data.xml/parse to do so, which returns a lazy sequence. So, short >> of writing my own XML parser, I have to go via a lazy sequence at some >> point along the way? >> >> Secondly, the primary point of this is to exploit parallelism, not >> reducers per-se. So supporting CollReduce isn't enough, I also need to >> support CollFold. Which is exactly what foldable-seq does - I'm not sure >> how that differs from what you're proposing? >> >> Am I missing something? >> >> -- >> paul.butcher->msgCount++ >> >> Snetterton, Castle Combe, Cadwell Park... >> Who says I have a one track mind? >> >> http://www.paulbutcher.com/ >> LinkedIn: http://www.linkedin.com/in/paulbutcher >> MSN: pa...@paulbutcher.com >> AIM: paulrabutcher >> Skype: paulrabutcher >> >>
-- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
