Can you look at the class of these keys and see if some are Long while others
are Integer?
Stu
> I'm running a large hadoop job in which merge-with is called millions
> of times to aggregate values among about 1000 keys. Basically we are
> counting the number of times the keys occur among all entries and
> using merge-with as the reduce function.
>
> In the output, the keys are often duplicated (I've attached a sample
> of the result map below so you can see what I mean, the keys 14336,
> 14464, 14528 have dups), with the repeat having a value of 1 or
> sometimes 2.
>
> This is coming right out of merge-with, so its baffling why there
> would be duplicate keys, unless there is some kind of bug in that
> function when called in a strenuous setting. Another theory is
> something going wrong during serialization in intermediate steps. I am
> using cascading-clojure. Thanks for your input.
>
> {14336 18176, 14336 2, 14368 19111, 14400 17161, 14432 17015, 14464
> 14604, 14464 1, 14496 20810, 14528 18759, 14528 1, 14560 28086, 13568
> 162, 14592 31956, 13600 437, 14624 38402, 13632 281, 14656 38429,
> 13664 1137, 14688 29531}
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
