Re: Fast HashSets HashMaps - Spark Collection Utils

2015-01-15 Thread Sean Owen
A recent discussion says these won't be public. However there are many optimized collection libs in Java. My favorite is Koloboke: https://github.com/OpenHFT/Koloboke/wiki/Koloboke:-roll-the-collection-implementation-with-features-you-need Carrot HPPC is good too. The only catch is that the

Fast HashSets HashMaps - Spark Collection Utils

2015-01-14 Thread Night Wolf
Hi all, I'd like to leverage some of the fast Spark collection implementations in my own code. Particularity for doing things like distinct counts in a mapPartitions loop. Are there any plans to make the org.apache.spark.util.collection implementations public? Is there any other library out