Re: Java Primitive Collections in Flink

2014-11-26 Thread Sean Owen
in context: > http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/Java-Primitive-Collections-in-Flink-tp563p2614.html > Sent from the Apache Flink (Incubator) Mailing List archive. mailing list > archive at Nabble.com.

Re: Java Primitive Collections in Flink

2014-11-25 Thread samkum
-mailing-list-archive.1008284.n3.nabble.com/Java-Primitive-Collections-in-Flink-tp563p2611.html Sent from the Apache Flink (Incubator) Mailing List archive. mailing list archive at Nabble.com.

Re: Java Primitive Collections in Flink

2014-11-25 Thread sirinath
message in context: http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/Java-Primitive-Collections-in-Flink-tp563p2614.html Sent from the Apache Flink (Incubator) Mailing List archive. mailing list archive at Nabble.com.

Re: Java Primitive Collections in Flink

2014-11-25 Thread Sean Owen
es. You can bench mark and see what works best. > > > > -- > View this message in context: > http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/Java-Primitive-Collections-in-Flink-tp563p2612.html > Sent from the Apache Flink (Incubator) Mailing Li

Re: Java Primitive Collections in Flink

2014-11-25 Thread sirinath
/Java-Primitive-Collections-in-Flink-tp563p2612.html Sent from the Apache Flink (Incubator) Mailing List archive. mailing list archive at Nabble.com.

Re: Java Primitive Collections in Flink

2014-06-24 Thread Stephan Ewen
I agree with Sean. Fastutil does all sorts of combinations of primitive key and value types, blowing the number of classes up. If you only need a simple growing array of primitive longs, it may be simpler to just implement it yourself.

Re: Java Primitive Collections in Flink

2014-06-24 Thread Sean Owen
It's own implementation of a primitive map. The needs were limited, and two Scala classes did the trick. On Tue, Jun 24, 2014 at 9:18 AM, Robert Metzger wrote: > What did Spark use instead of fastutil? > > > On Tue, Jun 24, 2014 at 9:40 AM, Sean Owen wrote: > >> A word of caution -- fastutil is

Re: Java Primitive Collections in Flink

2014-06-24 Thread Robert Metzger
What did Spark use instead of fastutil? On Tue, Jun 24, 2014 at 9:40 AM, Sean Owen wrote: > A word of caution -- fastutil is a massive library, 20MB or so and 10K > files if I recall correctly. It was pulled out of Spark just because > it was making the deployment jars huge (and wasn't used muc

Re: Java Primitive Collections in Flink

2014-06-24 Thread Sean Owen
A word of caution -- fastutil is a massive library, 20MB or so and 10K files if I recall correctly. It was pulled out of Spark just because it was making the deployment jars huge (and wasn't used much). Make sure it's worth it. On Tue, Jun 24, 2014 at 8:33 AM, Robert Waury wrote: > Okay, > > I'm

Re: Java Primitive Collections in Flink

2014-06-24 Thread Robert Waury
Okay, I'm going to add fastutil to the dependencies in my next pull request. Cheers, Robert On Jun 20, 2014 8:52 AM, "Sebastian Schelter" wrote: > +1 for fastutils > > > On 06/20/2014 08:50 AM, Robert Metzger wrote: > >> Hi Robert, >> >> The Apache Commons Primitives Collection project seems to

Re: Java Primitive Collections in Flink

2014-06-19 Thread Sebastian Schelter
+1 for fastutils On 06/20/2014 08:50 AM, Robert Metzger wrote: Hi Robert, The Apache Commons Primitives Collection project seems to be pretty inactive. The last release was in 2003, there are many dead links on the website. I would not suggest to use it. HPPC and fastutil seem pretty similar t

Re: Java Primitive Collections in Flink

2014-06-19 Thread Robert Metzger
Hi Robert, The Apache Commons Primitives Collection project seems to be pretty inactive. The last release was in 2003, there are many dead links on the website. I would not suggest to use it. HPPC and fastutil seem pretty similar to me. Both have a somewhat active mailing list and up-to-date relea

Java Primitive Collections in Flink

2014-06-19 Thread Robert Waury
Hi, I'm currently working on some code in Flink's runtime and want to use some Java Primitive Collections to improve performance. As fas as I can see no Primitive Collections library is in the dependencies so I wanted to ask if anybody has any preferences or input on which library the project sho