Re: ANN: Iota 1.0.2 release (Reducers for text files)

2013-03-10 Thread Alan Busby
Hi Bernard, I'm going to see if I can add some performance metrics in the test code to try and determine where various bottlenecks may exist. If that goes well I'll see if I can add a branch that returns Byte[] instead of String for iota/vec. I suspect this will just require modifying iota.FileVe

Re: ANN: Iota 1.0.2 release (Reducers for text files)

2013-03-08 Thread bernardH
Hi Alan, On Friday, March 8, 2013 4:02:18 PM UTC+1, Alan Busby wrote: > > Hi Bernard, > > I'd certainly like to add support for binary files, but as I haven't had a > need for it myself I haven't had a good place to start. > > As Java NIO's mmap() doesn't support ranges over 2GB, I've had to pa

Re: ANN: Iota 1.0.2 release (Reducers for text files)

2013-03-08 Thread Alan Busby
Hi Bernard, I'd certainly like to add support for binary files, but as I haven't had a need for it myself I haven't had a good place to start. As Java NIO's mmap() doesn't support ranges over 2GB, I've had to paste together multiple mmap's to cover files that are larger than 2GB. So if a record e

Re: ANN: Iota 1.0.2 release (Reducers for text files)

2013-03-07 Thread bernardH
Hi, On Wednesday, March 6, 2013 2:53:26 PM UTC+1, Alan Busby wrote: > > > With the release of Clojure 1.5 and it's new reducers, I figured this > would be a good time to release a library to help with file IO for > reducers. As reducers can only operate in parallel over specific > collections,

Re: ANN: Iota 1.0.2 release (Reducers for text files)

2013-03-06 Thread Paul deGrandis
I agree, this is a really cool project. Nice work! -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first

Re: ANN: Iota 1.0.2 release (Reducers for text files)

2013-03-06 Thread kovas boguta
Very cool! Been wanting to play with something like this. On Wed, Mar 6, 2013 at 5:53 AM, Alan Busby wrote: > Hi all, > > With the release of Clojure 1.5 and it's new reducers, I figured this would > be a good time to release a library to help with file IO for reducers. As > reducers can only o

Re: ANN: Iota 1.0.2 release (Reducers for text files)

2013-03-06 Thread Michael Klishin
2013/3/6 Alan Busby > Code is available on Github here: https://github.com/thebusby/iota > Library is available on Clojars here: https://clojars.org/iota > Alan, Please add dependency information to the README. Otherwise beginners won't be able to use your project. If you need an example: http

ANN: Iota 1.0.2 release (Reducers for text files)

2013-03-06 Thread Alan Busby
Hi all, With the release of Clojure 1.5 and it's new reducers, I figured this would be a good time to release a library to help with file IO for reducers. As reducers can only operate in parallel over specific collections, like vec, it requires some work to use them against files. Iota wraps a te