If you (vertigo.core/wrap "a-file-name"), that will use mmap under the covers, so if no one's tried it, it's easy enough to start.
With respect to non-fixed data layouts, that could be supported by a library which parses the framing information, and then layers Vertigo atop the actual data. In effect, that's what Gloss [1] is going to become, so keep watching the skies. Zach [1] https://github.com/ztellman/gloss On Sun, Jul 14, 2013 at 9:16 PM, kovas boguta <kovas.bog...@gmail.com>wrote: > This is pretty neat. > > Anyone try using this in conjunction with mmap? > > It would be nice to have some way to deal with strings & other > variable-length data. > > I'm also curious if its possible to make the analog of this for fressian, > basically to avoid unpacking objects that are not necessary for the > computation at hand. > > > > > > > On Tue, Jul 9, 2013 at 8:56 PM, Zach Tellman <ztell...@gmail.com> wrote: > >> Last year, I gave a talk at the Conj on my attempt to write an AI for the >> board game Go. Two things I discovered is that it was hard to get >> predictable performance, but even once I made sure I had all the right type >> hints, there was still a lot of room at the bottom for performance >> improvements. Towards the end [1], I mentioned a few ideas for >> improvements, one of which was simply using ByteBuffers rather than objects >> to host the data. This would remove all the levels of indirection, giving >> much better cache coherency, and also allow for fast unsynchronized >> mutability when the situation called for it. >> >> So, ten months and several supporting libraries [2] [3] later, here it >> is: https://github.com/ztellman/vertigo >> >> At a high level, this library is useful whenever your datatype has a >> fixed layout and is used more than once. Depending on your type, it will >> give you moderate to large memory savings, and if you're willing to forgo >> some of core library in favor of Vertigo's operators, you can get >> significant performance gains on batch operations. And, in the cases where >> performance doesn't matter, it will behave exactly like any other Clojure >> data structure. >> >> I want to point out that something like this would be more or less >> impossible in Java; reading from an offset in a ByteBuffer without the >> compile-time inference and validation provided by this library would be >> pointlessly risky. There's not a lot of low-level Clojure libraries, but >> there's an increasing amount of production usage where people are using >> Clojure for performance-sensitive work. I'm looking forward to seeing what >> people do with Vertigo and libraries like it. >> >> Zach >> >> [1] >> http://www.youtube.com/watch?feature=player_detailpage&v=v5dYE0CMmHQ#t=1828s >> [2] https://github.com/ztellman/primitive-math >> [3] https://github.com/ztellman/byte-streams >> >> -- >> -- >> You received this message because you are subscribed to the Google >> Groups "Clojure" group. >> To post to this group, send email to clojure@googlegroups.com >> Note that posts from new members are moderated - please be patient with >> your first post. >> To unsubscribe from this group, send email to >> clojure+unsubscr...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/clojure?hl=en >> --- >> You received this message because you are subscribed to the Google Groups >> "Clojure" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to clojure+unsubscr...@googlegroups.com. >> >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> > > -- > -- > You received this message because you are subscribed to the Google > Groups "Clojure" group. > To post to this group, send email to clojure@googlegroups.com > Note that posts from new members are moderated - please be patient with > your first post. > To unsubscribe from this group, send email to > clojure+unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/clojure?hl=en > --- > You received this message because you are subscribed to a topic in the > Google Groups "Clojure" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/clojure/BayfuaqMzvs/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > clojure+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
