If the concern is simply data size on the wire, in the meantime, you can simply enable buffer compression.
However, if you're also concerned about data size in memory, then indeed Run-End Encoding would help. Regards Antoine. On Mon, 23 Oct 2023 14:13:25 -0400 Elliott Bradshaw <elli...@tectonix.com> wrote: > Just as a simple example, if we’re shipping a table of events that has a > year, month, and day column, and it’s sorted by time, then each column will > have long runs of duplicate numbers that would benefit greatly from > compaction. > > It looks like REE is already built into the standard and supported by the > C, Go and Python apis. Would be awesome to have this support in Java as > well! > > Thanks, > Elliott > > On Mon, Oct 23, 2023 at 10:55 AM Felipe Oliveira Carvalho < > felipe...@gmail.com> wrote: > > > Hi Elliott, > > > > Not that I know of. But do you have concrete numbers and a practical case > > that could motivate someone to tackle the project? > > > > -- > > Felipe > > > > On Sun, Oct 22, 2023 at 10:05 AM Elliott Bradshaw <elli...@tectonix.com> > > wrote: > > > >> Hi Arrow Team, > >> > >> We love your work. Wondering if support for Run End Encoded Vectors is > >> on the horizon in Java? It would make it much more efficient to ship > >> column sorted tables over the wire. > >> > >> Thanks! > >> Elliott > >> > > >
