Hi Michael, We are targeting grouped aggregation for 4.0 as part of a general query engine buildout. We also intend to bring DataFrame functionality into core Arrow (which would probably include an analog of pandas' pivot_table), but the query engine work is a prerequisite.
Ben Kietzman On Fri, Mar 19, 2021, 08:19 Michael Lavina <michael.lav...@factset.com> wrote: > Hey Team, > > Sorry if this is answered already somewhere I tried searching emails and > issues but couldn’t find anything. I am wondering if there is a standard > way to encode row or column pivots in Arrow? > > I know Pandas does it already some way > https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.pivot_table.html > and there are libraries using Arrow like Perspective that may have their > internal solution for representation of pivots > https://perspective.finos.org/docs/md/view.html#row-pivots > > I am wondering if there is already a discussion or already a best practice > or standard for encoding this information. Or alternatively is this not > something that should be at all associated with Arrow. > > -Michael > > P.S. If anyone on the Perspective team or anyone who might know is on this > thread I would be interested in understanding more how Perspective, > specifically, encodes pivot information in Arrow. > >
