On Wed, Dec 15, 2021 at 6:22 PM Micah Kornfield wrote:
>>
>> In any case, having memory layouts that support O(# records)
>> selections on strings and nested data will greatly benefit some data
>> processing systems built on Arrow.
>
>
> Wes, something that still isn't clear to me, are we
>
> In any case, having memory layouts that support O(# records)
> selections on strings and nested data will greatly benefit some data
> processing systems built on Arrow.
Wes, something that still isn't clear to me, are we proposing these new
encoding for ONLY the C-ABI or do we want to plumb
On Wed, Dec 15, 2021 at 3:56 PM Micah Kornfield wrote:
>
> >
> > Big +1 in replacing our current representation of variable-sized arrays by
> > the "sequence view". atm I am -0.5 in adding it without removing the
> > [Large]Utf8Array / Binary / List, as I see the advantages as sufficiently
> >
>
> Big +1 in replacing our current representation of variable-sized arrays by
> the "sequence view". atm I am -0.5 in adding it without removing the
> [Large]Utf8Array / Binary / List, as I see the advantages as sufficiently
> large to break compatibility and deprecate the previous
> I am -0.5 in adding it without removing the
> [Large]Utf8Array / Binary / List
I'm not sure about dropping List.
Is SequenceView semantically equivalent to List / FixedSizeList? In
other words, is SequenceView a nested type? The document seems to
suggest it is but the use case you described
Hi,
Thanks a lot for this initiative and the write up.
I did a small bench for the sequence view and added a graph to the document
for evidence of what Wes is writing wrt to performance of "selection / take
/ filter".
Big +1 in replacing our current representation of variable-sized arrays by
Great. I've merged both your PRs into the new branch, so you can open a PR for
the integration tests against that branch next.
-David
On Wed, Dec 15, 2021, at 13:52, Rafael Telles wrote:
> Thanks everyone!
>
> @David, we prefer to already merge the existing PRs and then add the
> integration
Thanks everyone!
@David, we prefer to already merge the existing PRs and then add the
integration tests as soon as possible.
Em qua., 15 de dez. de 2021 às 11:11, David Li
escreveu:
> My vote: +1
>
> The vote passes with three +1 (binding) votes, one +1 (non binding) vote,
> and one -0.5
My vote: +1
The vote passes with three +1 (binding) votes, one +1 (non binding) vote, and
one -0.5 (binding) vote.
However, we will first merge into a separate branch and implement integration
tests before merging into the main branch. JIRA for integration tests:
