+1
Thanks all for pushing this forward!
On Fri, Sep 29, 2023, at 12:55, wish maple wrote:
> +1
>
> LGTM, thanks!
>
> Ian Cook <ianmc...@apache.org> 于2023年9月30日周六 00:49写道：
>
>> +1 (non-binding)
>>
>> Thanks very much Felipe for your persistence and your commitment to
>> addressing the numerous questions and comments that have been raised
>> since the beginning of the discussion on this in April.
>>
>> On Fri, Sep 29, 2023 at 12:34 PM Benjamin Kietzman <bengil...@gmail.com>
>> wrote:
>> >
>> > +1
>> >
>> > On Fri, Sep 29, 2023 at 10:51 AM Felipe Oliveira Carvalho <
>> > felipe...@gmail.com> wrote:
>> >
>> > > Yes, ListView is an implementation of Velox's ArrayVector [1] ("vector
>> of
>> > > arrays"). In Arrow we would naturally refer to them as "array of
>> lists",
>> > > but `ListArray` is taken by the existing offset-only list formats.
>> > > Following the pattern adopted by other types in Arrow that use offsets
>> and
>> > > sizes, we adopt the suffix -View to differentiate list-views from
>> lists.
>> > >
>> > > Velox doesn't offer the 64-bit variation, but since Arrow has both
>> List and
>> > > LargeList, it was natural to pair them with ListView and LargeListView.
>> > >
>> > > [2] is a link to the point of a talk by Mark Raasveldt where he
>> describes
>> > > the DuckDB list representation. Early in the talk, one of the slides
>> [3]
>> > > mentions how these formats were "co-designed together with Velox team".
>> > >
>> > > --
>> > > Felipe
>> > >
>> > > [1]
>> > >
>> https://facebookincubator.github.io/velox/develop/vectors.html#arrayvector
>> > > [2] https://youtu.be/bZOvAKGkzpQ?si=wgSwew3Ck8utteOI&t=1569
>> > > [3] https://15721.courses.cs.cmu.edu/spring2023/slides/22-duckdb.pdf
>> > >
>> > > On Fri, Sep 29, 2023 at 9:32 AM Raphael Taylor-Davies
>> > > <r.taylordav...@googlemail.com.invalid> wrote:
>> > >
>> > > > Hi Felipe,
>> > > >
>> > > > Can I confirm that DuckDB and Velox use the same encoding for these
>> > > > types, and so we aren't going to run into similar issues as [1]?
>> > > >
>> > > > Kind Regards,
>> > > >
>> > > > Raphael Taylor-Davies
>> > > >
>> > > > [1]:
>> https://lists.apache.org/thread/l8t1vj5x1wdf75mdw3wfjvnxrfy5xomy
>> > > >
>> > > > On 29/09/2023 13:09, Felipe Oliveira Carvalho wrote:
>> > > > > Hello,
>> > > > >
>> > > > > I'd like to propose adding ListView and LargeListView arrays to the
>> > > Arrow
>> > > > > format.
>> > > > > Previous discussion in [1][2], columnar format description and
>> > > > flatbuffers
>> > > > > changes in [3].
>> > > > >
>> > > > > There are implementations available in both C++ [4] and Go [5]. I'm
>> > > > working
>> > > > > on the integration tests which I will push to one of the PR
>> branches
>> > > > before
>> > > > > they are merged. I've made a graph illustrating how this addition
>> > > > affects,
>> > > > > in a backwards compatible way, the type predicates and inheritance
>> > > chain
>> > > > on
>> > > > > the C++ implementation. [6]
>> > > > >
>> > > > > The vote will be open for at least 72 hours not counting the
>> weekend.
>> > > > >
>> > > > > [ ] +1 add the proposed ListView and LargeListView types to the
>> Apache
>> > > > > Arrow format
>> > > > > [ ] -1 do not add the proposed ListView and LargeListView types to
>> the
>> > > > > Apache Arrow format
>> > > > > because...
>> > > > >
>> > > > > Sincerely,
>> > > > > Felipe
>> > > > >
>> > > > > [1]
>> https://lists.apache.org/thread/r28rw5n39jwtvn08oljl09d4q2c1ysvb
>> > > > > [2]
>> https://lists.apache.org/thread/dcwdzhz15fftoyj6xp89ool9vdk3rh19
>> > > > > [3] https://github.com/apache/arrow/pull/37877
>> > > > > [4] https://github.com/apache/arrow/pull/35345
>> > > > > [5] https://github.com/apache/arrow/pull/37468
>> > > > > [6]
>> https://gist.github.com/felipecrv/3c02f3784221d946dec1b031c6d400db
>> > > > >
>> > > >
>> > >
>>
