Well, I finally have a PR[1] that adds full ArrayData validation ready for review Any thoughts or review comments would be most helpful.
Thank you, Andrew [1] https://github.com/apache/arrow-rs/pull/921 On Fri, Oct 29, 2021 at 6:23 AM Andrew Lamb <al...@influxdata.com> wrote: > There is more discussion about the RUSTSEC process here: > https://github.com/rustsec/advisory-db/issues/1092 > > On Wed, Oct 6, 2021 at 10:52 AM Andrew Lamb <al...@influxdata.com> wrote: > >> I have incorporated feedback into a proposal [1] of how to handle >> validation of arguments to ArrayData::new, and would appreciate further >> review >> >> [1] https://github.com/apache/arrow-rs/issues/817 >> >> On Fri, Oct 1, 2021 at 6:44 AM Andrew Lamb <al...@influxdata.com> wrote: >> >>> Thank you Antoine, >>> >>> The C++ validation routine is super helpful, especially with respect to >>> validating nested structures. I will follow its lead >>> >>> Andrew >>> >>> On Fri, Oct 1, 2021 at 3:25 AM Antoine Pitrou <anto...@python.org> >>> wrote: >>> >>>> >>>> In C++ we have dedicated validation routines that can be used against >>>> untrusted input. (*) We also have fuzzing set up using OSS-Fuzz to >>>> validate that invalid input cannot crash the IPC reader. >>>> >>>> (*) >>>> >>>> https://github.com/apache/arrow/blob/master/cpp/src/arrow/array/validate.h >>>> >>>> >>>> Le 01/10/2021 à 00:13, Andrew Lamb a écrit : >>>> > I have created a WIP PR for initial feedback on the approach of >>>> validating >>>> > ArrayData upon creation[1]. If there are no objections to the >>>> approach I >>>> > will complete the implementation over the next few days >>>> > >>>> > The approach that Sergey describes of `get` and `unsafe get_unchecked` >>>> > sounds like a good one to me if performance testing shows we need a >>>> bypass. >>>> > >>>> > Andrew >>>> > >>>> > [1] https://github.com/apache/arrow-rs/pull/810 >>>> >>>
