Re: [VOTE][RUST] Release Apache Arrow Rust 9.0.3 RC3

[Neville Dipale]

Sorry, that should be a -1 because of the bug

On Thu, 10 Feb 2022 at 09:42, Neville Dipale  wrote:

> [X] +0 binding
>
> There's a test failure on aarch64, I've opened
> https://github.com/apache/arrow-rs/issues/1294
>
> thread
> 'util::bit_chunk_iterator::tests::test_unaligned_bit_chunk_iterator'
> panicked at 'assertion failed: ALIGNMENT > 64',
> arrow/src/util/bit_chunk_iterator.rs:470:9
>
> On Wed, 9 Feb 2022 at 18:57, Andrew Lamb  wrote:
>
>> Hi,
>>
>> I would like to propose a release of Apache Arrow Rust Implementation,
>> version 9.0.2.
>>
>> This is the third RC[5] for the 9.x  version. While somewhat inconvenient
>> I
>> am happy to say our process has found and prevented two critical bugs
>> prior
>> from being released! I hope that "third time's the charm" this time.
>>
>> This release candidate is based on commit:
>> ab7c2904ccc03d0c05687ef416fbc5f4ed92f125 [1]
>>
>> The proposed release tarball and signatures are hosted at [2].
>>
>> The changelog is located at [3].
>>
>> Please download, verify checksums and signatures, run the unit tests,
>> and vote on the release. There is a script [4] that automates some of
>> the verification.
>>
>> The vote will be open for at least 72 hours.
>>
>> [ ] +1 Release this as Apache Arrow Rust
>> [ ] +0
>> [ ] -1 Do not release this as Apache Arrow Rust  because...
>>
>> [1]:
>>
>> https://github.com/apache/arrow-rs/tree/ab7c2904ccc03d0c05687ef416fbc5f4ed92f125
>> [2]:
>> https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-rs-9.0.2-rc3
>> [3]:
>>
>> https://github.com/apache/arrow-rs/blob/ab7c2904ccc03d0c05687ef416fbc5f4ed92f125/CHANGELOG.md
>> [4]:
>>
>> https://github.com/apache/arrow-rs/blob/master/dev/release/verify-release-candidate.sh
>> [5]: https://lists.apache.org/thread/gtvv5okt49jyr0hxn5kho7vwp25rz5dk
>> -
>> Running rat license checker on
>>
>> /Users/alamb/Software/arrow-rs/dev/dist/apache-arrow-rs-9.0.2-rc3/apache-arrow-rs-9.0.2.tar.gz
>>
>

Re: [VOTE][RUST] Release Apache Arrow Rust 9.0.3 RC3

[Neville Dipale]

[X] +0 binding

There's a test failure on aarch64, I've opened
https://github.com/apache/arrow-rs/issues/1294

thread 'util::bit_chunk_iterator::tests::test_unaligned_bit_chunk_iterator'
panicked at 'assertion failed: ALIGNMENT > 64',
arrow/src/util/bit_chunk_iterator.rs:470:9

On Wed, 9 Feb 2022 at 18:57, Andrew Lamb  wrote:

> Hi,
>
> I would like to propose a release of Apache Arrow Rust Implementation,
> version 9.0.2.
>
> This is the third RC[5] for the 9.x  version. While somewhat inconvenient I
> am happy to say our process has found and prevented two critical bugs prior
> from being released! I hope that "third time's the charm" this time.
>
> This release candidate is based on commit:
> ab7c2904ccc03d0c05687ef416fbc5f4ed92f125 [1]
>
> The proposed release tarball and signatures are hosted at [2].
>
> The changelog is located at [3].
>
> Please download, verify checksums and signatures, run the unit tests,
> and vote on the release. There is a script [4] that automates some of
> the verification.
>
> The vote will be open for at least 72 hours.
>
> [ ] +1 Release this as Apache Arrow Rust
> [ ] +0
> [ ] -1 Do not release this as Apache Arrow Rust  because...
>
> [1]:
>
> https://github.com/apache/arrow-rs/tree/ab7c2904ccc03d0c05687ef416fbc5f4ed92f125
> [2]:
> https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-rs-9.0.2-rc3
> [3]:
>
> https://github.com/apache/arrow-rs/blob/ab7c2904ccc03d0c05687ef416fbc5f4ed92f125/CHANGELOG.md
> [4]:
>
> https://github.com/apache/arrow-rs/blob/master/dev/release/verify-release-candidate.sh
> [5]: https://lists.apache.org/thread/gtvv5okt49jyr0hxn5kho7vwp25rz5dk
> -
> Running rat license checker on
>
> /Users/alamb/Software/arrow-rs/dev/dist/apache-arrow-rs-9.0.2-rc3/apache-arrow-rs-9.0.2.tar.gz
>

Re: Managing usage of the @ApacheArrow Twitter handle and other social media

[Wes McKinney]

> In my opinion, any PMC member should be allowed to use the Twitter account 
> without any other checks, balances, or friction.

I agree with this. Without objections, then, I would say we make sure
as many PMC members have access to the account as have interest. I do
not think I am empowered to grant access (Jacques may be the only one
since he created the account initially)

On Tue, Feb 1, 2022 at 4:44 PM Julian Hyde  wrote:
>
> In my opinion, any PMC member should be allowed to use the Twitter account 
> without any other checks, balances, or friction. They know that they are 
> speaking for the project, and only for the project. They are PMC members so 
> we trust them to do the right thing.
>
> If committers and other non-PMC community members would like to suggest a 
> tweet then a GitHub PR is a reasonable process. But unlike a code change, it 
> needs to be approved by a PMC member.
>
> Quite frankly the easiest process for non-PMC members is just to tweet as 
> yourself (or your company, or project) and mention @ApacheArrow. A PMC member 
> will retweet or quote tweet if it deserves wider circulation.
>
> Julian
>
>
>
> > On Feb 1, 2022, at 11:00 AM, QP Hou  wrote:
> >
> > To be clear I haven't used that twitter-together action myself, was
> > just using it as an example for how such a workflow could be set up. I
> > imagine it won't be too much work for us to write our own action if
> > needed.
> >
> > Thanks,
> > QP Hou
> >
> > On Tue, Feb 1, 2022 at 5:31 AM Neal Richardson
> >  wrote:
> >>
> >> I like the idea too. If we were to use the github action to send tweets, we
> >> would need to get INFRA to add it to the allowlist since they restrict
> >> which actions we can use. I don't think it's a blocker, just would add some
> >> extra time in the process of getting it set up.
> >>
> >> Neal
> >>
> >>
> >> On Tue, Feb 1, 2022 at 5:25 AM Alessandro Molina <
> >> alessan...@ursacomputing.com> wrote:
> >>
> >>> I never used https://github.com/gr2m/twitter-together previously, in the
> >>> past I used Hootsuite to set up approval workflows, but I think that the
> >>> idea of setting up a workflow through github PRs looks like a good idea. 
> >>> It
> >>> would be able to leverage committer/pmc membership to merge the PRs and
> >>> would allow anyone to contribute with social media content.
> >>>
> >>> On Tue, Feb 1, 2022 at 12:43 AM QP Hou  wrote:
> >>>
>  I don't know how other projects manage this, but one solution we could
>  evaluate is using github PRs to manage the twitter account. For
>  example, here is a github action that does exactly this
>  https://github.com/gr2m/twitter-together.
> 
>  On Mon, Jan 31, 2022 at 3:14 PM Wes McKinney 
> >>> wrote:
> >
> > hi all,
> >
> > The project is approaching it's 6th birthday and we have come a long
> >>> way!
> >
> > We have a relatively seldom-used Twitter handle
> > twitter.com/ApacheArrow and only a handful of people in the community
> > have access to it. I know that Jacques and I do, but I am not sure who
> > else.
> >
> > I wanted to discuss a few things:
> >
> > * Giving more committers/PMC members access to the Twitter handle — I
> > think clearly there should be more people with access (I tweet through
> > TweetDeck, e.g. I just posted about a newly posted blog post)
> > * Consider if there are any other social media channels where we might
> > want to promote Arrow content
> > * Discuss a social media policy more broadly for the project
> >
> > On the latter point, my feelings are:
> >
> > * Promote content and usage of Apache Arrow, but not companies or
> > products (Apache projects are independent)
> > * Provide a way for the community to submit ideas/materials for social
>  media
> >
> > Does anyone know if other ASF projects have policies/conventions about
> > how they decide how to use their social media properties to best serve
> > the community?
> >
> > Thanks,
> > Wes
> 
> >>>
>

[VOTE][RUST] Release Apache Arrow Rust 9.0.3 RC3

[Andrew Lamb]

Hi,

I would like to propose a release of Apache Arrow Rust Implementation,
version 9.0.2.

This is the third RC[5] for the 9.x  version. While somewhat inconvenient I
am happy to say our process has found and prevented two critical bugs prior
from being released! I hope that "third time's the charm" this time.

This release candidate is based on commit:
ab7c2904ccc03d0c05687ef416fbc5f4ed92f125 [1]

The proposed release tarball and signatures are hosted at [2].

The changelog is located at [3].

Please download, verify checksums and signatures, run the unit tests,
and vote on the release. There is a script [4] that automates some of
the verification.

The vote will be open for at least 72 hours.

[ ] +1 Release this as Apache Arrow Rust
[ ] +0
[ ] -1 Do not release this as Apache Arrow Rust  because...

[1]:
https://github.com/apache/arrow-rs/tree/ab7c2904ccc03d0c05687ef416fbc5f4ed92f125
[2]: https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-rs-9.0.2-rc3
[3]:
https://github.com/apache/arrow-rs/blob/ab7c2904ccc03d0c05687ef416fbc5f4ed92f125/CHANGELOG.md
[4]:
https://github.com/apache/arrow-rs/blob/master/dev/release/verify-release-candidate.sh
[5]: https://lists.apache.org/thread/gtvv5okt49jyr0hxn5kho7vwp25rz5dk
-
Running rat license checker on
/Users/alamb/Software/arrow-rs/dev/dist/apache-arrow-rs-9.0.2-rc3/apache-arrow-rs-9.0.2.tar.gz

Re: [VOTE] Release Apache Arrow 7.0.0 - RC10

[Ian Joiner]

Really thanks! Yes I can confirm that.

Ian

On Wednesday, February 9, 2022, Joris Van den Bossche <
jorisvandenboss...@gmail.com> wrote:

> It's not that simple though ;) For me, clicking on "7.0.0" works fine,
> and results in going to the stable docs. And the dropdown shows "7.0
> (stable)" and "8.0 (dev)".
>
> You might need a hard refresh of the browser page (the list of
> versions might be cached).
>
> Joris
>
>
> On Wed, 9 Feb 2022 at 07:03, Ian Joiner  wrote:
> >
> > Sure. Let’s actually keep it simple. Just go to
> > https://arrow.apache.org/docs/index.html and click on “7.0.0”. The
> problem
> > will immediately manifest itself.
> >
> > Ian
> >
> > On Wednesday, February 9, 2022, Sutou Kouhei  wrote:
> >
> > > Thanks.
> > > But we can't attach an image to this mailing list. Could you
> > > upload it to somewhere such as https://gist.github.com/ ?
> > >
> > > In  >
> > >   "Re: [VOTE] Release Apache Arrow 7.0.0 - RC10" on Tue, 8 Feb 2022
> > > 20:51:16 -0500,
> > >   Ian Joiner  wrote:
> > >
> > > > The URLs are good. It is the values in the drop-down list that still
> > > > need to be fixed. Please see the attached photo.
> > > >
> > > > Ian
> > > >
> > > >
> > > >
> > > > On Tuesday, February 8, 2022, Sutou Kouhei 
> wrote:
> > > >>
> > > >> Could you show URL that is occurred?
> > > >>
> > > >> It seems that the following URLs show correct versions:
> > > >>
> > > >> * https://arrow.apache.org/docs/6.0/index.html
> > > >> * https://arrow.apache.org/docs/index.html
> > > >> * https://arrow.apache.org/docs/dev/index.html
> > > >>
> > > >>
> > > >> Thanks,
> > > >> --
> > > >> kou
> > > >>
> > > >> In  gmail.com>
> > > >>   "Re: [VOTE] Release Apache Arrow 7.0.0 - RC10" on Tue, 8 Feb 2022
> > > 17:35:47 -0500,
> > > >>   Ian Joiner  wrote:
> > > >>
> > > >> > Really thanks!
> > > >> >
> > > >> > I do need to mention that versioning in the docs is still not
> > > displayed
> > > >> > properly (7.0 is labelled “6.0 (stable)” while 8.0 is labelled
> “7.0
> > > (dev)”).
> > > >> >
> > > >> > Ian
> > > >> >
> > > >> > On Tuesday, February 8, 2022, Sutou Kouhei 
> > > wrote:
> > > >> >
> > > >> >> Homebrew, MSYS2 and RubyGems are done:
> > > >> >>
> > > >> >> 1. [done] make the released version as "RELEASED" on JIRA
> > > >> >> 2. [done] start the new version on JIRA
> > > >> >> 4. [done] upload source
> > > >> >> 5. [done] upload binaries
> > > >> >> 6. [done] update website
> > > >> >> 7. [done] update Homebrew packages
> > > >> >> 8. [done] update MSYS2 package
> > > >> >> 9. [done] upload RubyGems
> > > >> >> 10. [done] upload JS packages
> > > >> >> 11. [done] upload C# packages
> > > >> >> 12. [todo:unassigned] update conda recipes
> > > >> >> 13. [done] upload wheels/sdist to pypi
> > > >> >> 14. [todo:kszucs] publish Maven artifacts
> > > >> >> 15. [todo:nealrichardson] update R packages
> > > >> >> 16. [todo:ianmcook] update vcpkg port
> > > >> >> 17. [done] bump versions
> > > >> >> 18. [done] update tags for Go modules
> > > >> >> 19. [done] update docs
> > > >> >>
> > > >> >> In  > > mail.gmail.com>
> > > >> >>   "Re: [VOTE] Release Apache Arrow 7.0.0 - RC10" on Thu, 3 Feb
> 2022
> > > >> >> 23:31:45 +0100,
> > > >> >>   Krisztián Szűcs  wrote:
> > > >> >>
> > > >> >> > I managed to upload the wheels to pypi. Here is the current
> status
> > > >> >> > with the updated assignments:
> > > >> >> >
> > > >> >> > 1. [done] make the released version as "RELEASED" on JIRA
> > > >> >> > 2. [done] start the new version on JIRA
> > > >> >> > 4. [done] upload source
> > > >> >> > 5. [done] upload binaries
> > > >> >> > 6. [in-pr] update website
> > > >> >> > 7. [todo:kou] update Homebrew packages
> > > >> >> > 8. [todo:kou] update MSYS2 package
> > > >> >> > 9. [todo:kou] upload RubyGems
> > > >> >> > 10. [done] upload JS packages
> > > >> >> > 11. [done] upload C# packages
> > > >> >> > 12. [todo:unassigned] update conda recipes
> > > >> >> > 13. [done] upload wheels/sdist to pypi
> > > >> >> > 14. [todo:kszucs] publish Maven artifacts
> > > >> >> > 15. [todo:nealrichardson] update R packages
> > > >> >> > 16. [todo:ianmcook] update vcpkg port
> > > >> >> > 17. [done] bump versions
> > > >> >> > 18. [done] update tags for Go modules
> > > >> >> > 19. [in-pr] update docs
> > > >> >> >
> > > >> >> > On Thu, Feb 3, 2022 at 10:26 PM Neal Richardson
> > > >> >> >  wrote:
> > > >> >> >>
> > > >> >> >> I will handle the R package submission to CRAN.
> > > >> >> > Thanks Neal!
> > > >> >> >>
> > > >> >> >> Neal
> > > >> >> >>
> > > >> >> >> On Thu, Feb 3, 2022 at 4:11 PM Sutou Kouhei <
> k...@clear-code.com>
> > > wrote:
> > > >> >> >>
> > > >> >> >> > Hi,
> > > >> >> >> >
> > > >> >> >> > I'll update/upload Homebrew, MSYS2 and RubyGems.
> > > >> >> > Thanks Kou!
> > > >> >> >> >
> > > >> >> >> > 1. [done] make the released version as "RELEASED" on JIRA
> > > >> >> >> > 2. [done] start the new version on JIRA
> > > >> >> >> > 4. [done] upload source
> > > >> >> >> > 5. [done] upload binaries
> > > >> >> >> > 

Re: [DISCUSS] Binary Values in Key value pairs WAS: Re: [INFO_REQUEST][FLIGHT] - Dynamic schema changes in ArrowFlight streams

[Phillip Cloud]

I don't think memcpy is feasible. The bytes may be different for different
languages.

Many languages' compilers reorder struct fields and pad structs for
efficiency reasons so the bytes in metadata coming from language X may be
meaningless to language Y.


On Wed, Feb 9, 2022, 08:41 Dewey Dunnington  wrote:

> I'm new to this discussion and Arrow generally and appreciate Joris
> bringing this up. While the forward- and backward-compatability of this is
> over my head, I think that it is important to provide a path for extension
> types to have serialized metadata as binary because the alternatives are
> (in my opinion) not great and might lead to extension developers inventing
> their own key/value encoding.
>
> With binary on the table there are options like protobuf, flatbuffers, or
> memcpy of a struct that extension type implementations can use depending on
> the complexity of the extension metadata that needs to be communicated and
> whether or not the metadata is intended to be used in compiled code (i.e.,
> are those values needed to do anything useful with the bytes in the
> buffers?).
>
> Without binary on the table, an extension type implementation has to vendor
> in a JSON reader/writer, vendor in a base64 encoder/decoder, or invent some
> custom serialization that's easier to parse than either of those.
>
> Cheers,
>
> -dewey
>
> On Fri, Feb 4, 2022 at 7:06 AM Joris Van den Bossche <
> jorisvandenboss...@gmail.com> wrote:
>
> > Reviving this thread, I don't think anything happened in the meantime on
> > this topic?
> >
> > From rereading the thread, it seems David mentioned two possible ideas:
> > - A new [byte] binary_value field in the existing KeyValue type, next to
> > the existing string value field. And if you have valid string metadata,
> the
> > same value can be put in both fields by reusing the offset
> > - Relax the existing string value field to [byte], but also require
> > implementations to null-terminate binary values regardless to ensure
> > compatibility for old readers
> >  - And in addition to those options, there is also the safest option of
> > adding a new BinaryKeyValue type and add that next to KeyValue everywhere
> > this is used.
> >
> > And we need to further assess which option is viable given the
> > forward/backward compatibility constraints?
> >
> > I want to point out though that in pyarrow we already _do_ use binary
> > metadata for extension types for several years. The built-in
> > PyExtensionType class uses a pickle dump as the serialized metadata (so
> > which is put in the ARROW:extension:metadata key in the key-value
> metadata
> > of the Field), and the bytes from a pickle dump are certainly not valid
> > UTF-8 I think.
> > I don't know how much this is used in the wild. In pandas, our arrow
> > extension types don't use this pickle-based PyExtensionType but define
> > their own serialization which currently returns a valid UTF-8 string. But
> > for example the Ray project seems to use PyExtensionType (
> >
> >
> https://docs.ray.io/en/latest/_modules/ray/data/extensions/tensor_extension.html#ArrowTensorType
> > ).
> >
> > Joris
> >
> > On Thu, 2 Sept 2021 at 07:01, Micah Kornfield 
> > wrote:
> >
> > > It still sounds like adding a new type might be the safest approach
> (and
> > > marking the old type as discouraged).
> > >
> > > On Mon, Aug 23, 2021 at 11:18 AM David Li  wrote:
> > >
> > > > I believe so.
> > > >
> > > > The encoding of a string in Flatbuffers is [byte] with a null
> > terminator
> > > > not included in the length, so old files should still be readable
> (they
> > > > would simply not see the terminator anymore). And conversely,
> > continuing
> > > to
> > > > write the null terminator means new files should still be readable by
> > > > existing implementations, so long as they don't have binary metadata
> > (but
> > > > we could increment the metadata version before allowing that). There
> > > still
> > > > remains the issue of non-UTF8 data inside the "string" value; on that
> > > > point, this is a non-answer (or I guess, it canonicalizes the current
> > > > unintentional behavior).
> > > >
> > > > It would complicate the Java API as that currently works with
> Strings,
> > > but
> > > > that will be an issue for any attempt to add binary metadata.
> > > >
> > > > -David
> > > >
> > > > On Mon, Aug 23, 2021, at 13:40, Antoine Pitrou wrote:
> > > > >
> > > > > Le 23/08/2021 à 17:52, David Li a écrit :
> > > > > > Another way forward might be to relax the value type to [byte],
> but
> > > > also require implementations to null-terminate binary values
> > regardless.
> > > > The C++ Flatbuffers implementation does this already [1] (though not
> > the
> > > > Java one [2]). Old implementations validating UTF8-ness would still
> be
> > > > unable to read anything with binary metadata (which is the case
> they're
> > > > already in), but implementations just validating for null-termination
> > > would
> > > > continue to work. And again,

Re: [DISCUSS] Binary Values in Key value pairs WAS: Re: [INFO_REQUEST][FLIGHT] - Dynamic schema changes in ArrowFlight streams

[Dewey Dunnington]

I'm new to this discussion and Arrow generally and appreciate Joris
bringing this up. While the forward- and backward-compatability of this is
over my head, I think that it is important to provide a path for extension
types to have serialized metadata as binary because the alternatives are
(in my opinion) not great and might lead to extension developers inventing
their own key/value encoding.

With binary on the table there are options like protobuf, flatbuffers, or
memcpy of a struct that extension type implementations can use depending on
the complexity of the extension metadata that needs to be communicated and
whether or not the metadata is intended to be used in compiled code (i.e.,
are those values needed to do anything useful with the bytes in the
buffers?).

Without binary on the table, an extension type implementation has to vendor
in a JSON reader/writer, vendor in a base64 encoder/decoder, or invent some
custom serialization that's easier to parse than either of those.

Cheers,

-dewey

On Fri, Feb 4, 2022 at 7:06 AM Joris Van den Bossche <
jorisvandenboss...@gmail.com> wrote:

> Reviving this thread, I don't think anything happened in the meantime on
> this topic?
>
> From rereading the thread, it seems David mentioned two possible ideas:
> - A new [byte] binary_value field in the existing KeyValue type, next to
> the existing string value field. And if you have valid string metadata, the
> same value can be put in both fields by reusing the offset
> - Relax the existing string value field to [byte], but also require
> implementations to null-terminate binary values regardless to ensure
> compatibility for old readers
>  - And in addition to those options, there is also the safest option of
> adding a new BinaryKeyValue type and add that next to KeyValue everywhere
> this is used.
>
> And we need to further assess which option is viable given the
> forward/backward compatibility constraints?
>
> I want to point out though that in pyarrow we already _do_ use binary
> metadata for extension types for several years. The built-in
> PyExtensionType class uses a pickle dump as the serialized metadata (so
> which is put in the ARROW:extension:metadata key in the key-value metadata
> of the Field), and the bytes from a pickle dump are certainly not valid
> UTF-8 I think.
> I don't know how much this is used in the wild. In pandas, our arrow
> extension types don't use this pickle-based PyExtensionType but define
> their own serialization which currently returns a valid UTF-8 string. But
> for example the Ray project seems to use PyExtensionType (
>
> https://docs.ray.io/en/latest/_modules/ray/data/extensions/tensor_extension.html#ArrowTensorType
> ).
>
> Joris
>
> On Thu, 2 Sept 2021 at 07:01, Micah Kornfield 
> wrote:
>
> > It still sounds like adding a new type might be the safest approach (and
> > marking the old type as discouraged).
> >
> > On Mon, Aug 23, 2021 at 11:18 AM David Li  wrote:
> >
> > > I believe so.
> > >
> > > The encoding of a string in Flatbuffers is [byte] with a null
> terminator
> > > not included in the length, so old files should still be readable (they
> > > would simply not see the terminator anymore). And conversely,
> continuing
> > to
> > > write the null terminator means new files should still be readable by
> > > existing implementations, so long as they don't have binary metadata
> (but
> > > we could increment the metadata version before allowing that). There
> > still
> > > remains the issue of non-UTF8 data inside the "string" value; on that
> > > point, this is a non-answer (or I guess, it canonicalizes the current
> > > unintentional behavior).
> > >
> > > It would complicate the Java API as that currently works with Strings,
> > but
> > > that will be an issue for any attempt to add binary metadata.
> > >
> > > -David
> > >
> > > On Mon, Aug 23, 2021, at 13:40, Antoine Pitrou wrote:
> > > >
> > > > Le 23/08/2021 à 17:52, David Li a écrit :
> > > > > Another way forward might be to relax the value type to [byte], but
> > > also require implementations to null-terminate binary values
> regardless.
> > > The C++ Flatbuffers implementation does this already [1] (though not
> the
> > > Java one [2]). Old implementations validating UTF8-ness would still be
> > > unable to read anything with binary metadata (which is the case they're
> > > already in), but implementations just validating for null-termination
> > would
> > > continue to work. And again, I believe this is in-spec for Flatbuffers
> > > since string lengths don't include the null terminator.
> > > > >
> > > > > [1]:
> > >
> >
> https://github.com/google/flatbuffers/blob/273f6084e55285e26ff8b4cdd55a9c0e2d9a48b7/include/flatbuffers/flatbuffers.h#L1612-L1670
> > > > > [2]:
> > >
> >
> https://github.com/google/flatbuffers/blob/273f6084e55285e26ff8b4cdd55a9c0e2d9a48b7/java/com/google/flatbuffers/FlatBufferBuilder.java#L594-L637
> > > > > Though it seems we could emulate C++ here by calling addByte(0)
> > > our

Re: [VOTE][RUST] Release Apache Arrow Rust 9.0.1 RC1

[Andrew Lamb]

There is a new report[1] of an additional issue with the release candidate.
I will look into it later today if no one else has a chance to

[1]
https://github.com/apache/arrow-rs/commit/411171ac7ad3efe9391b2a0f8709f7fe97672a20#commitcomment-66299402

On Mon, Feb 7, 2022 at 2:27 PM Jörn Horstmann 
wrote:

> +1 (non-binding)
>
> Updated to the release candidate and ran the test suite of our query
> engine. Thanks a lot Andrew for the quick reaction.
>
> On Mon, Feb 7, 2022 at 6:51 PM Andrew Lamb  wrote:
>
> > I would like to propose a release of Apache Arrow Rust Implementation,
> > version 9.0.1.
> >
> > This release candidate is based on commit:
> > e5003bf40bceb942c1c78cc2311e7d7a7e87ebf6 [1]
> >
> > The proposed release tarball and signatures are hosted at [2].
> >
> > The changelog is located at [3].
> >
> > Please download, verify checksums and signatures, run the unit tests,
> > and vote on the release. There is a script [4] that automates some of
> > the verification.
> >
> > The vote will be open for at least 72 hours.
> >
> > [ ] +1 Release this as Apache Arrow Rust
> > [ ] +0
> > [ ] -1 Do not release this as Apache Arrow Rust  because...
> >
> > [1]:
> >
> >
> https://github.com/apache/arrow-rs/tree/e5003bf40bceb942c1c78cc2311e7d7a7e87ebf6
> > [2]:
> > https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-rs-9.0.1-rc1
> > [3]:
> >
> >
> https://github.com/apache/arrow-rs/blob/e5003bf40bceb942c1c78cc2311e7d7a7e87ebf6/CHANGELOG.md
> > [4]:
> >
> >
> https://github.com/apache/arrow-rs/blob/master/dev/release/verify-release-candidate.sh
> >
> >
> > On Mon, Feb 7, 2022 at 11:57 AM Andrew Lamb 
> wrote:
> >
> > > Testing has found a critical issue[1] in RC1. I'll be creating an RC 2
> > > shortly with the fix.
> > >
> > > Sorry about that,
> > > Andrew
> > >
> > > [1] https://github.com/apache/arrow-rs/issues/1285
> > >
> > > On Fri, Feb 4, 2022 at 5:33 PM Andy Grove 
> wrote:
> > >
> > >> +1 (binding)
> > >>
> > >> Verified on Ubuntu 20.04.3 LTS
> > >>
> > >> On Fri, Feb 4, 2022 at 7:57 AM Wang Xudong 
> > >> wrote:
> > >>
> > >> > +1 non-binding
> > >> > verify on macOS.
> > >> > "Release candidate looks good!"
> > >> >
> > >> > Thanks @alamb
> > >> > ---
> > >> > xudong
> > >> >
> > >> > Andrew Lamb  于2022年2月4日周五 21:14写道：
> > >> >
> > >> > > Hi,
> > >> > >
> > >> > > I would like to propose a release of Apache Arrow Rust
> > Implementation,
> > >> > > version 9.0.0. I am also working on a blog post draft which I will
> > >> share
> > >> > on
> > >> > > this mailing list shortly.
> > >> > >
> > >> > > This release candidate is based on commit:
> > >> > > fa728736bffb44af2d2e7fd3a890678f3e807cfd [1]
> > >> > >
> > >> > > The proposed release tarball and signatures are hosted at [2].
> > >> > >
> > >> > > The changelog is located at [3].
> > >> > >
> > >> > > Please download, verify checksums and signatures, run the unit
> > tests,
> > >> > > and vote on the release. There is a script [4] that automates some
> > of
> > >> > > the verification.
> > >> > >
> > >> > > The vote will be open for at least 72 hours.
> > >> > >
> > >> > > [ ] +1 Release this as Apache Arrow Rust
> > >> > > [ ] +0
> > >> > > [ ] -1 Do not release this as Apache Arrow Rust  because...
> > >> > >
> > >> > > [1]:
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> https://github.com/apache/arrow-rs/tree/fa728736bffb44af2d2e7fd3a890678f3e807cfd
> > >> > > [2]:
> > >> > >
> > >>
> https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-rs-9.0.0-rc1
> > >> > > [3]:
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> https://github.com/apache/arrow-rs/blob/fa728736bffb44af2d2e7fd3a890678f3e807cfd/CHANGELOG.md
> > >> > > [4]:
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> https://github.com/apache/arrow-rs/blob/master/dev/release/verify-release-candidate.sh
> > >> > >
> > >> >
> > >>
> > >
> >
>
>
> --
> *Jörn Horstmann* | Senior Backend Engineer
>
> www.signavio.com
> Kurfürstenstraße 111, 10787 Berlin, Germany
>
> Work with us! 
>
>
>
> 
>    
> 
> 
>
> 
>
> HRB 121584 B Charlottenburg District Court, VAT ID: DE265675123
> Managing Directors: Dr. Gero Decker, Rouven Morato Adam
>

Re: [VOTE] Release Apache Arrow 7.0.0 - RC10

[Joris Van den Bossche]

It's not that simple though ;) For me, clicking on "7.0.0" works fine,
and results in going to the stable docs. And the dropdown shows "7.0
(stable)" and "8.0 (dev)".

You might need a hard refresh of the browser page (the list of
versions might be cached).

Joris


On Wed, 9 Feb 2022 at 07:03, Ian Joiner  wrote:
>
> Sure. Let’s actually keep it simple. Just go to
> https://arrow.apache.org/docs/index.html and click on “7.0.0”. The problem
> will immediately manifest itself.
>
> Ian
>
> On Wednesday, February 9, 2022, Sutou Kouhei  wrote:
>
> > Thanks.
> > But we can't attach an image to this mailing list. Could you
> > upload it to somewhere such as https://gist.github.com/ ?
> >
> > In 
> >   "Re: [VOTE] Release Apache Arrow 7.0.0 - RC10" on Tue, 8 Feb 2022
> > 20:51:16 -0500,
> >   Ian Joiner  wrote:
> >
> > > The URLs are good. It is the values in the drop-down list that still
> > > need to be fixed. Please see the attached photo.
> > >
> > > Ian
> > >
> > >
> > >
> > > On Tuesday, February 8, 2022, Sutou Kouhei  wrote:
> > >>
> > >> Could you show URL that is occurred?
> > >>
> > >> It seems that the following URLs show correct versions:
> > >>
> > >> * https://arrow.apache.org/docs/6.0/index.html
> > >> * https://arrow.apache.org/docs/index.html
> > >> * https://arrow.apache.org/docs/dev/index.html
> > >>
> > >>
> > >> Thanks,
> > >> --
> > >> kou
> > >>
> > >> In 
> > >>   "Re: [VOTE] Release Apache Arrow 7.0.0 - RC10" on Tue, 8 Feb 2022
> > 17:35:47 -0500,
> > >>   Ian Joiner  wrote:
> > >>
> > >> > Really thanks!
> > >> >
> > >> > I do need to mention that versioning in the docs is still not
> > displayed
> > >> > properly (7.0 is labelled “6.0 (stable)” while 8.0 is labelled “7.0
> > (dev)”).
> > >> >
> > >> > Ian
> > >> >
> > >> > On Tuesday, February 8, 2022, Sutou Kouhei 
> > wrote:
> > >> >
> > >> >> Homebrew, MSYS2 and RubyGems are done:
> > >> >>
> > >> >> 1. [done] make the released version as "RELEASED" on JIRA
> > >> >> 2. [done] start the new version on JIRA
> > >> >> 4. [done] upload source
> > >> >> 5. [done] upload binaries
> > >> >> 6. [done] update website
> > >> >> 7. [done] update Homebrew packages
> > >> >> 8. [done] update MSYS2 package
> > >> >> 9. [done] upload RubyGems
> > >> >> 10. [done] upload JS packages
> > >> >> 11. [done] upload C# packages
> > >> >> 12. [todo:unassigned] update conda recipes
> > >> >> 13. [done] upload wheels/sdist to pypi
> > >> >> 14. [todo:kszucs] publish Maven artifacts
> > >> >> 15. [todo:nealrichardson] update R packages
> > >> >> 16. [todo:ianmcook] update vcpkg port
> > >> >> 17. [done] bump versions
> > >> >> 18. [done] update tags for Go modules
> > >> >> 19. [done] update docs
> > >> >>
> > >> >> In  > mail.gmail.com>
> > >> >>   "Re: [VOTE] Release Apache Arrow 7.0.0 - RC10" on Thu, 3 Feb 2022
> > >> >> 23:31:45 +0100,
> > >> >>   Krisztián Szűcs  wrote:
> > >> >>
> > >> >> > I managed to upload the wheels to pypi. Here is the current status
> > >> >> > with the updated assignments:
> > >> >> >
> > >> >> > 1. [done] make the released version as "RELEASED" on JIRA
> > >> >> > 2. [done] start the new version on JIRA
> > >> >> > 4. [done] upload source
> > >> >> > 5. [done] upload binaries
> > >> >> > 6. [in-pr] update website
> > >> >> > 7. [todo:kou] update Homebrew packages
> > >> >> > 8. [todo:kou] update MSYS2 package
> > >> >> > 9. [todo:kou] upload RubyGems
> > >> >> > 10. [done] upload JS packages
> > >> >> > 11. [done] upload C# packages
> > >> >> > 12. [todo:unassigned] update conda recipes
> > >> >> > 13. [done] upload wheels/sdist to pypi
> > >> >> > 14. [todo:kszucs] publish Maven artifacts
> > >> >> > 15. [todo:nealrichardson] update R packages
> > >> >> > 16. [todo:ianmcook] update vcpkg port
> > >> >> > 17. [done] bump versions
> > >> >> > 18. [done] update tags for Go modules
> > >> >> > 19. [in-pr] update docs
> > >> >> >
> > >> >> > On Thu, Feb 3, 2022 at 10:26 PM Neal Richardson
> > >> >> >  wrote:
> > >> >> >>
> > >> >> >> I will handle the R package submission to CRAN.
> > >> >> > Thanks Neal!
> > >> >> >>
> > >> >> >> Neal
> > >> >> >>
> > >> >> >> On Thu, Feb 3, 2022 at 4:11 PM Sutou Kouhei 
> > wrote:
> > >> >> >>
> > >> >> >> > Hi,
> > >> >> >> >
> > >> >> >> > I'll update/upload Homebrew, MSYS2 and RubyGems.
> > >> >> > Thanks Kou!
> > >> >> >> >
> > >> >> >> > 1. [done] make the released version as "RELEASED" on JIRA
> > >> >> >> > 2. [done] start the new version on JIRA
> > >> >> >> > 4. [done] upload source
> > >> >> >> > 5. [done] upload binaries
> > >> >> >> > 6. [in-pr] update website
> > >> >> >> > 7. [todo:kou] update Homebrew packages
> > >> >> >> > 8. [todo:kou] update MSYS2 package
> > >> >> >> > 9. [todo:kou] upload RubyGems
> > >> >> >> > 10. [done] upload JS packages
> > >> >> >> > 11. [done] upload C# packages
> > >> >> >> > 12. [todo:unassigned] update conda recipes
> > >> >> >> > 13. [blocked:kszucs] upload wheels/sdist to pypi
> > >> >> >> > "Project size too large. Limit for project 'pyarrow' tot

Re: [VOTE] Release Apache Arrow 7.0.0 - RC10

[Krisztián Szűcs]

On 2022. Feb 9., Wed at 6:11, Sutou Kouhei  wrote:

> Thanks for confirming them!
> I've pressed the "Release" button on
> https://repository.apache.org/ .


Thanks Chao for checking and Kou for publishing the artifacts!
Can we incorporate the staged artifact checking to the verification
script/tasks somehow?

>
>
> In 
>   "Re: [VOTE] Release Apache Arrow 7.0.0 - RC10" on Tue, 8 Feb 2022
> 21:06:18 -0800,
>   Chao Sun  wrote:
>
> > The Java Maven artifacts look good to me. I was able to upgrade Apache
> > Spark to use the staging artifacts and all the tests successfully passed.
> > It's also great to see the `arrow-c-data` module finally get published in
> > this release!
> >
> > Chao
> >
> > On Tue, Feb 8, 2022 at 5:51 PM Ian Joiner  wrote:
> >
> >> The URLs are good. It is the values in the drop-down list that still
> >> need to be fixed. Please see the attached photo.
> >>
> >> Ian
> >>
> >>
> >>
> >> On Tuesday, February 8, 2022, Sutou Kouhei  wrote:
> >> >
> >> > Could you show URL that is occurred?
> >> >
> >> > It seems that the following URLs show correct versions:
> >> >
> >> > * https://arrow.apache.org/docs/6.0/index.html
> >> > * https://arrow.apache.org/docs/index.html
> >> > * https://arrow.apache.org/docs/dev/index.html
> >> >
> >> >
> >> > Thanks,
> >> > --
> >> > kou
> >> >
> >> > In  9s-ngvqnyb...@mail.gmail.com>
> >> >   "Re: [VOTE] Release Apache Arrow 7.0.0 - RC10" on Tue, 8 Feb 2022
> >> 17:35:47 -0500,
> >> >   Ian Joiner  wrote:
> >> >
> >> > > Really thanks!
> >> > >
> >> > > I do need to mention that versioning in the docs is still not
> displayed
> >> > > properly (7.0 is labelled “6.0 (stable)” while 8.0 is labelled “7.0
> >> (dev)”).
> >> > >
> >> > > Ian
> >> > >
> >> > > On Tuesday, February 8, 2022, Sutou Kouhei 
> wrote:
> >> > >
> >> > >> Homebrew, MSYS2 and RubyGems are done:
> >> > >>
> >> > >> 1. [done] make the released version as "RELEASED" on JIRA
> >> > >> 2. [done] start the new version on JIRA
> >> > >> 4. [done] upload source
> >> > >> 5. [done] upload binaries
> >> > >> 6. [done] update website
> >> > >> 7. [done] update Homebrew packages
> >> > >> 8. [done] update MSYS2 package
> >> > >> 9. [done] upload RubyGems
> >> > >> 10. [done] upload JS packages
> >> > >> 11. [done] upload C# packages
> >> > >> 12. [todo:unassigned] update conda recipes
> >> > >> 13. [done] upload wheels/sdist to pypi
> >> > >> 14. [todo:kszucs] publish Maven artifacts
> >> > >> 15. [todo:nealrichardson] update R packages
> >> > >> 16. [todo:ianmcook] update vcpkg port
> >> > >> 17. [done] bump versions
> >> > >> 18. [done] update tags for Go modules
> >> > >> 19. [done] update docs
> >> > >>
> >> > >> In <
> >> cahm19a4se5wf8_fj3mwk-pksjdthe+d_sry3xdnbtjm+jjn...@mail.gmail.com>
> >> > >>   "Re: [VOTE] Release Apache Arrow 7.0.0 - RC10" on Thu, 3 Feb 2022
> >> > >> 23:31:45 +0100,
> >> > >>   Krisztián Szűcs  wrote:
> >> > >>
> >> > >> > I managed to upload the wheels to pypi. Here is the current
> status
> >> > >> > with the updated assignments:
> >> > >> >
> >> > >> > 1. [done] make the released version as "RELEASED" on JIRA
> >> > >> > 2. [done] start the new version on JIRA
> >> > >> > 4. [done] upload source
> >> > >> > 5. [done] upload binaries
> >> > >> > 6. [in-pr] update website
> >> > >> > 7. [todo:kou] update Homebrew packages
> >> > >> > 8. [todo:kou] update MSYS2 package
> >> > >> > 9. [todo:kou] upload RubyGems
> >> > >> > 10. [done] upload JS packages
> >> > >> > 11. [done] upload C# packages
> >> > >> > 12. [todo:unassigned] update conda recipes
> >> > >> > 13. [done] upload wheels/sdist to pypi
> >> > >> > 14. [todo:kszucs] publish Maven artifacts
> >> > >> > 15. [todo:nealrichardson] update R packages
> >> > >> > 16. [todo:ianmcook] update vcpkg port
> >> > >> > 17. [done] bump versions
> >> > >> > 18. [done] update tags for Go modules
> >> > >> > 19. [in-pr] update docs
> >> > >> >
> >> > >> > On Thu, Feb 3, 2022 at 10:26 PM Neal Richardson
> >> > >> >  wrote:
> >> > >> >>
> >> > >> >> I will handle the R package submission to CRAN.
> >> > >> > Thanks Neal!
> >> > >> >>
> >> > >> >> Neal
> >> > >> >>
> >> > >> >> On Thu, Feb 3, 2022 at 4:11 PM Sutou Kouhei  >
> >> wrote:
> >> > >> >>
> >> > >> >> > Hi,
> >> > >> >> >
> >> > >> >> > I'll update/upload Homebrew, MSYS2 and RubyGems.
> >> > >> > Thanks Kou!
> >> > >> >> >
> >> > >> >> > 1. [done] make the released version as "RELEASED" on JIRA
> >> > >> >> > 2. [done] start the new version on JIRA
> >> > >> >> > 4. [done] upload source
> >> > >> >> > 5. [done] upload binaries
> >> > >> >> > 6. [in-pr] update website
> >> > >> >> > 7. [todo:kou] update Homebrew packages
> >> > >> >> > 8. [todo:kou] update MSYS2 package
> >> > >> >> > 9. [todo:kou] upload RubyGems
> >> > >> >> > 10. [done] upload JS packages
> >> > >> >> > 11. [done] upload C# packages
> >> > >> >> > 12. [todo:unassigned] update conda recipes
> >> > >> >> > 13. [blocked:kszucs] upload wheels/sdist to pypi
> >> > >> >> > "Project size too large. Limit for pr