Re: Is 7.0.0 release missing the Java arrow-flight POM?
Hi, I found that "dev/release/04-binary-download.sh 7.0.0 10 --task-filter 'java-jars'" doesn't download arrow-flight*.pom. I think that we need to add arrow-flight*.pom to https://github.com/apache/arrow/blob/master/dev/tasks/tasks.yml#L761 . Thanks, -- kou In <20220220.142345.1095495044811966896@clear-code.com> "Re: Is 7.0.0 release missing the Java arrow-flight POM?" on Sun, 20 Feb 2022 14:23:45 +0900 (JST), Sutou Kouhei wrote: > Hi, > > I tried "dev/release/06-java-upload.sh 7.0.0 10" and upload > the log to > https://gist.github.com/kou/b6d8aa2b9420baa086a7cf0763a9bf37 > . It seems that arrow-flight isn't uploaded... > > You can see uploaded files at > https://repository.apache.org/#stagingRepositories with your > ASF account. > Note that you MUST not press the "Close" button! I'll remove > them by pressing "Drop" button when we fix this. > > > Thanks, > -- > kou > > In > "Re: Is 7.0.0 release missing the Java arrow-flight POM?" on Fri, 18 Feb > 2022 12:51:59 -0800, > Bryan Cutler wrote: > >> I wasn't able to able to run the entire process, so I downloaded a few >> artifacts from the nightly java-jars and pointed the script there to see >> the output: >> >> dev/release/06-java-upload.sh 7.0.0 10 >> >> deploy:deploy-file -Durl= >> https://repository.apache.org/service/local/staging/deploy/maven2 >> -DrepositoryId=apache.releases.https >> -DpomFile=./java-jars/arrow-flight-8.0.0.dev82.pom >> -Dfile=./java-jars/arrow-flight-8.0.0.dev82.pom -Dfiles= -Dtypes= >> -Dclassifiers= >> >> deploy:deploy-file -Durl= >> https://repository.apache.org/service/local/staging/deploy/maven2 >> -DrepositoryId=apache.releases.https >> -DpomFile=./java-jars/arrow-java-root-8.0.0.dev82.pom >> -Dfile=./java-jars/arrow-java-root-8.0.0.dev82.pom -Dfiles= -Dtypes= >> -Dclassifiers= >> >> deploy:deploy-file -Durl= >> https://repository.apache.org/service/local/staging/deploy/maven2 >> -DrepositoryId=apache.releases.https >> -DpomFile=./java-jars/flight-core-8.0.0.dev82.pom >> -Dfile=./java-jars/flight-core-8.0.0.dev82.jar -Dfiles= -Dtypes= >> -Dclassifiers= >> >> Based on that, it looks like the maven command is correct, it should deploy >> it just like the arrow-java-root pom. Is there any way to see the log >> output for when the release artifacts were deployed? >> >> On Thu, Feb 17, 2022 at 10:06 PM Bryan Cutler wrote: >> >>> Sure, I'll take a look at the script. >>> >>> On Thu, Feb 17, 2022 at 4:39 PM Sutou Kouhei wrote: >>> Hi, Ah, arrow-flight-*.pom exists on our CI artifacts: https://github.com/ursacomputing/crossbow/releases/tag/nightly-2022-02-17-0-github-java-jars I don't know why our upload script https://github.com/apache/arrow/blob/master/dev/release/06-java-upload.sh doesn't upload it... Could you take a look at it? Thanks, -- kou In "Re: Is 7.0.0 release missing the Java arrow-flight POM?" on Thu, 17 Feb 2022 14:14:52 -0800, Bryan Cutler wrote: > Yes, it's a little confusing. The original arrow-flight was when the Flight > implementation was a single module, then it got split into flight-core and > flight-grpc and arrow-flight was no more Now it's back but only as a parent > POM. I think the current structure seems correct, so I'm not sure if we > would want to go back to using flight-core as the parent. Is it possible to > deploy just the arrow-flight POM? Otherwise, people can't use Flight for > this release without some kind of hack. > > On Thu, Feb 17, 2022 at 12:45 PM Sutou Kouhei wrote: > >> Hi, >> >> It seems that arrow-flight isn't released after 0.15.1: >> https://repo1.maven.org/maven2/org/apache/arrow/arrow-flight/ >> >> But flight-core is released since 0.16.0: >> https://repo1.maven.org/maven2/org/apache/arrow/flight-core/ >> >> flight-grpc is also released: >> https://repo1.maven.org/maven2/org/apache/arrow/flight-grpc/ >> >> Can we use flight-core (and flight-grpc) instead of >> arrow-flight? >> >> Thanks, >> -- >> kou >> >> In >>> > >> "Is 7.0.0 release missing the Java arrow-flight POM?" on Thu, 17 Feb >> 2022 09:48:57 -0800, >> Bryan Cutler wrote: >> >> > Hi All, >> > >> > Congrats on the 7.0.0 release! I was trying it out and got an error not >> > being able to find arrow-flight-7.0.0.pom. This looks like a new parent >> POM >> > for Flight, so I checked maven central and don't see it deployed there. >> Not >> > sure what could have happened, but maybe it's only me. Anyone else seeing >> > the same issue? >> > >> > Thanks, >> > Bryan >> >>>
Re: Is 7.0.0 release missing the Java arrow-flight POM?
Hi, I tried "dev/release/06-java-upload.sh 7.0.0 10" and upload the log to https://gist.github.com/kou/b6d8aa2b9420baa086a7cf0763a9bf37 . It seems that arrow-flight isn't uploaded... You can see uploaded files at https://repository.apache.org/#stagingRepositories with your ASF account. Note that you MUST not press the "Close" button! I'll remove them by pressing "Drop" button when we fix this. Thanks, -- kou In "Re: Is 7.0.0 release missing the Java arrow-flight POM?" on Fri, 18 Feb 2022 12:51:59 -0800, Bryan Cutler wrote: > I wasn't able to able to run the entire process, so I downloaded a few > artifacts from the nightly java-jars and pointed the script there to see > the output: > > dev/release/06-java-upload.sh 7.0.0 10 > > deploy:deploy-file -Durl= > https://repository.apache.org/service/local/staging/deploy/maven2 > -DrepositoryId=apache.releases.https > -DpomFile=./java-jars/arrow-flight-8.0.0.dev82.pom > -Dfile=./java-jars/arrow-flight-8.0.0.dev82.pom -Dfiles= -Dtypes= > -Dclassifiers= > > deploy:deploy-file -Durl= > https://repository.apache.org/service/local/staging/deploy/maven2 > -DrepositoryId=apache.releases.https > -DpomFile=./java-jars/arrow-java-root-8.0.0.dev82.pom > -Dfile=./java-jars/arrow-java-root-8.0.0.dev82.pom -Dfiles= -Dtypes= > -Dclassifiers= > > deploy:deploy-file -Durl= > https://repository.apache.org/service/local/staging/deploy/maven2 > -DrepositoryId=apache.releases.https > -DpomFile=./java-jars/flight-core-8.0.0.dev82.pom > -Dfile=./java-jars/flight-core-8.0.0.dev82.jar -Dfiles= -Dtypes= > -Dclassifiers= > > Based on that, it looks like the maven command is correct, it should deploy > it just like the arrow-java-root pom. Is there any way to see the log > output for when the release artifacts were deployed? > > On Thu, Feb 17, 2022 at 10:06 PM Bryan Cutler wrote: > >> Sure, I'll take a look at the script. >> >> On Thu, Feb 17, 2022 at 4:39 PM Sutou Kouhei wrote: >> >>> Hi, >>> >>> Ah, arrow-flight-*.pom exists on our CI artifacts: >>> >>> https://github.com/ursacomputing/crossbow/releases/tag/nightly-2022-02-17-0-github-java-jars >>> >>> I don't know why our upload script >>> https://github.com/apache/arrow/blob/master/dev/release/06-java-upload.sh >>> doesn't upload it... >>> >>> Could you take a look at it? >>> >>> >>> Thanks, >>> -- >>> kou >>> >>> In >>> "Re: Is 7.0.0 release missing the Java arrow-flight POM?" on Thu, 17 >>> Feb 2022 14:14:52 -0800, >>> Bryan Cutler wrote: >>> >>> > Yes, it's a little confusing. The original arrow-flight was when the >>> Flight >>> > implementation was a single module, then it got split into flight-core >>> and >>> > flight-grpc and arrow-flight was no more Now it's back but only as a >>> parent >>> > POM. I think the current structure seems correct, so I'm not sure if we >>> > would want to go back to using flight-core as the parent. Is it >>> possible to >>> > deploy just the arrow-flight POM? Otherwise, people can't use Flight for >>> > this release without some kind of hack. >>> > >>> > On Thu, Feb 17, 2022 at 12:45 PM Sutou Kouhei >>> wrote: >>> > >>> >> Hi, >>> >> >>> >> It seems that arrow-flight isn't released after 0.15.1: >>> >> https://repo1.maven.org/maven2/org/apache/arrow/arrow-flight/ >>> >> >>> >> But flight-core is released since 0.16.0: >>> >> https://repo1.maven.org/maven2/org/apache/arrow/flight-core/ >>> >> >>> >> flight-grpc is also released: >>> >> https://repo1.maven.org/maven2/org/apache/arrow/flight-grpc/ >>> >> >>> >> Can we use flight-core (and flight-grpc) instead of >>> >> arrow-flight? >>> >> >>> >> Thanks, >>> >> -- >>> >> kou >>> >> >>> >> In >> > >>> >> "Is 7.0.0 release missing the Java arrow-flight POM?" on Thu, 17 Feb >>> >> 2022 09:48:57 -0800, >>> >> Bryan Cutler wrote: >>> >> >>> >> > Hi All, >>> >> > >>> >> > Congrats on the 7.0.0 release! I was trying it out and got an error >>> not >>> >> > being able to find arrow-flight-7.0.0.pom. This looks like a new >>> parent >>> >> POM >>> >> > for Flight, so I checked maven central and don't see it deployed >>> there. >>> >> Not >>> >> > sure what could have happened, but maybe it's only me. Anyone else >>> seeing >>> >> > the same issue? >>> >> > >>> >> > Thanks, >>> >> > Bryan >>> >> >>> >>
Re: [VOTE][RUST] Release Apache Arrow Rust 9.1.0 RC1
Hi, It seems that Neville forgot to add Neville's PGP key to * https://dist.apache.org/repos/dist/dev/arrow/KEYS * https://dist.apache.org/repos/dist/release/arrow/KEYS See the header comment of them how to add a PGP key. Committers can update them by Subversion client with their ASF account. e.g.: $ svn co https://dist.apache.org/repos/dist/dev/arrow $ cd arrow $ editor KEYS $ svn ci KEYS If Neville doesn't have a PGP key, https://infra.apache.org/release-signing.html#generate may be helpful. Thanks, -- kou In "Re: [VOTE][RUST] Release Apache Arrow Rust 9.1.0 RC1" on Sat, 19 Feb 2022 11:52:44 -0700, Andy Grove wrote: > I'm using the same process as usual (as far as I am aware) and the script > failed with: > > gpg: Signature made Sat 19 Feb 2022 10:01:41 AM MST > gpg:using EDDSA key 3905F254F9E504B40FFF6CF6000488D7717D3FB2 > gpg: Can't check signature: No public key > + cleanup > + '[' no = yes ']' > + echo 'Failed to verify release candidate. See /tmp/arrow-9.1.0.62KW4 for > details.' > > > On Sat, Feb 19, 2022 at 10:04 AM Neville Dipale > wrote: > >> Hi, >> >> I would like to propose a release of Apache Arrow Rust Implementation, >> version 9.1.0. >> >> This release candidate is based on commit: >> ecba7dc0830dbde6aa6dd9432519b776e40c1e85 [1] >> >> The proposed release tarball and signatures are hosted at [2]. >> >> The changelog is located at [3]. >> >> Please download, verify checksums and signatures, run the unit tests, >> and vote on the release. There is a script [4] that automates some of >> the verification. >> >> The vote will be open for at least 72 hours. >> >> [ ] +1 Release this as Apache Arrow Rust >> [ ] +0 >> [ ] -1 Do not release this as Apache Arrow Rust because... >> >> [1]: >> >> https://github.com/apache/arrow-rs/tree/ecba7dc0830dbde6aa6dd9432519b776e40c1e85 >> [2]: >> https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-rs-9.1.0-rc1 >> [3]: >> >> https://github.com/apache/arrow-rs/blob/ecba7dc0830dbde6aa6dd9432519b776e40c1e85/CHANGELOG.md >> [4]: >> >> https://github.com/apache/arrow-rs/blob/master/dev/release/verify-release-candidate.sh >>
Re: [VOTE][RUST] Release Apache Arrow Rust 9.1.0 RC1
My gpg signature is freshly added, and might not be on the directory yet as I understand it to be refreshed once daily. May you please try again later? I'll also check if the signature appears in a few hours. On Sat, 19 Feb 2022, 20:53 Andy Grove, wrote: > I'm using the same process as usual (as far as I am aware) and the script > failed with: > > gpg: Signature made Sat 19 Feb 2022 10:01:41 AM MST > gpg:using EDDSA key > 3905F254F9E504B40FFF6CF6000488D7717D3FB2 > gpg: Can't check signature: No public key > + cleanup > + '[' no = yes ']' > + echo 'Failed to verify release candidate. See /tmp/arrow-9.1.0.62KW4 for > details.' > > > On Sat, Feb 19, 2022 at 10:04 AM Neville Dipale > wrote: > > > Hi, > > > > I would like to propose a release of Apache Arrow Rust Implementation, > > version 9.1.0. > > > > This release candidate is based on commit: > > ecba7dc0830dbde6aa6dd9432519b776e40c1e85 [1] > > > > The proposed release tarball and signatures are hosted at [2]. > > > > The changelog is located at [3]. > > > > Please download, verify checksums and signatures, run the unit tests, > > and vote on the release. There is a script [4] that automates some of > > the verification. > > > > The vote will be open for at least 72 hours. > > > > [ ] +1 Release this as Apache Arrow Rust > > [ ] +0 > > [ ] -1 Do not release this as Apache Arrow Rust because... > > > > [1]: > > > > > https://github.com/apache/arrow-rs/tree/ecba7dc0830dbde6aa6dd9432519b776e40c1e85 > > [2]: > > https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-rs-9.1.0-rc1 > > [3]: > > > > > https://github.com/apache/arrow-rs/blob/ecba7dc0830dbde6aa6dd9432519b776e40c1e85/CHANGELOG.md > > [4]: > > > > > https://github.com/apache/arrow-rs/blob/master/dev/release/verify-release-candidate.sh > > >
Re: [VOTE][RUST] Release Apache Arrow Rust 9.1.0 RC1
I'm using the same process as usual (as far as I am aware) and the script failed with: gpg: Signature made Sat 19 Feb 2022 10:01:41 AM MST gpg:using EDDSA key 3905F254F9E504B40FFF6CF6000488D7717D3FB2 gpg: Can't check signature: No public key + cleanup + '[' no = yes ']' + echo 'Failed to verify release candidate. See /tmp/arrow-9.1.0.62KW4 for details.' On Sat, Feb 19, 2022 at 10:04 AM Neville Dipale wrote: > Hi, > > I would like to propose a release of Apache Arrow Rust Implementation, > version 9.1.0. > > This release candidate is based on commit: > ecba7dc0830dbde6aa6dd9432519b776e40c1e85 [1] > > The proposed release tarball and signatures are hosted at [2]. > > The changelog is located at [3]. > > Please download, verify checksums and signatures, run the unit tests, > and vote on the release. There is a script [4] that automates some of > the verification. > > The vote will be open for at least 72 hours. > > [ ] +1 Release this as Apache Arrow Rust > [ ] +0 > [ ] -1 Do not release this as Apache Arrow Rust because... > > [1]: > > https://github.com/apache/arrow-rs/tree/ecba7dc0830dbde6aa6dd9432519b776e40c1e85 > [2]: > https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-rs-9.1.0-rc1 > [3]: > > https://github.com/apache/arrow-rs/blob/ecba7dc0830dbde6aa6dd9432519b776e40c1e85/CHANGELOG.md > [4]: > > https://github.com/apache/arrow-rs/blob/master/dev/release/verify-release-candidate.sh >
[VOTE][RUST] Release Apache Arrow Rust 9.1.0 RC1
Hi, I would like to propose a release of Apache Arrow Rust Implementation, version 9.1.0. This release candidate is based on commit: ecba7dc0830dbde6aa6dd9432519b776e40c1e85 [1] The proposed release tarball and signatures are hosted at [2]. The changelog is located at [3]. Please download, verify checksums and signatures, run the unit tests, and vote on the release. There is a script [4] that automates some of the verification. The vote will be open for at least 72 hours. [ ] +1 Release this as Apache Arrow Rust [ ] +0 [ ] -1 Do not release this as Apache Arrow Rust because... [1]: https://github.com/apache/arrow-rs/tree/ecba7dc0830dbde6aa6dd9432519b776e40c1e85 [2]: https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-rs-9.1.0-rc1 [3]: https://github.com/apache/arrow-rs/blob/ecba7dc0830dbde6aa6dd9432519b776e40c1e85/CHANGELOG.md [4]: https://github.com/apache/arrow-rs/blob/master/dev/release/verify-release-candidate.sh
Re: [ANNOUNCE] New Arrow PMC member: QP Hou
Congratulations QP! 🥳 On Fri, 18 Feb 2022 at 09:42, Daniël Heres wrote: > Congratulations QP! > > On Fri, Feb 18, 2022, 06:55 Benson Muite > wrote: > > > Congratulations QP! > > On 2/18/22 8:35 AM, Jiayu Liu wrote: > > > Congratulations QP! > > > > > > On Fri, Feb 18, 2022 at 1:32 PM Micah Kornfield > > > > wrote: > > > > > >> Congrats! > > >> > > >> On Thu, Feb 17, 2022 at 7:27 PM Weston Pace > > wrote: > > >> > > >>> Congratulations QP! > > >>> > > >>> On Thu, Feb 17, 2022 at 3:22 PM hao Yang <1371656737...@gmail.com> > > >> wrote: > > > > Congratulations QP! > > > > On Fri, 18 Feb 2022 at 09:14, Vibhatha Abeykoon > > > >>> wrote: > > > > > Congratulations! > > > > > > On Fri, Feb 18, 2022 at 5:51 AM Yijie Shen < > > >> henry.yijies...@gmail.com> > > > wrote: > > > > > >> Congratulations QP! > > >> > > >> On Fri, Feb 18, 2022 at 6:17 AM Phillip Cloud > > >>> wrote: > > >> > > >>> Congratulations!! > > >>> > > >>> On Thu, Feb 17, 2022 at 5:12 PM Neal Richardson < > > >>> neal.p.richard...@gmail.com> > > >>> wrote: > > >>> > > Congratulations! > > > > Neal > > > > On Thu, Feb 17, 2022 at 4:48 PM Rok Mihevc < > > >> rok.mih...@gmail.com > > > > >> wrote: > > > > > Congrats QP! > > > > > > Rok > > > > > > On Thu, Feb 17, 2022 at 10:41 PM David Li < > > >> lidav...@apache.org > > > > >> wrote: > > >> > > >> Congrats QP! > > >> > > >> On Thu, Feb 17, 2022, at 16:26, Matthew Turner wrote: > > >>> Congratulations, QP! Appreciate all of your contributions > > >>> and > > guidance. > > >>> > > >>> From: Sutou Kouhei > > >>> Date: Thursday, February 17, 2022 at 3:17 PM > > >>> To: dev@arrow.apache.org > > >>> Subject: [ANNOUNCE] New Arrow PMC member: QP Hou > > >>> The Project Management Committee (PMC) for Apache Arrow > > >> has > > >> invited > > >>> QP Hou to become a PMC member and we are pleased to > > >>> announce > > >>> that QP Hou has accepted. > > >>> > > >>> Congratulations and welcome! > > > > > > > >>> > > >> > > > -- > > > Vibhatha Abeykoon > > > > > >>> > > >> > > > > > > > >
Re: [JavaScript] Appending Nulls to a Struct (Bug)
Le 18/02/2022 à 23:44, Micah Kornfield a écrit : Ok, then perhaps you might have some thoughts on the original question: is the JavaScript implementation currently incorrect? I think whether it is a bug or not depends on the contract of the builder. If the contract is that the builder assumes users will ensure equal lengths of all the children, then it is probably correct as is. If it is more consistent with the code that the builder of the struct should manage appending a placeholder-value to its children then it is a reasonable change. Agreed with Micah. What stands is that in a struct array, the child arrays must have the same length as the parent. Then it's a matter of how the builder API is defined, and different implementations may choose different strategies. Regards Antoine. I seem to recall at least in C++ that we actually changed the behavior of builders in this regard at some point, but I might be misremembering (the change might have been appending a place-holder value instead appends nulls, to lower the chances of needing validity buffers on children arrays if all values in the struct are null). Whatever the implementation is, the post-condition for the resulting struct array is that its length is equal to the length of all of its children arrays. Cheers, Micah On Fri, Feb 18, 2022 at 1:12 PM Phillip Cloud wrote: On Fri, Feb 18, 2022 at 3:44 PM Antoine Pitrou wrote: Le 18/02/2022 à 21:32, Phillip Cloud a écrit : I am really struggling to see how anything I've said is inconsistent with the spec or what you are saying here. To recap what I've said: 1. Appending a null sentinel to the values buffer isn't _required_ unless the type requires it. Ex: "joemark" in the spec example. No sentinels were append for the two null values in the parent struct array. There is no notion of sentinel in the Arrow format, so I don't understand what you're saying. The word "sentinel" is a linguistic placeholder for "some set of bytes". Hopefully that's clear from the context. (a sentinel is a physical value having a specific meaning, for example a data format that has no separate validity bitmap could use the integer value 42 to indicate null values in an integer array; the Arrow format has a separate validity bitmap and therefore doesn't make use of sentinel values) 2. Appending a null value sentinel is _allowed_ to be there if the type does not require it. Ex: "joefoofoomark" extending the spec example, assuming the other associated buffers (validity, offsets) are correctly constructed. Is either of those statements incorrect? To me, they simply don't make sense given that sentinels don't exist in Arrow. Do they make sense after substituting in "a null entry in a string array with a non-zero number of bytes"? That said, a null entry in a string array can be backed by a non-zero number of bytes in the values buffer. That is unrelated to the question about struct arrays. For example, "joefoofoomark" can very well be the values buffer for a string array with the logical values ["joe", null, "mark"]. In this case, the offsets will be [0, 3, 9, 13]. Ok, then perhaps you might have some thoughts on the original question: is the JavaScript implementation currently incorrect? Regards Antoine.
