Hi Ryan I raise this question because I want to make sure whether it's a bug in the current code base, if sure then we will need to fix this before the next 0.10.0 RC.
Now I think we've confirmed that it's NOT a bug, so let's continue the next release RC voting. Thanks for the attention. On Wed, Nov 4, 2020 at 1:31 AM Ryan Blue <rb...@netflix.com.invalid> wrote: > OpenInx, is that a general question or is it related to the release? It > doesn't look related, but I want to make sure. > > On Tue, Nov 3, 2020 at 5:41 AM OpenInx <open...@gmail.com> wrote: > >> Hi >> >> I will suggest taking a look at this discussion: >> https://github.com/apache/iceberg/pull/1391#discussion_r516518978. >> >> It sounds like a bug. >> >> Thanks. >> >> >> On Tue, Nov 3, 2020 at 12:08 PM Anton Okolnychyi >> <aokolnyc...@apple.com.invalid> wrote: >> >>> -1 due to the ORC correctness bug. I’ll build a new RC. >>> >>> Thanks everyone for checking the candidate! >>> >>> - Anton >>> >>> On 2 Nov 2020, at 19:01, Ryan Blue <rb...@netflix.com.INVALID> wrote: >>> >>> I think we are going to build another RC to get the ORC correctness >>> problem that Shardul just fixed in. >>> >>> On Mon, Nov 2, 2020 at 6:40 PM Jingsong Li <jingsongl...@gmail.com> >>> wrote: >>> >>>> +1 >>>> >>>> 1. Download the source tarball, signature (.asc), and checksum >>>> (.sha512): OK >>>> 2. Import gpg keys: download KEYS and run gpg --import >>>> /path/to/downloaded/KEYS (optional if this hasn’t changed) : OK >>>> 3. Verify the signature by running: gpg --verify >>>> apache-iceberg-xx-incubating.tar.gz.asc: OK >>>> 4. Verify the checksum by running: sha512sum -c >>>> apache-iceberg-xx-incubating.tar.gz.sha512 : OK >>>> 5. Untar the archive and go into the source directory: tar xzf >>>> apache-iceberg-xx-incubating.tar.gz && cd apache-iceberg-xx-incubating: OK >>>> 6. Run RAT checks to validate license headers: dev/check-license: OK >>>> 7. Build and test the project: ./gradlew build (use Java 8) : OK >>>> >>>> Best, >>>> Jingsong >>>> >>>> On Tue, Nov 3, 2020 at 7:10 AM Jungtaek Lim < >>>> kabhwan.opensou...@gmail.com> wrote: >>>> >>>>> Anton, thanks for the interest. It's across modules, and looks to be >>>>> consistent. Looks like others have no problem with UT runs, probably need >>>>> to have another mail thread or Github issue for this. >>>>> >>>>> On Tue, Nov 3, 2020 at 1:31 AM Anton Okolnychyi <aokolnyc...@apple.com> >>>>> wrote: >>>>> >>>>>> Jungtaek, do you hit this issue only in a specific suite/module? Is >>>>>> it consistent? >>>>>> >>>>>> I remember we had similar issues for our Spark tests. If I am not >>>>>> mistaken, the underlying problem was related to the number of connections >>>>>> we were making to HMS and the pool size that was configured in our test >>>>>> HMS. >>>>>> >>>>>> - Anton >>>>>> >>>>>> On 2 Nov 2020, at 07:02, Ryan Murray <rym...@dremio.com> wrote: >>>>>> >>>>>> +1 (non-binding) >>>>>> >>>>>> Ran through steps 1-7, completed successfully. Also tested locally >>>>>> against nessie and all looked good. >>>>>> >>>>>> Best, >>>>>> Ryan >>>>>> >>>>>> On Mon, Nov 2, 2020 at 2:28 PM Mass Dosage <massdos...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> +1 (non-binding) >>>>>>> >>>>>>> I ran the RC against a set of integration tests I have for a subset >>>>>>> of the Hive2 read functionality on a distributed cluster and it worked >>>>>>> fine. >>>>>>> >>>>>>> On Mon, 2 Nov 2020 at 04:05, Simon Su <barley0...@gmail.com> wrote: >>>>>>> >>>>>>>> + 1 (non-binding) >>>>>>>> 1. Build code pass all UTs. >>>>>>>> 2. Test Flink iceberg sink failover, test exactly-once. >>>>>>>> >>>>>>>> Junjie Chen <chenjunjied...@gmail.com> 于2020年11月2日周一 上午11:35写道： >>>>>>>> >>>>>>>>> + 1 (non-binding) >>>>>>>>> >>>>>>>>> I ran step 1-7 on my cloud virtual machine (centos 7, java >>>>>>>>> 1.8.0_171), all passed. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Mon, Nov 2, 2020 at 10:36 AM OpenInx <open...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> +1 for 0.10.0 RC2 >>>>>>>>>> >>>>>>>>>> 1. Download the source tarball, signature (.asc), and checksum >>>>>>>>>> (.sha512): OK >>>>>>>>>> 2. Import gpg keys: download KEYS and run gpg --import >>>>>>>>>> /path/to/downloaded/KEYS (optional if this hasn’t changed) : OK >>>>>>>>>> 3. Verify the signature by running: gpg --verify >>>>>>>>>> apache-iceberg-xx-incubating.tar.gz.asc: OK >>>>>>>>>> 4. Verify the checksum by running: sha512sum -c >>>>>>>>>> apache-iceberg-xx-incubating.tar.gz.sha512 : OK >>>>>>>>>> 5. Untar the archive and go into the source directory: tar xzf >>>>>>>>>> apache-iceberg-xx-incubating.tar.gz && cd >>>>>>>>>> apache-iceberg-xx-incubating: OK >>>>>>>>>> 6. Run RAT checks to validate license headers: dev/check-license: >>>>>>>>>> OK >>>>>>>>>> 7. Build and test the project: ./gradlew build (use Java 8) : OK >>>>>>>>>> >>>>>>>>>> BTW, I think we may need a bash script to do the above >>>>>>>>>> verification automatically , so I created an issue for it: >>>>>>>>>> https://github.com/apache/iceberg/issues/1700 >>>>>>>>>> >>>>>>>>>> Thanks all for the work. >>>>>>>>>> >>>>>>>>>> On Sun, Nov 1, 2020 at 5:40 AM Dongjoon Hyun < >>>>>>>>>> dongjoon.h...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> +1 for 0.10.0 RC2 (non-binding) >>>>>>>>>>> >>>>>>>>>>> I checked the followings. >>>>>>>>>>> 1. Checksum and GPG signatures >>>>>>>>>>> 2. Gradle build and tests on Java 1.8.0_272 >>>>>>>>>>> 3. Manual integration tests with Hive Metastore 2.3.7 and Apache >>>>>>>>>>> Spark 2.3.7/3.0.1. <http://3.0.0.1/> >>>>>>>>>>> >>>>>>>>>>> Thank you! >>>>>>>>>>> >>>>>>>>>>> Bests, >>>>>>>>>>> Dongjoon. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Fri, Oct 30, 2020 at 3:25 PM Russell Spitzer < >>>>>>>>>>> russell.spit...@gmail.com> wrote: >>>>>>>>>>> >>>>>>>>>>>> +1 (non-binding) Downloaded and ran build with Java HotSpot(TM) >>>>>>>>>>>> 64-Bit Server VM 18.9 (build 11.0.7+8-LTS, mixed mode). All tests >>>>>>>>>>>> passed :) >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Oct 30, 2020 at 4:05 PM Anton Okolnychyi < >>>>>>>>>>>> aokolnyc...@apple.com.invalid> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Here is the link to steps we normally use to validate a >>>>>>>>>>>>> release candidate: >>>>>>>>>>>>> >>>>>>>>>>>>> https://lists.apache.org/thread.html/rd5e6b1656ac80252a9a7d473b36b6227da91d07d86d4ba4bee10df66%40%3Cdev.iceberg.apache.org%3E >>>>>>>>>>>>> <https://lists.apache.org/thread.html/rd5e6b1656ac80252a9a7d473b36b6227da91d07d86d4ba4bee10df66@%3Cdev.iceberg.apache.org%3E> >>>>>>>>>>>>> >>>>>>>>>>>>> - Anton >>>>>>>>>>>>> >>>>>>>>>>>>> On 30 Oct 2020, at 14:03, Anton Okolnychyi < >>>>>>>>>>>>> aokolnyc...@apple.com.INVALID> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi everyone, >>>>>>>>>>>>> >>>>>>>>>>>>> I propose the following RC to be released as official Apache >>>>>>>>>>>>> Iceberg 0.10.0 release. >>>>>>>>>>>>> >>>>>>>>>>>>> The commit id is 37f21b72fb55503e6e40b1555b7ea1af61dfdfc7 >>>>>>>>>>>>> * This corresponds to the tag: apache-iceberg-0.10.0-rc2 >>>>>>>>>>>>> * >>>>>>>>>>>>> https://github.com/apache/iceberg/commits/apache-iceberg-0.10.0-rc2 >>>>>>>>>>>>> * >>>>>>>>>>>>> https://github.com/apache/iceberg/tree/37f21b72fb55503e6e40b1555b7ea1af61dfdfc7 >>>>>>>>>>>>> >>>>>>>>>>>>> The release tarball, signature, and checksums are here: >>>>>>>>>>>>> * >>>>>>>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-0.10.0-rc2/ >>>>>>>>>>>>> >>>>>>>>>>>>> You can find the KEYS file here (make sure to import the new >>>>>>>>>>>>> key that was used to sign the release): >>>>>>>>>>>>> * https://dist.apache.org/repos/dist/dev/iceberg/KEYS >>>>>>>>>>>>> >>>>>>>>>>>>> Convenience binary artifacts are staged in Nexus. The Maven >>>>>>>>>>>>> repository URL is: >>>>>>>>>>>>> * >>>>>>>>>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1011/ >>>>>>>>>>>>> >>>>>>>>>>>>> This release includes important changes: >>>>>>>>>>>>> >>>>>>>>>>>>> * Flink support >>>>>>>>>>>>> * Hive read support >>>>>>>>>>>>> * ORC support fixes and improvements >>>>>>>>>>>>> * Application of row-level delete files on read >>>>>>>>>>>>> * Snapshot partition summary >>>>>>>>>>>>> * Ability to load LocationProvider dynamically >>>>>>>>>>>>> * Sort spec >>>>>>>>>>>>> >>>>>>>>>>>>> Please download, verify, and test. >>>>>>>>>>>>> >>>>>>>>>>>>> Please vote in the next 72 hours. >>>>>>>>>>>>> >>>>>>>>>>>>> [ ] +1 Release this as Apache Iceberg 0.10.0 >>>>>>>>>>>>> [ ] +0 >>>>>>>>>>>>> [ ] -1 Do not release this because… >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Anton >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Best Regards >>>>>>>>> >>>>>>>> >>>>>> >>>> >>>> -- >>>> Best, Jingsong Lee >>>> >>> >>> >>> -- >>> Ryan Blue >>> Software Engineer >>> Netflix >>> >>> >>> > > -- > Ryan Blue > Software Engineer > Netflix >
