+1 (binding) verified sigs/sums/license/build/tests
As for the detached commit, I believe I commented on this in a prior release and the parent commit is the head of the 0.13.x branch and the detached commit is just the version bump, so I'm ok with it, but it sure would be nice if that wasn't detached. -Dan On Sun, Jun 5, 2022 at 10:27 PM Kyle Bendickson <[email protected]> wrote: > Update: > > Running the test suite in IntelliJ that was (and is) having consistent > test failures via CLI, the issue seems to be resolved. > So I do think it is indeed a local JVM set up issue. > > Investigating the differences now, but the class in question is > *org.apache.iceberg.mr.hive.TestHiveIcebergStorageHandlerWithEngine* > > It seems to be caused by a NoClassDefFoundError, specifically for > org.xerial.snappy.Snappy. It also happens for ORC, but not for parquet. > > Included is a sample output: > ``` > java.lang.NoClassDefFoundError: Could not initialize class > org.xerial.snappy.Snappy > at org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:99) > ~[snappy-java-1.1.8.jar:1.1.8] > at org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:91) > ~[snappy-java-1.1.8.jar:1.1.8] > at org.xerial.snappy.SnappyOutputStream.<init>(SnappyOutputStream.java:81) > ~[snappy-java-1.1.8.jar:1.1.8] > at > org.apache.tez.common.TezUtils.createByteStringFromConf(TezUtils.java:81) > ~[tez-api-0.10.1.jar:0.10.1] > ``` > > Apologies for speaking too soon. *I'm now +0 [non-binding] *provided we > fix the 0.13.x branch and associated commitId to not be in a detached > state. The tag *apache-iceberg-0.13.2-rc1 *works just fine, but the > 0.13.x branch doesn't have the commit ID in question. Not sure if that's a > major concern or not. > > Cheers, > Kyle > > On Sun, Jun 5, 2022 at 11:51 AM Kyle Bendickson <[email protected]> wrote: > >> Thanks Eduard! >> >> I have: >> - verified the signature >> - verified the checksum in the file given as well as of the artifact >> - ran all unit tests on Java 11, all passed >> - ran all unit tests on Java 8, some hive-3 tests consistently fail (I do >> notice they passed on Github - but the tests which fail are consistent >> despite giving the JVM more memory and checking for OOM) >> - ran a simple smoke test suite of CRUD on namespaces and v1 and v2 >> tables with Spark (3.2, 3.1) and Flink (1.13 and 1.14). >> - ran some upsert related tests on Flink 1.13 and 1.14 (1.12 is provided >> a deprecation notice) >> >> *Problems:* >> I did notice that the *given commit ID is considered unattached (and I >> wasn't able to check it out).* I am running my tests by using the >> provided JAR with engines and then running unit tests locally for the >> commit just prior (with commit ID >> *fae977a9f0a79266a04647b0df2ab540cf0dcff4*). >> >> Not sure if this is a huge issue, but outside of this unattached commit, >> my only concern is the `iceberg-hive3` failing tests, but as they passed in >> CI it's possibly an issue with my local setup locally. >> >> Running hive-3 test suite alone, the same tests failed multiple times but >> again might be something to do with my computer / JVM configuration. >> >> *I am -1 (non-binding)*, primarily based on the detached commit (as I >> had quite a good bit of trouble trying to fetch it through my normal >> processes) as well as the failing hive3 tests (though that's not exactly >> within my area of expertise). >> >> If the hive3 test failures are only something that occurs for me, then if >> we fix the "Add version.txt commit" in branch 0.3.x such that when I fetch >> branch 0.3.x it's present, I'd be +1. Unfortunately, I can't help with >> cleaning up with the release branch outside of advising somebody else (if >> desired), but I'm happy to help with that. >> >> The hive3 test failures for me seem to be OOM related, but I raised my >> >> Find attached a picture of the detached commit ID, >> *0784d64a659abd4fdaa82cdb599a250a7514facf*, per Github. >> >> [image: image.png] >> >> Example test failures >> org.apache.iceberg.mr.hive.TestHiveIcebergStorageHandlerWithEngine > >> testCBOWithSelectedColumnsOverlapJoin[fileFormat=AVRO, engine=tez, >> catalog=HIVE_CATALOG, isVectorized=false] FAILED >> java.lang.IllegalArgumentException: Failed to execute Hive query >> 'SELECT c.first_name, o.order_id FROM default.orders o JOIN >> default.customers c ON o.customer_id = c.customer_id ORDER BY o.order_id >> DESC': Error while processing statement: FAILED: Execution Error, return >> code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask >> at >> org.apache.iceberg.mr.hive.TestHiveShell.executeStatement(TestHiveShell.java:152) >> at >> org.apache.iceberg.mr.hive.TestHiveIcebergStorageHandlerWithEngine.testCBOWithSelectedColumnsOverlapJoin(TestHiveIcebergStorageHandlerWithEngine.java:236) >> >> Caused by: >> org.apache.hive.service.cli.HiveSQLException: Error while >> processing statement: FAILED: Execution Error, return code 1 from >> org.apache.hadoop.hive.ql.exec.tez.TezTask >> at >> org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:335) >> at >> org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:226) >> at >> org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:263) >> at >> org.apache.hive.service.cli.operation.Operation.run(Operation.java:247) >> at >> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:541) >> at >> org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveSessionImpl.java:510) >> at >> org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:267) >> at >> org.apache.iceberg.mr.hive.TestHiveShell.executeStatement(TestHiveShell.java:139) >> ... 1 more >> >> Thanks you for working on this, >> Kyle >> >> >> >> >> On Wed, Jun 1, 2022 at 11:12 PM Eduard Tudenhoefner <[email protected]> >> wrote: >> >>> Hi Everyone, >>> >>> I propose that we release the following RC as the official Apache >>> Iceberg 0.13.2 release. >>> >>> The commit ID is *0784d64a659abd4fdaa82cdb599a250a7514facf* >>> >>> >>> - This corresponds to the tag: *apache-iceberg-0.13.2-rc1* >>> - https://github.com/apache/iceberg/commits/apache-iceberg-0.13.2-rc1 >>> - >>> >>> https://github.com/apache/iceberg/tree/0784d64a659abd4fdaa82cdb599a250a7514facf >>> >>> >>> The release tarball, signature, and checksums are here: >>> >>> - >>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-0.13.2-rc1 >>> >>> >>> You can find the KEYS file here: >>> >>> - https://dist.apache.org/repos/dist/dev/iceberg/KEYS >>> >>> >>> Convenience binary artifacts are staged on Nexus. The Maven repository >>> URL is: >>> >>> - >>> https://repository.apache.org/content/repositories/orgapacheiceberg-1088/ >>> >>> >>> Please download, verify, and test. >>> >>> Please vote in the next 72 hours. >>> >>> [ ] +1 Release this as Apache Iceberg <VERSION> >>> [ ] +0 >>> [ ] -1 Do not release this because... >>> >>
