Hi everyone, unfortunately my vote is -1 (although non-binding) due to a classpath error which prevents queries involving Tez to complete (all the details at the end of the email, apologies for the lengthy text but I wanted to provide all the context).
- verified gpg signature: OK $ wget https://www.apache.org/dist/hive/KEYS $ gpg --import KEYS ... $ gpg --verify apache-hive-4.0.0-alpha-2-bin.tar.gz.asc apache-hive-4.0.0-alpha-2-bin.tar.gz gpg: Signature made Thu 27 Oct 15:11:48 2022 CEST gpg: using RSA key 50606DE1BDBD5CF862A595A907C5682DAFC73125 gpg: issuer "dkuzme...@apache.org" gpg: Good signature from "Denys Kuzmenko (CODE SIGNING KEY) < dkuzme...@apache.org>" [unknown] gpg: WARNING: The key's User ID is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: 5060 6DE1 BDBD 5CF8 62A5 95A9 07C5 682D AFC7 3125 $ gpg --verify apache-hive-4.0.0-alpha-2-src.tar.gz.asc apache-hive-4.0.0-alpha-2-src.tar.gz gpg: Signature made Thu 27 Oct 15:12:08 2022 CEST gpg: using RSA key 50606DE1BDBD5CF862A595A907C5682DAFC73125 gpg: issuer "dkuzme...@apache.org" gpg: Good signature from "Denys Kuzmenko (CODE SIGNING KEY) < dkuzme...@apache.org>" [unknown] gpg: WARNING: The key's User ID is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: 5060 6DE1 BDBD 5CF8 62A5 95A9 07C5 682D AFC7 3125 (AFAIK, this warning is OK) - verified package checksum: OK $ diff <(cat apache-hive-4.0.0-alpha-2-src.tar.gz.sha256) <(shasum -a 256 apache-hive-4.0.0-alpha-2-src.tar.gz) $ diff <(cat apache-hive-4.0.0-alpha-2-bin.tar.gz.sha256) <(shasum -a 256 apache-hive-4.0.0-alpha-2-bin.tar.gz) - verified maven build (no tests): OK $ mvn clean install -DskipTests ... [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 04:31 min - checked release notes: OK - checked few modules in Nexus: OK - environment used: $ sw_vers ProductName: macOS ProductVersion: 11.6.8 BuildVersion: 20G730 $ mvn --version Apache Maven 3.8.1 (05c21c65bdfed0f71a2f2ada8b84da59348c4c5d) Maven home: .../.sdkman/candidates/maven/current Java version: 1.8.0_292, vendor: AdoptOpenJDK, runtime: .../.sdkman/candidates/java/8.0.292.hs-adpt/jre Default locale: en_IE, platform encoding: UTF-8 OS name: "mac os x", version: "10.16", arch: "x86_64", family: "mac" $ java -version openjdk version "1.8.0_292" OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_292-b10) OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.292-b10, mixed mode) Testing in hive-dev-box (https://github.com/kgyrtkirk/hive-dev-box): KO This is the setup I have used: $ sw hadoop 3.1.0 $ sw tez 0.10.0 (tried also 0.10.1) $ sw hive https://people.apache.org/~dkuzmenko/release-4.0.0-alpha-2-rc0/apache-hive-4.0.0-alpha-2-bin.tar.gz In what follows the test data and query I have tried, with associated stacktrace for the error. It seems a classpath issue, probably there are multiple versions of the class ending up in the CP and the classloader happened to load the “wrong one”. CREATE TABLE test_stats_a (a int, b int) STORED AS ORC; > INSERT INTO test_stats_a (a, b) VALUES (0, 2); > INSERT INTO test_stats_a (a, b) VALUES (1, 2); > INSERT INTO test_stats_a (a, b) VALUES (2, 2); > INSERT INTO test_stats_a (a, b) VALUES (3, 2); > INSERT INTO test_stats_a (a, b) VALUES (4, 2); > INSERT INTO test_stats_a (a, b) VALUES (5, 2); > INSERT INTO test_stats_a (a, b) VALUES (6, 2); > INSERT INTO test_stats_a (a, b) VALUES (7, 2); > INSERT INTO test_stats_a (a, b) VALUES (8, 3); > INSERT INTO test_stats_a (a, b) VALUES (9, 4); > INSERT INTO test_stats_a (a, b) VALUES (10, 5); > INSERT INTO test_stats_a (a, b) VALUES (11, 6); > INSERT INTO test_stats_a (a, b) VALUES (12, 7); > INSERT INTO test_stats_a (a, b) VALUES (13, NULL); > INSERT INTO test_stats_a (a, b) VALUES (14, NULL); > CREATE TABLE test_stats_b (a int, b int) STORED AS ORC; > INSERT INTO test_stats_b (a, b) VALUES (0, 2); > INSERT INTO test_stats_b (a, b) VALUES (1, 2); > INSERT INTO test_stats_b (a, b) VALUES (2, 2); > INSERT INTO test_stats_b (a, b) VALUES (3, 2); > INSERT INTO test_stats_b (a, b) VALUES (4, 2); > INSERT INTO test_stats_b (a, b) VALUES (5, 2); > INSERT INTO test_stats_b (a, b) VALUES (6, 2); > INSERT INTO test_stats_b (a, b) VALUES (7, 2); > INSERT INTO test_stats_b (a, b) VALUES (8, 3); > INSERT INTO test_stats_b (a, b) VALUES (9, 4); > INSERT INTO test_stats_b (a, b) VALUES (10, 5); > INSERT INTO test_stats_b (a, b) VALUES (11, 6); > INSERT INTO test_stats_b (a, b) VALUES (12, 7); > INSERT INTO test_stats_b (a, b) VALUES (13, NULL); > INSERT INTO test_stats_b (a, b) VALUES (14, NULL); CREATE TABLE test_stats_c (a string, b int) STORED AS PARQUET; > INSERT INTO test_stats_c (a, b) VALUES ("a", 2); > INSERT INTO test_stats_c (a, b) VALUES ("b", 2); > INSERT INTO test_stats_c (a, b) VALUES ("c", 2); > INSERT INTO test_stats_c (a, b) VALUES ("d", 2); > INSERT INTO test_stats_c (a, b) VALUES ("e", 2); > INSERT INTO test_stats_c (a, b) VALUES ("f", 2); > INSERT INTO test_stats_c (a, b) VALUES ("g", 2); > INSERT INTO test_stats_c (a, b) VALUES ("h", 2); > INSERT INTO test_stats_c (a, b) VALUES ("i", 3); > INSERT INTO test_stats_c (a, b) VALUES ("j", 4); > INSERT INTO test_stats_c (a, b) VALUES ("k", 5); > INSERT INTO test_stats_c (a, b) VALUES ("l", 6); > INSERT INTO test_stats_c (a, b) VALUES ("m", 7); > INSERT INTO test_stats_c (a, b) VALUES ("n", NULL); > INSERT INTO test_stats_c (a, b) VALUES ("o", NULL); SELECT * FROM test_stats_a t1 JOIN test_stats_b t2 ON (t1.a = t2.a) WHERE > t1.b < 3 AND t2.b > 1; INFO : Completed compiling > command(queryId=dev_20221027162822_de90e30b-a80a-427e-869a-b71799222f4b); > Time taken: > 4.171 seconds > INFO : Operation QUERY obtained 0 locks > INFO : Executing > command(queryId=dev_20221027162822_de90e30b-a80a-427e-869a-b71799222f4b): > SELECT * FROM test_sta > ts_a t1 JOIN test_stats_b t2 ON (t1.a = t2.a) WHERE t1.b < 3 AND t2.b > 1 > INFO : Query ID = dev_20221027162822_de90e30b-a80a-427e-869a-b71799222f4b > INFO : Total jobs = 1 > INFO : Launching Job 1 out of 1 > INFO : Starting task [Stage-1:MAPRED] in serial mode > DEBUG : Task getting executed using mapred tag : > dev_20221027162822_de90e30b-a80a-427e-869a-b71799222f4b,userid=de > v > INFO : Subscribed to counters: [] for queryId: > dev_20221027162822_de90e30b-a80a-427e-869a-b71799222f4b > INFO : Tez session hasn't been created yet. Opening session > DEBUG : No local resources to process (other than hive-exec) > INFO : Dag name: SELECT * FROM test_st...... < 3 AND t2.b > 1 (Stage-1) > DEBUG : DagInfo: {"context":"Hive","description":"SELECT * FROM > test_stats_a t1 JOIN test_stats_b t2 ON (t1.a = t2 > .a) WHERE t1.b < 3 AND t2.b > 1"} > DEBUG : Setting Tez DAG access for > queryId=dev_20221027162822_de90e30b-a80a-427e-869a-b71799222f4b with > viewAclStr > ing=dev, modifyStr=dev > INFO : Setting tez.task.scale.memory.reserve-fraction to > 0.30000001192092896 > INFO : HS2 Host: [alpha2], Query ID: > [dev_20221027162822_de90e30b-a80a-427e-869a-b71799222f4b], Dag ID: > [dag_1666888075798_0001_1], DAG Session ID: [application_1666888075798_0001] > INFO : Status: Running (Executing on YARN cluster with App id > application_1666888075798_0001) > ERROR : Status: Failed > ERROR : Vertex failed, vertexName=Map 2, > vertexId=vertex_1666888075798_0001_1_01, diagnostics=[Vertex > vertex_1666888075798_0001_1_01 [Map 2] killed/failed due > to:ROOT_INPUT_INIT_FAILURE, Vertex Input: t2 initializer failed, > vertex=vertex_1666888075798_0001_1_01 [Map 2], java.lang.NoSuchMethodError: > org.apache.hadoop.fs.Path.compareTo(Lorg/apache/hadoop/fs/Path;)I at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator$InputSplitComparator.compare(HiveSplitGenerator.java:415) > at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator$InputSplitComparator.compare(HiveSplitGenerator.java:401) > at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) > at java.util.TimSort.sort(TimSort.java:220) > at java.util.Arrays.sort(Arrays.java:1438) at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:254) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:281) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:272) at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:272) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:256) > at > com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) at > com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) > ] > ERROR : Vertex failed, vertexName=Map 1, > vertexId=vertex_1666888075798_0001_1_00, diagnostics=[Vertex > vertex_1666888075798_0001_1_00 [Map 1] killed/failed due > to:ROOT_INPUT_INIT_FAILURE, Vertex Input: t1 initializer failed, > vertex=vertex_1666888075798_0001_1_00 [Map 1], java.lang.NoSuchMethodError: > org.apache.hadoop.fs.Path.compareTo(Lorg/apache/hadoop/fs/Path;)I at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator$InputSplitComparator.compare(HiveSplitGenerator.java:415) > at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator$InputSplitComparator.compare(HiveSplitGenerator.java:401) > at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) > at java.util.TimSort.sort(TimSort.java:220) > at java.util.Arrays.sort(Arrays.java:1438) at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:254) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:281) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:272) at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:272) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:256) > at > com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > ] > ERROR : DAG did not succeed due to VERTEX_FAILURE. failedVertices:2 > killedVertices:0 > ERROR : FAILED: Execution Error, return code 2 from > org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map > 2, vertexId=vertex_1666888075798_0001_1_01, diagnostics=[Vertex > vertex_1666888075798_0001_1_01 [Map 2] killed/failed due > to:ROOT_INPUT_INIT_FAILURE, Vertex Input: t2 initializer failed, > vertex=vertex_1666888075798_0001_1_01 [Map 2], java.lang.NoSuchMethodError: > org.apache.hadoop.fs.Path.compareTo(Lorg/apache/hadoop/fs/Path;)I at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator$InputSplitComparator.compare(HiveSplitGenerator.java:415) > at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator$InputSplitComparator.compare(HiveSplitGenerator.java:401) > at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) > at java.util.TimSort.sort(TimSort.java:220) > at java.util.Arrays.sort(Arrays.java:1438) > at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:254) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:281) at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:272) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:272) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:256) > at > com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > ]Vertex failed, vertexName=Map 1, vertexId=vertex_1666888075798_0001_1_00, > diagnostics=[Vertex vertex_1666888075798_0001_1_00 [Map 1] killed/failed > due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: t1 initializer failed, > vertex=vertex_1666888075798_0001_1_00 [Map 1], java.lang.NoSuchMethodError: > org.apache.hadoop.fs.Path.compareTo(Lorg/apache/hadoop/fs/Path;)I > at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator$InputSplitComparator.compare(HiveSplitGenerator.java:415) > at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator$InputSplitComparator.compare(HiveSplitGenerator.java:401) > at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) > at java.util.TimSort.sort(TimSort.java:220) > at java.util.Arrays.sort(Arrays.java:1438) > at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:254) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:281) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:272) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:272) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:256) > at > com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > ]DAG did not succeed due to VERTEX_FAILURE. failedVertices:2 > killedVertices:0 > DEBUG : Shutting down query SELECT * FROM test_stats_a t1 JOIN > test_stats_b t2 ON (t1.a = t2.a) WHERE t1.b < 3 AND t2.b > 1 > INFO : Completed executing > command(queryId=dev_20221027162822_de90e30b-a80a-427e-869a-b71799222f4b); > Time taken: 6.983 seconds > DEBUG : Shutting down query SELECT * FROM test_stats_a t1 JOIN > test_stats_b t2 ON (t1.a = t2.a) WHERE t1.b < 3 AND t2.b > 1 > Error: Error while compiling statement: FAILED: Execution Error, return > code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, > vertexName=Map 2, vertexId=vertex_1666888075798_0001_1_01, > diagnostics=[Vertex vertex_1666888075798_0001_1_01 [Map 2] killed/failed > due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: t2 initializer failed, > vertex=vertex_1666888075798_0001_1_01 [Map 2], java.lang.NoSuchMethodError: > org.apache.hadoop.fs.Path.compareTo(Lorg/apache/hadoop/fs/Path;)I > at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator$InputSplitComparator.compare(HiveSplitGenerator.java:415) > at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator$InputSplitComparator.compare(HiveSplitGenerator.java:401) > at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) > at java.util.TimSort.sort(TimSort.java:220) > at java.util.Arrays.sort(Arrays.java:1438) > at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:254) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:281) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:272) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:272) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:256) > at > com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > ]Vertex failed, vertexName=Map 1, vertexId=vertex_1666888075798_0001_1_00, > diagnostics=[Vertex vertex_1666888075798_0001_1_00 [Map 1] killed/failed > due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: t1 initializer failed, > vertex=vertex_1666888075798_0001_1_00 [Map 1], java.lang.NoSuchMethodError: > org.apache.hadoop.fs.Path.compareTo(Lorg/apache/hadoop/fs/Path;)I > at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator$InputSplitComparator.compare(HiveSplitGenerator.java:415) > at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator$InputSplitComparator.compare(HiveSplitGenerator.java:401) > at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) > at java.util.TimSort.sort(TimSort.java:220) > at java.util.Arrays.sort(Arrays.java:1438) > at > org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:254) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:281) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:272) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:272) > at > org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:256) > at > com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) > at > com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) > at > com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > ]DAG did not succeed due to VERTEX_FAILURE. failedVertices:2 > killedVertices:0 (state=08S01,code=2) at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > Best regards, Alessandro On Thu, 27 Oct 2022 at 19:01, Ayush Saxena <ayush...@gmail.com> wrote: > Chris, > The KEYS file is at: > https://downloads.apache.org/hive/KEYS > > -Ayush > > On Thu, 27 Oct 2022 at 21:58, Chris Nauroth <cnaur...@apache.org> wrote: > > > Could someone please point me toward the right KEYS file to import so > that > > I can verify signatures? Thanks! > > > > I'm seeing numerous test failures due to "Insufficient configured > threads" > > while trying to start the HTTP server. One example is > > TestBeelinePasswordOption. Is anyone else seeing this? I noticed that > > HIVE-24484 set hive.server2.webui.max.threads to 4 in > > /data/conf/hive-site.xml. (The default in HiveConf.java is 50.) > > > > [INFO] Running org.apache.hive.beeline.TestBeelinePasswordOption > > [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: > > 11.742 s <<< FAILURE! - in > > org.apache.hive.beeline.TestBeelinePasswordOption > > [ERROR] org.apache.hive.beeline.TestBeelinePasswordOption Time elapsed: > > 11.733 s <<< ERROR! > > org.apache.hive.service.ServiceException: > java.lang.IllegalStateException: > > Insufficient configured threads: required=4 < max=4 for > > > > > QueuedThreadPool[hiveserver2-web]@628bd77e{STARTED,4<=4<=4,i=4,r=-1,q=0}[ReservedThreadExecutor@cfacf0 > > {s=0/1,p=0}] > > at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:733) > > at org.apache.hive.jdbc.miniHS2.MiniHS2.start(MiniHS2.java:395) > > at > > > > > org.apache.hive.beeline.TestBeelinePasswordOption.preTests(TestBeelinePasswordOption.java:60) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:498) > > at > > > > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > > at > > > > > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > > at > > > > > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > > at > > > > > org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) > > at > > > > > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) > > at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) > > at org.junit.runners.ParentRunner.run(ParentRunner.java:413) > > at > > > > > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) > > at > > > > > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) > > at > > > > > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) > > at > > > > > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) > > at > > > > > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377) > > at > > > > > org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138) > > at > org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465) > > at > > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451) > > Caused by: java.lang.IllegalStateException: Insufficient configured > > threads: required=4 < max=4 for > > > > > QueuedThreadPool[hiveserver2-web]@628bd77e{STARTED,4<=4<=4,i=4,r=-1,q=0}[ReservedThreadExecutor@cfacf0 > > {s=0/1,p=0}] > > at > > > > > org.eclipse.jetty.util.thread.ThreadPoolBudget.check(ThreadPoolBudget.java:165) > > at > > > > > org.eclipse.jetty.util.thread.ThreadPoolBudget.leaseTo(ThreadPoolBudget.java:141) > > at > > > > > org.eclipse.jetty.util.thread.ThreadPoolBudget.leaseFrom(ThreadPoolBudget.java:191) > > at org.eclipse.jetty.io > .SelectorManager.doStart(SelectorManager.java:255) > > at > > > > > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > > at > > > > > org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) > > at > > > > > org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) > > at > > > > > org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:321) > > at > > > > > org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:81) > > at > > > org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:234) > > at > > > > > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > > at org.eclipse.jetty.server.Server.doStart(Server.java:401) > > at > > > > > org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) > > at org.apache.hive.http.HttpServer.start(HttpServer.java:335) > > at org.apache.hive.service.server.HiveServer2.start(HiveServer2.java:729) > > ... 21 more > > > > Chris Nauroth > > > > > > On Thu, Oct 27, 2022 at 7:48 AM Ádám Szita <sz...@apache.org> wrote: > > > > > Hi, > > > > > > Thanks for rebuilding this RC, Denys. > > > > > > Alessandro: IMHO since there was no vote cast yet and we're talking > about > > > a build option change only, I guess it just doesn't worth rebuilding > the > > > whole stuff from scratch to create a new RC. > > > > > > I give +1 (binding) to this RC, I verified the checksum, binary > content, > > > source content, built Hive from source and also tried out the artifacts > > in > > > a mini cluster environment. Built an HMS DB with the schema scripts > > > provided, did table creation, insert, delete, rollback (Iceberg). > > > > > > Thanks again, Denys for taking this up. > > > > > > On 2022/10/27 13:29:36 Alessandro Solimando wrote: > > > > Hi Denys, > > > > in other Apache communities I generally see that votes are cancelled > > and > > > a > > > > new RC is prepared when there are changes or blocking issues like in > > this > > > > case, not sure how things are done in Hive though. > > > > > > > > Best regards, > > > > Alessandro > > > > > > > > On Thu, 27 Oct 2022 at 15:22, Denys Kuzmenko <dkuzme...@cloudera.com > > > .invalid> > > > > wrote: > > > > > > > > > Hi Adam, > > > > > > > > > > Thanks for pointing that out! Upstream release guide is outdated. > > Once > > > I > > > > > receive the edit rights, I'll amend the instructions. > > > > > Updated the release artifacts and checksums: > > > > > > > > > > Apache Hive 4.0.0-alpha-2 Release Candidate 0 is available > > > > > here: > https://people.apache.org/~dkuzmenko/release-4.0.0-alpha-2-rc0/ > > > > > > > > > > > > > > > The checksums are these: > > > > > - b4dbaac5530694f631af13677ffe5443addc148bd94176b27a109a6da67f5e0f > > > > > apache-hive-4.0.0-alpha-2-bin.tar.gz > > > > > - 8c4639915e9bf649f4a55cd9adb9d266aa15d8fa48ddfadb28ebead2c0aee4d0 > > > > > apache-hive-4.0.0-alpha-2-src.tar.gz > > > > > > > > > > Maven artifacts are available > > > > > here: > > > > > > > https://repository.apache.org/content/repositories/orgapachehive-1117/ > > > > > > > > > > The tag release-4.0.0-alpha-2-rc0 has been applied to the source > for > > > > > this release in github, you can see it at > > > > > https://github.com/apache/hive/tree/release-4.0.0-alpha-2-rc0 > > > > > > > > > > The git commit hash > > > > > is: > > > > > > > > > > > https://github.com/apache/hive/commit/da146200e003712e324496bf560a1702485d231c > > > > > > > > > > > > > > > Please check again. > > > > > > > > > > > > > > > Thanks, > > > > > Denys > > > > > > > > > > On Thu, Oct 27, 2022 at 2:53 PM Ádám Szita <sz...@apache.org> > wrote: > > > > > > > > > > > Hi Denys, > > > > > > > > > > > > Unfortutantely I can't give a plus 1 on this yet, as the Iceberg > > > > > artifacts > > > > > > are missing from the binary tar.gz. Perhaps -Piceberg flag was > > > missing > > > > > > during build, can you please rebuild? > > > > > > > > > > > > Thanks, > > > > > > Adam > > > > > > > > > > > > On 2022/10/25 11:20:23 Denys Kuzmenko wrote: > > > > > > > Hi team, > > > > > > > > > > > > > > > > > > > > > Apache Hive 4.0.0-alpha-2 Release Candidate 0 is available > > > > > > > here: > > > https://people.apache.org/~dkuzmenko/release-4.0.0-alpha-2-rc0/ > > > > > > > > > > > > > > > > > > > > > The checksums are these: > > > > > > > - > > 7d4c54ecfe2b04cabc283a84defcc1e8a02eed0e13baba2a2c91ae882b6bfaf7 > > > > > > > apache-hive-4.0.0-alpha-2-bin.tar.gz > > > > > > > - > > 8c4639915e9bf649f4a55cd9adb9d266aa15d8fa48ddfadb28ebead2c0aee4d0 > > > > > > > apache-hive-4.0.0-alpha-2-src.tar.gz > > > > > > > > > > > > > > Maven artifacts are available > > > > > > > here: > > > > > > > > > https://repository.apache.org/content/repositories/orgapachehive-1117/ > > > > > > > > > > > > > > The tag release-4.0.0-alpha-2-rc0 has been applied to the > source > > > for > > > > > > > this release in github, you can see it at > > > > > > > https://github.com/apache/hive/tree/release-4.0.0-alpha-2-rc0 > > > > > > > > > > > > > > The git commit hash > > > > > > > is: > > > > > > > > > > > > > > > > > https://github.com/apache/hive/commit/da146200e003712e324496bf560a1702485d231c > > > > > > > > > > > > > > Voting will conclude in 72 hours. > > > > > > > > > > > > > > Hive PMC Members: Please test and vote. > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > > >
