[jira] [Updated] (HIVE-28290) TestMetastoreAuthorizationProvider to run on Tez
[ https://issues.apache.org/jira/browse/HIVE-28290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-28290: Description: when I was doing HIVE-27972, it passed locally, but failed in precommit, need to investigate further {code} Error The dir: /home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir on HDFS should be writable. Current permissions are: rwxr-xr-x Stacktrace java.lang.RuntimeException: The dir: /home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir on HDFS should be writable. Current permissions are: rwxr-xr-x at org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:4966) at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:843) at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:784) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:708) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:669) at org.apache.hadoop.hive.ql.security.TestMetastoreAuthorizationProvider.setUp(TestMetastoreAuthorizationProvider.java:126) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.ParentRunner.run(ParentRunner.java:413) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138) at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451) {code} > TestMetastoreAuthorizationProvider to run on Tez > > > Key: HIVE-28290 > URL: https://issues.apache.org/jira/browse/HIVE-28290 > Project: Hive > Issue Type: Sub-task >Reporter: László Bodor >Priority: Major > > when I was doing HIVE-27972, it passed locally, but failed in precommit, need > to investigate further > {code} > Error > The dir: > /home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir > on HDFS should be writable. Current permissions are: rwxr-xr-x > Stacktrace > java.lang.RuntimeException: The dir: > /home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir > on HDFS should be writable. Current permissions are: rwxr-xr-x > at > org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:4966) > at > org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:843) > at > org.apache.hadoop.hive.ql.session.SessionState.c
[jira] [Updated] (HIVE-26018) The result of UNIQUEJOIN on Hive on Tez is inconsistent with that of MR
[ https://issues.apache.org/jira/browse/HIVE-26018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-26018: -- Labels: pull-request-available (was: ) > The result of UNIQUEJOIN on Hive on Tez is inconsistent with that of MR > --- > > Key: HIVE-26018 > URL: https://issues.apache.org/jira/browse/HIVE-26018 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 3.1.0, 4.0.0 >Reporter: GuangMing Lu >Priority: Major > Labels: pull-request-available > > The result of UNIQUEJOIN on Hive on Tez is inconsistent with that of MR, and > the result Is not correct, for example: > CREATE TABLE T1_n1x(key STRING, val STRING) STORED AS orc; > CREATE TABLE T2_n1x(key STRING, val STRING) STORED AS orc; > insert into T1_n1x values('aaa', '111'),('bbb', '222'),('ccc', '333'); > insert into T2_n1x values('aaa', '111'),('ddd', '444'),('ccc', '333'); > SELECT a.key, b.key FROM UNIQUEJOIN PRESERVE T1_n1x a (a.key), PRESERVE > T2_n1x b (b.key); > Hive on Tez result: wrong > |a.key |b.key | > |aaa |aaa | > |bbb |NULL | > |ccc |ccc | > |NULL |ddd | > +--+ > Hive on MR result: right > |a.key |b.key | > |aaa |aaa | > |bbb |NULL | > |ccc |ccc | > +-+ > SELECT a.key, b.key FROM UNIQUEJOIN T1_n1x a (a.key), T2_n1x b (b.key); > Hive on Tez result: wrong > +---+ > |a.key |b.key | > |aaa |aaa | > |bbb |NULL | > |ccc |ccc | > |NULL |ddd | > +-+ > Hive on MR result: right > |a.key |b.key | > |aaa |aaa | > |ccc |ccc | > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-28284) TestJdbcWithMiniHS2/TestJdbcWithMiniHS2ErasureCoding to run on Tez
[ https://issues.apache.org/jira/browse/HIVE-28284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-28284: Summary: TestJdbcWithMiniHS2/TestJdbcWithMiniHS2ErasureCoding to run on Tez (was: TestJdbcWithMiniHS2 to run on Tez) > TestJdbcWithMiniHS2/TestJdbcWithMiniHS2ErasureCoding to run on Tez > -- > > Key: HIVE-28284 > URL: https://issues.apache.org/jira/browse/HIVE-28284 > Project: Hive > Issue Type: Sub-task >Reporter: László Bodor >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-28291) Fix Partition spec is incorrect in getPartitionsByFilter RPC
dzcxzl created HIVE-28291: - Summary: Fix Partition spec is incorrect in getPartitionsByFilter RPC Key: HIVE-28291 URL: https://issues.apache.org/jira/browse/HIVE-28291 Project: Hive Issue Type: Bug Components: Metastore Reporter: dzcxzl -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-28291) Fix Partition spec is incorrect in getPartitionsByFilter RPC
[ https://issues.apache.org/jira/browse/HIVE-28291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-28291: -- Labels: pull-request-available (was: ) > Fix Partition spec is incorrect in getPartitionsByFilter RPC > > > Key: HIVE-28291 > URL: https://issues.apache.org/jira/browse/HIVE-28291 > Project: Hive > Issue Type: Bug > Components: Metastore >Reporter: dzcxzl >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-28282) Merging into iceberg table fails with copy on write when values clause has a function call
[ https://issues.apache.org/jira/browse/HIVE-28282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa resolved HIVE-28282. --- Fix Version/s: 4.1.0 Resolution: Fixed Merged to master. Thanks [~dkuzmenko] for the review! > Merging into iceberg table fails with copy on write when values clause has a > function call > -- > > Key: HIVE-28282 > URL: https://issues.apache.org/jira/browse/HIVE-28282 > Project: Hive > Issue Type: Bug > Components: Iceberg integration, Query Planning >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Labels: pull-request-available > Fix For: 4.1.0 > > > {code} > create external table target_ice(a int, b string, c int) stored by iceberg > tblproperties ('format-version'='2', 'write.merge.mode'='copy-on-write'); > create table source(a int, b string, c int); > explain > merge into target_ice as t using source src ON t.a = src.a > when matched and t.a > 100 THEN DELETE > when not matched then insert (a, b) values (src.a, concat(src.b, '-merge new > 2')); > {code} > {code} > org.apache.hadoop.hive.ql.parse.SemanticException: Encountered parse error > while parsing rewritten merge/update or delete query > at > org.apache.hadoop.hive.ql.parse.ParseUtils.parseRewrittenQuery(ParseUtils.java:721) > at > org.apache.hadoop.hive.ql.parse.rewrite.CopyOnWriteMergeRewriter.rewrite(CopyOnWriteMergeRewriter.java:84) > at > org.apache.hadoop.hive.ql.parse.rewrite.CopyOnWriteMergeRewriter.rewrite(CopyOnWriteMergeRewriter.java:48) > at > org.apache.hadoop.hive.ql.parse.RewriteSemanticAnalyzer.rewriteAndAnalyze(RewriteSemanticAnalyzer.java:93) > at > org.apache.hadoop.hive.ql.parse.MergeSemanticAnalyzer.analyze(MergeSemanticAnalyzer.java:201) > at > org.apache.hadoop.hive.ql.parse.RewriteSemanticAnalyzer.analyze(RewriteSemanticAnalyzer.java:84) > at > org.apache.hadoop.hive.ql.parse.RewriteSemanticAnalyzer.analyzeInternal(RewriteSemanticAnalyzer.java:72) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327) > at > org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:180) > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327) > at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:224) > at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:107) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:519) > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:471) > at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:436) > at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:430) > at > org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121) > at > org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:229) > at > org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:257) > at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:425) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:356) > at > org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:732) > at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:702) > at > org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:115) > at > org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:157) > at > org.apache.hadoop.hive.cli.TestIcebergLlapLocalCliDriver.testCliDriver(TestIcebergLlapLocalCliDriver.java:60) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) > at > org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:135) > at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:3
[jira] [Resolved] (HIVE-28287) Attempt make the scratch directory writable before failing
[ https://issues.apache.org/jira/browse/HIVE-28287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HIVE-28287. - Fix Version/s: 4.1.0 Resolution: Fixed > Attempt make the scratch directory writable before failing > -- > > Key: HIVE-28287 > URL: https://issues.apache.org/jira/browse/HIVE-28287 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Agnes Tevesz >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > Fix For: 4.1.0 > > > When hive is starting up it checks the tmp/hive directory privileges. Even if > the Azure managed identity have write access on an Azure storage account, the > rwx-wx-wx privileges still enforced, so in case rwxr-xr-x is set, the hive > process startup will fail. > See the logs of a startup that resulted in failure. > {code} > WARNING: An illegal reflective access operation has occurred > WARNING: Illegal reflective access by > org.apache.hadoop.hive.common.StringInternUtils > (file:/usr/lib/hive/lib/hive-common-3.1.3000.2023.0.16.0-142.jar) to field > java.net.URI.string > WARNING: Please consider reporting this to the maintainers of > org.apache.hadoop.hive.common.StringInternUtils > WARNING: Use --illegal-access=warn to enable warnings of further illegal > reflective access operations > WARNING: All illegal access operations will be denied in a future release > Exception in thread "main" java.lang.RuntimeException: Error applying > authorization policy on hive configuration: The dir: /tmp/hive on HDFS should > be writable. Current permissions are: rwxr-xr-x > at org.apache.hive.service.cli.CLIService.init(CLIService.java:121) > at > org.apache.hive.service.cli.thrift.EmbeddedThriftBinaryCLIService.init(EmbeddedThriftBinaryCLIService.java:63) > at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:357) > at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:287) > at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107) > at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677) > at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:228) > at > org.apache.hadoop.hive.metastore.tools.schematool.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:88) > at > org.apache.hadoop.hive.metastore.tools.schematool.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:103) > at > org.apache.hadoop.hive.metastore.CDHMetaStoreSchemaInfo.getMetaStoreSchemaVersion(CDHMetaStoreSchemaInfo.java:323) > at > org.apache.hadoop.hive.metastore.tools.schematool.SchemaToolTaskInitOrUpgrade.execute(SchemaToolTaskInitOrUpgrade.java:41) > at > org.apache.hadoop.hive.metastore.tools.schematool.MetastoreSchemaTool.run(MetastoreSchemaTool.java:482) > at > org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:143) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at org.apache.hadoop.util.RunJar.run(RunJar.java:318) > at org.apache.hadoop.util.RunJar.main(RunJar.java:232) > Caused by: java.lang.RuntimeException: The dir: /tmp/hive on HDFS should be > writable. Current permissions are: rwxr-xr-x > at > org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:5088) > at > org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:896) > at > org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:837) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:749) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:708) > at > org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:133) > at org.apache.hive.service.cli.CLIService.init(CLIService.java:118) > ... 18 more > Information schema initialization failed! > + '[' 1 -eq 0 ']' > + echo 'Information schema initialization failed!' > + exit 1 > {code} > To overcome this issue one approach could be to try to set the tmp/hive > folder privileges to rwx-wx-wx before failing the startup. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-28287) Attempt make the scratch directory writable before failing
[ https://issues.apache.org/jira/browse/HIVE-28287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17850794#comment-17850794 ] Ayush Saxena commented on HIVE-28287: - Committed to master. Thanx [~simhadri-g] for the review!!! > Attempt make the scratch directory writable before failing > -- > > Key: HIVE-28287 > URL: https://issues.apache.org/jira/browse/HIVE-28287 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Agnes Tevesz >Assignee: Ayush Saxena >Priority: Major > Labels: pull-request-available > > When hive is starting up it checks the tmp/hive directory privileges. Even if > the Azure managed identity have write access on an Azure storage account, the > rwx-wx-wx privileges still enforced, so in case rwxr-xr-x is set, the hive > process startup will fail. > See the logs of a startup that resulted in failure. > {code} > WARNING: An illegal reflective access operation has occurred > WARNING: Illegal reflective access by > org.apache.hadoop.hive.common.StringInternUtils > (file:/usr/lib/hive/lib/hive-common-3.1.3000.2023.0.16.0-142.jar) to field > java.net.URI.string > WARNING: Please consider reporting this to the maintainers of > org.apache.hadoop.hive.common.StringInternUtils > WARNING: Use --illegal-access=warn to enable warnings of further illegal > reflective access operations > WARNING: All illegal access operations will be denied in a future release > Exception in thread "main" java.lang.RuntimeException: Error applying > authorization policy on hive configuration: The dir: /tmp/hive on HDFS should > be writable. Current permissions are: rwxr-xr-x > at org.apache.hive.service.cli.CLIService.init(CLIService.java:121) > at > org.apache.hive.service.cli.thrift.EmbeddedThriftBinaryCLIService.init(EmbeddedThriftBinaryCLIService.java:63) > at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:357) > at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:287) > at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107) > at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677) > at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:228) > at > org.apache.hadoop.hive.metastore.tools.schematool.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:88) > at > org.apache.hadoop.hive.metastore.tools.schematool.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:103) > at > org.apache.hadoop.hive.metastore.CDHMetaStoreSchemaInfo.getMetaStoreSchemaVersion(CDHMetaStoreSchemaInfo.java:323) > at > org.apache.hadoop.hive.metastore.tools.schematool.SchemaToolTaskInitOrUpgrade.execute(SchemaToolTaskInitOrUpgrade.java:41) > at > org.apache.hadoop.hive.metastore.tools.schematool.MetastoreSchemaTool.run(MetastoreSchemaTool.java:482) > at > org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:143) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.base/java.lang.reflect.Method.invoke(Method.java:566) > at org.apache.hadoop.util.RunJar.run(RunJar.java:318) > at org.apache.hadoop.util.RunJar.main(RunJar.java:232) > Caused by: java.lang.RuntimeException: The dir: /tmp/hive on HDFS should be > writable. Current permissions are: rwxr-xr-x > at > org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:5088) > at > org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:896) > at > org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:837) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:749) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:708) > at > org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:133) > at org.apache.hive.service.cli.CLIService.init(CLIService.java:118) > ... 18 more > Information schema initialization failed! > + '[' 1 -eq 0 ']' > + echo 'Information schema initialization failed!' > + exit 1 > {code} > To overcome this issue one approach could be to try to set the tmp/hive > folder privileges to rwx-wx-wx before failing the startup. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-28292) Optimize SHOW TABLES|VIEWS statements
Wechar created HIVE-28292: - Summary: Optimize SHOW TABLES|VIEWS statements Key: HIVE-28292 URL: https://issues.apache.org/jira/browse/HIVE-28292 Project: Hive Issue Type: Improvement Components: Hive Reporter: Wechar Assignee: Wechar -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HIVE-28292) Optimize SHOW TABLES|VIEWS statements
[ https://issues.apache.org/jira/browse/HIVE-28292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-28292 started by Wechar. - > Optimize SHOW TABLES|VIEWS statements > - > > Key: HIVE-28292 > URL: https://issues.apache.org/jira/browse/HIVE-28292 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > > There are some improvement points in {{SHOW TABLES}} statement: > 1. pattern is not pushed to metastore server > 2. get table objects even if only table name and table type is needed. > 3. index {{UNIQUETABLE(TBL_NAME, DB_ID)}} in {{TBLS}} table is not efficient > enough compared to index {{(DB_ID, TBL_NAME)}} when querying tables by name > pattern. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-28292) Optimize SHOW TABLES|VIEWS statements
[ https://issues.apache.org/jira/browse/HIVE-28292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wechar updated HIVE-28292: -- Description: There are some improvement points in {{SHOW TABLES}} statement: 1. pattern is not pushed to metastore server 2. get table objects even if only table name and table type is needed. 3. index {{UNIQUETABLE(TBL_NAME, DB_ID)}} in {{TBLS}} table is not efficient enough compared to index {{(DB_ID, TBL_NAME)}} when querying tables by name pattern. > Optimize SHOW TABLES|VIEWS statements > - > > Key: HIVE-28292 > URL: https://issues.apache.org/jira/browse/HIVE-28292 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > > There are some improvement points in {{SHOW TABLES}} statement: > 1. pattern is not pushed to metastore server > 2. get table objects even if only table name and table type is needed. > 3. index {{UNIQUETABLE(TBL_NAME, DB_ID)}} in {{TBLS}} table is not efficient > enough compared to index {{(DB_ID, TBL_NAME)}} when querying tables by name > pattern. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-28292) Optimize SHOW TABLES|VIEWS statements
[ https://issues.apache.org/jira/browse/HIVE-28292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-28292: -- Labels: pull-request-available (was: ) > Optimize SHOW TABLES|VIEWS statements > - > > Key: HIVE-28292 > URL: https://issues.apache.org/jira/browse/HIVE-28292 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Wechar >Assignee: Wechar >Priority: Major > Labels: pull-request-available > > There are some improvement points in {{SHOW TABLES}} statement: > 1. pattern is not pushed to metastore server > 2. get table objects even if only table name and table type is needed. > 3. index {{UNIQUETABLE(TBL_NAME, DB_ID)}} in {{TBLS}} table is not efficient > enough compared to index {{(DB_ID, TBL_NAME)}} when querying tables by name > pattern. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-28290) TestMetastoreAuthorizationProvider to run on Tez
[ https://issues.apache.org/jira/browse/HIVE-28290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor reassigned HIVE-28290: --- Assignee: László Bodor > TestMetastoreAuthorizationProvider to run on Tez > > > Key: HIVE-28290 > URL: https://issues.apache.org/jira/browse/HIVE-28290 > Project: Hive > Issue Type: Sub-task >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > > when I was doing HIVE-27972, it passed locally, but failed in precommit, need > to investigate further > {code} > Error > The dir: > /home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir > on HDFS should be writable. Current permissions are: rwxr-xr-x > Stacktrace > java.lang.RuntimeException: The dir: > /home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir > on HDFS should be writable. Current permissions are: rwxr-xr-x > at > org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:4966) > at > org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:843) > at > org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:784) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:708) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:669) > at > org.apache.hadoop.hive.ql.security.TestMetastoreAuthorizationProvider.setUp(TestMetastoreAuthorizationProvider.java:126) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) > at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) > at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) > at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) > at org.junit.runners.ParentRunner.run(ParentRunner.java:413) > at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) > at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) > at > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377) > at > org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138) > at > org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465) > at > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451) > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-28288) TestHiveMetaTool to run on Tez
[ https://issues.apache.org/jira/browse/HIVE-28288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor reassigned HIVE-28288: --- Assignee: László Bodor > TestHiveMetaTool to run on Tez > -- > > Key: HIVE-28288 > URL: https://issues.apache.org/jira/browse/HIVE-28288 > Project: Hive > Issue Type: Sub-task >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > > when I was doing HIVE-27972, it passed locally, but failed in precommit, need > to investigate further > {code} > Error > java.lang.RuntimeException: The dir: > /home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir > on HDFS should be writable. Current permissions are: rwxr-xr-x > Stacktrace > java.lang.Exception: java.lang.RuntimeException: The dir: > /home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir > on HDFS should be writable. Current permissions are: rwxr-xr-x > at > org.apache.hadoop.hive.metastore.tools.metatool.TestHiveMetaTool.setUp(TestHiveMetaTool.java:137) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) > at > org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) > at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) > at > org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) > at > org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) > at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) > at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) > at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) > at org.junit.runners.ParentRunner.run(ParentRunner.java:413) > at > org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273) > at > org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238) > at > org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159) > at > org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377) > at > org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138) > at > org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465) > at > org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451) > Caused by: java.lang.RuntimeException: The dir: > /home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir > on HDFS should be writable. Current permissions are: rwxr-xr-x > at > org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:4966) > at > org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:843) > at > org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:784) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:708) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:669) > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:649) > at > org.apache.hadoop.hive.metastore.tools.metatool.TestHiveMetaTool.setUp(TestHiveMetaTool.java:130) > ... 30 more > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)