[jira] [Updated] (HIVE-28290) TestMetastoreAuthorizationProvider to run on Tez
[
https://issues.apache.org/jira/browse/HIVE-28290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-28290:
Description:
when I was doing HIVE-27972, it passed locally, but failed in precommit, need
to investigate further
{code}
Error
The dir:
/home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir
on HDFS should be writable. Current permissions are: rwxr-xr-x
Stacktrace
java.lang.RuntimeException: The dir:
/home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir
on HDFS should be writable. Current permissions are: rwxr-xr-x
at
org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:4966)
at
org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:843)
at
org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:784)
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:708)
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:669)
at
org.apache.hadoop.hive.ql.security.TestMetastoreAuthorizationProvider.setUp(TestMetastoreAuthorizationProvider.java:126)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
at
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
at
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
at
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377)
at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138)
at
org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465)
at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451)
{code}
> TestMetastoreAuthorizationProvider to run on Tez
>
>
> Key: HIVE-28290
> URL: https://issues.apache.org/jira/browse/HIVE-28290
> Project: Hive
> Issue Type: Sub-task
>Reporter: László Bodor
>Priority: Major
>
> when I was doing HIVE-27972, it passed locally, but failed in precommit, need
> to investigate further
> {code}
> Error
> The dir:
> /home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir
> on HDFS should be writable. Current permissions are: rwxr-xr-x
> Stacktrace
> java.lang.RuntimeException: The dir:
> /home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir
> on HDFS should be writable. Current permissions are: rwxr-xr-x
> at
> org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:4966)
> at
> org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:843)
> at
> org.apache.hadoop.hive.ql.session.SessionState.c
[jira] [Updated] (HIVE-26018) The result of UNIQUEJOIN on Hive on Tez is inconsistent with that of MR
[
https://issues.apache.org/jira/browse/HIVE-26018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-26018:
--
Labels: pull-request-available (was: )
> The result of UNIQUEJOIN on Hive on Tez is inconsistent with that of MR
> ---
>
> Key: HIVE-26018
> URL: https://issues.apache.org/jira/browse/HIVE-26018
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Affects Versions: 3.1.0, 4.0.0
>Reporter: GuangMing Lu
>Priority: Major
> Labels: pull-request-available
>
> The result of UNIQUEJOIN on Hive on Tez is inconsistent with that of MR, and
> the result Is not correct, for example:
> CREATE TABLE T1_n1x(key STRING, val STRING) STORED AS orc;
> CREATE TABLE T2_n1x(key STRING, val STRING) STORED AS orc;
> insert into T1_n1x values('aaa', '111'),('bbb', '222'),('ccc', '333');
> insert into T2_n1x values('aaa', '111'),('ddd', '444'),('ccc', '333');
> SELECT a.key, b.key FROM UNIQUEJOIN PRESERVE T1_n1x a (a.key), PRESERVE
> T2_n1x b (b.key);
> Hive on Tez result: wrong
> |a.key |b.key |
> |aaa |aaa |
> |bbb |NULL |
> |ccc |ccc |
> |NULL |ddd |
> +--+
> Hive on MR result: right
> |a.key |b.key |
> |aaa |aaa |
> |bbb |NULL |
> |ccc |ccc |
> +-+
> SELECT a.key, b.key FROM UNIQUEJOIN T1_n1x a (a.key), T2_n1x b (b.key);
> Hive on Tez result: wrong
> +---+
> |a.key |b.key |
> |aaa |aaa |
> |bbb |NULL |
> |ccc |ccc |
> |NULL |ddd |
> +-+
> Hive on MR result: right
> |a.key |b.key |
> |aaa |aaa |
> |ccc |ccc |
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-28284) TestJdbcWithMiniHS2/TestJdbcWithMiniHS2ErasureCoding to run on Tez
[ https://issues.apache.org/jira/browse/HIVE-28284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-28284: Summary: TestJdbcWithMiniHS2/TestJdbcWithMiniHS2ErasureCoding to run on Tez (was: TestJdbcWithMiniHS2 to run on Tez) > TestJdbcWithMiniHS2/TestJdbcWithMiniHS2ErasureCoding to run on Tez > -- > > Key: HIVE-28284 > URL: https://issues.apache.org/jira/browse/HIVE-28284 > Project: Hive > Issue Type: Sub-task >Reporter: László Bodor >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-28291) Fix Partition spec is incorrect in getPartitionsByFilter RPC
dzcxzl created HIVE-28291: - Summary: Fix Partition spec is incorrect in getPartitionsByFilter RPC Key: HIVE-28291 URL: https://issues.apache.org/jira/browse/HIVE-28291 Project: Hive Issue Type: Bug Components: Metastore Reporter: dzcxzl -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-28291) Fix Partition spec is incorrect in getPartitionsByFilter RPC
[ https://issues.apache.org/jira/browse/HIVE-28291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-28291: -- Labels: pull-request-available (was: ) > Fix Partition spec is incorrect in getPartitionsByFilter RPC > > > Key: HIVE-28291 > URL: https://issues.apache.org/jira/browse/HIVE-28291 > Project: Hive > Issue Type: Bug > Components: Metastore >Reporter: dzcxzl >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-28282) Merging into iceberg table fails with copy on write when values clause has a function call
[
https://issues.apache.org/jira/browse/HIVE-28282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa resolved HIVE-28282.
---
Fix Version/s: 4.1.0
Resolution: Fixed
Merged to master. Thanks [~dkuzmenko] for the review!
> Merging into iceberg table fails with copy on write when values clause has a
> function call
> --
>
> Key: HIVE-28282
> URL: https://issues.apache.org/jira/browse/HIVE-28282
> Project: Hive
> Issue Type: Bug
> Components: Iceberg integration, Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> {code}
> create external table target_ice(a int, b string, c int) stored by iceberg
> tblproperties ('format-version'='2', 'write.merge.mode'='copy-on-write');
> create table source(a int, b string, c int);
> explain
> merge into target_ice as t using source src ON t.a = src.a
> when matched and t.a > 100 THEN DELETE
> when not matched then insert (a, b) values (src.a, concat(src.b, '-merge new
> 2'));
> {code}
> {code}
> org.apache.hadoop.hive.ql.parse.SemanticException: Encountered parse error
> while parsing rewritten merge/update or delete query
> at
> org.apache.hadoop.hive.ql.parse.ParseUtils.parseRewrittenQuery(ParseUtils.java:721)
> at
> org.apache.hadoop.hive.ql.parse.rewrite.CopyOnWriteMergeRewriter.rewrite(CopyOnWriteMergeRewriter.java:84)
> at
> org.apache.hadoop.hive.ql.parse.rewrite.CopyOnWriteMergeRewriter.rewrite(CopyOnWriteMergeRewriter.java:48)
> at
> org.apache.hadoop.hive.ql.parse.RewriteSemanticAnalyzer.rewriteAndAnalyze(RewriteSemanticAnalyzer.java:93)
> at
> org.apache.hadoop.hive.ql.parse.MergeSemanticAnalyzer.analyze(MergeSemanticAnalyzer.java:201)
> at
> org.apache.hadoop.hive.ql.parse.RewriteSemanticAnalyzer.analyze(RewriteSemanticAnalyzer.java:84)
> at
> org.apache.hadoop.hive.ql.parse.RewriteSemanticAnalyzer.analyzeInternal(RewriteSemanticAnalyzer.java:72)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:180)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:224)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:107)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:519)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:471)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:436)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:430)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:121)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:229)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:257)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:201)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:127)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:425)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:356)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:732)
> at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:702)
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:115)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:157)
> at
> org.apache.hadoop.hive.cli.TestIcebergLlapLocalCliDriver.testCliDriver(TestIcebergLlapLocalCliDriver.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:135)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:3
[jira] [Resolved] (HIVE-28287) Attempt make the scratch directory writable before failing
[
https://issues.apache.org/jira/browse/HIVE-28287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena resolved HIVE-28287.
-
Fix Version/s: 4.1.0
Resolution: Fixed
> Attempt make the scratch directory writable before failing
> --
>
> Key: HIVE-28287
> URL: https://issues.apache.org/jira/browse/HIVE-28287
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Agnes Tevesz
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> When hive is starting up it checks the tmp/hive directory privileges. Even if
> the Azure managed identity have write access on an Azure storage account, the
> rwx-wx-wx privileges still enforced, so in case rwxr-xr-x is set, the hive
> process startup will fail.
> See the logs of a startup that resulted in failure.
> {code}
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by
> org.apache.hadoop.hive.common.StringInternUtils
> (file:/usr/lib/hive/lib/hive-common-3.1.3000.2023.0.16.0-142.jar) to field
> java.net.URI.string
> WARNING: Please consider reporting this to the maintainers of
> org.apache.hadoop.hive.common.StringInternUtils
> WARNING: Use --illegal-access=warn to enable warnings of further illegal
> reflective access operations
> WARNING: All illegal access operations will be denied in a future release
> Exception in thread "main" java.lang.RuntimeException: Error applying
> authorization policy on hive configuration: The dir: /tmp/hive on HDFS should
> be writable. Current permissions are: rwxr-xr-x
> at org.apache.hive.service.cli.CLIService.init(CLIService.java:121)
> at
> org.apache.hive.service.cli.thrift.EmbeddedThriftBinaryCLIService.init(EmbeddedThriftBinaryCLIService.java:63)
> at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:357)
> at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:287)
> at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
> at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
> at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:228)
> at
> org.apache.hadoop.hive.metastore.tools.schematool.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:88)
> at
> org.apache.hadoop.hive.metastore.tools.schematool.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:103)
> at
> org.apache.hadoop.hive.metastore.CDHMetaStoreSchemaInfo.getMetaStoreSchemaVersion(CDHMetaStoreSchemaInfo.java:323)
> at
> org.apache.hadoop.hive.metastore.tools.schematool.SchemaToolTaskInitOrUpgrade.execute(SchemaToolTaskInitOrUpgrade.java:41)
> at
> org.apache.hadoop.hive.metastore.tools.schematool.MetastoreSchemaTool.run(MetastoreSchemaTool.java:482)
> at
> org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:143)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> Caused by: java.lang.RuntimeException: The dir: /tmp/hive on HDFS should be
> writable. Current permissions are: rwxr-xr-x
> at
> org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:5088)
> at
> org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:896)
> at
> org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:837)
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:749)
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:708)
> at
> org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:133)
> at org.apache.hive.service.cli.CLIService.init(CLIService.java:118)
> ... 18 more
> Information schema initialization failed!
> + '[' 1 -eq 0 ']'
> + echo 'Information schema initialization failed!'
> + exit 1
> {code}
> To overcome this issue one approach could be to try to set the tmp/hive
> folder privileges to rwx-wx-wx before failing the startup.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-28287) Attempt make the scratch directory writable before failing
[
https://issues.apache.org/jira/browse/HIVE-28287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17850794#comment-17850794
]
Ayush Saxena commented on HIVE-28287:
-
Committed to master.
Thanx [~simhadri-g] for the review!!!
> Attempt make the scratch directory writable before failing
> --
>
> Key: HIVE-28287
> URL: https://issues.apache.org/jira/browse/HIVE-28287
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Agnes Tevesz
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
>
> When hive is starting up it checks the tmp/hive directory privileges. Even if
> the Azure managed identity have write access on an Azure storage account, the
> rwx-wx-wx privileges still enforced, so in case rwxr-xr-x is set, the hive
> process startup will fail.
> See the logs of a startup that resulted in failure.
> {code}
> WARNING: An illegal reflective access operation has occurred
> WARNING: Illegal reflective access by
> org.apache.hadoop.hive.common.StringInternUtils
> (file:/usr/lib/hive/lib/hive-common-3.1.3000.2023.0.16.0-142.jar) to field
> java.net.URI.string
> WARNING: Please consider reporting this to the maintainers of
> org.apache.hadoop.hive.common.StringInternUtils
> WARNING: Use --illegal-access=warn to enable warnings of further illegal
> reflective access operations
> WARNING: All illegal access operations will be denied in a future release
> Exception in thread "main" java.lang.RuntimeException: Error applying
> authorization policy on hive configuration: The dir: /tmp/hive on HDFS should
> be writable. Current permissions are: rwxr-xr-x
> at org.apache.hive.service.cli.CLIService.init(CLIService.java:121)
> at
> org.apache.hive.service.cli.thrift.EmbeddedThriftBinaryCLIService.init(EmbeddedThriftBinaryCLIService.java:63)
> at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:357)
> at org.apache.hive.jdbc.HiveConnection.(HiveConnection.java:287)
> at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107)
> at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
> at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:228)
> at
> org.apache.hadoop.hive.metastore.tools.schematool.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:88)
> at
> org.apache.hadoop.hive.metastore.tools.schematool.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:103)
> at
> org.apache.hadoop.hive.metastore.CDHMetaStoreSchemaInfo.getMetaStoreSchemaVersion(CDHMetaStoreSchemaInfo.java:323)
> at
> org.apache.hadoop.hive.metastore.tools.schematool.SchemaToolTaskInitOrUpgrade.execute(SchemaToolTaskInitOrUpgrade.java:41)
> at
> org.apache.hadoop.hive.metastore.tools.schematool.MetastoreSchemaTool.run(MetastoreSchemaTool.java:482)
> at
> org.apache.hive.beeline.schematool.HiveSchemaTool.main(HiveSchemaTool.java:143)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:232)
> Caused by: java.lang.RuntimeException: The dir: /tmp/hive on HDFS should be
> writable. Current permissions are: rwxr-xr-x
> at
> org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:5088)
> at
> org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:896)
> at
> org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:837)
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:749)
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:708)
> at
> org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:133)
> at org.apache.hive.service.cli.CLIService.init(CLIService.java:118)
> ... 18 more
> Information schema initialization failed!
> + '[' 1 -eq 0 ']'
> + echo 'Information schema initialization failed!'
> + exit 1
> {code}
> To overcome this issue one approach could be to try to set the tmp/hive
> folder privileges to rwx-wx-wx before failing the startup.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-28292) Optimize SHOW TABLES|VIEWS statements
Wechar created HIVE-28292: - Summary: Optimize SHOW TABLES|VIEWS statements Key: HIVE-28292 URL: https://issues.apache.org/jira/browse/HIVE-28292 Project: Hive Issue Type: Improvement Components: Hive Reporter: Wechar Assignee: Wechar -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HIVE-28292) Optimize SHOW TABLES|VIEWS statements
[
https://issues.apache.org/jira/browse/HIVE-28292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-28292 started by Wechar.
-
> Optimize SHOW TABLES|VIEWS statements
> -
>
> Key: HIVE-28292
> URL: https://issues.apache.org/jira/browse/HIVE-28292
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
>
> There are some improvement points in {{SHOW TABLES}} statement:
> 1. pattern is not pushed to metastore server
> 2. get table objects even if only table name and table type is needed.
> 3. index {{UNIQUETABLE(TBL_NAME, DB_ID)}} in {{TBLS}} table is not efficient
> enough compared to index {{(DB_ID, TBL_NAME)}} when querying tables by name
> pattern.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-28292) Optimize SHOW TABLES|VIEWS statements
[
https://issues.apache.org/jira/browse/HIVE-28292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wechar updated HIVE-28292:
--
Description:
There are some improvement points in {{SHOW TABLES}} statement:
1. pattern is not pushed to metastore server
2. get table objects even if only table name and table type is needed.
3. index {{UNIQUETABLE(TBL_NAME, DB_ID)}} in {{TBLS}} table is not efficient
enough compared to index {{(DB_ID, TBL_NAME)}} when querying tables by name
pattern.
> Optimize SHOW TABLES|VIEWS statements
> -
>
> Key: HIVE-28292
> URL: https://issues.apache.org/jira/browse/HIVE-28292
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
>
> There are some improvement points in {{SHOW TABLES}} statement:
> 1. pattern is not pushed to metastore server
> 2. get table objects even if only table name and table type is needed.
> 3. index {{UNIQUETABLE(TBL_NAME, DB_ID)}} in {{TBLS}} table is not efficient
> enough compared to index {{(DB_ID, TBL_NAME)}} when querying tables by name
> pattern.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-28292) Optimize SHOW TABLES|VIEWS statements
[
https://issues.apache.org/jira/browse/HIVE-28292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-28292:
--
Labels: pull-request-available (was: )
> Optimize SHOW TABLES|VIEWS statements
> -
>
> Key: HIVE-28292
> URL: https://issues.apache.org/jira/browse/HIVE-28292
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
>
> There are some improvement points in {{SHOW TABLES}} statement:
> 1. pattern is not pushed to metastore server
> 2. get table objects even if only table name and table type is needed.
> 3. index {{UNIQUETABLE(TBL_NAME, DB_ID)}} in {{TBLS}} table is not efficient
> enough compared to index {{(DB_ID, TBL_NAME)}} when querying tables by name
> pattern.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HIVE-28290) TestMetastoreAuthorizationProvider to run on Tez
[
https://issues.apache.org/jira/browse/HIVE-28290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor reassigned HIVE-28290:
---
Assignee: László Bodor
> TestMetastoreAuthorizationProvider to run on Tez
>
>
> Key: HIVE-28290
> URL: https://issues.apache.org/jira/browse/HIVE-28290
> Project: Hive
> Issue Type: Sub-task
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
>
> when I was doing HIVE-27972, it passed locally, but failed in precommit, need
> to investigate further
> {code}
> Error
> The dir:
> /home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir
> on HDFS should be writable. Current permissions are: rwxr-xr-x
> Stacktrace
> java.lang.RuntimeException: The dir:
> /home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir
> on HDFS should be writable. Current permissions are: rwxr-xr-x
> at
> org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:4966)
> at
> org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:843)
> at
> org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:784)
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:708)
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:669)
> at
> org.apache.hadoop.hive.ql.security.TestMetastoreAuthorizationProvider.setUp(TestMetastoreAuthorizationProvider.java:126)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138)
> at
> org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HIVE-28288) TestHiveMetaTool to run on Tez
[
https://issues.apache.org/jira/browse/HIVE-28288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor reassigned HIVE-28288:
---
Assignee: László Bodor
> TestHiveMetaTool to run on Tez
> --
>
> Key: HIVE-28288
> URL: https://issues.apache.org/jira/browse/HIVE-28288
> Project: Hive
> Issue Type: Sub-task
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
>
> when I was doing HIVE-27972, it passed locally, but failed in precommit, need
> to investigate further
> {code}
> Error
> java.lang.RuntimeException: The dir:
> /home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir
> on HDFS should be writable. Current permissions are: rwxr-xr-x
> Stacktrace
> java.lang.Exception: java.lang.RuntimeException: The dir:
> /home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir
> on HDFS should be writable. Current permissions are: rwxr-xr-x
> at
> org.apache.hadoop.hive.metastore.tools.metatool.TestHiveMetaTool.setUp(TestHiveMetaTool.java:137)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:377)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:138)
> at
> org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:465)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:451)
> Caused by: java.lang.RuntimeException: The dir:
> /home/jenkins/agent/workspace/hive-precommit_PR-4973/itests/hive-unit/target/tmp/scratchdir
> on HDFS should be writable. Current permissions are: rwxr-xr-x
> at
> org.apache.hadoop.hive.ql.exec.Utilities.ensurePathIsWritable(Utilities.java:4966)
> at
> org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:843)
> at
> org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:784)
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:708)
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:669)
> at
> org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:649)
> at
> org.apache.hadoop.hive.metastore.tools.metatool.TestHiveMetaTool.setUp(TestHiveMetaTool.java:130)
> ... 30 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
