[jira] [Created] (HIVE-26799) Make authorizations on custom UDFs involved in tables/view configurable.
Sai Hemanth Gantasala created HIVE-26799:
Summary: Make authorizations on custom UDFs involved in
tables/view configurable.
Key: HIVE-26799
URL: https://issues.apache.org/jira/browse/HIVE-26799
Project: Hive
Issue Type: New Feature
Components: HiveServer2, Security
Affects Versions: 4.0.0-alpha-2
Reporter: Sai Hemanth Gantasala
Assignee: Sai Hemanth Gantasala
When Hive is using Ranger/Sentry as an authorization service, consider the
following scenario.
{code:java}
> create table test_udf(st string); // privileged user operation
> create function Udf_UPPER as 'openkb.hive.udf.MyUpper' using jar
> 'hdfs:///tmp/MyUpperUDF-1.0.0.jar'; // privileged user operation
> create view v1_udf as select udf_upper(st) from test_udf; // privileged user
> operation
//unprivileged user test_user is given select permissions on view v1_udf
> select * from v1_udf; {code}
It is expected that test_user needs to have select privilege on v1_udf and
select permissions on udf_upper custom UDF in order to do a select query on
view.
This patch introduces a configuration
"hive.security.authorization.functions.in.view"=false which disables
authorization on views associated with views/tables during the select query. In
this mode, only UDFs explicitly stated in the query would still be authorized
as it is currently.
The reason for making these custom UDFs associated with view/tables
authorizable is that currently, test_user will need to be granted select
permissions on the custom udf. and the test_user can use this UDF and query
against any other table, which is a security concern.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-26798) Revert HIVE-26763
Zsolt Miskolczi created HIVE-26798: -- Summary: Revert HIVE-26763 Key: HIVE-26798 URL: https://issues.apache.org/jira/browse/HIVE-26798 Project: Hive Issue Type: Bug Components: Testing Infrastructure Reporter: Zsolt Miskolczi Reason: https://issues.apache.org/jira/browse/HIVE-26796 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-26797) Tests are excluded after Zookeeper and curator upgrade
Zsolt Miskolczi created HIVE-26797: -- Summary: Tests are excluded after Zookeeper and curator upgrade Key: HIVE-26797 URL: https://issues.apache.org/jira/browse/HIVE-26797 Project: Hive Issue Type: Bug Components: HiveServer2, Standalone Metastore, Tests Reporter: Zsolt Miskolczi Attachments: screenshot-1.png https://issues.apache.org/jira/browse/HIVE-26763 upgraded Zookeeper and curator. After that, we have noticed `StartMiniHS2Cluster` doesn't run at all. It doesn't throw an error. It just excluded. The whole hive-unit is affected as well so that I made a comparison between the Zookeeper change test run and the previous precommit test run before that: - The Zookeeper update ran 42430 tests: http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3787/7/tests/ - The previous change ran 46838: http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3784/3/tests/ It is about 4000 test cases as a difference. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-26796) All tests in hive-unit module are skipped silently
Stamatis Zampetakis created HIVE-26796:
--
Summary: All tests in hive-unit module are skipped silently
Key: HIVE-26796
URL: https://issues.apache.org/jira/browse/HIVE-26796
Project: Hive
Issue Type: Bug
Components: Testing Infrastructure
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis
In current master (7207a62def246b3290f1ece529e65b79012a3578) the tests in
hive-unit module are not running.
{noformat}
$ cd itests/hive-unit && mvn test
[INFO] --- maven-surefire-plugin:3.0.0-M4:test (default-test) @ hive-it-unit ---
[INFO]
[INFO] ---
[INFO] T E S T S
[INFO] ---
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] BUILD SUCCESS
[INFO]
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-26795) Iceberg integration: clean up temporary files in case of statement cancel
Krisztian Kasa created HIVE-26795: - Summary: Iceberg integration: clean up temporary files in case of statement cancel Key: HIVE-26795 URL: https://issues.apache.org/jira/browse/HIVE-26795 Project: Hive Issue Type: Bug Components: Iceberg integration Reporter: Krisztian Kasa Iceberg write operations are performed in the Tez task but the Iceberg commit of these writes are happening in the move task. To inform the MoveTask what writes has to be committed temp files are created with the path of the actual datafiles. Also in case of ctas statements the table is created by the ddl task is serialized into a temp file to be available for the Tez task which does the writes into the newly created table. Normally the cleanup of these temp files are happening in the move task but this task is not executed in case of cancel or an error in tez task. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-26794) Explore changing TxnHandler#connPoolMutex to NoPoolConnectionPool
Zhihua Deng created HIVE-26794: -- Summary: Explore changing TxnHandler#connPoolMutex to NoPoolConnectionPool Key: HIVE-26794 URL: https://issues.apache.org/jira/browse/HIVE-26794 Project: Hive Issue Type: Improvement Components: Standalone Metastore Reporter: Zhihua Deng Instead of creating a fixed size connection pool for TxnHandler#MutexAPI, the pool can be assigned to NoPoolConnectionPool due to: * TxnHandler#MutexAPI is primarily designed to provide coarse-grained mutex support to maintenance tasks running inside the Metastore, these tasks are not user faced; * A fixed size connection pool as same as the pool used in ObjectStore is a waste for other non leaders in the warehouse; The NoPoolConnectionPool provides connection on demand, and TxnHandler#MutexAPI only uses getConnection method to fetch a connection from the pool, so it's doable to change the pool to NoPoolConnectionPool, this would make the HMS more scaleable. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [Help] How to create a new table when automatically generating schema?
Hi, Alessandro Thanks for your response. I have implemented my feature by manually creating a schema. But I can't run the schematool because I want to use the new schema in the unit test that belongs to another project. In the unit test, I use hadoop-mini-clusters-hivemetastore to initialize my local hive metastore. There should be a script or code for initializing the derby database in hive metastore, and I miss it. Would anyone know about it? On Wed, 30 Nov 2022 at 18:03, Alessandro Solimando < alessandro.solima...@gmail.com> wrote: > Hi Jiajun, > how are you running the schematool? > > "schematool -verbose -dbType derby -initSchema" <-- are you maybe missing > the "-initSchema" bit? > > Can you provide the exact list of commands you are using? > > I am not very familiar with Hive 1.x, but maybe we can get it working. > > Best regards, > Alessandro > > On Wed, 30 Nov 2022 at 09:23, Jiajun Xie > wrote: > > > Hello~ > > I need to create a new table in the metastore. > > > > I tried to update `metastore/src/model/package.jdo` and > > `metastore/scripts/upgrade/derby/hive-schema-1.2.0.derby.sql`. (My > feature > > is based on branch-1.2) > > Then I set > > `datanucleus.schema.autoCreateTables`, > > `datanucleus.schema.generateDatabase.createScript`. > > None of them work. > > > > How to create a new table when automatically generating schema? Thank you > > very much. > > >
Re: [Help] How to create a new table when automatically generating schema?
Hi Jiajun, how are you running the schematool? "schematool -verbose -dbType derby -initSchema" <-- are you maybe missing the "-initSchema" bit? Can you provide the exact list of commands you are using? I am not very familiar with Hive 1.x, but maybe we can get it working. Best regards, Alessandro On Wed, 30 Nov 2022 at 09:23, Jiajun Xie wrote: > Hello~ > I need to create a new table in the metastore. > > I tried to update `metastore/src/model/package.jdo` and > `metastore/scripts/upgrade/derby/hive-schema-1.2.0.derby.sql`. (My feature > is based on branch-1.2) > Then I set > `datanucleus.schema.autoCreateTables`, > `datanucleus.schema.generateDatabase.createScript`. > None of them work. > > How to create a new table when automatically generating schema? Thank you > very much. >
[jira] [Created] (HIVE-26793) Create a new configuration to override "no compaction" for tables
Kokila N created HIVE-26793:
---
Summary: Create a new configuration to override "no compaction"
for tables
Key: HIVE-26793
URL: https://issues.apache.org/jira/browse/HIVE-26793
Project: Hive
Issue Type: Improvement
Reporter: Kokila N
Assignee: Kokila N
Currently a simple user can create a table with
{color:#6a8759}no_auto_compaction=true{color} table property and create an
aborted write transaction writing to this table. This way a malicious user can
prevent cleaning up data for the aborted transaction, creating performance
degradation.
This configuration should be allowed to overridden on a database level: adding
{color:#6a8759}no_auto_compaction=false{color} should override the table level
setting forcing the initiator to schedule compaction for all tables.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[Help] How to create a new table when automatically generating schema?
Hello~ I need to create a new table in the metastore. I tried to update `metastore/src/model/package.jdo` and `metastore/scripts/upgrade/derby/hive-schema-1.2.0.derby.sql`. (My feature is based on branch-1.2) Then I set `datanucleus.schema.autoCreateTables`, `datanucleus.schema.generateDatabase.createScript`. None of them work. How to create a new table when automatically generating schema? Thank you very much.
