[jira] [Updated] (HIVE-28240) Compaction requests remain in Working state after TimeoutException
[ https://issues.apache.org/jira/browse/HIVE-28240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-28240: -- Labels: pull-request-available (was: ) > Compaction requests remain in Working state after TimeoutException > -- > > Key: HIVE-28240 > URL: https://issues.apache.org/jira/browse/HIVE-28240 > Project: Hive > Issue Type: Bug >Reporter: Dmitriy Fingerman >Assignee: Dmitriy Fingerman >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-28240) Compaction requests remain in Working state after TimeoutException
Dmitriy Fingerman created HIVE-28240: Summary: Compaction requests remain in Working state after TimeoutException Key: HIVE-28240 URL: https://issues.apache.org/jira/browse/HIVE-28240 Project: Hive Issue Type: Bug Reporter: Dmitriy Fingerman Assignee: Dmitriy Fingerman -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-28214) HPLSQL not using the hive variables passed through beeline using --hivevar option
[
https://issues.apache.org/jira/browse/HIVE-28214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-28214:
--
Labels: pull-request-available (was: )
> HPLSQL not using the hive variables passed through beeline using --hivevar
> option
> -
>
> Key: HIVE-28214
> URL: https://issues.apache.org/jira/browse/HIVE-28214
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Reporter: Dayakar M
>Assignee: Dayakar M
>Priority: Major
> Labels: pull-request-available
>
> HPLSQL not using the hive variables passed through beeline using --hivevar
> option.
> Steps to reproduce:
> {noformat}
> beeline -u
> 'jdbc:hive2://localhost:1/default;user=hive;password=hive;mode=hplsql'
> --hivevar hivedb=sys --hivevar hivetbl=tbls{noformat}
> {noformat}
> 0: jdbc:hive2://localhost> DECLARE hivedb_tbl string;
> . . . . . . . . . . . . . . . . . . . . . . .> SELECT hivedb || '.' ||
> hivetbl into hivedb_tbl;
> . . . . . . . . . . . . . . . . . . . . . . .> PRINT hivedb_tbl;
> . . . . . . . . . . . . . . . . . . . . . . .> /
> INFO : Compiling
> command(queryId=hive_20240424145826_617acb79-0b27-46eb-aa05-1332703c94fb):
> SELECT CONCAT(hivedb, '.', hivetbl)
> ERROR : FAILED: SemanticException [Error 10004]: Line 1:14 Invalid table
> alias or column reference 'hivedb': (possible column names are: )
> org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:14 Invalid table
> alias or column reference 'hivedb': (possible column names are: )
>
> INFO : Completed compiling
> command(queryId=hive_20240424145826_617acb79-0b27-46eb-aa05-1332703c94fb);
> Time taken: 3.976 seconds
> ERROR : Unhandled exception in HPL/SQL
> No rows affected (4.901 seconds)
> 0: jdbc:hive2://localhost>
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-28239) Fix bug on HMSHandler#checkLimitNumberOfPartitions
Wechar created HIVE-28239:
-
Summary: Fix bug on HMSHandler#checkLimitNumberOfPartitions
Key: HIVE-28239
URL: https://issues.apache.org/jira/browse/HIVE-28239
Project: Hive
Issue Type: Bug
Components: Hive
Reporter: Wechar
Assignee: Wechar
{{HMSHandler#checkLimitNumberOfPartitions}} should not compare request size,
which can cause the incorrect limit check.
Assume that HMS configure {{metastore.limit.partition.request}} as 100, the
client calls {{get_partitions_by_filter}} with maxParts as 101, and the
matching partition number is 50, in this case the HMS server should not throw
MetaException by partition limit check.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-28238) Open Hive ACID txn only for transactional resources
[ https://issues.apache.org/jira/browse/HIVE-28238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-28238: -- Labels: pull-request-available (was: ) > Open Hive ACID txn only for transactional resources > --- > > Key: HIVE-28238 > URL: https://issues.apache.org/jira/browse/HIVE-28238 > Project: Hive > Issue Type: Task >Reporter: Denys Kuzmenko >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-28238) Open Hive ACID txn only for transactional resources
Denys Kuzmenko created HIVE-28238: - Summary: Open Hive ACID txn only for transactional resources Key: HIVE-28238 URL: https://issues.apache.org/jira/browse/HIVE-28238 Project: Hive Issue Type: Task Reporter: Denys Kuzmenko -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27791) Eliminate totalSize check from test
[ https://issues.apache.org/jira/browse/HIVE-27791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor resolved HIVE-27791. - Resolution: Fixed > Eliminate totalSize check from test > --- > > Key: HIVE-27791 > URL: https://issues.apache.org/jira/browse/HIVE-27791 > Project: Hive > Issue Type: Improvement >Reporter: Zoltán Rátkai >Assignee: Zoltán Rátkai >Priority: Major > Labels: pull-request-available > Fix For: 4.1.0 > > > As discussed in this ticket, totalSize checks need to be eliminated from > tests: > https://github.com/apache/hive/pull/4690 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27791) Eliminate totalSize check from test
[ https://issues.apache.org/jira/browse/HIVE-27791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842329#comment-17842329 ] László Bodor commented on HIVE-27791: - merged to master, thanks [~zratkai] for taking care of this and [~ayushtkn], [~zhangbutao], [~sbadhya], [~aturoczy] for the reviews! > Eliminate totalSize check from test > --- > > Key: HIVE-27791 > URL: https://issues.apache.org/jira/browse/HIVE-27791 > Project: Hive > Issue Type: Improvement >Reporter: Zoltán Rátkai >Assignee: Zoltán Rátkai >Priority: Major > Labels: pull-request-available > Fix For: 4.1.0 > > > As discussed in this ticket, totalSize checks need to be eliminated from > tests: > https://github.com/apache/hive/pull/4690 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27791) Eliminate totalSize check from test
[ https://issues.apache.org/jira/browse/HIVE-27791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27791: Fix Version/s: 4.1.0 > Eliminate totalSize check from test > --- > > Key: HIVE-27791 > URL: https://issues.apache.org/jira/browse/HIVE-27791 > Project: Hive > Issue Type: Improvement >Reporter: Zoltán Rátkai >Assignee: Zoltán Rátkai >Priority: Major > Labels: pull-request-available > Fix For: 4.1.0 > > > As discussed in this ticket, totalSize checks need to be eliminated from > tests: > https://github.com/apache/hive/pull/4690 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-28237) Add a default tez-site.xml to data/conf
[
https://issues.apache.org/jira/browse/HIVE-28237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-28237:
--
Labels: pull-request-available (was: )
> Add a default tez-site.xml to data/conf
> ---
>
> Key: HIVE-28237
> URL: https://issues.apache.org/jira/browse/HIVE-28237
> Project: Hive
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
>
> currently, data/conf looks like:
> {code}
> ls -la data/conf
> total 88
> drwx-- 15 laszlobodor staff480 Apr 30 11:52 .
> drwx-- 7 laszlobodor staff224 Sep 20 2023 ..
> -rw-r--r-- 1 laszlobodor staff442 Aug 8 2023 fair-scheduler-test.xml
> -rw-r--r-- 1 laszlobodor staff 5862 Mar 20 11:53 hive-log4j2.properties
> -rw-r--r-- 1 laszlobodor staff 6706 Aug 8 2023 hive-site-old.xml
> -rw-r--r-- 1 laszlobodor staff 11934 Apr 30 11:43 hive-site.xml
> -rw-r--r-- 1 laszlobodor staff 1681 Sep 26 2023 hivemetastore-site.xml
> -rw-r--r-- 1 laszlobodor staff 1582 Aug 8 2023 hiveserver2-site.xml
> drwxr-xr-x 4 laszlobodor staff128 Mar 20 11:53 iceberg
> drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 impala
> drwx-- 5 laszlobodor staff160 Apr 29 20:08 llap
> drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 perf
> drwx-- 4 laszlobodor staff128 Mar 20 11:53 rlist
> drwx-- 4 laszlobodor staff128 Apr 29 20:08 tez
> {code}
> so there is no default tez-site.xml, making it confusing for users e.g. of
> the StartMiniHS2Cluster
> StartMiniHS2Cluster works from this folder, so it has a target (after a mvn
> clean install command) like:
> {code}
> ls -la itests/hive-unit/target/testconf
> total 88
> drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:52 .
> drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:53 ..
> -rw-r--r-- 1 laszlobodor staff442 Apr 30 11:52 fair-scheduler-test.xml
> -rw-r--r-- 1 laszlobodor staff 5862 Apr 30 11:52 hive-log4j2.properties
> -rw-r--r-- 1 laszlobodor staff 6706 Apr 30 11:52 hive-site-old.xml
> -rw-r--r-- 1 laszlobodor staff 11934 Apr 30 11:52 hive-site.xml
> -rw-r--r-- 1 laszlobodor staff 1681 Apr 30 11:52 hivemetastore-site.xml
> -rw-r--r-- 1 laszlobodor staff 1582 Apr 30 11:52 hiveserver2-site.xml
> drwxr-xr-x 4 laszlobodor staff128 Apr 30 11:52 iceberg
> drwxr-xr-x 3 laszlobodor staff 96 Apr 30 11:52 impala
> drwxr-xr-x 5 laszlobodor staff160 Apr 30 11:52 llap
> drwxr-xr-x 3 laszlobodor staff 96 Apr 30 11:52 perf
> drwxr-xr-x 4 laszlobodor staff128 Apr 30 11:52 rlist
> drwxr-xr-x 4 laszlobodor staff128 Apr 30 11:52 tez
> {code}
> for a dev, it's confusing as even the default TezConfiguration adds
> tez-site.xml as a default resource, it's not obvious where a tez site config
> is supposed to come from, ending up hacking tez options into hive-site.xml,
> which doesn't work (TezConfiguration object doesn't have hive-site.xml as a
> default resource)
> the log message of StartMiniHS2Cluster shows:
> {code}
> 2024-04-30T02:53:15,120 INFO [main] conf.HiveConf: Found configuration file
> file:/Users/laszlobodor/CDH/hive/itests/hive-unit/target/testconf/hive-site.xml
> {code}
> which is because we can inject a hive-site.xml directly for testing purposes:
> https://github.com/apache/hive/blob/a6069ee816804477c42c254fbec47228e3491982/itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/StartMiniHS2Cluster.java#L61
> but we cannot do the same for tez-site.xml, and it's okay, and here is where
> this jira is supposed to help by adding a tez-site.xml to data/conf that can
> be picked up in scenarios where data/conf is used as a resource folder
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-28237) Add a default tez-site.xml to data/conf
[
https://issues.apache.org/jira/browse/HIVE-28237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-28237:
Description:
currently, data/conf looks like:
{code}
ls -la data/conf
total 88
drwx-- 15 laszlobodor staff480 Apr 30 11:52 .
drwx-- 7 laszlobodor staff224 Sep 20 2023 ..
-rw-r--r-- 1 laszlobodor staff442 Aug 8 2023 fair-scheduler-test.xml
-rw-r--r-- 1 laszlobodor staff 5862 Mar 20 11:53 hive-log4j2.properties
-rw-r--r-- 1 laszlobodor staff 6706 Aug 8 2023 hive-site-old.xml
-rw-r--r-- 1 laszlobodor staff 11934 Apr 30 11:43 hive-site.xml
-rw-r--r-- 1 laszlobodor staff 1681 Sep 26 2023 hivemetastore-site.xml
-rw-r--r-- 1 laszlobodor staff 1582 Aug 8 2023 hiveserver2-site.xml
drwxr-xr-x 4 laszlobodor staff128 Mar 20 11:53 iceberg
drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 impala
drwx-- 5 laszlobodor staff160 Apr 29 20:08 llap
drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 perf
drwx-- 4 laszlobodor staff128 Mar 20 11:53 rlist
drwx-- 4 laszlobodor staff128 Apr 29 20:08 tez
{code}
so there is no default tez-site.xml, making it confusing for users e.g. of the
StartMiniHS2Cluster
StartMiniHS2Cluster works from this folder, so it has a target (after a mvn
clean install command) like:
{code}
ls -la itests/hive-unit/target/testconf
total 88
drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:52 .
drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:53 ..
-rw-r--r-- 1 laszlobodor staff442 Apr 30 11:52 fair-scheduler-test.xml
-rw-r--r-- 1 laszlobodor staff 5862 Apr 30 11:52 hive-log4j2.properties
-rw-r--r-- 1 laszlobodor staff 6706 Apr 30 11:52 hive-site-old.xml
-rw-r--r-- 1 laszlobodor staff 11934 Apr 30 11:52 hive-site.xml
-rw-r--r-- 1 laszlobodor staff 1681 Apr 30 11:52 hivemetastore-site.xml
-rw-r--r-- 1 laszlobodor staff 1582 Apr 30 11:52 hiveserver2-site.xml
drwxr-xr-x 4 laszlobodor staff128 Apr 30 11:52 iceberg
drwxr-xr-x 3 laszlobodor staff 96 Apr 30 11:52 impala
drwxr-xr-x 5 laszlobodor staff160 Apr 30 11:52 llap
drwxr-xr-x 3 laszlobodor staff 96 Apr 30 11:52 perf
drwxr-xr-x 4 laszlobodor staff128 Apr 30 11:52 rlist
drwxr-xr-x 4 laszlobodor staff128 Apr 30 11:52 tez
{code}
for a dev, it's confusing as even the default TezConfiguration adds
tez-site.xml as a default resource, it's not obvious where a tez site config is
supposed to come from, ending up hacking tez options into hive-site.xml, which
doesn't work (TezConfiguration object doesn't have hive-site.xml as a default
resource)
the log message of StartMiniHS2Cluster shows:
{code}
2024-04-30T02:53:15,120 INFO [main] conf.HiveConf: Found configuration file
file:/Users/laszlobodor/CDH/hive/itests/hive-unit/target/testconf/hive-site.xml
{code}
which is because we can inject a hive-site.xml directly for testing purposes:
https://github.com/apache/hive/blob/a6069ee816804477c42c254fbec47228e3491982/itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/StartMiniHS2Cluster.java#L61
but we cannot do the same for tez-site.xml, and it's okay, and here is where
this jira is supposed to help by adding a tez-site.xml to data/conf that can be
picked up in scenarios where data/conf is used as a resource folder
was:
currently, data/conf looks like:
{code}
ls -la data/conf
total 88
drwx-- 15 laszlobodor staff480 Apr 30 11:52 .
drwx-- 7 laszlobodor staff224 Sep 20 2023 ..
-rw-r--r-- 1 laszlobodor staff442 Aug 8 2023 fair-scheduler-test.xml
-rw-r--r-- 1 laszlobodor staff 5862 Mar 20 11:53 hive-log4j2.properties
-rw-r--r-- 1 laszlobodor staff 6706 Aug 8 2023 hive-site-old.xml
-rw-r--r-- 1 laszlobodor staff 11934 Apr 30 11:43 hive-site.xml
-rw-r--r-- 1 laszlobodor staff 1681 Sep 26 2023 hivemetastore-site.xml
-rw-r--r-- 1 laszlobodor staff 1582 Aug 8 2023 hiveserver2-site.xml
drwxr-xr-x 4 laszlobodor staff128 Mar 20 11:53 iceberg
drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 impala
drwx-- 5 laszlobodor staff160 Apr 29 20:08 llap
drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 perf
drwx-- 4 laszlobodor staff128 Mar 20 11:53 rlist
drwx-- 4 laszlobodor staff128 Apr 29 20:08 tez
{code}
so there is no default tez-site.xml, making it confusing for users e.g. of the
StartMiniHS2Cluster
StartMiniHS2Cluster works from this folder, so it has a target (after a mvn
clean install command) like:
{code}
ls -la itests/hive-unit/target/testconf
total 88
drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:52 .
drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:53 ..
-rw-r--r-- 1 laszlobodor staff442 Apr 30 11:52 fair-scheduler-test.xml
-rw-r--r-- 1 laszlobodor staff 5862 Apr 30 11:52 hive-log4j2.properties
-rw-r--r-- 1 laszlobodor staff 6706 Apr 30 11:52
[jira] [Updated] (HIVE-28237) Add a default tez-site.xml to data/conf
[
https://issues.apache.org/jira/browse/HIVE-28237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-28237:
Description:
currently, data/conf looks like:
{code}
ls -la data/conf
total 88
drwx-- 15 laszlobodor staff480 Apr 30 11:52 .
drwx-- 7 laszlobodor staff224 Sep 20 2023 ..
-rw-r--r-- 1 laszlobodor staff442 Aug 8 2023 fair-scheduler-test.xml
-rw-r--r-- 1 laszlobodor staff 5862 Mar 20 11:53 hive-log4j2.properties
-rw-r--r-- 1 laszlobodor staff 6706 Aug 8 2023 hive-site-old.xml
-rw-r--r-- 1 laszlobodor staff 11934 Apr 30 11:43 hive-site.xml
-rw-r--r-- 1 laszlobodor staff 1681 Sep 26 2023 hivemetastore-site.xml
-rw-r--r-- 1 laszlobodor staff 1582 Aug 8 2023 hiveserver2-site.xml
drwxr-xr-x 4 laszlobodor staff128 Mar 20 11:53 iceberg
drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 impala
drwx-- 5 laszlobodor staff160 Apr 29 20:08 llap
drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 perf
drwx-- 4 laszlobodor staff128 Mar 20 11:53 rlist
drwx-- 4 laszlobodor staff128 Apr 29 20:08 tez
{code}
so there is no default tez-site.xml, making it confusing for users e.g. of the
StartMiniHS2Cluster
StartMiniHS2Cluster works from this folder, so it has a target (after a mvn
clean install command) like:
{code}
ls -la itests/hive-unit/target/testconf
total 88
drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:52 .
drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:53 ..
-rw-r--r-- 1 laszlobodor staff442 Apr 30 11:52 fair-scheduler-test.xml
-rw-r--r-- 1 laszlobodor staff 5862 Apr 30 11:52 hive-log4j2.properties
-rw-r--r-- 1 laszlobodor staff 6706 Apr 30 11:52 hive-site-old.xml
-rw-r--r-- 1 laszlobodor staff 11934 Apr 30 11:52 hive-site.xml
-rw-r--r-- 1 laszlobodor staff 1681 Apr 30 11:52 hivemetastore-site.xml
-rw-r--r-- 1 laszlobodor staff 1582 Apr 30 11:52 hiveserver2-site.xml
drwxr-xr-x 4 laszlobodor staff128 Apr 30 11:52 iceberg
drwxr-xr-x 3 laszlobodor staff 96 Apr 30 11:52 impala
drwxr-xr-x 5 laszlobodor staff160 Apr 30 11:52 llap
drwxr-xr-x 3 laszlobodor staff 96 Apr 30 11:52 perf
drwxr-xr-x 4 laszlobodor staff128 Apr 30 11:52 rlist
drwxr-xr-x 4 laszlobodor staff128 Apr 30 11:52 tez
{code}
for a dev, it's confusing as even the default TezConfiguration adds
tez-site.xml as a default resource, it's not obvious where a tez site config is
supposed to come from, ending up hacking tez options into hive-site.xml, which
doesn't work (TezConfiguration object doesn't have hive-site.xml as a default
resource)
the log message of StartMiniHS2Cluster shows:
{code}
2024-04-30T02:53:15,120 INFO [main] conf.HiveConf: Found configuration file
file:/Users/laszlobodor/CDH/hive/itests/hive-unit/target/testconf/hive-site.xml
{code}
which is because we can inject a hive-site.xml directly for testing purposes:
https://github.com/apache/hive/blob/a6069ee816804477c42c254fbec47228e3491982/itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/StartMiniHS2Cluster.java#L61
but we cannot do the same for tez-site.xml, and it's okay
was:
currently, data/conf looks like:
{code}
ls -la data/conf
total 88
drwx-- 15 laszlobodor staff480 Apr 30 11:52 .
drwx-- 7 laszlobodor staff224 Sep 20 2023 ..
-rw-r--r-- 1 laszlobodor staff442 Aug 8 2023 fair-scheduler-test.xml
-rw-r--r-- 1 laszlobodor staff 5862 Mar 20 11:53 hive-log4j2.properties
-rw-r--r-- 1 laszlobodor staff 6706 Aug 8 2023 hive-site-old.xml
-rw-r--r-- 1 laszlobodor staff 11934 Apr 30 11:43 hive-site.xml
-rw-r--r-- 1 laszlobodor staff 1681 Sep 26 2023 hivemetastore-site.xml
-rw-r--r-- 1 laszlobodor staff 1582 Aug 8 2023 hiveserver2-site.xml
drwxr-xr-x 4 laszlobodor staff128 Mar 20 11:53 iceberg
drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 impala
drwx-- 5 laszlobodor staff160 Apr 29 20:08 llap
drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 perf
drwx-- 4 laszlobodor staff128 Mar 20 11:53 rlist
drwx-- 4 laszlobodor staff128 Apr 29 20:08 tez
{code}
so there is no default tez-site.xml, makes it confusing for users e.g. of the
StartMiniHS2Cluster
StartMiniHS2Cluster works from this folder, so it has a target (after a mvn
clean install command) like:
{code}
ls -la itests/hive-unit/target/testconf
total 88
drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:52 .
drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:53 ..
-rw-r--r-- 1 laszlobodor staff442 Apr 30 11:52 fair-scheduler-test.xml
-rw-r--r-- 1 laszlobodor staff 5862 Apr 30 11:52 hive-log4j2.properties
-rw-r--r-- 1 laszlobodor staff 6706 Apr 30 11:52 hive-site-old.xml
-rw-r--r-- 1 laszlobodor staff 11934 Apr 30 11:52 hive-site.xml
-rw-r--r-- 1 laszlobodor staff 1681 Apr 30 11:52 hivemetastore-site.xml
-rw-r--r-- 1
[jira] [Updated] (HIVE-28237) Add a default tez-site.xml to data/conf
[
https://issues.apache.org/jira/browse/HIVE-28237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-28237:
Description:
currently, data/conf looks like:
{code}
ls -la data/conf
total 88
drwx-- 15 laszlobodor staff480 Apr 30 11:52 .
drwx-- 7 laszlobodor staff224 Sep 20 2023 ..
-rw-r--r-- 1 laszlobodor staff442 Aug 8 2023 fair-scheduler-test.xml
-rw-r--r-- 1 laszlobodor staff 5862 Mar 20 11:53 hive-log4j2.properties
-rw-r--r-- 1 laszlobodor staff 6706 Aug 8 2023 hive-site-old.xml
-rw-r--r-- 1 laszlobodor staff 11934 Apr 30 11:43 hive-site.xml
-rw-r--r-- 1 laszlobodor staff 1681 Sep 26 2023 hivemetastore-site.xml
-rw-r--r-- 1 laszlobodor staff 1582 Aug 8 2023 hiveserver2-site.xml
drwxr-xr-x 4 laszlobodor staff128 Mar 20 11:53 iceberg
drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 impala
drwx-- 5 laszlobodor staff160 Apr 29 20:08 llap
drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 perf
drwx-- 4 laszlobodor staff128 Mar 20 11:53 rlist
drwx-- 4 laszlobodor staff128 Apr 29 20:08 tez
{code}
so there is no default tez-site.xml, makes it confusing for users e.g. of the
StartMiniHS2Cluster
StartMiniHS2Cluster works from this folder, so it has a target (after a mvn
clean install command) like:
{code}
ls -la itests/hive-unit/target/testconf
total 88
drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:52 .
drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:53 ..
-rw-r--r-- 1 laszlobodor staff442 Apr 30 11:52 fair-scheduler-test.xml
-rw-r--r-- 1 laszlobodor staff 5862 Apr 30 11:52 hive-log4j2.properties
-rw-r--r-- 1 laszlobodor staff 6706 Apr 30 11:52 hive-site-old.xml
-rw-r--r-- 1 laszlobodor staff 11934 Apr 30 11:52 hive-site.xml
-rw-r--r-- 1 laszlobodor staff 1681 Apr 30 11:52 hivemetastore-site.xml
-rw-r--r-- 1 laszlobodor staff 1582 Apr 30 11:52 hiveserver2-site.xml
drwxr-xr-x 4 laszlobodor staff128 Apr 30 11:52 iceberg
drwxr-xr-x 3 laszlobodor staff 96 Apr 30 11:52 impala
drwxr-xr-x 5 laszlobodor staff160 Apr 30 11:52 llap
drwxr-xr-x 3 laszlobodor staff 96 Apr 30 11:52 perf
drwxr-xr-x 4 laszlobodor staff128 Apr 30 11:52 rlist
drwxr-xr-x 4 laszlobodor staff128 Apr 30 11:52 tez
{code}
for a dev it's confusing as the default TezConfiguration adds tez-site.xml as a
default resource, but it's not obvious where it's coming from, moreover, adding
tez options to hive-site.xml simply doesn't work
the log message of StartMiniHS2Cluster shows:
{code}
2024-04-30T02:53:15,120 INFO [main] conf.HiveConf: Found configuration file
file:/Users/laszlobodor/CDH/hive/itests/hive-unit/target/testconf/hive-site.xml
{code}
which is because we can inject a hive-site.xml directly for testing purposes:
https://github.com/apache/hive/blob/a6069ee816804477c42c254fbec47228e3491982/itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/StartMiniHS2Cluster.java#L61
but we cannot do the same for tez-site.xml, and it's okay
was:
currently, data/conf looks like:
{code}
ls -la data/conf
total 88
drwx-- 15 laszlobodor staff480 Apr 30 11:52 .
drwx-- 7 laszlobodor staff224 Sep 20 2023 ..
-rw-r--r-- 1 laszlobodor staff442 Aug 8 2023 fair-scheduler-test.xml
-rw-r--r-- 1 laszlobodor staff 5862 Mar 20 11:53 hive-log4j2.properties
-rw-r--r-- 1 laszlobodor staff 6706 Aug 8 2023 hive-site-old.xml
-rw-r--r-- 1 laszlobodor staff 11934 Apr 30 11:43 hive-site.xml
-rw-r--r-- 1 laszlobodor staff 1681 Sep 26 2023 hivemetastore-site.xml
-rw-r--r-- 1 laszlobodor staff 1582 Aug 8 2023 hiveserver2-site.xml
drwxr-xr-x 4 laszlobodor staff128 Mar 20 11:53 iceberg
drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 impala
drwx-- 5 laszlobodor staff160 Apr 29 20:08 llap
drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 perf
drwx-- 4 laszlobodor staff128 Mar 20 11:53 rlist
drwx-- 4 laszlobodor staff128 Apr 29 20:08 tez
{code}
so there is not default tez-site.xml, makes it confusing for users e.g. of the
StartMiniHS2Cluster
StartMiniHS2Cluster works from this folder, so it has a target (after a mvn
clean install command) like:
{code}
ls -la itests/hive-unit/target/testconf
total 88
drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:52 .
drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:53 ..
-rw-r--r-- 1 laszlobodor staff442 Apr 30 11:52 fair-scheduler-test.xml
-rw-r--r-- 1 laszlobodor staff 5862 Apr 30 11:52 hive-log4j2.properties
-rw-r--r-- 1 laszlobodor staff 6706 Apr 30 11:52 hive-site-old.xml
-rw-r--r-- 1 laszlobodor staff 11934 Apr 30 11:52 hive-site.xml
-rw-r--r-- 1 laszlobodor staff 1681 Apr 30 11:52 hivemetastore-site.xml
-rw-r--r-- 1 laszlobodor staff 1582 Apr 30 11:52 hiveserver2-site.xml
drwxr-xr-x 4 laszlobodor staff128 Apr 30
[jira] [Updated] (HIVE-28237) Add a default tez-site.xml to data/conf
[
https://issues.apache.org/jira/browse/HIVE-28237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-28237:
Description:
currently, data/conf looks like:
{code}
ls -la data/conf
total 88
drwx-- 15 laszlobodor staff480 Apr 30 11:52 .
drwx-- 7 laszlobodor staff224 Sep 20 2023 ..
-rw-r--r-- 1 laszlobodor staff442 Aug 8 2023 fair-scheduler-test.xml
-rw-r--r-- 1 laszlobodor staff 5862 Mar 20 11:53 hive-log4j2.properties
-rw-r--r-- 1 laszlobodor staff 6706 Aug 8 2023 hive-site-old.xml
-rw-r--r-- 1 laszlobodor staff 11934 Apr 30 11:43 hive-site.xml
-rw-r--r-- 1 laszlobodor staff 1681 Sep 26 2023 hivemetastore-site.xml
-rw-r--r-- 1 laszlobodor staff 1582 Aug 8 2023 hiveserver2-site.xml
drwxr-xr-x 4 laszlobodor staff128 Mar 20 11:53 iceberg
drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 impala
drwx-- 5 laszlobodor staff160 Apr 29 20:08 llap
drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 perf
drwx-- 4 laszlobodor staff128 Mar 20 11:53 rlist
drwx-- 4 laszlobodor staff128 Apr 29 20:08 tez
{code}
so there is not default tez-site.xml, makes it confusing for users e.g. of the
StartMiniHS2Cluster
StartMiniHS2Cluster works from this folder, so it has a target (after a mvn
clean install command) like:
{code}
ls -la itests/hive-unit/target/testconf
total 88
drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:52 .
drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:53 ..
-rw-r--r-- 1 laszlobodor staff442 Apr 30 11:52 fair-scheduler-test.xml
-rw-r--r-- 1 laszlobodor staff 5862 Apr 30 11:52 hive-log4j2.properties
-rw-r--r-- 1 laszlobodor staff 6706 Apr 30 11:52 hive-site-old.xml
-rw-r--r-- 1 laszlobodor staff 11934 Apr 30 11:52 hive-site.xml
-rw-r--r-- 1 laszlobodor staff 1681 Apr 30 11:52 hivemetastore-site.xml
-rw-r--r-- 1 laszlobodor staff 1582 Apr 30 11:52 hiveserver2-site.xml
drwxr-xr-x 4 laszlobodor staff128 Apr 30 11:52 iceberg
drwxr-xr-x 3 laszlobodor staff 96 Apr 30 11:52 impala
drwxr-xr-x 5 laszlobodor staff160 Apr 30 11:52 llap
drwxr-xr-x 3 laszlobodor staff 96 Apr 30 11:52 perf
drwxr-xr-x 4 laszlobodor staff128 Apr 30 11:52 rlist
drwxr-xr-x 4 laszlobodor staff128 Apr 30 11:52 tez
{code}
for a dev it's confusing as the default TezConfiguration adds tez-site.xml as a
default resource, but it's not obvious where it's coming from, moreover, adding
tez options to hive-site.xml simply doesn't work
the log message of StartMiniHS2Cluster shows:
{code}
2024-04-30T02:53:15,120 INFO [main] conf.HiveConf: Found configuration file
file:/Users/laszlobodor/CDH/hive/itests/hive-unit/target/testconf/hive-site.xml
{code}
which is because we can inject a hive-site.xml directly for testing purposes:
https://github.com/apache/hive/blob/a6069ee816804477c42c254fbec47228e3491982/itests/hive-unit/src/test/java/org/apache/hive/jdbc/miniHS2/StartMiniHS2Cluster.java#L61
but we cannot do the same for tez-site.xml, and it's okay
> Add a default tez-site.xml to data/conf
> ---
>
> Key: HIVE-28237
> URL: https://issues.apache.org/jira/browse/HIVE-28237
> Project: Hive
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
>
> currently, data/conf looks like:
> {code}
> ls -la data/conf
> total 88
> drwx-- 15 laszlobodor staff480 Apr 30 11:52 .
> drwx-- 7 laszlobodor staff224 Sep 20 2023 ..
> -rw-r--r-- 1 laszlobodor staff442 Aug 8 2023 fair-scheduler-test.xml
> -rw-r--r-- 1 laszlobodor staff 5862 Mar 20 11:53 hive-log4j2.properties
> -rw-r--r-- 1 laszlobodor staff 6706 Aug 8 2023 hive-site-old.xml
> -rw-r--r-- 1 laszlobodor staff 11934 Apr 30 11:43 hive-site.xml
> -rw-r--r-- 1 laszlobodor staff 1681 Sep 26 2023 hivemetastore-site.xml
> -rw-r--r-- 1 laszlobodor staff 1582 Aug 8 2023 hiveserver2-site.xml
> drwxr-xr-x 4 laszlobodor staff128 Mar 20 11:53 iceberg
> drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 impala
> drwx-- 5 laszlobodor staff160 Apr 29 20:08 llap
> drwxr-xr-x 3 laszlobodor staff 96 Mar 20 11:53 perf
> drwx-- 4 laszlobodor staff128 Mar 20 11:53 rlist
> drwx-- 4 laszlobodor staff128 Apr 29 20:08 tez
> {code}
> so there is not default tez-site.xml, makes it confusing for users e.g. of
> the StartMiniHS2Cluster
> StartMiniHS2Cluster works from this folder, so it has a target (after a mvn
> clean install command) like:
> {code}
> ls -la itests/hive-unit/target/testconf
> total 88
> drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:52 .
> drwxr-xr-x 15 laszlobodor staff480 Apr 30 11:53 ..
> -rw-r--r-- 1 laszlobodor staff442 Apr 30 11:52 fair-scheduler-test.xml
>
[jira] [Assigned] (HIVE-28237) Add a default tez-site.xml to data/conf
[ https://issues.apache.org/jira/browse/HIVE-28237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor reassigned HIVE-28237: --- Assignee: László Bodor > Add a default tez-site.xml to data/conf > --- > > Key: HIVE-28237 > URL: https://issues.apache.org/jira/browse/HIVE-28237 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-28237) Add a default tez-site.xml to data/conf
László Bodor created HIVE-28237: --- Summary: Add a default tez-site.xml to data/conf Key: HIVE-28237 URL: https://issues.apache.org/jira/browse/HIVE-28237 Project: Hive Issue Type: Improvement Reporter: László Bodor -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-20016) Investigate TestJdbcWithMiniHS2.testParallelCompilation3 random failure
[ https://issues.apache.org/jira/browse/HIVE-20016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-20016: -- Labels: pull-request-available (was: ) > Investigate TestJdbcWithMiniHS2.testParallelCompilation3 random failure > --- > > Key: HIVE-20016 > URL: https://issues.apache.org/jira/browse/HIVE-20016 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Yongzhi Chen >Assignee: Yongzhi Chen >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20016.1.patch > > > org.apache.hive.jdbc.TestJdbcWithMiniHS2.testParallelCompilation3 failed with: > java.lang.AssertionError: Concurrent Statement failed: > org.apache.hive.service.cli.HiveSQLException: java.lang.AssertionError: > Authorization plugins not initialized! > at org.junit.Assert.fail(Assert.java:88) > at > org.apache.hive.jdbc.TestJdbcWithMiniHS2.finishTasks(TestJdbcWithMiniHS2.java:374) > at > org.apache.hive.jdbc.TestJdbcWithMiniHS2.testParallelCompilation3(TestJdbcWithMiniHS2.java:304) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-28167) Full table deletion fails when converting to truncate for Iceberg and ACID tables
[
https://issues.apache.org/jira/browse/HIVE-28167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842311#comment-17842311
]
Sourabh Badhya commented on HIVE-28167:
---
[~zabetak] , Issuing a truncate command is far more efficient than performing a
full-table delete operation which would just create more files. Hence we would
like this to be the default behaviour. The config is deprecated since it was
already released in 4.0.0. We would like to deprecate it first and then
completely remove it in the next release.
> Full table deletion fails when converting to truncate for Iceberg and ACID
> tables
> -
>
> Key: HIVE-28167
> URL: https://issues.apache.org/jira/browse/HIVE-28167
> Project: Hive
> Issue Type: Bug
>Reporter: Sourabh Badhya
>Assignee: Sourabh Badhya
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> A simple repro -
> {code:java}
> create table ice01 (id int, key int) stored by iceberg stored as orc
> tblproperties ('format-version'='2', 'write.delete.mode'='copy-on-write');
> insert into ice01 values (1,1),(2,1),(3,1),(4,1);
> insert into ice01 values (1,2),(2,2),(3,2),(4,2);
> insert into ice01 values (1,3),(2,3),(3,3),(4,3);
> insert into ice01 values (1,4),(2,4),(3,4),(4,4);
> insert into ice01 values (1,5),(2,5),(3,5),(4,5);
> explain analyze delete from ice01;
> delete from ice01;
> select count(*) from ice01;
> select * from ice01;
> describe formatted ice01; {code}
> The solution is to convert full table deletion to a truncate operation on the
> table.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-28167) Full table deletion fails when converting to truncate for Iceberg and ACID tables
[
https://issues.apache.org/jira/browse/HIVE-28167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842307#comment-17842307
]
Stamatis Zampetakis commented on HIVE-28167:
It seems that hive.optimize.delete.all property was deprecated as part of this
ticket but there are no explanation why. [~sbadhya] Can you please elaborate a
bit here on why this was done?
> Full table deletion fails when converting to truncate for Iceberg and ACID
> tables
> -
>
> Key: HIVE-28167
> URL: https://issues.apache.org/jira/browse/HIVE-28167
> Project: Hive
> Issue Type: Bug
>Reporter: Sourabh Badhya
>Assignee: Sourabh Badhya
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> A simple repro -
> {code:java}
> create table ice01 (id int, key int) stored by iceberg stored as orc
> tblproperties ('format-version'='2', 'write.delete.mode'='copy-on-write');
> insert into ice01 values (1,1),(2,1),(3,1),(4,1);
> insert into ice01 values (1,2),(2,2),(3,2),(4,2);
> insert into ice01 values (1,3),(2,3),(3,3),(4,3);
> insert into ice01 values (1,4),(2,4),(3,4),(4,4);
> insert into ice01 values (1,5),(2,5),(3,5),(4,5);
> explain analyze delete from ice01;
> delete from ice01;
> select count(*) from ice01;
> select * from ice01;
> describe formatted ice01; {code}
> The solution is to convert full table deletion to a truncate operation on the
> table.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-28236) Upgrade DropWizard to 4.x
Devaspati Krishnatri created HIVE-28236: --- Summary: Upgrade DropWizard to 4.x Key: HIVE-28236 URL: https://issues.apache.org/jira/browse/HIVE-28236 Project: Hive Issue Type: Task Reporter: Devaspati Krishnatri Assignee: Devaspati Krishnatri Upgrade DropWizard to 4.x to resolve CVEs -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-28227) Change the description of HIVE_OPTIMIZE_METADATA_DELETE config
[
https://issues.apache.org/jira/browse/HIVE-28227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sourabh Badhya resolved HIVE-28227.
---
Fix Version/s: 4.1.0
Resolution: Fixed
Merged to master.
Thanks [~ayushtkn] for the reviews.
> Change the description of HIVE_OPTIMIZE_METADATA_DELETE config
> --
>
> Key: HIVE-28227
> URL: https://issues.apache.org/jira/browse/HIVE-28227
> Project: Hive
> Issue Type: Task
>Reporter: Sourabh Badhya
>Assignee: Sourabh Badhya
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.1.0
>
>
> The current description is misleading. Change it to -
> {code:java}
> "Optimize delete using filters provided by the query. This uses the metadata
> of the table provided by table formats like Iceberg." {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-28235) Hive should honour heapsize set in hive-env
[ https://issues.apache.org/jira/browse/HIVE-28235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-28235: -- Labels: pull-request-available (was: ) > Hive should honour heapsize set in hive-env > --- > > Key: HIVE-28235 > URL: https://issues.apache.org/jira/browse/HIVE-28235 > Project: Hive > Issue Type: Bug >Reporter: Anmol Sundaram >Priority: Major > Labels: pull-request-available > > Hive processes do not honour the heap size set in hive-env. Right now, they > only consider the value set in hadoop-env and ignore those set in hive-env. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-28235) Hive should honour heapsize set in hive-env
Anmol Sundaram created HIVE-28235: - Summary: Hive should honour heapsize set in hive-env Key: HIVE-28235 URL: https://issues.apache.org/jira/browse/HIVE-28235 Project: Hive Issue Type: Bug Reporter: Anmol Sundaram Assignee: Anmol Sundaram Hive processes do not honour the heap size set in hive-env. Right now, they only consider the value set in hadoop-env and ignore those set in hive-env. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-28235) Hive should honour heapsize set in hive-env
[ https://issues.apache.org/jira/browse/HIVE-28235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anmol Sundaram reassigned HIVE-28235: - Assignee: (was: Anmol Sundaram) > Hive should honour heapsize set in hive-env > --- > > Key: HIVE-28235 > URL: https://issues.apache.org/jira/browse/HIVE-28235 > Project: Hive > Issue Type: Bug >Reporter: Anmol Sundaram >Priority: Major > > Hive processes do not honour the heap size set in hive-env. Right now, they > only consider the value set in hadoop-env and ignore those set in hive-env. -- This message was sent by Atlassian Jira (v8.20.10#820010)
