[jira] [Resolved] (HIVE-27353) Show saved snapshot of materialized view source tables
[
https://issues.apache.org/jira/browse/HIVE-27353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa resolved HIVE-27353.
---
Resolution: Fixed
Merged to master. Thanks [~veghlaci05] for review.
> Show saved snapshot of materialized view source tables
> --
>
> Key: HIVE-27353
> URL: https://issues.apache.org/jira/browse/HIVE-27353
> Project: Hive
> Issue Type: Improvement
> Components: Materialized views
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Minor
> Labels: pull-request-available
>
> HIVE-25745 introduced a new section into
> {code:java}
> DESCRIBE FORMATTED ;
> {code}
> output:
> {code:java}
> # Materialized View Source table information
> Table nameI/U/D since last rebuild
> hive.default.src_txn 0/0/0
> hive.default.src_txn_20/0/0
> {code}
> Unfortunately transactional stats are not reliable because such stats are
> supposed to be saved along with basic stats.
> If something blocks saving the stats like
> {code:java}
> set hive.stats.autogather=false;
> {code}
> basic stats still can be refreshed using
> {code:java}
> analyze table compute statistics;
> {code}
> but it won't collect and update transactional since the amount of rows
> affected by recent transactions are no longer available and can not be
> calculated.
>
> The goal of this jira is to print the saved snapshot information of each
> source table instead:
> * writeId in case of native acid tables
> * snapshotId in case of iceberg tables
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HIVE-27398) SHOW CREATE TABLE doesn't output backticks for CLUSTERED by Col names
[
https://issues.apache.org/jira/browse/HIVE-27398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naveen Gangam resolved HIVE-27398.
--
Fix Version/s: 4.0.0
Resolution: Fixed
Fix has been merged to master. Thank you for the fix [~rtrivedi12]. Closing the
jira.
> SHOW CREATE TABLE doesn't output backticks for CLUSTERED by Col names
> -
>
> Key: HIVE-27398
> URL: https://issues.apache.org/jira/browse/HIVE-27398
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Minor
> Labels: pull-request-available
> Fix For: 4.0.0
>
>
> SHOW CREATE TABLE output uses backticks for all column names and partition
> column names but does not include backticks for CLUSTERED BY column names.
> This causes ParseException during table creation when any bucket column
> identifier matches reserved keywords
> {code:java}
> CREATE TABLE `test_ts_reserved_keyword7`(
> `member_id` varchar(8),
> `plan_nr` varchar(11),
> `timestamp` timestamp,
> `shared_ind` varchar(1),
> `user_id` varchar(8))
> CLUSTERED BY (
> member_nr,
> plan_nr,
> `timestamp`)
> INTO 4 BUCKETS;
> SHOW CREATE TABLE test_ts_reserved_keyword7;
> CREATE TABLE `test_ts_reserved_keyword7`(
> `member_id` varchar(8),
> `plan_nr` varchar(11),
> `timestamp` timestamp,
> `shared_ind` varchar(1),
> `user_id` varchar(8))
> CLUSTERED BY (
> member_id,
> plan_nr,
> timestamp)
> INTO 4 BUCKETS
> ROW FORMAT
> SERDE'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'STORED AS
> INPUTFORMAT'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'OUTPUTFORMAT'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat';
> This fails with "Error while compiling statement: FAILED: ParseException line
> 13:0 cannot recognize input near 'timestamp' ')' 'INTO' in column name"{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work started] (HIVE-27398) SHOW CREATE TABLE doesn't output backticks for CLUSTERED by Col names
[
https://issues.apache.org/jira/browse/HIVE-27398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-27398 started by Riju Trivedi.
---
> SHOW CREATE TABLE doesn't output backticks for CLUSTERED by Col names
> -
>
> Key: HIVE-27398
> URL: https://issues.apache.org/jira/browse/HIVE-27398
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Riju Trivedi
>Assignee: Riju Trivedi
>Priority: Minor
> Labels: pull-request-available
>
> SHOW CREATE TABLE output uses backticks for all column names and partition
> column names but does not include backticks for CLUSTERED BY column names.
> This causes ParseException during table creation when any bucket column
> identifier matches reserved keywords
> {code:java}
> CREATE TABLE `test_ts_reserved_keyword7`(
> `member_id` varchar(8),
> `plan_nr` varchar(11),
> `timestamp` timestamp,
> `shared_ind` varchar(1),
> `user_id` varchar(8))
> CLUSTERED BY (
> member_nr,
> plan_nr,
> `timestamp`)
> INTO 4 BUCKETS;
> SHOW CREATE TABLE test_ts_reserved_keyword7;
> CREATE TABLE `test_ts_reserved_keyword7`(
> `member_id` varchar(8),
> `plan_nr` varchar(11),
> `timestamp` timestamp,
> `shared_ind` varchar(1),
> `user_id` varchar(8))
> CLUSTERED BY (
> member_id,
> plan_nr,
> timestamp)
> INTO 4 BUCKETS
> ROW FORMAT
> SERDE'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'STORED AS
> INPUTFORMAT'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'OUTPUTFORMAT'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat';
> This fails with "Error while compiling statement: FAILED: ParseException line
> 13:0 cannot recognize input near 'timestamp' ')' 'INTO' in column name"{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27352) Support both LDAP and Kerberos Auth in HS2
[ https://issues.apache.org/jira/browse/HIVE-27352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng updated HIVE-27352: --- Fix Version/s: 4.0.0 > Support both LDAP and Kerberos Auth in HS2 > -- > > Key: HIVE-27352 > URL: https://issues.apache.org/jira/browse/HIVE-27352 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > > Currently, HS2 supports a single form of auth in binary mode, and some > limited multiple auth methods in http mode. Some analysis tools based on Hive > JDBC provide support for mixing both Kerberos and LDAP, however HS2 doesn't > make itself heard for this auth type, both in http and binary mode. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (HIVE-27352) Support both LDAP and Kerberos Auth in HS2
[ https://issues.apache.org/jira/browse/HIVE-27352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729681#comment-17729681 ] Zhihua Deng edited comment on HIVE-27352 at 6/6/23 10:16 AM: - Thanks for the review and merge, [~hemanth619]! For Http mode, we must configure the new auth method to the end of original hive.server2.authentication in order to keep compatible with the old HS2 client. was (Author: dengzh): Thanks for the review and merge, [~hemanth619]! For Http mode, we must configure the new auth method to the end of original hive.server2.authentication in order to keep compatible with the old HS2 client. > Support both LDAP and Kerberos Auth in HS2 > -- > > Key: HIVE-27352 > URL: https://issues.apache.org/jira/browse/HIVE-27352 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > > Currently, HS2 supports a single form of auth in binary mode, and some > limited multiple auth methods in http mode. Some analysis tools based on Hive > JDBC provide support for mixing both Kerberos and LDAP, however HS2 doesn't > make itself heard for this auth type, both in http and binary mode. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27352) Support both LDAP and Kerberos Auth in HS2
[ https://issues.apache.org/jira/browse/HIVE-27352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhihua Deng resolved HIVE-27352. Target Version/s: 4.0.0 Resolution: Fixed Thanks for the review and merge, [~hemanth619]! For Http mode, we must configure the new auth method to the end of original hive.server2.authentication in order to keep compatible with the old HS2 client. > Support both LDAP and Kerberos Auth in HS2 > -- > > Key: HIVE-27352 > URL: https://issues.apache.org/jira/browse/HIVE-27352 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Zhihua Deng >Assignee: Zhihua Deng >Priority: Major > Labels: pull-request-available > > Currently, HS2 supports a single form of auth in binary mode, and some > limited multiple auth methods in http mode. Some analysis tools based on Hive > JDBC provide support for mixing both Kerberos and LDAP, however HS2 doesn't > make itself heard for this auth type, both in http and binary mode. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27413) Bump jettison from 1.5.3 to 1.5.4
[ https://issues.apache.org/jira/browse/HIVE-27413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729639#comment-17729639 ] Akshat Mathur commented on HIVE-27413: -- :) > Bump jettison from 1.5.3 to 1.5.4 > - > > Key: HIVE-27413 > URL: https://issues.apache.org/jira/browse/HIVE-27413 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Priority: Major > Fix For: 4.0.0 > > > PR from *[dependabot|https://github.com/apps/dependabot]* > Creating for tracking -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27413) Bump jettison from 1.5.3 to 1.5.4
[ https://issues.apache.org/jira/browse/HIVE-27413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HIVE-27413. - Fix Version/s: 4.0.0 Resolution: Fixed > Bump jettison from 1.5.3 to 1.5.4 > - > > Key: HIVE-27413 > URL: https://issues.apache.org/jira/browse/HIVE-27413 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Priority: Major > Fix For: 4.0.0 > > > PR from *[dependabot|https://github.com/apps/dependabot]* > Creating for tracking -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27413) Bump jettison from 1.5.3 to 1.5.4
[ https://issues.apache.org/jira/browse/HIVE-27413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729638#comment-17729638 ] Ayush Saxena commented on HIVE-27413: - Committed to master. Thanx *[dependabot|https://github.com/apps/dependabot]* for the contribution *:)* and [~akshatm] for the review!!! > Bump jettison from 1.5.3 to 1.5.4 > - > > Key: HIVE-27413 > URL: https://issues.apache.org/jira/browse/HIVE-27413 > Project: Hive > Issue Type: Bug >Reporter: Ayush Saxena >Priority: Major > > PR from *[dependabot|https://github.com/apps/dependabot]* > Creating for tracking -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27413) Bump jettison from 1.5.3 to 1.5.4
Ayush Saxena created HIVE-27413: --- Summary: Bump jettison from 1.5.3 to 1.5.4 Key: HIVE-27413 URL: https://issues.apache.org/jira/browse/HIVE-27413 Project: Hive Issue Type: Bug Reporter: Ayush Saxena PR from *[dependabot|https://github.com/apps/dependabot]* Creating for tracking -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27401) Upgrade arrow to 12.0.0
[ https://issues.apache.org/jira/browse/HIVE-27401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HIVE-27401. - Fix Version/s: 4.0.0 Resolution: Fixed > Upgrade arrow to 12.0.0 > --- > > Key: HIVE-27401 > URL: https://issues.apache.org/jira/browse/HIVE-27401 > Project: Hive > Issue Type: Task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27401) Upgrade arrow to 12.0.0
[ https://issues.apache.org/jira/browse/HIVE-27401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729636#comment-17729636 ] Ayush Saxena commented on HIVE-27401: - Committed to master. Thanx [~amanraj2520] for the contribution, [~aturoczy], [~akshatm] & [~InvisibleProgrammer] for the reviews!!! > Upgrade arrow to 12.0.0 > --- > > Key: HIVE-27401 > URL: https://issues.apache.org/jira/browse/HIVE-27401 > Project: Hive > Issue Type: Task >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27393) Upgrade ant to 1.10.13 to fix CVE's
[ https://issues.apache.org/jira/browse/HIVE-27393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729634#comment-17729634 ] Ayush Saxena commented on HIVE-27393: - Committed to master. Thanx [~Aggarwal_Raghav] for the contribution & [~akshatm] for the review!!! > Upgrade ant to 1.10.13 to fix CVE's > --- > > Key: HIVE-27393 > URL: https://issues.apache.org/jira/browse/HIVE-27393 > Project: Hive > Issue Type: Improvement >Reporter: Raghav Aggarwal >Assignee: Raghav Aggarwal >Priority: Major > Labels: pull-request-available > > ant 1.10.12 contains 2 CVE: > # > [CVE-2022-23437|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23437] > # > [CVE-2020-14338|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-14338] > Upgrading to ant 1.10.13 will fix it -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27393) Upgrade ant to 1.10.13 to fix CVE's
[ https://issues.apache.org/jira/browse/HIVE-27393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HIVE-27393. - Fix Version/s: 4.0.0 Resolution: Fixed > Upgrade ant to 1.10.13 to fix CVE's > --- > > Key: HIVE-27393 > URL: https://issues.apache.org/jira/browse/HIVE-27393 > Project: Hive > Issue Type: Improvement >Reporter: Raghav Aggarwal >Assignee: Raghav Aggarwal >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > > ant 1.10.12 contains 2 CVE: > # > [CVE-2022-23437|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2022-23437] > # > [CVE-2020-14338|https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-14338] > Upgrading to ant 1.10.13 will fix it -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27386) Avoid duplicate audit log in cleanupHandlerContext
[
https://issues.apache.org/jira/browse/HIVE-27386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729633#comment-17729633
]
Ayush Saxena commented on HIVE-27386:
-
Committed to master.
Thanx [~wechar] for the contribution & others for the review!!!
> Avoid duplicate audit log in cleanupHandlerContext
> --
>
> Key: HIVE-27386
> URL: https://issues.apache.org/jira/browse/HIVE-27386
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Wechar
>Assignee: Wechar
>Priority: Major
> Labels: pull-request-available
> Attachments: screenshot-1.png
>
>
> {{HMSHandler#cleanupHandlerContext}} will be called twice if
> {{IMetaStoreClient#close}} was called. In this case, where will create two
> same audit log: Done cleaning up thread local RawStore.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-27408) Parquet file opened for reading stats is never closed
[ https://issues.apache.org/jira/browse/HIVE-27408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729632#comment-17729632 ] Sourabh Badhya commented on HIVE-27408: --- Thanks [~ayushtkn] , [~aturoczy] , [~akshatm] for the reviews. > Parquet file opened for reading stats is never closed > - > > Key: HIVE-27408 > URL: https://issues.apache.org/jira/browse/HIVE-27408 > Project: Hive > Issue Type: Bug >Reporter: Sourabh Badhya >Assignee: Sourabh Badhya >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > > ParquetRecordWriterWrapper while closing the writer tries to collect the > stats by creating a reader (opening the file). But it never closes the reader > (never closes the file). This can leave the file open hence consuming memory > and associated file handle resources. > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/ParquetRecordWriterWrapper.java#L143-L155] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27408) Parquet file opened for reading stats is never closed
[ https://issues.apache.org/jira/browse/HIVE-27408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17729631#comment-17729631 ] Ayush Saxena commented on HIVE-27408: - Committed to master. Thanx [~sbadhya] for the contribution, [~akshatm] and [~aturoczy] for the reviews!!! > Parquet file opened for reading stats is never closed > - > > Key: HIVE-27408 > URL: https://issues.apache.org/jira/browse/HIVE-27408 > Project: Hive > Issue Type: Bug >Reporter: Sourabh Badhya >Assignee: Sourabh Badhya >Priority: Major > Labels: pull-request-available > > ParquetRecordWriterWrapper while closing the writer tries to collect the > stats by creating a reader (opening the file). But it never closes the reader > (never closes the file). This can leave the file open hence consuming memory > and associated file handle resources. > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/ParquetRecordWriterWrapper.java#L143-L155] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27408) Parquet file opened for reading stats is never closed
[ https://issues.apache.org/jira/browse/HIVE-27408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena resolved HIVE-27408. - Fix Version/s: 4.0.0 Resolution: Fixed > Parquet file opened for reading stats is never closed > - > > Key: HIVE-27408 > URL: https://issues.apache.org/jira/browse/HIVE-27408 > Project: Hive > Issue Type: Bug >Reporter: Sourabh Badhya >Assignee: Sourabh Badhya >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > > ParquetRecordWriterWrapper while closing the writer tries to collect the > stats by creating a reader (opening the file). But it never closes the reader > (never closes the file). This can leave the file open hence consuming memory > and associated file handle resources. > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/write/ParquetRecordWriterWrapper.java#L143-L155] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27378) Backport of HIVE-19133: HS2 WebUI phase-wise performance metrics not showing correctly
[ https://issues.apache.org/jira/browse/HIVE-27378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan resolved HIVE-27378. - Fix Version/s: 3.2.0 Resolution: Fixed > Backport of HIVE-19133: HS2 WebUI phase-wise performance metrics not showing > correctly > -- > > Key: HIVE-27378 > URL: https://issues.apache.org/jira/browse/HIVE-27378 > Project: Hive > Issue Type: Sub-task >Reporter: Diksha >Assignee: Diksha >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > > Backport of HIVE-19133: HS2 WebUI phase-wise performance metrics not showing > correctly -- This message was sent by Atlassian Jira (v8.20.10#820010)
