[jira] [Work logged] (HIVE-26569) LlapTokenRenewer: TezAM (LlapTaskCommunicator) to renew LLAP_TOKENs
[ https://issues.apache.org/jira/browse/HIVE-26569?focusedWorklogId=830955=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-830955 ] ASF GitHub Bot logged work on HIVE-26569: - Author: ASF GitHub Bot Created on: 05/Dec/22 07:47 Start Date: 05/Dec/22 07:47 Worklog Time Spent: 10m Work Description: abstractdog opened a new pull request, #3626: URL: https://github.com/apache/hive/pull/3626 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? Issue Time Tracking --- Worklog Id: (was: 830955) Time Spent: 1h 10m (was: 1h) > LlapTokenRenewer: TezAM (LlapTaskCommunicator) to renew LLAP_TOKENs > --- > > Key: HIVE-26569 > URL: https://issues.apache.org/jira/browse/HIVE-26569 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Time Spent: 1h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26221) Add histogram-based column statistics
[
https://issues.apache.org/jira/browse/HIVE-26221?focusedWorklogId=830948=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-830948
]
ASF GitHub Bot logged work on HIVE-26221:
-
Author: ASF GitHub Bot
Created on: 05/Dec/22 07:24
Start Date: 05/Dec/22 07:24
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on code in PR #3137:
URL: https://github.com/apache/hive/pull/3137#discussion_r1039223978
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java:
##
@@ -1064,6 +1118,9 @@ public static void
setFieldsIntoOldStats(ColumnStatisticsObj oldStatObj,
if (newDecimalStatsData.isSetBitVectors()) {
oldDecimalStatsData.setBitVectors(newDecimalStatsData.getBitVectors());
}
+ if (newDecimalStatsData.isSetHistogram()) {
+oldDecimalStatsData.setHistogram(newDecimalStatsData.getHistogram());
+ }
Review Comment:
should also deal with date and timestamp type?
##
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/StatObjectConverter.java:
##
@@ -1064,6 +1118,9 @@ public static void
setFieldsIntoOldStats(ColumnStatisticsObj oldStatObj,
if (newDecimalStatsData.isSetBitVectors()) {
oldDecimalStatsData.setBitVectors(newDecimalStatsData.getBitVectors());
}
+ if (newDecimalStatsData.isSetHistogram()) {
+oldDecimalStatsData.setHistogram(newDecimalStatsData.getHistogram());
+ }
Review Comment:
Should also deal with date and timestamp type?
Issue Time Tracking
---
Worklog Id: (was: 830948)
Time Spent: 2.5h (was: 2h 20m)
> Add histogram-based column statistics
> -
>
> Key: HIVE-26221
> URL: https://issues.apache.org/jira/browse/HIVE-26221
> Project: Hive
> Issue Type: Improvement
> Components: CBO, Metastore, Statistics
>Affects Versions: 4.0.0-alpha-2
>Reporter: Alessandro Solimando
>Assignee: Alessandro Solimando
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> Hive does not support histogram statistics, which are particularly useful for
> skewed data (which is very common in practice) and range predicates.
> Hive's current selectivity estimation for range predicates is based on a
> hard-coded value of 1/3 (see
> [FilterSelectivityEstimator.java#L138-L144|https://github.com/apache/hive/blob/56c336268ea8c281d23c22d89271af37cb7e2572/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/FilterSelectivityEstimator.java#L138-L144]).])
> The current proposal aims at integrating histogram as an additional column
> statistics, stored into the Hive metastore at the table (or partition) level.
> The main requirements for histogram integration are the following:
> * efficiency: the approach must scale and support billions of rows
> * merge-ability: partition-level histograms have to be merged to form
> table-level histograms
> * explicit and configurable trade-off between memory footprint and accuracy
> Hive already integrates [KLL data
> sketches|https://datasketches.apache.org/docs/KLL/KLLSketch.html] UDAF.
> Datasketches are small, stateful programs that process massive data-streams
> and can provide approximate answers, with mathematical guarantees, to
> computationally difficult queries orders-of-magnitude faster than
> traditional, exact methods.
> We propose to use KLL, and more specifically the cumulative distribution
> function (CDF), as the underlying data structure for our histogram statistics.
> The current proposal targets numeric data types (float, integer and numeric
> families) and temporal data types (date and timestamp).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-26770) Make "end of loop" compaction logs appear more selectively
[
https://issues.apache.org/jira/browse/HIVE-26770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17643124#comment-17643124
]
Akshat Mathur commented on HIVE-26770:
--
Due to timeout the test are failing, blocking merge
> Make "end of loop" compaction logs appear more selectively
> --
>
> Key: HIVE-26770
> URL: https://issues.apache.org/jira/browse/HIVE-26770
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0-alpha-1
>Reporter: Akshat Mathur
>Assignee: Akshat Mathur
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> Currently Initiator, Worker, and Cleaner threads log something like "finished
> one loop" on INFO level.
> This is useful to figure out if one of these threads is taking too long to
> finish a loop, but expensive in general.
>
> Suggested Time: 20mins
> Logging this should be changed in the following way
> # If loop finished within a predefined amount of time, level should be DEBUG
> and message should look like: *Initiator loop took \{ellapsedTime} seconds to
> finish.*
> # If loop ran longer than this predefined amount, level should be WARN and
> message should look like: *Possible Initiator slowdown, loop took
> \{ellapsedTime} seconds to finish.*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-26770) Make "end of loop" compaction logs appear more selectively
[
https://issues.apache.org/jira/browse/HIVE-26770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akshat Mathur updated HIVE-26770:
-
Affects Version/s: 4.0.0-alpha-1
Status: Patch Available (was: In Progress)
> Make "end of loop" compaction logs appear more selectively
> --
>
> Key: HIVE-26770
> URL: https://issues.apache.org/jira/browse/HIVE-26770
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0-alpha-1
>Reporter: Akshat Mathur
>Assignee: Akshat Mathur
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> Currently Initiator, Worker, and Cleaner threads log something like "finished
> one loop" on INFO level.
> This is useful to figure out if one of these threads is taking too long to
> finish a loop, but expensive in general.
>
> Suggested Time: 20mins
> Logging this should be changed in the following way
> # If loop finished within a predefined amount of time, level should be DEBUG
> and message should look like: *Initiator loop took \{ellapsedTime} seconds to
> finish.*
> # If loop ran longer than this predefined amount, level should be WARN and
> message should look like: *Possible Initiator slowdown, loop took
> \{ellapsedTime} seconds to finish.*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work started] (HIVE-26770) Make "end of loop" compaction logs appear more selectively
[
https://issues.apache.org/jira/browse/HIVE-26770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-26770 started by Akshat Mathur.
> Make "end of loop" compaction logs appear more selectively
> --
>
> Key: HIVE-26770
> URL: https://issues.apache.org/jira/browse/HIVE-26770
> Project: Hive
> Issue Type: Improvement
>Reporter: Akshat Mathur
>Assignee: Akshat Mathur
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> Currently Initiator, Worker, and Cleaner threads log something like "finished
> one loop" on INFO level.
> This is useful to figure out if one of these threads is taking too long to
> finish a loop, but expensive in general.
>
> Suggested Time: 20mins
> Logging this should be changed in the following way
> # If loop finished within a predefined amount of time, level should be DEBUG
> and message should look like: *Initiator loop took \{ellapsedTime} seconds to
> finish.*
> # If loop ran longer than this predefined amount, level should be WARN and
> message should look like: *Possible Initiator slowdown, loop took
> \{ellapsedTime} seconds to finish.*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26787) Pushdown Timestamp data type to metastore via direct sql / JDO
[ https://issues.apache.org/jira/browse/HIVE-26787?focusedWorklogId=830934=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-830934 ] ASF GitHub Bot logged work on HIVE-26787: - Author: ASF GitHub Bot Created on: 05/Dec/22 03:53 Start Date: 05/Dec/22 03:53 Worklog Time Spent: 10m Work Description: shameersss1 commented on PR #3811: URL: https://github.com/apache/hive/pull/3811#issuecomment-1336709749 @kasakrisz Could you please review this? Issue Time Tracking --- Worklog Id: (was: 830934) Time Spent: 1h (was: 50m) > Pushdown Timestamp data type to metastore via direct sql / JDO > -- > > Key: HIVE-26787 > URL: https://issues.apache.org/jira/browse/HIVE-26787 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Syed Shameerur Rahman >Assignee: Syed Shameerur Rahman >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > make timestamp data type push down to hive metastore during partition > pruning. This is in similar lines with the jira: > https://issues.apache.org/jira/browse/HIVE-26778 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26683) Sum over window produces 0 when row contains null
[
https://issues.apache.org/jira/browse/HIVE-26683?focusedWorklogId=830927=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-830927
]
ASF GitHub Bot logged work on HIVE-26683:
-
Author: ASF GitHub Bot
Created on: 05/Dec/22 03:07
Start Date: 05/Dec/22 03:07
Worklog Time Spent: 10m
Work Description: ramesh0201 commented on PR #3800:
URL: https://github.com/apache/hive/pull/3800#issuecomment-1336672534
Thanks @scarlin-cloudera. +1. LGTM
Issue Time Tracking
---
Worklog Id: (was: 830927)
Time Spent: 2h (was: 1h 50m)
> Sum over window produces 0 when row contains null
> -
>
> Key: HIVE-26683
> URL: https://issues.apache.org/jira/browse/HIVE-26683
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Steve Carlin
>Assignee: Steve Carlin
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> Ran the following sql:
>
> {code:java}
> create table sum_window_test_small (id int, tinyint_col tinyint);
> insert into sum_window_test_small values (5,5), (10, NULL), (11,1);
> select id,
> tinyint_col,
> sum(tinyint_col) over (order by id nulls last rows between 1 following and 1
> following)
> from sum_window_test_small order by id;
> select id,
> tinyint_col,
> sum(tinyint_col) over (order by id nulls last rows between current row and 1
> following)
> from sum_window_test_small order by id;
> {code}
> The result is
> {code:java}
> +-+--+---+
> | id | tinyint_col | sum_window_0 |
> +-+--+---+
> | 5 | 5 | 0 |
> | 10 | NULL | 1 |
> | 11 | 1 | NULL |
> +-+--+---+{code}
> The first row should have the sum as NULL
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26794) Explore changing TxnHandler#connPoolMutex to NoPoolConnectionPool
[ https://issues.apache.org/jira/browse/HIVE-26794?focusedWorklogId=830920=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-830920 ] ASF GitHub Bot logged work on HIVE-26794: - Author: ASF GitHub Bot Created on: 05/Dec/22 02:31 Start Date: 05/Dec/22 02:31 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3817: URL: https://github.com/apache/hive/pull/3817#issuecomment-1336649363 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3817) [](https://sonarcloud.io/project/issues?id=apache_hive=3817=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3817=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3817=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3817=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3817=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3817=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3817=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3817=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3817=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3817=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3817=false=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3817=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3817=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3817=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 830920) Time Spent: 1h 40m (was: 1.5h) > Explore changing TxnHandler#connPoolMutex to NoPoolConnectionPool > - > > Key: HIVE-26794 > URL: https://issues.apache.org/jira/browse/HIVE-26794 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Zhihua Deng >Priority: Major > Labels: pull-request-available > Time Spent: 1h 40m > Remaining Estimate: 0h > > Instead of creating a fixed size connection pool for TxnHandler#MutexAPI, the > pool can be assigned to NoPoolConnectionPool due to: > * TxnHandler#MutexAPI is primarily designed to provide coarse-grained mutex > support to maintenance tasks running inside the Metastore, these tasks are > not user faced; > * A fixed size connection pool as same as the pool used in ObjectStore is a > waste for other non leaders in the warehouse; > The NoPoolConnectionPool provides connection on demand, and > TxnHandler#MutexAPI only uses getConnection method to fetch a connection from > the pool, so it's doable to change the pool to NoPoolConnectionPool, this > would make the HMS more scaleable. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26574) hive.merge.tezfiles does not work with LLAP hive.llap.execution.mode = all
[ https://issues.apache.org/jira/browse/HIVE-26574?focusedWorklogId=830911=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-830911 ] ASF GitHub Bot logged work on HIVE-26574: - Author: ASF GitHub Bot Created on: 05/Dec/22 00:20 Start Date: 05/Dec/22 00:20 Worklog Time Spent: 10m Work Description: github-actions[bot] closed pull request #3631: [WIP] HIVE-26574: Make TaskGraphWalker walk ConditionalTask children URL: https://github.com/apache/hive/pull/3631 Issue Time Tracking --- Worklog Id: (was: 830911) Time Spent: 1h (was: 50m) > hive.merge.tezfiles does not work with LLAP hive.llap.execution.mode = all > -- > > Key: HIVE-26574 > URL: https://issues.apache.org/jira/browse/HIVE-26574 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 4.0.0-alpha-2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > set hive.execution.mode=llap; > set hive.llap.execution.mode=all; > set hive.merge.tezfiles=true; > with the above settings the Merge work generated does not get converted to > LLAP work and thus fails on platform that require LLAP. > This will either end up causing the query compilation to fail OR a > NullPointerException at execution time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26569) LlapTokenRenewer: TezAM (LlapTaskCommunicator) to renew LLAP_TOKENs
[ https://issues.apache.org/jira/browse/HIVE-26569?focusedWorklogId=830912=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-830912 ] ASF GitHub Bot logged work on HIVE-26569: - Author: ASF GitHub Bot Created on: 05/Dec/22 00:20 Start Date: 05/Dec/22 00:20 Worklog Time Spent: 10m Work Description: github-actions[bot] closed pull request #3626: HIVE-26569: LlapTokenRenewer: TezAM (LlapTaskCommunicator) to renew LLAP_TOKENs URL: https://github.com/apache/hive/pull/3626 Issue Time Tracking --- Worklog Id: (was: 830912) Time Spent: 1h (was: 50m) > LlapTokenRenewer: TezAM (LlapTaskCommunicator) to renew LLAP_TOKENs > --- > > Key: HIVE-26569 > URL: https://issues.apache.org/jira/browse/HIVE-26569 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26568) Upgrade Log4j2 to 2.18.0 due to CVEs
[ https://issues.apache.org/jira/browse/HIVE-26568?focusedWorklogId=830913=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-830913 ] ASF GitHub Bot logged work on HIVE-26568: - Author: ASF GitHub Bot Created on: 05/Dec/22 00:20 Start Date: 05/Dec/22 00:20 Worklog Time Spent: 10m Work Description: github-actions[bot] closed pull request #3623: HIVE-26568: Upgrade log4j to 2.18.0 due to CVEs (Naveen Gangam) URL: https://github.com/apache/hive/pull/3623 Issue Time Tracking --- Worklog Id: (was: 830913) Time Spent: 40m (was: 0.5h) > Upgrade Log4j2 to 2.18.0 due to CVEs > > > Key: HIVE-26568 > URL: https://issues.apache.org/jira/browse/HIVE-26568 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.2 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0-alpha-2 > > Time Spent: 40m > Remaining Estimate: 0h > > High security vulnerability in Log4J - CVE-2021-44832 bundled with Hive -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26770) Make "end of loop" compaction logs appear more selectively
[
https://issues.apache.org/jira/browse/HIVE-26770?focusedWorklogId=830906=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-830906
]
ASF GitHub Bot logged work on HIVE-26770:
-
Author: ASF GitHub Bot
Created on: 04/Dec/22 20:18
Start Date: 04/Dec/22 20:18
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #3803:
URL: https://github.com/apache/hive/pull/3803#issuecomment-1336507862
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3803)
[](https://sonarcloud.io/project/issues?id=apache_hive=3803=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=3803=false=BUG)
[1
Bug](https://sonarcloud.io/project/issues?id=apache_hive=3803=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=3803=false=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive=3803=false=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3803=false=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3803=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3803=false=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3803=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive=3803=false=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive=3803=false=CODE_SMELL)
[10 Code
Smells](https://sonarcloud.io/project/issues?id=apache_hive=3803=false=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive=3803=coverage=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive=3803=duplicated_lines_density=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 830906)
Time Spent: 4h 40m (was: 4.5h)
> Make "end of loop" compaction logs appear more selectively
> --
>
> Key: HIVE-26770
> URL: https://issues.apache.org/jira/browse/HIVE-26770
> Project: Hive
> Issue Type: Improvement
>Reporter: Akshat Mathur
>Assignee: Akshat Mathur
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> Currently Initiator, Worker, and Cleaner threads log something like "finished
> one loop" on INFO level.
> This is useful to figure out if one of these threads is taking too long to
> finish a loop, but expensive in general.
>
> Suggested Time: 20mins
> Logging this should be changed in the following way
> # If loop finished within a predefined amount of time, level should be DEBUG
> and message should look like: *Initiator loop took \{ellapsedTime} seconds to
> finish.*
> # If loop ran longer than this predefined amount, level should be WARN and
> message should look like: *Possible Initiator slowdown, loop took
> \{ellapsedTime} seconds to finish.*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-26758) Allow use scratchdir for staging final job
[ https://issues.apache.org/jira/browse/HIVE-26758?focusedWorklogId=830903=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-830903 ] ASF GitHub Bot logged work on HIVE-26758: - Author: ASF GitHub Bot Created on: 04/Dec/22 19:15 Start Date: 04/Dec/22 19:15 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3781: URL: https://github.com/apache/hive/pull/3781#issuecomment-1336495148 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3781) [](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3781=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3781=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3781=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3781=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3781=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 830903) Time Spent: 2h 40m (was: 2.5h) > Allow use scratchdir for staging final job > -- > > Key: HIVE-26758 > URL: https://issues.apache.org/jira/browse/HIVE-26758 > Project: Hive > Issue Type: New Feature > Components: Query Planning >Affects Versions: 4.0.0-alpha-2 >Reporter: Yi Zhang >Assignee: Yi Zhang >Priority: Minor > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > The query results are staged in stagingdir that is relative to the > destination path // > during blobstorage optimzation HIVE-17620 final job is set to use stagingdir. > HIVE-15215 mentioned the possibility of using scratch for staging when write > to S3 but it was long time ago and no activity. > > This is to allow final job to use hive.exec.scratchdir as the interim jobs, > with a configuration > hive.use.scratchdir.for.staging > This is useful for cross Filesystem, user can use local source filesystem > instead of remote filesystem for the staging. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work logged] (HIVE-26799) Make authorizations on custom UDFs involved in tables/view configurable.
[
https://issues.apache.org/jira/browse/HIVE-26799?focusedWorklogId=830899=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-830899
]
ASF GitHub Bot logged work on HIVE-26799:
-
Author: ASF GitHub Bot
Created on: 04/Dec/22 18:42
Start Date: 04/Dec/22 18:42
Worklog Time Spent: 10m
Work Description: sonarcloud[bot] commented on PR #3821:
URL: https://github.com/apache/hive/pull/3821#issuecomment-1336488099
Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3821)
[](https://sonarcloud.io/project/issues?id=apache_hive=3821=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=3821=false=BUG)
[0
Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3821=false=BUG)
[](https://sonarcloud.io/project/issues?id=apache_hive=3821=false=VULNERABILITY)
[](https://sonarcloud.io/project/issues?id=apache_hive=3821=false=VULNERABILITY)
[0
Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3821=false=VULNERABILITY)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3821=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3821=false=SECURITY_HOTSPOT)
[0 Security
Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3821=false=SECURITY_HOTSPOT)
[](https://sonarcloud.io/project/issues?id=apache_hive=3821=false=CODE_SMELL)
[](https://sonarcloud.io/project/issues?id=apache_hive=3821=false=CODE_SMELL)
[1 Code
Smell](https://sonarcloud.io/project/issues?id=apache_hive=3821=false=CODE_SMELL)
[](https://sonarcloud.io/component_measures?id=apache_hive=3821=coverage=list)
No Coverage information
[](https://sonarcloud.io/component_measures?id=apache_hive=3821=duplicated_lines_density=list)
No Duplication information
Issue Time Tracking
---
Worklog Id: (was: 830899)
Time Spent: 1h (was: 50m)
> Make authorizations on custom UDFs involved in tables/view configurable.
>
>
> Key: HIVE-26799
> URL: https://issues.apache.org/jira/browse/HIVE-26799
> Project: Hive
> Issue Type: New Feature
> Components: HiveServer2, Security
>Affects Versions: 4.0.0-alpha-2
>Reporter: Sai Hemanth Gantasala
>Assignee: Sai Hemanth Gantasala
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> When Hive is using Ranger/Sentry as an authorization service, consider the
> following scenario.
> {code:java}
> > create table test_udf(st string); // privileged user operation
> > create function Udf_UPPER as 'openkb.hive.udf.MyUpper' using jar
> > 'hdfs:///tmp/MyUpperUDF-1.0.0.jar'; // privileged user operation
> > create view v1_udf as select udf_upper(st) from test_udf; // privileged
> > user operation
> //unprivileged user test_user is given select permissions on view v1_udf
> > select * from v1_udf; {code}
> It is expected that test_user needs to have select privilege on v1_udf and
> select permissions on udf_upper custom UDF in order to do a select query on
> view.
> This patch introduces a configuration
> "hive.security.authorization.functions.in.view"=false which disables
> authorization on views associated with views/tables during the
[jira] [Work logged] (HIVE-26758) Allow use scratchdir for staging final job
[ https://issues.apache.org/jira/browse/HIVE-26758?focusedWorklogId=830886=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-830886 ] ASF GitHub Bot logged work on HIVE-26758: - Author: ASF GitHub Bot Created on: 04/Dec/22 10:59 Start Date: 04/Dec/22 10:59 Worklog Time Spent: 10m Work Description: sonarcloud[bot] commented on PR #3781: URL: https://github.com/apache/hive/pull/3781#issuecomment-1336379430 Kudos, SonarCloud Quality Gate passed! [](https://sonarcloud.io/dashboard?id=apache_hive=3781) [](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=BUG) [0 Bugs](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=BUG) [](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=VULNERABILITY) [](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=VULNERABILITY) [0 Vulnerabilities](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=VULNERABILITY) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3781=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3781=false=SECURITY_HOTSPOT) [0 Security Hotspots](https://sonarcloud.io/project/security_hotspots?id=apache_hive=3781=false=SECURITY_HOTSPOT) [](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=CODE_SMELL) [](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=CODE_SMELL) [2 Code Smells](https://sonarcloud.io/project/issues?id=apache_hive=3781=false=CODE_SMELL) [](https://sonarcloud.io/component_measures?id=apache_hive=3781=coverage=list) No Coverage information [](https://sonarcloud.io/component_measures?id=apache_hive=3781=duplicated_lines_density=list) No Duplication information Issue Time Tracking --- Worklog Id: (was: 830886) Time Spent: 2.5h (was: 2h 20m) > Allow use scratchdir for staging final job > -- > > Key: HIVE-26758 > URL: https://issues.apache.org/jira/browse/HIVE-26758 > Project: Hive > Issue Type: New Feature > Components: Query Planning >Affects Versions: 4.0.0-alpha-2 >Reporter: Yi Zhang >Assignee: Yi Zhang >Priority: Minor > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > The query results are staged in stagingdir that is relative to the > destination path // > during blobstorage optimzation HIVE-17620 final job is set to use stagingdir. > HIVE-15215 mentioned the possibility of using scratch for staging when write > to S3 but it was long time ago and no activity. > > This is to allow final job to use hive.exec.scratchdir as the interim jobs, > with a configuration > hive.use.scratchdir.for.staging > This is useful for cross Filesystem, user can use local source filesystem > instead of remote filesystem for the staging. -- This message was sent by Atlassian Jira (v8.20.10#820010)
