[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=843338=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-843338
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 03/Feb/23 00:21
Start Date: 03/Feb/23 00:21
Worklog Time Spent: 10m
Work Description: github-actions[bot] closed pull request #3602:
[HIVE-25616][FOLLOWUP] Set auth privileges should respect getauth
URL: https://github.com/apache/hive/pull/3602
Issue Time Tracking
---
Worklog Id: (was: 843338)
Time Spent: 4h (was: 3h 50m)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 4h
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=841885=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-841885
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 27/Jan/23 00:21
Start Date: 27/Jan/23 00:21
Worklog Time Spent: 10m
Work Description: github-actions[bot] commented on PR #3602:
URL: https://github.com/apache/hive/pull/3602#issuecomment-1405849812
This pull request has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the d...@hive.apache.org list if the patch is in
need of reviews.
Issue Time Tracking
---
Worklog Id: (was: 841885)
Time Spent: 3h 50m (was: 3h 40m)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=829063=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-829063
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 26/Nov/22 09:39
Start Date: 26/Nov/22 09:39
Worklog Time Spent: 10m
Work Description: pan3793 commented on PR #3602:
URL: https://github.com/apache/hive/pull/3602#issuecomment-1328014826
ping @nssalian @vihangk1, it's a backport issue
Issue Time Tracking
---
Worklog Id: (was: 829063)
Time Spent: 3h 40m (was: 3.5h)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=829042=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-829042
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 26/Nov/22 00:19
Start Date: 26/Nov/22 00:19
Worklog Time Spent: 10m
Work Description: github-actions[bot] commented on PR #3602:
URL: https://github.com/apache/hive/pull/3602#issuecomment-1327938630
This pull request has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the d...@hive.apache.org list if the patch is in
need of reviews.
Issue Time Tracking
---
Worklog Id: (was: 829042)
Time Spent: 3.5h (was: 3h 20m)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=812062=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-812062
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 26/Sep/22 11:17
Start Date: 26/Sep/22 11:17
Worklog Time Spent: 10m
Work Description: pan3793 commented on PR #3602:
URL: https://github.com/apache/hive/pull/3602#issuecomment-1257875300
would you like to take a look? @vihangk1
Issue Time Tracking
---
Worklog Id: (was: 812062)
Time Spent: 3h 20m (was: 3h 10m)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=812060=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-812060
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 26/Sep/22 11:16
Start Date: 26/Sep/22 11:16
Worklog Time Spent: 10m
Work Description: pan3793 commented on PR #3602:
URL: https://github.com/apache/hive/pull/3602#issuecomment-1257874445
This recovers all `testPartitions` UTs.
http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-3602/7/tests
Issue Time Tracking
---
Worklog Id: (was: 812060)
Time Spent: 3h 10m (was: 3h)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=809743=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-809743
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 17/Sep/22 09:51
Start Date: 17/Sep/22 09:51
Worklog Time Spent: 10m
Work Description: pan3793 commented on PR #3602:
URL: https://github.com/apache/hive/pull/3602#issuecomment-1250040404
ping @nssalian @vihangk1
Issue Time Tracking
---
Worklog Id: (was: 809743)
Time Spent: 3h (was: 2h 50m)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=809742=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-809742
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 17/Sep/22 09:11
Start Date: 17/Sep/22 09:11
Worklog Time Spent: 10m
Work Description: pan3793 opened a new pull request, #3602:
URL: https://github.com/apache/hive/pull/3602
### What changes were proposed in this pull request?
Set auth privileges should respect getauth.
### Why are the changes needed?
Recover tests
`org.apache.hadoop.hive.metastore.TestHiveMetaStore.testPartition`
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass `org.apache.hadoop.hive.metastore.TestHiveMetaStore.testPartition`
Issue Time Tracking
---
Worklog Id: (was: 809742)
Time Spent: 2h 50m (was: 2h 40m)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=672077=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-672077
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 29/Oct/21 17:21
Start Date: 29/Oct/21 17:21
Worklog Time Spent: 10m
Work Description: vihangk1 merged pull request #2733:
URL: https://github.com/apache/hive/pull/2733
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 672077)
Time Spent: 2h 40m (was: 2.5h)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=672076=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-672076
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 29/Oct/21 17:20
Start Date: 29/Oct/21 17:20
Worklog Time Spent: 10m
Work Description: vihangk1 commented on pull request #2733:
URL: https://github.com/apache/hive/pull/2733#issuecomment-954913202
This is similar patch to the one which we ported to Hive 2.x. I don't think
we have regular precommits running on Hive 3.1 branch and we don't have a
baseline of how many tests are failing without this patch. I am going to
override the test failure vote and merge this patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 672076)
Time Spent: 2.5h (was: 2h 20m)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=671201=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-671201
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 28/Oct/21 04:48
Start Date: 28/Oct/21 04:48
Worklog Time Spent: 10m
Work Description: nssalian commented on pull request #2733:
URL: https://github.com/apache/hive/pull/2733#issuecomment-953499315
@vihangk1 can we merge this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 671201)
Time Spent: 2h 20m (was: 2h 10m)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=668545=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-668545
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 21/Oct/21 17:03
Start Date: 21/Oct/21 17:03
Worklog Time Spent: 10m
Work Description: nssalian commented on pull request #2733:
URL: https://github.com/apache/hive/pull/2733#issuecomment-948815091
Thanks for the review @vihangk1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 668545)
Time Spent: 2h 10m (was: 2h)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=667353=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-667353
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 19/Oct/21 23:18
Start Date: 19/Oct/21 23:18
Worklog Time Spent: 10m
Work Description: nssalian commented on pull request #2733:
URL: https://github.com/apache/hive/pull/2733#issuecomment-947175884
CC: @vihangk1 . Same patch but on 3.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 667353)
Time Spent: 2h (was: 1h 50m)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=667351=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-667351
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 19/Oct/21 23:15
Start Date: 19/Oct/21 23:15
Worklog Time Spent: 10m
Work Description: nssalian opened a new pull request #2733:
URL: https://github.com/apache/hive/pull/2733
What changes were proposed in this pull request?
This PR is a backport of #1948 wherein the the performance of
get_partitions_ps_with_auth is improved into the Hive-3.1 branch.
Why are the changes needed?
Performance improvement for a HMS API on the Hive-3.1 branch.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Same tests as original patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 667351)
Time Spent: 1h 50m (was: 1h 40m)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=667231=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-667231
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 19/Oct/21 19:51
Start Date: 19/Oct/21 19:51
Worklog Time Spent: 10m
Work Description: vihangk1 commented on pull request #2730:
URL: https://github.com/apache/hive/pull/2730#issuecomment-947055116
Thanks @nssalian for your contribution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 667231)
Time Spent: 1h 40m (was: 1.5h)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=667230=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-667230
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 19/Oct/21 19:50
Start Date: 19/Oct/21 19:50
Worklog Time Spent: 10m
Work Description: vihangk1 merged pull request #2730:
URL: https://github.com/apache/hive/pull/2730
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 667230)
Time Spent: 1.5h (was: 1h 20m)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=667228=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-667228
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 19/Oct/21 19:49
Start Date: 19/Oct/21 19:49
Worklog Time Spent: 10m
Work Description: vihangk1 commented on pull request #2730:
URL: https://github.com/apache/hive/pull/2730#issuecomment-947053721
I am overriding the tests failed label in this case. Hive 2.x is a pretty
old branch. We don't have a baseline of how stable are the tests on that branch
in the first place.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 667228)
Time Spent: 1h 20m (was: 1h 10m)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Assignee: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=667177=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-667177
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 19/Oct/21 17:47
Start Date: 19/Oct/21 17:47
Worklog Time Spent: 10m
Work Description: nssalian commented on pull request #2730:
URL: https://github.com/apache/hive/pull/2730#issuecomment-946957642
Thanks @vihangk1 for the review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 667177)
Time Spent: 1h 10m (was: 1h)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=667174=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-667174
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 19/Oct/21 17:46
Start Date: 19/Oct/21 17:46
Worklog Time Spent: 10m
Work Description: vihangk1 commented on pull request #2730:
URL: https://github.com/apache/hive/pull/2730#issuecomment-946956766
Thanks for addressing the changes. I don't think the precommits are stable
for Hive 2.x branch so I am willing to override testing results and do a force
merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 667174)
Time Spent: 1h (was: 50m)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=666561=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666561
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 18/Oct/21 21:32
Start Date: 18/Oct/21 21:32
Worklog Time Spent: 10m
Work Description: nssalian commented on a change in pull request #2730:
URL: https://github.com/apache/hive/pull/2730#discussion_r731331838
##
File path: metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
##
@@ -2571,8 +2571,8 @@ private Collection getPartitionPsQueryResults(String
dbName, String tableName,
* all the partitions and hence we can attempt to use a directSQL equivalent
API which
* is considerably faster.
* @param partVals The partitions values used to filter out the partitions.
- * @return true if partVals is empty or if all the values in partVals is
empty strings.
- * other wise false. If user or groups is valid then returns false since the
directSQL
+ * @return false if partVals is empty, true if all the values in partVals
are empty strings and
Review comment:
makes sense. Fixing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 666561)
Time Spent: 50m (was: 40m)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=666559=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666559
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 18/Oct/21 21:23
Start Date: 18/Oct/21 21:23
Worklog Time Spent: 10m
Work Description: vihangk1 commented on a change in pull request #2730:
URL: https://github.com/apache/hive/pull/2730#discussion_r731326469
##
File path: metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
##
@@ -2571,8 +2571,8 @@ private Collection getPartitionPsQueryResults(String
dbName, String tableName,
* all the partitions and hence we can attempt to use a directSQL equivalent
API which
* is considerably faster.
* @param partVals The partitions values used to filter out the partitions.
- * @return true if partVals is empty or if all the values in partVals is
empty strings.
- * other wise false. If user or groups is valid then returns false since the
directSQL
+ * @return false if partVals is empty, true if all the values in partVals
are empty strings and
Review comment:
the comment should be "...and otherwise false for all the other cases"
Or more simply, the method returns true only when partVals is non-empty and
contains only empty strings, otherwise false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 666559)
Time Spent: 40m (was: 0.5h)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=666535=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666535
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 18/Oct/21 20:54
Start Date: 18/Oct/21 20:54
Worklog Time Spent: 10m
Work Description: nssalian commented on a change in pull request #2730:
URL: https://github.com/apache/hive/pull/2730#discussion_r731308987
##
File path: metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
##
@@ -2565,34 +2566,68 @@ private Collection getPartitionPsQueryResults(String
dbName, String tableName,
return (Collection) query.execute(dbName, tableName, partNameMatcher);
}
+ /**
+ * If partVals all the values are empty strings, it means we are returning
+ * all the partitions and hence we can attempt to use a directSQL equivalent
API which
+ * is considerably faster.
+ * @param partVals The partitions values used to filter out the partitions.
+ * @return true if partVals is empty or if all the values in partVals is
empty strings.
Review comment:
Will fix the docstring
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 666535)
Time Spent: 0.5h (was: 20m)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=666531=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666531
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 18/Oct/21 20:44
Start Date: 18/Oct/21 20:44
Worklog Time Spent: 10m
Work Description: vihangk1 commented on a change in pull request #2730:
URL: https://github.com/apache/hive/pull/2730#discussion_r731302621
##
File path: metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
##
@@ -2565,34 +2566,68 @@ private Collection getPartitionPsQueryResults(String
dbName, String tableName,
return (Collection) query.execute(dbName, tableName, partNameMatcher);
}
+ /**
+ * If partVals all the values are empty strings, it means we are returning
+ * all the partitions and hence we can attempt to use a directSQL equivalent
API which
+ * is considerably faster.
+ * @param partVals The partitions values used to filter out the partitions.
+ * @return true if partVals is empty or if all the values in partVals is
empty strings.
Review comment:
I know this is from the original patch but looks like this javadoc is
incorrect. The method actually returns false if partVals is empty. I don't
really remember now why that was the case. Perhaps you can look at this if you
have some cycles? If not, please fix the java doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 666531)
Time Spent: 20m (was: 10m)
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-25616) Backport HIVE-24741 to Hive 2, 3
[
https://issues.apache.org/jira/browse/HIVE-25616?focusedWorklogId=666511=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-666511
]
ASF GitHub Bot logged work on HIVE-25616:
-
Author: ASF GitHub Bot
Created on: 18/Oct/21 20:16
Start Date: 18/Oct/21 20:16
Worklog Time Spent: 10m
Work Description: nssalian commented on pull request #2730:
URL: https://github.com/apache/hive/pull/2730#issuecomment-946130817
CC: @nrg4878 , @vihangk1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 666511)
Remaining Estimate: 0h
Time Spent: 10m
> Backport HIVE-24741 to Hive 2, 3
> -
>
> Key: HIVE-25616
> URL: https://issues.apache.org/jira/browse/HIVE-25616
> Project: Hive
> Issue Type: Improvement
>Reporter: Neelesh Srinivas Salian
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HIVE-24741 adds a major improvement to the `{{get_partitions_ps_with_auth}}`
> API that is used by Spark to retrieve all partitions of the table.
> This has caused problems in Spark 3 - running on a Hive 2.3.x metastore.
> Patched this in my org and it helped get past large partitioned tables having
> problems reading metadata through Spark.
> [~vihangk1], thoughts?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
