[jira] [Updated] (HIVE-27702) Remove PowerMock from beeline and upgrade mockito to 4.11
[ https://issues.apache.org/jira/browse/HIVE-27702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Kunwar updated HIVE-27702: - Status: Patch Available (was: Open) > Remove PowerMock from beeline and upgrade mockito to 4.11 > - > > Key: HIVE-27702 > URL: https://issues.apache.org/jira/browse/HIVE-27702 > Project: Hive > Issue Type: Task > Components: HiveServer2 >Reporter: Zsolt Miskolczi >Assignee: Mayank Kunwar >Priority: Major > Labels: newbie, pull-request-available, starter > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27711) Allow creating a branch from tag name
[ https://issues.apache.org/jira/browse/HIVE-27711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27711: -- Labels: pull-request-available (was: ) > Allow creating a branch from tag name > - > > Key: HIVE-27711 > URL: https://issues.apache.org/jira/browse/HIVE-27711 > Project: Hive > Issue Type: Sub-task >Reporter: Ayush Saxena >Assignee: zhangbutao >Priority: Major > Labels: pull-request-available > > Allow creating a branch from tag name. > If a tag is already there we should be able to create a branch with the same > snapshot id that corresponds to that tag -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27711) Allow creating a branch from tag name
[ https://issues.apache.org/jira/browse/HIVE-27711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhangbutao reassigned HIVE-27711: - Assignee: zhangbutao > Allow creating a branch from tag name > - > > Key: HIVE-27711 > URL: https://issues.apache.org/jira/browse/HIVE-27711 > Project: Hive > Issue Type: Sub-task >Reporter: Ayush Saxena >Assignee: zhangbutao >Priority: Major > > Allow creating a branch from tag name. > If a tag is already there we should be able to create a branch with the same > snapshot id that corresponds to that tag -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HIVE-27715) Backport of HIVE-25235: Remove ThreadPoolExecutorWithOomHook
[ https://issues.apache.org/jira/browse/HIVE-27715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan resolved HIVE-27715. - Fix Version/s: 3.2.0 Resolution: Fixed > Backport of HIVE-25235: Remove ThreadPoolExecutorWithOomHook > > > Key: HIVE-27715 > URL: https://issues.apache.org/jira/browse/HIVE-27715 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.1.3 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > Fix For: 3.2.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27721) Backport of HIVE-20267, HIVE-20682, HIVE-23396 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-27721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Raj reassigned HIVE-27721: --- Assignee: Aman Raj > Backport of HIVE-20267, HIVE-20682, HIVE-23396 to branch-3 > -- > > Key: HIVE-27721 > URL: https://issues.apache.org/jira/browse/HIVE-27721 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.2.0 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27721) Backport of HIVE-20267, HIVE-20682, HIVE-23396 to branch-3
[ https://issues.apache.org/jira/browse/HIVE-27721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Raj updated HIVE-27721: Summary: Backport of HIVE-20267, HIVE-20682, HIVE-23396 to branch-3 (was: Backport of HIVE-23396: Many fixes and improvements to stabilize tests) > Backport of HIVE-20267, HIVE-20682, HIVE-23396 to branch-3 > -- > > Key: HIVE-27721 > URL: https://issues.apache.org/jira/browse/HIVE-27721 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.2.0 >Reporter: Aman Raj >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27715) Backport of HIVE-25235: Remove ThreadPoolExecutorWithOomHook
[ https://issues.apache.org/jira/browse/HIVE-27715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-27715: Affects Version/s: 3.1.3 (was: 3.2.0) > Backport of HIVE-25235: Remove ThreadPoolExecutorWithOomHook > > > Key: HIVE-27715 > URL: https://issues.apache.org/jira/browse/HIVE-27715 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.1.3 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27715) Backport of HIVE-25235: Remove ThreadPoolExecutorWithOomHook
[ https://issues.apache.org/jira/browse/HIVE-27715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan reassigned HIVE-27715: --- Assignee: Aman Raj > Backport of HIVE-25235: Remove ThreadPoolExecutorWithOomHook > > > Key: HIVE-27715 > URL: https://issues.apache.org/jira/browse/HIVE-27715 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.2.0 >Reporter: Aman Raj >Assignee: Aman Raj >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27721) Backport of HIVE-23396: Many fixes and improvements to stabilize tests
[ https://issues.apache.org/jira/browse/HIVE-27721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27721: -- Labels: pull-request-available (was: ) > Backport of HIVE-23396: Many fixes and improvements to stabilize tests > -- > > Key: HIVE-27721 > URL: https://issues.apache.org/jira/browse/HIVE-27721 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.2.0 >Reporter: Aman Raj >Priority: Major > Labels: pull-request-available > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27721) Backport of HIVE-23396: Many fixes and improvements to stabilize tests
Aman Raj created HIVE-27721: --- Summary: Backport of HIVE-23396: Many fixes and improvements to stabilize tests Key: HIVE-27721 URL: https://issues.apache.org/jira/browse/HIVE-27721 Project: Hive Issue Type: Sub-task Affects Versions: 3.2.0 Reporter: Aman Raj -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27720) Optimize NotificationLog Cleaner
Naveen Gangam created HIVE-27720: Summary: Optimize NotificationLog Cleaner Key: HIVE-27720 URL: https://issues.apache.org/jira/browse/HIVE-27720 Project: Hive Issue Type: Improvement Reporter: Naveen Gangam Assignee: Naveen Gangam Currently this thread fetches the events first in batches of 10k (or whatever configured) and then deletes them. This is not very optimal. This table has an index on the EVENT_ID which are unique. So the records have high cardinality. The cleaner thread deletes using high EVENT_TIME which also has high cardinality. So the fetching of this events appears to be doing a full table scan each time even for a following batches. Instead of fetching, we should be able to do a delete directly using JDO. This way we can delete them all in one go at the DB level. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27716) Precommit: Save log files for first 10 failures
[ https://issues.apache.org/jira/browse/HIVE-27716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27716: Description: This is about to simply applying https://github.com/kgyrtkirk/hive-test-kube/pull/11 to hive's master [Jenkinsfile|https://github.com/apache/hive/blob/master/Jenkinsfile] current artifacts are simply useless, flaky tests are impossible to investigate (no, they don't reproduce in flaky-check, damn) looks like the job simply picks up Jenkinsfile from the root: http://ci.hive.apache.org/job/hive-precommit/configure !Screenshot 2023-09-21 at 11.49.16.png! was: This is about to simply apply https://github.com/kgyrtkirk/hive-test-kube/pull/11 to hive's master [Jenkinsfile|https://github.com/apache/hive/blob/master/Jenkinsfile] current artifacts are simply useless, flaky tests are impossible to investigate (no, they don't reproduce in flaky-check, damn) looks like the job simply picks up Jenkinsfile from the root: http://ci.hive.apache.org/job/hive-precommit/configure !Screenshot 2023-09-21 at 11.49.16.png! > Precommit: Save log files for first 10 failures > --- > > Key: HIVE-27716 > URL: https://issues.apache.org/jira/browse/HIVE-27716 > Project: Hive > Issue Type: Sub-task >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Attachments: Screenshot 2023-09-21 at 11.49.16.png > > > This is about to simply applying > https://github.com/kgyrtkirk/hive-test-kube/pull/11 to hive's master > [Jenkinsfile|https://github.com/apache/hive/blob/master/Jenkinsfile] > current artifacts are simply useless, flaky tests are impossible to > investigate (no, they don't reproduce in flaky-check, damn) > looks like the job simply picks up Jenkinsfile from the root: > http://ci.hive.apache.org/job/hive-precommit/configure > !Screenshot 2023-09-21 at 11.49.16.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27718) TestMiniTezCliDriver: save application logs for failed tests
[
https://issues.apache.org/jira/browse/HIVE-27718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-27718:
--
Labels: pull-request-available (was: )
> TestMiniTezCliDriver: save application logs for failed tests
>
>
> Key: HIVE-27718
> URL: https://issues.apache.org/jira/browse/HIVE-27718
> Project: Hive
> Issue Type: Sub-task
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
> Labels: pull-request-available
>
> 1. locate tez app logs for a TestMiniTezCliDriver test
> {code}
> ls -laR itests/qtest/target/tmp/hive/yarn-*/hive-logDir-nm-*
> {code}
> 2. add them similarly to HIVE-27716
> important to note that tez app logs files are not specific to a particular
> test, so we can collect those for the whole module in case of an error
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-27695) Intermittent OOM when running TestMiniTezCliDriver
[
https://issues.apache.org/jira/browse/HIVE-27695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767548#comment-17767548
]
Stamatis Zampetakis commented on HIVE-27695:
To obtain the heapdump from the TestMiniTezCliDriver I modified the following
properties in "data/conf/tez/hive-site.xml" file as follows:
{code:xml}
hive.tez.java.opts
-Dlog4j.configurationFile=tez-container-log4j2.properties
-Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/home/stamatis/Issues/tez_union_oom/child
tez.am.launch.cmd-opts
-Dlog4j.configurationFile=tez-container-log4j2.properties
-Dtez.container.log.level=INFO -Dtez.container.root.logger=CLA
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/home/stamatis/Issues/tez_union_oom/am
{code}
> Intermittent OOM when running TestMiniTezCliDriver
> --
>
> Key: HIVE-27695
> URL: https://issues.apache.org/jira/browse/HIVE-27695
> Project: Hive
> Issue Type: Bug
> Components: Test
>Affects Versions: 4.0.0-beta-1
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
> Attachments: am_heap_dumps.tar.xz, leak_suspect_1.png
>
>
> Running all the tests under TestMiniTezCliDriver very frequently (but still
> intermittently) leads to OutOfMemory errors.
> {noformat}
> cd itests/qtest && mvn test -Dtest=TestMiniTezCliDriver
> {noformat}
> I set {{-XX:+HeapDumpOnOutOfMemoryError}} and the respective heapdumps are
> attached to this ticket.
> The OOM is thrown from the application master and a quick inspection of the
> dumps shows that it comes mainly from the accumulation of Configuration
> objects (~1MB each) by various classes.
> The max heap size for application master is pretty low (~100MB) so it is
> quite easy to reach. The heap size is explicitly very low for testing
> purposes but maybe we should re-evaluate the current configurations for the
> tests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-27695) Intermittent OOM when running TestMiniTezCliDriver
[
https://issues.apache.org/jira/browse/HIVE-27695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767538#comment-17767538
]
László Bodor commented on HIVE-27695:
-
can you please clarify how did you get the heapdump? it could be used in the
scope of HIVE-27719 with minimal or no modification
> Intermittent OOM when running TestMiniTezCliDriver
> --
>
> Key: HIVE-27695
> URL: https://issues.apache.org/jira/browse/HIVE-27695
> Project: Hive
> Issue Type: Bug
> Components: Test
>Affects Versions: 4.0.0-beta-1
>Reporter: Stamatis Zampetakis
>Assignee: Stamatis Zampetakis
>Priority: Major
> Attachments: am_heap_dumps.tar.xz, leak_suspect_1.png
>
>
> Running all the tests under TestMiniTezCliDriver very frequently (but still
> intermittently) leads to OutOfMemory errors.
> {noformat}
> cd itests/qtest && mvn test -Dtest=TestMiniTezCliDriver
> {noformat}
> I set {{-XX:+HeapDumpOnOutOfMemoryError}} and the respective heapdumps are
> attached to this ticket.
> The OOM is thrown from the application master and a quick inspection of the
> dumps shows that it comes mainly from the accumulation of Configuration
> objects (~1MB each) by various classes.
> The max heap size for application master is pretty low (~100MB) so it is
> quite easy to reach. The heap size is explicitly very low for testing
> purposes but maybe we should re-evaluate the current configurations for the
> tests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27718) TestMiniTezCliDriver: save application logs for failed tests
[
https://issues.apache.org/jira/browse/HIVE-27718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-27718:
Description:
1. locate tez app logs for a TestMiniTezCliDriver test
{code}
ls -laR itests/qtest/target/tmp/hive/yarn-*/hive-logDir-nm-*
{code}
2. add them similarly to HIVE-27716
important to note that tez app logs files are not specific to a particular
test, so we can collect those for the whole module in case of an error
was:
1. locate tez app logs for a TestMiniTezCliDriver test
{code}
ls -laR itests/qtest/target/tmp/hive/yarn-*/hive-logDir-nm-*
{code}
2. add them similarly to HIVE-27716
> TestMiniTezCliDriver: save application logs for failed tests
>
>
> Key: HIVE-27718
> URL: https://issues.apache.org/jira/browse/HIVE-27718
> Project: Hive
> Issue Type: Sub-task
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
>
> 1. locate tez app logs for a TestMiniTezCliDriver test
> {code}
> ls -laR itests/qtest/target/tmp/hive/yarn-*/hive-logDir-nm-*
> {code}
> 2. add them similarly to HIVE-27716
> important to note that tez app logs files are not specific to a particular
> test, so we can collect those for the whole module in case of an error
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (HIVE-27718) TestMiniTezCliDriver: save application logs for failed tests
[
https://issues.apache.org/jira/browse/HIVE-27718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767536#comment-17767536
]
László Bodor edited comment on HIVE-27718 at 9/21/23 12:26 PM:
---
sounds good [~zabetak], looking forward to seeing the actual fix for the root
cause, thanks for linking HIVE-27695
was (Author: abstractdog):
sounds good [~zabetak], looking forward to seeing the actual fix for the root
cause
> TestMiniTezCliDriver: save application logs for failed tests
>
>
> Key: HIVE-27718
> URL: https://issues.apache.org/jira/browse/HIVE-27718
> Project: Hive
> Issue Type: Sub-task
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
>
> 1. locate tez app logs for a TestMiniTezCliDriver test
> {code}
> ls -laR itests/qtest/target/tmp/hive/yarn-*/hive-logDir-nm-*
> {code}
> 2. add them similarly to HIVE-27716
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-27718) TestMiniTezCliDriver: save application logs for failed tests
[
https://issues.apache.org/jira/browse/HIVE-27718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767536#comment-17767536
]
László Bodor commented on HIVE-27718:
-
sounds good [~zabetak], looking forward to seeing the actual fix for the root
cause
> TestMiniTezCliDriver: save application logs for failed tests
>
>
> Key: HIVE-27718
> URL: https://issues.apache.org/jira/browse/HIVE-27718
> Project: Hive
> Issue Type: Sub-task
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
>
> 1. locate tez app logs for a TestMiniTezCliDriver test
> {code}
> ls -laR itests/qtest/target/tmp/hive/yarn-*/hive-logDir-nm-*
> {code}
> 2. add them similarly to HIVE-27716
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Work started] (HIVE-27718) TestMiniTezCliDriver: save application logs for failed tests
[
https://issues.apache.org/jira/browse/HIVE-27718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-27718 started by László Bodor.
---
> TestMiniTezCliDriver: save application logs for failed tests
>
>
> Key: HIVE-27718
> URL: https://issues.apache.org/jira/browse/HIVE-27718
> Project: Hive
> Issue Type: Sub-task
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
>
> 1. locate tez app logs for a TestMiniTezCliDriver test
> {code}
> ls -laR itests/qtest/target/tmp/hive/yarn-*/hive-logDir-nm-*
> {code}
> 2. add them similarly to HIVE-27716
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-27718) TestMiniTezCliDriver: save application logs for failed tests
[
https://issues.apache.org/jira/browse/HIVE-27718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767518#comment-17767518
]
Stamatis Zampetakis commented on HIVE-27718:
FYI regarding the flakiness of TestMiniTezCliDriver there is some WIP under
HIVE-27695.
> TestMiniTezCliDriver: save application logs for failed tests
>
>
> Key: HIVE-27718
> URL: https://issues.apache.org/jira/browse/HIVE-27718
> Project: Hive
> Issue Type: Sub-task
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
>
> 1. locate tez app logs for a TestMiniTezCliDriver test
> {code}
> ls -laR itests/qtest/target/tmp/hive/yarn-*/hive-logDir-nm-*
> {code}
> 2. add them similarly to HIVE-27716
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HIVE-27718) TestMiniTezCliDriver: save application logs for failed tests
[
https://issues.apache.org/jira/browse/HIVE-27718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
László Bodor updated HIVE-27718:
Description:
1. locate tez app logs for a TestMiniTezCliDriver test
{code}
ls -laR itests/qtest/target/tmp/hive/yarn-*/hive-logDir-nm-*
{code}
2. add them similarly to HIVE-27716
was:
1. locate tez app logs for a TestMiniTezCliDriver test
2. add them similarly to HIVE-27716
> TestMiniTezCliDriver: save application logs for failed tests
>
>
> Key: HIVE-27718
> URL: https://issues.apache.org/jira/browse/HIVE-27718
> Project: Hive
> Issue Type: Sub-task
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Major
>
> 1. locate tez app logs for a TestMiniTezCliDriver test
> {code}
> ls -laR itests/qtest/target/tmp/hive/yarn-*/hive-logDir-nm-*
> {code}
> 2. add them similarly to HIVE-27716
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-27718) TestMiniTezCliDriver: save application logs for failed tests
[
https://issues.apache.org/jira/browse/HIVE-27718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767497#comment-17767497
]
László Bodor commented on HIVE-27718:
-
{code}
ls -laR itests/qtest/target/tmp/hive/yarn-*/hive-logDir-nm-*
itests/qtest/target/tmp/hive/yarn-168379286/hive-logDir-nm-0_0:
total 0
drwxr-xr-x 3 laszlobodor staff 96 Sep 21 12:24 .
drwxr-xr-x 19 laszlobodor staff 608 Sep 21 12:24 ..
drwx--x--- 3 laszlobodor staff 96 Sep 21 12:24
application_1695291860552_0001
itests/qtest/target/tmp/hive/yarn-168379286/hive-logDir-nm-0_0/application_1695291860552_0001:
total 0
drwx--x--- 3 laszlobodor staff 96 Sep 21 12:24 .
drwxr-xr-x 3 laszlobodor staff 96 Sep 21 12:24 ..
drwx--x--- 2 laszlobodor staff 64 Sep 21 12:24
container_1695291860552_0001_01_01
itests/qtest/target/tmp/hive/yarn-168379286/hive-logDir-nm-0_0/application_1695291860552_0001/container_1695291860552_0001_01_01:
total 0
drwx--x--- 2 laszlobodor staff 64 Sep 21 12:24 .
drwx--x--- 3 laszlobodor staff 96 Sep 21 12:24 ..
itests/qtest/target/tmp/hive/yarn-168379286/hive-logDir-nm-0_1:
total 0
drwxr-xr-x 3 laszlobodor staff 96 Sep 21 12:24 .
drwxr-xr-x 19 laszlobodor staff 608 Sep 21 12:24 ..
drwx--x--- 3 laszlobodor staff 96 Sep 21 12:24
application_1695291860552_0001
itests/qtest/target/tmp/hive/yarn-168379286/hive-logDir-nm-0_1/application_1695291860552_0001:
total 0
drwx--x--- 3 laszlobodor staff 96 Sep 21 12:24 .
drwxr-xr-x 3 laszlobodor staff 96 Sep 21 12:24 ..
drwx--x--- 2 laszlobodor staff 64 Sep 21 12:24
container_1695291860552_0001_01_01
itests/qtest/target/tmp/hive/yarn-168379286/hive-logDir-nm-0_1/application_1695291860552_0001/container_1695291860552_0001_01_01:
total 0
drwx--x--- 2 laszlobodor staff 64 Sep 21 12:24 .
drwx--x--- 3 laszlobodor staff 96 Sep 21 12:24 ..
itests/qtest/target/tmp/hive/yarn-168379286/hive-logDir-nm-0_2:
total 0
drwxr-xr-x 3 laszlobodor staff 96 Sep 21 12:24 .
drwxr-xr-x 19 laszlobodor staff 608 Sep 21 12:24 ..
drwx--x--- 3 laszlobodor staff 96 Sep 21 12:24
application_1695291860552_0001
itests/qtest/target/tmp/hive/yarn-168379286/hive-logDir-nm-0_2/application_1695291860552_0001:
total 0
drwx--x--- 3 laszlobodor staff 96 Sep 21 12:24 .
drwxr-xr-x 3 laszlobodor staff 96 Sep 21 12:24 ..
drwx--x--- 9 laszlobodor staff 288 Sep 21 12:24
container_1695291860552_0001_01_01
itests/qtest/target/tmp/hive/yarn-168379286/hive-logDir-nm-0_2/application_1695291860552_0001/container_1695291860552_0001_01_01:
total 432
drwx--x--- 9 laszlobodor staff288 Sep 21 12:24 .
drwx--x--- 3 laszlobodor staff 96 Sep 21 12:24 ..
-rw-r--r-- 1 laszlobodor staff 3272 Sep 21 12:24 directory.info
-rw-r- 1 laszlobodor staff 69647 Sep 21 12:24 launch_container.sh
-rw-r--r-- 1 laszlobodor staff 0 Sep 21 12:24 prelaunch.err
-rw-r--r-- 1 laszlobodor staff100 Sep 21 12:24 prelaunch.out
-rw-r--r-- 1 laszlobodor staff935 Sep 21 12:24 stderr
-rw-r--r-- 1 laszlobodor staff 0 Sep 21 12:24 stdout
-rw-r--r-- 1 laszlobodor staff 80658 Sep 21 12:27 syslog
itests/qtest/target/tmp/hive/yarn-168379286/hive-logDir-nm-0_3:
total 0
drwxr-xr-x 3 laszlobodor staff 96 Sep 21 12:24 .
drwxr-xr-x 19 laszlobodor staff 608 Sep 21 12:24 ..
drwx--x--- 3 laszlobodor staff 96 Sep 21 12:24
application_1695291860552_0001
itests/qtest/target/tmp/hive/yarn-168379286/hive-logDir-nm-0_3/application_1695291860552_0001:
total 0
drwx--x--- 3 laszlobodor staff 96 Sep 21 12:24 .
drwxr-xr-x 3 laszlobodor staff 96 Sep 21 12:24 ..
drwx--x--- 3 laszlobodor staff 96 Sep 21 12:24
container_1695291860552_0001_01_01
itests/qtest/target/tmp/hive/yarn-168379286/hive-logDir-nm-0_3/application_1695291860552_0001/container_1695291860552_0001_01_01:
total 416
drwx--x--- 3 laszlobodor staff 96 Sep 21 12:24 .
drwx--x--- 3 laszlobodor staff 96 Sep 21 12:24 ..
-rw-rw-rw- 1 laszlobodor staff 212992 Sep 21 12:24
history.txt.appattempt_1695291860552_0001_01
itests/qtest/target/tmp/hive/yarn-168379286/hive-logDir-nm-1_0:
total 0
drwxr-xr-x 2 laszlobodor staff 64 Sep 21 12:24 .
drwxr-xr-x 19 laszlobodor staff 608 Sep 21 12:24 ..
itests/qtest/target/tmp/hive/yarn-168379286/hive-logDir-nm-1_1:
total 0
drwxr-xr-x 2 laszlobodor staff 64 Sep 21 12:24 .
drwxr-xr-x 19 laszlobodor staff 608 Sep 21 12:24 ..
itests/qtest/target/tmp/hive/yarn-168379286/hive-logDir-nm-1_2:
total 0
drwxr-xr-x 2 laszlobodor staff 64 Sep 21 12:24 .
drwxr-xr-x 19 laszlobodor staff 608 Sep 21 12:24 ..
itests/qtest/target/tmp/hive/yarn-168379286/hive-logDir-nm-1_3:
total 0
drwxr-xr-x 2 laszlobodor staff 64 Sep 21 12:24 .
drwxr-xr-x 19 laszlobodor staff 608 Sep 21 12:24 ..
{code}
> TestMiniTezCliDriver: save application logs for failed tests
>
>
[jira] [Updated] (HIVE-27718) TestMiniTezCliDriver: save application logs for failed tests
[ https://issues.apache.org/jira/browse/HIVE-27718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27718: Description: 1. locate tez app logs for a TestMiniTezCliDriver test 2. add them similarly to > TestMiniTezCliDriver: save application logs for failed tests > > > Key: HIVE-27718 > URL: https://issues.apache.org/jira/browse/HIVE-27718 > Project: Hive > Issue Type: Sub-task >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > > 1. locate tez app logs for a TestMiniTezCliDriver test > 2. add them similarly to -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27718) TestMiniTezCliDriver: save application logs for failed tests
[ https://issues.apache.org/jira/browse/HIVE-27718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27718: Description: 1. locate tez app logs for a TestMiniTezCliDriver test 2. add them similarly to HIVE-27716 was: 1. locate tez app logs for a TestMiniTezCliDriver test 2. add them similarly to > TestMiniTezCliDriver: save application logs for failed tests > > > Key: HIVE-27718 > URL: https://issues.apache.org/jira/browse/HIVE-27718 > Project: Hive > Issue Type: Sub-task >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > > 1. locate tez app logs for a TestMiniTezCliDriver test > 2. add them similarly to HIVE-27716 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27718) TestMiniTezCliDriver: save application logs for failed tests
[ https://issues.apache.org/jira/browse/HIVE-27718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor reassigned HIVE-27718: --- Assignee: László Bodor > TestMiniTezCliDriver: save application logs for failed tests > > > Key: HIVE-27718 > URL: https://issues.apache.org/jira/browse/HIVE-27718 > Project: Hive > Issue Type: Sub-task >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27719) Save heapdump in case of OOM
[ https://issues.apache.org/jira/browse/HIVE-27719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor reassigned HIVE-27719: --- Assignee: László Bodor > Save heapdump in case of OOM > > > Key: HIVE-27719 > URL: https://issues.apache.org/jira/browse/HIVE-27719 > Project: Hive > Issue Type: Sub-task >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > > This applies to 2 places: > 1. mini llap tests: 1 single JVM has everything (HS2, AM, tasks) > 2. mini tez test: tez app containers -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27719) Save heapdump in case of OOM
[ https://issues.apache.org/jira/browse/HIVE-27719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27719: Description: This applies to 2 places: 1. mini llap tests: 1 single JVM has everything (HS2, AM, tasks) 2. mini tez test: tez app containers > Save heapdump in case of OOM > > > Key: HIVE-27719 > URL: https://issues.apache.org/jira/browse/HIVE-27719 > Project: Hive > Issue Type: Sub-task >Reporter: László Bodor >Priority: Major > > This applies to 2 places: > 1. mini llap tests: 1 single JVM has everything (HS2, AM, tasks) > 2. mini tez test: tez app containers -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27718) TestMiniTezCliDriver: save application logs for failed tests
László Bodor created HIVE-27718: --- Summary: TestMiniTezCliDriver: save application logs for failed tests Key: HIVE-27718 URL: https://issues.apache.org/jira/browse/HIVE-27718 Project: Hive Issue Type: Sub-task Reporter: László Bodor -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27719) Save heapdump in case of OOM
László Bodor created HIVE-27719: --- Summary: Save heapdump in case of OOM Key: HIVE-27719 URL: https://issues.apache.org/jira/browse/HIVE-27719 Project: Hive Issue Type: Sub-task Reporter: László Bodor -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27716) Precommit: Save log files for first 10 failures
[ https://issues.apache.org/jira/browse/HIVE-27716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27716: Parent: HIVE-27717 Issue Type: Sub-task (was: Improvement) > Precommit: Save log files for first 10 failures > --- > > Key: HIVE-27716 > URL: https://issues.apache.org/jira/browse/HIVE-27716 > Project: Hive > Issue Type: Sub-task >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Attachments: Screenshot 2023-09-21 at 11.49.16.png > > > This is about to simply apply > https://github.com/kgyrtkirk/hive-test-kube/pull/11 to hive's master > [Jenkinsfile|https://github.com/apache/hive/blob/master/Jenkinsfile] > current artifacts are simply useless, flaky tests are impossible to > investigate (no, they don't reproduce in flaky-check, damn) > looks like the job simply picks up Jenkinsfile from the root: > http://ci.hive.apache.org/job/hive-precommit/configure > !Screenshot 2023-09-21 at 11.49.16.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27717) Improve precommit logging to address flaky tests easier
László Bodor created HIVE-27717: --- Summary: Improve precommit logging to address flaky tests easier Key: HIVE-27717 URL: https://issues.apache.org/jira/browse/HIVE-27717 Project: Hive Issue Type: Improvement Reporter: László Bodor -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27717) Improve precommit logging to address flaky tests easier
[ https://issues.apache.org/jira/browse/HIVE-27717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor reassigned HIVE-27717: --- Assignee: László Bodor > Improve precommit logging to address flaky tests easier > --- > > Key: HIVE-27717 > URL: https://issues.apache.org/jira/browse/HIVE-27717 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27716) Precommit: Save log files for first 10 failures
[ https://issues.apache.org/jira/browse/HIVE-27716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27716: Description: This is about to simply apply https://github.com/kgyrtkirk/hive-test-kube/pull/11 to hive's master [Jenkinsfile|https://github.com/apache/hive/blob/master/Jenkinsfile] current artifacts are simply useless, flaky tests are impossible to investigate (no, they don't reproduce in flaky-check, damn) looks like the job simply picks up Jenkinsfile from the root: http://ci.hive.apache.org/job/hive-precommit/configure !Screenshot 2023-09-21 at 11.49.16.png! was: This is about to simply apply https://github.com/kgyrtkirk/hive-test-kube/pull/11 to hive's master [Jenkinsfile|https://github.com/apache/hive/blob/master/Jenkinsfile] current artifacts are simply useless, flaky tests are impossible to investigate (no, they don't reproduce in flaky-check, damn) looks like the job simply picks up Jenkinsfile from the root: http://ci.hive.apache.org/job/hive-precommit/configure > Precommit: Save log files for first 10 failures > --- > > Key: HIVE-27716 > URL: https://issues.apache.org/jira/browse/HIVE-27716 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Attachments: Screenshot 2023-09-21 at 11.49.16.png > > > This is about to simply apply > https://github.com/kgyrtkirk/hive-test-kube/pull/11 to hive's master > [Jenkinsfile|https://github.com/apache/hive/blob/master/Jenkinsfile] > current artifacts are simply useless, flaky tests are impossible to > investigate (no, they don't reproduce in flaky-check, damn) > looks like the job simply picks up Jenkinsfile from the root: > http://ci.hive.apache.org/job/hive-precommit/configure > !Screenshot 2023-09-21 at 11.49.16.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27716) Precommit: Save log files for first 10 failures
[ https://issues.apache.org/jira/browse/HIVE-27716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27716: Description: This is about to simply apply https://github.com/kgyrtkirk/hive-test-kube/pull/11 to hive's master [Jenkinsfile|https://github.com/apache/hive/blob/master/Jenkinsfile] current artifacts are simply useless, flaky tests are impossible to investigate (no, they don't reproduce in flaky-check, damn) looks like the job simply picks up Jenkinsfile from the root: http://ci.hive.apache.org/job/hive-precommit/configure was: This is about to simply apply https://github.com/kgyrtkirk/hive-test-kube/pull/11 to hive's master [Jenkinsfile|https://github.com/apache/hive/blob/master/Jenkinsfile] current artifacts are simply useless, flaky tests are impossible to investigate (no, they don't reproduce in flaky-check, damn) > Precommit: Save log files for first 10 failures > --- > > Key: HIVE-27716 > URL: https://issues.apache.org/jira/browse/HIVE-27716 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Attachments: Screenshot 2023-09-21 at 11.49.16.png > > > This is about to simply apply > https://github.com/kgyrtkirk/hive-test-kube/pull/11 to hive's master > [Jenkinsfile|https://github.com/apache/hive/blob/master/Jenkinsfile] > current artifacts are simply useless, flaky tests are impossible to > investigate (no, they don't reproduce in flaky-check, damn) > looks like the job simply picks up Jenkinsfile from the root: > http://ci.hive.apache.org/job/hive-precommit/configure -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27716) Precommit: Save log files for first 10 failures
[ https://issues.apache.org/jira/browse/HIVE-27716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27716: Attachment: Screenshot 2023-09-21 at 11.49.16.png > Precommit: Save log files for first 10 failures > --- > > Key: HIVE-27716 > URL: https://issues.apache.org/jira/browse/HIVE-27716 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Attachments: Screenshot 2023-09-21 at 11.49.16.png > > > This is about to simply apply > https://github.com/kgyrtkirk/hive-test-kube/pull/11 to hive's master > [Jenkinsfile|https://github.com/apache/hive/blob/master/Jenkinsfile] > current artifacts are simply useless, flaky tests are impossible to > investigate (no, they don't reproduce in flaky-check, damn) > looks like the job simply picks up Jenkinsfile from the root: > http://ci.hive.apache.org/job/hive-precommit/configure -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27716) Precommit: Save log files for first 10 failures
[ https://issues.apache.org/jira/browse/HIVE-27716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27716: -- Labels: pull-request-available (was: ) > Precommit: Save log files for first 10 failures > --- > > Key: HIVE-27716 > URL: https://issues.apache.org/jira/browse/HIVE-27716 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > Labels: pull-request-available > Attachments: Screenshot 2023-09-21 at 11.49.16.png > > > This is about to simply apply > https://github.com/kgyrtkirk/hive-test-kube/pull/11 to hive's master > [Jenkinsfile|https://github.com/apache/hive/blob/master/Jenkinsfile] > current artifacts are simply useless, flaky tests are impossible to > investigate (no, they don't reproduce in flaky-check, damn) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27716) Precommit: Save log files for first 10 failures
[ https://issues.apache.org/jira/browse/HIVE-27716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-27716: Description: This is about to simply apply https://github.com/kgyrtkirk/hive-test-kube/pull/11 to hive's master [Jenkinsfile|https://github.com/apache/hive/blob/master/Jenkinsfile] current artifacts are simply useless, flaky tests are impossible to investigate (no, they don't reproduce in flaky-check, damn) > Precommit: Save log files for first 10 failures > --- > > Key: HIVE-27716 > URL: https://issues.apache.org/jira/browse/HIVE-27716 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > > This is about to simply apply > https://github.com/kgyrtkirk/hive-test-kube/pull/11 to hive's master > [Jenkinsfile|https://github.com/apache/hive/blob/master/Jenkinsfile] > current artifacts are simply useless, flaky tests are impossible to > investigate (no, they don't reproduce in flaky-check, damn) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Work started] (HIVE-27716) Precommit: Save log files for first 10 failures
[ https://issues.apache.org/jira/browse/HIVE-27716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-27716 started by László Bodor. --- > Precommit: Save log files for first 10 failures > --- > > Key: HIVE-27716 > URL: https://issues.apache.org/jira/browse/HIVE-27716 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > > This is about to simply apply > https://github.com/kgyrtkirk/hive-test-kube/pull/11 to hive's master > [Jenkinsfile|https://github.com/apache/hive/blob/master/Jenkinsfile] > current artifacts are simply useless, flaky tests are impossible to > investigate (no, they don't reproduce in flaky-check, damn) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (HIVE-27716) Precommit: Save log files for first 10 failures
[ https://issues.apache.org/jira/browse/HIVE-27716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor reassigned HIVE-27716: --- Assignee: László Bodor > Precommit: Save log files for first 10 failures > --- > > Key: HIVE-27716 > URL: https://issues.apache.org/jira/browse/HIVE-27716 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: László Bodor >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27716) Precommit: Save log files for first 10 failures
László Bodor created HIVE-27716: --- Summary: Precommit: Save log files for first 10 failures Key: HIVE-27716 URL: https://issues.apache.org/jira/browse/HIVE-27716 Project: Hive Issue Type: Improvement Reporter: László Bodor -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-25615) Hive on tez will generate at least one MapContainer per 0 length file
[ https://issues.apache.org/jira/browse/HIVE-25615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767458#comment-17767458 ] ZhengBowen commented on HIVE-25615: --- I also encountered this problem, this is indeed a very poor design > Hive on tez will generate at least one MapContainer per 0 length file > - > > Key: HIVE-25615 > URL: https://issues.apache.org/jira/browse/HIVE-25615 > Project: Hive > Issue Type: Bug > Components: Query Planning, Query Processor, Tez >Affects Versions: 3.1.2 > Environment: hive-3.1.2 > tez-0.10.1 >Reporter: JackYan >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > When tez read a table with many partitions and those partitions contain 0 > length file only, ColumnarSplitSizeEstimator will return Integer.MAX_VALUE > bytes length for every 0 length file.Then,TezSplitGrouper will treat those > files as big files,and generate at least one MapContainer per 0 file to > handle it.This is incorrect and even wasteful. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27702) Remove PowerMock from beeline and upgrade mockito to 4.11
[ https://issues.apache.org/jira/browse/HIVE-27702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27702: -- Labels: newbie pull-request-available starter (was: newbie starter) > Remove PowerMock from beeline and upgrade mockito to 4.11 > - > > Key: HIVE-27702 > URL: https://issues.apache.org/jira/browse/HIVE-27702 > Project: Hive > Issue Type: Task > Components: HiveServer2 >Reporter: Zsolt Miskolczi >Assignee: Mayank Kunwar >Priority: Major > Labels: newbie, pull-request-available, starter > -- This message was sent by Atlassian Jira (v8.20.10#820010)
