[jira] [Commented] (HIVE-20807) Refactor LlapStatusServiceDriver

2018-11-12 Thread Miklos Gergely (JIRA) 


[ 
https://issues.apache.org/jira/browse/HIVE-20807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684486#comment-16684486
 ] 

Miklos Gergely commented on HIVE-20807:
---

Thank you [~sershe]!

About the line lengths: there is kind of a confusion here, the following are 
contradicting each other:

https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-CodingConventions

https://github.com/apache/hive/blob/bc39c49988c8a5d881a23ed7dd5d4adba0509ee9/checkstyle/checkstyle.xml#L163

The latter was introduced this January by [~kgyrtkirk], reviewed by 
[~ashutoshc], and there is some discussion about it at HIVE-18222.

I personally prefer 120, but will go with 100 and fix all lines going forward 
in may patches to that, but then we should modify the checkstyle.xml as well.

> Refactor LlapStatusServiceDriver
> 
>
> Key: HIVE-20807
> URL: https://issues.apache.org/jira/browse/HIVE-20807
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20807.01.patch, HIVE-20807.02.patch, 
> HIVE-20807.03.patch, HIVE-20807.04.patch, HIVE-20807.05.patch, 
> HIVE-20807.06.patch
>
>
> LlapStatusServiceDriver is the class used to determine if LLAP has started. 
> The following problems should be solved by refactoring:
> 1. The main class is more than 800 lines long,should be cut into multiple 
> smaller classes.
> 2. The current design makes it extremely hard to write unit tests.
> 3. There are some overcomplicated, over-engineered parts of the code.
> 4. Most of the code is under org.apache.hadoop.hive.llap.cli, but some parts 
> are under org.apache.hadoop.hive.llap.cli.status. The whole program could be 
> moved to the latter.
> 5. LlapStatusHelpers serves as a class for holding classes, which doesn't 
> make much sense.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-20807) Refactor LlapStatusServiceDriver

2018-11-12 Thread Sergey Shelukhin (JIRA) 


[ 
https://issues.apache.org/jira/browse/HIVE-20807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16684439#comment-16684439
 ] 

Sergey Shelukhin commented on HIVE-20807:
-

+1
Nit: some lines are made longer than 100 chars that used to be shorter, and 
there weren't any logic changes to them.
I think your auto-formatting might need to be set to a 100-char limit.

> Refactor LlapStatusServiceDriver
> 
>
> Key: HIVE-20807
> URL: https://issues.apache.org/jira/browse/HIVE-20807
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20807.01.patch, HIVE-20807.02.patch, 
> HIVE-20807.03.patch, HIVE-20807.04.patch, HIVE-20807.05.patch, 
> HIVE-20807.06.patch
>
>
> LlapStatusServiceDriver is the class used to determine if LLAP has started. 
> The following problems should be solved by refactoring:
> 1. The main class is more than 800 lines long,should be cut into multiple 
> smaller classes.
> 2. The current design makes it extremely hard to write unit tests.
> 3. There are some overcomplicated, over-engineered parts of the code.
> 4. Most of the code is under org.apache.hadoop.hive.llap.cli, but some parts 
> are under org.apache.hadoop.hive.llap.cli.status. The whole program could be 
> moved to the latter.
> 5. LlapStatusHelpers serves as a class for holding classes, which doesn't 
> make much sense.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-20807) Refactor LlapStatusServiceDriver

2018-11-10 Thread Miklos Gergely (JIRA) 


[ 
https://issues.apache.org/jira/browse/HIVE-20807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682636#comment-16682636
 ] 

Miklos Gergely commented on HIVE-20807:
---

[~sershe] please don't forget to review this, before some other changes is made 
to the program, which would mean a lot of extra work updating this patch.

> Refactor LlapStatusServiceDriver
> 
>
> Key: HIVE-20807
> URL: https://issues.apache.org/jira/browse/HIVE-20807
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20807.01.patch, HIVE-20807.02.patch, 
> HIVE-20807.03.patch, HIVE-20807.04.patch, HIVE-20807.05.patch, 
> HIVE-20807.06.patch
>
>
> LlapStatusServiceDriver is the class used to determine if LLAP has started. 
> The following problems should be solved by refactoring:
> 1. The main class is more than 800 lines long,should be cut into multiple 
> smaller classes.
> 2. The current design makes it extremely hard to write unit tests.
> 3. There are some overcomplicated, over-engineered parts of the code.
> 4. Most of the code is under org.apache.hadoop.hive.llap.cli, but some parts 
> are under org.apache.hadoop.hive.llap.cli.status. The whole program could be 
> moved to the latter.
> 5. LlapStatusHelpers serves as a class for holding classes, which doesn't 
> make much sense.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-20807) Refactor LlapStatusServiceDriver

2018-11-07 Thread Miklos Gergely (JIRA) 


[ 
https://issues.apache.org/jira/browse/HIVE-20807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16678508#comment-16678508
 ] 

Miklos Gergely commented on HIVE-20807:
---

[~sershe] could you please review the changes?

> Refactor LlapStatusServiceDriver
> 
>
> Key: HIVE-20807
> URL: https://issues.apache.org/jira/browse/HIVE-20807
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20807.01.patch, HIVE-20807.02.patch, 
> HIVE-20807.03.patch, HIVE-20807.04.patch, HIVE-20807.05.patch, 
> HIVE-20807.06.patch
>
>
> LlapStatusServiceDriver is the class used to determine if LLAP has started. 
> The following problems should be solved by refactoring:
> 1. The main class is more than 800 lines long,should be cut into multiple 
> smaller classes.
> 2. The current design makes it extremely hard to write unit tests.
> 3. There are some overcomplicated, over-engineered parts of the code.
> 4. Most of the code is under org.apache.hadoop.hive.llap.cli, but some parts 
> are under org.apache.hadoop.hive.llap.cli.status. The whole program could be 
> moved to the latter.
> 5. LlapStatusHelpers serves as a class for holding classes, which doesn't 
> make much sense.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-20807) Refactor LlapStatusServiceDriver

2018-11-07 Thread Hive QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/HIVE-20807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16678131#comment-16678131
 ] 

Hive QA commented on HIVE-20807:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12947142/HIVE-20807.06.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 15526 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/14788/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14788/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14788/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12947142 - PreCommit-HIVE-Build

> Refactor LlapStatusServiceDriver
> 
>
> Key: HIVE-20807
> URL: https://issues.apache.org/jira/browse/HIVE-20807
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20807.01.patch, HIVE-20807.02.patch, 
> HIVE-20807.03.patch, HIVE-20807.04.patch, HIVE-20807.05.patch, 
> HIVE-20807.06.patch
>
>
> LlapStatusServiceDriver is the class used to determine if LLAP has started. 
> The following problems should be solved by refactoring:
> 1. The main class is more than 800 lines long,should be cut into multiple 
> smaller classes.
> 2. The current design makes it extremely hard to write unit tests.
> 3. There are some overcomplicated, over-engineered parts of the code.
> 4. Most of the code is under org.apache.hadoop.hive.llap.cli, but some parts 
> are under org.apache.hadoop.hive.llap.cli.status. The whole program could be 
> moved to the latter.
> 5. LlapStatusHelpers serves as a class for holding classes, which doesn't 
> make much sense.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-20807) Refactor LlapStatusServiceDriver

2018-11-07 Thread Hive QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/HIVE-20807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16678098#comment-16678098
 ] 

Hive QA commented on HIVE-20807:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
 9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  6m 
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
38s{color} | {color:blue} llap-server in master has 83 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
35s{color} | {color:blue} service in master has 48 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  6m 
32s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
7s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  6m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  6m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
20s{color} | {color:green} The patch . passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
11s{color} | {color:green} llap-server: The patch generated 0 new + 4 unchanged 
- 98 fixed = 4 total (was 102) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
11s{color} | {color:green} The patch service passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
47s{color} | {color:green} llap-server generated 0 new + 81 unchanged - 2 fixed 
= 81 total (was 83) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
40s{color} | {color:green} service in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  6m 
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m  3s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14788/dev-support/hive-personality.sh
 |
| git revision | master / 6d713b6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: . llap-server service U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-14788/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Refactor LlapStatusServiceDriver
> 
>
> Key: HIVE-20807
> URL: https://issues.apache.org/jira/browse/HIVE-20807
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20807.01.patch, HIVE-20807.02.patch, 
> HIVE-20807.03.patch, HIVE-20807.04.patch, HIVE-20807.05.patch, 
> HIVE-20807.06.patch
>
>
> LlapStatusServiceDriver is the class used to determine if LLAP has started. 
>

[jira] [Commented] (HIVE-20807) Refactor LlapStatusServiceDriver

2018-11-06 Thread Hive QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/HIVE-20807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677348#comment-16677348
 ] 

Hive QA commented on HIVE-20807:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12947033/HIVE-20807.05.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15524 tests 
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=196)

[druidmini_masking.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_joins.q,druid_timestamptz.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2] 
(batchId=85)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeOnTezEdges (batchId=323)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/14775/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14775/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14775/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12947033 - PreCommit-HIVE-Build

> Refactor LlapStatusServiceDriver
> 
>
> Key: HIVE-20807
> URL: https://issues.apache.org/jira/browse/HIVE-20807
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20807.01.patch, HIVE-20807.02.patch, 
> HIVE-20807.03.patch, HIVE-20807.04.patch, HIVE-20807.05.patch
>
>
> LlapStatusServiceDriver is the class used to determine if LLAP has started. 
> The following problems should be solved by refactoring:
> 1. The main class is more than 800 lines long,should be cut into multiple 
> smaller classes.
> 2. The current design makes it extremely hard to write unit tests.
> 3. There are some overcomplicated, over-engineered parts of the code.
> 4. Most of the code is under org.apache.hadoop.hive.llap.cli, but some parts 
> are under org.apache.hadoop.hive.llap.cli.status. The whole program could be 
> moved to the latter.
> 5. LlapStatusHelpers serves as a class for holding classes, which doesn't 
> make much sense.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-20807) Refactor LlapStatusServiceDriver

2018-11-06 Thread Hive QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/HIVE-20807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677328#comment-16677328
 ] 

Hive QA commented on HIVE-20807:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  6m 
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
39s{color} | {color:blue} llap-server in master has 83 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
35s{color} | {color:blue} service in master has 48 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  6m 
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
7s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
 1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  6m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  6m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
19s{color} | {color:green} The patch . passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
12s{color} | {color:green} llap-server: The patch generated 0 new + 4 unchanged 
- 98 fixed = 4 total (was 102) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
11s{color} | {color:green} The patch service passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
46s{color} | {color:green} llap-server generated 0 new + 81 unchanged - 2 fixed 
= 81 total (was 83) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
42s{color} | {color:green} service in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  6m 
42s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 27s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14775/dev-support/hive-personality.sh
 |
| git revision | master / f4a48fd |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: . llap-server service U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-14775/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Refactor LlapStatusServiceDriver
> 
>
> Key: HIVE-20807
> URL: https://issues.apache.org/jira/browse/HIVE-20807
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20807.01.patch, HIVE-20807.02.patch, 
> HIVE-20807.03.patch, HIVE-20807.04.patch, HIVE-20807.05.patch
>
>
> LlapStatusServiceDriver is the class used to determine if LLAP has started. 
> The following problems

[jira] [Commented] (HIVE-20807) Refactor LlapStatusServiceDriver

2018-11-01 Thread Hive QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/HIVE-20807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671150#comment-16671150
 ] 

Hive QA commented on HIVE-20807:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12946146/HIVE-20807.03.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15520 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
 (batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[strict_managed_tables_sysdb]
 (batchId=172)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/14693/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14693/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14693/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12946146 - PreCommit-HIVE-Build

> Refactor LlapStatusServiceDriver
> 
>
> Key: HIVE-20807
> URL: https://issues.apache.org/jira/browse/HIVE-20807
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20807.01.patch, HIVE-20807.02.patch, 
> HIVE-20807.03.patch
>
>
> LlapStatusServiceDriver is the class used to determine if LLAP has started. 
> The following problems should be solved by refactoring:
> 1. The main class is more than 800 lines long,should be cut into multiple 
> smaller classes.
> 2. The current design makes it extremely hard to write unit tests.
> 3. There are some overcomplicated, over-engineered parts of the code.
> 4. Most of the code is under org.apache.hadoop.hive.llap.cli, but some parts 
> are under org.apache.hadoop.hive.llap.cli.status. The whole program could be 
> moved to the latter.
> 5. LlapStatusHelpers serves as a class for holding classes, which doesn't 
> make much sense.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-20807) Refactor LlapStatusServiceDriver

2018-10-31 Thread Hive QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/HIVE-20807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16671138#comment-16671138
 ] 

Hive QA commented on HIVE-20807:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
 9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  6m 
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
41s{color} | {color:blue} llap-server in master has 83 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
35s{color} | {color:blue} service in master has 48 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  6m 
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
 2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  6m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  6m 
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
12s{color} | {color:red} llap-server: The patch generated 1 new + 4 unchanged - 
98 fixed = 5 total (was 102) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
47s{color} | {color:green} llap-server generated 0 new + 81 unchanged - 2 fixed 
= 81 total (was 83) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
43s{color} | {color:green} service in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  6m 
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 39s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14693/dev-support/hive-personality.sh
 |
| git revision | master / 08349b1 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-14693/yetus/diff-checkstyle-llap-server.txt
 |
| modules | C: . llap-server service U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-14693/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Refactor LlapStatusServiceDriver
> 
>
> Key: HIVE-20807
> URL: https://issues.apache.org/jira/browse/HIVE-20807
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20807.01.patch, HIVE-20807.02.patch, 
> HIVE-20807.03.patch
>
>
> LlapStatusServiceDriver is the class used to determine if LLAP has started. 
> The following problems should be solved by refactoring:
> 1. The main class is more than 800 lines long,should be cut into multiple 
> smaller classes.
> 2. The current design makes it extremely hard to write unit tests.
> 3. There are some overcomplicated,

[jira] [Commented] (HIVE-20807) Refactor LlapStatusServiceDriver

2018-10-27 Thread Hive QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/HIVE-20807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1274#comment-1274
 ] 

Hive QA commented on HIVE-20807:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12945662/HIVE-20807.02.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/14654/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14654/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14654/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL 
https://issues.apache.org/jira/secure/attachment/12945662/HIVE-20807.02.patch 
was found in seen patch url's cache and a test was probably run already on it. 
Aborting...
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12945662 - PreCommit-HIVE-Build

> Refactor LlapStatusServiceDriver
> 
>
> Key: HIVE-20807
> URL: https://issues.apache.org/jira/browse/HIVE-20807
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20807.01.patch, HIVE-20807.02.patch
>
>
> LlapStatusServiceDriver is the class used to determine if LLAP has started. 
> The following problems should be solved by refactoring:
> 1. The main class is more than 800 lines long,should be cut into multiple 
> smaller classes.
> 2. The current design makes it extremely hard to write unit tests.
> 3. There are some overcomplicated, over-engineered parts of the code.
> 4. Most of the code is under org.apache.hadoop.hive.llap.cli, but some parts 
> are under org.apache.hadoop.hive.llap.cli.status. The whole program could be 
> moved to the latter.
> 5. LlapStatusHelpers serves as a class for holding classes, which doesn't 
> make much sense.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-20807) Refactor LlapStatusServiceDriver

2018-10-27 Thread Hive QA (JIRA) 


[ 
https://issues.apache.org/jira/browse/HIVE-20807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1209#comment-1209
 ] 

Hive QA commented on HIVE-20807:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12945662/HIVE-20807.02.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/14651/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14651/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14651/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-10-27 19:41:20.336
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-14651/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-10-27 19:41:20.339
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 1002e89 HIVE-20638 : Upgrade version of Jetty to 
9.3.25.v20180904 (Laszlo Bodor  via Thejas Nair)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 1002e89 HIVE-20638 : Upgrade version of Jetty to 
9.3.25.v20180904 (Laszlo Bodor  via Thejas Nair)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-10-27 19:41:21.152
+ rm -rf ../yetus_PreCommit-HIVE-Build-14651
+ mkdir ../yetus_PreCommit-HIVE-Build-14651
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-14651
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-14651/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh 
/data/hiveptest/working/scratch/build.patch
error: a/bin/ext/llapstatus.sh: does not exist in index
error: 
a/llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapSliderUtils.java: 
does not exist in index
error: 
a/llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapStatusOptionsProcessor.java:
 does not exist in index
error: 
a/llap-server/src/java/org/apache/hadoop/hive/llap/cli/LlapStatusServiceDriver.java:
 does not exist in index
error: 
a/llap-server/src/java/org/apache/hadoop/hive/llap/cli/status/LlapStatusHelpers.java:
 does not exist in index
error: 
a/llap-server/src/test/org/apache/hadoop/hive/llap/cli/TestLlapStatusServiceDriver.java:
 does not exist in index
Going to apply patch with: git apply -p1
/data/hiveptest/working/scratch/build.patch:3395: new blank line at EOF.
+
/data/hiveptest/working/scratch/build.patch:3625: new blank line at EOF.
+
warning: 2 lines add whitespace errors.
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q 
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc3519613450051859971.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc3519613450051859971.exe, 
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
 
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
 
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator  Version 3.5.2
protoc-jar: executing: [/tmp/protoc4407913131038094891.exe, --version]
libprotoc 2.5.0
ANTLR Parser Generator  Version 3.5.2
Output file

[jira] [Commented] (HIVE-20807) Refactor LlapStatusServiceDriver

2018-10-25 Thread Miklos Gergely (JIRA) 


[ 
https://issues.apache.org/jira/browse/HIVE-20807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16664269#comment-16664269
 ] 

Miklos Gergely commented on HIVE-20807:
---

This patch is the first step of refactoring the program, now all of it 
components are moved under the package org.apache.hadoop.hive.llap.cli.status, 
all the classes and enums are put into a separate file, the overcomplicated 
parts of the command line parsing are replaced with a more simple structure, 
and the findbugs and checkstyle warnings are fixed.

> Refactor LlapStatusServiceDriver
> 
>
> Key: HIVE-20807
> URL: https://issues.apache.org/jira/browse/HIVE-20807
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-20807.01.patch, HIVE-20807.02.patch
>
>
> LlapStatusServiceDriver is the class used to determine if LLAP has started. 
> The following problems should be solved by refactoring:
> 1. The main class is more than 800 lines long,should be cut into multiple 
> smaller classes.
> 2. The current design makes it extremely hard to write unit tests.
> 3. There are some overcomplicated, over-engineered parts of the code.
> 4. Most of the code is under org.apache.hadoop.hive.llap.cli, but some parts 
> are under org.apache.hadoop.hive.llap.cli.status. The whole program could be 
> moved to the latter.
> 5. LlapStatusHelpers serves as a class for holding classes, which doesn't 
> make much sense.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)