[jira] [Commented] (HIVE-12952) Show query sub-pages on webui

2016-02-02 Thread Szehon Ho (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15128814#comment-15128814
 ] 

Szehon Ho commented on HIVE-12952:
--

TestHiveMetaStorePartitionSpecs failures could not reproduce locally.. and does 
not seem related as this is touching only HS2 code.

> Show query sub-pages on webui
> -
>
> Key: HIVE-12952
> URL: https://issues.apache.org/jira/browse/HIVE-12952
> Project: Hive
>  Issue Type: Sub-task
>  Components: Diagnosability
>Reporter: Szehon Ho
>Assignee: Szehon Ho
> Attachments: Error - Query Info Expired.png, HIVE-12952.2.patch, 
> HIVE-12952.3.patch, HIVE-12952.4.patch, HIVE-12952.patch, Query Drilldown 
> link.png, Tab 3 - Stages.png, Tab1- Base Profile.png, Tab2 - Query Plan.png, 
> Tab4 - Perf Logging.png
>
>
> Today the queries showing in Running and Completed lists have some basic 
> information like query string, elapsed time, state, user, etc.
> It may be helpful to have even more information like:
> * Job URL's, job status
> * Explain plan (configurable)
> * Error message (if failure)
> * Dynamic metrics, incl:
>  ** Number of Tables/partitions fetched
>  ** Time taken in each method via perf-logger.
> These should go in another page, so as not to clog the summary hiveserver.jsp 
> page.
> This JIRA aims to tackle some of those.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-12952) Show query sub-pages on webui

2016-02-02 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15128713#comment-15128713
 ] 

Hive QA commented on HIVE-12952:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12785649/HIVE-12952.4.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10048 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testAddPartitions
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testFetchingPartitionsWithDifferentSchemas
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testGetPartitionSpecs_WithAndWithoutPartitionGrouping
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6840/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6840/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6840/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12785649 - PreCommit-HIVE-TRUNK-Build

> Show query sub-pages on webui
> -
>
> Key: HIVE-12952
> URL: https://issues.apache.org/jira/browse/HIVE-12952
> Project: Hive
>  Issue Type: Sub-task
>  Components: Diagnosability
>Reporter: Szehon Ho
>Assignee: Szehon Ho
> Attachments: Error - Query Info Expired.png, HIVE-12952.2.patch, 
> HIVE-12952.3.patch, HIVE-12952.4.patch, HIVE-12952.patch, Query Drilldown 
> link.png, Tab 3 - Stages.png, Tab1- Base Profile.png, Tab2 - Query Plan.png, 
> Tab4 - Perf Logging.png
>
>
> Today the queries showing in Running and Completed lists have some basic 
> information like query string, elapsed time, state, user, etc.
> It may be helpful to have even more information like:
> * Job URL's, job status
> * Explain plan (configurable)
> * Error message (if failure)
> * Dynamic metrics, incl:
>  ** Number of Tables/partitions fetched
>  ** Time taken in each method via perf-logger.
> These should go in another page, so as not to clog the summary hiveserver.jsp 
> page.
> This JIRA aims to tackle some of those.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-12952) Show query sub-pages on webui

2016-02-02 Thread Szehon Ho (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15128701#comment-15128701
 ] 

Szehon Ho commented on HIVE-12952:
--

That's really good ideas Aihua, it sounds more organized, I'll file the 
follow-up JIRA's right after this one goes in.

> Show query sub-pages on webui
> -
>
> Key: HIVE-12952
> URL: https://issues.apache.org/jira/browse/HIVE-12952
> Project: Hive
>  Issue Type: Sub-task
>  Components: Diagnosability
>Reporter: Szehon Ho
>Assignee: Szehon Ho
> Attachments: Error - Query Info Expired.png, HIVE-12952.2.patch, 
> HIVE-12952.3.patch, HIVE-12952.4.patch, HIVE-12952.patch, Query Drilldown 
> link.png, Tab 3 - Stages.png, Tab1- Base Profile.png, Tab2 - Query Plan.png, 
> Tab4 - Perf Logging.png
>
>
> Today the queries showing in Running and Completed lists have some basic 
> information like query string, elapsed time, state, user, etc.
> It may be helpful to have even more information like:
> * Job URL's, job status
> * Explain plan (configurable)
> * Error message (if failure)
> * Dynamic metrics, incl:
>  ** Number of Tables/partitions fetched
>  ** Time taken in each method via perf-logger.
> These should go in another page, so as not to clog the summary hiveserver.jsp 
> page.
> This JIRA aims to tackle some of those.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-12952) Show query sub-pages on webui

2016-02-02 Thread Aihua Xu (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15128315#comment-15128315
 ] 

Aihua Xu commented on HIVE-12952:
-

+1. The patch looks good.

Possible future improvement to consider:
1. In Perf-log tab, if we can add a total time for compile and execution, that 
could be helpful for us to see if compile or execution has the bottleneck. 
2. The entries for compile stage should be divided into logical subtasks, like 
how much time is spent on Parser, Logical Optimizer, Physical Optimizer, etc. 
And each of them can be divided further. 
{noformat}
Compile-time 
Semantic Analysis  50
Parser20
Optimizer50

Total  5600
{noformat}

> Show query sub-pages on webui
> -
>
> Key: HIVE-12952
> URL: https://issues.apache.org/jira/browse/HIVE-12952
> Project: Hive
>  Issue Type: Sub-task
>  Components: Diagnosability
>Reporter: Szehon Ho
>Assignee: Szehon Ho
> Attachments: Error - Query Info Expired.png, HIVE-12952.2.patch, 
> HIVE-12952.3.patch, HIVE-12952.4.patch, HIVE-12952.patch, Query Drilldown 
> link.png, Tab 3 - Stages.png, Tab1- Base Profile.png, Tab2 - Query Plan.png, 
> Tab4 - Perf Logging.png
>
>
> Today the queries showing in Running and Completed lists have some basic 
> information like query string, elapsed time, state, user, etc.
> It may be helpful to have even more information like:
> * Job URL's, job status
> * Explain plan (configurable)
> * Error message (if failure)
> * Dynamic metrics, incl:
>  ** Number of Tables/partitions fetched
>  ** Time taken in each method via perf-logger.
> These should go in another page, so as not to clog the summary hiveserver.jsp 
> page.
> This JIRA aims to tackle some of those.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-12952) Show query sub-pages on webui

2016-02-01 Thread Jimmy Xiang (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15126908#comment-15126908
 ] 

Jimmy Xiang commented on HIVE-12952:


This is great.

> Show query sub-pages on webui
> -
>
> Key: HIVE-12952
> URL: https://issues.apache.org/jira/browse/HIVE-12952
> Project: Hive
>  Issue Type: Sub-task
>  Components: Diagnosability
>Reporter: Szehon Ho
>Assignee: Szehon Ho
> Attachments: Error - Query Info Expired.png, HIVE-12952.patch, Query 
> Drilldown link.png, Tab 3 - Stages.png, Tab 4 - Perf Logging.png, Tab1- Base 
> Profile.png, Tab2 - Query Plan.png
>
>
> Today the queries showing in Running and Completed lists have some basic 
> information like query string, elapsed time, state, user, etc.
> It may be helpful to have even more information like:
> * Job URL's, job status
> * Explain plan (configurable)
> * Error message (if failure)
> * Dynamic metrics, incl:
>  ** Number of Tables/partitions fetched
>  ** Time taken in each method via perf-logger.
> These should go in another page, so as not to clog the summary hiveserver.jsp 
> page.
> This JIRA aims to tackle some of those.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-12952) Show query sub-pages on webui

2016-02-01 Thread Mohit Sabharwal (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15126797#comment-15126797
 ] 

Mohit Sabharwal commented on HIVE-12952:


Thanks, [~szehon], could you add RB please.

> Show query sub-pages on webui
> -
>
> Key: HIVE-12952
> URL: https://issues.apache.org/jira/browse/HIVE-12952
> Project: Hive
>  Issue Type: Sub-task
>  Components: Diagnosability
>Reporter: Szehon Ho
>Assignee: Szehon Ho
> Attachments: Error - Query Info Expired.png, HIVE-12952.patch, Query 
> Drilldown link.png, Tab 3 - Stages.png, Tab 4 - Perf Logging.png, Tab1- Base 
> Profile.png, Tab2 - Query Plan.png
>
>
> Today the queries showing in Running and Completed lists have some basic 
> information like query string, elapsed time, state, user, etc.
> It may be helpful to have even more information like:
> * Job URL's, job status
> * Explain plan (configurable)
> * Error message (if failure)
> * Dynamic metrics, incl:
>  ** Number of Tables/partitions fetched
>  ** Time taken in each method via perf-logger.
> These should go in another page, so as not to clog the summary hiveserver.jsp 
> page.
> This JIRA aims to tackle some of those.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-12952) Show query sub-pages on webui

2016-01-31 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-12952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15125746#comment-15125746
 ] 

Hive QA commented on HIVE-12952:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12785327/HIVE-12952.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 9990 tests executed
*Failed tests:*
{noformat}
TestMiniSparkOnYarnCliDriver - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6820/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/6820/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-6820/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12785327 - PreCommit-HIVE-TRUNK-Build

> Show query sub-pages on webui
> -
>
> Key: HIVE-12952
> URL: https://issues.apache.org/jira/browse/HIVE-12952
> Project: Hive
>  Issue Type: Sub-task
>  Components: Diagnosability
>Reporter: Szehon Ho
>Assignee: Szehon Ho
> Attachments: Error - Query Info Expired.png, HIVE-12952.patch, Query 
> Drilldown link.png, Tab 3 - Stages.png, Tab 4 - Perf Logging.png, Tab1- Base 
> Profile.png, Tab2 - Query Plan.png
>
>
> Today the queries showing in Running and Completed lists have some basic 
> information like query string, elapsed time, state, user, etc.
> It may be helpful to have even more information like:
> * Job URL's, job status
> * Explain plan (configurable)
> * Error message (if failure)
> * Dynamic metrics, incl:
>  ** Number of Tables/partitions fetched
>  ** Time taken in each method via perf-logger.
> These should go in another page, so as not to clog the summary hiveserver.jsp 
> page.
> This JIRA aims to tackle some of those.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)