[jira] [Comment Edited] (TEZ-2814) ATSImportTool has a return statement in a finally block

2015-10-29 Thread Rajesh Balamohan (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980686#comment-14980686
 ] 

Rajesh Balamohan edited comment on TEZ-2814 at 10/29/15 4:06 PM:
-

[~hitesh] - Please review when you find time.


was (Author: rajesh.balamohan):
[~hitesh] - Please review when you find sometime.

> ATSImportTool has a return statement in a finally block 
> 
>
> Key: TEZ-2814
> URL: https://issues.apache.org/jira/browse/TEZ-2814
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Hitesh Shah
>Assignee: Rajesh Balamohan
> Attachments: TEZ-2814.1.patch
>
>
> {code}
>  } catch (Exception e) {
>   LOG.error("Error in processing ", e);
>   throw e;
> } finally {
>  return result;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2679) Admin forms of launch env settings

2015-10-29 Thread Hitesh Shah (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980626#comment-14980626
 ] 

Hitesh Shah commented on TEZ-2679:
--

+1. Please mark this as incompatible in the changelog. Might be useful to add a 
release note to this jira too describing the behavior change. 

> Admin forms of launch env settings
> --
>
> Key: TEZ-2679
> URL: https://issues.apache.org/jira/browse/TEZ-2679
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Jason Lowe
>Assignee: Jonathan Eagles
> Attachments: TEZ-2679.1.patch, TEZ-2679.2.patch, TEZ-2679.3.patch, 
> TEZ-2679.4.patch
>
>
> Tez should provide corresponding admin configs for launch env and java opt 
> settings that can be specified in tez-site.xml by cluster admins.  For 
> example, if users override the launch env settings to setup LD_LIBRARY_PATH 
> to just reference their libs then we can fail to load the native Hadoop libs.
> This is currently mitigated by having Tez rely on the MapReduce admin 
> properties in many cases, but Tez should provide its own admin settings to 
> match the Tez-specific non-admin settings.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2814) ATSImportTool has a return statement in a finally block

2015-10-29 Thread Hitesh Shah (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980716#comment-14980716
 ] 

Hitesh Shah commented on TEZ-2814:
--

+1 pending pre-commit 

> ATSImportTool has a return statement in a finally block 
> 
>
> Key: TEZ-2814
> URL: https://issues.apache.org/jira/browse/TEZ-2814
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Hitesh Shah
>Assignee: Rajesh Balamohan
> Attachments: TEZ-2814.1.patch
>
>
> {code}
>  } catch (Exception e) {
>   LOG.error("Error in processing ", e);
>   throw e;
> } finally {
>  return result;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2679) Admin forms of launch env settings

2015-10-29 Thread Jonathan Eagles (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980517#comment-14980517
 ] 

Jonathan Eagles commented on TEZ-2679:
--

[~hitesh], can you give a final review due? Specifically looking at the 
behavior change?

> Admin forms of launch env settings
> --
>
> Key: TEZ-2679
> URL: https://issues.apache.org/jira/browse/TEZ-2679
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Jason Lowe
>Assignee: Jonathan Eagles
> Attachments: TEZ-2679.1.patch, TEZ-2679.2.patch, TEZ-2679.3.patch, 
> TEZ-2679.4.patch
>
>
> Tez should provide corresponding admin configs for launch env and java opt 
> settings that can be specified in tez-site.xml by cluster admins.  For 
> example, if users override the launch env settings to setup LD_LIBRARY_PATH 
> to just reference their libs then we can fail to load the native Hadoop libs.
> This is currently mitigated by having Tez rely on the MapReduce admin 
> properties in many cases, but Tez should provide its own admin settings to 
> match the Tez-specific non-admin settings.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.

2015-10-29 Thread Hitesh Shah (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980598#comment-14980598
 ] 

Hitesh Shah edited comment on TEZ-2909 at 10/29/15 3:14 PM:


Does the applicationId from allDags link to the application details page within 
the Tez UI or does it link directly to the YARN ResourceManager page for that 
application? 

If it links to the application details page, then we should always create a 
link regardless of whether TEZ_APPLICATION exists or not. 

Now on the application details page, depending on whether we have data or not, 
we can show the data if possible or show unavailable but that should not drive 
the link logic on the AllDAGs page. 


was (Author: hitesh):
Does the applicationId from allDags link to the application details page or 
does it link directly to the ResourceManager page? 

If it links to the application details page, then we should always create a 
link regardless of whether TEZ_APPLICATION exists or not. 

Now on the application details page, depending on whether we have data or not, 
we can show the data if possible or show unavailable but that should not drive 
the link logic on the AllDAGs page. 

> Tez UI: Application link in All DAGs table is disable when applicationhistory 
> is unavailable.
> -
>
> Key: TEZ-2909
> URL: https://issues.apache.org/jira/browse/TEZ-2909
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Attachments: TEZ-2909.1.patch
>
>
> Application link in All DAGs table is disable when applicationhistory is 
> unavailable. It must be available when TEZ_APPLICATION is available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.

2015-10-29 Thread Hitesh Shah (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980598#comment-14980598
 ] 

Hitesh Shah commented on TEZ-2909:
--

Does the applicationId from allDags link to the application details page or 
does it link directly to the ResourceManager page? 

> Tez UI: Application link in All DAGs table is disable when applicationhistory 
> is unavailable.
> -
>
> Key: TEZ-2909
> URL: https://issues.apache.org/jira/browse/TEZ-2909
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Attachments: TEZ-2909.1.patch
>
>
> Application link in All DAGs table is disable when applicationhistory is 
> unavailable. It must be available when TEZ_APPLICATION is available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2553) Tez UI: Tez UI Nits

2015-10-29 Thread Hitesh Shah (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980605#comment-14980605
 ] 

Hitesh Shah commented on TEZ-2553:
--

"Substring search ..." is not really more readable. The user does not get 
clarity on how the search box is meant to be used. Maybe just change to a 
simple "Search ..." for now. 

> Tez UI: Tez UI Nits
> ---
>
> Key: TEZ-2553
> URL: https://issues.apache.org/jira/browse/TEZ-2553
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
>Priority: Minor
> Attachments: TEZ-2553.1.patch
>
>
> - Tooltips are not disappearing at times
> - Make search message readable
> - Remove Application Attempt Id row from Tez Application Description in app 
> details page.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (TEZ-2916) Show counts of running tasks on the DAG visualization page

2015-10-29 Thread Siddharth Seth (JIRA)
Siddharth Seth created TEZ-2916:
---

 Summary: Show counts of running tasks on the DAG visualization page
 Key: TEZ-2916
 URL: https://issues.apache.org/jira/browse/TEZ-2916
 Project: Apache Tez
  Issue Type: Improvement
Reporter: Siddharth Seth


While a DAG is running, it's useful to have task counts for running / total / 
complete tasks on the DAG visualization page. This can be used to figure out 
dependencies, and which tasks/vertices are blocking others.

As part of this - some indication of which vertices are currently running will 
be a useful addition.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.

2015-10-29 Thread TezQA (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980707#comment-14980707
 ] 

TezQA commented on TEZ-2909:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12769489/TEZ-2909.1.patch
  against master revision 65f4575.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 3.0.1) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-TEZ-Build/1261//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1261//console

This message is automatically generated.

> Tez UI: Application link in All DAGs table is disable when applicationhistory 
> is unavailable.
> -
>
> Key: TEZ-2909
> URL: https://issues.apache.org/jira/browse/TEZ-2909
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Attachments: TEZ-2909.1.patch
>
>
> Application link in All DAGs table is disable when applicationhistory is 
> unavailable. It must be available when TEZ_APPLICATION is available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Failed: TEZ-2909 PreCommit Build #1261

2015-10-29 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-2909
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1261/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 3430 lines...]
[INFO] Total time: 42:34 min
[INFO] Finished at: 2015-10-29T16:17:12+00:00
[INFO] Final Memory: 86M/921M
[INFO] 




{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12769489/TEZ-2909.1.patch
  against master revision 65f4575.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 3.0.1) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-TEZ-Build/1261//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1261//console

This message is automatically generated.


==
==
Adding comment to Jira.
==
==


Comment added.
dffe4daa81d333d6365bf64ffca28cfe70369dcf logged out


==
==
Finished build.
==
==


Build step 'Execute shell' marked build as failure
Archiving artifacts
Compressed 3.02 MB of artifacts by 26.9% relative to #1258
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
All tests passed

[jira] [Commented] (TEZ-2553) Tez UI: Tez UI Nits

2015-10-29 Thread Hitesh Shah (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980607#comment-14980607
 ] 

Hitesh Shah commented on TEZ-2553:
--

+1 to commit after the search message is changed.  

> Tez UI: Tez UI Nits
> ---
>
> Key: TEZ-2553
> URL: https://issues.apache.org/jira/browse/TEZ-2553
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
>Priority: Minor
> Attachments: TEZ-2553.1.patch
>
>
> - Tooltips are not disappearing at times
> - Make search message readable
> - Remove Application Attempt Id row from Tez Application Description in app 
> details page.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Assigned] (TEZ-2814) ATSImportTool has a return statement in a finally block

2015-10-29 Thread Rajesh Balamohan (JIRA)

 [ 
https://issues.apache.org/jira/browse/TEZ-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajesh Balamohan reassigned TEZ-2814:
-

Assignee: Rajesh Balamohan

> ATSImportTool has a return statement in a finally block 
> 
>
> Key: TEZ-2814
> URL: https://issues.apache.org/jira/browse/TEZ-2814
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Hitesh Shah
>Assignee: Rajesh Balamohan
> Attachments: TEZ-2814.1.patch
>
>
> {code}
>  } catch (Exception e) {
>   LOG.error("Error in processing ", e);
>   throw e;
> } finally {
>  return result;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.

2015-10-29 Thread Hitesh Shah (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980598#comment-14980598
 ] 

Hitesh Shah edited comment on TEZ-2909 at 10/29/15 3:14 PM:


Does the applicationId from allDags link to the application details page or 
does it link directly to the ResourceManager page? 

If it links to the application details page, then we should always create a 
link regardless of whether TEZ_APPLICATION exists or not. 

Now on the application details page, depending on whether we have data or not, 
we can show the data if possible or show unavailable but that should not drive 
the link logic on the AllDAGs page. 


was (Author: hitesh):
Does the applicationId from allDags link to the application details page or 
does it link directly to the ResourceManager page? 

> Tez UI: Application link in All DAGs table is disable when applicationhistory 
> is unavailable.
> -
>
> Key: TEZ-2909
> URL: https://issues.apache.org/jira/browse/TEZ-2909
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Attachments: TEZ-2909.1.patch
>
>
> Application link in All DAGs table is disable when applicationhistory is 
> unavailable. It must be available when TEZ_APPLICATION is available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Success: TEZ-2814 PreCommit Build #1262

2015-10-29 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-2814
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1262/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 3424 lines...]
[INFO] Tez ... SUCCESS [  0.026 s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 42:19 min
[INFO] Finished at: 2015-10-29T17:07:36+00:00
[INFO] Final Memory: 81M/912M
[INFO] 




{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12769555/TEZ-2814.1.patch
  against master revision 65f4575.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 3.0.1) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-TEZ-Build/1262//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1262//console

This message is automatically generated.


==
==
Adding comment to Jira.
==
==


Comment added.
1b5d0012f848d85e29a39957e10ec2d7907b9bd5 logged out


==
==
Finished build.
==
==


Archiving artifacts
[description-setter] Description set: TEZ-2814
Recording test results
Email was triggered for: Success
Sending email for trigger: Success



###
## FAILED TESTS (if any) 
##
All tests passed

[jira] [Commented] (TEZ-2679) Admin forms of launch env settings

2015-10-29 Thread Hitesh Shah (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980908#comment-14980908
 ] 

Hitesh Shah commented on TEZ-2679:
--

I think we may need to file an INFRA jira for that. Would you mind filing that? 

FWIW, [~jeagles] you are a Tez JIRA admin too so both of us have the same level 
of permissions. In any case, a more descriptive message in CHANGES.txt should 
suffice for now. 

> Admin forms of launch env settings
> --
>
> Key: TEZ-2679
> URL: https://issues.apache.org/jira/browse/TEZ-2679
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Jason Lowe
>Assignee: Jonathan Eagles
> Fix For: 0.7.1, 0.8.2
>
> Attachments: TEZ-2679.1.patch, TEZ-2679.2.patch, TEZ-2679.3.patch, 
> TEZ-2679.4.patch
>
>
> Tez should provide corresponding admin configs for launch env and java opt 
> settings that can be specified in tez-site.xml by cluster admins.  For 
> example, if users override the launch env settings to setup LD_LIBRARY_PATH 
> to just reference their libs then we can fail to load the native Hadoop libs.
> This is currently mitigated by having Tez rely on the MapReduce admin 
> properties in many cases, but Tez should provide its own admin settings to 
> match the Tez-specific non-admin settings.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.

2015-10-29 Thread Sreenath Somarajapuram (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980793#comment-14980793
 ] 

Sreenath Somarajapuram commented on TEZ-2909:
-

Thanks [~hitesh]
Will make the required changes, also that would improve the All DAGs, page UX.

> Tez UI: Application link in All DAGs table is disable when applicationhistory 
> is unavailable.
> -
>
> Key: TEZ-2909
> URL: https://issues.apache.org/jira/browse/TEZ-2909
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Attachments: TEZ-2909.1.patch
>
>
> Application link in All DAGs table is disable when applicationhistory is 
> unavailable. It must be available when TEZ_APPLICATION is available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2814) ATSImportTool has a return statement in a finally block

2015-10-29 Thread TezQA (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980814#comment-14980814
 ] 

TezQA commented on TEZ-2814:


{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12769555/TEZ-2814.1.patch
  against master revision 65f4575.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 3.0.1) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-TEZ-Build/1262//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1262//console

This message is automatically generated.

> ATSImportTool has a return statement in a finally block 
> 
>
> Key: TEZ-2814
> URL: https://issues.apache.org/jira/browse/TEZ-2814
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Hitesh Shah
>Assignee: Rajesh Balamohan
> Attachments: TEZ-2814.1.patch
>
>
> {code}
>  } catch (Exception e) {
>   LOG.error("Error in processing ", e);
>   throw e;
> } finally {
>  return result;
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2679) Admin forms of launch env settings

2015-10-29 Thread Jonathan Eagles (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980747#comment-14980747
 ] 

Jonathan Eagles commented on TEZ-2679:
--

Committed to master and branch-0.7.

[~hitesh], the Tez project is not configured to have the _Release note_ field. 
This seems like a useful feature to add to the project. Can you enable this if 
you have the permissions to add this field to the project? 

> Admin forms of launch env settings
> --
>
> Key: TEZ-2679
> URL: https://issues.apache.org/jira/browse/TEZ-2679
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Jason Lowe
>Assignee: Jonathan Eagles
> Fix For: 0.7.1, 0.8.2
>
> Attachments: TEZ-2679.1.patch, TEZ-2679.2.patch, TEZ-2679.3.patch, 
> TEZ-2679.4.patch
>
>
> Tez should provide corresponding admin configs for launch env and java opt 
> settings that can be specified in tez-site.xml by cluster admins.  For 
> example, if users override the launch env settings to setup LD_LIBRARY_PATH 
> to just reference their libs then we can fail to load the native Hadoop libs.
> This is currently mitigated by having Tez rely on the MapReduce admin 
> properties in many cases, but Tez should provide its own admin settings to 
> match the Tez-specific non-admin settings.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.

2015-10-29 Thread Sreenath Somarajapuram (JIRA)

 [ 
https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sreenath Somarajapuram updated TEZ-2909:

Attachment: TEZ-2909.2.patch

Attaching a fresh patch with the changes advised by Hitesh.

The link is enabled by default, and if TEZ_APPLICATION fetch fails an error bar 
will be displayed as expected.

> Tez UI: Application link in All DAGs table is disable when applicationhistory 
> is unavailable.
> -
>
> Key: TEZ-2909
> URL: https://issues.apache.org/jira/browse/TEZ-2909
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Attachments: TEZ-2909.1.patch, TEZ-2909.2.patch
>
>
> Application link in All DAGs table is disable when applicationhistory is 
> unavailable. It must be available when TEZ_APPLICATION is available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (TEZ-2553) Tez UI: Tez UI Nits

2015-10-29 Thread Sreenath Somarajapuram (JIRA)

 [ 
https://issues.apache.org/jira/browse/TEZ-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sreenath Somarajapuram updated TEZ-2553:

Attachment: TEZ-2553.2.patch

Attaching patch with the text change.
Committing shortly.

> Tez UI: Tez UI Nits
> ---
>
> Key: TEZ-2553
> URL: https://issues.apache.org/jira/browse/TEZ-2553
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
>Priority: Minor
> Fix For: 0.7.1, 0.8.2
>
> Attachments: TEZ-2553.1.patch, TEZ-2553.2.patch
>
>
> - Tooltips are not disappearing at times
> - Make search message readable
> - Remove Application Attempt Id row from Tez Application Description in app 
> details page.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.

2015-10-29 Thread Sreenath Somarajapuram (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981035#comment-14981035
 ] 

Sreenath Somarajapuram edited comment on TEZ-2909 at 10/29/15 7:05 PM:
---

bq. Does the applicationId from allDags link to the application details page 
within the Tez UI or does it link directly to the YARN ResourceManager page for 
that application?
ApplicationId from allDags link to the application details page

Attaching a fresh patch with the changes advised by Hitesh.

The link is enabled by default, and if TEZ_APPLICATION fetch fails an error bar 
will be displayed as expected.


was (Author: sreenath):
Attaching a fresh patch with the changes advised by Hitesh.

The link is enabled by default, and if TEZ_APPLICATION fetch fails an error bar 
will be displayed as expected.

> Tez UI: Application link in All DAGs table is disable when applicationhistory 
> is unavailable.
> -
>
> Key: TEZ-2909
> URL: https://issues.apache.org/jira/browse/TEZ-2909
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Attachments: TEZ-2909.1.patch, TEZ-2909.2.patch
>
>
> Application link in All DAGs table is disable when applicationhistory is 
> unavailable. It must be available when TEZ_APPLICATION is available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2679) Admin forms of launch env settings

2015-10-29 Thread Jonathan Eagles (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981037#comment-14981037
 ] 

Jonathan Eagles commented on TEZ-2679:
--

Filed INFRA-10684 to add Release notes to TEZ project expect time before fix is 
about a week.

> Admin forms of launch env settings
> --
>
> Key: TEZ-2679
> URL: https://issues.apache.org/jira/browse/TEZ-2679
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Jason Lowe
>Assignee: Jonathan Eagles
> Fix For: 0.7.1, 0.8.2
>
> Attachments: TEZ-2679.1.patch, TEZ-2679.2.patch, TEZ-2679.3.patch, 
> TEZ-2679.4.patch
>
>
> Tez should provide corresponding admin configs for launch env and java opt 
> settings that can be specified in tez-site.xml by cluster admins.  For 
> example, if users override the launch env settings to setup LD_LIBRARY_PATH 
> to just reference their libs then we can fail to load the native Hadoop libs.
> This is currently mitigated by having Tez rely on the MapReduce admin 
> properties in many cases, but Tez should provide its own admin settings to 
> match the Tez-specific non-admin settings.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (TEZ-2553) Tez UI: Tez UI Nits

2015-10-29 Thread Sreenath Somarajapuram (JIRA)

 [ 
https://issues.apache.org/jira/browse/TEZ-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sreenath Somarajapuram updated TEZ-2553:

Fix Version/s: 0.8.2
   0.7.1

> Tez UI: Tez UI Nits
> ---
>
> Key: TEZ-2553
> URL: https://issues.apache.org/jira/browse/TEZ-2553
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
>Priority: Minor
> Fix For: 0.7.1, 0.8.2
>
> Attachments: TEZ-2553.1.patch, TEZ-2553.2.patch
>
>
> - Tooltips are not disappearing at times
> - Make search message readable
> - Remove Application Attempt Id row from Tez Application Description in app 
> details page.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-808) Handle task attempts that are not making progress

2015-10-29 Thread Jason Lowe (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981205#comment-14981205
 ] 

Jason Lowe commented on TEZ-808:


Thanks for updating the patch, Bikas!  Patch still doesn't treat 0 as a 
disabled timeout but otherwise lgtm.

> Handle task attempts that are not making progress
> -
>
> Key: TEZ-808
> URL: https://issues.apache.org/jira/browse/TEZ-808
> Project: Apache Tez
>  Issue Type: Sub-task
>Reporter: Bikas Saha
>Assignee: Bikas Saha
> Attachments: TEZ-808.1.patch, TEZ-808.2.patch
>
>
> If a task attempt is not making progress then it may cause the job to hang. 
> We may want to kill and restart the attempt. With speculation support and 
> free resources we may want to run another version in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Failed: TEZ-2909 PreCommit Build #1264

2015-10-29 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-2909
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1264/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 3428 lines...]
[INFO] Total time: 42:11 min
[INFO] Finished at: 2015-10-29T22:28:11+00:00
[INFO] Final Memory: 94M/1144M
[INFO] 




{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12769600/TEZ-2909.2.patch
  against master revision dcc5108.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 3.0.1) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-TEZ-Build/1264//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1264//console

This message is automatically generated.


==
==
Adding comment to Jira.
==
==


Comment added.
4a20a661f8438475a101db422a067d654a4ffbb6 logged out


==
==
Finished build.
==
==


Build step 'Execute shell' marked build as failure
Archiving artifacts
Compressed 2.99 MB of artifacts by 29.3% relative to #1262
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
All tests passed

[jira] [Commented] (TEZ-2917) change some logs from info to debug

2015-10-29 Thread Sergey Shelukhin (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981456#comment-14981456
 ] 

Sergey Shelukhin commented on TEZ-2917:
---

[~sseth] can you take a look?

> change some logs from info to debug
> ---
>
> Key: TEZ-2917
> URL: https://issues.apache.org/jira/browse/TEZ-2917
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: TEZ-2917.patch
>
>
> I've done a highly unscientific summarization of the logs from some random 
> queries, and will now change some log statements that are the most prevalent 
> and not extremely useful from info to debug.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Moved] (TEZ-2917) change some logs from info to debug

2015-10-29 Thread Sergey Shelukhin (JIRA)

 [ 
https://issues.apache.org/jira/browse/TEZ-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Shelukhin moved HIVE-12298 to TEZ-2917:
--

Key: TEZ-2917  (was: HIVE-12298)
Project: Apache Tez  (was: Hive)

> change some logs from info to debug
> ---
>
> Key: TEZ-2917
> URL: https://issues.apache.org/jira/browse/TEZ-2917
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>
> I've done a highly unscientific summarization of the logs from some random 
> queries, and will now change some log statements that are the most prevalent 
> and not extremely useful from info to debug.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2917) change some logs from info to debug

2015-10-29 Thread Hitesh Shah (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981477#comment-14981477
 ] 

Hitesh Shah commented on TEZ-2917:
--

How much general savings in terms of log size does this provide on a 
decent-size query?

> change some logs from info to debug
> ---
>
> Key: TEZ-2917
> URL: https://issues.apache.org/jira/browse/TEZ-2917
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: TEZ-2917.patch
>
>
> I've done a highly unscientific summarization of the logs from some random 
> queries, and will now change some log statements that are the most prevalent 
> and not extremely useful from info to debug.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2917) change some logs from info to debug

2015-10-29 Thread Sergey Shelukhin (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981494#comment-14981494
 ] 

Sergey Shelukhin commented on TEZ-2917:
---

This is mostly coming from the complaint that logs are hard to read, not from 
size.

> change some logs from info to debug
> ---
>
> Key: TEZ-2917
> URL: https://issues.apache.org/jira/browse/TEZ-2917
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: TEZ-2917.patch
>
>
> I've done a highly unscientific summarization of the logs from some random 
> queries, and will now change some log statements that are the most prevalent 
> and not extremely useful from info to debug.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.

2015-10-29 Thread TezQA (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981448#comment-14981448
 ] 

TezQA commented on TEZ-2909:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12769600/TEZ-2909.2.patch
  against master revision dcc5108.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 3.0.1) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-TEZ-Build/1264//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1264//console

This message is automatically generated.

> Tez UI: Application link in All DAGs table is disable when applicationhistory 
> is unavailable.
> -
>
> Key: TEZ-2909
> URL: https://issues.apache.org/jira/browse/TEZ-2909
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Attachments: TEZ-2909.1.patch, TEZ-2909.2.patch
>
>
> Application link in All DAGs table is disable when applicationhistory is 
> unavailable. It must be available when TEZ_APPLICATION is available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (TEZ-2917) change some logs from info to debug

2015-10-29 Thread Sergey Shelukhin (JIRA)

 [ 
https://issues.apache.org/jira/browse/TEZ-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Shelukhin updated TEZ-2917:
--
Attachment: TEZ-2917.patch

> change some logs from info to debug
> ---
>
> Key: TEZ-2917
> URL: https://issues.apache.org/jira/browse/TEZ-2917
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: TEZ-2917.patch
>
>
> I've done a highly unscientific summarization of the logs from some random 
> queries, and will now change some log statements that are the most prevalent 
> and not extremely useful from info to debug.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2553) Tez UI: Tez UI Nits

2015-10-29 Thread TezQA (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981487#comment-14981487
 ] 

TezQA commented on TEZ-2553:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12769601/TEZ-2553.2.patch
  against master revision dcc5108.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 3.0.1) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-TEZ-Build/1263//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1263//console

This message is automatically generated.

> Tez UI: Tez UI Nits
> ---
>
> Key: TEZ-2553
> URL: https://issues.apache.org/jira/browse/TEZ-2553
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
>Priority: Minor
> Fix For: 0.7.1, 0.8.2
>
> Attachments: TEZ-2553.1.patch, TEZ-2553.2.patch
>
>
> - Tooltips are not disappearing at times
> - Make search message readable
> - Remove Application Attempt Id row from Tez Application Description in app 
> details page.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Failed: TEZ-2553 PreCommit Build #1263

2015-10-29 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-2553
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1263/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 3430 lines...]
[INFO] 
[INFO] Total time: 50:50 min
[INFO] Finished at: 2015-10-29T22:44:33+00:00
[INFO] Final Memory: 100M/1405M
[INFO] 




{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12769601/TEZ-2553.2.patch
  against master revision dcc5108.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 3.0.1) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-TEZ-Build/1263//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1263//console

This message is automatically generated.


==
==
Adding comment to Jira.
==
==


Comment added.
905a1e44efb897ae56451ed1183174271626f88b logged out


==
==
Finished build.
==
==


Build step 'Execute shell' marked build as failure
Archiving artifacts
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
All tests passed

[jira] [Commented] (TEZ-2917) change some logs from info to debug

2015-10-29 Thread Sergey Shelukhin (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981500#comment-14981500
 ] 

Sergey Shelukhin commented on TEZ-2917:
---

I ran some queries, grouped logs by first 30 characters with numbers removed, 
and went thru them manually for those that were present in relatively large 
numbers. So this is how these were removed.

> change some logs from info to debug
> ---
>
> Key: TEZ-2917
> URL: https://issues.apache.org/jira/browse/TEZ-2917
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: TEZ-2917.patch
>
>
> I've done a highly unscientific summarization of the logs from some random 
> queries, and will now change some log statements that are the most prevalent 
> and not extremely useful from info to debug.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (TEZ-2915) Tez UI: Getting back to the DAG details page is difficult

2015-10-29 Thread Sreenath Somarajapuram (JIRA)
Sreenath Somarajapuram created TEZ-2915:
---

 Summary: Tez UI: Getting back to the DAG details page is difficult
 Key: TEZ-2915
 URL: https://issues.apache.org/jira/browse/TEZ-2915
 Project: Apache Tez
  Issue Type: Bug
Reporter: Sreenath Somarajapuram
Assignee: Sreenath Somarajapuram
 Fix For: 0.7.1, 0.8.2


An example while tracking a running DAG.
1. Go to the DAG page. (Shows the progress table).
2. Click on a specific vertex - goes to the vertex table.
3. Click the DAG link on the top left of the page. Expecting this to go back to 
the DAG page, Instead it goes to the All Vertices page - which has similar 
information to the All DAG page, but no task level info.
Getting back to the DAG progress page is difficult



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (TEZ-2915) Tez UI: Getting back to the DAG details page is difficult

2015-10-29 Thread Sreenath Somarajapuram (JIRA)

 [ 
https://issues.apache.org/jira/browse/TEZ-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sreenath Somarajapuram updated TEZ-2915:

Attachment: TEZ-2915.1.patch

[~dbhowmick]
Please review the patch.

> Tez UI: Getting back to the DAG details page is difficult
> -
>
> Key: TEZ-2915
> URL: https://issues.apache.org/jira/browse/TEZ-2915
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Fix For: 0.7.1, 0.8.2
>
> Attachments: TEZ-2915.1.patch
>
>
> On clicking breadcrumb, go to the details page than the tables.
> An example while tracking a running DAG.
> 1. Go to the DAG page. (Shows the progress table).
> 2. Click on a specific vertex - goes to the vertex table.
> 3. Click the DAG link on the top left of the page. Expecting this to go back 
> to the DAG page, Instead it goes to the All Vertices page - which has similar 
> information to the All DAG page, but no task level info.
> Getting back to the DAG progress page is difficult



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (TEZ-2346) TEZ-UI: Lazy load other info / counter data

2015-10-29 Thread Sreenath Somarajapuram (JIRA)

 [ 
https://issues.apache.org/jira/browse/TEZ-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sreenath Somarajapuram updated TEZ-2346:

Attachment: TEZ-2346.2.patch

[~jeagles]
- Attaching patch for lazy loading otherinfo & counters.
- Further more, the UI works smoother with Application ID & Queue columns 
disabled.

Please help in reviewing the patch.

> TEZ-UI: Lazy load other info / counter data
> ---
>
> Key: TEZ-2346
> URL: https://issues.apache.org/jira/browse/TEZ-2346
> Project: Apache Tez
>  Issue Type: Sub-task
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
>Priority: Critical
> Attachments: Screen-Shot-2015-04-21-at-1.56.28-AM.jpg, 
> TEZ-2346.1.patch, TEZ-2346.2.patch, TEZ-2346.wip.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (TEZ-2553) Tez UI: Tez UI Nits

2015-10-29 Thread Sreenath Somarajapuram (JIRA)

 [ 
https://issues.apache.org/jira/browse/TEZ-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sreenath Somarajapuram updated TEZ-2553:

Attachment: TEZ-2553.1.patch

[~dbhowmick]

Please review the patch.

> Tez UI: Tez UI Nits
> ---
>
> Key: TEZ-2553
> URL: https://issues.apache.org/jira/browse/TEZ-2553
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
>Priority: Minor
> Attachments: TEZ-2553.1.patch
>
>
> - Tooltips are not disappearing at times
> - Make search message readable
> - Remove Application Attempt Id row from Tez Application Description in app 
> details page.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.

2015-10-29 Thread Sreenath Somarajapuram (JIRA)

 [ 
https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sreenath Somarajapuram updated TEZ-2909:

Attachment: TEZ-2909.1.patch

[~dbhowmick]

Please review the patch.

> Tez UI: Application link in All DAGs table is disable when applicationhistory 
> is unavailable.
> -
>
> Key: TEZ-2909
> URL: https://issues.apache.org/jira/browse/TEZ-2909
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Attachments: TEZ-2909.1.patch
>
>
> Application link in All DAGs table is disable when applicationhistory is 
> unavailable. It must be available when TEZ_APPLICATION is available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (TEZ-2915) Tez UI: Getting back to the DAG details page is difficult

2015-10-29 Thread Sreenath Somarajapuram (JIRA)

 [ 
https://issues.apache.org/jira/browse/TEZ-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sreenath Somarajapuram updated TEZ-2915:

Description: 
On clicking breadcrumb, go to the details page than the tables.

An example while tracking a running DAG.
1. Go to the DAG page. (Shows the progress table).
2. Click on a specific vertex - goes to the vertex table.
3. Click the DAG link on the top left of the page. Expecting this to go back to 
the DAG page, Instead it goes to the All Vertices page - which has similar 
information to the All DAG page, but no task level info.
Getting back to the DAG progress page is difficult

  was:
An example while tracking a running DAG.
1. Go to the DAG page. (Shows the progress table).
2. Click on a specific vertex - goes to the vertex table.
3. Click the DAG link on the top left of the page. Expecting this to go back to 
the DAG page, Instead it goes to the All Vertices page - which has similar 
information to the All DAG page, but no task level info.
Getting back to the DAG progress page is difficult


> Tez UI: Getting back to the DAG details page is difficult
> -
>
> Key: TEZ-2915
> URL: https://issues.apache.org/jira/browse/TEZ-2915
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Fix For: 0.7.1, 0.8.2
>
>
> On clicking breadcrumb, go to the details page than the tables.
> An example while tracking a running DAG.
> 1. Go to the DAG page. (Shows the progress table).
> 2. Click on a specific vertex - goes to the vertex table.
> 3. Click the DAG link on the top left of the page. Expecting this to go back 
> to the DAG page, Instead it goes to the All Vertices page - which has similar 
> information to the All DAG page, but no task level info.
> Getting back to the DAG progress page is difficult



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2346) TEZ-UI: Lazy load other info / counter data

2015-10-29 Thread TezQA (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980111#comment-14980111
 ] 

TezQA commented on TEZ-2346:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12769484/TEZ-2346.2.patch
  against master revision 65f4575.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 3.0.1) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The following test timeouts occurred in :
 org.apache.tez.test.TestFaultTolerance

Test results: 
https://builds.apache.org/job/PreCommit-TEZ-Build/1259//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1259//console

This message is automatically generated.

> TEZ-UI: Lazy load other info / counter data
> ---
>
> Key: TEZ-2346
> URL: https://issues.apache.org/jira/browse/TEZ-2346
> Project: Apache Tez
>  Issue Type: Sub-task
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
>Priority: Critical
> Attachments: Screen-Shot-2015-04-21-at-1.56.28-AM.jpg, 
> TEZ-2346.1.patch, TEZ-2346.2.patch, TEZ-2346.wip.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2581) Umbrella for Tez Recovery Redesign

2015-10-29 Thread Bikas Saha (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980142#comment-14980142
 ] 

Bikas Saha commented on TEZ-2581:
-

bq. Vertex/VertexGroup has completed committing. But its full recovery events 
are not seen. 
Why is that bad?

> Umbrella for Tez Recovery Redesign
> --
>
> Key: TEZ-2581
> URL: https://issues.apache.org/jira/browse/TEZ-2581
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Jeff Zhang
>Assignee: Jeff Zhang
> Attachments: TEZ-2581-WIP-1.patch, TEZ-2581-WIP-2.patch, 
> TEZ-2581-WIP-3.patch, TEZ-2581-WIP-4.patch, TEZ-2581-WIP-5.patch, 
> TEZ-2581-WIP-6.patch, TezRecoveryRedesignProposal.pdf, 
> TezRecoveryRedesignV1.1.pdf
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


Failed: TEZ-2346 PreCommit Build #1259

2015-10-29 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-2346
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1259/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 2911 lines...]
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :tez-tests




{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12769484/TEZ-2346.2.patch
  against master revision 65f4575.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 3.0.1) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The following test timeouts occurred in :
 org.apache.tez.test.TestFaultTolerance

Test results: 
https://builds.apache.org/job/PreCommit-TEZ-Build/1259//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1259//console

This message is automatically generated.


==
==
Adding comment to Jira.
==
==


Comment added.
352fd47b5d84fd979c218e13b5bb97c45dff2a95 logged out


==
==
Finished build.
==
==


Build step 'Execute shell' marked build as failure
Archiving artifacts
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
All tests passed

Failed: TEZ-2553 PreCommit Build #1260

2015-10-29 Thread Apache Jenkins Server
Jira: https://issues.apache.org/jira/browse/TEZ-2553
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1260/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 2858 lines...]
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :tez-dag




{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12769487/TEZ-2553.1.patch
  against master revision 65f4575.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 3.0.1) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in :
   org.apache.tez.dag.app.TestPreemption

Test results: 
https://builds.apache.org/job/PreCommit-TEZ-Build/1260//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1260//console

This message is automatically generated.


==
==
Adding comment to Jira.
==
==


Comment added.
9db5381c69e590c6bba574458a9a6618a28c8e22 logged out


==
==
Finished build.
==
==


Build step 'Execute shell' marked build as failure
Archiving artifacts
[description-setter] Could not determine description.
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



###
## FAILED TESTS (if any) 
##
1 tests failed.
FAILED:  org.apache.tez.dag.app.TestPreemption.testPreemptionWithSession

Error Message:
App master already running a DAG
 at 
org.apache.tez.dag.app.DAGAppMaster.submitDAGToAppMaster(DAGAppMaster.java:1284)
 at 
org.apache.tez.dag.api.client.DAGClientHandler.submitDAG(DAGClientHandler.java:120)
 at 
org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.submitDAG(DAGClientAMProtocolBlockingPBServerImpl.java:161)
 at 
org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7471)
 at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
 at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
 at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039)
 at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
 at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
 at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033)


Stack Trace:
org.apache.tez.dag.api.TezException: App master already running a DAG
at 
org.apache.tez.dag.app.DAGAppMaster.submitDAGToAppMaster(DAGAppMaster.java:1284)
at 
org.apache.tez.dag.api.client.DAGClientHandler.submitDAG(DAGClientHandler.java:120)
at 
org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.submitDAG(DAGClientAMProtocolBlockingPBServerImpl.java:161)
at 
org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7471)
at 
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)

[jira] [Commented] (TEZ-2581) Umbrella for Tez Recovery Redesign

2015-10-29 Thread Jeff Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980224#comment-14980224
 ] 

Jeff Zhang commented on TEZ-2581:
-

bq. Why rename to setParallelism2?
After removing parameter recovering,  these 2 methods' signature become same. 
The only difference is one Map type of sourceEdgeProperties, but the generic 
type info is erased for java compiler. 
bq. Why is RecoverTransition needed?
It is for recovering of completed DAG. Only recover DAG & Vertex, but not Task 
& TaskAttempt.

> Umbrella for Tez Recovery Redesign
> --
>
> Key: TEZ-2581
> URL: https://issues.apache.org/jira/browse/TEZ-2581
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Jeff Zhang
>Assignee: Jeff Zhang
> Attachments: TEZ-2581-WIP-1.patch, TEZ-2581-WIP-2.patch, 
> TEZ-2581-WIP-3.patch, TEZ-2581-WIP-4.patch, TEZ-2581-WIP-5.patch, 
> TEZ-2581-WIP-6.patch, TezRecoveryRedesignProposal.pdf, 
> TezRecoveryRedesignV1.1.pdf
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2581) Umbrella for Tez Recovery Redesign

2015-10-29 Thread Jeff Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980173#comment-14980173
 ] 

Jeff Zhang commented on TEZ-2581:
-

bq. Does the client transition to using ATS for completed DAGs? If yes, then 
perhaps we dont need this since the client will transition to using the ATS.
getDAGStatus & getVertexStatus is RPC call, it doesn't matter with ATS.
bq. In this same context, what happens to ATS data? 
Yes, this is known issue that there may be inconsistency between the ATS data 
and recovery data.  The processing recovery events and ATS events are now in 
different threads, currently have no good solution for that. 

> Umbrella for Tez Recovery Redesign
> --
>
> Key: TEZ-2581
> URL: https://issues.apache.org/jira/browse/TEZ-2581
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Jeff Zhang
>Assignee: Jeff Zhang
> Attachments: TEZ-2581-WIP-1.patch, TEZ-2581-WIP-2.patch, 
> TEZ-2581-WIP-3.patch, TEZ-2581-WIP-4.patch, TEZ-2581-WIP-5.patch, 
> TEZ-2581-WIP-6.patch, TezRecoveryRedesignProposal.pdf, 
> TezRecoveryRedesignV1.1.pdf
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2553) Tez UI: Tez UI Nits

2015-10-29 Thread TezQA (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980178#comment-14980178
 ] 

TezQA commented on TEZ-2553:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12769487/TEZ-2553.1.patch
  against master revision 65f4575.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 3.0.1) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 core tests{color}.  The patch failed these unit tests in :
   org.apache.tez.dag.app.TestPreemption

Test results: 
https://builds.apache.org/job/PreCommit-TEZ-Build/1260//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1260//console

This message is automatically generated.

> Tez UI: Tez UI Nits
> ---
>
> Key: TEZ-2553
> URL: https://issues.apache.org/jira/browse/TEZ-2553
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
>Priority: Minor
> Attachments: TEZ-2553.1.patch
>
>
> - Tooltips are not disappearing at times
> - Make search message readable
> - Remove Application Attempt Id row from Tez Application Description in app 
> details page.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2581) Umbrella for Tez Recovery Redesign

2015-10-29 Thread Jeff Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980181#comment-14980181
 ] 

Jeff Zhang commented on TEZ-2581:
-

bq. Can we check that the post recovery value of various times like 
startTime/launchTime/finishTime etc. remain at their pre recovery values?
I plan to add more tests in TestDAGRecovery to verify the new state machine 
transition.  Will put the time checking in these unit tests.

> Umbrella for Tez Recovery Redesign
> --
>
> Key: TEZ-2581
> URL: https://issues.apache.org/jira/browse/TEZ-2581
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Jeff Zhang
>Assignee: Jeff Zhang
> Attachments: TEZ-2581-WIP-1.patch, TEZ-2581-WIP-2.patch, 
> TEZ-2581-WIP-3.patch, TEZ-2581-WIP-4.patch, TEZ-2581-WIP-5.patch, 
> TEZ-2581-WIP-6.patch, TezRecoveryRedesignProposal.pdf, 
> TezRecoveryRedesignV1.1.pdf
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (TEZ-2581) Umbrella for Tez Recovery Redesign

2015-10-29 Thread Jeff Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980181#comment-14980181
 ] 

Jeff Zhang edited comment on TEZ-2581 at 10/29/15 10:01 AM:


bq. Can we check that the post recovery value of various times like 
startTime/launchTime/finishTime etc. remain at their pre recovery values?
I plan to add more tests to verify the new state machine transition for 
recovery.  Will put the time checking in these unit tests.


was (Author: zjffdu):
bq. Can we check that the post recovery value of various times like 
startTime/launchTime/finishTime etc. remain at their pre recovery values?
I plan to add more tests in TestDAGRecovery to verify the new state machine 
transition.  Will put the time checking in these unit tests.

> Umbrella for Tez Recovery Redesign
> --
>
> Key: TEZ-2581
> URL: https://issues.apache.org/jira/browse/TEZ-2581
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Jeff Zhang
>Assignee: Jeff Zhang
> Attachments: TEZ-2581-WIP-1.patch, TEZ-2581-WIP-2.patch, 
> TEZ-2581-WIP-3.patch, TEZ-2581-WIP-4.patch, TEZ-2581-WIP-5.patch, 
> TEZ-2581-WIP-6.patch, TezRecoveryRedesignProposal.pdf, 
> TezRecoveryRedesignV1.1.pdf
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Comment Edited] (TEZ-2581) Umbrella for Tez Recovery Redesign

2015-10-29 Thread Jeff Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980181#comment-14980181
 ] 

Jeff Zhang edited comment on TEZ-2581 at 10/29/15 10:04 AM:


bq. TezEventUpdaterTransition - we could accumulate events and then write them 
upon finish once - any downsides to that?
The only concern is memory consumption. But after second thought, we will 
always keep the TezEvents in memory for downstream consumption, so I think 
accumulating them should be OK.

bq. Can we check that the post recovery value of various times like 
startTime/launchTime/finishTime etc. remain at their pre recovery values?
I plan to add more tests to verify the new state machine transition for 
recovery.  Will put the time checking in these unit tests.


was (Author: zjffdu):
bq. Can we check that the post recovery value of various times like 
startTime/launchTime/finishTime etc. remain at their pre recovery values?
I plan to add more tests to verify the new state machine transition for 
recovery.  Will put the time checking in these unit tests.

> Umbrella for Tez Recovery Redesign
> --
>
> Key: TEZ-2581
> URL: https://issues.apache.org/jira/browse/TEZ-2581
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Jeff Zhang
>Assignee: Jeff Zhang
> Attachments: TEZ-2581-WIP-1.patch, TEZ-2581-WIP-2.patch, 
> TEZ-2581-WIP-3.patch, TEZ-2581-WIP-4.patch, TEZ-2581-WIP-5.patch, 
> TEZ-2581-WIP-6.patch, TezRecoveryRedesignProposal.pdf, 
> TezRecoveryRedesignV1.1.pdf
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2581) Umbrella for Tez Recovery Redesign

2015-10-29 Thread Jeff Zhang (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980261#comment-14980261
 ] 

Jeff Zhang commented on TEZ-2581:
-

bq. Is the ordering of DM events and completion events via serializing through 
the Vertex not needed anymore? Does the same ordering apply to the event being 
sent to the TA -> DM events must be sent before completion events? 
Yes, here's DM events must be sent before TA completion event. Will add comment 
for that. Besides, the DM events will also be sent to Vertex directly for 
performance consideration. DM events can be sent to TA and TA send them to 
Vertex for downstream consumption, but may lose performance because we route 
the event in dispatcher twice. Will add more comments to explain that. 

bq. Event serde code should be in Event or some common place (e.g. 
ProtoConverters) - not in RecoverConverters where others will not try to search 
for.
This is due to dependency issue. To achieve that, I need to move EventMetaData 
from tez-runtime-internal to tez-api 

> Umbrella for Tez Recovery Redesign
> --
>
> Key: TEZ-2581
> URL: https://issues.apache.org/jira/browse/TEZ-2581
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Jeff Zhang
>Assignee: Jeff Zhang
> Attachments: TEZ-2581-WIP-1.patch, TEZ-2581-WIP-2.patch, 
> TEZ-2581-WIP-3.patch, TEZ-2581-WIP-4.patch, TEZ-2581-WIP-5.patch, 
> TEZ-2581-WIP-6.patch, TezRecoveryRedesignProposal.pdf, 
> TezRecoveryRedesignV1.1.pdf
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-808) Handle task attempts that are not making progress

2015-10-29 Thread Bikas Saha (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981818#comment-14981818
 ] 

Bikas Saha commented on TEZ-808:


Thanks! I will change that and also update the javadoc advisory on calling this 
infrequently and commit.

> Handle task attempts that are not making progress
> -
>
> Key: TEZ-808
> URL: https://issues.apache.org/jira/browse/TEZ-808
> Project: Apache Tez
>  Issue Type: Sub-task
>Reporter: Bikas Saha
>Assignee: Bikas Saha
> Attachments: TEZ-808.1.patch, TEZ-808.2.patch
>
>
> If a task attempt is not making progress then it may cause the job to hang. 
> We may want to kill and restart the attempt. With speculation support and 
> free resources we may want to run another version in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (TEZ-808) Handle task attempts that are not making progress

2015-10-29 Thread Bikas Saha (JIRA)

 [ 
https://issues.apache.org/jira/browse/TEZ-808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bikas Saha updated TEZ-808:
---
Attachment: TEZ-808.3.patch

Attaching commit patch. Changed config name to be shorter.

> Handle task attempts that are not making progress
> -
>
> Key: TEZ-808
> URL: https://issues.apache.org/jira/browse/TEZ-808
> Project: Apache Tez
>  Issue Type: Sub-task
>Reporter: Bikas Saha
>Assignee: Bikas Saha
> Attachments: TEZ-808.1.patch, TEZ-808.2.patch, TEZ-808.3.patch
>
>
> If a task attempt is not making progress then it may cause the job to hang. 
> We may want to kill and restart the attempt. With speculation support and 
> free resources we may want to run another version in parallel.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.

2015-10-29 Thread Hitesh Shah (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981787#comment-14981787
 ] 

Hitesh Shah commented on TEZ-2909:
--

+1

> Tez UI: Application link in All DAGs table is disable when applicationhistory 
> is unavailable.
> -
>
> Key: TEZ-2909
> URL: https://issues.apache.org/jira/browse/TEZ-2909
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sreenath Somarajapuram
>Assignee: Sreenath Somarajapuram
> Attachments: TEZ-2909.1.patch, TEZ-2909.2.patch
>
>
> Application link in All DAGs table is disable when applicationhistory is 
> unavailable. It must be available when TEZ_APPLICATION is available.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (TEZ-2917) change some logs from info to debug

2015-10-29 Thread TezQA (JIRA)

[ 
https://issues.apache.org/jira/browse/TEZ-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981689#comment-14981689
 ] 

TezQA commented on TEZ-2917:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12769651/TEZ-2917.patch
  against master revision 9ca6947.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 3.0.1) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: 
https://builds.apache.org/job/PreCommit-TEZ-Build/1265//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1265//console

This message is automatically generated.

> change some logs from info to debug
> ---
>
> Key: TEZ-2917
> URL: https://issues.apache.org/jira/browse/TEZ-2917
> Project: Apache Tez
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: TEZ-2917.patch
>
>
> I've done a highly unscientific summarization of the logs from some random 
> queries, and will now change some log statements that are the most prevalent 
> and not extremely useful from info to debug.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)