[jira] [Comment Edited] (TEZ-2814) ATSImportTool has a return statement in a finally block
[ https://issues.apache.org/jira/browse/TEZ-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980686#comment-14980686 ] Rajesh Balamohan edited comment on TEZ-2814 at 10/29/15 4:06 PM: - [~hitesh] - Please review when you find time. was (Author: rajesh.balamohan): [~hitesh] - Please review when you find sometime. > ATSImportTool has a return statement in a finally block > > > Key: TEZ-2814 > URL: https://issues.apache.org/jira/browse/TEZ-2814 > Project: Apache Tez > Issue Type: Bug >Reporter: Hitesh Shah >Assignee: Rajesh Balamohan > Attachments: TEZ-2814.1.patch > > > {code} > } catch (Exception e) { > LOG.error("Error in processing ", e); > throw e; > } finally { > return result; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2679) Admin forms of launch env settings
[ https://issues.apache.org/jira/browse/TEZ-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980626#comment-14980626 ] Hitesh Shah commented on TEZ-2679: -- +1. Please mark this as incompatible in the changelog. Might be useful to add a release note to this jira too describing the behavior change. > Admin forms of launch env settings > -- > > Key: TEZ-2679 > URL: https://issues.apache.org/jira/browse/TEZ-2679 > Project: Apache Tez > Issue Type: Improvement >Reporter: Jason Lowe >Assignee: Jonathan Eagles > Attachments: TEZ-2679.1.patch, TEZ-2679.2.patch, TEZ-2679.3.patch, > TEZ-2679.4.patch > > > Tez should provide corresponding admin configs for launch env and java opt > settings that can be specified in tez-site.xml by cluster admins. For > example, if users override the launch env settings to setup LD_LIBRARY_PATH > to just reference their libs then we can fail to load the native Hadoop libs. > This is currently mitigated by having Tez rely on the MapReduce admin > properties in many cases, but Tez should provide its own admin settings to > match the Tez-specific non-admin settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2814) ATSImportTool has a return statement in a finally block
[ https://issues.apache.org/jira/browse/TEZ-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980716#comment-14980716 ] Hitesh Shah commented on TEZ-2814: -- +1 pending pre-commit > ATSImportTool has a return statement in a finally block > > > Key: TEZ-2814 > URL: https://issues.apache.org/jira/browse/TEZ-2814 > Project: Apache Tez > Issue Type: Bug >Reporter: Hitesh Shah >Assignee: Rajesh Balamohan > Attachments: TEZ-2814.1.patch > > > {code} > } catch (Exception e) { > LOG.error("Error in processing ", e); > throw e; > } finally { > return result; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2679) Admin forms of launch env settings
[ https://issues.apache.org/jira/browse/TEZ-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980517#comment-14980517 ] Jonathan Eagles commented on TEZ-2679: -- [~hitesh], can you give a final review due? Specifically looking at the behavior change? > Admin forms of launch env settings > -- > > Key: TEZ-2679 > URL: https://issues.apache.org/jira/browse/TEZ-2679 > Project: Apache Tez > Issue Type: Improvement >Reporter: Jason Lowe >Assignee: Jonathan Eagles > Attachments: TEZ-2679.1.patch, TEZ-2679.2.patch, TEZ-2679.3.patch, > TEZ-2679.4.patch > > > Tez should provide corresponding admin configs for launch env and java opt > settings that can be specified in tez-site.xml by cluster admins. For > example, if users override the launch env settings to setup LD_LIBRARY_PATH > to just reference their libs then we can fail to load the native Hadoop libs. > This is currently mitigated by having Tez rely on the MapReduce admin > properties in many cases, but Tez should provide its own admin settings to > match the Tez-specific non-admin settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.
[ https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980598#comment-14980598 ] Hitesh Shah edited comment on TEZ-2909 at 10/29/15 3:14 PM: Does the applicationId from allDags link to the application details page within the Tez UI or does it link directly to the YARN ResourceManager page for that application? If it links to the application details page, then we should always create a link regardless of whether TEZ_APPLICATION exists or not. Now on the application details page, depending on whether we have data or not, we can show the data if possible or show unavailable but that should not drive the link logic on the AllDAGs page. was (Author: hitesh): Does the applicationId from allDags link to the application details page or does it link directly to the ResourceManager page? If it links to the application details page, then we should always create a link regardless of whether TEZ_APPLICATION exists or not. Now on the application details page, depending on whether we have data or not, we can show the data if possible or show unavailable but that should not drive the link logic on the AllDAGs page. > Tez UI: Application link in All DAGs table is disable when applicationhistory > is unavailable. > - > > Key: TEZ-2909 > URL: https://issues.apache.org/jira/browse/TEZ-2909 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2909.1.patch > > > Application link in All DAGs table is disable when applicationhistory is > unavailable. It must be available when TEZ_APPLICATION is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.
[ https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980598#comment-14980598 ] Hitesh Shah commented on TEZ-2909: -- Does the applicationId from allDags link to the application details page or does it link directly to the ResourceManager page? > Tez UI: Application link in All DAGs table is disable when applicationhistory > is unavailable. > - > > Key: TEZ-2909 > URL: https://issues.apache.org/jira/browse/TEZ-2909 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2909.1.patch > > > Application link in All DAGs table is disable when applicationhistory is > unavailable. It must be available when TEZ_APPLICATION is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2553) Tez UI: Tez UI Nits
[ https://issues.apache.org/jira/browse/TEZ-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980605#comment-14980605 ] Hitesh Shah commented on TEZ-2553: -- "Substring search ..." is not really more readable. The user does not get clarity on how the search box is meant to be used. Maybe just change to a simple "Search ..." for now. > Tez UI: Tez UI Nits > --- > > Key: TEZ-2553 > URL: https://issues.apache.org/jira/browse/TEZ-2553 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram >Priority: Minor > Attachments: TEZ-2553.1.patch > > > - Tooltips are not disappearing at times > - Make search message readable > - Remove Application Attempt Id row from Tez Application Description in app > details page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-2916) Show counts of running tasks on the DAG visualization page
Siddharth Seth created TEZ-2916: --- Summary: Show counts of running tasks on the DAG visualization page Key: TEZ-2916 URL: https://issues.apache.org/jira/browse/TEZ-2916 Project: Apache Tez Issue Type: Improvement Reporter: Siddharth Seth While a DAG is running, it's useful to have task counts for running / total / complete tasks on the DAG visualization page. This can be used to figure out dependencies, and which tasks/vertices are blocking others. As part of this - some indication of which vertices are currently running will be a useful addition. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.
[ https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980707#comment-14980707 ] TezQA commented on TEZ-2909: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12769489/TEZ-2909.1.patch against master revision 65f4575. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 3.0.1) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/1261//testReport/ Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1261//console This message is automatically generated. > Tez UI: Application link in All DAGs table is disable when applicationhistory > is unavailable. > - > > Key: TEZ-2909 > URL: https://issues.apache.org/jira/browse/TEZ-2909 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2909.1.patch > > > Application link in All DAGs table is disable when applicationhistory is > unavailable. It must be available when TEZ_APPLICATION is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Failed: TEZ-2909 PreCommit Build #1261
Jira: https://issues.apache.org/jira/browse/TEZ-2909 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1261/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 3430 lines...] [INFO] Total time: 42:34 min [INFO] Finished at: 2015-10-29T16:17:12+00:00 [INFO] Final Memory: 86M/921M [INFO] {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12769489/TEZ-2909.1.patch against master revision 65f4575. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 3.0.1) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/1261//testReport/ Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1261//console This message is automatically generated. == == Adding comment to Jira. == == Comment added. dffe4daa81d333d6365bf64ffca28cfe70369dcf logged out == == Finished build. == == Build step 'Execute shell' marked build as failure Archiving artifacts Compressed 3.02 MB of artifacts by 26.9% relative to #1258 [description-setter] Could not determine description. Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (TEZ-2553) Tez UI: Tez UI Nits
[ https://issues.apache.org/jira/browse/TEZ-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980607#comment-14980607 ] Hitesh Shah commented on TEZ-2553: -- +1 to commit after the search message is changed. > Tez UI: Tez UI Nits > --- > > Key: TEZ-2553 > URL: https://issues.apache.org/jira/browse/TEZ-2553 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram >Priority: Minor > Attachments: TEZ-2553.1.patch > > > - Tooltips are not disappearing at times > - Make search message readable > - Remove Application Attempt Id row from Tez Application Description in app > details page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (TEZ-2814) ATSImportTool has a return statement in a finally block
[ https://issues.apache.org/jira/browse/TEZ-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-2814: - Assignee: Rajesh Balamohan > ATSImportTool has a return statement in a finally block > > > Key: TEZ-2814 > URL: https://issues.apache.org/jira/browse/TEZ-2814 > Project: Apache Tez > Issue Type: Bug >Reporter: Hitesh Shah >Assignee: Rajesh Balamohan > Attachments: TEZ-2814.1.patch > > > {code} > } catch (Exception e) { > LOG.error("Error in processing ", e); > throw e; > } finally { > return result; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.
[ https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980598#comment-14980598 ] Hitesh Shah edited comment on TEZ-2909 at 10/29/15 3:14 PM: Does the applicationId from allDags link to the application details page or does it link directly to the ResourceManager page? If it links to the application details page, then we should always create a link regardless of whether TEZ_APPLICATION exists or not. Now on the application details page, depending on whether we have data or not, we can show the data if possible or show unavailable but that should not drive the link logic on the AllDAGs page. was (Author: hitesh): Does the applicationId from allDags link to the application details page or does it link directly to the ResourceManager page? > Tez UI: Application link in All DAGs table is disable when applicationhistory > is unavailable. > - > > Key: TEZ-2909 > URL: https://issues.apache.org/jira/browse/TEZ-2909 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2909.1.patch > > > Application link in All DAGs table is disable when applicationhistory is > unavailable. It must be available when TEZ_APPLICATION is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Success: TEZ-2814 PreCommit Build #1262
Jira: https://issues.apache.org/jira/browse/TEZ-2814 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1262/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 3424 lines...] [INFO] Tez ... SUCCESS [ 0.026 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 42:19 min [INFO] Finished at: 2015-10-29T17:07:36+00:00 [INFO] Final Memory: 81M/912M [INFO] {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12769555/TEZ-2814.1.patch against master revision 65f4575. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 3.0.1) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/1262//testReport/ Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1262//console This message is automatically generated. == == Adding comment to Jira. == == Comment added. 1b5d0012f848d85e29a39957e10ec2d7907b9bd5 logged out == == Finished build. == == Archiving artifacts [description-setter] Description set: TEZ-2814 Recording test results Email was triggered for: Success Sending email for trigger: Success ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (TEZ-2679) Admin forms of launch env settings
[ https://issues.apache.org/jira/browse/TEZ-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980908#comment-14980908 ] Hitesh Shah commented on TEZ-2679: -- I think we may need to file an INFRA jira for that. Would you mind filing that? FWIW, [~jeagles] you are a Tez JIRA admin too so both of us have the same level of permissions. In any case, a more descriptive message in CHANGES.txt should suffice for now. > Admin forms of launch env settings > -- > > Key: TEZ-2679 > URL: https://issues.apache.org/jira/browse/TEZ-2679 > Project: Apache Tez > Issue Type: Improvement >Reporter: Jason Lowe >Assignee: Jonathan Eagles > Fix For: 0.7.1, 0.8.2 > > Attachments: TEZ-2679.1.patch, TEZ-2679.2.patch, TEZ-2679.3.patch, > TEZ-2679.4.patch > > > Tez should provide corresponding admin configs for launch env and java opt > settings that can be specified in tez-site.xml by cluster admins. For > example, if users override the launch env settings to setup LD_LIBRARY_PATH > to just reference their libs then we can fail to load the native Hadoop libs. > This is currently mitigated by having Tez rely on the MapReduce admin > properties in many cases, but Tez should provide its own admin settings to > match the Tez-specific non-admin settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.
[ https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980793#comment-14980793 ] Sreenath Somarajapuram commented on TEZ-2909: - Thanks [~hitesh] Will make the required changes, also that would improve the All DAGs, page UX. > Tez UI: Application link in All DAGs table is disable when applicationhistory > is unavailable. > - > > Key: TEZ-2909 > URL: https://issues.apache.org/jira/browse/TEZ-2909 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2909.1.patch > > > Application link in All DAGs table is disable when applicationhistory is > unavailable. It must be available when TEZ_APPLICATION is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2814) ATSImportTool has a return statement in a finally block
[ https://issues.apache.org/jira/browse/TEZ-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980814#comment-14980814 ] TezQA commented on TEZ-2814: {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12769555/TEZ-2814.1.patch against master revision 65f4575. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 3.0.1) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/1262//testReport/ Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1262//console This message is automatically generated. > ATSImportTool has a return statement in a finally block > > > Key: TEZ-2814 > URL: https://issues.apache.org/jira/browse/TEZ-2814 > Project: Apache Tez > Issue Type: Bug >Reporter: Hitesh Shah >Assignee: Rajesh Balamohan > Attachments: TEZ-2814.1.patch > > > {code} > } catch (Exception e) { > LOG.error("Error in processing ", e); > throw e; > } finally { > return result; > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2679) Admin forms of launch env settings
[ https://issues.apache.org/jira/browse/TEZ-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980747#comment-14980747 ] Jonathan Eagles commented on TEZ-2679: -- Committed to master and branch-0.7. [~hitesh], the Tez project is not configured to have the _Release note_ field. This seems like a useful feature to add to the project. Can you enable this if you have the permissions to add this field to the project? > Admin forms of launch env settings > -- > > Key: TEZ-2679 > URL: https://issues.apache.org/jira/browse/TEZ-2679 > Project: Apache Tez > Issue Type: Improvement >Reporter: Jason Lowe >Assignee: Jonathan Eagles > Fix For: 0.7.1, 0.8.2 > > Attachments: TEZ-2679.1.patch, TEZ-2679.2.patch, TEZ-2679.3.patch, > TEZ-2679.4.patch > > > Tez should provide corresponding admin configs for launch env and java opt > settings that can be specified in tez-site.xml by cluster admins. For > example, if users override the launch env settings to setup LD_LIBRARY_PATH > to just reference their libs then we can fail to load the native Hadoop libs. > This is currently mitigated by having Tez rely on the MapReduce admin > properties in many cases, but Tez should provide its own admin settings to > match the Tez-specific non-admin settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.
[ https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2909: Attachment: TEZ-2909.2.patch Attaching a fresh patch with the changes advised by Hitesh. The link is enabled by default, and if TEZ_APPLICATION fetch fails an error bar will be displayed as expected. > Tez UI: Application link in All DAGs table is disable when applicationhistory > is unavailable. > - > > Key: TEZ-2909 > URL: https://issues.apache.org/jira/browse/TEZ-2909 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2909.1.patch, TEZ-2909.2.patch > > > Application link in All DAGs table is disable when applicationhistory is > unavailable. It must be available when TEZ_APPLICATION is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2553) Tez UI: Tez UI Nits
[ https://issues.apache.org/jira/browse/TEZ-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2553: Attachment: TEZ-2553.2.patch Attaching patch with the text change. Committing shortly. > Tez UI: Tez UI Nits > --- > > Key: TEZ-2553 > URL: https://issues.apache.org/jira/browse/TEZ-2553 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram >Priority: Minor > Fix For: 0.7.1, 0.8.2 > > Attachments: TEZ-2553.1.patch, TEZ-2553.2.patch > > > - Tooltips are not disappearing at times > - Make search message readable > - Remove Application Attempt Id row from Tez Application Description in app > details page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.
[ https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981035#comment-14981035 ] Sreenath Somarajapuram edited comment on TEZ-2909 at 10/29/15 7:05 PM: --- bq. Does the applicationId from allDags link to the application details page within the Tez UI or does it link directly to the YARN ResourceManager page for that application? ApplicationId from allDags link to the application details page Attaching a fresh patch with the changes advised by Hitesh. The link is enabled by default, and if TEZ_APPLICATION fetch fails an error bar will be displayed as expected. was (Author: sreenath): Attaching a fresh patch with the changes advised by Hitesh. The link is enabled by default, and if TEZ_APPLICATION fetch fails an error bar will be displayed as expected. > Tez UI: Application link in All DAGs table is disable when applicationhistory > is unavailable. > - > > Key: TEZ-2909 > URL: https://issues.apache.org/jira/browse/TEZ-2909 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2909.1.patch, TEZ-2909.2.patch > > > Application link in All DAGs table is disable when applicationhistory is > unavailable. It must be available when TEZ_APPLICATION is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2679) Admin forms of launch env settings
[ https://issues.apache.org/jira/browse/TEZ-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981037#comment-14981037 ] Jonathan Eagles commented on TEZ-2679: -- Filed INFRA-10684 to add Release notes to TEZ project expect time before fix is about a week. > Admin forms of launch env settings > -- > > Key: TEZ-2679 > URL: https://issues.apache.org/jira/browse/TEZ-2679 > Project: Apache Tez > Issue Type: Improvement >Reporter: Jason Lowe >Assignee: Jonathan Eagles > Fix For: 0.7.1, 0.8.2 > > Attachments: TEZ-2679.1.patch, TEZ-2679.2.patch, TEZ-2679.3.patch, > TEZ-2679.4.patch > > > Tez should provide corresponding admin configs for launch env and java opt > settings that can be specified in tez-site.xml by cluster admins. For > example, if users override the launch env settings to setup LD_LIBRARY_PATH > to just reference their libs then we can fail to load the native Hadoop libs. > This is currently mitigated by having Tez rely on the MapReduce admin > properties in many cases, but Tez should provide its own admin settings to > match the Tez-specific non-admin settings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2553) Tez UI: Tez UI Nits
[ https://issues.apache.org/jira/browse/TEZ-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2553: Fix Version/s: 0.8.2 0.7.1 > Tez UI: Tez UI Nits > --- > > Key: TEZ-2553 > URL: https://issues.apache.org/jira/browse/TEZ-2553 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram >Priority: Minor > Fix For: 0.7.1, 0.8.2 > > Attachments: TEZ-2553.1.patch, TEZ-2553.2.patch > > > - Tooltips are not disappearing at times > - Make search message readable > - Remove Application Attempt Id row from Tez Application Description in app > details page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-808) Handle task attempts that are not making progress
[ https://issues.apache.org/jira/browse/TEZ-808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981205#comment-14981205 ] Jason Lowe commented on TEZ-808: Thanks for updating the patch, Bikas! Patch still doesn't treat 0 as a disabled timeout but otherwise lgtm. > Handle task attempts that are not making progress > - > > Key: TEZ-808 > URL: https://issues.apache.org/jira/browse/TEZ-808 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Bikas Saha >Assignee: Bikas Saha > Attachments: TEZ-808.1.patch, TEZ-808.2.patch > > > If a task attempt is not making progress then it may cause the job to hang. > We may want to kill and restart the attempt. With speculation support and > free resources we may want to run another version in parallel. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Failed: TEZ-2909 PreCommit Build #1264
Jira: https://issues.apache.org/jira/browse/TEZ-2909 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1264/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 3428 lines...] [INFO] Total time: 42:11 min [INFO] Finished at: 2015-10-29T22:28:11+00:00 [INFO] Final Memory: 94M/1144M [INFO] {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12769600/TEZ-2909.2.patch against master revision dcc5108. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 3.0.1) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/1264//testReport/ Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1264//console This message is automatically generated. == == Adding comment to Jira. == == Comment added. 4a20a661f8438475a101db422a067d654a4ffbb6 logged out == == Finished build. == == Build step 'Execute shell' marked build as failure Archiving artifacts Compressed 2.99 MB of artifacts by 29.3% relative to #1262 [description-setter] Could not determine description. Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (TEZ-2917) change some logs from info to debug
[ https://issues.apache.org/jira/browse/TEZ-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981456#comment-14981456 ] Sergey Shelukhin commented on TEZ-2917: --- [~sseth] can you take a look? > change some logs from info to debug > --- > > Key: TEZ-2917 > URL: https://issues.apache.org/jira/browse/TEZ-2917 > Project: Apache Tez > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: TEZ-2917.patch > > > I've done a highly unscientific summarization of the logs from some random > queries, and will now change some log statements that are the most prevalent > and not extremely useful from info to debug. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Moved] (TEZ-2917) change some logs from info to debug
[ https://issues.apache.org/jira/browse/TEZ-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin moved HIVE-12298 to TEZ-2917: -- Key: TEZ-2917 (was: HIVE-12298) Project: Apache Tez (was: Hive) > change some logs from info to debug > --- > > Key: TEZ-2917 > URL: https://issues.apache.org/jira/browse/TEZ-2917 > Project: Apache Tez > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > > I've done a highly unscientific summarization of the logs from some random > queries, and will now change some log statements that are the most prevalent > and not extremely useful from info to debug. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2917) change some logs from info to debug
[ https://issues.apache.org/jira/browse/TEZ-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981477#comment-14981477 ] Hitesh Shah commented on TEZ-2917: -- How much general savings in terms of log size does this provide on a decent-size query? > change some logs from info to debug > --- > > Key: TEZ-2917 > URL: https://issues.apache.org/jira/browse/TEZ-2917 > Project: Apache Tez > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: TEZ-2917.patch > > > I've done a highly unscientific summarization of the logs from some random > queries, and will now change some log statements that are the most prevalent > and not extremely useful from info to debug. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2917) change some logs from info to debug
[ https://issues.apache.org/jira/browse/TEZ-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981494#comment-14981494 ] Sergey Shelukhin commented on TEZ-2917: --- This is mostly coming from the complaint that logs are hard to read, not from size. > change some logs from info to debug > --- > > Key: TEZ-2917 > URL: https://issues.apache.org/jira/browse/TEZ-2917 > Project: Apache Tez > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: TEZ-2917.patch > > > I've done a highly unscientific summarization of the logs from some random > queries, and will now change some log statements that are the most prevalent > and not extremely useful from info to debug. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.
[ https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981448#comment-14981448 ] TezQA commented on TEZ-2909: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12769600/TEZ-2909.2.patch against master revision dcc5108. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 3.0.1) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/1264//testReport/ Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1264//console This message is automatically generated. > Tez UI: Application link in All DAGs table is disable when applicationhistory > is unavailable. > - > > Key: TEZ-2909 > URL: https://issues.apache.org/jira/browse/TEZ-2909 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2909.1.patch, TEZ-2909.2.patch > > > Application link in All DAGs table is disable when applicationhistory is > unavailable. It must be available when TEZ_APPLICATION is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2917) change some logs from info to debug
[ https://issues.apache.org/jira/browse/TEZ-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated TEZ-2917: -- Attachment: TEZ-2917.patch > change some logs from info to debug > --- > > Key: TEZ-2917 > URL: https://issues.apache.org/jira/browse/TEZ-2917 > Project: Apache Tez > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: TEZ-2917.patch > > > I've done a highly unscientific summarization of the logs from some random > queries, and will now change some log statements that are the most prevalent > and not extremely useful from info to debug. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2553) Tez UI: Tez UI Nits
[ https://issues.apache.org/jira/browse/TEZ-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981487#comment-14981487 ] TezQA commented on TEZ-2553: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12769601/TEZ-2553.2.patch against master revision dcc5108. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 3.0.1) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/1263//testReport/ Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1263//console This message is automatically generated. > Tez UI: Tez UI Nits > --- > > Key: TEZ-2553 > URL: https://issues.apache.org/jira/browse/TEZ-2553 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram >Priority: Minor > Fix For: 0.7.1, 0.8.2 > > Attachments: TEZ-2553.1.patch, TEZ-2553.2.patch > > > - Tooltips are not disappearing at times > - Make search message readable > - Remove Application Attempt Id row from Tez Application Description in app > details page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Failed: TEZ-2553 PreCommit Build #1263
Jira: https://issues.apache.org/jira/browse/TEZ-2553 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1263/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 3430 lines...] [INFO] [INFO] Total time: 50:50 min [INFO] Finished at: 2015-10-29T22:44:33+00:00 [INFO] Final Memory: 100M/1405M [INFO] {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12769601/TEZ-2553.2.patch against master revision dcc5108. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 3.0.1) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/1263//testReport/ Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1263//console This message is automatically generated. == == Adding comment to Jira. == == Comment added. 905a1e44efb897ae56451ed1183174271626f88b logged out == == Finished build. == == Build step 'Execute shell' marked build as failure Archiving artifacts [description-setter] Could not determine description. Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any ### ## FAILED TESTS (if any) ## All tests passed
[jira] [Commented] (TEZ-2917) change some logs from info to debug
[ https://issues.apache.org/jira/browse/TEZ-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981500#comment-14981500 ] Sergey Shelukhin commented on TEZ-2917: --- I ran some queries, grouped logs by first 30 characters with numbers removed, and went thru them manually for those that were present in relatively large numbers. So this is how these were removed. > change some logs from info to debug > --- > > Key: TEZ-2917 > URL: https://issues.apache.org/jira/browse/TEZ-2917 > Project: Apache Tez > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: TEZ-2917.patch > > > I've done a highly unscientific summarization of the logs from some random > queries, and will now change some log statements that are the most prevalent > and not extremely useful from info to debug. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (TEZ-2915) Tez UI: Getting back to the DAG details page is difficult
Sreenath Somarajapuram created TEZ-2915: --- Summary: Tez UI: Getting back to the DAG details page is difficult Key: TEZ-2915 URL: https://issues.apache.org/jira/browse/TEZ-2915 Project: Apache Tez Issue Type: Bug Reporter: Sreenath Somarajapuram Assignee: Sreenath Somarajapuram Fix For: 0.7.1, 0.8.2 An example while tracking a running DAG. 1. Go to the DAG page. (Shows the progress table). 2. Click on a specific vertex - goes to the vertex table. 3. Click the DAG link on the top left of the page. Expecting this to go back to the DAG page, Instead it goes to the All Vertices page - which has similar information to the All DAG page, but no task level info. Getting back to the DAG progress page is difficult -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2915) Tez UI: Getting back to the DAG details page is difficult
[ https://issues.apache.org/jira/browse/TEZ-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2915: Attachment: TEZ-2915.1.patch [~dbhowmick] Please review the patch. > Tez UI: Getting back to the DAG details page is difficult > - > > Key: TEZ-2915 > URL: https://issues.apache.org/jira/browse/TEZ-2915 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Fix For: 0.7.1, 0.8.2 > > Attachments: TEZ-2915.1.patch > > > On clicking breadcrumb, go to the details page than the tables. > An example while tracking a running DAG. > 1. Go to the DAG page. (Shows the progress table). > 2. Click on a specific vertex - goes to the vertex table. > 3. Click the DAG link on the top left of the page. Expecting this to go back > to the DAG page, Instead it goes to the All Vertices page - which has similar > information to the All DAG page, but no task level info. > Getting back to the DAG progress page is difficult -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2346) TEZ-UI: Lazy load other info / counter data
[ https://issues.apache.org/jira/browse/TEZ-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2346: Attachment: TEZ-2346.2.patch [~jeagles] - Attaching patch for lazy loading otherinfo & counters. - Further more, the UI works smoother with Application ID & Queue columns disabled. Please help in reviewing the patch. > TEZ-UI: Lazy load other info / counter data > --- > > Key: TEZ-2346 > URL: https://issues.apache.org/jira/browse/TEZ-2346 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram >Priority: Critical > Attachments: Screen-Shot-2015-04-21-at-1.56.28-AM.jpg, > TEZ-2346.1.patch, TEZ-2346.2.patch, TEZ-2346.wip.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2553) Tez UI: Tez UI Nits
[ https://issues.apache.org/jira/browse/TEZ-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2553: Attachment: TEZ-2553.1.patch [~dbhowmick] Please review the patch. > Tez UI: Tez UI Nits > --- > > Key: TEZ-2553 > URL: https://issues.apache.org/jira/browse/TEZ-2553 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram >Priority: Minor > Attachments: TEZ-2553.1.patch > > > - Tooltips are not disappearing at times > - Make search message readable > - Remove Application Attempt Id row from Tez Application Description in app > details page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.
[ https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2909: Attachment: TEZ-2909.1.patch [~dbhowmick] Please review the patch. > Tez UI: Application link in All DAGs table is disable when applicationhistory > is unavailable. > - > > Key: TEZ-2909 > URL: https://issues.apache.org/jira/browse/TEZ-2909 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2909.1.patch > > > Application link in All DAGs table is disable when applicationhistory is > unavailable. It must be available when TEZ_APPLICATION is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-2915) Tez UI: Getting back to the DAG details page is difficult
[ https://issues.apache.org/jira/browse/TEZ-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sreenath Somarajapuram updated TEZ-2915: Description: On clicking breadcrumb, go to the details page than the tables. An example while tracking a running DAG. 1. Go to the DAG page. (Shows the progress table). 2. Click on a specific vertex - goes to the vertex table. 3. Click the DAG link on the top left of the page. Expecting this to go back to the DAG page, Instead it goes to the All Vertices page - which has similar information to the All DAG page, but no task level info. Getting back to the DAG progress page is difficult was: An example while tracking a running DAG. 1. Go to the DAG page. (Shows the progress table). 2. Click on a specific vertex - goes to the vertex table. 3. Click the DAG link on the top left of the page. Expecting this to go back to the DAG page, Instead it goes to the All Vertices page - which has similar information to the All DAG page, but no task level info. Getting back to the DAG progress page is difficult > Tez UI: Getting back to the DAG details page is difficult > - > > Key: TEZ-2915 > URL: https://issues.apache.org/jira/browse/TEZ-2915 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Fix For: 0.7.1, 0.8.2 > > > On clicking breadcrumb, go to the details page than the tables. > An example while tracking a running DAG. > 1. Go to the DAG page. (Shows the progress table). > 2. Click on a specific vertex - goes to the vertex table. > 3. Click the DAG link on the top left of the page. Expecting this to go back > to the DAG page, Instead it goes to the All Vertices page - which has similar > information to the All DAG page, but no task level info. > Getting back to the DAG progress page is difficult -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2346) TEZ-UI: Lazy load other info / counter data
[ https://issues.apache.org/jira/browse/TEZ-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980111#comment-14980111 ] TezQA commented on TEZ-2346: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12769484/TEZ-2346.2.patch against master revision 65f4575. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 3.0.1) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The following test timeouts occurred in : org.apache.tez.test.TestFaultTolerance Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/1259//testReport/ Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1259//console This message is automatically generated. > TEZ-UI: Lazy load other info / counter data > --- > > Key: TEZ-2346 > URL: https://issues.apache.org/jira/browse/TEZ-2346 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram >Priority: Critical > Attachments: Screen-Shot-2015-04-21-at-1.56.28-AM.jpg, > TEZ-2346.1.patch, TEZ-2346.2.patch, TEZ-2346.wip.1.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2581) Umbrella for Tez Recovery Redesign
[ https://issues.apache.org/jira/browse/TEZ-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980142#comment-14980142 ] Bikas Saha commented on TEZ-2581: - bq. Vertex/VertexGroup has completed committing. But its full recovery events are not seen. Why is that bad? > Umbrella for Tez Recovery Redesign > -- > > Key: TEZ-2581 > URL: https://issues.apache.org/jira/browse/TEZ-2581 > Project: Apache Tez > Issue Type: Improvement >Reporter: Jeff Zhang >Assignee: Jeff Zhang > Attachments: TEZ-2581-WIP-1.patch, TEZ-2581-WIP-2.patch, > TEZ-2581-WIP-3.patch, TEZ-2581-WIP-4.patch, TEZ-2581-WIP-5.patch, > TEZ-2581-WIP-6.patch, TezRecoveryRedesignProposal.pdf, > TezRecoveryRedesignV1.1.pdf > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Failed: TEZ-2346 PreCommit Build #1259
Jira: https://issues.apache.org/jira/browse/TEZ-2346 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1259/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 2911 lines...] [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :tez-tests {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12769484/TEZ-2346.2.patch against master revision 65f4575. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 3.0.1) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The following test timeouts occurred in : org.apache.tez.test.TestFaultTolerance Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/1259//testReport/ Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1259//console This message is automatically generated. == == Adding comment to Jira. == == Comment added. 352fd47b5d84fd979c218e13b5bb97c45dff2a95 logged out == == Finished build. == == Build step 'Execute shell' marked build as failure Archiving artifacts [description-setter] Could not determine description. Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any ### ## FAILED TESTS (if any) ## All tests passed
Failed: TEZ-2553 PreCommit Build #1260
Jira: https://issues.apache.org/jira/browse/TEZ-2553 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/1260/ ### ## LAST 60 LINES OF THE CONSOLE ### [...truncated 2858 lines...] [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :tez-dag {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12769487/TEZ-2553.1.patch against master revision 65f4575. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 3.0.1) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in : org.apache.tez.dag.app.TestPreemption Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/1260//testReport/ Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1260//console This message is automatically generated. == == Adding comment to Jira. == == Comment added. 9db5381c69e590c6bba574458a9a6618a28c8e22 logged out == == Finished build. == == Build step 'Execute shell' marked build as failure Archiving artifacts [description-setter] Could not determine description. Recording test results Email was triggered for: Failure - Any Sending email for trigger: Failure - Any ### ## FAILED TESTS (if any) ## 1 tests failed. FAILED: org.apache.tez.dag.app.TestPreemption.testPreemptionWithSession Error Message: App master already running a DAG at org.apache.tez.dag.app.DAGAppMaster.submitDAGToAppMaster(DAGAppMaster.java:1284) at org.apache.tez.dag.api.client.DAGClientHandler.submitDAG(DAGClientHandler.java:120) at org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.submitDAG(DAGClientAMProtocolBlockingPBServerImpl.java:161) at org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7471) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033) Stack Trace: org.apache.tez.dag.api.TezException: App master already running a DAG at org.apache.tez.dag.app.DAGAppMaster.submitDAGToAppMaster(DAGAppMaster.java:1284) at org.apache.tez.dag.api.client.DAGClientHandler.submitDAG(DAGClientHandler.java:120) at org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolBlockingPBServerImpl.submitDAG(DAGClientAMProtocolBlockingPBServerImpl.java:161) at org.apache.tez.dag.api.client.rpc.DAGClientAMProtocolRPC$DAGClientAMProtocol$2.callBlockingMethod(DAGClientAMProtocolRPC.java:7471) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)
[jira] [Commented] (TEZ-2581) Umbrella for Tez Recovery Redesign
[ https://issues.apache.org/jira/browse/TEZ-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980224#comment-14980224 ] Jeff Zhang commented on TEZ-2581: - bq. Why rename to setParallelism2? After removing parameter recovering, these 2 methods' signature become same. The only difference is one Map type of sourceEdgeProperties, but the generic type info is erased for java compiler. bq. Why is RecoverTransition needed? It is for recovering of completed DAG. Only recover DAG & Vertex, but not Task & TaskAttempt. > Umbrella for Tez Recovery Redesign > -- > > Key: TEZ-2581 > URL: https://issues.apache.org/jira/browse/TEZ-2581 > Project: Apache Tez > Issue Type: Improvement >Reporter: Jeff Zhang >Assignee: Jeff Zhang > Attachments: TEZ-2581-WIP-1.patch, TEZ-2581-WIP-2.patch, > TEZ-2581-WIP-3.patch, TEZ-2581-WIP-4.patch, TEZ-2581-WIP-5.patch, > TEZ-2581-WIP-6.patch, TezRecoveryRedesignProposal.pdf, > TezRecoveryRedesignV1.1.pdf > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2581) Umbrella for Tez Recovery Redesign
[ https://issues.apache.org/jira/browse/TEZ-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980173#comment-14980173 ] Jeff Zhang commented on TEZ-2581: - bq. Does the client transition to using ATS for completed DAGs? If yes, then perhaps we dont need this since the client will transition to using the ATS. getDAGStatus & getVertexStatus is RPC call, it doesn't matter with ATS. bq. In this same context, what happens to ATS data? Yes, this is known issue that there may be inconsistency between the ATS data and recovery data. The processing recovery events and ATS events are now in different threads, currently have no good solution for that. > Umbrella for Tez Recovery Redesign > -- > > Key: TEZ-2581 > URL: https://issues.apache.org/jira/browse/TEZ-2581 > Project: Apache Tez > Issue Type: Improvement >Reporter: Jeff Zhang >Assignee: Jeff Zhang > Attachments: TEZ-2581-WIP-1.patch, TEZ-2581-WIP-2.patch, > TEZ-2581-WIP-3.patch, TEZ-2581-WIP-4.patch, TEZ-2581-WIP-5.patch, > TEZ-2581-WIP-6.patch, TezRecoveryRedesignProposal.pdf, > TezRecoveryRedesignV1.1.pdf > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2553) Tez UI: Tez UI Nits
[ https://issues.apache.org/jira/browse/TEZ-2553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980178#comment-14980178 ] TezQA commented on TEZ-2553: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12769487/TEZ-2553.1.patch against master revision 65f4575. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 3.0.1) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:red}-1 core tests{color}. The patch failed these unit tests in : org.apache.tez.dag.app.TestPreemption Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/1260//testReport/ Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1260//console This message is automatically generated. > Tez UI: Tez UI Nits > --- > > Key: TEZ-2553 > URL: https://issues.apache.org/jira/browse/TEZ-2553 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram >Priority: Minor > Attachments: TEZ-2553.1.patch > > > - Tooltips are not disappearing at times > - Make search message readable > - Remove Application Attempt Id row from Tez Application Description in app > details page. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2581) Umbrella for Tez Recovery Redesign
[ https://issues.apache.org/jira/browse/TEZ-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980181#comment-14980181 ] Jeff Zhang commented on TEZ-2581: - bq. Can we check that the post recovery value of various times like startTime/launchTime/finishTime etc. remain at their pre recovery values? I plan to add more tests in TestDAGRecovery to verify the new state machine transition. Will put the time checking in these unit tests. > Umbrella for Tez Recovery Redesign > -- > > Key: TEZ-2581 > URL: https://issues.apache.org/jira/browse/TEZ-2581 > Project: Apache Tez > Issue Type: Improvement >Reporter: Jeff Zhang >Assignee: Jeff Zhang > Attachments: TEZ-2581-WIP-1.patch, TEZ-2581-WIP-2.patch, > TEZ-2581-WIP-3.patch, TEZ-2581-WIP-4.patch, TEZ-2581-WIP-5.patch, > TEZ-2581-WIP-6.patch, TezRecoveryRedesignProposal.pdf, > TezRecoveryRedesignV1.1.pdf > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TEZ-2581) Umbrella for Tez Recovery Redesign
[ https://issues.apache.org/jira/browse/TEZ-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980181#comment-14980181 ] Jeff Zhang edited comment on TEZ-2581 at 10/29/15 10:01 AM: bq. Can we check that the post recovery value of various times like startTime/launchTime/finishTime etc. remain at their pre recovery values? I plan to add more tests to verify the new state machine transition for recovery. Will put the time checking in these unit tests. was (Author: zjffdu): bq. Can we check that the post recovery value of various times like startTime/launchTime/finishTime etc. remain at their pre recovery values? I plan to add more tests in TestDAGRecovery to verify the new state machine transition. Will put the time checking in these unit tests. > Umbrella for Tez Recovery Redesign > -- > > Key: TEZ-2581 > URL: https://issues.apache.org/jira/browse/TEZ-2581 > Project: Apache Tez > Issue Type: Improvement >Reporter: Jeff Zhang >Assignee: Jeff Zhang > Attachments: TEZ-2581-WIP-1.patch, TEZ-2581-WIP-2.patch, > TEZ-2581-WIP-3.patch, TEZ-2581-WIP-4.patch, TEZ-2581-WIP-5.patch, > TEZ-2581-WIP-6.patch, TezRecoveryRedesignProposal.pdf, > TezRecoveryRedesignV1.1.pdf > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (TEZ-2581) Umbrella for Tez Recovery Redesign
[ https://issues.apache.org/jira/browse/TEZ-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980181#comment-14980181 ] Jeff Zhang edited comment on TEZ-2581 at 10/29/15 10:04 AM: bq. TezEventUpdaterTransition - we could accumulate events and then write them upon finish once - any downsides to that? The only concern is memory consumption. But after second thought, we will always keep the TezEvents in memory for downstream consumption, so I think accumulating them should be OK. bq. Can we check that the post recovery value of various times like startTime/launchTime/finishTime etc. remain at their pre recovery values? I plan to add more tests to verify the new state machine transition for recovery. Will put the time checking in these unit tests. was (Author: zjffdu): bq. Can we check that the post recovery value of various times like startTime/launchTime/finishTime etc. remain at their pre recovery values? I plan to add more tests to verify the new state machine transition for recovery. Will put the time checking in these unit tests. > Umbrella for Tez Recovery Redesign > -- > > Key: TEZ-2581 > URL: https://issues.apache.org/jira/browse/TEZ-2581 > Project: Apache Tez > Issue Type: Improvement >Reporter: Jeff Zhang >Assignee: Jeff Zhang > Attachments: TEZ-2581-WIP-1.patch, TEZ-2581-WIP-2.patch, > TEZ-2581-WIP-3.patch, TEZ-2581-WIP-4.patch, TEZ-2581-WIP-5.patch, > TEZ-2581-WIP-6.patch, TezRecoveryRedesignProposal.pdf, > TezRecoveryRedesignV1.1.pdf > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2581) Umbrella for Tez Recovery Redesign
[ https://issues.apache.org/jira/browse/TEZ-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980261#comment-14980261 ] Jeff Zhang commented on TEZ-2581: - bq. Is the ordering of DM events and completion events via serializing through the Vertex not needed anymore? Does the same ordering apply to the event being sent to the TA -> DM events must be sent before completion events? Yes, here's DM events must be sent before TA completion event. Will add comment for that. Besides, the DM events will also be sent to Vertex directly for performance consideration. DM events can be sent to TA and TA send them to Vertex for downstream consumption, but may lose performance because we route the event in dispatcher twice. Will add more comments to explain that. bq. Event serde code should be in Event or some common place (e.g. ProtoConverters) - not in RecoverConverters where others will not try to search for. This is due to dependency issue. To achieve that, I need to move EventMetaData from tez-runtime-internal to tez-api > Umbrella for Tez Recovery Redesign > -- > > Key: TEZ-2581 > URL: https://issues.apache.org/jira/browse/TEZ-2581 > Project: Apache Tez > Issue Type: Improvement >Reporter: Jeff Zhang >Assignee: Jeff Zhang > Attachments: TEZ-2581-WIP-1.patch, TEZ-2581-WIP-2.patch, > TEZ-2581-WIP-3.patch, TEZ-2581-WIP-4.patch, TEZ-2581-WIP-5.patch, > TEZ-2581-WIP-6.patch, TezRecoveryRedesignProposal.pdf, > TezRecoveryRedesignV1.1.pdf > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-808) Handle task attempts that are not making progress
[ https://issues.apache.org/jira/browse/TEZ-808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981818#comment-14981818 ] Bikas Saha commented on TEZ-808: Thanks! I will change that and also update the javadoc advisory on calling this infrequently and commit. > Handle task attempts that are not making progress > - > > Key: TEZ-808 > URL: https://issues.apache.org/jira/browse/TEZ-808 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Bikas Saha >Assignee: Bikas Saha > Attachments: TEZ-808.1.patch, TEZ-808.2.patch > > > If a task attempt is not making progress then it may cause the job to hang. > We may want to kill and restart the attempt. With speculation support and > free resources we may want to run another version in parallel. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (TEZ-808) Handle task attempts that are not making progress
[ https://issues.apache.org/jira/browse/TEZ-808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated TEZ-808: --- Attachment: TEZ-808.3.patch Attaching commit patch. Changed config name to be shorter. > Handle task attempts that are not making progress > - > > Key: TEZ-808 > URL: https://issues.apache.org/jira/browse/TEZ-808 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Bikas Saha >Assignee: Bikas Saha > Attachments: TEZ-808.1.patch, TEZ-808.2.patch, TEZ-808.3.patch > > > If a task attempt is not making progress then it may cause the job to hang. > We may want to kill and restart the attempt. With speculation support and > free resources we may want to run another version in parallel. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2909) Tez UI: Application link in All DAGs table is disable when applicationhistory is unavailable.
[ https://issues.apache.org/jira/browse/TEZ-2909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981787#comment-14981787 ] Hitesh Shah commented on TEZ-2909: -- +1 > Tez UI: Application link in All DAGs table is disable when applicationhistory > is unavailable. > - > > Key: TEZ-2909 > URL: https://issues.apache.org/jira/browse/TEZ-2909 > Project: Apache Tez > Issue Type: Bug >Reporter: Sreenath Somarajapuram >Assignee: Sreenath Somarajapuram > Attachments: TEZ-2909.1.patch, TEZ-2909.2.patch > > > Application link in All DAGs table is disable when applicationhistory is > unavailable. It must be available when TEZ_APPLICATION is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-2917) change some logs from info to debug
[ https://issues.apache.org/jira/browse/TEZ-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981689#comment-14981689 ] TezQA commented on TEZ-2917: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12769651/TEZ-2917.patch against master revision 9ca6947. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 3.0.1) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/1265//testReport/ Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/1265//console This message is automatically generated. > change some logs from info to debug > --- > > Key: TEZ-2917 > URL: https://issues.apache.org/jira/browse/TEZ-2917 > Project: Apache Tez > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: TEZ-2917.patch > > > I've done a highly unscientific summarization of the logs from some random > queries, and will now change some log statements that are the most prevalent > and not extremely useful from info to debug. -- This message was sent by Atlassian JIRA (v6.3.4#6332)