date:20150324

[jira] [Commented] (TEZ-986) Make conf set on DAG and vertex available in jobhistory

2015-03-24 Thread Rohini Palaniswamy (JIRA)

[
https://issues.apache.org/jira/browse/TEZ-986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377568#comment-14377568
]

Rohini Palaniswamy commented on TEZ-986:

bq. viewable in Tez UI after the job completes. This is very essential for
debugging jobs.
Just wanted to mention that we need this for Pig and would be good to have
it in one of the upcoming releases. While debugging some of the recent issues,
realized that I don't have access to pig script if user ran it from a gateway
(for Oozie I can get it from launcher job), because pig.script setting is only
set on the DAG config and few other settings useful for debugging like pig
version. For now, I have other workarounds to get this info or resort to asking
the user.

The vertex config also has some important debugging info like what feature is
being run (group by, etc), input/output dirs, etc. Even for this can manage for
the short term and figure these out with the explain output of the script. But
life would be easier if those are shown in the UI.

Make conf set on DAG and vertex available in jobhistory
---

Key: TEZ-986
URL: https://issues.apache.org/jira/browse/TEZ-986
Project: Apache Tez
Issue Type: Sub-task
Components: UI
Reporter: Rohini Palaniswamy
Priority: Blocker

Would like to have the conf set on DAG and Vertex
1) viewable in Tez UI after the job completes. This is very essential for
debugging jobs.
2) We have processes, that parse jobconf.xml from job history (hdfs) and
load them into hive tables for analysis. Would like to have Tez also make all
the configuration (byte array) available in job history so that we can
similarly parse them. 1) mandates that you store it in hdfs. 2) is just to
say make the format stored as a contract others can rely on for parsing.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TEZ-2224) EventQueue empty doesn't mean events are consumed in RecoveryService

2015-03-24 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/TEZ-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1433#comment-1433
 ] 

Hadoop QA commented on TEZ-2224:


{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12706880/TEZ-2224-1.patch
  against master revision f53942c.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/338//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/338//console

This message is automatically generated.

 EventQueue empty doesn't mean events are consumed in RecoveryService
 

 Key: TEZ-2224
 URL: https://issues.apache.org/jira/browse/TEZ-2224
 Project: Apache Tez
  Issue Type: Bug
Reporter: Jeff Zhang
Assignee: Jeff Zhang
 Attachments: TEZ-2224-1.patch


 If the event queue is empty, the event may still been processing. Should fix 
 it like AsyncDispatcher



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Success: TEZ-2224 PreCommit Build #338

2015-03-24 Thread Apache Jenkins Server

Jira: https://issues.apache.org/jira/browse/TEZ-2224
Build: https://builds.apache.org/job/PreCommit-TEZ-Build/338/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 2754 lines...]
[INFO] Final Memory: 72M/762M
[INFO] 




{color:green}+1 overall{color}.  Here are the results of testing the latest 
attachment
  http://issues.apache.org/jira/secure/attachment/12706880/TEZ-2224-1.patch
  against master revision f53942c.

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+1 tests included{color}.  The patch appears to include 1 new 
or modified test files.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

{color:green}+1 findbugs{color}.  The patch does not introduce any new 
Findbugs (version 2.0.3) warnings.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/338//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/338//console

This message is automatically generated.


==
==
Adding comment to Jira.
==
==


Comment added.
d98f08814b5cfb8ee2fbc55e0c5bb28a28e2e817 logged out


==
==
Finished build.
==
==


Archiving artifacts
Sending artifact delta relative to PreCommit-TEZ-Build #335
Archived 44 artifacts
Archive block size is 32768
Received 4 blocks and 2622022 bytes
Compression is 4.8%
Took 1.3 sec
Description set: TEZ-2224
Recording test results
Email was triggered for: Success
Sending email for trigger: Success



###
## FAILED TESTS (if any) 
##
All tests passed

[jira] [Comment Edited] (TEZ-2224) EventQueue empty doesn't mean events are consumed in RecoveryService

2015-03-24 Thread Jeff Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/TEZ-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377733#comment-14377733
 ] 

Jeff Zhang edited comment on TEZ-2224 at 3/24/15 11:39 AM:
---

Upload patch for this issue. Just like how AsyncDispatcher does. [~hitesh] 
Please help review it.


was (Author: zjffdu):
Upload patch for this issue. Just like how AsyncDispatcher does

 EventQueue empty doesn't mean events are consumed in RecoveryService
 

 Key: TEZ-2224
 URL: https://issues.apache.org/jira/browse/TEZ-2224
 Project: Apache Tez
  Issue Type: Bug
Reporter: Jeff Zhang
Assignee: Jeff Zhang
 Attachments: TEZ-2224-1.patch


 If the event queue is empty, the event may still been processing. Should fix 
 it like AsyncDispatcher



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TEZ-2205) Tez still tries to post to ATS when yarn.timeline-service.enabled=false

2015-03-24 Thread Rohini Palaniswamy (JIRA)

[
https://issues.apache.org/jira/browse/TEZ-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377562#comment-14377562
]

Rohini Palaniswamy commented on TEZ-2205:
-

I prefer 3). For jobs launched through Oozie it is easy to turn off ATS via
Oozie server side setting and this might be required in the near future now and
then considering the issues we are facing with ATS. Since tez-site.xml for
those jobs come from HDFS, it is not easy to change the tez ATS Logger easily
(replacing the file on HDFS is more manual and can cause running jobs to fail
as LocalResource time has changed) and so do not like 1). Also having to change
multiple settings to turn off something is cumbersome. 2) is what is happening
now but the problem I see is that it impacts performance as time is wasted
trying to connect to ATS and failing due to lack of authentication.

Tez still tries to post to ATS when yarn.timeline-service.enabled=false
---

Key: TEZ-2205
URL: https://issues.apache.org/jira/browse/TEZ-2205
Project: Apache Tez
Issue Type: Sub-task
Affects Versions: 0.6.1
Reporter: Chang Li
Assignee: Chang Li
Attachments: TEZ-2205.wip.patch

when set yarn.timeline-service.enabled=false, Tez still tries posting to ATS,
but hits error as token is not found. Does not fail the job because of the
fix to not fail job when there is error posting to ATS. But it should not be
trying to post to ATS in the first place.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TEZ-2224) EventQueue empty doesn't mean events are consumed in RecoveryService

2015-03-24 Thread Jeff Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/TEZ-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14377733#comment-14377733
 ] 

Jeff Zhang commented on TEZ-2224:
-

Upload patch for this issue. Just like how AsyncDispatcher does

 EventQueue empty doesn't mean events are consumed in RecoveryService
 

 Key: TEZ-2224
 URL: https://issues.apache.org/jira/browse/TEZ-2224
 Project: Apache Tez
  Issue Type: Bug
Reporter: Jeff Zhang
Assignee: Jeff Zhang
 Attachments: TEZ-2224-1.patch


 If the event queue is empty, the event may still been processing. Should fix 
 it like AsyncDispatcher



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (TEZ-2224) EventQueue empty doesn't mean events are consumed in RecoveryService

2015-03-24 Thread Jeff Zhang (JIRA)


 [ 
https://issues.apache.org/jira/browse/TEZ-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Zhang updated TEZ-2224:

Attachment: TEZ-2224-1.patch

 EventQueue empty doesn't mean events are consumed in RecoveryService
 

 Key: TEZ-2224
 URL: https://issues.apache.org/jira/browse/TEZ-2224
 Project: Apache Tez
  Issue Type: Bug
Reporter: Jeff Zhang
Assignee: Jeff Zhang
 Attachments: TEZ-2224-1.patch






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (TEZ-2224) EventQueue empty doesn't mean events are consumed in RecoveryService

2015-03-24 Thread Jeff Zhang (JIRA)


 [ 
https://issues.apache.org/jira/browse/TEZ-2224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeff Zhang updated TEZ-2224:

Description: If the event queue is empty, the event may still been 
processing. Should fix it like AsyncDispatcher

 EventQueue empty doesn't mean events are consumed in RecoveryService
 

 Key: TEZ-2224
 URL: https://issues.apache.org/jira/browse/TEZ-2224
 Project: Apache Tez
  Issue Type: Bug
Reporter: Jeff Zhang
Assignee: Jeff Zhang
 Attachments: TEZ-2224-1.patch


 If the event queue is empty, the event may still been processing. Should fix 
 it like AsyncDispatcher



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

56 matches

Mail list logo