[jira] [Commented] (TEZ-3957) Report TASK_DURATION_MILLIS as a Counter for completed tasks
[
https://issues.apache.org/jira/browse/TEZ-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16664655#comment-16664655
]
Eric Wohlstadter commented on TEZ-3957:
---
[~sershe]
Do you want the counter to be continuously updated and reported back to the AM
from running task containers?
I was planning to just add the counter at DAG completion time, from information
in the AM about completed tasks. That's what I meant by "Report
TASK_DURATION_MILLIS as a Counter for completed tasks".
What I had in mind is less useful in general than what I understand this patch
is doing, but it doesn't add any overhead to AM <-> Container messaging.
Wanted to check that there is a use-case where having this as a running counter
is useful (it wasn't necessary for my use-case).
> Report TASK_DURATION_MILLIS as a Counter for completed tasks
>
>
> Key: TEZ-3957
> URL: https://issues.apache.org/jira/browse/TEZ-3957
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Eric Wohlstadter
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: TEZ-3957.patch
>
>
> timeTaken is already being reported by {{TaskAttemptFinishedEvent}}, but not
> as a Counter.
> Combined with TEZ-3911, this provides min(timeTaken), max(timeTaken),
> avg(timeTaken).
> The value will be: {{finishTime - launchTime}}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (TEZ-3957) Report TASK_DURATION_MILLIS as a Counter for completed tasks
[
https://issues.apache.org/jira/browse/TEZ-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated TEZ-3957:
--
Attachment: TEZ-3957.patch
> Report TASK_DURATION_MILLIS as a Counter for completed tasks
>
>
> Key: TEZ-3957
> URL: https://issues.apache.org/jira/browse/TEZ-3957
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Eric Wohlstadter
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: TEZ-3957.patch
>
>
> timeTaken is already being reported by {{TaskAttemptFinishedEvent}}, but not
> as a Counter.
> Combined with TEZ-3911, this provides min(timeTaken), max(timeTaken),
> avg(timeTaken).
> The value will be: {{finishTime - launchTime}}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (TEZ-3957) Report TASK_DURATION_MILLIS as a Counter for completed tasks
[
https://issues.apache.org/jira/browse/TEZ-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin reassigned TEZ-3957:
-
Assignee: Sergey Shelukhin (was: Eric Wohlstadter)
> Report TASK_DURATION_MILLIS as a Counter for completed tasks
>
>
> Key: TEZ-3957
> URL: https://issues.apache.org/jira/browse/TEZ-3957
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Eric Wohlstadter
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: TEZ-3957.patch
>
>
> timeTaken is already being reported by {{TaskAttemptFinishedEvent}}, but not
> as a Counter.
> Combined with TEZ-3911, this provides min(timeTaken), max(timeTaken),
> avg(timeTaken).
> The value will be: {{finishTime - launchTime}}
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TEZ-4012) Add docker support for Tez.
[
https://issues.apache.org/jira/browse/TEZ-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16664491#comment-16664491
]
TezQA commented on TEZ-4012:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} hadolint {color} | {color:blue} 0m
0s{color} | {color:blue} hadolint was not available. {color} |
| {color:blue}0{color} | {color:blue} shelldocs {color} | {color:blue} 0m
0s{color} | {color:blue} Shelldocs was not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
9s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 0m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | TEZ-4012 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12945672/TEZ-4012.002.patch |
| Optional Tests | dupname asflicense hadolint shellcheck shelldocs |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-143-generic #192-Ubuntu SMP Tue
Feb 27 10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/yetus/precommit/personality/tez.sh
|
| git revision | master / 79af4e8 |
| maven | version: Apache Maven 3.5.4
(1edded0938998edf8bf061f1ceb3cfdeccf443fe; 2018-06-17T18:33:14Z) |
| shellcheck | v0.3.3 (This is an old version that has serious bugs. Consider
upgrading.) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/PreCommit-TEZ-Build/44/console
|
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Add docker support for Tez.
> ---
>
> Key: TEZ-4012
> URL: https://issues.apache.org/jira/browse/TEZ-4012
> Project: Apache Tez
> Issue Type: Sub-task
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
>Priority: Major
> Attachments: TEZ-4012.001.patch, TEZ-4012.002.patch
>
>
> Hadoop label builds contain a mix of development tools and versions. In
> particular H11-H20 are unusable by tez since protoc -version is 2.6.x and
> hadoop only supports 2.5.0. This jira will allow builds across all H1-H20
> jenkins machines.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (TEZ-4012) Add docker support for Tez.
[ https://issues.apache.org/jira/browse/TEZ-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-4012: - Attachment: TEZ-4012.002.patch > Add docker support for Tez. > --- > > Key: TEZ-4012 > URL: https://issues.apache.org/jira/browse/TEZ-4012 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles >Priority: Major > Attachments: TEZ-4012.001.patch, TEZ-4012.002.patch > > > Hadoop label builds contain a mix of development tools and versions. In > particular H11-H20 are unusable by tez since protoc -version is 2.6.x and > hadoop only supports 2.5.0. This jira will allow builds across all H1-H20 > jenkins machines. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (TEZ-4012) Add docker support for Tez.
[
https://issues.apache.org/jira/browse/TEZ-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16664322#comment-16664322
]
Kuhu Shukla edited comment on TEZ-4012 at 10/25/18 9:50 PM:
I am not super familiar with yetus and precommit builds but can the shell
script name be changed here from {{hadoop_env_checks}} to maybe
{{tez_env_checks}}?
The only place it is referenced seems to be in the Dockerfile.
was (Author: kshukla):
I am not super familiar with yetus and precommit builds but can the shell
script name be changed here from {{hadoop_env_checks}} to maybe
{{tez_env_checks}}?
> Add docker support for Tez.
> ---
>
> Key: TEZ-4012
> URL: https://issues.apache.org/jira/browse/TEZ-4012
> Project: Apache Tez
> Issue Type: Sub-task
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
>Priority: Major
> Attachments: TEZ-4012.001.patch
>
>
> Hadoop label builds contain a mix of development tools and versions. In
> particular H11-H20 are unusable by tez since protoc -version is 2.6.x and
> hadoop only supports 2.5.0. This jira will allow builds across all H1-H20
> jenkins machines.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TEZ-4012) Add docker support for Tez.
[
https://issues.apache.org/jira/browse/TEZ-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16664322#comment-16664322
]
Kuhu Shukla commented on TEZ-4012:
--
I am not super familiar with yetus and precommit builds but can the shell
script name be changed here from {{hadoop_env_checks}} to maybe
{{tez_env_checks}}?
> Add docker support for Tez.
> ---
>
> Key: TEZ-4012
> URL: https://issues.apache.org/jira/browse/TEZ-4012
> Project: Apache Tez
> Issue Type: Sub-task
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
>Priority: Major
> Attachments: TEZ-4012.001.patch
>
>
> Hadoop label builds contain a mix of development tools and versions. In
> particular H11-H20 are unusable by tez since protoc -version is 2.6.x and
> hadoop only supports 2.5.0. This jira will allow builds across all H1-H20
> jenkins machines.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
