[jira] [Commented] (TEZ-3952) Allow Tez task speculation to allow for greater customization
[
https://issues.apache.org/jira/browse/TEZ-3952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510201#comment-16510201
]
TezQA commented on TEZ-3952:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12927525/TEZ-3952.003.patch
against master revision 3f2373e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter
org.apache.tez.runtime.library.common.shuffle.orderedgrouped.TestMergeManager
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2837//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2837//console
This message is automatically generated.
> Allow Tez task speculation to allow for greater customization
> -
>
> Key: TEZ-3952
> URL: https://issues.apache.org/jira/browse/TEZ-3952
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Nishant Dash
>Assignee: Nishant Dash
>Priority: Major
> Attachments: TEZ-3952.001.patch, TEZ-3952.002.patch,
> TEZ-3952.003.patch
>
>
> Many of the settings for Tez task speculation are hardcoded and should
> instead be configurable. For example, there's no equivalent config settings
> for the following MapReduce settings:
> - mapreduce.job.speculative.speculative-cap-running-tasks
> - mapreduce.job.speculative.retry-after-no-speculate
> - mapreduce.job.speculative.retry-after-speculate
> - mapreduce.job.speculative.minimum-allowed-tasks
> - mapreduce.job.speculative.speculative-cap-total-tasks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TEZ-3952) Allow Tez task speculation to allow for greater customization
[ https://issues.apache.org/jira/browse/TEZ-3952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510055#comment-16510055 ] Jonathan Eagles commented on TEZ-3952: -- [~nishantdash], thanks for this patch. Couple of minor nits. Clean up the unnecessary space changes and switch method names to camel case and this will be a great addition. > Allow Tez task speculation to allow for greater customization > - > > Key: TEZ-3952 > URL: https://issues.apache.org/jira/browse/TEZ-3952 > Project: Apache Tez > Issue Type: Improvement >Reporter: Nishant Dash >Assignee: Nishant Dash >Priority: Major > Attachments: TEZ-3952.001.patch, TEZ-3952.002.patch > > > Many of the settings for Tez task speculation are hardcoded and should > instead be configurable. For example, there's no equivalent config settings > for the following MapReduce settings: > - mapreduce.job.speculative.speculative-cap-running-tasks > - mapreduce.job.speculative.retry-after-no-speculate > - mapreduce.job.speculative.retry-after-speculate > - mapreduce.job.speculative.minimum-allowed-tasks > - mapreduce.job.speculative.speculative-cap-total-tasks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-3952) Allow Tez task speculation to allow for greater customization
[ https://issues.apache.org/jira/browse/TEZ-3952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510017#comment-16510017 ] Nishant Dash commented on TEZ-3952: --- TestFaultTolerance is a know flaky test. Reopened TEZ-2686. TestUnorderedPartitionedKVWriter is due to HADOOP-15450. [~kshukla] could you take a look at the patch? > Allow Tez task speculation to allow for greater customization > - > > Key: TEZ-3952 > URL: https://issues.apache.org/jira/browse/TEZ-3952 > Project: Apache Tez > Issue Type: Improvement >Reporter: Nishant Dash >Assignee: Nishant Dash >Priority: Major > Attachments: TEZ-3952.001.patch, TEZ-3952.002.patch > > > Many of the settings for Tez task speculation are hardcoded and should > instead be configurable. For example, there's no equivalent config settings > for the following MapReduce settings: > - mapreduce.job.speculative.speculative-cap-running-tasks > - mapreduce.job.speculative.retry-after-no-speculate > - mapreduce.job.speculative.retry-after-speculate > - mapreduce.job.speculative.minimum-allowed-tasks > - mapreduce.job.speculative.speculative-cap-total-tasks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-3952) Allow Tez task speculation to allow for greater customization
[
https://issues.apache.org/jira/browse/TEZ-3952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509835#comment-16509835
]
TezQA commented on TEZ-3952:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12927477/TEZ-3952.002.patch
against master revision 3f2373e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.test.TestFaultTolerance
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2836//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2836//console
This message is automatically generated.
> Allow Tez task speculation to allow for greater customization
> -
>
> Key: TEZ-3952
> URL: https://issues.apache.org/jira/browse/TEZ-3952
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Nishant Dash
>Assignee: Nishant Dash
>Priority: Major
> Attachments: TEZ-3952.001.patch, TEZ-3952.002.patch
>
>
> Many of the settings for Tez task speculation are hardcoded and should
> instead be configurable. For example, there's no equivalent config settings
> for the following MapReduce settings:
> - mapreduce.job.speculative.speculative-cap-running-tasks
> - mapreduce.job.speculative.retry-after-no-speculate
> - mapreduce.job.speculative.retry-after-speculate
> - mapreduce.job.speculative.minimum-allowed-tasks
> - mapreduce.job.speculative.speculative-cap-total-tasks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TEZ-3952) Allow Tez task speculation to allow for greater customization
[ https://issues.apache.org/jira/browse/TEZ-3952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509731#comment-16509731 ] Nishant Dash commented on TEZ-3952: --- Updated patch has been posted, which corrects some naming issues and modifies current test cases to test for the added functionality mentioned in this jira. The core tests that fail are due to HADOOP-15450. > Allow Tez task speculation to allow for greater customization > - > > Key: TEZ-3952 > URL: https://issues.apache.org/jira/browse/TEZ-3952 > Project: Apache Tez > Issue Type: Improvement >Reporter: Nishant Dash >Assignee: Nishant Dash >Priority: Major > Attachments: TEZ-3952.001.patch, TEZ-3952.002.patch > > > Many of the settings for Tez task speculation are hardcoded and should > instead be configurable. For example, there's no equivalent config settings > for the following MapReduce settings: > - mapreduce.job.speculative.speculative-cap-running-tasks > - mapreduce.job.speculative.retry-after-no-speculate > - mapreduce.job.speculative.retry-after-speculate > - mapreduce.job.speculative.minimum-allowed-tasks > - mapreduce.job.speculative.speculative-cap-total-tasks -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-3952) Allow Tez task speculation to allow for greater customization
[
https://issues.apache.org/jira/browse/TEZ-3952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502640#comment-16502640
]
TezQA commented on TEZ-3952:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12926650/TEZ-3952.001.patch
against master revision 09102e5.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 3.0.1) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in :
org.apache.tez.runtime.library.common.writers.TestUnorderedPartitionedKVWriter
The following test timeouts occurred in :
org.apache.tez.dag.app.rm.TestTaskScheduler
Test results:
https://builds.apache.org/job/PreCommit-TEZ-Build/2827//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/2827//console
This message is automatically generated.
> Allow Tez task speculation to allow for greater customization
> -
>
> Key: TEZ-3952
> URL: https://issues.apache.org/jira/browse/TEZ-3952
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Nishant Dash
>Assignee: Nishant Dash
>Priority: Major
> Attachments: TEZ-3952.001.patch
>
>
> Many of the settings for Tez task speculation are hardcoded and should
> instead be configurable. For example, there's no equivalent config settings
> for the following MapReduce settings:
> - mapreduce.job.speculative.speculative-cap-running-tasks
> - mapreduce.job.speculative.retry-after-no-speculate
> - mapreduce.job.speculative.retry-after-speculate
> - mapreduce.job.speculative.minimum-allowed-tasks
> - mapreduce.job.speculative.speculative-cap-total-tasks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
