[jira] [Commented] (TEZ-1928) Tez local mode hang in Pig tez local mode
[ https://issues.apache.org/jira/browse/TEZ-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14313057#comment-14313057 ] Siddharth Seth commented on TEZ-1928: - +1 for the patch. > Tez local mode hang in Pig tez local mode > - > > Key: TEZ-1928 > URL: https://issues.apache.org/jira/browse/TEZ-1928 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.5.2 >Reporter: Daniel Dai >Assignee: Hitesh Shah > Attachments: TEZ-1928.1.patch, TestMultiQuery.log, > TestMultiQueryBasic.log, TestScalarAliasesLocal.log > > > Pig tez local mode tests hang under some scenario. I attached several stack > trace of hanging tests. > By setting "tez.am.inline.task.execution.max-tasks", the test does not hang. > However, we cannot make it general since Pig backend code is not designed to > be multithread-safe. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-1928) Tez local mode hang in Pig tez local mode
[ https://issues.apache.org/jira/browse/TEZ-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14313032#comment-14313032 ] Daniel Dai commented on TEZ-1928: - I've already tested with the proposed config, it works for Pig. So +1. > Tez local mode hang in Pig tez local mode > - > > Key: TEZ-1928 > URL: https://issues.apache.org/jira/browse/TEZ-1928 > Project: Apache Tez > Issue Type: Bug >Affects Versions: 0.5.2 >Reporter: Daniel Dai >Assignee: Hitesh Shah > Attachments: TEZ-1928.1.patch, TestMultiQuery.log, > TestMultiQueryBasic.log, TestScalarAliasesLocal.log > > > Pig tez local mode tests hang under some scenario. I attached several stack > trace of hanging tests. > By setting "tez.am.inline.task.execution.max-tasks", the test does not hang. > However, we cannot make it general since Pig backend code is not designed to > be multithread-safe. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-1928) Tez local mode hang in Pig tez local mode
[
https://issues.apache.org/jira/browse/TEZ-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14286581#comment-14286581
]
Hadoop QA commented on TEZ-1928:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12693736/TEZ-1928.1.patch
against master revision de904ae.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-TEZ-Build/69//testReport/
Console output: https://builds.apache.org/job/PreCommit-TEZ-Build/69//console
This message is automatically generated.
> Tez local mode hang in Pig tez local mode
> -
>
> Key: TEZ-1928
> URL: https://issues.apache.org/jira/browse/TEZ-1928
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.5.2
>Reporter: Daniel Dai
>Assignee: Hitesh Shah
> Attachments: TEZ-1928.1.patch, TestMultiQuery.log,
> TestMultiQueryBasic.log, TestScalarAliasesLocal.log
>
>
> Pig tez local mode tests hang under some scenario. I attached several stack
> trace of hanging tests.
> By setting "tez.am.inline.task.execution.max-tasks", the test does not hang.
> However, we cannot make it general since Pig backend code is not designed to
> be multithread-safe.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (TEZ-1928) Tez local mode hang in Pig tez local mode
[ https://issues.apache.org/jira/browse/TEZ-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14272132#comment-14272132 ] Siddharth Seth commented on TEZ-1928: - Yes, you can do that. This primarily happens because some vertices may start up early - before tasks from dependent vertices are complete. Pre-emption is used to handle this when there isn't enough capacity, but does not work in LocalMode. This scheduler should become the default in a future release - once a few more features are added to it. For now, it just guarantees ordering. > Tez local mode hang in Pig tez local mode > - > > Key: TEZ-1928 > URL: https://issues.apache.org/jira/browse/TEZ-1928 > Project: Apache Tez > Issue Type: Bug >Reporter: Daniel Dai > Attachments: TestMultiQuery.log, TestMultiQueryBasic.log, > TestScalarAliasesLocal.log > > > Pig tez local mode tests hang under some scenario. I attached several stack > trace of hanging tests. > By setting "tez.am.inline.task.execution.max-tasks", the test does not hang. > However, we cannot make it general since Pig backend code is not designed to > be multithread-safe. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-1928) Tez local mode hang in Pig tez local mode
[ https://issues.apache.org/jira/browse/TEZ-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14271959#comment-14271959 ] Daniel Dai commented on TEZ-1928: - It seems work. Shall we use this always in local mode? > Tez local mode hang in Pig tez local mode > - > > Key: TEZ-1928 > URL: https://issues.apache.org/jira/browse/TEZ-1928 > Project: Apache Tez > Issue Type: Bug >Reporter: Daniel Dai > Attachments: TestMultiQuery.log, TestMultiQueryBasic.log, > TestScalarAliasesLocal.log > > > Pig tez local mode tests hang under some scenario. I attached several stack > trace of hanging tests. > By setting "tez.am.inline.task.execution.max-tasks", the test does not hang. > However, we cannot make it general since Pig backend code is not designed to > be multithread-safe. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (TEZ-1928) Tez local mode hang in Pig tez local mode
[
https://issues.apache.org/jira/browse/TEZ-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14268920#comment-14268920
]
Siddharth Seth commented on TEZ-1928:
-
[~daijy] - if you're running with Tez-0.5.3 or higher, can you try setting the
following - either programmatically or via tez-site
{code}
tez.am.dag.scheduler.class
org.apache.tez.dag.app.dag.impl.DAGSchedulerNaturalOrderControlled
{code}
> Tez local mode hang in Pig tez local mode
> -
>
> Key: TEZ-1928
> URL: https://issues.apache.org/jira/browse/TEZ-1928
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Daniel Dai
> Attachments: TestMultiQuery.log, TestMultiQueryBasic.log,
> TestScalarAliasesLocal.log
>
>
> Pig tez local mode tests hang under some scenario. I attached several stack
> trace of hanging tests.
> By setting "tez.am.inline.task.execution.max-tasks", the test does not hang.
> However, we cannot make it general since Pig backend code is not designed to
> be multithread-safe.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
