[jira] [Updated] (TEZ-1493) WordCount example fails in recovery

2014-08-25 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-1493: Description: {code} 14/08/25 17:37:03 INFO client.TezClient: Submitting DAG to YARN,

[jira] [Updated] (TEZ-1493) WordCount example fails in recovery

2014-08-25 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-1493: Attachment: Tez-1493.patch WordCount example fails in recovery ---

[jira] [Commented] (TEZ-1493) WordCount example fails in recovery

2014-08-25 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109015#comment-14109015 ] Jeff Zhang commented on TEZ-1493: - Attach the patch. The reason of this issue is that when

[jira] [Created] (TEZ-1494) DAG hangs waiting for ShuffleManager.getNextInput()

2014-08-25 Thread Rajesh Balamohan (JIRA)
Rajesh Balamohan created TEZ-1494: - Summary: DAG hangs waiting for ShuffleManager.getNextInput() Key: TEZ-1494 URL: https://issues.apache.org/jira/browse/TEZ-1494 Project: Apache Tez Issue

[jira] [Updated] (TEZ-1494) DAG hangs waiting for ShuffleManager.getNextInput()

2014-08-25 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1494: -- Description: Attaching the DAG and the stack trace of the hung process. Thread 30071: (state

[jira] [Updated] (TEZ-1494) DAG hangs waiting for ShuffleManager.getNextInput()

2014-08-25 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1494: -- Attachment: TEZ-1494-DAG.dot DAG hangs waiting for ShuffleManager.getNextInput()

[jira] [Commented] (TEZ-1490) dagid reported is incorrect in TezClient.java

2014-08-25 Thread Prakash Ramachandran (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109136#comment-14109136 ] Prakash Ramachandran commented on TEZ-1490: --- Thanks for the patch. yes its mostly

[jira] [Updated] (TEZ-1486) TezUncheckedException when using dynamic partition pruning

2014-08-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-1486: Attachment: TEZ-1486.1.txt Patch skips routing events if target vertex parallelism is 0. I've left

[jira] [Commented] (TEZ-1083) Enable IFile RLE for DefaultSorter

2014-08-25 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109453#comment-14109453 ] Gopal V commented on TEZ-1083: -- This looks alright - this just needs a roll-over check for the

[jira] [Commented] (TEZ-1494) DAG hangs waiting for ShuffleManager.getNextInput()

2014-08-25 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109454#comment-14109454 ] Hitesh Shah commented on TEZ-1494: -- [~rajesh.balamohan] Is this an issue present in the

[jira] [Updated] (TEZ-1493) WordCount example fails in recovery

2014-08-25 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-1493: - Priority: Blocker (was: Major) Target Version/s: 0.5.1 WordCount example fails in recovery

[jira] [Resolved] (TEZ-1473) TEZ_RUNTIME_SHUFFLE_BUFFER is too large by default

2014-08-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth resolved TEZ-1473. - Resolution: Invalid TEZ_RUNTIME_SHUFFLE_BUFFER is too large by default

[jira] [Commented] (TEZ-1492) IFile RLE not kicking in due to bug in BufferUtils.compare()

2014-08-25 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109461#comment-14109461 ] Gopal V commented on TEZ-1492: -- The BufferUtils class needs re-namespacing as well, as part of

[jira] [Commented] (TEZ-1489) Broadcast Shuffle should call freeResources() on FetchedInput

2014-08-25 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109472#comment-14109472 ] Gopal V commented on TEZ-1489: -- The buffer is being cleared up correctly - but the unreserve()

[jira] [Commented] (TEZ-1489) Broadcast Shuffle should call freeResources() on FetchedInput

2014-08-25 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109476#comment-14109476 ] Gopal V commented on TEZ-1489: -- UnorderedKVReader::moveToNextInput() maybe is a good place for

[jira] [Commented] (TEZ-1489) Broadcast Shuffle should call freeResources() on FetchedInput

2014-08-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109481#comment-14109481 ] Siddharth Seth commented on TEZ-1489: - Do you have some logs which can be looked at -

[jira] [Commented] (TEZ-1360) Provide vertex parallelism to each vertex task

2014-08-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109484#comment-14109484 ] Siddharth Seth commented on TEZ-1360: - +1. Committing. Thanks [~gopalv], [~oae],

[jira] [Commented] (TEZ-1471) Additional supplement for TEZ local mode document

2014-08-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109518#comment-14109518 ] Siddharth Seth commented on TEZ-1471: - [~airbots] - the changes look good. Could you

[jira] [Created] (TEZ-1495) ATS integration for TezClient

2014-08-25 Thread Prakash Ramachandran (JIRA)
Prakash Ramachandran created TEZ-1495: - Summary: ATS integration for TezClient Key: TEZ-1495 URL: https://issues.apache.org/jira/browse/TEZ-1495 Project: Apache Tez Issue Type: Bug

[jira] [Updated] (TEZ-1495) ATS integration for TezClient

2014-08-25 Thread Prakash Ramachandran (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prakash Ramachandran updated TEZ-1495: -- Attachment: TEZ-1495.WIP.1.patch Changes in TEZ-1495.WIP.1.patch - getDagStatus,

[jira] [Commented] (TEZ-1476) DAGClient waitForCompletion output is confusing

2014-08-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109649#comment-14109649 ] Siddharth Seth commented on TEZ-1476: - +1. Looks good. DAGClient waitForCompletion

[jira] [Commented] (TEZ-1433) Invalid credentials can be used when a DAG is submitted to a session which has timed out

2014-08-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109672#comment-14109672 ] Siddharth Seth commented on TEZ-1433: - [~jeagles] - it seems like we have some races

[jira] [Commented] (TEZ-1471) Additional supplement for TEZ local mode document

2014-08-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109800#comment-14109800 ] Siddharth Seth commented on TEZ-1471: - +1. Thanks [~airbots] Additional supplement for

[jira] [Commented] (TEZ-1486) TezUncheckedException when using dynamic partition pruning

2014-08-25 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109823#comment-14109823 ] Bikas Saha commented on TEZ-1486: - Looks good. Some comments. Log message needs fixing.

[jira] [Created] (TEZ-1496) Multi MR inputs can not be configured without accessing internal proto structures

2014-08-25 Thread Vikram Dixit K (JIRA)
Vikram Dixit K created TEZ-1496: --- Summary: Multi MR inputs can not be configured without accessing internal proto structures Key: TEZ-1496 URL: https://issues.apache.org/jira/browse/TEZ-1496 Project:

[jira] [Created] (TEZ-1497) Add tez-broadcast-example into tez-examples/

2014-08-25 Thread Gopal V (JIRA)
Gopal V created TEZ-1497: Summary: Add tez-broadcast-example into tez-examples/ Key: TEZ-1497 URL: https://issues.apache.org/jira/browse/TEZ-1497 Project: Apache Tez Issue Type: Bug

[jira] [Commented] (TEZ-1492) IFile RLE not kicking in due to bug in BufferUtils.compare()

2014-08-25 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109874#comment-14109874 ] Rajesh Balamohan commented on TEZ-1492: --- [~gopalv] BufferUtils is placed in o.a.h.io

[jira] [Comment Edited] (TEZ-1492) IFile RLE not kicking in due to bug in BufferUtils.compare()

2014-08-25 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109874#comment-14109874 ] Rajesh Balamohan edited comment on TEZ-1492 at 8/25/14 10:28 PM:

[jira] [Commented] (TEZ-1494) DAG hangs waiting for ShuffleManager.getNextInput()

2014-08-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109872#comment-14109872 ] Siddharth Seth commented on TEZ-1494: - [~rajesh.balamohan] - have you investigated this

[jira] [Commented] (TEZ-1476) DAGClient waitForCompletion output is confusing

2014-08-25 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109873#comment-14109873 ] Jonathan Eagles commented on TEZ-1476: -- Thanks [~sseth]] and [~zjffdu] for the reviews.

[jira] [Commented] (TEZ-1044) Need to consider map/reduce java.opts values for container reuse

2014-08-25 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109951#comment-14109951 ] Hitesh Shah commented on TEZ-1044: -- [~yeshavora] Please provide more details on this issue.

[jira] [Updated] (TEZ-1496) Multi MR inputs can not be configured without accessing internal proto structures

2014-08-25 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-1496: - Target Version/s: 0.5.0 Multi MR inputs can not be configured without accessing internal proto

[jira] [Commented] (TEZ-1496) Multi MR inputs can not be configured without accessing internal proto structures

2014-08-25 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109960#comment-14109960 ] Hitesh Shah commented on TEZ-1496: -- [~vikram.dixit] Can you provide more details on what

[jira] [Updated] (TEZ-1496) Multi MR inputs can not be configured without accessing internal proto structures

2014-08-25 Thread Hitesh Shah (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-1496: - Affects Version/s: (was: 0.5.1) Multi MR inputs can not be configured without accessing internal

[jira] [Updated] (TEZ-1476) DAGClient waitForCompletion output is confusing

2014-08-25 Thread Siddharth Seth (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-1476: Fix Version/s: 0.5.1 DAGClient waitForCompletion output is confusing

[jira] [Commented] (TEZ-1490) dagid reported is incorrect in TezClient.java

2014-08-25 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110023#comment-14110023 ] Jonathan Eagles commented on TEZ-1490: -- [~bikassaha], could you give a review and

[jira] [Commented] (TEZ-1497) Add tez-broadcast-example into tez-examples/

2014-08-25 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110042#comment-14110042 ] Bikas Saha commented on TEZ-1497: - The JoinExample in tez-examples is already illustrating

[jira] [Commented] (TEZ-1433) Invalid credentials can be used when a DAG is submitted to a session which has timed out

2014-08-25 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110053#comment-14110053 ] Jonathan Eagles commented on TEZ-1433: -- A true rollback can be difficult to implement

[jira] [Commented] (TEZ-1490) dagid reported is incorrect in TezClient.java

2014-08-25 Thread Bikas Saha (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110060#comment-14110060 ] Bikas Saha commented on TEZ-1490: - Clearly, the DO NOT CHANGE this comment did not help in

[jira] [Commented] (TEZ-1430) Javadoc generation should not generate docs for classes annotated as private

2014-08-25 Thread Jonathan Eagles (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110226#comment-14110226 ] Jonathan Eagles commented on TEZ-1430: -- [~hitesh], can you please review as you added

[jira] [Updated] (TEZ-1492) IFile RLE not kicking in due to bug in BufferUtils.compare()

2014-08-25 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1492: -- Attachment: TEZ-1492.3.patch Moved BufferUtils and FastByteComparisons to

[jira] [Updated] (TEZ-1493) Tez examples fail in recovery sometimes

2014-08-25 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang updated TEZ-1493: Summary: Tez examples fail in recovery sometimes (was: WordCount example fails in recovery) Tez examples

[jira] [Created] (TEZ-1498) Usage info is not printed when error number of arguments for JoinExample

2014-08-25 Thread Jeff Zhang (JIRA)
Jeff Zhang created TEZ-1498: --- Summary: Usage info is not printed when error number of arguments for JoinExample Key: TEZ-1498 URL: https://issues.apache.org/jira/browse/TEZ-1498 Project: Apache Tez

[jira] [Commented] (TEZ-1492) IFile RLE not kicking in due to bug in BufferUtils.compare()

2014-08-25 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110264#comment-14110264 ] Gopal V commented on TEZ-1492: -- Thanks [~rajesh.balamohan], can I have the same diff with git

[jira] [Updated] (TEZ-1492) IFile RLE not kicking in due to bug in BufferUtils.compare()

2014-08-25 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1492: -- Attachment: TEZ-1492.4.patch No code change (but regenerated the patch with git diff -M

[jira] [Resolved] (TEZ-1497) Add tez-broadcast-example into tez-examples/

2014-08-25 Thread Gopal V (JIRA)
[ https://issues.apache.org/jira/browse/TEZ-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V resolved TEZ-1497. -- Resolution: Not a Problem Add tez-broadcast-example into tez-examples/