[jira] [Created] (TEZ-1513) Move to Apache Content Management System

Jonathan Eagles created TEZ-1513: Summary: Move to Apache Content Management System Key: TEZ-1513 URL: https://issues.apache.org/jira/browse/TEZ-1513 Project: Apache Tez Issue Type:

[jira] [Commented] (TEZ-1512) VertexImpl.getTask(int) can be CPU intensive when lots of tasks are present in the vertex

[ https://issues.apache.org/jira/browse/TEZ-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14113388#comment-14113388 ] Siddharth Seth commented on TEZ-1512: - [~rajesh.balamohan] - can this just be changed to

[jira] [Commented] (TEZ-1310) Update website documentation framework

[ https://issues.apache.org/jira/browse/TEZ-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14113392#comment-14113392 ] Jonathan Eagles commented on TEZ-1310: -- Addressed the markdown conversion bugs -

[jira] [Updated] (TEZ-1513) Move to Apache Content Management System

[ https://issues.apache.org/jira/browse/TEZ-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-1513: - Description: It is recommended for projects to use Apache Content Management System (CMS)

[jira] [Commented] (TEZ-1310) Update website documentation framework

[ https://issues.apache.org/jira/browse/TEZ-1310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14113404#comment-14113404 ] Jonathan Eagles commented on TEZ-1310: -- [~hitesh], This is patch is ready for review.

[jira] [Assigned] (TEZ-1508) Log a warning if Xmx is configured incorrectly.

[ https://issues.apache.org/jira/browse/TEZ-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles reassigned TEZ-1508: Assignee: Jonathan Eagles Log a warning if Xmx is configured incorrectly.

[jira] [Commented] (TEZ-853) Support counters recovery

[ https://issues.apache.org/jira/browse/TEZ-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14113433#comment-14113433 ] Jeff Zhang commented on TEZ-853: [~hitesh] bq. Do TaskImpl and VertexImpl write counters to

[jira] [Updated] (TEZ-1512) VertexImpl.getTask(int) can be CPU intensive when lots of tasks are present in the vertex

[ https://issues.apache.org/jira/browse/TEZ-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1512: -- Attachment: TEZ-1512.2.patch Yes [~sseth]. Addressed it in the current patch.

[jira] [Created] (TEZ-1514) Add a logging mechanism for Tasks - similar to App History

Siddharth Seth created TEZ-1514: --- Summary: Add a logging mechanism for Tasks - similar to App History Key: TEZ-1514 URL: https://issues.apache.org/jira/browse/TEZ-1514 Project: Apache Tez

[jira] [Commented] (TEZ-1358) Display better diagnostics when tasks fail to launch

[ https://issues.apache.org/jira/browse/TEZ-1358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14113442#comment-14113442 ] Jeff Zhang commented on TEZ-1358: - [~hitesh] verify it that it will show up in the RM UI.

[jira] [Assigned] (TEZ-1357) Display better diagnostics when AM fails to launch

[ https://issues.apache.org/jira/browse/TEZ-1357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Zhang reassigned TEZ-1357: --- Assignee: Jeff Zhang Display better diagnostics when AM fails to launch

[jira] [Assigned] (TEZ-1512) VertexImpl.getTask(int) can be CPU intensive when lots of tasks are present in the vertex

[ https://issues.apache.org/jira/browse/TEZ-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-1512: - Assignee: Rajesh Balamohan VertexImpl.getTask(int) can be CPU intensive when lots of

[jira] [Resolved] (TEZ-1512) VertexImpl.getTask(int) can be CPU intensive when lots of tasks are present in the vertex

[ https://issues.apache.org/jira/browse/TEZ-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved TEZ-1512. --- Resolution: Fixed Fix Version/s: 0.6.0 Hadoop Flags: Reviewed Thanks [~sseth].

[jira] [Updated] (TEZ-1515) DAGAppMaster : Thread contentions due to org.apache.tez.common.counters.ResourceBundles

[ https://issues.apache.org/jira/browse/TEZ-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1515: -- Attachment: detailed_sample_stack_trace.txt HistoryLoggingService.png

[jira] [Created] (TEZ-1515) DAGAppMaster : Thread contentions due to org.apache.tez.common.counters.ResourceBundles

Rajesh Balamohan created TEZ-1515: - Summary: DAGAppMaster : Thread contentions due to org.apache.tez.common.counters.ResourceBundles Key: TEZ-1515 URL: https://issues.apache.org/jira/browse/TEZ-1515

[jira] [Updated] (TEZ-1495) ATS integration for TezClient

[ https://issues.apache.org/jira/browse/TEZ-1495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prakash Ramachandran updated TEZ-1495: -- Attachment: TEZ-1495.2.patch - removed the wrong unit test and added a unit test for

[jira] [Commented] (TEZ-1345) Add checks to guarantee all init events are written to recovery to consider vertex initialized

[ https://issues.apache.org/jira/browse/TEZ-1345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114025#comment-14114025 ] Hitesh Shah commented on TEZ-1345: -- bq. So overall IMO, I prefer to ignore the init events

[jira] [Commented] (TEZ-1509) Set a useful default value for java opts

[ https://issues.apache.org/jira/browse/TEZ-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114049#comment-14114049 ] Bikas Saha commented on TEZ-1509: - Can someone review/comment so that this can be committed

[jira] [Commented] (TEZ-1501) Add a test dag to generate load on the getTask RPC

[ https://issues.apache.org/jira/browse/TEZ-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114052#comment-14114052 ] Gopal V commented on TEZ-1501: -- Looks good - +1 Just needs an fs.deleteOnExit() for the

[jira] [Reopened] (TEZ-1510) TezConfiguration should not add tez-site.xml as a default resource.

[ https://issues.apache.org/jira/browse/TEZ-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah reopened TEZ-1510: -- TezConfiguration should not add tez-site.xml as a default resource.

[jira] [Assigned] (TEZ-1510) TezConfiguration should not add tez-site.xml as a default resource.

[ https://issues.apache.org/jira/browse/TEZ-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah reassigned TEZ-1510: Assignee: Hitesh Shah TezConfiguration should not add tez-site.xml as a default resource.

[jira] [Updated] (TEZ-1501) Add a test dag to generate load on the getTask RPC

[ https://issues.apache.org/jira/browse/TEZ-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-1501: Attachment: TEZ-1501.3.txt Updated patch. Committing. Thanks for the review. Add a test dag to

[jira] [Updated] (TEZ-1510) TezConfiguration should not add tez-site.xml as a default resource.

[ https://issues.apache.org/jira/browse/TEZ-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-1510: - Attachment: TEZ-1510.1.patch [~sseth] review please. TezConfiguration should not add tez-site.xml as a

[jira] [Commented] (TEZ-1510) TezConfiguration should not add tez-site.xml as a default resource.

[ https://issues.apache.org/jira/browse/TEZ-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114123#comment-14114123 ] Bikas Saha commented on TEZ-1510: - Does the test fail without the changes?

[jira] [Commented] (TEZ-1511) MROutputConfigBuilder sets OutputFormat as String class if OutputFormat is not provided

[ https://issues.apache.org/jira/browse/TEZ-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114131#comment-14114131 ] Hitesh Shah commented on TEZ-1511: -- The use of NEW_API_CONFIG seems incorrect. There should

[jira] [Updated] (TEZ-1280) Timeline server integration for DAG history

[ https://issues.apache.org/jira/browse/TEZ-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-1280: - Assignee: Prakash Ramachandran Timeline server integration for DAG history

[jira] [Updated] (TEZ-1516) Log transfer rate for Broadcast Fetch

[ https://issues.apache.org/jira/browse/TEZ-1516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-1516: Attachment: TEZ-1516.1.txt Simple patch to log transfer times for individual fetches, as well as an

[jira] [Updated] (TEZ-1504) Ordered Input Shuffle can hang if there's errors while creating the Fetcher

[ https://issues.apache.org/jira/browse/TEZ-1504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-1504: - Priority: Critical (was: Major) Ordered Input Shuffle can hang if there's errors while creating the

[jira] [Created] (TEZ-1517) Avoid sending routed events via the AsyncDispatcher

Siddharth Seth created TEZ-1517: --- Summary: Avoid sending routed events via the AsyncDispatcher Key: TEZ-1517 URL: https://issues.apache.org/jira/browse/TEZ-1517 Project: Apache Tez Issue Type:

[jira] [Updated] (TEZ-1510) TezConfiguration should not add tez-site.xml as a default resource.

[ https://issues.apache.org/jira/browse/TEZ-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-1510: - Attachment: TEZ-1510.2.patch Modified test to compile without patch. TezConfiguration should not add

[jira] [Updated] (TEZ-1517) Avoid sending routed events via the AsyncDispatcher

[ https://issues.apache.org/jira/browse/TEZ-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-1517: Attachment: TEZ-1517.1.txt Simple patch, to send events directly to tasks. [~bikassaha], [~hitesh]

[jira] [Updated] (TEZ-1511) MROutputConfigBuilder sets OutputFormat as String class if OutputFormat is not provided

[ https://issues.apache.org/jira/browse/TEZ-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated TEZ-1511: Attachment: TEZ-1511.3.patch Made the changes. Changed a test to use this version of the API.

[jira] [Updated] (TEZ-1376) Investigate independent parallel DAGs execution in Local Mode

[ https://issues.apache.org/jira/browse/TEZ-1376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen He updated TEZ-1376: - Summary: Investigate independent parallel DAGs execution in Local Mode (was: Support independent parallel DAGs

[jira] [Assigned] (TEZ-1376) Investigate independent parallel DAGs execution in Local Mode

[ https://issues.apache.org/jira/browse/TEZ-1376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen He reassigned TEZ-1376: Assignee: Chen He Investigate independent parallel DAGs execution in Local Mode

[jira] [Created] (TEZ-1518) Clean up ID caches on DAG completion

Siddharth Seth created TEZ-1518: --- Summary: Clean up ID caches on DAG completion Key: TEZ-1518 URL: https://issues.apache.org/jira/browse/TEZ-1518 Project: Apache Tez Issue Type: Improvement

[jira] [Commented] (TEZ-1508) Log a warning if Xmx is configured incorrectly.

[ https://issues.apache.org/jira/browse/TEZ-1508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114346#comment-14114346 ] Siddharth Seth commented on TEZ-1508: - Should we just be failing if this is configured

[jira] [Updated] (TEZ-1510) TezConfiguration should not add tez-site.xml as a default resource.

[ https://issues.apache.org/jira/browse/TEZ-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated TEZ-1510: - Attachment: TEZ-1510.3.addendum.patch TezConfiguration should not add tez-site.xml as a default resource.

[jira] [Updated] (TEZ-1376) Investigate independent parallel DAGs execution in Local Mode

[ https://issues.apache.org/jira/browse/TEZ-1376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen He updated TEZ-1376: - Attachment: (was: differentParallelJob-surefireOut.txt) Investigate independent parallel DAGs execution in

[jira] [Updated] (TEZ-1376) Investigate independent parallel DAGs execution in Local Mode

[ https://issues.apache.org/jira/browse/TEZ-1376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen He updated TEZ-1376: - Attachment: independantDAG2.data independantDAG1.data independantDAG.pig a pig

[jira] [Comment Edited] (TEZ-1376) Investigate independent parallel DAGs execution in Local Mode

[ https://issues.apache.org/jira/browse/TEZ-1376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114392#comment-14114392 ] Chen He edited comment on TEZ-1376 at 8/28/14 9:37 PM: --- a pig example

[jira] [Issue Comment Deleted] (TEZ-1376) Investigate independent parallel DAGs execution in Local Mode

[ https://issues.apache.org/jira/browse/TEZ-1376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen He updated TEZ-1376: - Comment: was deleted (was: The sequential number looks incorrect since LocalClient use appIdNumber++ to create

[jira] [Commented] (TEZ-1488) Implement HashComparatorBytesWritable in TezBytesComparator

[ https://issues.apache.org/jira/browse/TEZ-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114428#comment-14114428 ] Siddharth Seth commented on TEZ-1488: - +1. Looks good. Some documentation on the

[jira] [Commented] (TEZ-1488) Implement HashComparatorBytesWritable in TezBytesComparator

[ https://issues.apache.org/jira/browse/TEZ-1488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114448#comment-14114448 ] Gopal V commented on TEZ-1488: -- I think we should call the interface what it really - a

[jira] [Commented] (TEZ-1515) DAGAppMaster : Thread contentions due to org.apache.tez.common.counters.ResourceBundles

[ https://issues.apache.org/jira/browse/TEZ-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114486#comment-14114486 ] Siddharth Seth commented on TEZ-1515: - [~rajesh.balamohan] - should we just remove

[jira] [Commented] (TEZ-1517) Avoid sending routed events via the AsyncDispatcher

[ https://issues.apache.org/jira/browse/TEZ-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114503#comment-14114503 ] Siddharth Seth commented on TEZ-1517: - It's still running on the EventHandler thread -

[jira] [Updated] (TEZ-1517) Avoid sending routed events via the AsyncDispatcher

[ https://issues.apache.org/jira/browse/TEZ-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth updated TEZ-1517: Attachment: TEZ-1517.2.txt This does require minimal locking even here - which was earlier being

[jira] [Updated] (TEZ-1515) DAGAppMaster : Thread contentions due to org.apache.tez.common.counters.ResourceBundles

[ https://issues.apache.org/jira/browse/TEZ-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-1515: -- Attachment: TEZ-1515.1.patch Attachment removes the resource bundle/localization usage. [~sseth]

[jira] [Commented] (TEZ-1515) DAGAppMaster : Thread contentions due to org.apache.tez.common.counters.ResourceBundles

[ https://issues.apache.org/jira/browse/TEZ-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114598#comment-14114598 ] Siddharth Seth commented on TEZ-1515: - And open a follow up jira to support displayNames

[jira] [Commented] (TEZ-1510) TezConfiguration should not add tez-site.xml as a default resource.

[ https://issues.apache.org/jira/browse/TEZ-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114771#comment-14114771 ] Bikas Saha commented on TEZ-1510: - Please comment on whether this is safe for 0.5.0 or not.

[jira] [Commented] (TEZ-1511) MROutputConfigBuilder sets OutputFormat as String class if OutputFormat is not provided

[ https://issues.apache.org/jira/browse/TEZ-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114773#comment-14114773 ] Bikas Saha commented on TEZ-1511: - UnionExample is run today as an e2e test for the build.

[jira] [Commented] (TEZ-1510) TezConfiguration should not add tez-site.xml as a default resource.

[ https://issues.apache.org/jira/browse/TEZ-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14114851#comment-14114851 ] Hitesh Shah commented on TEZ-1510: -- Ran a few example jobs and did not see any issues after