[jira] [Commented] (TEZ-3719) DAGImpl.computeProgress slows down dispatcher and ipc threads

[ https://issues.apache.org/jira/browse/TEZ-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005930#comment-16005930 ] TezQA commented on TEZ-3719: {color:red}-1 overall{color}. Here are the results of testing the latest

Failed: TEZ-3719 PreCommit Build #2428

Jira: https://issues.apache.org/jira/browse/TEZ-3719 Build: https://builds.apache.org/job/PreCommit-TEZ-Build/2428/ ### ## LAST 60 LINES OF THE CONSOLE ###

[jira] [Commented] (TEZ-3719) DAGImpl.computeProgress slows down dispatcher and ipc threads

[ https://issues.apache.org/jira/browse/TEZ-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005870#comment-16005870 ] Rajesh Balamohan commented on TEZ-3719: --- Patch lgtm. +1 pending jenkins tests. >

[jira] [Created] (TEZ-3721) Extend CallableWithNdc to also clone MDC

Prasanth Jayachandran created TEZ-3721: -- Summary: Extend CallableWithNdc to also clone MDC Key: TEZ-3721 URL: https://issues.apache.org/jira/browse/TEZ-3721 Project: Apache Tez Issue

[jira] [Commented] (TEZ-3719) DAGImpl.computeProgress slows down dispatcher and ipc threads

[ https://issues.apache.org/jira/browse/TEZ-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005809#comment-16005809 ] Gopal V commented on TEZ-3719: -- [~jeagles]: I think I have made all of those changes and increased my AM->RM

[jira] [Updated] (TEZ-3719) DAGImpl.computeProgress slows down dispatcher and ipc threads

[ https://issues.apache.org/jira/browse/TEZ-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3719: - Description: The Vertex readLock is held while computing progress, which prevents scheduling of tasks from

[jira] [Updated] (TEZ-3720) Optimize SimpleHistoryLoggingService: FileSystem$Stats updates are expensive than writes

[ https://issues.apache.org/jira/browse/TEZ-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3720: - Priority: Minor (was: Major) > Optimize SimpleHistoryLoggingService: FileSystem$Stats updates are expensive >

[jira] [Updated] (TEZ-3720) Optimize SimpleHistoryLoggingService: FileSystem$Stats updates are expensive than writes

[ https://issues.apache.org/jira/browse/TEZ-3720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3720: -- Attachment: FS$Stats.incrementBytesWritten.png > Optimize SimpleHistoryLoggingService:

[jira] [Created] (TEZ-3720) Optimize SimpleHistoryLoggingService: FileSystem$Stats updates are expensive than writes

Rajesh Balamohan created TEZ-3720: - Summary: Optimize SimpleHistoryLoggingService: FileSystem$Stats updates are expensive than writes Key: TEZ-3720 URL: https://issues.apache.org/jira/browse/TEZ-3720

[jira] [Commented] (TEZ-3708) Improve parallelism and auto grouping of unpartitioned cartesian product

[ https://issues.apache.org/jira/browse/TEZ-3708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005671#comment-16005671 ] Siddharth Seth commented on TEZ-3708: - - {code} +minNumRecordForEstimation = + (long)

[jira] [Updated] (TEZ-3719) DAGImpl.computeProgress slows down dispatcher and ipc threads

[ https://issues.apache.org/jira/browse/TEZ-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated TEZ-3719: - Attachment: TEZ-3719.1.patch > DAGImpl.computeProgress slows down dispatcher and ipc threads >

[jira] [Commented] (TEZ-3719) DAGImpl.computeProgress slows down dispatcher and ipc threads

[ https://issues.apache.org/jira/browse/TEZ-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005429#comment-16005429 ] Gopal V commented on TEZ-3719: -- One of these getProgress() calls is coming from the AM-RM heartbeat, to display

[jira] [Commented] (TEZ-3712) Use Local FileContext for deleting dag level directories

[ https://issues.apache.org/jira/browse/TEZ-3712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005420#comment-16005420 ] Kuhu Shukla commented on TEZ-3712: -- Did some more analysis as to why we see discrepancy in the output of

[jira] [Commented] (TEZ-3718) Better handling of 'bad' nodes

[ https://issues.apache.org/jira/browse/TEZ-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005282#comment-16005282 ] Jason Lowe commented on TEZ-3718: - Last I checked, MapReduce doesn't have task duration factored into the

[jira] [Commented] (TEZ-3719) DAGImpl.computeProgress slows down dispatcher and ipc threads

[ https://issues.apache.org/jira/browse/TEZ-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005251#comment-16005251 ] Jonathan Eagles commented on TEZ-3719: -- I noticed this exact slow down when I was profiling tez

[jira] [Comment Edited] (TEZ-3718) Better handling of 'bad' nodes

[ https://issues.apache.org/jira/browse/TEZ-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005247#comment-16005247 ] Siddharth Seth edited comment on TEZ-3718 at 5/10/17 7:15 PM: -- bq. Not

[jira] [Commented] (TEZ-3718) Better handling of 'bad' nodes

[ https://issues.apache.org/jira/browse/TEZ-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16005247#comment-16005247 ] Siddharth Seth commented on TEZ-3718: - bq. Not specifically related to bad node handling, but we could

[jira] [Updated] (TEZ-3719) DAGImpl.computeProgress slows down dispatcher and ipc threads

[ https://issues.apache.org/jira/browse/TEZ-3719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-3719: -- Attachment: tez_computeProgress_waiting_for_lock.png dispatcher_holding_lock.png

[jira] [Created] (TEZ-3719) DAGImpl.computeProgress slows down dispatcher and ipc threads

Rajesh Balamohan created TEZ-3719: - Summary: DAGImpl.computeProgress slows down dispatcher and ipc threads Key: TEZ-3719 URL: https://issues.apache.org/jira/browse/TEZ-3719 Project: Apache Tez