[jira] [Commented] (TEZ-4128) Logging: Fix ArrayOutOfBound in PipelineSorter

[ https://issues.apache.org/jira/browse/TEZ-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17048695#comment-17048695 ] Rajesh Balamohan commented on TEZ-4128: --- Can you share the sort mb for this job? This

[jira] [Created] (TEZ-4135) Improve memory allocation when executing in-memory reads

Rajesh Balamohan created TEZ-4135: - Summary: Improve memory allocation when executing in-memory reads Key: TEZ-4135 URL: https://issues.apache.org/jira/browse/TEZ-4135 Project: Apache Tez Iss

[jira] [Commented] (TEZ-4135) Improve memory allocation when executing in-memory reads

[ https://issues.apache.org/jira/browse/TEZ-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17060232#comment-17060232 ] Rajesh Balamohan commented on TEZ-4135: --- Example scenario can be Q78 in Hive. > Impr

[jira] [Assigned] (TEZ-4087) Shuffle: Fix shuffle cleanup to prevent thread leaks

[ https://issues.apache.org/jira/browse/TEZ-4087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-4087: - Assignee: Rajesh Balamohan > Shuffle: Fix shuffle cleanup to prevent thread leaks > --

[jira] [Updated] (TEZ-4087) Shuffle: Fix shuffle cleanup to prevent thread leaks

[ https://issues.apache.org/jira/browse/TEZ-4087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4087: -- Attachment: TEZ-4087.1.patch > Shuffle: Fix shuffle cleanup to prevent thread leaks > ---

[jira] [Commented] (TEZ-4135) Improve memory allocation when executing in-memory reads

[ https://issues.apache.org/jira/browse/TEZ-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17063053#comment-17063053 ] Rajesh Balamohan commented on TEZ-4135: --- Sure, TEZ-3645 should be helpful as well in.

[jira] [Commented] (TEZ-4135) Improve memory allocation when executing in-memory reads

[ https://issues.apache.org/jira/browse/TEZ-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17064632#comment-17064632 ] Rajesh Balamohan commented on TEZ-4135: --- Yes, something like "((Configurable) codec).

[jira] [Created] (TEZ-4138) Provide an option to retain tez recovery file in job logs

Rajesh Balamohan created TEZ-4138: - Summary: Provide an option to retain tez recovery file in job logs Key: TEZ-4138 URL: https://issues.apache.org/jira/browse/TEZ-4138 Project: Apache Tez Is

[jira] [Created] (TEZ-4139) Tez should consider node information for computing failure fraction

Rajesh Balamohan created TEZ-4139: - Summary: Tez should consider node information for computing failure fraction Key: TEZ-4139 URL: https://issues.apache.org/jira/browse/TEZ-4139 Project: Apache Tez

[jira] [Commented] (TEZ-4139) Tez should consider node information for computing failure fraction

[ https://issues.apache.org/jira/browse/TEZ-4139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076762#comment-17076762 ] Rajesh Balamohan commented on TEZ-4139: --- For ref: Observed this when analyzing Q28

[jira] [Created] (TEZ-4143) Provide an option to disable DAG graph (.dot) generation for latency sensitive jobs

Rajesh Balamohan created TEZ-4143: - Summary: Provide an option to disable DAG graph (.dot) generation for latency sensitive jobs Key: TEZ-4143 URL: https://issues.apache.org/jira/browse/TEZ-4143 Proje

[jira] [Created] (TEZ-4145) Reduce lock contention in TezSpillRecord

Rajesh Balamohan created TEZ-4145: - Summary: Reduce lock contention in TezSpillRecord Key: TEZ-4145 URL: https://issues.apache.org/jira/browse/TEZ-4145 Project: Apache Tez Issue Type: Improve

[jira] [Commented] (TEZ-4145) Reduce lock contention in TezSpillRecord

[ https://issues.apache.org/jira/browse/TEZ-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082242#comment-17082242 ] Rajesh Balamohan commented on TEZ-4145: --- Note: TezSpillRecord is in the hot path in f

[jira] [Commented] (TEZ-4145) Reduce lock contention in TezSpillRecord

[ https://issues.apache.org/jira/browse/TEZ-4145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082758#comment-17082758 ] Rajesh Balamohan commented on TEZ-4145: --- Yes, {{TezSpillRecord::writeToFile}} would h

[jira] [Commented] (TEZ-4143) Provide an option to disable DAG graph (.dot) generation for latency sensitive jobs

[ https://issues.apache.org/jira/browse/TEZ-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17082760#comment-17082760 ] Rajesh Balamohan commented on TEZ-4143: --- Thanks for sharing the patch [~abstractdog]

[jira] [Created] (TEZ-4146) Register RUNNING state in DAG's state change callback

Rajesh Balamohan created TEZ-4146: - Summary: Register RUNNING state in DAG's state change callback Key: TEZ-4146 URL: https://issues.apache.org/jira/browse/TEZ-4146 Project: Apache Tez Issue

[jira] [Updated] (TEZ-4146) Register RUNNING state in DAG's state change callback

[ https://issues.apache.org/jira/browse/TEZ-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4146: -- Attachment: TEZ-4146.1.patch > Register RUNNING state in DAG's state change callback > --

[jira] [Assigned] (TEZ-4146) Register RUNNING state in DAG's state change callback

[ https://issues.apache.org/jira/browse/TEZ-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-4146: - Assignee: Rajesh Balamohan > Register RUNNING state in DAG's state change callback > -

[jira] [Updated] (TEZ-4146) Register RUNNING state in DAG's state change callback

[ https://issues.apache.org/jira/browse/TEZ-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4146: -- Attachment: TEZ-4146.2.patch > Register RUNNING state in DAG's state change callback > --

[jira] [Updated] (TEZ-4146) Register RUNNING state in DAG's state change callback

[ https://issues.apache.org/jira/browse/TEZ-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4146: -- Attachment: TEZ-4146.3.patch > Register RUNNING state in DAG's state change callback > --

[jira] [Commented] (TEZ-4146) Register RUNNING state in DAG's state change callback

[ https://issues.apache.org/jira/browse/TEZ-4146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083776#comment-17083776 ] Rajesh Balamohan commented on TEZ-4146: --- Thanks [~gopalv] . Addressed the comments in

[jira] [Commented] (TEZ-4143) Provide an option to disable DAG graph (.dot) generation for latency sensitive jobs

[ https://issues.apache.org/jira/browse/TEZ-4143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083779#comment-17083779 ] Rajesh Balamohan commented on TEZ-4143: --- +1. Thanks [~abstractdog]. > Provide an opt

[jira] [Commented] (TEZ-4139) Tez should consider node information for computing failure fraction

[ https://issues.apache.org/jira/browse/TEZ-4139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083818#comment-17083818 ] Rajesh Balamohan commented on TEZ-4139: --- [~abstractdog]: Thanks for sharing the wip p

[jira] [Created] (TEZ-4147) Reduce NN calls in RecoveryService::handleRecoveryEvent

Rajesh Balamohan created TEZ-4147: - Summary: Reduce NN calls in RecoveryService::handleRecoveryEvent Key: TEZ-4147 URL: https://issues.apache.org/jira/browse/TEZ-4147 Project: Apache Tez Issu

[jira] [Created] (TEZ-4155) Remove sync bottlneck in counters

Rajesh Balamohan created TEZ-4155: - Summary: Remove sync bottlneck in counters Key: TEZ-4155 URL: https://issues.apache.org/jira/browse/TEZ-4155 Project: Apache Tez Issue Type: Improvement

[jira] [Created] (TEZ-4156) Fix Tez to reuse IPC connections

Rajesh Balamohan created TEZ-4156: - Summary: Fix Tez to reuse IPC connections Key: TEZ-4156 URL: https://issues.apache.org/jira/browse/TEZ-4156 Project: Apache Tez Issue Type: Improvement

[jira] [Assigned] (TEZ-4156) Fix Tez to reuse IPC connections

[ https://issues.apache.org/jira/browse/TEZ-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-4156: - Assignee: Rajesh Balamohan > Fix Tez to reuse IPC connections > --

[jira] [Updated] (TEZ-4156) Fix Tez to reuse IPC connections

[ https://issues.apache.org/jira/browse/TEZ-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4156: -- Attachment: TEZ-4156.1.patch > Fix Tez to reuse IPC connections > ---

[jira] [Commented] (TEZ-4155) Remove sync bottleneck in counters

[ https://issues.apache.org/jira/browse/TEZ-4155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093163#comment-17093163 ] Rajesh Balamohan commented on TEZ-4155: --- LGTM. +1. Can you remove "violation()" metho

[jira] [Commented] (TEZ-4156) Fix Tez to reuse IPC connections

[ https://issues.apache.org/jira/browse/TEZ-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17093613#comment-17093613 ] Rajesh Balamohan commented on TEZ-4156: --- [~jeagles]: Yes, the cluster where I tried o

[jira] [Updated] (TEZ-4156) Fix Tez to reuse IPC connections

[ https://issues.apache.org/jira/browse/TEZ-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4156: -- Attachment: TEZ-4156.2.patch > Fix Tez to reuse IPC connections > ---

[jira] [Commented] (TEZ-4156) Fix Tez to reuse IPC connections

[ https://issues.apache.org/jira/browse/TEZ-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094077#comment-17094077 ] Rajesh Balamohan commented on TEZ-4156: --- [~sseth]: This is encountered frequently in

[jira] [Updated] (TEZ-4156) Fix Tez to reuse IPC connections

[ https://issues.apache.org/jira/browse/TEZ-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4156: -- Attachment: TEZ-4156.3.patch > Fix Tez to reuse IPC connections > ---

[jira] [Updated] (TEZ-4156) Fix Tez to reuse IPC connections

[ https://issues.apache.org/jira/browse/TEZ-4156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4156: -- Attachment: TEZ-4156.4.patch > Fix Tez to reuse IPC connections > ---

[jira] [Commented] (TEZ-4153) IndexCache totalMemoryUsed differs from cache contents.

[ https://issues.apache.org/jira/browse/TEZ-4153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17098771#comment-17098771 ] Rajesh Balamohan commented on TEZ-4153: --- [~jeagles]: LGTM. +1. Minor comment: Should

[jira] [Created] (TEZ-4170) RootInputInitializerManager could make use of ThreadPool from appContext

Rajesh Balamohan created TEZ-4170: - Summary: RootInputInitializerManager could make use of ThreadPool from appContext Key: TEZ-4170 URL: https://issues.apache.org/jira/browse/TEZ-4170 Project: Apache

[jira] [Created] (TEZ-4171) DAGImp::getDAGStatus should try to report RUNNING state information correctly

Rajesh Balamohan created TEZ-4171: - Summary: DAGImp::getDAGStatus should try to report RUNNING state information correctly Key: TEZ-4171 URL: https://issues.apache.org/jira/browse/TEZ-4171 Project: Ap

[jira] [Assigned] (TEZ-4171) DAGImp::getDAGStatus should try to report RUNNING state information correctly

[ https://issues.apache.org/jira/browse/TEZ-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-4171: - Assignee: Rajesh Balamohan > DAGImp::getDAGStatus should try to report RUNNING state infor

[jira] [Updated] (TEZ-4171) DAGImp::getDAGStatus should try to report RUNNING state information correctly

[ https://issues.apache.org/jira/browse/TEZ-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4171: -- Attachment: TEZ-4171.1.patch > DAGImp::getDAGStatus should try to report RUNNING state informatio

[jira] [Updated] (TEZ-4171) DAGImp::getDAGStatus should try to report RUNNING state information correctly

[ https://issues.apache.org/jira/browse/TEZ-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4171: -- Attachment: TEZ-4171.2.patch > DAGImp::getDAGStatus should try to report RUNNING state informatio

[jira] [Updated] (TEZ-4171) DAGImp::getDAGStatus should try to report RUNNING state information correctly

[ https://issues.apache.org/jira/browse/TEZ-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4171: -- Attachment: TEZ-4171.3.patch > DAGImp::getDAGStatus should try to report RUNNING state informatio

[jira] [Updated] (TEZ-4171) DAGImp::getDAGStatus should try to report RUNNING state information correctly

[ https://issues.apache.org/jira/browse/TEZ-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4171: -- Attachment: TEZ-4171.4.patch > DAGImp::getDAGStatus should try to report RUNNING state informatio

[jira] [Updated] (TEZ-4171) DAGImp::getDAGStatus should try to report RUNNING state information correctly

[ https://issues.apache.org/jira/browse/TEZ-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4171: -- Attachment: TEZ-4171.5.patch > DAGImp::getDAGStatus should try to report RUNNING state informatio

[jira] [Updated] (TEZ-4171) DAGImp::getDAGStatus should try to report RUNNING state information correctly

[ https://issues.apache.org/jira/browse/TEZ-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4171: -- Attachment: TEZ-4171.6.patch > DAGImp::getDAGStatus should try to report RUNNING state informatio

[jira] [Commented] (TEZ-4171) DAGImp::getDAGStatus should try to report RUNNING state information correctly

[ https://issues.apache.org/jira/browse/TEZ-4171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17102598#comment-17102598 ] Rajesh Balamohan commented on TEZ-4171: --- Thanks [~gopalv] , [~jeagles] . Yes, SUBMITT

[jira] [Commented] (TEZ-2672) Allow specifying a new payload for plugins when a new DAG starts

[ https://issues.apache.org/jira/browse/TEZ-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17103983#comment-17103983 ] Rajesh Balamohan commented on TEZ-2672: --- Thanks for sharing the patch [~abstractdog]

[jira] [Commented] (TEZ-4174) [Kubernetes] Fetcher should connection failure on SocketException

[ https://issues.apache.org/jira/browse/TEZ-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17104045#comment-17104045 ] Rajesh Balamohan commented on TEZ-4174: --- LGTM. +1. > [Kubernetes] Fetcher should con

[jira] [Commented] (TEZ-2672) Allow specifying a new payload for plugins when a new DAG starts

[ https://issues.apache.org/jira/browse/TEZ-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17104315#comment-17104315 ] Rajesh Balamohan commented on TEZ-2672: --- Thanks [~abstractdog]. Can {{Configuration g

[jira] [Commented] (TEZ-2672) Allow specifying a new payload for plugins when a new DAG starts

[ https://issues.apache.org/jira/browse/TEZ-2672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17105024#comment-17105024 ] Rajesh Balamohan commented on TEZ-2672: --- .4 patch LGTM. +1. Thanks [~abstractdog] >

[jira] [Commented] (TEZ-4174) [Kubernetes] Fetcher should connection failure on SocketException

[ https://issues.apache.org/jira/browse/TEZ-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17106747#comment-17106747 ] Rajesh Balamohan commented on TEZ-4174: --- Committed to master. Thanks [~prasanth_j] >

[jira] [Commented] (TEZ-4179) [Kubernetes] Extend NodeId in tez to support unique worker identity

[ https://issues.apache.org/jira/browse/TEZ-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110697#comment-17110697 ] Rajesh Balamohan commented on TEZ-4179: --- This would increase the mem footprint in AM

[jira] [Commented] (TEZ-4179) [Kubernetes] Extend NodeId in tez to support unique worker identity

[ https://issues.apache.org/jira/browse/TEZ-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17110767#comment-17110767 ] Rajesh Balamohan commented on TEZ-4179: --- Minor change before commit: Mark the files a

[jira] [Commented] (TEZ-4186) Limits: Fix init order regression from TEZ-4155

[ https://issues.apache.org/jira/browse/TEZ-4186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17112754#comment-17112754 ] Rajesh Balamohan commented on TEZ-4186: --- +1. Thanks for the patch [~gopalv] .  > Lim

[jira] [Commented] (TEZ-4170) RootInputInitializerManager could make use of ThreadPool from appContext

[ https://issues.apache.org/jira/browse/TEZ-4170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17118331#comment-17118331 ] Rajesh Balamohan commented on TEZ-4170: --- Thanks for sharing the patch. 1. handleInit

[jira] [Created] (TEZ-4194) NPE in FetcherOrderedGrouped

Rajesh Balamohan created TEZ-4194: - Summary: NPE in FetcherOrderedGrouped Key: TEZ-4194 URL: https://issues.apache.org/jira/browse/TEZ-4194 Project: Apache Tez Issue Type: Bug Affects Ve

[jira] [Updated] (TEZ-4194) NPE in FetcherOrderedGrouped

[ https://issues.apache.org/jira/browse/TEZ-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4194: -- Attachment: NPE_TASK_syslog_attempt_1592898862823_0002_1_01_000120_0_apache.log > NPE in Fetcher

[jira] [Created] (TEZ-4199) MergeManager::finalMerge should make use of compression

Rajesh Balamohan created TEZ-4199: - Summary: MergeManager::finalMerge should make use of compression Key: TEZ-4199 URL: https://issues.apache.org/jira/browse/TEZ-4199 Project: Apache Tez Iss

[jira] [Commented] (TEZ-4203) Findbugs: MergeThread.shuffleSchedulerThread; locked 80% of time

[ https://issues.apache.org/jira/browse/TEZ-4203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17162529#comment-17162529 ] Rajesh Balamohan commented on TEZ-4203: --- This isn't a real sync issue. "shuffleSchedu

[jira] [Commented] (TEZ-4128) Logging: Fix ArrayOutOfBound in PipelineSorter

[ https://issues.apache.org/jira/browse/TEZ-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17163447#comment-17163447 ] Rajesh Balamohan commented on TEZ-4128: --- [~rameshkumar]: Is this still an issue? I be

[jira] [Commented] (TEZ-4175) Consider removing YarnConfiguration where it's possible

[ https://issues.apache.org/jira/browse/TEZ-4175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17164098#comment-17164098 ] Rajesh Balamohan commented on TEZ-4175: --- [~abstractdog] , thanks for sharing the patc

[jira] [Moved] (TEZ-4207) Provide approximate number of input records to be processed in broadcast reader

[ https://issues.apache.org/jira/browse/TEZ-4207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan moved HIVE-23936 to TEZ-4207: -- Key: TEZ-4207 (was: HIVE-23936) Project: Apache Tez (was: Hive)

[jira] [Updated] (TEZ-4207) Provide approximate number of input records to be processed in UnorderedKVInput

[ https://issues.apache.org/jira/browse/TEZ-4207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4207: -- Summary: Provide approximate number of input records to be processed in UnorderedKVInput (was: P

[jira] [Updated] (TEZ-4207) Provide approximate number of input records to be processed in UnorderedKVInput

[ https://issues.apache.org/jira/browse/TEZ-4207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4207: -- Attachment: TEZ-4207.wip.patch > Provide approximate number of input records to be processed in

[jira] [Updated] (TEZ-4208) Pipelinesorter uses single SortSpan after spill

[ https://issues.apache.org/jira/browse/TEZ-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4208: -- Attachment: q67_sorter.log > Pipelinesorter uses single SortSpan after spill > --

[jira] [Created] (TEZ-4208) Pipelinesorter uses single SortSpan after spill

Rajesh Balamohan created TEZ-4208: - Summary: Pipelinesorter uses single SortSpan after spill Key: TEZ-4208 URL: https://issues.apache.org/jira/browse/TEZ-4208 Project: Apache Tez Issue Type:

[jira] [Updated] (TEZ-4208) Pipelinesorter uses single SortSpan after spill

[ https://issues.apache.org/jira/browse/TEZ-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4208: -- Attachment: TEZ-4208.1.patch > Pipelinesorter uses single SortSpan after spill >

[jira] [Commented] (TEZ-4208) Pipelinesorter uses single SortSpan after spill

[ https://issues.apache.org/jira/browse/TEZ-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166857#comment-17166857 ] Rajesh Balamohan commented on TEZ-4208: --- Q67 runtime with/without patch in internal c

[jira] [Comment Edited] (TEZ-4208) Pipelinesorter uses single SortSpan after spill

[ https://issues.apache.org/jira/browse/TEZ-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166857#comment-17166857 ] Rajesh Balamohan edited comment on TEZ-4208 at 7/29/20, 3:59 AM:

[jira] [Updated] (TEZ-4203) Findbugs: MergeThread.shuffleSchedulerThread; locked 80% of time

[ https://issues.apache.org/jira/browse/TEZ-4203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4203: -- Attachment: TEZ-4203.1.patch > Findbugs: MergeThread.shuffleSchedulerThread; locked 80% of time >

[jira] [Updated] (TEZ-4207) Provide approximate number of input records to be processed in UnorderedKVInput

[ https://issues.apache.org/jira/browse/TEZ-4207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4207: -- Attachment: TEZ-4207.1.patch > Provide approximate number of input records to be processed in >

[jira] [Created] (TEZ-4209) Use task counter information to compute keycount during hashtable loading

Rajesh Balamohan created TEZ-4209: - Summary: Use task counter information to compute keycount during hashtable loading Key: TEZ-4209 URL: https://issues.apache.org/jira/browse/TEZ-4209 Project: Apache

[jira] [Resolved] (TEZ-4209) Use task counter information to compute keycount during hashtable loading

[ https://issues.apache.org/jira/browse/TEZ-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved TEZ-4209. --- Resolution: Won't Fix > Use task counter information to compute keycount during hashtable loadi

[jira] [Commented] (TEZ-4209) Use task counter information to compute keycount during hashtable loading

[ https://issues.apache.org/jira/browse/TEZ-4209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17167635#comment-17167635 ] Rajesh Balamohan commented on TEZ-4209: --- Supposed to be created in Hive project. Igno

[jira] [Created] (TEZ-4210) Use task counter information to compute keycount during hashtable loading

Rajesh Balamohan created TEZ-4210: - Summary: Use task counter information to compute keycount during hashtable loading Key: TEZ-4210 URL: https://issues.apache.org/jira/browse/TEZ-4210 Project: Apache

[jira] [Resolved] (TEZ-4210) Use task counter information to compute keycount during hashtable loading

[ https://issues.apache.org/jira/browse/TEZ-4210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan resolved TEZ-4210. --- Resolution: Won't Fix > Use task counter information to compute keycount during hashtable loadi

[jira] [Created] (TEZ-4211) Optimise MergeManager final merge

Rajesh Balamohan created TEZ-4211: - Summary: Optimise MergeManager final merge Key: TEZ-4211 URL: https://issues.apache.org/jira/browse/TEZ-4211 Project: Apache Tez Issue Type: Bug

[jira] [Updated] (TEZ-4211) Optimise MergeManager final merge

[ https://issues.apache.org/jira/browse/TEZ-4211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4211: -- Attachment: TEZ-4211.wip.patch > Optimise MergeManager final merge >

[jira] [Commented] (TEZ-4211) Optimise MergeManager final merge

[ https://issues.apache.org/jira/browse/TEZ-4211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17167729#comment-17167729 ] Rajesh Balamohan commented on TEZ-4211: --- Attaching wip patch > Optimise MergeManager

[jira] [Updated] (TEZ-4211) Optimise MergeManager final merge

[ https://issues.apache.org/jira/browse/TEZ-4211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4211: -- Attachment: TEZ-4211.2.patch > Optimise MergeManager final merge > --

[jira] [Updated] (TEZ-4208) Pipelinesorter uses single SortSpan after spill

[ https://issues.apache.org/jira/browse/TEZ-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4208: -- Attachment: TEZ-4208.2.patch > Pipelinesorter uses single SortSpan after spill >

[jira] [Commented] (TEZ-4208) Pipelinesorter uses single SortSpan after spill

[ https://issues.apache.org/jira/browse/TEZ-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17170723#comment-17170723 ] Rajesh Balamohan commented on TEZ-4208: --- Attaching .2 patch with test case. > Pipeli

[jira] [Created] (TEZ-4216) RLE check in MergeManager::finalMerge could be disabled

Rajesh Balamohan created TEZ-4216: - Summary: RLE check in MergeManager::finalMerge could be disabled Key: TEZ-4216 URL: https://issues.apache.org/jira/browse/TEZ-4216 Project: Apache Tez Issu

[jira] [Updated] (TEZ-4216) RLE check in MergeManager::finalMerge could be disabled

[ https://issues.apache.org/jira/browse/TEZ-4216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4216: -- Attachment: TEZ-4216.1.patch > RLE check in MergeManager::finalMerge could be disabled >

[jira] [Created] (TEZ-4222) Sync issues during IFile::Writer init phase due to SerializationFactory

Rajesh Balamohan created TEZ-4222: - Summary: Sync issues during IFile::Writer init phase due to SerializationFactory Key: TEZ-4222 URL: https://issues.apache.org/jira/browse/TEZ-4222 Project: Apache T

[jira] [Commented] (TEZ-4211) Optimise MergeManager final merge

[ https://issues.apache.org/jira/browse/TEZ-4211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17172227#comment-17172227 ] Rajesh Balamohan commented on TEZ-4211: --- Thanks for the note [~abstractdog] . I had o

[jira] [Updated] (TEZ-4222) Sync issues during IFile::Writer init phase due to SerializationFactory

[ https://issues.apache.org/jira/browse/TEZ-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4222: -- Attachment: image-2020-08-07-06-49-38-729.png > Sync issues during IFile::Writer init phase due t

[jira] [Commented] (TEZ-4222) Sync issues during IFile::Writer init phase due to SerializationFactory

[ https://issues.apache.org/jira/browse/TEZ-4222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17172762#comment-17172762 ] Rajesh Balamohan commented on TEZ-4222: --- Yes [~jeagles] , TEZ-3645 fixes the issue.

[jira] [Commented] (TEZ-3645) Reuse SerializationFactory while sorting, merging, and writing IFiles

[ https://issues.apache.org/jira/browse/TEZ-3645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17172763#comment-17172763 ] Rajesh Balamohan commented on TEZ-3645: --- [~jeagles]: May have to rebase the patch for

[jira] [Commented] (TEZ-4223) Adding new jars or resources after the first DAG runs does not work.

[ https://issues.apache.org/jira/browse/TEZ-4223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176731#comment-17176731 ] Rajesh Balamohan commented on TEZ-4223: --- LGTM. +1 > Adding new jars or resources aft

[jira] [Commented] (TEZ-4207) Provide approximate number of input records to be processed in UnorderedKVInput

[ https://issues.apache.org/jira/browse/TEZ-4207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17176856#comment-17176856 ] Rajesh Balamohan commented on TEZ-4207: --- Thanks for the review [~ashutoshc]. Committe

[jira] [Assigned] (TEZ-4207) Provide approximate number of input records to be processed in UnorderedKVInput

[ https://issues.apache.org/jira/browse/TEZ-4207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan reassigned TEZ-4207: - Fix Version/s: 0.10.1 Assignee: Rajesh Balamohan Resolution: Fixed > Provi

[jira] [Commented] (TEZ-4233) Map task should be blamed earlier for local fetch failures

[ https://issues.apache.org/jira/browse/TEZ-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17197624#comment-17197624 ] Rajesh Balamohan commented on TEZ-4233: --- Thanks [~abstractdog]  for the patch. I went

[jira] [Commented] (TEZ-4233) Map task should be blamed earlier for local fetch failures

[ https://issues.apache.org/jira/browse/TEZ-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200433#comment-17200433 ] Rajesh Balamohan commented on TEZ-4233: --- Thanks [~abstractdog] for the revised patch.

[jira] [Commented] (TEZ-4233) Map task should be blamed earlier for local fetch failures

[ https://issues.apache.org/jira/browse/TEZ-4233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17201142#comment-17201142 ] Rajesh Balamohan commented on TEZ-4233: --- LGTM. +1. > Map task should be blamed earli

[jira] [Commented] (TEZ-4234) Compressor can cause IllegalArgumentException in Buffer.limit where limit exceeds capacity

[ https://issues.apache.org/jira/browse/TEZ-4234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17208010#comment-17208010 ] Rajesh Balamohan commented on TEZ-4234: --- LGTM. +1. Resetting the conf in the codec

[jira] [Created] (TEZ-4244) Consider using RawLocalFileSystem in LocalDiskFetchedInput

Rajesh Balamohan created TEZ-4244: - Summary: Consider using RawLocalFileSystem in LocalDiskFetchedInput Key: TEZ-4244 URL: https://issues.apache.org/jira/browse/TEZ-4244 Project: Apache Tez I

[jira] [Created] (TEZ-4245) Optimise split grouping when locality information is set to null/empty

Rajesh Balamohan created TEZ-4245: - Summary: Optimise split grouping when locality information is set to null/empty Key: TEZ-4245 URL: https://issues.apache.org/jira/browse/TEZ-4245 Project: Apache Te

[jira] [Updated] (TEZ-4245) Optimise split grouping when locality information is set to null/empty

[ https://issues.apache.org/jira/browse/TEZ-4245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated TEZ-4245: -- Attachment: TEZ-4245.1.patch > Optimise split grouping when locality information is set to null/e

[jira] [Commented] (TEZ-4245) Optimise split grouping when locality information is set to null/empty

[ https://issues.apache.org/jira/browse/TEZ-4245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17221238#comment-17221238 ] Rajesh Balamohan commented on TEZ-4245: --- PR: [https://github.com/apache/tez/pull/78]

[jira] [Commented] (TEZ-4246) Avoid uneven local disk usage for spills

[ https://issues.apache.org/jira/browse/TEZ-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17228365#comment-17228365 ] Rajesh Balamohan commented on TEZ-4246: --- Can you share more details on this [~okumin]

  1   2   3   4   5   6   7   8   9   10   >