[
https://issues.apache.org/jira/browse/TEZ-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16221394#comment-16221394
]
Ming Ma commented on TEZ-3666:
--
Thanks [~aplusplus]!
> Integer overflow in ShuffleVertexManage
[
https://issues.apache.org/jira/browse/TEZ-3818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma reassigned TEZ-3818:
Assignee: Ming Ma
> Support a new data routing policy for small partitions
> ---
[
https://issues.apache.org/jira/browse/TEZ-3818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3818:
-
Attachment: TEZ-3818.patch
Here is the draft patch. Besides the added unit test, I tested on a hadoop
cluster with
Ming Ma created TEZ-3818:
Summary: Support a new data routing policy for small partitions
Key: TEZ-3818
URL: https://issues.apache.org/jira/browse/TEZ-3818
Project: Apache Tez
Issue Type: Sub-task
[
https://issues.apache.org/jira/browse/TEZ-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3722:
-
Attachment: TEZ-3722.patch
Here is the draft patch. Applications will first get the configuration object
created by
Ming Ma created TEZ-3722:
Summary: Have MultiMROutput support different configurations for
different writers
Key: TEZ-3722
URL: https://issues.apache.org/jira/browse/TEZ-3722
Project: Apache Tez
Iss
[
https://issues.apache.org/jira/browse/TEZ-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3666:
-
Attachment: TEZ-3666-2.patch
Thanks [~aplusplus]! Here is the updated patch to address your comments.
Regarding the
Ming Ma created TEZ-3676:
Summary: Desired number of tasks in TezGroupedSplitsInputFormat
could be negative
Key: TEZ-3676
URL: https://issues.apache.org/jira/browse/TEZ-3676
Project: Apache Tez
Issu
[
https://issues.apache.org/jira/browse/TEZ-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3666:
-
Attachment: TEZ-3666.patch
Here is the draft patch. The code path with the issue is only used by
FairShuffleVertexM
[
https://issues.apache.org/jira/browse/TEZ-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma reassigned TEZ-3666:
Assignee: Ming Ma
> Integer overflow in ShuffleVertexManagerBase
> --
Ming Ma created TEZ-3666:
Summary: Integer overflow in ShuffleVertexManagerBase
Key: TEZ-3666
URL: https://issues.apache.org/jira/browse/TEZ-3666
Project: Apache Tez
Issue Type: Bug
Repor
[
https://issues.apache.org/jira/browse/TEZ-3588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15838238#comment-15838238
]
Ming Ma commented on TEZ-3588:
--
Configuration of NM metrics is a global change that applies to
[
https://issues.apache.org/jira/browse/TEZ-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15832276#comment-15832276
]
Ming Ma commented on TEZ-3458:
--
Thanks [~kshukla]. It seems TestFaultTolerance have issue witho
[
https://issues.apache.org/jira/browse/TEZ-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15827090#comment-15827090
]
Ming Ma commented on TEZ-3458:
--
[~aplusplus] apologies for the delay. +1 for the patch. Can you
Ming Ma created TEZ-3578:
Summary: Enable RLE during merge in TezMerger
Key: TEZ-3578
URL: https://issues.apache.org/jira/browse/TEZ-3578
Project: Apache Tez
Issue Type: Improvement
Repor
Ming Ma created TEZ-3577:
Summary: DefaultSorter doesn't compute RLE properly
Key: TEZ-3577
URL: https://issues.apache.org/jira/browse/TEZ-3577
Project: Apache Tez
Issue Type: Bug
Reporte
[
https://issues.apache.org/jira/browse/TEZ-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3552:
-
Assignee: Zhiyuan Yang (was: Ming Ma)
Thanks [~aplusplus]! Given you provided the fix, assign the jira to you.
> S
[
https://issues.apache.org/jira/browse/TEZ-2132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15727703#comment-15727703
]
Ming Ma commented on TEZ-2132:
--
[~rajesh.balamohan], any plan to work on it? It appears the iss
[
https://issues.apache.org/jira/browse/TEZ-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15727694#comment-15727694
]
Ming Ma commented on TEZ-3552:
--
[~aplusplus] any comment on the unit tests? Please modify if ne
[
https://issues.apache.org/jira/browse/TEZ-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3552:
-
Attachment: (was: TEZ-3552-2.patch)
> Shuffle split array when size-based sorting is turned off
> --
[
https://issues.apache.org/jira/browse/TEZ-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3552:
-
Attachment: TEZ-3552-2.patch
> Shuffle split array when size-based sorting is turned off
> -
[
https://issues.apache.org/jira/browse/TEZ-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3552:
-
Attachment: TEZ-3552-2.patch
oh, I didn't upload the patch. Thanks [~aplusplus]. To get the unit test pass
and veri
[
https://issues.apache.org/jira/browse/TEZ-3222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15717500#comment-15717500
]
Ming Ma commented on TEZ-3222:
--
+1. Thanks [~jeagles].
> Reduce messaging overhead for auto-re
[
https://issues.apache.org/jira/browse/TEZ-3222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15710215#comment-15710215
]
Ming Ma commented on TEZ-3222:
--
Thanks for the clarification!
For the wording for "CompositeRo
Ming Ma created TEZ-3552:
Summary: Shuffle split array when size-based sorting is turned off
Key: TEZ-3552
URL: https://issues.apache.org/jira/browse/TEZ-3552
Project: Apache Tez
Issue Type: Improvem
[
https://issues.apache.org/jira/browse/TEZ-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3430:
-
Issue Type: Improvement (was: Bug)
> Make split sorting optional
> ---
>
>
[
https://issues.apache.org/jira/browse/TEZ-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15685626#comment-15685626
]
Ming Ma commented on TEZ-3463:
--
We also want MR jars to be excluded for the same reason [~jeagl
[
https://issues.apache.org/jira/browse/TEZ-3222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677445#comment-15677445
]
Ming Ma edited comment on TEZ-3222 at 11/18/16 6:56 PM:
[~jeagles],
[
https://issues.apache.org/jira/browse/TEZ-3222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15677445#comment-15677445
]
Ming Ma commented on TEZ-3222:
--
[~jeagles], the fair routing part looks good. Couple questions:
[
https://issues.apache.org/jira/browse/TEZ-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15665901#comment-15665901
]
Ming Ma commented on TEZ-3458:
--
Thanks [~aplusplus].
* Grouper abstraction is nice. Maybe
Fai
[
https://issues.apache.org/jira/browse/TEZ-2104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15649457#comment-15649457
]
Ming Ma commented on TEZ-2104:
--
ok. Thanks for the jira. It is useful.
> A CrossProductEdge wh
[
https://issues.apache.org/jira/browse/TEZ-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15648157#comment-15648157
]
Ming Ma commented on TEZ-3477:
--
+1. Thanks!
> MRInputHelpers generateInputSplitsToMem public A
[
https://issues.apache.org/jira/browse/TEZ-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646080#comment-15646080
]
Ming Ma commented on TEZ-3477:
--
Thanks [~jeagles] for the clarification. Is the missing annotat
[
https://issues.apache.org/jira/browse/TEZ-2104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15646078#comment-15646078
]
Ming Ma commented on TEZ-2104:
--
[~aplusplus] is there a scenario where applications want to do
[
https://issues.apache.org/jira/browse/TEZ-3465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15643079#comment-15643079
]
Ming Ma commented on TEZ-3465:
--
+1.
BTW, do you expect that is how applications will do cartes
[
https://issues.apache.org/jira/browse/TEZ-2442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637351#comment-15637351
]
Ming Ma commented on TEZ-2442:
--
Nice work.
bq. The doc mentioned "For skewed intermediate outp
[
https://issues.apache.org/jira/browse/TEZ-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15637341#comment-15637341
]
Ming Ma commented on TEZ-3458:
--
Cool. For the partitioned case, do we also need to do something
[
https://issues.apache.org/jira/browse/TEZ-3465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15623698#comment-15623698
]
Ming Ma commented on TEZ-3465:
--
[~aplusplus] sorry for the delay. The patch looks good overall.
[
https://issues.apache.org/jira/browse/TEZ-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3269:
-
Attachment: TEZ-3269-5.patch
Update findbugs-exclude.xml as it isn't a real issue.
> Provide basic fair routing and
[
https://issues.apache.org/jira/browse/TEZ-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15621405#comment-15621405
]
Ming Ma edited comment on TEZ-3269 at 10/31/16 6:29 AM:
Thanks [~sse
[
https://issues.apache.org/jira/browse/TEZ-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15621405#comment-15621405
]
Ming Ma edited comment on TEZ-3269 at 10/31/16 6:28 AM:
Thanks [~sse
[
https://issues.apache.org/jira/browse/TEZ-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3269:
-
Attachment: TEZ-3269-4.patch
Thanks @sseth.
bq. final where possible
Fixed.
bq. Would be good to have some more doc
[
https://issues.apache.org/jira/browse/TEZ-3500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3500:
-
Summary: Fair routing support for multiple source vertices (was: Support
for multiple source vertices)
> Fair rout
Ming Ma created TEZ-3500:
Summary: Support for multiple source vertices
Key: TEZ-3500
URL: https://issues.apache.org/jira/browse/TEZ-3500
Project: Apache Tez
Issue Type: Sub-task
Reporter
[
https://issues.apache.org/jira/browse/TEZ-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15613988#comment-15613988
]
Ming Ma commented on TEZ-3477:
--
[~jeagles] sorry about that.
* The patch added protected crea
[
https://issues.apache.org/jira/browse/TEZ-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3215:
-
Attachment: TEZ-3215-6.patch
Thanks [~sseth]. Regarding OutputCommitter, it should be ok as long as basePath
specif
[
https://issues.apache.org/jira/browse/TEZ-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585906#comment-15585906
]
Ming Ma commented on TEZ-3452:
--
+1. Thanks [~jeagles].
> Auto-reduce parallelism calculation c
[
https://issues.apache.org/jira/browse/TEZ-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15569234#comment-15569234
]
Ming Ma commented on TEZ-3269:
--
[~aplusplus] [~sseth] or anyone else, any additional comments?
[
https://issues.apache.org/jira/browse/TEZ-3452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15550811#comment-15550811
]
Ming Ma commented on TEZ-3452:
--
[~jeagles], perhaps BigInteger could also address the issue. Wh
[
https://issues.apache.org/jira/browse/TEZ-3454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15550618#comment-15550618
]
Ming Ma commented on TEZ-3454:
--
[~darion], is it for the unordered shuffle scenario? It will be
[
https://issues.apache.org/jira/browse/TEZ-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3269:
-
Attachment: TEZ-3269-3.patch
Thank you [~aplusplus]! The new patch has addressed the issues you raised.
Regarding "
[
https://issues.apache.org/jira/browse/TEZ-3000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3000:
-
Hadoop Flags: Reviewed
> Fix TestContainerReuse
> --
>
> Key: TEZ-3000
>
[
https://issues.apache.org/jira/browse/TEZ-3000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3000:
-
Summary: Fix TestContainerReuse (was:
TestContainerReuse.testReuseWithTaskSpecificLaunchCmdOption fails)
> Fix Tes
[
https://issues.apache.org/jira/browse/TEZ-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15508274#comment-15508274
]
Ming Ma commented on TEZ-3429:
--
Thanks [~sseth]! Per
https://builds.apache.org/job/PreCommit-T
[
https://issues.apache.org/jira/browse/TEZ-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15507477#comment-15507477
]
Ming Ma commented on TEZ-3215:
--
[~aplusplus] [~hitesh] any additional comments?
> Support for
[
https://issues.apache.org/jira/browse/TEZ-3000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3000:
-
Attachment: TEZ-3000-2.patch
Updated patch to exclude the find bug warning.
> TestContainerReuse.testReuseWithTaskS
[
https://issues.apache.org/jira/browse/TEZ-2905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma resolved TEZ-2905.
--
Resolution: Duplicate
> TestContainerReuse fails in jenkins
> ---
>
>
[
https://issues.apache.org/jira/browse/TEZ-3000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3000:
-
Attachment: TEZ-3000.patch
Linked another similar jira. Here is the draft path. There are couple issues.
* {{TestTa
[
https://issues.apache.org/jira/browse/TEZ-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3430:
-
Attachment: TEZ-3430.patch
Here is the draft patch. It adds a new option sortSplitsEnabled to
configuration.
> Mak
[
https://issues.apache.org/jira/browse/TEZ-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3429:
-
Attachment: TEZ-3429.patch
Here is the draft patch. It also enables info level logging during recovery
which should
[
https://issues.apache.org/jira/browse/TEZ-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3215:
-
Attachment: TEZ-3215-5.patch
bq. Why do we want to remove this line in MROutput.KeyValueWriter.write?
Yeah, I also
[
https://issues.apache.org/jira/browse/TEZ-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3215:
-
Attachment: TEZ-3215-4.patch
Thanks [~aplusplus]!
bq. LAZY_OUTPUTFORMAT_OUTPUTFORMAT looks like a config for mapred
[
https://issues.apache.org/jira/browse/TEZ-3215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3215:
-
Attachment: TEZ-3215-3.patch
The failed test is unrelated. Here is the rebased patch.
> Support for MultipleOutputs
Ming Ma created TEZ-3430:
Summary: Make split sorting optional
Key: TEZ-3430
URL: https://issues.apache.org/jira/browse/TEZ-3430
Project: Apache Tez
Issue Type: Bug
Reporter: Ming Ma
Th
Ming Ma created TEZ-3429:
Summary: Set reconfigureDoneTime on VertexConfigurationDoneEvent
properly
Key: TEZ-3429
URL: https://issues.apache.org/jira/browse/TEZ-3429
Project: Apache Tez
Issue Type:
[
https://issues.apache.org/jira/browse/TEZ-3239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma resolved TEZ-3239.
--
Resolution: Invalid
Verified that the issue no longer exists in the master branch.
> ShuffleVertexManager recover
[
https://issues.apache.org/jira/browse/TEZ-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3269:
-
Attachment: TEZ-3269-2.patch
Here is the updated patch to include the followings:
* exclude file update to fix find
[
https://issues.apache.org/jira/browse/TEZ-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15459809#comment-15459809
]
Ming Ma commented on TEZ-3230:
--
+1. I will commit it earlier next week in case others have comm
[
https://issues.apache.org/jira/browse/TEZ-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma resolved TEZ-3427.
--
Resolution: Fixed
Hadoop Flags: Reviewed
Thanks [~hitesh]. I have committed it to the master branch.
> Add
[
https://issues.apache.org/jira/browse/TEZ-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3427:
-
Attachment: TEZ-3427.patch
Verified per
https://cwiki.apache.org/confluence/display/TEZ/Updating+the+Tez+Website. C
Ming Ma created TEZ-3427:
Summary: Add committer mingma to the Tez Team List
Key: TEZ-3427
URL: https://issues.apache.org/jira/browse/TEZ-3427
Project: Apache Tez
Issue Type: Task
Reporte
[
https://issues.apache.org/jira/browse/TEZ-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15453559#comment-15453559
]
Ming Ma commented on TEZ-3230:
--
[~aplusplus] the latest patch looks good. One minor thing, in
[
https://issues.apache.org/jira/browse/TEZ-3269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3269:
-
Attachment: TEZ-3269.patch
Here is the draft patch. It supports two polices w.r.t. fair routing.
* One policy is au
[
https://issues.apache.org/jira/browse/TEZ-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15429095#comment-15429095
]
Ming Ma commented on TEZ-3230:
--
Thanks [~aplusplus]. Couple more comments:
bq. But since there
[
https://issues.apache.org/jira/browse/TEZ-3395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15428666#comment-15428666
]
Ming Ma commented on TEZ-3395:
--
Thanks [~sseth]!
> Refactor ShuffleVertexManager to make parts
[
https://issues.apache.org/jira/browse/TEZ-3395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15428543#comment-15428543
]
Ming Ma commented on TEZ-3395:
--
Oops, yes please remove it. Thank you!
> Refactor ShuffleVerte
[
https://issues.apache.org/jira/browse/TEZ-3395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3395:
-
Attachment: TEZ-3395-4.patch
Thanks [~sseth]. Here is the patch with findbugs-exclude.xml's update.
> Refactor Shuf
[
https://issues.apache.org/jira/browse/TEZ-3395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3395:
-
Attachment: TEZ-3395-3.patch
Test failure and javac warnings aren't related. The synchronization issue in
findbugs
[
https://issues.apache.org/jira/browse/TEZ-3395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3395:
-
Attachment: TEZ-3395-2.patch
Thanks [~sseth].
bq. ReconfigVertexParams - static class, and final params
fixed
bq.
[
https://issues.apache.org/jira/browse/TEZ-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419756#comment-15419756
]
Ming Ma commented on TEZ-3230:
--
About "Only on demand routing" question, I am trying to underst
[
https://issues.apache.org/jira/browse/TEZ-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419672#comment-15419672
]
Ming Ma edited comment on TEZ-3230 at 8/12/16 11:17 PM:
Thanks [~apl
[
https://issues.apache.org/jira/browse/TEZ-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419672#comment-15419672
]
Ming Ma commented on TEZ-3230:
--
bq. For now, cDME is more prevalent given it won't consume too
[
https://issues.apache.org/jira/browse/TEZ-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15417693#comment-15417693
]
Ming Ma commented on TEZ-3209:
--
Thanks [~aplusplus]! Yes you are right about FairShuffleEdgeMan
[
https://issues.apache.org/jira/browse/TEZ-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413956#comment-15413956
]
Ming Ma commented on TEZ-3230:
--
Couple suggestions w.r.t. EdgeManager:
* {{CartesianProductEdg
[
https://issues.apache.org/jira/browse/TEZ-3397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15413834#comment-15413834
]
Ming Ma commented on TEZ-3397:
--
Is that only for specific custom edge scenario or it applies to
[
https://issues.apache.org/jira/browse/TEZ-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15412516#comment-15412516
]
Ming Ma commented on TEZ-3369:
--
bq. For task level progress, we could do the same - just need t
[
https://issues.apache.org/jira/browse/TEZ-3230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15409872#comment-15409872
]
Ming Ma commented on TEZ-3230:
--
Thanks [~aplusplus]. Nice patch! Some comments are about design
[
https://issues.apache.org/jira/browse/TEZ-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15404939#comment-15404939
]
Ming Ma commented on TEZ-3209:
--
Thanks [~sseth]. The refactoring work will be done in TEZ-3395.
[
https://issues.apache.org/jira/browse/TEZ-3395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3395:
-
Attachment: TEZ-3395.patch
Here is the draft patch. The major changes are:
* Move event handling to {{ShuffleVertex
[
https://issues.apache.org/jira/browse/TEZ-3395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3395:
-
Summary: Refactor ShuffleVertexManager (was: refactor ShuffleVertexManager)
> Refactor ShuffleVertexManager
> -
Ming Ma created TEZ-3395:
Summary: refactor ShuffleVertexManager
Key: TEZ-3395
URL: https://issues.apache.org/jira/browse/TEZ-3395
Project: Apache Tez
Issue Type: Sub-task
Reporter: Ming
[
https://issues.apache.org/jira/browse/TEZ-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15402906#comment-15402906
]
Ming Ma commented on TEZ-3303:
--
Nit: The TODO comment for TEZ_RUNTIME_REPORT_PARTITION_STATS in
[
https://issues.apache.org/jira/browse/TEZ-3113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15394807#comment-15394807
]
Ming Ma commented on TEZ-3113:
--
We also ran into similar issue.
The pipeline sorter setting:
[
https://issues.apache.org/jira/browse/TEZ-3209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15385449#comment-15385449
]
Ming Ma commented on TEZ-3209:
--
Thanks [~sseth]!
bq. primarily targeted towards Unordered Data
[
https://issues.apache.org/jira/browse/TEZ-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15379771#comment-15379771
]
Ming Ma commented on TEZ-3331:
--
[~hitesh], thanks for the explanation. So to make this patch co
[
https://issues.apache.org/jira/browse/TEZ-3331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15373947#comment-15373947
]
Ming Ma commented on TEZ-3331:
--
[~hitesh] thanks for the info about hadoop shim.
bq. Mind addi
[
https://issues.apache.org/jira/browse/TEZ-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated TEZ-3340:
-
Description:
YARN provides shared cache in functionality YARN-1492. According to [~ctrezzo]
most of the YARN functi
Ming Ma created TEZ-3342:
Summary: Have Tez AM generate thread dump on task attempts timeout
before killing them
Key: TEZ-3342
URL: https://issues.apache.org/jira/browse/TEZ-3342
Project: Apache Tez
Ming Ma created TEZ-3341:
Summary: Add thread dump support to Tez webUI
Key: TEZ-3341
URL: https://issues.apache.org/jira/browse/TEZ-3341
Project: Apache Tez
Issue Type: Improvement
Repor
[
https://issues.apache.org/jira/browse/TEZ-3340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma moved YARN-5365 to TEZ-3340:
Key: TEZ-3340 (was: YARN-5365)
Project: Apache Tez (was: Hadoop YARN)
> Add support
1 - 100 of 179 matches
Mail list logo
