Muhammad Samir Khan created TEZ-3753:
Summary: Improve error message in IFile for buffer length overflow
Key: TEZ-3753
URL: https://issues.apache.org/jira/browse/TEZ-3753
Project: Apache Tez
[
https://issues.apache.org/jira/browse/TEZ-3753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan reassigned TEZ-3753:
Assignee: Muhammad Samir Khan
> Improve error message in IFile for buffer length over
[
https://issues.apache.org/jira/browse/TEZ-3753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3753:
-
Attachment: tez-3753.001.patch
Check if the byte array doubling will result in the array siz
[
https://issues.apache.org/jira/browse/TEZ-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan reassigned TEZ-3283:
Assignee: Muhammad Samir Khan
> Add a max cap to the heap computation done by Tez
> -
[
https://issues.apache.org/jira/browse/TEZ-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3283:
-
Attachment: tez-3283.001.patch
Added a new parameter to specify the max java non heap memory
[
https://issues.apache.org/jira/browse/TEZ-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan reassigned TEZ-3693:
Assignee: Muhammad Samir Khan
> ControlledClock is not used
> ---
[
https://issues.apache.org/jira/browse/TEZ-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3693:
-
Attachment: TEZ-3693.001.patch
Removed ControlledClock
> ControlledClock is not used
>
Muhammad Samir Khan created TEZ-3772:
Summary: Allow slowstart for small jobs to be treated differently
Key: TEZ-3772
URL: https://issues.apache.org/jira/browse/TEZ-3772
Project: Apache Tez
[
https://issues.apache.org/jira/browse/TEZ-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3772:
-
Attachment: tez-3772.001.patch
Added new configurations to distinguish between small/regular
[
https://issues.apache.org/jira/browse/TEZ-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3772:
-
Summary: Allow slowstart for small vertices to be treated differently
(was: Allow slowstart
[
https://issues.apache.org/jira/browse/TEZ-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16063760#comment-16063760
]
Muhammad Samir Khan commented on TEZ-3772:
--
For example, when there is a single red
[
https://issues.apache.org/jira/browse/TEZ-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3772:
-
Attachment: tez-3772.002.patch
-Updated defaults for slowstart for small vertices.
-Fixed ja
[
https://issues.apache.org/jira/browse/TEZ-3753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3753:
-
Attachment: tez-3753.002.patch
Updated patch to include a unit test and some changes based o
[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan reopened TEZ-3212:
--
> IFile throws NegativeArraySizeException for value sizes between 1GB and 2GB
> --
[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan reassigned TEZ-3212:
Assignee: Muhammad Samir Khan (was: Jonathan Eagles)
> IFile throws NegativeArraySiz
[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3212:
-
Attachment: tez-3212.002.patch
> IFile throws NegativeArraySizeException for value sizes bet
[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16096906#comment-16096906
]
Muhammad Samir Khan commented on TEZ-3212:
--
Added unit test and throw an error mess
[
https://issues.apache.org/jira/browse/TEZ-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3752:
-
Attachment: TEZ-3752.001.patch
Replace using BoundedByteArrayOutputStream with a byte array
[
https://issues.apache.org/jira/browse/TEZ-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102046#comment-16102046
]
Muhammad Samir Khan commented on TEZ-3752:
--
JOL dump:
Before:
-internals:
{code}
#
[
https://issues.apache.org/jira/browse/TEZ-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16103716#comment-16103716
]
Muhammad Samir Khan commented on TEZ-3752:
--
Ran orderedwordcount with
-Dtez.shuffl
[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3212:
-
Attachment: tez-3212.003.patch
1) Added MAX_BUFFER_SIZE not final to findbugs-exclude.xml
2)
[
https://issues.apache.org/jira/browse/TEZ-3752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16103880#comment-16103880
]
Muhammad Samir Khan commented on TEZ-3752:
--
However, this test doesn't actually hit
[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3212:
-
Attachment: tez-3212.004.patch
Thanks [~rajesh.balamohan] for the suggestion. I moved the co
[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3212:
-
Attachment: tez-3212.005.patch
Made createLargerArray() static.
> IFile throws NegativeArra
Muhammad Samir Khan created TEZ-3807:
Summary: InMemoryWriter is not tested with RLE enabled
Key: TEZ-3807
URL: https://issues.apache.org/jira/browse/TEZ-3807
Project: Apache Tez
Issue Ty
[
https://issues.apache.org/jira/browse/TEZ-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3807:
-
Attachment: TEZ-3807.001.patch
Fix testInMemoryWriter() and add test cases to testWithRLEMar
[
https://issues.apache.org/jira/browse/TEZ-3212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16105671#comment-16105671
]
Muhammad Samir Khan commented on TEZ-3212:
--
The failing tests pass locally.
> IFil
[
https://issues.apache.org/jira/browse/TEZ-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3807:
-
Attachment: TEZ-3807.002.patch
> InMemoryWriter is not tested with RLE enabled
> ---
[
https://issues.apache.org/jira/browse/TEZ-3807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16108089#comment-16108089
]
Muhammad Samir Khan commented on TEZ-3807:
--
Fixed both.
> InMemoryWriter is not te
Muhammad Samir Khan created TEZ-3809:
Summary: The buffer size allocated for InMemoryMapOutput can be
optimized
Key: TEZ-3809
URL: https://issues.apache.org/jira/browse/TEZ-3809
Project: Apache Te
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3809:
-
Description:
Related jiras: TEZ-3752 and TEZ-3732.
-When shuffling input to memory, the dec
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3809:
-
Attachment: TEZ-3809.001.patch
Initial patch with the changes mentioned in the description.
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3809:
-
Description:
Related jiras: TEZ-3752 and TEZ-3732.
-When shuffling input to memory, the dec
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111794#comment-16111794
]
Muhammad Samir Khan commented on TEZ-3809:
--
Took a heap dump on ordered word count
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111794#comment-16111794
]
Muhammad Samir Khan edited comment on TEZ-3809 at 8/2/17 10:03 PM:
---
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111794#comment-16111794
]
Muhammad Samir Khan edited comment on TEZ-3809 at 8/2/17 10:04 PM:
---
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111794#comment-16111794
]
Muhammad Samir Khan edited comment on TEZ-3809 at 8/2/17 10:04 PM:
---
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111794#comment-16111794
]
Muhammad Samir Khan edited comment on TEZ-3809 at 8/2/17 10:07 PM:
---
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16113366#comment-16113366
]
Muhammad Samir Khan commented on TEZ-3809:
--
Also tested with memory-to-memory merge
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16113369#comment-16113369
]
Muhammad Samir Khan commented on TEZ-3809:
--
Request for comments [~rajesh.balamohan
Muhammad Samir Khan created TEZ-3813:
Summary: Reduce Object size of MemoryFetchedInput for large jobs
Key: TEZ-3813
URL: https://issues.apache.org/jira/browse/TEZ-3813
Project: Apache Tez
[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3813:
-
Attachment: TEZ-3813.001.patch
Replace using BoundedByteArrayOutputStream with a byte array
[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16113451#comment-16113451
]
Muhammad Samir Khan commented on TEZ-3813:
--
*JOL Dump:*
+Before:+
Internals:
{code}
[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16113454#comment-16113454
]
Muhammad Samir Khan commented on TEZ-3813:
--
Tested with filterLinesByWord and compa
[
https://issues.apache.org/jira/browse/TEZ-3809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3809:
-
Attachment: TEZ-3809.002.patch
Removed the unused imports and fixed the names for new method
[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3813:
-
Attachment: TEZ-3813.002.patch
Some more optimizations related to enums, similar to TEZ-3732
[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3813:
-
Attachment: TEZ-3813.003.patch
Fixed a NPE when getSize() is called after a free().
Removed
[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3813:
-
Attachment: TEZ-3813.004.patch
Added getBytes() to findbugs exclude file.
> Reduce Object s
[
https://issues.apache.org/jira/browse/TEZ-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan reassigned TEZ-3159:
Assignee: Muhammad Samir Khan
> Reduce memory utilization while serializing keys and
[
https://issues.apache.org/jira/browse/TEZ-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3159:
-
Attachment: TEZ-3159.001.patch
Added a new class that grows buffers via "doubling" until it
[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3813:
-
Attachment: TEZ-3813.005.patch
Removed int size from MemoryFetchedInput.
*JOL Dump:*
+After
[
https://issues.apache.org/jira/browse/TEZ-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3159:
-
Attachment: TEZ-3159.002.patch
Fixed findbugs and release audit warnings.
> Reduce memory u
[
https://issues.apache.org/jira/browse/TEZ-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3813:
-
Attachment: TEZ-3813.006.patch
Fixed
> Reduce Object size of MemoryFetchedInput for large j
Muhammad Samir Khan created TEZ-3816:
Summary: Ability to automatically speculate single-task vertices
Key: TEZ-3816
URL: https://issues.apache.org/jira/browse/TEZ-3816
Project: Apache Tez
[
https://issues.apache.org/jira/browse/TEZ-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3816:
-
Attachment: TEZ-3816.001.patch
Added ability to LegacyTaskRuntimeEstimator to speculate base
[
https://issues.apache.org/jira/browse/TEZ-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3816:
-
Description: When a single-task vertex is unlucky, it lands on a very slow
node. Speculation
[
https://issues.apache.org/jira/browse/TEZ-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3159:
-
Attachment: TEZ-3159.003.patch
Some changes to ensureExtraCapacity to create a separate meth
[
https://issues.apache.org/jira/browse/TEZ-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3816:
-
Attachment: TEZ-3816.002.patch
Thanks [~sseth] for the comments. I updated the patch to put
[
https://issues.apache.org/jira/browse/TEZ-3816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3816:
-
Attachment: TEZ-3816.003.patch
Added a couple of comments and speculationValue returns ON_SC
[
https://issues.apache.org/jira/browse/TEZ-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3159:
-
Attachment: TEZ-3159.004.patch
Incorporated comments from the review. Removed the alignment
[
https://issues.apache.org/jira/browse/TEZ-3159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Muhammad Samir Khan updated TEZ-3159:
-
Attachment: TEZ-3159.005.patch
Fixes to findbugs etc. Added unit test for FlexibleDataInputB
61 matches
Mail list logo
