[jira] [Commented] (TEZ-1083) Enable IFile RLE for DefaultSorter

2014-08-25 Thread Gopal V (JIRA) 

[ 
https://issues.apache.org/jira/browse/TEZ-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14109453#comment-14109453
 ] 

Gopal V commented on TEZ-1083:
--

This looks alright - this just needs a roll-over check for the sameKey long 
variable.

The worst-case value for that is near O(n^2), so it might overflow before 
totalKeys does.

For performance, it can be assumed that if sameKeys is < 0, isRLENeeded == true 
- instead of checking within the loop.

> Enable IFile RLE for DefaultSorter
> --
>
> Key: TEZ-1083
> URL: https://issues.apache.org/jira/browse/TEZ-1083
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Gopal V
> Attachments: TEZ-1083.1.patch
>
>
> Generate RLE IFiles for DefaultSorter and use it to fast-forward map-side 
> merge.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Commented] (TEZ-1083) Enable IFile RLE for DefaultSorter

2014-09-08 Thread Rajesh Balamohan (JIRA) 

[ 
https://issues.apache.org/jira/browse/TEZ-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14126687#comment-14126687
 ] 

Rajesh Balamohan commented on TEZ-1083:
---

[~gopalv] - Can you please have a look at this when you have time?

> Enable IFile RLE for DefaultSorter
> --
>
> Key: TEZ-1083
> URL: https://issues.apache.org/jira/browse/TEZ-1083
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Gopal V
> Attachments: TEZ-1083.1.patch, TEZ-1083.2.patch
>
>
> Generate RLE IFiles for DefaultSorter and use it to fast-forward map-side 
> merge.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TEZ-1083) Enable IFile RLE for DefaultSorter

2014-10-13 Thread Rajesh Balamohan (JIRA) 

[ 
https://issues.apache.org/jira/browse/TEZ-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14170517#comment-14170517
 ] 

Rajesh Balamohan commented on TEZ-1083:
---

[~gopalv] - Can you please have a look at this?

> Enable IFile RLE for DefaultSorter
> --
>
> Key: TEZ-1083
> URL: https://issues.apache.org/jira/browse/TEZ-1083
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Gopal V
> Attachments: TEZ-1083.1.patch, TEZ-1083.2.patch
>
>
> Generate RLE IFiles for DefaultSorter and use it to fast-forward map-side 
> merge.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (TEZ-1083) Enable IFile RLE for DefaultSorter

2014-10-14 Thread Gopal V (JIRA) 

[ 
https://issues.apache.org/jira/browse/TEZ-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14171963#comment-14171963
 ] 

Gopal V commented on TEZ-1083:
--

+1 - this enables RLE on the map-side spill.

Reduce-side TezMerger needs the equivalent impl, as a different JIRA.

> Enable IFile RLE for DefaultSorter
> --
>
> Key: TEZ-1083
> URL: https://issues.apache.org/jira/browse/TEZ-1083
> Project: Apache Tez
>  Issue Type: Improvement
>Reporter: Siddharth Seth
>Assignee: Gopal V
> Attachments: TEZ-1083.1.patch, TEZ-1083.2.patch
>
>
> Generate RLE IFiles for DefaultSorter and use it to fast-forward map-side 
> merge.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)