[jira] [Commented] (PIG-3754) InputSizeReducerEstimator.getTotalInputFileSize reports incorrect size
[ https://issues.apache.org/jira/browse/PIG-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923641#comment-13923641 ] Aniket Mokashi commented on PIG-3754: - Committed to trunk. Thanks [~cheolsoo] and [~julienledem] for the review. > InputSizeReducerEstimator.getTotalInputFileSize reports incorrect size > -- > > Key: PIG-3754 > URL: https://issues.apache.org/jira/browse/PIG-3754 > Project: Pig > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: Aniket Mokashi >Assignee: Aniket Mokashi >Priority: Trivial > Fix For: 0.13.0 > > Attachments: PIG-3754-1.patch, PIG-3754-2.patch > > > If you have more than one input, > InputSizeReducerEstimator.getTotalInputFileSize can return incorrect value if > one of the loader returns \-1 and is not file based (eg- hbase). This causes > incorrect reducer estimation and problems in auto.local mode. > If size of input is not found in for any of the inputs, we should bail out > with return value of -1. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (PIG-3754) InputSizeReducerEstimator.getTotalInputFileSize reports incorrect size
[ https://issues.apache.org/jira/browse/PIG-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923639#comment-13923639 ] Aniket Mokashi commented on PIG-3754: - Attached PIG-3754-2.patch with tests. Will commit to trunk. > InputSizeReducerEstimator.getTotalInputFileSize reports incorrect size > -- > > Key: PIG-3754 > URL: https://issues.apache.org/jira/browse/PIG-3754 > Project: Pig > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: Aniket Mokashi >Assignee: Aniket Mokashi >Priority: Trivial > Fix For: 0.13.0 > > Attachments: PIG-3754-1.patch, PIG-3754-2.patch > > > If you have more than one input, > InputSizeReducerEstimator.getTotalInputFileSize can return incorrect value if > one of the loader returns \-1 and is not file based (eg- hbase). This causes > incorrect reducer estimation and problems in auto.local mode. > If size of input is not found in for any of the inputs, we should bail out > with return value of -1. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (PIG-3754) InputSizeReducerEstimator.getTotalInputFileSize reports incorrect size
[ https://issues.apache.org/jira/browse/PIG-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923365#comment-13923365 ] Julien Le Dem commented on PIG-3754: LGTM too > InputSizeReducerEstimator.getTotalInputFileSize reports incorrect size > -- > > Key: PIG-3754 > URL: https://issues.apache.org/jira/browse/PIG-3754 > Project: Pig > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: Aniket Mokashi >Assignee: Aniket Mokashi >Priority: Trivial > Fix For: 0.13.0 > > Attachments: PIG-3754-1.patch > > > If you have more than one input, > InputSizeReducerEstimator.getTotalInputFileSize can return incorrect value if > one of the loader returns \-1 and is not file based (eg- hbase). This causes > incorrect reducer estimation and problems in auto.local mode. > If size of input is not found in for any of the inputs, we should bail out > with return value of -1. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (PIG-3754) InputSizeReducerEstimator.getTotalInputFileSize reports incorrect size
[ https://issues.apache.org/jira/browse/PIG-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923166#comment-13923166 ] Aniket Mokashi commented on PIG-3754: - Will do. > InputSizeReducerEstimator.getTotalInputFileSize reports incorrect size > -- > > Key: PIG-3754 > URL: https://issues.apache.org/jira/browse/PIG-3754 > Project: Pig > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: Aniket Mokashi >Assignee: Aniket Mokashi >Priority: Trivial > Fix For: 0.13.0 > > Attachments: PIG-3754-1.patch > > > If you have more than one input, > InputSizeReducerEstimator.getTotalInputFileSize can return incorrect value if > one of the loader returns \-1 and is not file based (eg- hbase). This causes > incorrect reducer estimation and problems in auto.local mode. > If size of input is not found in for any of the inputs, we should bail out > with return value of -1. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (PIG-3754) InputSizeReducerEstimator.getTotalInputFileSize reports incorrect size
[ https://issues.apache.org/jira/browse/PIG-3754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923158#comment-13923158 ] Cheolsoo Park commented on PIG-3754: Looks good. Would you mind adding a test case for this corner case to TestInputSizeReducerEstimator? > InputSizeReducerEstimator.getTotalInputFileSize reports incorrect size > -- > > Key: PIG-3754 > URL: https://issues.apache.org/jira/browse/PIG-3754 > Project: Pig > Issue Type: Bug >Affects Versions: 0.13.0 >Reporter: Aniket Mokashi >Assignee: Aniket Mokashi >Priority: Trivial > Fix For: 0.13.0 > > Attachments: PIG-3754-1.patch > > > If you have more than one input, > InputSizeReducerEstimator.getTotalInputFileSize can return incorrect value if > one of the loader returns \-1 and is not file based (eg- hbase). This causes > incorrect reducer estimation and problems in auto.local mode. > If size of input is not found in for any of the inputs, we should bail out > with return value of -1. -- This message was sent by Atlassian JIRA (v6.2#6252)
