[
https://issues.apache.org/jira/browse/PIG-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13240970#comment-13240970
]
Julien Le Dem commented on PIG-2574:
------------------------------------
Overall, this looks good to me.
Here are my comments, mostly details. Some of them are about existing code that
has been moved.
1)
{code}
return listener != null ? (PigProgressNotificationListener) listener : null;
{code}
if listener is null you can still cast it.
just return (PigProgressNotificationListener) listener;
2)
{code}
int reducers = (int)Math.ceil((totalInputFileSize+0.0) / bytesPerReducer);
reducers = Math.max(1, reducers);
reducers = Math.min(maxReducers, reducers);
{code}
use a cast (double)totalInputFileSize to convert to double.
cast the final result to int instead. That will avoid bad conversion to integer
if bytesPerReducer is small.
3)
{code}
public static Object instantiateObjectFromParams(Configuration conf, String
classParamKey, String argParamKey) {
{code}
you can do the following so that classes using it don't need to cast. You still
get a ClassCastException if it's not the correct type.
{code}
@SuppressWarnings("unchecked")
public static <T> T instantiateObjectFromParams(Configuration conf, String
classParamKey, String argParamKey) {
...
return (T)result;
}
{code}
If you want to handle the cast with a better message you can do this (notice we
don't need to suppress warnings anymore):
{code}
public static <T> T get(Configuration conf, String classParamKey, String
argParamKey, Class<T> clazz) {
try {
...
return clazz.cast(result);
} catch (ClassCastException e) {
throw new ExecException("the class defined by "+classParamKey+" in conf
is not of type "+clazz.getName(), e);
}
}
{code}
> Make reducer estimator plugable
> -------------------------------
>
> Key: PIG-2574
> URL: https://issues.apache.org/jira/browse/PIG-2574
> Project: Pig
> Issue Type: Improvement
> Reporter: Bill Graham
> Assignee: Bill Graham
> Labels: 0.10_blocker
> Fix For: 0.10, 0.11
>
> Attachments: pig-2574_1.patch, pig-2574_2.patch, pig-2574_3.patch,
> pig-2574_4.patch, pig-2574_5.patch
>
>
> I'd like to refactor the logic contained in this method into a pluggable
> interface:
> {noformat}
> static int JobControlCompiler.estimateNumberOfReducers(Configuration conf,
> List<POLoad> lds);
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
