[
https://issues.apache.org/jira/browse/FALCON-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14169608#comment-14169608
]
Sowmya Ramesh edited comment on FALCON-20 at 10/13/14 5:40 PM:
---------------------------------------------------------------
bq. Replication will pick distcp jar from oozie sharelib. Is falcon's
FeedReplicator guaranteed to work with any distcp in terms of code signature?
Few changes was required in CustomReplicator and only change in FeedReplicator
was to use setMapBandwidth. Code changes has been tested and replication works.
bq. If latest distcp doesn't support mapBandwidth in KB, we should rename
falcon's param as well accordingly. But this is useful, not sure why it was
removed
In Inmobi DistCpOptions code for setMapBandwidthKB was
{noformat}
public void setMapBandwidthKB(int mapBandwidth) {
if (mapBandwidth <= 0) {
throw new IllegalArgumentException("Bandwidth " +
mapBandwidth + " is invalid (should be > 0)");
}
this.mapBandwidth = mapBandwidth;
}
{noformat}
In Hadoop tools DistCP code for setMapBandwidth is
{noformat}
public void setMapBandwidth(int mapBandwidth) {
assert mapBandwidth > 0 : "Bandwidth " + mapBandwidth + " is invalid
(should be > 0)";
this.mapBandwidth = mapBandwidth;
}
{noformat}
As both methods have same logic and mapBandwidth is int I just retained the
same param name.
was (Author: sowmyaramesh):
bq. Replication will pick distcp jar from oozie sharelib. Is falcon's
FeedReplicator guaranteed to work with any distcp in terms of code signature?
Few changes was required in CustomReplicator and only change in FeedReplicator
was to use setMapBandwidth. Code changes has been tested and replication works.
bq. If latest distcp doesn't support mapBandwidth in KB, we should rename
falcon's param as well accordingly. But this is useful, not sure why it was
removed
In Inmobi DistCpOptions code for setMapBandwidthKB was
{noquote}
public void setMapBandwidthKB(int mapBandwidth) {
if (mapBandwidth <= 0) {
throw new IllegalArgumentException("Bandwidth " +
mapBandwidth + " is invalid (should be > 0)");
}
this.mapBandwidth = mapBandwidth;
}
{noquote}
In Hadoop tools DistCP code for setMapBandwidth is
{noquote}
public void setMapBandwidth(int mapBandwidth) {
assert mapBandwidth > 0 : "Bandwidth " + mapBandwidth + " is invalid
(should be > 0)";
this.mapBandwidth = mapBandwidth;
}
{noquote}
As both methods have same logic and mapBandwidth is int I just retained the
same param name.
> Remove dependency on custom InMobi DistCp
> -----------------------------------------
>
> Key: FALCON-20
> URL: https://issues.apache.org/jira/browse/FALCON-20
> Project: Falcon
> Issue Type: Sub-task
> Components: replication
> Affects Versions: 0.3
> Reporter: Venkatesh Seetharam
> Assignee: Sowmya Ramesh
> Fix For: 0.6
>
> Attachments: FALCON-20-C.patch, FALCON-20.patch, FALCON-20.r1.patch,
> FALCON-20.v2.patch, cluster01.xml, cluster02.xml, feed.xml
>
>
> Falcon depends on custom DistCp 0.3 since DistCp v2 was part of Hadoop 2.x
> line.
> https://issues.apache.org/jira/browse/MAPREDUCE-5081 has back ported DistCp
> v2 to Hadoop 1.x line and Falcon should instead depend on that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
