I've created SPARK-3499 https://issues.apache.org/jira/browse/SPARK-3499 to
track creating a Spark-based distcp utility.
Nick
On Tue, Aug 12, 2014 at 4:20 PM, Matei Zaharia matei.zaha...@gmail.com
wrote:
Good question; I don't know of one but I believe people at Cloudera had
some thoughts of
We are probably still the minority, but our analytics platform based on
Spark + HDFS does not have map/reduce installed. I'm wondering if there is
a distcp equivalent that leverages Spark to do the work.
Our team is trying to find the best way to do cross-datacenter replication
of our HDFS data
Good question; I don't know of one but I believe people at Cloudera had some
thoughts of porting Sqoop to Spark in the future, and maybe they'd consider
DistCP as part of this effort. I agree it's missing right now.
Matei
On August 12, 2014 at 11:04:28 AM, Gary Malouf (malouf.g...@gmail.com)
