You can use node.js for this.
On Tue, Jun 23, 2015 at 8:15 PM, Divya Gehlot
wrote:
> Can you please elaborate it more.
> On 20 Jun 2015 2:46 pm, "SF Hadoop" wrote:
>
>> Really depends on your requirements for the format of the data.
>>
>> The easiest way I can think of is to "stream" batches of
Can you please elaborate it more.
On 20 Jun 2015 2:46 pm, "SF Hadoop" wrote:
> Really depends on your requirements for the format of the data.
>
> The easiest way I can think of is to "stream" batches of data into a pub
> sub system that the target system can access and then consume.
>
> Verify e
Really depends on your requirements for the format of the data.
The easiest way I can think of is to "stream" batches of data into a pub
sub system that the target system can access and then consume.
Verify each batch and then ditch them.
You can throttle the size of the intermediary infrastruct
Not to hijack this post but how would you deal with data that is maintained
by hive(Orc format file, hive created tables etc..)...Would we copy the
hivemetastore(MySQL) and move that over to new cluster?
On Friday, June 19, 2015, Joep Rottinghuis wrote:
> You can't set up a proxy ?
> You probabl
You can't set up a proxy ?
You probably want to avoid writing to local file system because aside from that
being slow, it limits the size of your file to the free space on your local
disc.
If you do need to go commando and go through a single client machine that can
see both clusters you probab
yes
On Fri, Jun 19, 2015 at 11:36 AM, Divya Gehlot
wrote:
> In thats It will be like three step process .
> 1. first cluster (secure zone) HDFS -> copytoLocal -> user local file
> system
> 2. user local space -> copy data -> second cluster user local file system
> 3. second cluster user local f
In thats It will be like three step process .
1. first cluster (secure zone) HDFS -> copytoLocal -> user local file
system
2. user local space -> copy data -> second cluster user local file system
3. second cluster user local file system -> copyfromlocal -> second
clusterHDFS
Am I on the right tr
What's the size of the data?
If you can not do distcp between clusters then other way is doing hdfs get
on the data and then hdfs put on another cluster
On 19-Jun-2015 9:56 am, "Divya Gehlot" wrote:
> Hi,
> I need to copy data from first hadoop cluster to second hadoop cluster.
> I cant access se
Hi,
I need to copy data from first hadoop cluster to second hadoop cluster.
I cant access second hadoop cluster from first hadoop cluster due to some
security issue.
Can any point me how can I do apart from distcp command.
For instance
Cluster 1 secured zone -> copy hdfs data to -> cluster 2 in no
