If S3 you can use distcp from HDFS to S3@Ceph. For example:
hadoop distcp -Dmapred.job.queue.name=queue_name Dfs.s3a.access.key= -Dfs.s3a.secret.key= -Dfs.s3a.endpoint= -Dfs.s3a.connection.ssl.enabled=false_or_true /hdfs_path/ s3a://path/ Regards -- Jarek pt., 6 lis 2020 o 12:29 Szabo, Istvan (Agoda) <istvan.sz...@agoda.com> napisał(a): > Objectstore > > On 2020. Nov 6., at 15:41, Jaroslaw Owsiewski < > jaroslaw.owsiew...@allegro.pl> wrote: > > > Email received from outside the company. If in doubt don't click links nor > open attachments! > ________________________________ > Hi, > > What protocol do you want to make this data available on the Ceph? > > -- > Jarek > > pt., 6 lis 2020 o 04:00 Szabo, Istvan (Agoda) <istvan.sz...@agoda.com > <mailto:istvan.sz...@agoda.com>> napisał(a): > Hi, > > Is there anybody tried to migrate data from Hadoop to Ceph? > If yes what is the right way? > > Thank you > > ________________________________ > This message is confidential and is for the sole use of the intended > recipient(s). It may also be privileged or otherwise protected by copyright > or other legal rules. If you have received it by mistake please let us know > by reply email and delete it from your system. It is prohibited to copy > this message or disclose its content to anyone. Any confidentiality or > privilege is not waived or lost by any mistaken delivery or unauthorized > disclosure of the message. All messages sent to and from Agoda may be > monitored to ensure compliance with company policies, to protect the > company's interests and to remove potential malware. Electronic messages > may be intercepted, amended, lost or deleted, or contain viruses. > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io<mailto:ceph-users@ceph.io> > To unsubscribe send an email to ceph-users-le...@ceph.io<mailto: > ceph-users-le...@ceph.io> > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io > _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
