Thanks for clearing that out Chris, I think that the capability of the filemgr should be considered as a test on its own and does not actually fall into the category which we are currently benchmarking for the baseline WAN/LAN performance of the tools.
There should be a separate test section for archiving... Thanks for the help! Etienne On Sep 10, 2014 6:15 PM, "Chris Mattmann" <mattm...@apache.org> wrote: > Hi Etienne, > > Thanks for your query. The default protocol used by the > RemoteDataTransfer is XML-RPC. It's a call-back mechanism > that I baked up myself with a configurable chunk size parameter. > > My suggestion is rather than use that DataTransfer, I would leverage: > > (1) a distributed filesystem like HadoopFS, Tachyon (from BDAS), GlusterFS, > or even NFS that logically mounts a set of local shared nothing disks as a > virtual global mount > > (2) use the LocalDataTransfer in the File Manager, with #1 in place > > The main reason for this is that it allows the people who work on > distributed > and reliable filesystems that solve those problems in a great way, it > allows > us in the OODT community to take advantage of that work without having to > write > all the complex functionality ourselves. That way we just have > light-weight insulated > pieces/components in OODT that rely on the workhorses that do things right. > > HTH! > > Cheers, > Chris > > ------------------------ > Chris Mattmann > chris.mattm...@gmail.com > > > > > -----Original Message----- > From: Etienne Koen <koe...@gmail.com> > Date: Wednesday, September 10, 2014 1:57 AM > To: Shakeh Khudikyan <shakeh.e.khudik...@jpl.nasa.gov> > Cc: Chris Mattmann <chris.mattm...@gmail.com> > Subject: filemgr ingestion > > >Hi Shakeh, > >What is the default protocol that filemgr uses for RemoteDataTransfer > >ingestion? > > > >Also, is the a way to specify the protocol? > > > >Cheers > >Etienne > > > > >
