Thanks David that is very informative. May I ask what partitioning method you utilized?
I've got Oracle licensing covered so that should not be an issue. From: David Robson <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Wednesday, June 18, 2014 7:43 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: RE: Sqoop to Oracle transfer rates Hi Brenden, I did some benchmarking a while ago on a 62858MB folder. The best I achieved with 24 mappers was 867 seconds (72.5MB/s). This was on a 4 node Hadoop cluster that was on some old servers we had so I’m sure with some better hardware you could get a lot faster. I’ll attach some graphs which you might find interesting – basically you can see OraOop reduces load on the DB a little bit with a few optimizations – but once you use direct path and partition exchange loading the benefits are massive. Of course this requires you to have the relevant Oracle licenses so may not be an option. David From: Brenden Cobb [mailto:[email protected]] Sent: Thursday, 19 June 2014 6:25 AM To: [email protected]<mailto:[email protected]> Subject: Sqoop to Oracle transfer rates Looking for some benchmarks on Sqoop (or Oraoop) exports. Appreciate if anyone feels like sharing some metrics. Rough numbers for MB/s would suffice, perhaps number of mappers used. I'm working specifically with Oracle exports, but other RDBMS export rates would be enlightening as well. Thanks
