Hi David, Sorry for the slow reply, I haven't had access to the cluster for the last couple of weeks. This patch seems to have fixed the issue and i am now able to pull down evenly sized parquet files with Oraoop.
On the issue of the time taken by the block fetching. I forced a higher DOP for the session to make it faster and used the noparallel hint to stop the mappers from using additional resource. Though i could see the settings had taken effect from the query monitor, on repeated runs it didn't really take much off the initial query time. If anything it made it slightly longer. But the additional overhead for us isn't so important as we won't usually be dumping partitions more frequently than once a day. Thanks very much for your help. Josh On Wed, Nov 19, 2014 at 3:13 AM, David Robson < [email protected]> wrote: > Hi Joshua, > > > > I have submitted a patch to change the int to a long – would you be able > to test this and see if it resolves your issue? > > > > https://issues.apache.org/jira/browse/SQOOP-1764 > > > > Are you able to compile it – or I can send you a compiled version if you > like? > > > > David > > > > *From:* Joshua Baxter [mailto:[email protected]] > *Sent:* Tuesday, 11 November 2014 2:53 AM > *To:* [email protected] > *Subject:* Re: Using more than a single mapper per partition with OraOop > > > > Thanks for your help David. Can you let us know the issue number once it > is logged. > > > > Cheers > > > > Josh >
