Hi,
I'm wondering if there is expected performance increases with using the
--direct flag for exporting from hive to mysql. If so, how much speedup?
Also, I've been getting lock contention errors during export, and I'm
wondering if these are less likely using --direct mode? E.g. I'm getting
these sorts of exceptions on the sqoop console:
14/03/18 14:44:15 INFO mapred.JobClient: Task Id :
attempt_201403180842_0202_m_000002_1, Status : FAILED
java.io.IOException: Can't export data, please check failed map task logs
at
org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
at
org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
at
org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.io.IOException: java.sql.BatchUpdateException: Deadlock
found when trying to get lock; try restarting transaction
at
org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.write(AsyncSqlRecordWr
Thanks,
Jason
