play around with java vm heap setting https://altiscale.zendesk.com/hc/en-us/articles/200801519-Configuring-Memory-for-Mappers-and-Reducers-in-Hadoop-2
*Kind Regards,*Mario Amatucci On 27 July 2015 at 03:17, Eric <[email protected]> wrote: > Hi here! > > I tried to run a sqoop export command on Amazon EMR. I succeed running it > on a small dataset, but with a bigger one, I get this error on the cluster: > > Error: java.io.IOException: org.postgresql.util.PSQLException: Tried to > end inactive copy > at > org.apache.sqoop.mapreduce.postgresql.PostgreSQLCopyExportMapper.cleanup(PostgreSQLCopyExportMapper.java:145) > at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:160) > at > org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:773) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:175) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:170) > Caused by: org.postgresql.util.PSQLException: Tried to end inactive copy > at > org.postgresql.core.v3.QueryExecutorImpl.endCopy(QueryExecutorImpl.java:814) > at org.postgresql.core.v3.CopyInImpl.endCopy(CopyInImpl.java:59) > at > org.apache.sqoop.mapreduce.postgresql.PostgreSQLCopyExportMapper.cleanup(PostgreSQLCopyExportMapper.java:142) > ... 9 more > > > and here my command : ./sqoop export --direct --connect > jdbc:postgresql://xxxx --username xxxx --table updates_test --export-dir > s3://xxx -P --fields-terminated-by ',' --enclosed-by '\"' --null-string '' > --null-non-string '' > > > I was wondering if I missed anything , or if anyone here have any clue ? > > Thank you so much for your help. > Eric >
