[ https://issues.apache.org/jira/browse/SQOOP-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15415380#comment-15415380 ]
Jordi edited comment on SQOOP-2997 at 8/10/16 2:49 PM: ------------------------------------------------------- This is my fix proposal, already tested in my project. org.apache.sqoop.util.password.CryptoFileLoader Current sqoop-1.4.6 code: {code} @Override public String loadPassword(String p, Configuration configuration) throws IOException { LOG.debug("Fetching password from specified path: " + p); Path path = new Path(p); FileSystem fs = path.getFileSystem(configuration); byte [] encrypted; try { verifyPath(fs, path); encrypted = readBytes(fs, path); } finally { fs.close(); } {code} Fix proposal: @Override public String loadPassword(String p, Configuration configuration) throws IOException { LOG.debug("Fetching password from specified path: " + p); Path path = new Path(p); FileSystem fs = path.getFileSystem(configuration); // Not closing FileSystem object because of SQOOP-2997 verifyPath(fs, path); byte [] encrypted = readBytes(fs, path); ... was (Author: jordirodri): This is my fix proposal, already tested in my project. org.apache.sqoop.util.password.CryptoFileLoader Current sqoop-1.4.6 code: @Override public String loadPassword(String p, Configuration configuration) throws IOException { LOG.debug("Fetching password from specified path: " + p); Path path = new Path(p); FileSystem fs = path.getFileSystem(configuration); byte [] encrypted; try { verifyPath(fs, path); encrypted = readBytes(fs, path); } finally { fs.close(); } Fix proposal: @Override public String loadPassword(String p, Configuration configuration) throws IOException { LOG.debug("Fetching password from specified path: " + p); Path path = new Path(p); FileSystem fs = path.getFileSystem(configuration); // Not closing FileSystem object because of SQOOP-2997 verifyPath(fs, path); byte [] encrypted = readBytes(fs, path); ... > --password-file option triggers FileSystemClosed exception at end of Oozie > action > --------------------------------------------------------------------------------- > > Key: SQOOP-2997 > URL: https://issues.apache.org/jira/browse/SQOOP-2997 > Project: Sqoop > Issue Type: Bug > Affects Versions: 1.4.6 > Environment: Java 1.8 + CDH5.5.1 > Reporter: Jordi > > When using the --password-file option triggers FileSystemClosed exception at > end of Oozie action. > This error was fixed for FilePasswordLoader class for sqoop-1.4.3, but it is > also happening at CryptoFileLoader which extends from the fixed one. > https://issues.apache.org/jira/browse/SQOOP-1226 > In this case we are using --password-file option with an encrypted file so we > need to use CryptoFileLoder. > Error LOG: > Job commit failed: java.io.IOException: Filesystem closed > at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:837) > at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1720) > at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1662) > at > org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:404) > > at > org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:400) > > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > > at > org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:400) > > at > org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:343) > > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:917) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:898) > at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:795) > at > org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.touchz(CommitterEventHandler.java:265) > > at > org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobCommit(CommitterEventHandler.java:271) > > at > org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:237) > > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:745) > Average Map Time 57sec -- This message was sent by Atlassian JIRA (v6.3.4#6332)