Hanisha Koneru created HDFS-12164: ------------------------------------- Summary: CodecUtil throws stacktrace to the client on createRawEncoder/Decoder error Key: HDFS-12164 URL: https://issues.apache.org/jira/browse/HDFS-12164 Project: Hadoop HDFS Issue Type: Bug Components: erasure-coding Reporter: Hanisha Koneru Assignee: Hanisha Koneru
While doing filesystem operations on ec dirs/files, the _failure to create raw encoder/decoder_ warning at the client side is accompanied by the stacktrace of the error. {code} 2017-07-19 21:31:00,209 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2017-07-19 21:31:01,023 WARN erasurecode.CodecUtil: Failed to create raw erasure encoder rs_native, fallback to next codec if possible java.lang.ExceptionInInitializerError at org.apache.hadoop.io.erasurecode.rawcoder.NativeRSRawErasureCoderFactory.createEncoder(NativeRSRawErasureCoderFactory.java:35) at org.apache.hadoop.io.erasurecode.CodecUtil.createRawEncoderWithFallback(CodecUtil.java:177) at org.apache.hadoop.io.erasurecode.CodecUtil.createRawEncoder(CodecUtil.java:129) at org.apache.hadoop.hdfs.DFSStripedOutputStream.<init>(DFSStripedOutputStream.java:302) at org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:309) at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1216) at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1195) at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1133) at org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:451) at org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:448) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:462) at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:389) at org.apache.hadoop.fs.FilterFileSystem.create(FilterFileSystem.java:181) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1074) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1054) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:943) at org.apache.hadoop.fs.shell.CommandWithDestination$TargetFileSystem.create(CommandWithDestination.java:509) at org.apache.hadoop.fs.shell.CommandWithDestination$TargetFileSystem.writeStreamToFile(CommandWithDestination.java:484) at org.apache.hadoop.fs.shell.CommandWithDestination.copyStreamToTarget(CommandWithDestination.java:407) at org.apache.hadoop.fs.shell.CommandWithDestination.copyFileToTarget(CommandWithDestination.java:342) at org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:277) at org.apache.hadoop.fs.shell.CommandWithDestination.processPath(CommandWithDestination.java:262) at org.apache.hadoop.fs.shell.Command.processPaths(Command.java:331) at org.apache.hadoop.fs.shell.Command.processPathArgument(Command.java:303) at org.apache.hadoop.fs.shell.CommandWithDestination.processPathArgument(CommandWithDestination.java:257) at org.apache.hadoop.fs.shell.Command.processArgument(Command.java:285) at org.apache.hadoop.fs.shell.Command.processArguments(Command.java:269) at org.apache.hadoop.fs.shell.CommandWithDestination.processArguments(CommandWithDestination.java:228) at org.apache.hadoop.fs.shell.CopyCommands$Put.processArguments(CopyCommands.java:286) at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:119) at org.apache.hadoop.fs.shell.Command.run(Command.java:176) at org.apache.hadoop.fs.FsShell.run(FsShell.java:326) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90) at org.apache.hadoop.fs.FsShell.main(FsShell.java:389) Caused by: java.lang.RuntimeException: hadoop native library cannot be loaded. at org.apache.hadoop.io.erasurecode.ErasureCodeNative.checkNativeCodeLoaded(ErasureCodeNative.java:69) at org.apache.hadoop.io.erasurecode.rawcoder.NativeRSRawEncoder.<clinit>(NativeRSRawEncoder.java:33) {code} We can just show the warning message to the client and avoid throwing the full stacktrace. -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org