Liao Chunbo created HDFS-13903:
----------------------------------
Summary: write data into hdfs is very slow while the hdfs has
mounted to local file system via nfs3
Key: HDFS-13903
URL: https://issues.apache.org/jira/browse/HDFS-13903
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs
Affects Versions: 3.1.1
Reporter: Liao Chunbo
I have installed the hadoop 3.1.1, and mount hdfs to local filesystem via
command of "mount -t nfs -o vers=3,nolock,rw,async,wsize=32768,rsize=32768
hdfsserver:/ /mydir"
When I use use cp command to copy files to the mounted dir(/mydire). The speed
is very slow. And some exception occurred as the follow:
2018-09-07 11:04:51,952 INFO security.ShellBasedIdMapping: Update cache now
2018-09-07 11:08:48,305 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:08:48,806 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:08:48,816 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:08:48,829 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:08:49,001 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:10:24,975 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:11:19,747 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:11:19,756 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:11:19,768 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:12:17,220 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:12:17,233 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:12:35,109 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:12:36,398 ERROR nfs3.RpcProgramNfs3: Setting file size is not
supported when setattr, fileId: 27204
2018-09-07 11:12:44,424 ERROR nfs3.RpcProgramNfs3: Setting file size is not
supported when setattr, fileId: 27204
2018-09-07 11:12:53,382 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:12:53,394 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:12:59,658 ERROR nfs3.RpcProgramNfs3: Setting file size is not
supported when setattr, fileId: 27204
2018-09-07 11:13:17,519 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:13:17,533 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:13:18,602 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:13:18,613 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:13:18,933 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:13:42,596 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:13:42,608 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:18:30,308 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:19:23,438 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:19:23,446 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:19:23,666 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:19:31,833 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:19:51,967 INFO security.ShellBasedIdMapping: Update cache now
2018-09-07 11:20:02,884 WARN hdfs.DataStreamer: Exception for
BP-1952000504-10.56.233.182-1536129789677:blk_1073751181_10452
java.io.EOFException: Unexpected EOF while trying to read response from server
at
org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:549)
at
org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213)
at
org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1086)
2018-09-07 11:20:03,007 WARN hdfs.DataStreamer: Exception for
BP-1952000504-10.56.233.182-1536129789677:blk_1073751182_10453
java.io.EOFException: Unexpected EOF while trying to read response from server
at
org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:549)
at
org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213)
at
org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1086)
2018-09-07 11:20:03,377 WARN hdfs.DataStreamer: Exception for
BP-1952000504-10.56.233.182-1536129789677:blk_1073751185_10456
java.io.EOFException: Unexpected EOF while trying to read response from server
at
org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:549)
at
org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213)
at
org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1086)
2018-09-07 11:20:03,513 WARN hdfs.DataStreamer: Exception for
BP-1952000504-10.56.233.182-1536129789677:blk_1073751186_10457
java.io.EOFException: Unexpected EOF while trying to read response from server
at
org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:549)
at
org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213)
at
org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1086)
2018-09-07 11:20:03,704 WARN hdfs.DataStreamer: Exception for
BP-1952000504-10.56.233.182-1536129789677:blk_1073751188_10459
java.io.EOFException: Unexpected EOF while trying to read response from server
at
org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:549)
at
org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213)
at
org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1086)
2018-09-07 11:20:04,060 WARN hdfs.DataStreamer: Exception for
BP-1952000504-10.56.233.182-1536129789677:blk_1073751191_10462
java.io.EOFException: Unexpected EOF while trying to read response from server
at
org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:549)
at
org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213)
at
org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1086)
2018-09-07 11:20:08,213 WARN hdfs.DataStreamer: Exception for
BP-1952000504-10.56.233.182-1536129789677:blk_1073751206_10477
java.io.EOFException: Unexpected EOF while trying to read response from server
at
org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:549)
at
org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213)
at
org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1086)
2018-09-07 11:20:31,866 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:21:32,239 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:22:32,245 INFO security.ShellBasedIdMapping: Can't map group
supergroup. Use its string hashcode:-1710818332
2018-09-07 11:22:35,394 WARN hdfs.DataStreamer: Exception for
BP-1952000504-10.56.233.182-1536129789677:blk_1073751712_10983
java.io.EOFException: Unexpected EOF while trying to read response from server
at
org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:549)
at
org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213)
at
org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1086)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
