[jira] [Updated] (HDFS-2635) NNBench creates duplicate files if multiple maps are run from the same client
[
https://issues.apache.org/jira/browse/HDFS-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
cp...@kence.org updated HDFS-2635:
--
Target Version/s: 0.20.203.0
Status: Patch Available (was: Open)
--- NNBench.java2011-12-06 12:03:49.0 -0800
+++ NNBench.new 2011-12-06 12:03:31.0 -0800
@@ -722,16 +722,16 @@
if (barrier()) {
if (op.equals(OP_CREATE_WRITE)) {
startTimeTPmS = System.currentTimeMillis();
- doCreateWriteOp(file_ + hostName + _, reporter);
+ doCreateWriteOp(file_ + hostName + _ + key, reporter);
} else if (op.equals(OP_OPEN_READ)) {
startTimeTPmS = System.currentTimeMillis();
- doOpenReadOp(file_ + hostName + _, reporter);
+ doOpenReadOp(file_ + hostName + _ + key, reporter);
} else if (op.equals(OP_RENAME)) {
startTimeTPmS = System.currentTimeMillis();
- doRenameOp(file_ + hostName + _, reporter);
+ doRenameOp(file_ + hostName + _ + key, reporter);
} else if (op.equals(OP_DELETE)) {
startTimeTPmS = System.currentTimeMillis();
- doDeleteOp(file_ + hostName + _, reporter);
+ doDeleteOp(file_ + hostName + _ + key, reporter);
}
endTimeTPms = System.currentTimeMillis();
NNBench creates duplicate files if multiple maps are run from the same client
-
Key: HDFS-2635
URL: https://issues.apache.org/jira/browse/HDFS-2635
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Affects Versions: 0.20.205.0, 0.20.204.0, 0.20.203.0
Reporter: cp...@kence.org
Priority: Minor
NNBench creates files in the format:
file_hostname__filenum
This works seamlessly as long as all of the Hadoop clients in the cluster are
each running with a single map slot. If multiple map slots are available on a
single client, each mapper tries to create the same set of files. This can
result in lock contention on some non-Hadoop HDFS implementations thereby
defeating the purpose of the NNBench test.
Making the files unique per mapper, not per host, resolves this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2635) NNBench creates duplicate files if multiple maps are run from the same client
[ https://issues.apache.org/jira/browse/HDFS-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cp...@kence.org updated HDFS-2635: -- Status: Open (was: Patch Available) NNBench creates duplicate files if multiple maps are run from the same client - Key: HDFS-2635 URL: https://issues.apache.org/jira/browse/HDFS-2635 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.20.205.0, 0.20.204.0, 0.20.203.0 Reporter: cp...@kence.org Priority: Minor NNBench creates files in the format: file_hostname__filenum This works seamlessly as long as all of the Hadoop clients in the cluster are each running with a single map slot. If multiple map slots are available on a single client, each mapper tries to create the same set of files. This can result in lock contention on some non-Hadoop HDFS implementations thereby defeating the purpose of the NNBench test. Making the files unique per mapper, not per host, resolves this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2635) NNBench creates duplicate files if multiple maps are run from the same client
[ https://issues.apache.org/jira/browse/HDFS-2635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] cp...@kence.org updated HDFS-2635: -- Attachment: hdfs-2635.patch Lets try this again, the 'right' way. NNBench creates duplicate files if multiple maps are run from the same client - Key: HDFS-2635 URL: https://issues.apache.org/jira/browse/HDFS-2635 Project: Hadoop HDFS Issue Type: Bug Components: test Affects Versions: 0.20.203.0, 0.20.204.0, 0.20.205.0 Reporter: cp...@kence.org Priority: Minor Attachments: hdfs-2635.patch NNBench creates files in the format: file_hostname__filenum This works seamlessly as long as all of the Hadoop clients in the cluster are each running with a single map slot. If multiple map slots are available on a single client, each mapper tries to create the same set of files. This can result in lock contention on some non-Hadoop HDFS implementations thereby defeating the purpose of the NNBench test. Making the files unique per mapper, not per host, resolves this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
