[jira] [Commented] (HDFS-5868) Make hsync implementation pluggable
[ https://issues.apache.org/jira/browse/HDFS-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13906871#comment-13906871 ] Hudson commented on HDFS-5868: -- SUCCESS: Integrated in Hadoop-Yarn-trunk #487 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/487/]) HDFS-5868. Make hsync implementation pluggable. (Contributed by Buddy Taylor) (arp: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1569978) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/ReplicaOutputStreams.java Make hsync implementation pluggable --- Key: HDFS-5868 URL: https://issues.apache.org/jira/browse/HDFS-5868 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode Affects Versions: 2.2.0 Reporter: Buddy Fix For: 3.0.0, 2.4.0 Attachments: HDFS-5868-branch-2.patch, HDFS-5868a-branch-2.patch, HDFS-5868b-branch-2.patch The current implementation of hsync in BlockReceiver only works if the output streams are instances of FileOutputStream. Therefore, there is currently no way for a FSDatasetSpi plugin to implement hsync if it is not using standard OS files. One possible solution is to push the implementation of hsync into the ReplicaOutputStreams class. This class is constructed by the ReplicaInPipeline which is constructed by the FSDatasetSpi plugin, therefore it can be extended. Instead of directly calling sync on the output stream, BlockReceiver would call ReplicaOutputStream.sync. The default implementation of sync in ReplicaOutputStream would be the same as the current implementation in BlockReceiver. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HDFS-5868) Make hsync implementation pluggable
[ https://issues.apache.org/jira/browse/HDFS-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13906976#comment-13906976 ] Hudson commented on HDFS-5868: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1679 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1679/]) HDFS-5868. Make hsync implementation pluggable. (Contributed by Buddy Taylor) (arp: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1569978) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/ReplicaOutputStreams.java Make hsync implementation pluggable --- Key: HDFS-5868 URL: https://issues.apache.org/jira/browse/HDFS-5868 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode Affects Versions: 2.2.0 Reporter: Buddy Fix For: 3.0.0, 2.4.0 Attachments: HDFS-5868-branch-2.patch, HDFS-5868a-branch-2.patch, HDFS-5868b-branch-2.patch The current implementation of hsync in BlockReceiver only works if the output streams are instances of FileOutputStream. Therefore, there is currently no way for a FSDatasetSpi plugin to implement hsync if it is not using standard OS files. One possible solution is to push the implementation of hsync into the ReplicaOutputStreams class. This class is constructed by the ReplicaInPipeline which is constructed by the FSDatasetSpi plugin, therefore it can be extended. Instead of directly calling sync on the output stream, BlockReceiver would call ReplicaOutputStream.sync. The default implementation of sync in ReplicaOutputStream would be the same as the current implementation in BlockReceiver. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HDFS-5868) Make hsync implementation pluggable
[ https://issues.apache.org/jira/browse/HDFS-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13907027#comment-13907027 ] Hudson commented on HDFS-5868: -- SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1704 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1704/]) HDFS-5868. Make hsync implementation pluggable. (Contributed by Buddy Taylor) (arp: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1569978) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/ReplicaOutputStreams.java Make hsync implementation pluggable --- Key: HDFS-5868 URL: https://issues.apache.org/jira/browse/HDFS-5868 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode Affects Versions: 2.2.0 Reporter: Buddy Fix For: 3.0.0, 2.4.0 Attachments: HDFS-5868-branch-2.patch, HDFS-5868a-branch-2.patch, HDFS-5868b-branch-2.patch The current implementation of hsync in BlockReceiver only works if the output streams are instances of FileOutputStream. Therefore, there is currently no way for a FSDatasetSpi plugin to implement hsync if it is not using standard OS files. One possible solution is to push the implementation of hsync into the ReplicaOutputStreams class. This class is constructed by the ReplicaInPipeline which is constructed by the FSDatasetSpi plugin, therefore it can be extended. Instead of directly calling sync on the output stream, BlockReceiver would call ReplicaOutputStream.sync. The default implementation of sync in ReplicaOutputStream would be the same as the current implementation in BlockReceiver. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HDFS-5868) Make hsync implementation pluggable
[
https://issues.apache.org/jira/browse/HDFS-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13905924#comment-13905924
]
Arpit Agarwal commented on HDFS-5868:
-
Nitpcik: {{BlockReceiver#cout}} can be removed.
+1 otherwise.
Make hsync implementation pluggable
---
Key: HDFS-5868
URL: https://issues.apache.org/jira/browse/HDFS-5868
Project: Hadoop HDFS
Issue Type: Sub-task
Components: datanode
Affects Versions: 2.2.0
Reporter: Buddy
Attachments: HDFS-5868-branch-2.patch, HDFS-5868a-branch-2.patch
The current implementation of hsync in BlockReceiver only works if the output
streams are instances of FileOutputStream. Therefore, there is currently no
way for a FSDatasetSpi plugin to implement hsync if it is not using standard
OS files.
One possible solution is to push the implementation of hsync into the
ReplicaOutputStreams class. This class is constructed by the
ReplicaInPipeline which is constructed by the FSDatasetSpi plugin, therefore
it can be extended. Instead of directly calling sync on the output stream,
BlockReceiver would call ReplicaOutputStream.sync. The default
implementation of sync in ReplicaOutputStream would be the same as the
current implementation in BlockReceiver.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HDFS-5868) Make hsync implementation pluggable
[ https://issues.apache.org/jira/browse/HDFS-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13906049#comment-13906049 ] Arpit Agarwal commented on HDFS-5868: - +1 for the patch pending Jenkins. Make hsync implementation pluggable --- Key: HDFS-5868 URL: https://issues.apache.org/jira/browse/HDFS-5868 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode Affects Versions: 2.2.0 Reporter: Buddy Attachments: HDFS-5868-branch-2.patch, HDFS-5868a-branch-2.patch, HDFS-5868b-branch-2.patch The current implementation of hsync in BlockReceiver only works if the output streams are instances of FileOutputStream. Therefore, there is currently no way for a FSDatasetSpi plugin to implement hsync if it is not using standard OS files. One possible solution is to push the implementation of hsync into the ReplicaOutputStreams class. This class is constructed by the ReplicaInPipeline which is constructed by the FSDatasetSpi plugin, therefore it can be extended. Instead of directly calling sync on the output stream, BlockReceiver would call ReplicaOutputStream.sync. The default implementation of sync in ReplicaOutputStream would be the same as the current implementation in BlockReceiver. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HDFS-5868) Make hsync implementation pluggable
[
https://issues.apache.org/jira/browse/HDFS-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13906281#comment-13906281
]
Hadoop QA commented on HDFS-5868:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12629880/HDFS-5868b-branch-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.TestCacheDirectives
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6179//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6179//console
This message is automatically generated.
Make hsync implementation pluggable
---
Key: HDFS-5868
URL: https://issues.apache.org/jira/browse/HDFS-5868
Project: Hadoop HDFS
Issue Type: Sub-task
Components: datanode
Affects Versions: 2.2.0
Reporter: Buddy
Attachments: HDFS-5868-branch-2.patch, HDFS-5868a-branch-2.patch,
HDFS-5868b-branch-2.patch
The current implementation of hsync in BlockReceiver only works if the output
streams are instances of FileOutputStream. Therefore, there is currently no
way for a FSDatasetSpi plugin to implement hsync if it is not using standard
OS files.
One possible solution is to push the implementation of hsync into the
ReplicaOutputStreams class. This class is constructed by the
ReplicaInPipeline which is constructed by the FSDatasetSpi plugin, therefore
it can be extended. Instead of directly calling sync on the output stream,
BlockReceiver would call ReplicaOutputStream.sync. The default
implementation of sync in ReplicaOutputStream would be the same as the
current implementation in BlockReceiver.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HDFS-5868) Make hsync implementation pluggable
[ https://issues.apache.org/jira/browse/HDFS-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13906302#comment-13906302 ] Hudson commented on HDFS-5868: -- SUCCESS: Integrated in Hadoop-trunk-Commit #5192 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/5192/]) HDFS-5868. Make hsync implementation pluggable. (Contributed by Buddy Taylor) (arp: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1569978) * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/ReplicaOutputStreams.java Make hsync implementation pluggable --- Key: HDFS-5868 URL: https://issues.apache.org/jira/browse/HDFS-5868 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode Affects Versions: 2.2.0 Reporter: Buddy Fix For: 3.0.0, 2.4.0 Attachments: HDFS-5868-branch-2.patch, HDFS-5868a-branch-2.patch, HDFS-5868b-branch-2.patch The current implementation of hsync in BlockReceiver only works if the output streams are instances of FileOutputStream. Therefore, there is currently no way for a FSDatasetSpi plugin to implement hsync if it is not using standard OS files. One possible solution is to push the implementation of hsync into the ReplicaOutputStreams class. This class is constructed by the ReplicaInPipeline which is constructed by the FSDatasetSpi plugin, therefore it can be extended. Instead of directly calling sync on the output stream, BlockReceiver would call ReplicaOutputStream.sync. The default implementation of sync in ReplicaOutputStream would be the same as the current implementation in BlockReceiver. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HDFS-5868) Make hsync implementation pluggable
[ https://issues.apache.org/jira/browse/HDFS-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13896267#comment-13896267 ] David Powell commented on HDFS-5868: Buddy, Your latest patch looks great to me. The abstract class / interface approach definitely seems like the right way to go in the long term, but I can't speak with authority on when matters would weigh in favor of doing so. Make hsync implementation pluggable --- Key: HDFS-5868 URL: https://issues.apache.org/jira/browse/HDFS-5868 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode Affects Versions: 2.2.0 Reporter: Buddy Attachments: HDFS-5868-branch-2.patch, HDFS-5868a-branch-2.patch The current implementation of hsync in BlockReceiver only works if the output streams are instances of FileOutputStream. Therefore, there is currently no way for a FSDatasetSpi plugin to implement hsync if it is not using standard OS files. One possible solution is to push the implementation of hsync into the ReplicaOutputStreams class. This class is constructed by the ReplicaInPipeline which is constructed by the FSDatasetSpi plugin, therefore it can be extended. Instead of directly calling sync on the output stream, BlockReceiver would call ReplicaOutputStream.sync. The default implementation of sync in ReplicaOutputStream would be the same as the current implementation in BlockReceiver. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HDFS-5868) Make hsync implementation pluggable
[
https://issues.apache.org/jira/browse/HDFS-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13894619#comment-13894619
]
Buddy commented on HDFS-5868:
-
David,
Thanks very much for looking at this and for the suggestions.
* We see that LOG.warn in our logs and that is certainly an area that we would
like to clean up. However, I think that we need to make {{manageWriterOsCache}}
more pluggable to deal with that. The LOG.warn is telling us that some
functionality is being bypassed, which seems like a valid warning until the
FsDatasetSpi plugins are able to provide that functionality. Perhaps I should
file a new sub-jira on that?
* I fully agree with your comments on the sync(OutputStream) signature and I
like your solution. I have attached a new patch against the latest trunk.
Thanks very much for pointing that out.
In the latest patch, I still check that the streams are an instance of
FileOutputStream in the syncDataOut() and syncChecksumOut() methods. I think
that can be avoided by making ReplicaOutputStreams an abstract class with
abstract methods for syncDataOut() abd syncChecksumOut(). Then we could have
ReplicaFileOutputStreams and ReplicaSimulatedOutputStreams that extend the
abstract class and implement the sync methods, and FsDatasetSpi providers could
have their own subclass with their own implementations, as we do.
ReplicaFileOutputStreams would be constructed by
ReplicaInPipeline.createStreams since it currently uses FileOutputStreams.
ReplicaSimulatedOutputStreams would be constructed by SimulatedFSDataset, which
does not currently use FileOutputStreams.
In order to make this work, the abstract class ReplicaOutputStreams would also
have to use generics because the Streams are passed into the constructor.
It seems to me like this is a bit far to go for this patch, and that we should
consider solving the manageWriterOsCache problem before doing something like
this. But I would like to get your feedback and am happy to do it if the
community thinks it is a good solution at this point.
Make hsync implementation pluggable
---
Key: HDFS-5868
URL: https://issues.apache.org/jira/browse/HDFS-5868
Project: Hadoop HDFS
Issue Type: Sub-task
Components: datanode
Affects Versions: 2.2.0
Reporter: Buddy
Attachments: HDFS-5868-branch-2.patch
The current implementation of hsync in BlockReceiver only works if the output
streams are instances of FileOutputStream. Therefore, there is currently no
way for a FSDatasetSpi plugin to implement hsync if it is not using standard
OS files.
One possible solution is to push the implementation of hsync into the
ReplicaOutputStreams class. This class is constructed by the
ReplicaInPipeline which is constructed by the FSDatasetSpi plugin, therefore
it can be extended. Instead of directly calling sync on the output stream,
BlockReceiver would call ReplicaOutputStream.sync. The default
implementation of sync in ReplicaOutputStream would be the same as the
current implementation in BlockReceiver.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HDFS-5868) Make hsync implementation pluggable
[
https://issues.apache.org/jira/browse/HDFS-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13894769#comment-13894769
]
Hadoop QA commented on HDFS-5868:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12627634/HDFS-5868a-branch-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6069//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6069//console
This message is automatically generated.
Make hsync implementation pluggable
---
Key: HDFS-5868
URL: https://issues.apache.org/jira/browse/HDFS-5868
Project: Hadoop HDFS
Issue Type: Sub-task
Components: datanode
Affects Versions: 2.2.0
Reporter: Buddy
Attachments: HDFS-5868-branch-2.patch, HDFS-5868a-branch-2.patch
The current implementation of hsync in BlockReceiver only works if the output
streams are instances of FileOutputStream. Therefore, there is currently no
way for a FSDatasetSpi plugin to implement hsync if it is not using standard
OS files.
One possible solution is to push the implementation of hsync into the
ReplicaOutputStreams class. This class is constructed by the
ReplicaInPipeline which is constructed by the FSDatasetSpi plugin, therefore
it can be extended. Instead of directly calling sync on the output stream,
BlockReceiver would call ReplicaOutputStream.sync. The default
implementation of sync in ReplicaOutputStream would be the same as the
current implementation in BlockReceiver.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HDFS-5868) Make hsync implementation pluggable
[ https://issues.apache.org/jira/browse/HDFS-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13894069#comment-13894069 ] David Powell commented on HDFS-5868: Buddy, Very nice. A couple comments: * Given the acknowledgement of non-FileOutputStream OutputStreams, the LOG.warn() in BlockReceiver() when getDataOut() is not a FileOutputStream is probably producing unwanted output. * I'm not a fan of the sync(OutputStream) method signature. The premise is, I assume, to sync one of the streams associated with the ReplicaOutputStream. sync(OutputStream) lets one pass in *any* stream, which means an implementation will end up doing one of two things: *# Do the right thing if the OutputStream is the correct type or nothing if not. This works fine if callers only pass in correctly coordinated OutputStreams, but otherwise is useless. (This is what the attached patch does). *# Compare the OutputStream reference against the streams kept internally and perform the correct action depending on which it is. (This might be done if you had a simple OutputStream adapter subclass to state managed otherwise by the ReplicaOutputStream.) * In either case, the interface lets the caller pass in something wrong, which means the implementation has to test for that before performing the only action that should ever be performed for a correctly-written consumer. Instead, I recommend separate syncDataOut() and syncChecksumOut() methods that do exactly what they say and require no validity checking (a win for performance, maintenance, and testing). Make hsync implementation pluggable --- Key: HDFS-5868 URL: https://issues.apache.org/jira/browse/HDFS-5868 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode Affects Versions: 2.2.0 Reporter: Buddy Attachments: HDFS-5868-branch-2.patch The current implementation of hsync in BlockReceiver only works if the output streams are instances of FileOutputStream. Therefore, there is currently no way for a FSDatasetSpi plugin to implement hsync if it is not using standard OS files. One possible solution is to push the implementation of hsync into the ReplicaOutputStreams class. This class is constructed by the ReplicaInPipeline which is constructed by the FSDatasetSpi plugin, therefore it can be extended. Instead of directly calling sync on the output stream, BlockReceiver would call ReplicaOutputStream.sync. The default implementation of sync in ReplicaOutputStream would be the same as the current implementation in BlockReceiver. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HDFS-5868) Make hsync implementation pluggable
[
https://issues.apache.org/jira/browse/HDFS-5868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13891453#comment-13891453
]
Hadoop QA commented on HDFS-5868:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12626979/HDFS-5868-branch-2.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
-14 warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:red}-1 release audit{color}. The applied patch generated 1
release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-hdfs-project/hadoop-hdfs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/6026//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-HDFS-Build/6026//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6026//console
This message is automatically generated.
Make hsync implementation pluggable
---
Key: HDFS-5868
URL: https://issues.apache.org/jira/browse/HDFS-5868
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.2.0
Reporter: Buddy
Attachments: HDFS-5868-branch-2.patch
The current implementation of hsync in BlockReceiver only works if the output
streams are instances of FileOutputStream. Therefore, there is currently no
way for a FSDatasetSpi plugin to implement hsync if it is not using standard
OS files.
One possible solution is to push the implementation of hsync into the
ReplicaOutputStreams class. This class is constructed by the
ReplicaInPipeline which is constructed by the FSDatasetSpi plugin, therefore
it can be extended. Instead of directly calling sync on the output stream,
BlockReceiver would call ReplicaOutputStream.sync. The default
implementation of sync in ReplicaOutputStream would be the same as the
current implementation in BlockReceiver.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
