[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14726667#comment-14726667 ] Hudson commented on HADOOP-11710: - FAILURE: Integrated in HBase-TRUNK #6771 (See [https://builds.apache.org/job/HBase-TRUNK/6771/]) HBASE-13221 List Hadoop 2.6 as unsupported due to HADOOP-11710. (busbey: rev e54cb8abebb2971da68852c6dcef456dbc689afb) * src/main/asciidoc/_chapters/configuration.adoc > Make CryptoOutputStream behave like DFSOutputStream wrt synchronization > --- > > Key: HADOOP-11710 > URL: https://issues.apache.org/jira/browse/HADOOP-11710 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs >Affects Versions: 2.6.0 >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Labels: 2.6.1-candidate > Fix For: 2.6.1, 2.7.0 > > Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt, > HADOOP-11710.3.patch.txt > > > per discussion on parent, as an intermediate solution make CryptoOutputStream > behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[
https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362630#comment-14362630
]
Yi Liu commented on HADOOP-11710:
-
{quote}
I cherry-picked this to branch-2.7
{quote}
Oh, I missed that. Thanks for committing to branch-2.7, [~ozawa].
Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
---
Key: HADOOP-11710
URL: https://issues.apache.org/jira/browse/HADOOP-11710
Project: Hadoop Common
Issue Type: Sub-task
Components: fs
Affects Versions: 2.6.0
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Critical
Fix For: 2.7.0
Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt,
HADOOP-11710.3.patch.txt
per discussion on parent, as an intermediate solution make CryptoOutputStream
behave like DFSOutputStream
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362225#comment-14362225 ] Tsuyoshi Ozawa commented on HADOOP-11710: - [~hitliuyi], I cherry-picked this to branch-2.7 since it's not committed to the branch and you targeted this to the branch. Feel free to revert it if you find any problem. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization --- Key: HADOOP-11710 URL: https://issues.apache.org/jira/browse/HADOOP-11710 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 2.6.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Critical Fix For: 2.7.0 Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt, HADOOP-11710.3.patch.txt per discussion on parent, as an intermediate solution make CryptoOutputStream behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360248#comment-14360248 ] Hudson commented on HADOOP-11710: - SUCCESS: Integrated in Hadoop-Yarn-trunk #865 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/865/]) HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization. (Sean Busbey via yliu) (yliu: rev a85291003cf3e3fd79b6addcf59d4f43dc72d356) * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/CryptoOutputStream.java Make CryptoOutputStream behave like DFSOutputStream wrt synchronization --- Key: HADOOP-11710 URL: https://issues.apache.org/jira/browse/HADOOP-11710 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 2.6.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Critical Fix For: 2.7.0 Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt, HADOOP-11710.3.patch.txt per discussion on parent, as an intermediate solution make CryptoOutputStream behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360428#comment-14360428 ] Hudson commented on HADOOP-11710: - FAILURE: Integrated in Hadoop-Hdfs-trunk #2063 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2063/]) HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization. (Sean Busbey via yliu) (yliu: rev a85291003cf3e3fd79b6addcf59d4f43dc72d356) * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/CryptoOutputStream.java Make CryptoOutputStream behave like DFSOutputStream wrt synchronization --- Key: HADOOP-11710 URL: https://issues.apache.org/jira/browse/HADOOP-11710 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 2.6.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Critical Fix For: 2.7.0 Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt, HADOOP-11710.3.patch.txt per discussion on parent, as an intermediate solution make CryptoOutputStream behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360435#comment-14360435 ] Hudson commented on HADOOP-11710: - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #122 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/122/]) HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization. (Sean Busbey via yliu) (yliu: rev a85291003cf3e3fd79b6addcf59d4f43dc72d356) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/CryptoOutputStream.java * hadoop-common-project/hadoop-common/CHANGES.txt Make CryptoOutputStream behave like DFSOutputStream wrt synchronization --- Key: HADOOP-11710 URL: https://issues.apache.org/jira/browse/HADOOP-11710 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 2.6.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Critical Fix For: 2.7.0 Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt, HADOOP-11710.3.patch.txt per discussion on parent, as an intermediate solution make CryptoOutputStream behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360496#comment-14360496 ] Hudson commented on HADOOP-11710: - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #131 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/131/]) HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization. (Sean Busbey via yliu) (yliu: rev a85291003cf3e3fd79b6addcf59d4f43dc72d356) * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/CryptoOutputStream.java Make CryptoOutputStream behave like DFSOutputStream wrt synchronization --- Key: HADOOP-11710 URL: https://issues.apache.org/jira/browse/HADOOP-11710 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 2.6.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Critical Fix For: 2.7.0 Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt, HADOOP-11710.3.patch.txt per discussion on parent, as an intermediate solution make CryptoOutputStream behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14360361#comment-14360361 ] Hudson commented on HADOOP-11710: - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2081 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2081/]) HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization. (Sean Busbey via yliu) (yliu: rev a85291003cf3e3fd79b6addcf59d4f43dc72d356) * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/CryptoOutputStream.java * hadoop-common-project/hadoop-common/CHANGES.txt Make CryptoOutputStream behave like DFSOutputStream wrt synchronization --- Key: HADOOP-11710 URL: https://issues.apache.org/jira/browse/HADOOP-11710 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 2.6.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Critical Fix For: 2.7.0 Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt, HADOOP-11710.3.patch.txt per discussion on parent, as an intermediate solution make CryptoOutputStream behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359887#comment-14359887 ] Hudson commented on HADOOP-11710: - FAILURE: Integrated in Hadoop-trunk-Commit #7314 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7314/]) HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization. (Sean Busbey via yliu) (yliu: rev a85291003cf3e3fd79b6addcf59d4f43dc72d356) * hadoop-common-project/hadoop-common/CHANGES.txt * hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/CryptoOutputStream.java Make CryptoOutputStream behave like DFSOutputStream wrt synchronization --- Key: HADOOP-11710 URL: https://issues.apache.org/jira/browse/HADOOP-11710 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 2.6.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Critical Fix For: 2.7.0 Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt, HADOOP-11710.3.patch.txt per discussion on parent, as an intermediate solution make CryptoOutputStream behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359815#comment-14359815 ] Yi Liu commented on HADOOP-11710: - +1 pending Jenkins. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization --- Key: HADOOP-11710 URL: https://issues.apache.org/jira/browse/HADOOP-11710 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 2.6.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Critical Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt, HADOOP-11710.3.patch.txt per discussion on parent, as an intermediate solution make CryptoOutputStream behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[
https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359392#comment-14359392
]
Hadoop QA commented on HADOOP-11710:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704218/HADOOP-11710.1.patch.txt
against trunk revision b49c3a1.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-common-project/hadoop-common.
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/5929//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/5929//console
This message is automatically generated.
Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
---
Key: HADOOP-11710
URL: https://issues.apache.org/jira/browse/HADOOP-11710
Project: Hadoop Common
Issue Type: Sub-task
Components: fs
Affects Versions: 2.6.0
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Critical
Attachments: HADOOP-11710.1.patch.txt
per discussion on parent, as an intermediate solution make CryptoOutputStream
behave like DFSOutputStream
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359465#comment-14359465 ] Colin Patrick McCabe commented on HADOOP-11710: --- Thanks, Sean. I think unit testing is gonna be tough because this is adding synchronization. By their nature, synchronization bugs tend to show up very intermittently. Since this is just adding synchronized to 3 functions I would say +1 as-is. I will give others a bit to comment before I commit Make CryptoOutputStream behave like DFSOutputStream wrt synchronization --- Key: HADOOP-11710 URL: https://issues.apache.org/jira/browse/HADOOP-11710 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 2.6.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Critical Attachments: HADOOP-11710.1.patch.txt per discussion on parent, as an intermediate solution make CryptoOutputStream behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359395#comment-14359395 ] Sean Busbey commented on HADOOP-11710: -- I'm attempting to manually test, but I've had tons of problems reproducing the failure reliably on my own machines. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization --- Key: HADOOP-11710 URL: https://issues.apache.org/jira/browse/HADOOP-11710 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 2.6.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Critical Attachments: HADOOP-11710.1.patch.txt per discussion on parent, as an intermediate solution make CryptoOutputStream behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359269#comment-14359269 ] Sean Busbey commented on HADOOP-11710: -- knew I forgot something. thanks Steve! Make CryptoOutputStream behave like DFSOutputStream wrt synchronization --- Key: HADOOP-11710 URL: https://issues.apache.org/jira/browse/HADOOP-11710 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 2.6.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Critical Attachments: HADOOP-11710.1.patch.txt per discussion on parent, as an intermediate solution make CryptoOutputStream behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[
https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359661#comment-14359661
]
Hadoop QA commented on HADOOP-11710:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704284/HADOOP-11710.2.patch.txt
against trunk revision 8212877.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common:
org.apache.hadoop.crypto.TestCryptoStreamsForLocalFS
org.apache.hadoop.crypto.TestCryptoStreamsWithOpensslAesCtrCryptoCodec
org.apache.hadoop.crypto.TestCryptoStreams
org.apache.hadoop.crypto.TestCryptoStreamsNormal
Test results:
https://builds.apache.org/job/PreCommit-HADOOP-Build/5930//testReport/
Console output:
https://builds.apache.org/job/PreCommit-HADOOP-Build/5930//console
This message is automatically generated.
Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
---
Key: HADOOP-11710
URL: https://issues.apache.org/jira/browse/HADOOP-11710
Project: Hadoop Common
Issue Type: Sub-task
Components: fs
Affects Versions: 2.6.0
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Critical
Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt
per discussion on parent, as an intermediate solution make CryptoOutputStream
behave like DFSOutputStream
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359670#comment-14359670 ] Sean Busbey commented on HADOOP-11710: -- Those look related. Will take a look in a couple hours or so. -- Sean Make CryptoOutputStream behave like DFSOutputStream wrt synchronization --- Key: HADOOP-11710 URL: https://issues.apache.org/jira/browse/HADOOP-11710 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 2.6.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Critical Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt per discussion on parent, as an intermediate solution make CryptoOutputStream behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359568#comment-14359568 ] Steve Loughran commented on HADOOP-11710: - we don't have any formal FS concurrency tests. The load generator does this, but it's unlikely to be doing the operation replicating this problem. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization --- Key: HADOOP-11710 URL: https://issues.apache.org/jira/browse/HADOOP-11710 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 2.6.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Critical Attachments: HADOOP-11710.1.patch.txt per discussion on parent, as an intermediate solution make CryptoOutputStream behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[
https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359574#comment-14359574
]
Steve Loughran commented on HADOOP-11710:
-
Looking at the code, it seems OK to me, so +1
However, I would recommend one change, which is in {{close()}}, move the
close=true operation up immediately after the close check, just in case
something in {{freeBuffers() }} raised an exception or the parent did -it'll
stop a second close() call getting into a mess. This is not really related to
the rest of the patch, except in the general improve re-entrancy context
Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
---
Key: HADOOP-11710
URL: https://issues.apache.org/jira/browse/HADOOP-11710
Project: Hadoop Common
Issue Type: Sub-task
Components: fs
Affects Versions: 2.6.0
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Critical
Attachments: HADOOP-11710.1.patch.txt
per discussion on parent, as an intermediate solution make CryptoOutputStream
behave like DFSOutputStream
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359647#comment-14359647 ] Xiaoyu Yao commented on HADOOP-11710: - I have tested the v2 patch with my unit test that repros the original issue. +1 (non-binding). I can post the new unit test under a separate JIRA. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization --- Key: HADOOP-11710 URL: https://issues.apache.org/jira/browse/HADOOP-11710 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 2.6.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Critical Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt per discussion on parent, as an intermediate solution make CryptoOutputStream behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[
https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359737#comment-14359737
]
Yi Liu commented on HADOOP-11710:
-
Sean, don't move {{closed = true;}}. {{super.close();}} will invoke flush to
flush the remaining data in the buffer, if we set *closed* to true before
invoking {{super.close()}}, we will get error.
I think the test failure should be related to this.
Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
---
Key: HADOOP-11710
URL: https://issues.apache.org/jira/browse/HADOOP-11710
Project: Hadoop Common
Issue Type: Sub-task
Components: fs
Affects Versions: 2.6.0
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Critical
Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt
per discussion on parent, as an intermediate solution make CryptoOutputStream
behave like DFSOutputStream
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[ https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359741#comment-14359741 ] Sean Busbey commented on HADOOP-11710: -- Oh right, of course! Yes that's definitely the cause of the errors. Patch vers 1 should be used then, unless someone feels strongly about the placement wrt disposing of the direct buffers. Make CryptoOutputStream behave like DFSOutputStream wrt synchronization --- Key: HADOOP-11710 URL: https://issues.apache.org/jira/browse/HADOOP-11710 Project: Hadoop Common Issue Type: Sub-task Components: fs Affects Versions: 2.6.0 Reporter: Sean Busbey Assignee: Sean Busbey Priority: Critical Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt per discussion on parent, as an intermediate solution make CryptoOutputStream behave like DFSOutputStream -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-11710) Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
[
https://issues.apache.org/jira/browse/HADOOP-11710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14359743#comment-14359743
]
Yi Liu commented on HADOOP-11710:
-
Oh, I just see Steve's comments
{quote}
However, I would recommend one change, which is in close(), move the close=true
operation up immediately after the close check, just in case something in
{{freeBuffers() }} raised an exception or the parent did -it'll stop a second
close() call getting into a mess. This is not really related to the rest of the
patch, except in the general improve re-entrancy contex
{quote}
I agree we should make {{closed}} be set to false, also I think use sun's API
to release directbuffer rarely failed. Maybe we can put it in {{try...
finally}}.
Make CryptoOutputStream behave like DFSOutputStream wrt synchronization
---
Key: HADOOP-11710
URL: https://issues.apache.org/jira/browse/HADOOP-11710
Project: Hadoop Common
Issue Type: Sub-task
Components: fs
Affects Versions: 2.6.0
Reporter: Sean Busbey
Assignee: Sean Busbey
Priority: Critical
Attachments: HADOOP-11710.1.patch.txt, HADOOP-11710.2.patch.txt
per discussion on parent, as an intermediate solution make CryptoOutputStream
behave like DFSOutputStream
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
