[jira] [Commented] (HDFS-6813) WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable with thead-safe.
[
https://issues.apache.org/jira/browse/HDFS-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15805965#comment-15805965
]
Hadoop QA commented on HDFS-6813:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HDFS-6813 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HDFS-6813 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12659533/HDFS-6813.001.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/18072/console |
| Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable
> with thead-safe.
> ---
>
> Key: HDFS-6813
> URL: https://issues.apache.org/jira/browse/HDFS-6813
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 2.6.0
>Reporter: Yi Liu
>Assignee: Yi Liu
> Attachments: HDFS-6813.001.patch
>
>
> {{PositionedReadable}} definition requires the implementations for its
> interfaces should be thread-safe.
> OffsetUrlInputStream(WebHdfsFileSystem inputstream) doesn't implement these
> interfaces with tread-safe, this JIRA is to fix this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-6813) WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable with thead-safe.
[
https://issues.apache.org/jira/browse/HDFS-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14524873#comment-14524873
]
Hadoop QA commented on HDFS-6813:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12659533/HDFS-6813.001.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10640/console |
This message was automatically generated.
WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable
with thead-safe.
---
Key: HDFS-6813
URL: https://issues.apache.org/jira/browse/HDFS-6813
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.6.0
Reporter: Yi Liu
Assignee: Yi Liu
Attachments: HDFS-6813.001.patch
{{PositionedReadable}} definition requires the implementations for its
interfaces should be thread-safe.
OffsetUrlInputStream(WebHdfsFileSystem inputstream) doesn't implement these
interfaces with tread-safe, this JIRA is to fix this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6813) WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable with thead-safe.
[
https://issues.apache.org/jira/browse/HDFS-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14524889#comment-14524889
]
Hadoop QA commented on HDFS-6813:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | patch | 0m 0s | The patch command could not apply
the patch during dryrun. |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12659533/HDFS-6813.001.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / f1a152c |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/10645/console |
This message was automatically generated.
WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable
with thead-safe.
---
Key: HDFS-6813
URL: https://issues.apache.org/jira/browse/HDFS-6813
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.6.0
Reporter: Yi Liu
Assignee: Yi Liu
Attachments: HDFS-6813.001.patch
{{PositionedReadable}} definition requires the implementations for its
interfaces should be thread-safe.
OffsetUrlInputStream(WebHdfsFileSystem inputstream) doesn't implement these
interfaces with tread-safe, this JIRA is to fix this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HDFS-6813) WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable with thead-safe.
[
https://issues.apache.org/jira/browse/HDFS-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086007#comment-14086007
]
Steve Loughran commented on HDFS-6813:
--
Nicholas, can you look at HADOOP-10938 and see if we can pull this discussion
together.
For HBase they are looking at what could be done for concurrent HDFS
operations, that is what can be relaxed/unrelaxed.
The native IO streams are all generally wrapped by RawLocal and checksummed
filesystems, so they get their sync semantics from there.
What we do have, however is a model in which {{getPos() }} can block until a
potentially remote, long-haul IO operation completes, which I think is an
anomaly.
WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable
with thead-safe.
---
Key: HDFS-6813
URL: https://issues.apache.org/jira/browse/HDFS-6813
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.6.0
Reporter: Yi Liu
Assignee: Yi Liu
Attachments: HDFS-6813.001.patch
{{PositionedReadable}} definition requires the implementations for its
interfaces should be thread-safe.
OffsetUrlInputStream(WebHdfsFileSystem inputstream) doesn't implement these
interfaces with tread-safe, this JIRA is to fix this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6813) WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable with thead-safe.
[
https://issues.apache.org/jira/browse/HDFS-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086322#comment-14086322
]
Yi Liu commented on HDFS-6813:
--
Thanks [~szetszwo]
{quote}
if it is needed, it is very easy to be synchronized in the user applications
{quote}
Agree.
{quote}
I suggest simple update the javadoc of PositionedReadable. What do you think?
{quote}
I opened a new JIRA HADOOP-10938 for javadoc. Let's discuss with
[~ste...@apache.org] and other guys together to see what thread safety mode we
want. Then we can decide whether to modify the javadoc or make implementations
thread-safe.
WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable
with thead-safe.
---
Key: HDFS-6813
URL: https://issues.apache.org/jira/browse/HDFS-6813
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.6.0
Reporter: Yi Liu
Assignee: Yi Liu
Attachments: HDFS-6813.001.patch
{{PositionedReadable}} definition requires the implementations for its
interfaces should be thread-safe.
OffsetUrlInputStream(WebHdfsFileSystem inputstream) doesn't implement these
interfaces with tread-safe, this JIRA is to fix this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6813) WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable with thead-safe.
[
https://issues.apache.org/jira/browse/HDFS-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14086434#comment-14086434
]
Yi Liu commented on HDFS-6813:
--
Hi [~szetszwo] and [~ste...@apache.org], after further thinking, I think we
need to make thread-safe in DFSInputStream, my points are:
* Pread in org.apache.hadoop.hdfs.DFSInputStream doesn't modify the pos, so we
have opportunity to make read and pread concurrent. We need to synchronize few
object variables like {{failures}}, we can define local variable to resolve
this problem.
If Pread doesn't access object variable, then it's thread-safe and we don't
need to synchronize it with other Ops like read, seek.
* If we leave the synchronization to user applications, then read and pread
*cannot* be concurrent.
WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable
with thead-safe.
---
Key: HDFS-6813
URL: https://issues.apache.org/jira/browse/HDFS-6813
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.6.0
Reporter: Yi Liu
Assignee: Yi Liu
Attachments: HDFS-6813.001.patch
{{PositionedReadable}} definition requires the implementations for its
interfaces should be thread-safe.
OffsetUrlInputStream(WebHdfsFileSystem inputstream) doesn't implement these
interfaces with tread-safe, this JIRA is to fix this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6813) WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable with thead-safe.
[
https://issues.apache.org/jira/browse/HDFS-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14084688#comment-14084688
]
Tsz Wo Nicholas Sze commented on HDFS-6813:
---
The javadoc in PositionedReadable may be out dated. Even DFSInputStream does
not seem to be thread safe.
WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable
with thead-safe.
---
Key: HDFS-6813
URL: https://issues.apache.org/jira/browse/HDFS-6813
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.6.0
Reporter: Yi Liu
Assignee: Yi Liu
Attachments: HDFS-6813.001.patch
{{PositionedReadable}} definition requires the implementations for its
interfaces should be thread-safe.
OffsetUrlInputStream(WebHdfsFileSystem inputstream) doesn't implement these
interfaces with tread-safe, this JIRA is to fix this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6813) WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable with thead-safe.
[
https://issues.apache.org/jira/browse/HDFS-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14084734#comment-14084734
]
Yi Liu commented on HDFS-6813:
--
Hi [~szetszwo], thanks for your comments. I think the original intention in
{{DFSInputStream}} is to make thread-safe for PositionedReadable
implementation, we can see there are *synchronized* for other methods; normal
read is synchronized, and there is no reason that positioned read is not
synchronized (few object variables are used in this method, maybe forget to
synchronize?). I have checked other inputstreams in Hadoop which implement
PositionedReadable almost are thread-safe. If we don't want thread-safe, it's
better to remove synchronized, then it's a bit more efficient.
What's your suggestion about this?
WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable
with thead-safe.
---
Key: HDFS-6813
URL: https://issues.apache.org/jira/browse/HDFS-6813
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.6.0
Reporter: Yi Liu
Assignee: Yi Liu
Attachments: HDFS-6813.001.patch
{{PositionedReadable}} definition requires the implementations for its
interfaces should be thread-safe.
OffsetUrlInputStream(WebHdfsFileSystem inputstream) doesn't implement these
interfaces with tread-safe, this JIRA is to fix this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6813) WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable with thead-safe.
[
https://issues.apache.org/jira/browse/HDFS-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14085634#comment-14085634
]
Tsz Wo Nicholas Sze commented on HDFS-6813:
---
I think requiring PositionedReadable to be thread-safe may be overkill. I
believe few user applications need thread-safe and, if it is needed, it is very
easy to be synchronized in the user applications. Just have checked Java
InputStream and DataInputStream, they are not enforcing thread-safe. The
[DataInputStream
javadoc|http://docs.oracle.com/javase/7/docs/api/java/io/DataInputStream.html]
even emphasizes that it is not thread-safe:
bq. DataInputStream is not necessarily safe for multithreaded access. Thread
safety is optional and is the responsibility of users of methods in this class.
I suggest simple update the javadoc of PositionedReadable. What do you think?
WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable
with thead-safe.
---
Key: HDFS-6813
URL: https://issues.apache.org/jira/browse/HDFS-6813
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.6.0
Reporter: Yi Liu
Assignee: Yi Liu
Attachments: HDFS-6813.001.patch
{{PositionedReadable}} definition requires the implementations for its
interfaces should be thread-safe.
OffsetUrlInputStream(WebHdfsFileSystem inputstream) doesn't implement these
interfaces with tread-safe, this JIRA is to fix this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6813) WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable with thead-safe.
[
https://issues.apache.org/jira/browse/HDFS-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14083956#comment-14083956
]
Hadoop QA commented on HDFS-6813:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12659533/HDFS-6813.001.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
org.apache.hadoop.hdfs.server.namenode.ha.TestDFSZKFailoverController
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-HDFS-Build/7546//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/7546//console
This message is automatically generated.
WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable
with thead-safe.
---
Key: HDFS-6813
URL: https://issues.apache.org/jira/browse/HDFS-6813
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.6.0
Reporter: Yi Liu
Assignee: Yi Liu
Attachments: HDFS-6813.001.patch
{{PositionedReadable}} definition requires the implementations for its
interfaces should be thread-safe.
OffsetUrlInputStream(WebHdfsFileSystem inputstream) doesn't implement these
interfaces with tread-safe, this JIRA is to fix this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HDFS-6813) WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable with thead-safe.
[
https://issues.apache.org/jira/browse/HDFS-6813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14084056#comment-14084056
]
Uma Maheswara Rao G commented on HDFS-6813:
---
I think from PositionedReadable doc, this looks reasonable to me. But I also
noticed FsInputStream also have the APIs with out synchronized. Also the read
api in DFSInputStream also not synchronized, but sure that was left with
synchronization with intention.
[~szetszwo], can you please confirm if there is any reason for not synchronized
and did not follow the PositionedReadable java doc? Thanks
WebHdfsFileSystem#OffsetUrlInputStream should implement PositionedReadable
with thead-safe.
---
Key: HDFS-6813
URL: https://issues.apache.org/jira/browse/HDFS-6813
Project: Hadoop HDFS
Issue Type: Bug
Components: webhdfs
Affects Versions: 2.6.0
Reporter: Yi Liu
Assignee: Yi Liu
Attachments: HDFS-6813.001.patch
{{PositionedReadable}} definition requires the implementations for its
interfaces should be thread-safe.
OffsetUrlInputStream(WebHdfsFileSystem inputstream) doesn't implement these
interfaces with tread-safe, this JIRA is to fix this.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
