[jira] [Updated] (HDFS-2408) DFSClient#getNumCurrentReplicas is package private in 205 but public in branch-0.20-append
[
https://issues.apache.org/jira/browse/HDFS-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HDFS-2408:
-
Resolution: Fixed
Fix Version/s: 0.20.206.0
0.20.205.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Thanks Suresh. I've just committed this to branch-0.20-security and
branch-0.20-security-205.
> DFSClient#getNumCurrentReplicas is package private in 205 but public in
> branch-0.20-append
> --
>
> Key: HDFS-2408
> URL: https://issues.apache.org/jira/browse/HDFS-2408
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
>Affects Versions: 0.20.205.1
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Fix For: 0.20.205.0, 0.20.206.0
>
> Attachments: hdfs-2408.txt
>
>
> The below commit broke hdfs-826 for hbase in 205 rc1. It changes the
> accessiblity from public to package private on getNumCurrentReplicas and now
> current shipping hbase's at least cannot get at this method.
> {code}
> Revision 1174483 - (view) (download) (annotate) - [select for diffs]
> Modified Fri Sep 23 01:30:18 2011 UTC (13 days, 4 hours ago) by szetszwo
> File length: 136876 byte(s)
> Diff to previous 1174479 (colored)
> svn merge -c 1171137 from branch-0.20-security for HDFS-2333.
> {code}
> Here is diff between above change and one just previous:
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20-security-205/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java?view=diff&r1=1174479&r2=1174483&diff_format=u
> This is a critical facility for us.
> It seems like making this one method public again is all thats needed. I can
> make a patch like the below:
> diff --git a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> index b9cb053..39955c9 100644
> --- a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> +++ b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> @@ -3569,7 +3569,7 @@ public class DFSClient implements FSConstants,
> java.io.Closeable {
> * block is not yet allocated, then this API will return 0 because there
> are
> * no replicas in the pipeline.
> */
> -int getNumCurrentReplicas() throws IOException {
> +public int getNumCurrentReplicas() throws IOException {
>synchronized(dataQueue) {
> if (nodes == null) {
>return blockReplication;
> Can we get this into RC2?
> Thanks,
> St.Ack
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2408) DFSClient#getNumCurrentReplicas is package private in 205 but public in branch-0.20-append
[
https://issues.apache.org/jira/browse/HDFS-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121764#comment-13121764
]
Hadoop QA commented on HDFS-2408:
-
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12497951/hdfs-2408.txt
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1343//console
This message is automatically generated.
> DFSClient#getNumCurrentReplicas is package private in 205 but public in
> branch-0.20-append
> --
>
> Key: HDFS-2408
> URL: https://issues.apache.org/jira/browse/HDFS-2408
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
>Affects Versions: 0.20.205.1
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Attachments: hdfs-2408.txt
>
>
> The below commit broke hdfs-826 for hbase in 205 rc1. It changes the
> accessiblity from public to package private on getNumCurrentReplicas and now
> current shipping hbase's at least cannot get at this method.
> {code}
> Revision 1174483 - (view) (download) (annotate) - [select for diffs]
> Modified Fri Sep 23 01:30:18 2011 UTC (13 days, 4 hours ago) by szetszwo
> File length: 136876 byte(s)
> Diff to previous 1174479 (colored)
> svn merge -c 1171137 from branch-0.20-security for HDFS-2333.
> {code}
> Here is diff between above change and one just previous:
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20-security-205/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java?view=diff&r1=1174479&r2=1174483&diff_format=u
> This is a critical facility for us.
> It seems like making this one method public again is all thats needed. I can
> make a patch like the below:
> diff --git a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> index b9cb053..39955c9 100644
> --- a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> +++ b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> @@ -3569,7 +3569,7 @@ public class DFSClient implements FSConstants,
> java.io.Closeable {
> * block is not yet allocated, then this API will return 0 because there
> are
> * no replicas in the pipeline.
> */
> -int getNumCurrentReplicas() throws IOException {
> +public int getNumCurrentReplicas() throws IOException {
>synchronized(dataQueue) {
> if (nodes == null) {
>return blockReplication;
> Can we get this into RC2?
> Thanks,
> St.Ack
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2408) DFSClient#getNumCurrentReplicas is package private in 205 but public in branch-0.20-append
[
https://issues.apache.org/jira/browse/HDFS-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121761#comment-13121761
]
Suresh Srinivas commented on HDFS-2408:
---
Can you please commit this at the earlier. I can do that if you are busy. This
will help get rc2 out sooner.
> DFSClient#getNumCurrentReplicas is package private in 205 but public in
> branch-0.20-append
> --
>
> Key: HDFS-2408
> URL: https://issues.apache.org/jira/browse/HDFS-2408
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
>Affects Versions: 0.20.205.1
>Reporter: stack
>Assignee: stack
> Attachments: hdfs-2408.txt
>
>
> The below commit broke hdfs-826 for hbase in 205 rc1. It changes the
> accessiblity from public to package private on getNumCurrentReplicas and now
> current shipping hbase's at least cannot get at this method.
> {code}
> Revision 1174483 - (view) (download) (annotate) - [select for diffs]
> Modified Fri Sep 23 01:30:18 2011 UTC (13 days, 4 hours ago) by szetszwo
> File length: 136876 byte(s)
> Diff to previous 1174479 (colored)
> svn merge -c 1171137 from branch-0.20-security for HDFS-2333.
> {code}
> Here is diff between above change and one just previous:
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20-security-205/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java?view=diff&r1=1174479&r2=1174483&diff_format=u
> This is a critical facility for us.
> It seems like making this one method public again is all thats needed. I can
> make a patch like the below:
> diff --git a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> index b9cb053..39955c9 100644
> --- a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> +++ b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> @@ -3569,7 +3569,7 @@ public class DFSClient implements FSConstants,
> java.io.Closeable {
> * block is not yet allocated, then this API will return 0 because there
> are
> * no replicas in the pipeline.
> */
> -int getNumCurrentReplicas() throws IOException {
> +public int getNumCurrentReplicas() throws IOException {
>synchronized(dataQueue) {
> if (nodes == null) {
>return blockReplication;
> Can we get this into RC2?
> Thanks,
> St.Ack
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2408) DFSClient#getNumCurrentReplicas is package private in 205 but public in branch-0.20-append
[
https://issues.apache.org/jira/browse/HDFS-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suresh Srinivas updated HDFS-2408:
--
Priority: Blocker (was: Major)
> DFSClient#getNumCurrentReplicas is package private in 205 but public in
> branch-0.20-append
> --
>
> Key: HDFS-2408
> URL: https://issues.apache.org/jira/browse/HDFS-2408
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
>Affects Versions: 0.20.205.1
>Reporter: stack
>Assignee: stack
>Priority: Blocker
> Attachments: hdfs-2408.txt
>
>
> The below commit broke hdfs-826 for hbase in 205 rc1. It changes the
> accessiblity from public to package private on getNumCurrentReplicas and now
> current shipping hbase's at least cannot get at this method.
> {code}
> Revision 1174483 - (view) (download) (annotate) - [select for diffs]
> Modified Fri Sep 23 01:30:18 2011 UTC (13 days, 4 hours ago) by szetszwo
> File length: 136876 byte(s)
> Diff to previous 1174479 (colored)
> svn merge -c 1171137 from branch-0.20-security for HDFS-2333.
> {code}
> Here is diff between above change and one just previous:
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20-security-205/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java?view=diff&r1=1174479&r2=1174483&diff_format=u
> This is a critical facility for us.
> It seems like making this one method public again is all thats needed. I can
> make a patch like the below:
> diff --git a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> index b9cb053..39955c9 100644
> --- a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> +++ b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> @@ -3569,7 +3569,7 @@ public class DFSClient implements FSConstants,
> java.io.Closeable {
> * block is not yet allocated, then this API will return 0 because there
> are
> * no replicas in the pipeline.
> */
> -int getNumCurrentReplicas() throws IOException {
> +public int getNumCurrentReplicas() throws IOException {
>synchronized(dataQueue) {
> if (nodes == null) {
>return blockReplication;
> Can we get this into RC2?
> Thanks,
> St.Ack
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2408) DFSClient#getNumCurrentReplicas is package private in 205 but public in branch-0.20-append
[
https://issues.apache.org/jira/browse/HDFS-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121760#comment-13121760
]
Suresh Srinivas commented on HDFS-2408:
---
Aaron, this is 205 patch. Are you starting Jenkins pre-commit build?
> DFSClient#getNumCurrentReplicas is package private in 205 but public in
> branch-0.20-append
> --
>
> Key: HDFS-2408
> URL: https://issues.apache.org/jira/browse/HDFS-2408
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
>Affects Versions: 0.20.205.1
>Reporter: stack
>Assignee: stack
> Attachments: hdfs-2408.txt
>
>
> The below commit broke hdfs-826 for hbase in 205 rc1. It changes the
> accessiblity from public to package private on getNumCurrentReplicas and now
> current shipping hbase's at least cannot get at this method.
> {code}
> Revision 1174483 - (view) (download) (annotate) - [select for diffs]
> Modified Fri Sep 23 01:30:18 2011 UTC (13 days, 4 hours ago) by szetszwo
> File length: 136876 byte(s)
> Diff to previous 1174479 (colored)
> svn merge -c 1171137 from branch-0.20-security for HDFS-2333.
> {code}
> Here is diff between above change and one just previous:
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20-security-205/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java?view=diff&r1=1174479&r2=1174483&diff_format=u
> This is a critical facility for us.
> It seems like making this one method public again is all thats needed. I can
> make a patch like the below:
> diff --git a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> index b9cb053..39955c9 100644
> --- a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> +++ b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> @@ -3569,7 +3569,7 @@ public class DFSClient implements FSConstants,
> java.io.Closeable {
> * block is not yet allocated, then this API will return 0 because there
> are
> * no replicas in the pipeline.
> */
> -int getNumCurrentReplicas() throws IOException {
> +public int getNumCurrentReplicas() throws IOException {
>synchronized(dataQueue) {
> if (nodes == null) {
>return blockReplication;
> Can we get this into RC2?
> Thanks,
> St.Ack
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2408) DFSClient#getNumCurrentReplicas is package private in 205 but public in branch-0.20-append
[
https://issues.apache.org/jira/browse/HDFS-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121759#comment-13121759
]
Aaron T. Myers commented on HDFS-2408:
--
D'oh! I forgot that Jenkins isn't going to be able to apply the patch since the
patch is for 0.20-security. :)
Regardless, the patch is trivial. I'll commit it tomorrow unless there are any
objections in the mean time.
> DFSClient#getNumCurrentReplicas is package private in 205 but public in
> branch-0.20-append
> --
>
> Key: HDFS-2408
> URL: https://issues.apache.org/jira/browse/HDFS-2408
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
>Affects Versions: 0.20.205.1
>Reporter: stack
>Assignee: stack
> Attachments: hdfs-2408.txt
>
>
> The below commit broke hdfs-826 for hbase in 205 rc1. It changes the
> accessiblity from public to package private on getNumCurrentReplicas and now
> current shipping hbase's at least cannot get at this method.
> {code}
> Revision 1174483 - (view) (download) (annotate) - [select for diffs]
> Modified Fri Sep 23 01:30:18 2011 UTC (13 days, 4 hours ago) by szetszwo
> File length: 136876 byte(s)
> Diff to previous 1174479 (colored)
> svn merge -c 1171137 from branch-0.20-security for HDFS-2333.
> {code}
> Here is diff between above change and one just previous:
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20-security-205/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java?view=diff&r1=1174479&r2=1174483&diff_format=u
> This is a critical facility for us.
> It seems like making this one method public again is all thats needed. I can
> make a patch like the below:
> diff --git a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> index b9cb053..39955c9 100644
> --- a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> +++ b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> @@ -3569,7 +3569,7 @@ public class DFSClient implements FSConstants,
> java.io.Closeable {
> * block is not yet allocated, then this API will return 0 because there
> are
> * no replicas in the pipeline.
> */
> -int getNumCurrentReplicas() throws IOException {
> +public int getNumCurrentReplicas() throws IOException {
>synchronized(dataQueue) {
> if (nodes == null) {
>return blockReplication;
> Can we get this into RC2?
> Thanks,
> St.Ack
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2408) DFSClient#getNumCurrentReplicas is package private in 205 but public in branch-0.20-append
[
https://issues.apache.org/jira/browse/HDFS-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HDFS-2408:
-
Status: Patch Available (was: Open)
The patch looks good to me. Marking as patch available on Stack's behalf so
Jenkins gives it a spin.
+1 pending clean Jenkins results.
> DFSClient#getNumCurrentReplicas is package private in 205 but public in
> branch-0.20-append
> --
>
> Key: HDFS-2408
> URL: https://issues.apache.org/jira/browse/HDFS-2408
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
>Affects Versions: 0.20.205.1
>Reporter: stack
>Assignee: stack
> Attachments: hdfs-2408.txt
>
>
> The below commit broke hdfs-826 for hbase in 205 rc1. It changes the
> accessiblity from public to package private on getNumCurrentReplicas and now
> current shipping hbase's at least cannot get at this method.
> {code}
> Revision 1174483 - (view) (download) (annotate) - [select for diffs]
> Modified Fri Sep 23 01:30:18 2011 UTC (13 days, 4 hours ago) by szetszwo
> File length: 136876 byte(s)
> Diff to previous 1174479 (colored)
> svn merge -c 1171137 from branch-0.20-security for HDFS-2333.
> {code}
> Here is diff between above change and one just previous:
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20-security-205/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java?view=diff&r1=1174479&r2=1174483&diff_format=u
> This is a critical facility for us.
> It seems like making this one method public again is all thats needed. I can
> make a patch like the below:
> diff --git a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> index b9cb053..39955c9 100644
> --- a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> +++ b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> @@ -3569,7 +3569,7 @@ public class DFSClient implements FSConstants,
> java.io.Closeable {
> * block is not yet allocated, then this API will return 0 because there
> are
> * no replicas in the pipeline.
> */
> -int getNumCurrentReplicas() throws IOException {
> +public int getNumCurrentReplicas() throws IOException {
>synchronized(dataQueue) {
> if (nodes == null) {
>return blockReplication;
> Can we get this into RC2?
> Thanks,
> St.Ack
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2404) webhdfs liststatus json response is not correct
[ https://issues.apache.org/jira/browse/HDFS-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121757#comment-13121757 ] Suresh Srinivas commented on HDFS-2404: --- Nicholas, I have not problem compiling. Which patch does not compile? > webhdfs liststatus json response is not correct > --- > > Key: HDFS-2404 > URL: https://issues.apache.org/jira/browse/HDFS-2404 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Suresh Srinivas > Attachments: HDFS-2404.20.txt, HDFS-2404.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2408) DFSClient#getNumCurrentReplicas is package private in 205 but public in branch-0.20-append
[
https://issues.apache.org/jira/browse/HDFS-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HDFS-2408:
Attachment: hdfs-2408.txt
Change getNumCurrentReplicas back to public again.
> DFSClient#getNumCurrentReplicas is package private in 205 but public in
> branch-0.20-append
> --
>
> Key: HDFS-2408
> URL: https://issues.apache.org/jira/browse/HDFS-2408
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs client
>Affects Versions: 0.20.205.1
>Reporter: stack
>Assignee: stack
> Attachments: hdfs-2408.txt
>
>
> The below commit broke hdfs-826 for hbase in 205 rc1. It changes the
> accessiblity from public to package private on getNumCurrentReplicas and now
> current shipping hbase's at least cannot get at this method.
> {code}
> Revision 1174483 - (view) (download) (annotate) - [select for diffs]
> Modified Fri Sep 23 01:30:18 2011 UTC (13 days, 4 hours ago) by szetszwo
> File length: 136876 byte(s)
> Diff to previous 1174479 (colored)
> svn merge -c 1171137 from branch-0.20-security for HDFS-2333.
> {code}
> Here is diff between above change and one just previous:
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20-security-205/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java?view=diff&r1=1174479&r2=1174483&diff_format=u
> This is a critical facility for us.
> It seems like making this one method public again is all thats needed. I can
> make a patch like the below:
> diff --git a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> index b9cb053..39955c9 100644
> --- a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> +++ b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
> @@ -3569,7 +3569,7 @@ public class DFSClient implements FSConstants,
> java.io.Closeable {
> * block is not yet allocated, then this API will return 0 because there
> are
> * no replicas in the pipeline.
> */
> -int getNumCurrentReplicas() throws IOException {
> +public int getNumCurrentReplicas() throws IOException {
>synchronized(dataQueue) {
> if (nodes == null) {
>return blockReplication;
> Can we get this into RC2?
> Thanks,
> St.Ack
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2408) DFSClient#getNumCurrentReplicas is package private in 205 but public in branch-0.20-append
DFSClient#getNumCurrentReplicas is package private in 205 but public in
branch-0.20-append
--
Key: HDFS-2408
URL: https://issues.apache.org/jira/browse/HDFS-2408
Project: Hadoop HDFS
Issue Type: Bug
Components: hdfs client
Affects Versions: 0.20.205.1
Reporter: stack
Assignee: stack
The below commit broke hdfs-826 for hbase in 205 rc1. It changes the
accessiblity from public to package private on getNumCurrentReplicas and now
current shipping hbase's at least cannot get at this method.
{code}
Revision 1174483 - (view) (download) (annotate) - [select for diffs]
Modified Fri Sep 23 01:30:18 2011 UTC (13 days, 4 hours ago) by szetszwo
File length: 136876 byte(s)
Diff to previous 1174479 (colored)
svn merge -c 1171137 from branch-0.20-security for HDFS-2333.
{code}
Here is diff between above change and one just previous:
http://svn.apache.org/viewvc/hadoop/common/branches/branch-0.20-security-205/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java?view=diff&r1=1174479&r2=1174483&diff_format=u
This is a critical facility for us.
It seems like making this one method public again is all thats needed. I can
make a patch like the below:
diff --git a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
index b9cb053..39955c9 100644
--- a/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
+++ b/src/hdfs/org/apache/hadoop/hdfs/DFSClient.java
@@ -3569,7 +3569,7 @@ public class DFSClient implements FSConstants,
java.io.Closeable {
* block is not yet allocated, then this API will return 0 because there
are
* no replicas in the pipeline.
*/
-int getNumCurrentReplicas() throws IOException {
+public int getNumCurrentReplicas() throws IOException {
synchronized(dataQueue) {
if (nodes == null) {
return blockReplication;
Can we get this into RC2?
Thanks,
St.Ack
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[ https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-1973: - Attachment: HDFS-1973-HDFS-1623.patch Trivial change from the last patch to remove the findbugs warning. > HA: HDFS clients must handle namenode failover and switch over to the new > active namenode. > -- > > Key: HDFS-1973 > URL: https://issues.apache.org/jira/browse/HDFS-1973 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Suresh Srinivas >Assignee: Aaron T. Myers > Attachments: HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, > HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, > HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, hdfs-1973.0.patch > > > During failover, a client must detect the current active namenode failure and > switch over to the new active namenode. The switch over might make use of IP > failover or some thing more elaborate such as zookeeper to discover the new > active. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2407) getServerDefaults and getStats don't check operation category
[
https://issues.apache.org/jira/browse/HDFS-2407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HDFS-2407:
-
Attachment: HDFS-2407-HDFS-1623.patch
Patch which addresses the issue.
> getServerDefaults and getStats don't check operation category
> -
>
> Key: HDFS-2407
> URL: https://issues.apache.org/jira/browse/HDFS-2407
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
>Affects Versions: HA branch (HDFS-1623)
>Reporter: Aaron T. Myers
>Assignee: Aaron T. Myers
> Fix For: HA branch (HDFS-1623)
>
> Attachments: HDFS-2407-HDFS-1623.patch
>
>
> They should both check for {{OperationCategory.READ}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2407) getServerDefaults and getStats don't check operation category
getServerDefaults and getStats don't check operation category
-
Key: HDFS-2407
URL: https://issues.apache.org/jira/browse/HDFS-2407
Project: Hadoop HDFS
Issue Type: Sub-task
Components: name-node
Affects Versions: HA branch (HDFS-1623)
Reporter: Aaron T. Myers
Assignee: Aaron T. Myers
Fix For: HA branch (HDFS-1623)
They should both check for {{OperationCategory.READ}}.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[ https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121729#comment-13121729 ] Hadoop QA commented on HDFS-1973: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12497935/HDFS-1973-HDFS-1623.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1340//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1340//artifact/trunk/hadoop-hdfs-project/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1340//console This message is automatically generated. > HA: HDFS clients must handle namenode failover and switch over to the new > active namenode. > -- > > Key: HDFS-1973 > URL: https://issues.apache.org/jira/browse/HDFS-1973 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Suresh Srinivas >Assignee: Aaron T. Myers > Attachments: HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, > HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, > HDFS-1973-HDFS-1623.patch, hdfs-1973.0.patch > > > During failover, a client must detect the current active namenode failure and > switch over to the new active namenode. The switch over might make use of IP > failover or some thing more elaborate such as zookeeper to discover the new > active. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2393) Mark appropriate methods of ClientProtocol with the idempotent annotation
[
https://issues.apache.org/jira/browse/HDFS-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121717#comment-13121717
]
Aaron T. Myers commented on HDFS-2393:
--
I've been working on categorizing all of the 45 operations currently defined in
{{ClientProtocol}}. This is the breakdown of what I've come up with:
Read ops, therefore idempotent:
* getBlockLocations
* getPreferredBlockSize
* getListing
* getFileInfo
* getFileLinkInfo
* getStats
* getDatanodeReport
* listCorruptFileBlocks
* getContentSummary
* getLinkTarget
* getServerDefaults
Write ops that I believe should be marked idempotent:
* setReplication
* setPermission
* setOwner
* reportBadBlocks
* mkdirs (in fact, it looks like this op can never return false. It either
returns true or throws an exception.)
* renewLease
* setQuota
* fsync
* setTimes
* getDelegationToken
* renewDelegationToken
Write ops I'm unsure about:
* getAdditionalDatanode (I'm not sure this one should even be marked
OperationCategory.WRITE - it only gets the readLock)
* recoverLease
* updateBlockForPipeline
Write ops that I believe should *not* be idempotent:
* create
* append
* abandonBlock
* addBlock
* complete
* rename
* concat
* rename2
* delete
* createSymLink
* updatePipeline
* cancelDelegationToken
Ops which are more administrative, and arguably shouldn't be in ClientProtocol:
* setSafeMode
* saveNamespace
* restoreFailedStorage
* refreshNodes
* finalizeUpgrade
* distributedUpgradeProgress
* metaSave
* setBalancerBandwidth
Comments are most welcome. In particular, I'm interested in:
# Does anyone have any insight about the three operations I'm not sure about?
We could play it safe at first, and not mark them idempotent. Doing so will
result in a few more client failures during an NN fail over, but will not
result in correctness issues.
# What to do about the 8 operations which seem administrative in nature?
# Did I misidentify any of the write operations which I believe to be
idempotent?
Once we hash these things out a little bit, I'll go ahead and make a patch for
it.
> Mark appropriate methods of ClientProtocol with the idempotent annotation
> -
>
> Key: HDFS-2393
> URL: https://issues.apache.org/jira/browse/HDFS-2393
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs client
>Affects Versions: HA branch (HDFS-1623)
>Reporter: Aaron T. Myers
>Assignee: Aaron T. Myers
> Fix For: HA branch (HDFS-1623)
>
>
> HDFS-1973 will make the {{DFSClient}} take advantage of the annotation. This
> JIRA is to identify which methods can be marked as idempotent.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[ https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121690#comment-13121690 ] Hadoop QA commented on HDFS-1973: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12497942/HDFS-1973-HDFS-1623.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1341//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1341//artifact/trunk/hadoop-hdfs-project/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1341//console This message is automatically generated. > HA: HDFS clients must handle namenode failover and switch over to the new > active namenode. > -- > > Key: HDFS-1973 > URL: https://issues.apache.org/jira/browse/HDFS-1973 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Suresh Srinivas >Assignee: Aaron T. Myers > Attachments: HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, > HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, > HDFS-1973-HDFS-1623.patch, hdfs-1973.0.patch > > > During failover, a client must detect the current active namenode failure and > switch over to the new active namenode. The switch over might make use of IP > failover or some thing more elaborate such as zookeeper to discover the new > active. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[
https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HDFS-1973:
-
Attachment: HDFS-1973-HDFS-1623.patch
Here's an updated patch which reuses {{AppendTestUtil.(write|check)}}. Note
that I also had to add the {{@Idempotent}} annotation to
{{ClientProtocol.getFileInfo}} since {{AppendTestUtil.check}} calls
{{FileSystem.getFileStatus}}.
> HA: HDFS clients must handle namenode failover and switch over to the new
> active namenode.
> --
>
> Key: HDFS-1973
> URL: https://issues.apache.org/jira/browse/HDFS-1973
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Suresh Srinivas
>Assignee: Aaron T. Myers
> Attachments: HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch,
> HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch,
> HDFS-1973-HDFS-1623.patch, hdfs-1973.0.patch
>
>
> During failover, a client must detect the current active namenode failure and
> switch over to the new active namenode. The switch over might make use of IP
> failover or some thing more elaborate such as zookeeper to discover the new
> active.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2385) Support delegation token renewal in webhdfs
[ https://issues.apache.org/jira/browse/HDFS-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121679#comment-13121679 ] Jitendra Nath Pandey commented on HDFS-2385: h2385_20111004b_amendment2_0.20s.patch looks ok to me. +1 > Support delegation token renewal in webhdfs > --- > > Key: HDFS-2385 > URL: https://issues.apache.org/jira/browse/HDFS-2385 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Tsz Wo (Nicholas), SZE > Fix For: 0.20.205.0, 0.20.206.0 > > Attachments: h2358_20111001_0.20s.patch, h2385_20111004_0.20s.patch, > h2385_20111004b_0.20s.patch, h2385_20111004b_amendment2_0.20s.patch, > h2385_20111004b_amendment_0.20s.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2181) Seperate HDFS wire protocol data types
[ https://issues.apache.org/jira/browse/HDFS-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121673#comment-13121673 ] Sanjay Radia commented on HDFS-2181: There are two ways to support compatibility: (see the doc i posted in HADOOP-7347) # Maintain compatibility by simply adding new methods and new types (ie no changes to signatures of existing methods) # Have the server support multiple protocols and the client picking the right one (we need to figure out a way for the client to pick the right protocol based on what the server supports - i have ideas on this but have not supplied a patch so far.) I am proposing that we use #1 (within and across releases) till we move to PB or Avro. Say we move to PB in release 25 (hypothetically) then release 24 can continue to use R23. When we move to PB or Avro we use approach 2 since customer will expect compatibility across releases. BTW approach 2 can also be used to support compatibility between 20 and 23. I assume that I have your +1 having addressed your feedback and will commit this patch later tonight. > Seperate HDFS wire protocol data types > -- > > Key: HDFS-2181 > URL: https://issues.apache.org/jira/browse/HDFS-2181 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Sanjay Radia >Assignee: Sanjay Radia > Attachments: separateDataType1.patch, separateDataType10.patch, > separateDataType11.patch, separateDataType12.patch, separateDataType13.patch, > separateDataType14.patch, separateDataType16.patch, separateDataType19.patch, > separateDataType2.patch, separateDataType6.patch, separateDataType7.patch, > separateDataType8.patch, separateDataType9.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[
https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121662#comment-13121662
]
Todd Lipcon commented on HDFS-1973:
---
One small thing I didn't notice before (sorry):
- in the test cases, you have new methods verifyFileContents and
writeContentsToFile. These are very similar to AppendTestUtil.write and
AppndTestUtil.check. Can you use those instead? If not, you should fix the call
to {{in.read}} to use {{IOUtils.readFully}} since {{read}} isn't guaranteed to
read the whole length.
> HA: HDFS clients must handle namenode failover and switch over to the new
> active namenode.
> --
>
> Key: HDFS-1973
> URL: https://issues.apache.org/jira/browse/HDFS-1973
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Suresh Srinivas
>Assignee: Aaron T. Myers
> Attachments: HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch,
> HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, hdfs-1973.0.patch
>
>
> During failover, a client must detect the current active namenode failure and
> switch over to the new active namenode. The switch over might make use of IP
> failover or some thing more elaborate such as zookeeper to discover the new
> active.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2404) webhdfs liststatus json response is not correct
[ https://issues.apache.org/jira/browse/HDFS-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121661#comment-13121661 ] Tsz Wo (Nicholas), SZE commented on HDFS-2404: -- Hi Suresh, please also update WebHdfsFileSystem. Otherwise, it cannot be compiled. > webhdfs liststatus json response is not correct > --- > > Key: HDFS-2404 > URL: https://issues.apache.org/jira/browse/HDFS-2404 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Suresh Srinivas > Attachments: HDFS-2404.20.txt, HDFS-2404.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1973) HA: HDFS clients must handle namenode failover and switch over to the new active namenode.
[ https://issues.apache.org/jira/browse/HDFS-1973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aaron T. Myers updated HDFS-1973: - Attachment: HDFS-1973-HDFS-1623.patch Here's an updated patch which takes advantage of HADOOP-7717 by no longer needing to check for concurrent fail-overs in the event of concurrent method invocation failures. > HA: HDFS clients must handle namenode failover and switch over to the new > active namenode. > -- > > Key: HDFS-1973 > URL: https://issues.apache.org/jira/browse/HDFS-1973 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Suresh Srinivas >Assignee: Aaron T. Myers > Attachments: HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, > HDFS-1973-HDFS-1623.patch, HDFS-1973-HDFS-1623.patch, hdfs-1973.0.patch > > > During failover, a client must detect the current active namenode failure and > switch over to the new active namenode. The switch over might make use of IP > failover or some thing more elaborate such as zookeeper to discover the new > active. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2385) Support delegation token renewal in webhdfs
[ https://issues.apache.org/jira/browse/HDFS-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2385: - Attachment: h2385_20111004b_amendment2_0.20s.patch > ... Does it make sense to move createUri to SecurityUtil and use that to > create the uri in WebhdfsFileSystem as well? I moved it to DFSUtil since it is really security related and it is only used by HftpFileSystem and WebHdfsFileSystem. Here is the new patch. h2385_20111004b_amendment2_0.20s.patch > Support delegation token renewal in webhdfs > --- > > Key: HDFS-2385 > URL: https://issues.apache.org/jira/browse/HDFS-2385 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Tsz Wo (Nicholas), SZE > Fix For: 0.20.205.0, 0.20.206.0 > > Attachments: h2358_20111001_0.20s.patch, h2385_20111004_0.20s.patch, > h2385_20111004b_0.20s.patch, h2385_20111004b_amendment2_0.20s.patch, > h2385_20111004b_amendment_0.20s.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-1935) Build should not redownload ivy on every invocation
[ https://issues.apache.org/jira/browse/HDFS-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-1935: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Build should not redownload ivy on every invocation > --- > > Key: HDFS-1935 > URL: https://issues.apache.org/jira/browse/HDFS-1935 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 0.22.0 >Reporter: Todd Lipcon >Priority: Trivial > Labels: newbie > Fix For: 0.22.0 > > Attachments: diff, hdfs-1935.patch, hdfs-1935.txt > > > Currently we re-download ivy every time we build. If the jar already exists, > we should skip this. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2404) webhdfs liststatus json response is not correct
[ https://issues.apache.org/jira/browse/HDFS-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2404: -- Attachment: HDFS-2404.20.txt 20 version of the patch. > webhdfs liststatus json response is not correct > --- > > Key: HDFS-2404 > URL: https://issues.apache.org/jira/browse/HDFS-2404 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Suresh Srinivas > Attachments: HDFS-2404.20.txt, HDFS-2404.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2385) Support delegation token renewal in webhdfs
[ https://issues.apache.org/jira/browse/HDFS-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121639#comment-13121639 ] Jitendra Nath Pandey commented on HDFS-2385: Comment for h2385_20111004b_amendment_0.20s.patch: HftpFileSystem has a similar code. It uses a private static createUri to create the uri. createUri is a pretty generic method. Does it make sense to move createUri to SecurityUtil and use that to create the uri in WebhdfsFileSystem as well? > Support delegation token renewal in webhdfs > --- > > Key: HDFS-2385 > URL: https://issues.apache.org/jira/browse/HDFS-2385 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Tsz Wo (Nicholas), SZE > Fix For: 0.20.205.0, 0.20.206.0 > > Attachments: h2358_20111001_0.20s.patch, h2385_20111004_0.20s.patch, > h2385_20111004b_0.20s.patch, h2385_20111004b_amendment_0.20s.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2385) Support delegation token renewal in webhdfs
[ https://issues.apache.org/jira/browse/HDFS-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2385: - Attachment: h2385_20111004b_amendment_0.20s.patch On a second thought, let me post an amendment here. h2385_20111004b_amendment_0.20s.patch: use token service address instead of conf. > Support delegation token renewal in webhdfs > --- > > Key: HDFS-2385 > URL: https://issues.apache.org/jira/browse/HDFS-2385 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Tsz Wo (Nicholas), SZE > Fix For: 0.20.205.0, 0.20.206.0 > > Attachments: h2358_20111001_0.20s.patch, h2385_20111004_0.20s.patch, > h2385_20111004b_0.20s.patch, h2385_20111004b_amendment_0.20s.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2385) Support delegation token renewal in webhdfs
[ https://issues.apache.org/jira/browse/HDFS-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121612#comment-13121612 ] Tsz Wo (Nicholas), SZE commented on HDFS-2385: -- Let me fix it in HDFS-2403. > Support delegation token renewal in webhdfs > --- > > Key: HDFS-2385 > URL: https://issues.apache.org/jira/browse/HDFS-2385 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Tsz Wo (Nicholas), SZE > Fix For: 0.20.205.0, 0.20.206.0 > > Attachments: h2358_20111001_0.20s.patch, h2385_20111004_0.20s.patch, > h2385_20111004b_0.20s.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2385) Support delegation token renewal in webhdfs
[
https://issues.apache.org/jira/browse/HDFS-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121611#comment-13121611
]
Tsz Wo (Nicholas), SZE commented on HDFS-2385:
--
@Daryn, you are right. It should use {{InetSocketAddress serviceAddr =
SecurityUtil.getTokenServiceAddr(token);}}
> Support delegation token renewal in webhdfs
> ---
>
> Key: HDFS-2385
> URL: https://issues.apache.org/jira/browse/HDFS-2385
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.20.205.0, 0.20.206.0
>
> Attachments: h2358_20111001_0.20s.patch, h2385_20111004_0.20s.patch,
> h2385_20111004b_0.20s.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2181) Seperate HDFS wire protocol data types
[ https://issues.apache.org/jira/browse/HDFS-2181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121603#comment-13121603 ] Todd Lipcon commented on HDFS-2181: --- Hi Sanjay. One question: bq. As we add new types in future releases, the protocol version will NOT change (recall the rule - add new types and new methods) Is that still true, after this feature? I thought that, with this new protocol selection feature, we would simply duplicate the "protocolR23" package to "protocolR24" when R24 was released, etc? So we only had to worry about method/field compatibility within a release, but between releases, we could change the wire format as we liked? > Seperate HDFS wire protocol data types > -- > > Key: HDFS-2181 > URL: https://issues.apache.org/jira/browse/HDFS-2181 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Sanjay Radia >Assignee: Sanjay Radia > Attachments: separateDataType1.patch, separateDataType10.patch, > separateDataType11.patch, separateDataType12.patch, separateDataType13.patch, > separateDataType14.patch, separateDataType16.patch, separateDataType19.patch, > separateDataType2.patch, separateDataType6.patch, separateDataType7.patch, > separateDataType8.patch, separateDataType9.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-2395) webhdfs api's should return a root element in the json response
[ https://issues.apache.org/jira/browse/HDFS-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley resolved HDFS-2395. -- Resolution: Fixed Fix Version/s: (was: 0.20.206.0) > webhdfs api's should return a root element in the json response > --- > > Key: HDFS-2395 > URL: https://issues.apache.org/jira/browse/HDFS-2395 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Tsz Wo (Nicholas), SZE >Priority: Critical > Fix For: 0.20.205.0, 0.24.0 > > Attachments: h2395_20111003.patch, h2395_20111004.patch, > h2395_20111004_0.20s.patch, h2395_20111004b.patch, h2395_20111004c.patch, > h2395_20111004c_0.20s.patch, h2395_20111004d.patch, > h2395_20111004d_0.20s.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HDFS-2395) webhdfs api's should return a root element in the json response
[ https://issues.apache.org/jira/browse/HDFS-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley reopened HDFS-2395: -- Bug was erroneously marked "Duplicate" instead of "Fixed". Fixing... > webhdfs api's should return a root element in the json response > --- > > Key: HDFS-2395 > URL: https://issues.apache.org/jira/browse/HDFS-2395 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Tsz Wo (Nicholas), SZE >Priority: Critical > Fix For: 0.20.205.0, 0.24.0 > > Attachments: h2395_20111003.patch, h2395_20111004.patch, > h2395_20111004_0.20s.patch, h2395_20111004b.patch, h2395_20111004c.patch, > h2395_20111004c_0.20s.patch, h2395_20111004d.patch, > h2395_20111004d_0.20s.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2392) Dist with hftp is failing again
[ https://issues.apache.org/jira/browse/HDFS-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HDFS-2392: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) > Dist with hftp is failing again > --- > > Key: HDFS-2392 > URL: https://issues.apache.org/jira/browse/HDFS-2392 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20.205.0 >Reporter: Rajit Saha >Assignee: Daryn Sharp >Priority: Critical > Fix For: 0.20.205.0 > > Attachments: HDFS-2392.patch > > > $ hadoop distcp hftp://:50070/user/hadoopqa/input1/part-0 > /user/hadoopqa/out3 > 11/09/30 18:57:59 INFO tools.DistCp: srcPaths=[hftp:// Hostname>:50070/user/hadoopqa/input1/part-0] > 11/09/30 18:57:59 INFO tools.DistCp: destPath=/user/hadoopqa/out3 > 11/09/30 18:58:00 INFO security.TokenCache: Got dt for > hftp://:50070/user/hadoopqa/input1/part-0;uri= IP>:50470;t.service=:50470 > 11/09/30 18:58:00 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 24 > for hadoopqa on :8020 > 11/09/30 18:58:00 INFO security.TokenCache: Got dt for > /user/hadoopqa/out3;uri=:8020;t.service=:8020 > 11/09/30 18:58:00 INFO tools.DistCp: /user/hadoopqa/out3 does not exist. > 11/09/30 18:58:00 INFO tools.DistCp: sourcePathsCount=1 > 11/09/30 18:58:00 INFO tools.DistCp: filesToCopyCount=1 > 11/09/30 18:58:00 INFO tools.DistCp: bytesToCopyCount=1.0g > 11/09/30 18:58:01 INFO mapred.JobClient: Running job: job_201109300819_0007 > 11/09/30 18:58:02 INFO mapred.JobClient: map 0% reduce 0% > 11/09/30 18:58:25 INFO mapred.JobClient: Task Id : > attempt_201109300819_0007_m_00_0, Status : FAILED > java.io.IOException: Copied: 0 Skipped: 0 Failed: 1 > at > org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:582) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > 11/09/30 18:58:41 INFO mapred.JobClient: Task Id : > attempt_201109300819_0007_m_00_1, Status : FAILED > java.io.IOException: Copied: 0 Skipped: 0 Failed: 1 > at > org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:582) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > 11/09/30 18:58:56 INFO mapred.JobClient: Task Id : > attempt_201109300819_0007_m_00_2, Status : FAILED > java.io.IOException: Copied: 0 Skipped: 0 Failed: 1 > at > org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:582) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > 11/09/30 18:59:14 INFO mapred.JobClient: Job complete: job_201109300819_0007 > 11/09/30 18:59:14 INFO mapred.JobClient: Counters: 6 > 11/09/30 18:59:14 INFO mapred.JobClient: Job Counters > 11/09/30 18:59:14 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=62380 > 11/09/30 18:59:14 INFO mapred.JobClient: Total time spent by all reduces > waiting after reserving slots (ms)=0 > 11/09/30 18:59:14 INFO mapred.JobClient: Total time spent by all maps > waiting after reserving slots (ms)=0 > 11/09/30 18:59:14 INFO mapred.JobClient: Launched map tasks=4 > 11/09/30 18:59:14 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0 > 11/09/30 18:59:14 INFO mapred.JobClient: Failed map tasks=1 > 11/09/30 18:59:
[jira] [Updated] (HDFS-2404) webhdfs liststatus json response is not correct
[ https://issues.apache.org/jira/browse/HDFS-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suresh Srinivas updated HDFS-2404: -- Attachment: HDFS-2404.txt Attachment that removes the HDFSFileStatus for array elements. > webhdfs liststatus json response is not correct > --- > > Key: HDFS-2404 > URL: https://issues.apache.org/jira/browse/HDFS-2404 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Suresh Srinivas > Attachments: HDFS-2404.txt > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2358) NPE when the default filesystem's uri has no authority
[ https://issues.apache.org/jira/browse/HDFS-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated HDFS-2358: - Resolution: Fixed Fix Version/s: 0.20.205.0 Release Note: Give meaningful error message instead of NPE. Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) +1 for code review. Committed to both 0.20-sec and 0.20.205.0. Thanks, Daryn! > NPE when the default filesystem's uri has no authority > -- > > Key: HDFS-2358 > URL: https://issues.apache.org/jira/browse/HDFS-2358 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20.204.0 >Reporter: Rajit Saha >Assignee: Daryn Sharp > Fix For: 0.20.205.0 > > Attachments: HDFS-2358.patch > > > in core-site.xml if default filesystem's uri is misconfigured , NN does not > come up and in NN shows NPE > java.lang.NullPointerException > at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:142) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:195) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:225) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:259) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:458) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1226) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1235) > It is better to give meaningful error messege than NPE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2406) webhdfs shows wrong message when content is deleted
[ https://issues.apache.org/jira/browse/HDFS-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121548#comment-13121548 ] Arpit Gupta commented on HDFS-2406: --- user error :) > webhdfs shows wrong message when content is deleted > --- > > Key: HDFS-2406 > URL: https://issues.apache.org/jira/browse/HDFS-2406 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2406) webhdfs shows wrong message when content is deleted
webhdfs shows wrong message when content is deleted --- Key: HDFS-2406 URL: https://issues.apache.org/jira/browse/HDFS-2406 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.20.205.0 Reporter: Arpit Gupta -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-2406) webhdfs shows wrong message when content is deleted
[ https://issues.apache.org/jira/browse/HDFS-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Gupta resolved HDFS-2406. --- Resolution: Invalid > webhdfs shows wrong message when content is deleted > --- > > Key: HDFS-2406 > URL: https://issues.apache.org/jira/browse/HDFS-2406 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2358) NPE when the default filesystem's uri has no authority
[ https://issues.apache.org/jira/browse/HDFS-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121459#comment-13121459 ] Daryn Sharp commented on HDFS-2358: --- [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] > NPE when the default filesystem's uri has no authority > -- > > Key: HDFS-2358 > URL: https://issues.apache.org/jira/browse/HDFS-2358 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20.204.0 >Reporter: Rajit Saha >Assignee: Daryn Sharp > Attachments: HDFS-2358.patch > > > in core-site.xml if default filesystem's uri is misconfigured , NN does not > come up and in NN shows NPE > java.lang.NullPointerException > at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:142) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:195) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:225) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:259) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:458) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1226) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1235) > It is better to give meaningful error messege than NPE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2405) hadoop dfs command with webhdfs fails on secure hadoop
[ https://issues.apache.org/jira/browse/HDFS-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Gupta updated HDFS-2405: -- Summary: hadoop dfs command with webhdfs fails on secure hadoop (was: hadoop dfs command with webhdfs fails) Command ran bash-3.2$ hadoop dfs -ls webhdfs://host:50070/ ls: Exception obtaining parameters > hadoop dfs command with webhdfs fails on secure hadoop > -- > > Key: HDFS-2405 > URL: https://issues.apache.org/jira/browse/HDFS-2405 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Jitendra Nath Pandey >Priority: Critical > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2405) hadoop dfs command with webhdfs fails on secure hadoop
[ https://issues.apache.org/jira/browse/HDFS-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Gupta updated HDFS-2405: -- Command ran bash-3.2$ hadoop dfs -ls webhdfs://host:50070/ ls: Exception obtaining parameters > hadoop dfs command with webhdfs fails on secure hadoop > -- > > Key: HDFS-2405 > URL: https://issues.apache.org/jira/browse/HDFS-2405 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Jitendra Nath Pandey >Priority: Critical > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2405) hadoop dfs command with webhdfs fails
hadoop dfs command with webhdfs fails - Key: HDFS-2405 URL: https://issues.apache.org/jira/browse/HDFS-2405 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.20.205.0 Reporter: Arpit Gupta Assignee: Jitendra Nath Pandey Priority: Critical -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2358) NPE when the default filesystem's uri has no authority
[ https://issues.apache.org/jira/browse/HDFS-2358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121423#comment-13121423 ] Matt Foley commented on HDFS-2358: -- Daryn, please post your local test-patch results as a comment. Thanks. > NPE when the default filesystem's uri has no authority > -- > > Key: HDFS-2358 > URL: https://issues.apache.org/jira/browse/HDFS-2358 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20.204.0 >Reporter: Rajit Saha >Assignee: Daryn Sharp > Attachments: HDFS-2358.patch > > > in core-site.xml if default filesystem's uri is misconfigured , NN does not > come up and in NN shows NPE > java.lang.NullPointerException > at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:142) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:195) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:225) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:259) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:458) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1226) > at > org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1235) > It is better to give meaningful error messege than NPE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2404) webhdfs liststatus json response is not correct
[
https://issues.apache.org/jira/browse/HDFS-2404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121378#comment-13121378
]
Arpit Gupta commented on HDFS-2404:
---
curl -i "http://HOST:50070/webhdfs/?op=LISTSTATUS";
returns
{code}
{"HdfsFileStatus[]":[
{"HdfsFileStatus":{"accessTime":0,"blockSize":0,"group":"hadoop","isDir":true,"len":0,"localName":"hbase","modificationTime":1317832608532,"owner":"hrt_hdfs","permission":"700","replication":0}},
{"HdfsFileStatus":{"accessTime":0,"blockSize":0,"group":"hadoop","isDir":true,"len":0,"localName":"mapred","modificationTime":1317832616195,"owner":"mapred","permission":"755","replication":0}},
{code}
This when converted to xml is something like
{code}
0
0
hadoop
true
0
hbase
1317832608532
hrt_hdfs
700
0
0
0
hadoop
true
0
mapred
1317832616195
mapred
755
0
{code}
We should instead return something like
{code}
{"HdfsFileStatus":[
{"accessTime":0,"blockSize":0,"group":"hadoop","isDir":true,"len":0,"localName":"hbase","modificationTime":1317832608532,"owner":"hrt_hdfs","permission":"700","replication":0},
{"accessTime":0,"blockSize":0,"group":"hadoop","isDir":true,"len":0,"localName":"mapred","modificationTime":1317832616195,"owner":"mapred","permission":"755","replication":0},
{"accessTime":0,"blockSize":0,"group":"supergroup","isDir":true,"len":0,"localName":"tmp","modificationTime":1317832565771,"owner":"hdfs","permission":"777","replication":0},
{"accessTime":0,"blockSize":0,"group":"supergroup","isDir":true,"len":0,"localName":"user","modificationTime":1317832647984,"owner":"hdfs","permission":"755","replication":0}]}
{code}
which maps to xml like
{code}
0
0
hadoop
true
0
hbase
1317832608532
hrt_hdfs
700
0
0
0
hadoop
true
0
mapred
1317832616195
mapred
755
0
{code}
> webhdfs liststatus json response is not correct
> ---
>
> Key: HDFS-2404
> URL: https://issues.apache.org/jira/browse/HDFS-2404
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 0.20.205.0
>Reporter: Arpit Gupta
>Assignee: Suresh Srinivas
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2404) webhdfs liststatus json response is not correct
webhdfs liststatus json response is not correct --- Key: HDFS-2404 URL: https://issues.apache.org/jira/browse/HDFS-2404 Project: Hadoop HDFS Issue Type: Bug Affects Versions: 0.20.205.0 Reporter: Arpit Gupta Assignee: Suresh Srinivas -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2178) Contributing Hoop to HDFS, replacement for HDFS proxy with read/write capabilities
[ https://issues.apache.org/jira/browse/HDFS-2178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121354#comment-13121354 ] Sanjay Radia commented on HDFS-2178: Som thoughts on webhdfs & hoop: * webhdfs is a significant rewrite of hoop - indeed as much as hoop was a rewrite of hdfsproxy. This was due to a number of reasons. ** Fundamental differences between the proxy (hoop) and a built in file system. Examples are support for delegation tokens, support for trusted proxies (like oozie) and the need for redirecting to a DN, These three do not make sense for a proxy (Hoop or hdfsproxy). ** Code cleanup and integration into servlets of NN and DN. These changes can apply to hoop and we can share the code. ** Parameter and return type clean up (eg. the root element issue) - these also apply to hoop. As we move forward we need to consider two things. # What subset of webhdfs API makes sense for a proxy? # Alternatively shall we think of the proxy as a *pure* proxy - that is should it merely redirect requests to the webhdfs API. Indeed it can then be a proxy for *any* built-in rest API of the entire Hadoop system (not just HDFS). We can add features like user name and authentication mapping to the proxy. > Contributing Hoop to HDFS, replacement for HDFS proxy with read/write > capabilities > -- > > Key: HDFS-2178 > URL: https://issues.apache.org/jira/browse/HDFS-2178 > Project: Hadoop HDFS > Issue Type: Improvement >Affects Versions: 0.23.0 >Reporter: Alejandro Abdelnur >Assignee: Alejandro Abdelnur > Fix For: 0.23.0 > > Attachments: HdfsHttpAPI.pdf > > > We'd like to contribute Hoop to Hadoop HDFS as a replacement (an improvement) > for HDFS Proxy. > Hoop provides access to all Hadoop Distributed File System (HDFS) operations > (read and write) over HTTP/S. > The Hoop server component is a REST HTTP gateway to HDFS supporting all file > system operations. It can be accessed using standard HTTP tools (i.e. curl > and wget), HTTP libraries from different programing languages (i.e. Perl, > Java Script) as well as using the Hoop client. The Hoop server component is a > standard Java web-application and it has been implemented using Jersey > (JAX-RS). > The Hoop client component is an implementation of Hadoop FileSystem client > that allows using the familiar Hadoop filesystem API to access HDFS data > through a Hoop server. > Repo: https://github.com/cloudera/hoop > Docs: http://cloudera.github.com/hoop > Blog: http://www.cloudera.com/blog/2011/07/hoop-hadoop-hdfs-over-http/ > Hoop is a Maven based project that depends on Hadoop HDFS and Alfredo (for > Kerberos HTTP SPNEGO authentication). > To make the integration easy, HDFS Mavenization (HDFS-2096) would have to be > done first, as well as the Alfredo contribution (HADOOP-7119). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2385) Support delegation token renewal in webhdfs
[
https://issues.apache.org/jira/browse/HDFS-2385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121315#comment-13121315
]
Daryn Sharp commented on HDFS-2385:
---
Sorry I'm late to comment, but I'm confused by the webhdfs renew and cancel.
{code}
final String uri = WebHdfsFileSystem.SCHEME + "://"
+ conf.get("dfs.http.address");
...
return (WebHdfsFileSystem)FileSystem.get(new URI(uri), conf);
{code}
Does this work if webhdfs isn't being used to contact the default namenode? It
would appear that tokens will fail to renew during a distcp from a remote
namenode because the tokens are renewed against the local namenode. Why isn't
the service in the token being used?
> Support delegation token renewal in webhdfs
> ---
>
> Key: HDFS-2385
> URL: https://issues.apache.org/jira/browse/HDFS-2385
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Tsz Wo (Nicholas), SZE
>Assignee: Tsz Wo (Nicholas), SZE
> Fix For: 0.20.205.0, 0.20.206.0
>
> Attachments: h2358_20111001_0.20s.patch, h2385_20111004_0.20s.patch,
> h2385_20111004b_0.20s.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2231) Configuration changes for HA namenode
[
https://issues.apache.org/jira/browse/HDFS-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121191#comment-13121191
]
Aaron T. Myers commented on HDFS-2231:
--
Hey Suresh, patch looks pretty good to me. Two small comments:
# Why remove {{DFSUtil.getNameServiceIdKey}}, when you could have just
implemented this method in terms of {{DFSUtil.addSuffixes}}? Doing so will save
you having to change all the references to {{DFSUtil.getNameServiceIdKey}} that
are presently littered throughout the code.
# Should probably remove this: "{{System.out.println("Suresh returning key " +
key + " " + keySuffix);}}", as not all users will know who "Suresh" is. :)
> Configuration changes for HA namenode
> -
>
> Key: HDFS-2231
> URL: https://issues.apache.org/jira/browse/HDFS-2231
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Suresh Srinivas
>Assignee: Suresh Srinivas
> Fix For: HA branch (HDFS-1623)
>
> Attachments: HDFS-2231.txt
>
>
> This jira tracks the changes required for configuring HA setup for namenodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2403) The renewer in NamenodeWebHdfsMethods.generateDelegationToken(..) is not used
[ https://issues.apache.org/jira/browse/HDFS-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121035#comment-13121035 ] Hadoop QA commented on HDFS-2403: - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12497786/h2403_20111005.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1339//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1339//console This message is automatically generated. > The renewer in NamenodeWebHdfsMethods.generateDelegationToken(..) is not used > - > > Key: HDFS-2403 > URL: https://issues.apache.org/jira/browse/HDFS-2403 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Tsz Wo (Nicholas), SZE > Attachments: h2403_20111005.patch, h2403_20111005_0.20.patch > > > Below are some suggestions from Suresh. > # renewer not used in #generateDelegationToken > # put() does not use InputStream in and should not throw URISyntaxException > # post() does not use InputStream in and should not throw URISyntaxException > # get() should not throw URISyntaxException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2392) Dist with hftp is failing again
[ https://issues.apache.org/jira/browse/HDFS-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120963#comment-13120963 ] Daryn Sharp commented on HDFS-2392: --- I'll include it in the trunk merge of HADOOP-7510 since it was an oversight in that jira's changes. > Dist with hftp is failing again > --- > > Key: HDFS-2392 > URL: https://issues.apache.org/jira/browse/HDFS-2392 > Project: Hadoop HDFS > Issue Type: Bug > Components: name-node >Affects Versions: 0.20.205.0 >Reporter: Rajit Saha >Assignee: Daryn Sharp >Priority: Critical > Fix For: 0.20.205.0 > > Attachments: HDFS-2392.patch > > > $ hadoop distcp hftp://:50070/user/hadoopqa/input1/part-0 > /user/hadoopqa/out3 > 11/09/30 18:57:59 INFO tools.DistCp: srcPaths=[hftp:// Hostname>:50070/user/hadoopqa/input1/part-0] > 11/09/30 18:57:59 INFO tools.DistCp: destPath=/user/hadoopqa/out3 > 11/09/30 18:58:00 INFO security.TokenCache: Got dt for > hftp://:50070/user/hadoopqa/input1/part-0;uri= IP>:50470;t.service=:50470 > 11/09/30 18:58:00 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 24 > for hadoopqa on :8020 > 11/09/30 18:58:00 INFO security.TokenCache: Got dt for > /user/hadoopqa/out3;uri=:8020;t.service=:8020 > 11/09/30 18:58:00 INFO tools.DistCp: /user/hadoopqa/out3 does not exist. > 11/09/30 18:58:00 INFO tools.DistCp: sourcePathsCount=1 > 11/09/30 18:58:00 INFO tools.DistCp: filesToCopyCount=1 > 11/09/30 18:58:00 INFO tools.DistCp: bytesToCopyCount=1.0g > 11/09/30 18:58:01 INFO mapred.JobClient: Running job: job_201109300819_0007 > 11/09/30 18:58:02 INFO mapred.JobClient: map 0% reduce 0% > 11/09/30 18:58:25 INFO mapred.JobClient: Task Id : > attempt_201109300819_0007_m_00_0, Status : FAILED > java.io.IOException: Copied: 0 Skipped: 0 Failed: 1 > at > org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:582) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > 11/09/30 18:58:41 INFO mapred.JobClient: Task Id : > attempt_201109300819_0007_m_00_1, Status : FAILED > java.io.IOException: Copied: 0 Skipped: 0 Failed: 1 > at > org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:582) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > 11/09/30 18:58:56 INFO mapred.JobClient: Task Id : > attempt_201109300819_0007_m_00_2, Status : FAILED > java.io.IOException: Copied: 0 Skipped: 0 Failed: 1 > at > org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:582) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > 11/09/30 18:59:14 INFO mapred.JobClient: Job complete: job_201109300819_0007 > 11/09/30 18:59:14 INFO mapred.JobClient: Counters: 6 > 11/09/30 18:59:14 INFO mapred.JobClient: Job Counters > 11/09/30 18:59:14 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=62380 > 11/09/30 18:59:14 INFO mapred.JobClient: Total time spent by all reduces > waiting after reserving slots (ms)=0 > 11/09/30 18:59:14 INFO mapred.JobClient: Total time spent by all maps > waiting after reserving slots (ms)=0 > 11/09/30 18:59:14 INFO mapred.JobClient: Launched map tasks=4 > 11/09/30 18:59:14 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0 > 11/09/30 18
[jira] [Commented] (HDFS-2395) webhdfs api's should return a root element in the json response
[ https://issues.apache.org/jira/browse/HDFS-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120950#comment-13120950 ] Hudson commented on HDFS-2395: -- Integrated in Hadoop-Mapreduce-trunk #851 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/851/]) HDFS-2395. Add a root element in the JSON responses of webhdfs. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179169 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java > webhdfs api's should return a root element in the json response > --- > > Key: HDFS-2395 > URL: https://issues.apache.org/jira/browse/HDFS-2395 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Tsz Wo (Nicholas), SZE >Priority: Critical > Fix For: 0.20.205.0, 0.20.206.0, 0.24.0 > > Attachments: h2395_20111003.patch, h2395_20111004.patch, > h2395_20111004_0.20s.patch, h2395_20111004b.patch, h2395_20111004c.patch, > h2395_20111004c_0.20s.patch, h2395_20111004d.patch, > h2395_20111004d_0.20s.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2395) webhdfs api's should return a root element in the json response
[ https://issues.apache.org/jira/browse/HDFS-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120895#comment-13120895 ] Hudson commented on HDFS-2395: -- Integrated in Hadoop-Mapreduce-trunk-Commit #1036 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1036/]) HDFS-2395. Add a root element in the JSON responses of webhdfs. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179169 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java > webhdfs api's should return a root element in the json response > --- > > Key: HDFS-2395 > URL: https://issues.apache.org/jira/browse/HDFS-2395 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Tsz Wo (Nicholas), SZE >Priority: Critical > Fix For: 0.20.205.0, 0.20.206.0, 0.24.0 > > Attachments: h2395_20111003.patch, h2395_20111004.patch, > h2395_20111004_0.20s.patch, h2395_20111004b.patch, h2395_20111004c.patch, > h2395_20111004c_0.20s.patch, h2395_20111004d.patch, > h2395_20111004d_0.20s.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2395) webhdfs api's should return a root element in the json response
[ https://issues.apache.org/jira/browse/HDFS-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120875#comment-13120875 ] Hudson commented on HDFS-2395: -- Integrated in Hadoop-Hdfs-trunk #821 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/821/]) HDFS-2395. Add a root element in the JSON responses of webhdfs. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179169 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java > webhdfs api's should return a root element in the json response > --- > > Key: HDFS-2395 > URL: https://issues.apache.org/jira/browse/HDFS-2395 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Tsz Wo (Nicholas), SZE >Priority: Critical > Fix For: 0.20.205.0, 0.20.206.0, 0.24.0 > > Attachments: h2395_20111003.patch, h2395_20111004.patch, > h2395_20111004_0.20s.patch, h2395_20111004b.patch, h2395_20111004c.patch, > h2395_20111004c_0.20s.patch, h2395_20111004d.patch, > h2395_20111004d_0.20s.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2403) The renewer in NamenodeWebHdfsMethods.generateDelegationToken(..) is not used
[ https://issues.apache.org/jira/browse/HDFS-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2403: - Attachment: h2403_20111005.patch h2403_20111005_0.20.patch h2403_20111005_0.20.patch h2403_20111005.patch: fixed all 4 items. > The renewer in NamenodeWebHdfsMethods.generateDelegationToken(..) is not used > - > > Key: HDFS-2403 > URL: https://issues.apache.org/jira/browse/HDFS-2403 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Tsz Wo (Nicholas), SZE > Attachments: h2403_20111005.patch, h2403_20111005_0.20.patch > > > Below are some suggestions from Suresh. > # renewer not used in #generateDelegationToken > # put() does not use InputStream in and should not throw URISyntaxException > # post() does not use InputStream in and should not throw URISyntaxException > # get() should not throw URISyntaxException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2403) The renewer in NamenodeWebHdfsMethods.generateDelegationToken(..) is not used
[ https://issues.apache.org/jira/browse/HDFS-2403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2403: - Status: Patch Available (was: Open) > The renewer in NamenodeWebHdfsMethods.generateDelegationToken(..) is not used > - > > Key: HDFS-2403 > URL: https://issues.apache.org/jira/browse/HDFS-2403 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Tsz Wo (Nicholas), SZE >Assignee: Tsz Wo (Nicholas), SZE > Attachments: h2403_20111005.patch, h2403_20111005_0.20.patch > > > Below are some suggestions from Suresh. > # renewer not used in #generateDelegationToken > # put() does not use InputStream in and should not throw URISyntaxException > # post() does not use InputStream in and should not throw URISyntaxException > # get() should not throw URISyntaxException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HDFS-2403) The renewer in NamenodeWebHdfsMethods.generateDelegationToken(..) is not used
The renewer in NamenodeWebHdfsMethods.generateDelegationToken(..) is not used - Key: HDFS-2403 URL: https://issues.apache.org/jira/browse/HDFS-2403 Project: Hadoop HDFS Issue Type: Bug Reporter: Tsz Wo (Nicholas), SZE Assignee: Tsz Wo (Nicholas), SZE Below are some suggestions from Suresh. # renewer not used in #generateDelegationToken # put() does not use InputStream in and should not throw URISyntaxException # post() does not use InputStream in and should not throw URISyntaxException # get() should not throw URISyntaxException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2395) webhdfs api's should return a root element in the json response
[ https://issues.apache.org/jira/browse/HDFS-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120858#comment-13120858 ] Hadoop QA commented on HDFS-2395: - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12497762/h2395_20111004d.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1338//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1338//console This message is automatically generated. > webhdfs api's should return a root element in the json response > --- > > Key: HDFS-2395 > URL: https://issues.apache.org/jira/browse/HDFS-2395 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Tsz Wo (Nicholas), SZE >Priority: Critical > Fix For: 0.20.205.0, 0.20.206.0, 0.24.0 > > Attachments: h2395_20111003.patch, h2395_20111004.patch, > h2395_20111004_0.20s.patch, h2395_20111004b.patch, h2395_20111004c.patch, > h2395_20111004c_0.20s.patch, h2395_20111004d.patch, > h2395_20111004d_0.20s.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2395) webhdfs api's should return a root element in the json response
[ https://issues.apache.org/jira/browse/HDFS-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-2395: - Resolution: Duplicate Fix Version/s: 0.24.0 0.20.206.0 0.20.205.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I have committed this. > webhdfs api's should return a root element in the json response > --- > > Key: HDFS-2395 > URL: https://issues.apache.org/jira/browse/HDFS-2395 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Tsz Wo (Nicholas), SZE >Priority: Critical > Fix For: 0.20.205.0, 0.20.206.0, 0.24.0 > > Attachments: h2395_20111003.patch, h2395_20111004.patch, > h2395_20111004_0.20s.patch, h2395_20111004b.patch, h2395_20111004c.patch, > h2395_20111004c_0.20s.patch, h2395_20111004d.patch, > h2395_20111004d_0.20s.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2395) webhdfs api's should return a root element in the json response
[ https://issues.apache.org/jira/browse/HDFS-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120844#comment-13120844 ] Hudson commented on HDFS-2395: -- Integrated in Hadoop-Hdfs-trunk-Commit #1096 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1096/]) HDFS-2395. Add a root element in the JSON responses of webhdfs. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179169 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java > webhdfs api's should return a root element in the json response > --- > > Key: HDFS-2395 > URL: https://issues.apache.org/jira/browse/HDFS-2395 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Tsz Wo (Nicholas), SZE >Priority: Critical > Attachments: h2395_20111003.patch, h2395_20111004.patch, > h2395_20111004_0.20s.patch, h2395_20111004b.patch, h2395_20111004c.patch, > h2395_20111004c_0.20s.patch, h2395_20111004d.patch, > h2395_20111004d_0.20s.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2395) webhdfs api's should return a root element in the json response
[ https://issues.apache.org/jira/browse/HDFS-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120842#comment-13120842 ] Hudson commented on HDFS-2395: -- Integrated in Hadoop-Common-trunk-Commit #1018 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1018/]) HDFS-2395. Add a root element in the JSON responses of webhdfs. szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1179169 Files : * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/JsonUtil.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/web/WebHdfsFileSystem.java * /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestJsonUtil.java > webhdfs api's should return a root element in the json response > --- > > Key: HDFS-2395 > URL: https://issues.apache.org/jira/browse/HDFS-2395 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Tsz Wo (Nicholas), SZE >Priority: Critical > Attachments: h2395_20111003.patch, h2395_20111004.patch, > h2395_20111004_0.20s.patch, h2395_20111004b.patch, h2395_20111004c.patch, > h2395_20111004c_0.20s.patch, h2395_20111004d.patch, > h2395_20111004d_0.20s.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2395) webhdfs api's should return a root element in the json response
[ https://issues.apache.org/jira/browse/HDFS-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120763#comment-13120763 ] Hadoop QA commented on HDFS-2395: - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12497750/h2395_20111004c.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1337//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1337//console This message is automatically generated. > webhdfs api's should return a root element in the json response > --- > > Key: HDFS-2395 > URL: https://issues.apache.org/jira/browse/HDFS-2395 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Tsz Wo (Nicholas), SZE >Priority: Critical > Attachments: h2395_20111003.patch, h2395_20111004.patch, > h2395_20111004_0.20s.patch, h2395_20111004b.patch, h2395_20111004c.patch, > h2395_20111004c_0.20s.patch, h2395_20111004d.patch, > h2395_20111004d_0.20s.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2395) webhdfs api's should return a root element in the json response
[ https://issues.apache.org/jira/browse/HDFS-2395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120739#comment-13120739 ] Hadoop QA commented on HDFS-2395: - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12497746/h2395_20111004b.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1336//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1336//console This message is automatically generated. > webhdfs api's should return a root element in the json response > --- > > Key: HDFS-2395 > URL: https://issues.apache.org/jira/browse/HDFS-2395 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 0.20.205.0 >Reporter: Arpit Gupta >Assignee: Tsz Wo (Nicholas), SZE >Priority: Critical > Attachments: h2395_20111003.patch, h2395_20111004.patch, > h2395_20111004_0.20s.patch, h2395_20111004b.patch, h2395_20111004c.patch, > h2395_20111004c_0.20s.patch, h2395_20111004d.patch, > h2395_20111004d_0.20s.patch > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
