[jira] [Commented] (HDFS-17014) HttpFS Add Support getStatus API
[
https://issues.apache.org/jira/browse/HDFS-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17726005#comment-17726005
]
Shilun Fan commented on HDFS-17014:
---
[~ayushtkn] Thank you very much for your reply!
Sorry, we missed providing one of the logs. [~zhtttylzz] and I looked at this
issue together via email, and [~zhtttylzz] provided an error log and explained
the possible causes.
The link to the error log is as follows:
[https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1234/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt]
BaseTestHttpFSWith#testGetStatus
{code:java}
[INFO] Running org.apache.hadoop.fs.http.client.TestHttpFSWithHttpFSFileSystem
[ERROR] Tests run: 95, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 45.872
s <<< FAILURE! - in
org.apache.hadoop.fs.http.client.TestHttpFSWithHttpFSFileSystem
[ERROR]
testOperationDoAs[46](org.apache.hadoop.fs.http.client.TestHttpFSWithHttpFSFileSystem)
Time elapsed: 0.25 s <<< FAILURE!
java.lang.AssertionError: expected:<459519> but was:<459531>
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.failNotEquals(Assert.java:835)
at org.junit.Assert.assertEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:633) {code}
The cause of this error seems to be consistent with the explanation provided by
[~zhtttylzz] .
> HttpFS Add Support getStatus API
>
>
> Key: HDFS-17014
> URL: https://issues.apache.org/jira/browse/HDFS-17014
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 3.4.0
>Reporter: Hualong Zhang
>Assignee: Hualong Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: image-2023-05-24-21-58-28-674.png
>
>
> We should ensure that *WebHDFS* remains synchronized with {*}HttpFS{*}, as
> the former has already implemented the *getStatus* interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17014) HttpFS Add Support getStatus API
[
https://issues.apache.org/jira/browse/HDFS-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725905#comment-17725905
]
Ayush Saxena commented on HDFS-17014:
-
In Jenkins test classes are run in parallel, but the tests in a single test
class are running sequentially only, So, if you have MiniDfsCluster per class
that is ok, unless there is any cleanup issue with any individual test, which
creates issues later.
And I don't think the failures were around failing getUsed, some snapshot stuff
[https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1233/testReport/junit/org.apache.hadoop.fs.http.client/TestHttpFSWithHttpFSFileSystem/testOperationDoAs_38_/]
maybe some test created snap2 but didn't delete the snap2
then something else the other day
[https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1234/testReport/junit/org.apache.hadoop.fs.http.client/TestHttpFSFileSystemLocalFileSystem/testOperationDoAs_6_/]
but the good thing is that, it doesn't seems related to this PR since they
passed the next day:
> HttpFS Add Support getStatus API
>
>
> Key: HDFS-17014
> URL: https://issues.apache.org/jira/browse/HDFS-17014
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 3.4.0
>Reporter: Hualong Zhang
>Assignee: Hualong Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Attachments: image-2023-05-24-21-58-28-674.png
>
>
> We should ensure that *WebHDFS* remains synchronized with {*}HttpFS{*}, as
> the former has already implemented the *getStatus* interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17014) HttpFS Add Support getStatus API
[
https://issues.apache.org/jira/browse/HDFS-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725814#comment-17725814
]
Shilun Fan commented on HDFS-17014:
---
[~zhtttylzz] Good catch! There is indeed a concurrency issue in this area.
> HttpFS Add Support getStatus API
>
>
> Key: HDFS-17014
> URL: https://issues.apache.org/jira/browse/HDFS-17014
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 3.4.0
>Reporter: Hualong Zhang
>Assignee: Hualong Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> We should ensure that *WebHDFS* remains synchronized with {*}HttpFS{*}, as
> the former has already implemented the *getStatus* interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17014) HttpFS Add Support getStatus API
[
https://issues.apache.org/jira/browse/HDFS-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725809#comment-17725809
]
Hualong Zhang commented on HDFS-17014:
--
[~ayushtkn] Thank you for your reminder! I
After careful examination, we have discovered that this segment of logic in
{*}BaseTestHttpFSWith{*}#{*}testGetStatus{*} may potentially cause the issue.
{noformat}
FsStatus dfsFsStatus = dfs.getStatus(path);
FsStatus httpFsStatus = httpFs.getStatus(path);
// Validate used free and capacity are the same as DistributedFileSystem
assertEquals(dfsFsStatus.getUsed(), httpFsStatus.getUsed());
assertEquals(dfsFsStatus.getRemaining(), httpFsStatus.getRemaining());
assertEquals(dfsFsStatus.getCapacity(), httpFsStatus.getCapacity());
{noformat}
The reasons are as follows:
1.The *getStatus* API used to retrieve the used/remaining space of the
FileSystem may pose a problem. In {*}TestHttpFSWithHttpFSFileSystem{*}, all
*unit tests* share a single FileSystem and are executed in parallel, with file
writes or deletions occurring randomly. As a result, the usage size of the file
system may change at any time, causing the used/remaining values returned by
*httpFs.getStatus* to potentially differ from those returned by
{*}dfs.getStatus{*}, thereby resulting in test failures.
2.When conducting *unit tests* in {*}TestWebHDFS{*}, we utilized a separate
*MiniDFSCluster* to avoid encountering this issue.
We have the option of modifying the *unit tests* to only validate whether
used/remaining/capacity >= 0, or alternatively, using a new MiniDFSCluster in
this particular test to ensure that any writes or deletions performed by other
unit tests do not impact our test results.
> HttpFS Add Support getStatus API
>
>
> Key: HDFS-17014
> URL: https://issues.apache.org/jira/browse/HDFS-17014
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 3.4.0
>Reporter: Hualong Zhang
>Assignee: Hualong Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> We should ensure that *WebHDFS* remains synchronized with {*}HttpFS{*}, as
> the former has already implemented the *getStatus* interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17014) HttpFS Add Support getStatus API
[
https://issues.apache.org/jira/browse/HDFS-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725156#comment-17725156
]
Hualong Zhang commented on HDFS-17014:
--
[~ayushtkn] Thank you for bringing this to our attention!! We will investigate
the related issues immediately and resolve them as soon as possible.
> HttpFS Add Support getStatus API
>
>
> Key: HDFS-17014
> URL: https://issues.apache.org/jira/browse/HDFS-17014
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 3.4.0
>Reporter: Hualong Zhang
>Assignee: Hualong Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> We should ensure that *WebHDFS* remains synchronized with {*}HttpFS{*}, as
> the former has already implemented the *getStatus* interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17014) HttpFS Add Support getStatus API
[
https://issues.apache.org/jira/browse/HDFS-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725151#comment-17725151
]
Shilun Fan commented on HDFS-17014:
---
[~ayushtkn] Thank you for your reminder! I will check whether the error is
related to this pr as soon as possible.
> HttpFS Add Support getStatus API
>
>
> Key: HDFS-17014
> URL: https://issues.apache.org/jira/browse/HDFS-17014
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 3.4.0
>Reporter: Hualong Zhang
>Assignee: Hualong Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> We should ensure that *WebHDFS* remains synchronized with {*}HttpFS{*}, as
> the former has already implemented the *getStatus* interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17014) HttpFS Add Support getStatus API
[
https://issues.apache.org/jira/browse/HDFS-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17725148#comment-17725148
]
Ayush Saxena commented on HDFS-17014:
-
[~zhtttylzz] I think there are some test failures being reported:
[https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1233/#showFailuresLink]
[https://ci-hadoop.apache.org/view/Hadoop/job/hadoop-qbt-trunk-java8-linux-x86_64/1234/]
Can you check if related.
cc. [~slfan1989]
> HttpFS Add Support getStatus API
>
>
> Key: HDFS-17014
> URL: https://issues.apache.org/jira/browse/HDFS-17014
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 3.4.0
>Reporter: Hualong Zhang
>Assignee: Hualong Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> We should ensure that *WebHDFS* remains synchronized with {*}HttpFS{*}, as
> the former has already implemented the *getStatus* interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17014) HttpFS Add Support getStatus API
[
https://issues.apache.org/jira/browse/HDFS-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724576#comment-17724576
]
ASF GitHub Bot commented on HDFS-17014:
---
slfan1989 commented on PR #5660:
URL: https://github.com/apache/hadoop/pull/5660#issuecomment-1556059819
@ayushtkn Thank you very much for helping to merge the code! @zhtttylz
Thanks for your contribution!
> HttpFS Add Support getStatus API
>
>
> Key: HDFS-17014
> URL: https://issues.apache.org/jira/browse/HDFS-17014
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 3.4.0
>Reporter: Hualong Zhang
>Assignee: Hualong Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> We should ensure that *WebHDFS* remains synchronized with {*}HttpFS{*}, as
> the former has already implemented the *getStatus* interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17014) HttpFS Add Support getStatus API
[
https://issues.apache.org/jira/browse/HDFS-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724514#comment-17724514
]
ASF GitHub Bot commented on HDFS-17014:
---
zhtttylz commented on PR #5660:
URL: https://github.com/apache/hadoop/pull/5660#issuecomment-1555870815
@ayushtkn @slfan1989 Thank you for your assistance in reviewing the code!
> HttpFS Add Support getStatus API
>
>
> Key: HDFS-17014
> URL: https://issues.apache.org/jira/browse/HDFS-17014
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 3.4.0
>Reporter: Hualong Zhang
>Assignee: Hualong Zhang
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> We should ensure that *WebHDFS* remains synchronized with {*}HttpFS{*}, as
> the former has already implemented the *getStatus* interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17014) HttpFS Add Support getStatus API
[
https://issues.apache.org/jira/browse/HDFS-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724510#comment-17724510
]
Ayush Saxena commented on HDFS-17014:
-
Committed to trunk.
Thanx [~zhtttylzz] for the contribution & [~slfan1989] for the review!!!
> HttpFS Add Support getStatus API
>
>
> Key: HDFS-17014
> URL: https://issues.apache.org/jira/browse/HDFS-17014
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 3.4.0
>Reporter: Hualong Zhang
>Assignee: Hualong Zhang
>Priority: Major
> Labels: pull-request-available
>
> We should ensure that *WebHDFS* remains synchronized with {*}HttpFS{*}, as
> the former has already implemented the *getStatus* interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17014) HttpFS Add Support getStatus API
[
https://issues.apache.org/jira/browse/HDFS-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724509#comment-17724509
]
ASF GitHub Bot commented on HDFS-17014:
---
ayushtkn merged PR #5660:
URL: https://github.com/apache/hadoop/pull/5660
> HttpFS Add Support getStatus API
>
>
> Key: HDFS-17014
> URL: https://issues.apache.org/jira/browse/HDFS-17014
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 3.4.0
>Reporter: Hualong Zhang
>Assignee: Hualong Zhang
>Priority: Major
> Labels: pull-request-available
>
> We should ensure that *WebHDFS* remains synchronized with {*}HttpFS{*}, as
> the former has already implemented the *getStatus* interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17014) HttpFS Add Support getStatus API
[
https://issues.apache.org/jira/browse/HDFS-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724489#comment-17724489
]
ASF GitHub Bot commented on HDFS-17014:
---
hadoop-yetus commented on PR #5660:
URL: https://github.com/apache/hadoop/pull/5660#issuecomment-1555658752
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 12m 11s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 1 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 33m 4s | | trunk passed |
| +1 :green_heart: | compile | 0m 30s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | compile | 0m 28s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | checkstyle | 0m 36s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 44s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 33s | | trunk passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 29s | | trunk passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 1m 3s | | trunk passed |
| +1 :green_heart: | shadedclient | 20m 27s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 20s | | the patch passed |
| +1 :green_heart: | compile | 0m 21s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javac | 0m 21s | | the patch passed |
| +1 :green_heart: | compile | 0m 19s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | javac | 0m 19s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 19s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 27s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1 |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| +1 :green_heart: | spotbugs | 0m 48s | | the patch passed |
| +1 :green_heart: | shadedclient | 19m 55s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 5m 38s | | hadoop-hdfs-httpfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 37s | | The patch does not
generate ASF License warnings. |
| | | 103m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5660/5/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/5660 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets |
| uname | Linux 5c277d873bdc 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3
19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 87679d5277e49ed4ef1d39d766da452cb25aa095 |
| Default Java | Private Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.19+7-post-Ubuntu-0ubuntu120.04.1
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_362-8u372-ga~us1-0ubuntu1~20.04-b09 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5660/5/testReport/ |
| Max. process+thread count | 839 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-httpfs U:
hadoop-hdfs-project/hadoop-hdfs-httpfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5660/5/console |
| versions | git=2.25.1 maven=3.6.3 spotbugs=4.2.2 |
| Powered by | Apache Yetus 0.14.0 https://yetus.apache.org |
This message was automatically generated.
[jira] [Commented] (HDFS-17014) HttpFS Add Support getStatus API
[
https://issues.apache.org/jira/browse/HDFS-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724476#comment-17724476
]
ASF GitHub Bot commented on HDFS-17014:
---
zhtttylz commented on code in PR #5660:
URL: https://github.com/apache/hadoop/pull/5660#discussion_r1199540321
##
hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java:
##
@@ -2081,6 +2085,32 @@ private void testGetFileLinkStatus() throws Exception {
assertTrue(fs.getFileLinkStatus(linkToFile).isSymlink());
}
+ private void testGetStatus() throws Exception {
+if (isLocalFS()) {
+ // do not test the getStatus for local FS.
+ return;
+}
+final Path path = new Path("/foo");
+FileSystem fs = FileSystem.get(path.toUri(), this.getProxiedFSConf());
+if (fs instanceof DistributedFileSystem) {
+ DistributedFileSystem dfs =
+ (DistributedFileSystem) FileSystem.get(path.toUri(),
this.getProxiedFSConf());
+ FileSystem httpFs = this.getHttpFSFileSystem();
+
+ FsStatus dfsFsStatus = dfs.getStatus(path);
+ FsStatus httpFsStatus = httpFs.getStatus(path);
+
+ //Validate used free and capacity are the same as DistributedFileSystem
+ assertEquals(dfsFsStatus.getUsed(), httpFsStatus.getUsed());
+ assertEquals(dfsFsStatus.getRemaining(), httpFsStatus.getRemaining());
+ assertEquals(dfsFsStatus.getCapacity(), httpFsStatus.getCapacity());
+ httpFs.close();
+ dfs.close();
+}else{
+ Assert.fail(fs.getClass().getSimpleName() + " is not of type
DistributedFileSystem.");
+}
Review Comment:
Thank you very much for your valuable suggestion. I genuinely appreciate it
and will make the necessary adjustments to the code promptly.
> HttpFS Add Support getStatus API
>
>
> Key: HDFS-17014
> URL: https://issues.apache.org/jira/browse/HDFS-17014
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 3.4.0
>Reporter: Hualong Zhang
>Assignee: Hualong Zhang
>Priority: Major
> Labels: pull-request-available
>
> We should ensure that *WebHDFS* remains synchronized with {*}HttpFS{*}, as
> the former has already implemented the *getStatus* interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-17014) HttpFS Add Support getStatus API
[
https://issues.apache.org/jira/browse/HDFS-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17724463#comment-17724463
]
ASF GitHub Bot commented on HDFS-17014:
---
ayushtkn commented on code in PR #5660:
URL: https://github.com/apache/hadoop/pull/5660#discussion_r1199520649
##
hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/client/BaseTestHttpFSWith.java:
##
@@ -2081,6 +2085,32 @@ private void testGetFileLinkStatus() throws Exception {
assertTrue(fs.getFileLinkStatus(linkToFile).isSymlink());
}
+ private void testGetStatus() throws Exception {
+if (isLocalFS()) {
+ // do not test the getStatus for local FS.
+ return;
+}
+final Path path = new Path("/foo");
+FileSystem fs = FileSystem.get(path.toUri(), this.getProxiedFSConf());
+if (fs instanceof DistributedFileSystem) {
+ DistributedFileSystem dfs =
+ (DistributedFileSystem) FileSystem.get(path.toUri(),
this.getProxiedFSConf());
+ FileSystem httpFs = this.getHttpFSFileSystem();
+
+ FsStatus dfsFsStatus = dfs.getStatus(path);
+ FsStatus httpFsStatus = httpFs.getStatus(path);
+
+ //Validate used free and capacity are the same as DistributedFileSystem
+ assertEquals(dfsFsStatus.getUsed(), httpFsStatus.getUsed());
+ assertEquals(dfsFsStatus.getRemaining(), httpFsStatus.getRemaining());
+ assertEquals(dfsFsStatus.getCapacity(), httpFsStatus.getCapacity());
+ httpFs.close();
+ dfs.close();
+}else{
+ Assert.fail(fs.getClass().getSimpleName() + " is not of type
DistributedFileSystem.");
+}
Review Comment:
it is still not formatted, space before and after else.
```
} else {
```
> HttpFS Add Support getStatus API
>
>
> Key: HDFS-17014
> URL: https://issues.apache.org/jira/browse/HDFS-17014
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: webhdfs
>Affects Versions: 3.4.0
>Reporter: Hualong Zhang
>Assignee: Hualong Zhang
>Priority: Major
>
> We should ensure that *WebHDFS* remains synchronized with {*}HttpFS{*}, as
> the former has already implemented the *getStatus* interface.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
