[jira] [Work logged] (HDFS-16384) Upgrade Netty to 4.1.72.Final
[
https://issues.apache.org/jira/browse/HDFS-16384?focusedWorklogId=753852&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753852
]
ASF GitHub Bot logged work on HDFS-16384:
-
Author: ASF GitHub Bot
Created on: 07/Apr/22 05:58
Start Date: 07/Apr/22 05:58
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4132:
URL: https://github.com/apache/hadoop/pull/4132#issuecomment-1091110561
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 18m 7s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | shelldocs | 0m 1s | | Shelldocs was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 52s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 27m 58s | | trunk passed |
| +1 :green_heart: | compile | 25m 43s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 25m 41s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | mvnsite | 28m 57s | | trunk passed |
| +1 :green_heart: | javadoc | 8m 39s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 8m 27s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | shadedclient | 39m 35s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 59s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 26m 53s | | the patch passed |
| +1 :green_heart: | compile | 24m 33s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 24m 33s | | the patch passed |
| +1 :green_heart: | compile | 21m 30s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 30s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | mvnsite | 23m 3s | | the patch passed |
| +1 :green_heart: | shellcheck | 0m 0s | | No new issues. |
| +1 :green_heart: | xml | 0m 4s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 8m 22s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 8m 20s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | shadedclient | 40m 24s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 1030m 46s |
[/patch-unit-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4132/2/artifact/out/patch-unit-root.txt)
| root in the patch passed. |
| +1 :green_heart: | asflicense | 1m 16s | | The patch does not
generate ASF License warnings. |
| | | 1353m 49s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4132/2/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4132 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient codespell xml shellcheck shelldocs |
| uname | Linux d5d47f624876 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5
11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / a7cbfb64da26bedad326450fbc6c2427744f7585 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| T
[jira] [Work logged] (HDFS-16457) Make fs.getspaceused.classname reconfigurable
[ https://issues.apache.org/jira/browse/HDFS-16457?focusedWorklogId=753844&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753844 ] ASF GitHub Bot logged work on HDFS-16457: - Author: ASF GitHub Bot Created on: 07/Apr/22 05:08 Start Date: 07/Apr/22 05:08 Worklog Time Spent: 10m Work Description: tasanuma commented on PR #4069: URL: https://github.com/apache/hadoop/pull/4069#issuecomment-1091085883 @singer-bin It almost looks good to me. But, I have one more request. Please arrange the order of the import sentences alphabetically as much as possible. If you use IntelliJ, it will do it by pressing Alt + Enter key. https://user-images.githubusercontent.com/11712443/162124284-e5fc48ba-11ff-4dc9-bd45-61f91af070ed.png";> Issue Time Tracking --- Worklog Id: (was: 753844) Time Spent: 5.5h (was: 5h 20m) > Make fs.getspaceused.classname reconfigurable > - > > Key: HDFS-16457 > URL: https://issues.apache.org/jira/browse/HDFS-16457 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 3.3.0 >Reporter: yanbin.zhang >Assignee: yanbin.zhang >Priority: Major > Labels: pull-request-available > Time Spent: 5.5h > Remaining Estimate: 0h > > Now if we want to switch fs.getspaceused.classname we need to restart the > NameNode. It would be convenient if we can switch it at runtime. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16473) Make HDFS stat tool cross platform
[ https://issues.apache.org/jira/browse/HDFS-16473?focusedWorklogId=753839&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753839 ] ASF GitHub Bot logged work on HDFS-16473: - Author: ASF GitHub Bot Created on: 07/Apr/22 04:40 Start Date: 07/Apr/22 04:40 Worklog Time Spent: 10m Work Description: GauthamBanasandra opened a new pull request, #4145: URL: https://github.com/apache/hadoop/pull/4145 ### Description of PR The source files for `hdfs_stat` uses `getopt` for parsing the command line arguments. getopt is available only on Linux and thus, isn't cross platform. We need to replace getopt with `boost::program_options` to make this tool cross platform. ### How was this patch tested? # hdfs_stat This tool was testing locally with the Hadoop cluster setup in pseudo-distributed mode. I ran the tool by providing the path to a file as the argument. ```bash $ ./hdfs_stat ~/some-dir/some-data.txt ```  I verified the output against the information displayed in the UI.  I then ran the tool by providing the path to a directory as the argument. ```bash $ ./hdfs_stat ~/some-dir ```  I verified the output against the information displayed in the UI.  I then ran the tool by providing a non-existent path as the argument. ```bash $ ./hdfs_stat ~/some-non-existent-path ```  We note that an appropriate error message is reported in the above output. ### For code changes: - [x] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? Issue Time Tracking --- Worklog Id: (was: 753839) Remaining Estimate: 0h Time Spent: 10m > Make HDFS stat tool cross platform > -- > > Key: HDFS-16473 > URL: https://issues.apache.org/jira/browse/HDFS-16473 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, libhdfs++, tools >Affects Versions: 3.4.0 > Environment: Centos 7, Centos 8, Debian 10, Ubuntu Focal >Reporter: Gautham Banasandra >Assignee: Gautham Banasandra >Priority: Major > Labels: libhdfscpp > Time Spent: 10m > Remaining Estimate: 0h > > The source files for *hdfs_stat* uses *getopt* for parsing the command line > arguments. getopt is available only on Linux and thus, isn't cross platform. > We need to replace getopt with *boost::program_options* to make this tool > cross platform. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16473) Make HDFS stat tool cross platform
[ https://issues.apache.org/jira/browse/HDFS-16473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-16473: -- Labels: libhdfscpp pull-request-available (was: libhdfscpp) > Make HDFS stat tool cross platform > -- > > Key: HDFS-16473 > URL: https://issues.apache.org/jira/browse/HDFS-16473 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, libhdfs++, tools >Affects Versions: 3.4.0 > Environment: Centos 7, Centos 8, Debian 10, Ubuntu Focal >Reporter: Gautham Banasandra >Assignee: Gautham Banasandra >Priority: Major > Labels: libhdfscpp, pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > The source files for *hdfs_stat* uses *getopt* for parsing the command line > arguments. getopt is available only on Linux and thus, isn't cross platform. > We need to replace getopt with *boost::program_options* to make this tool > cross platform. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16473) Make HDFS stat tool cross platform
[ https://issues.apache.org/jira/browse/HDFS-16473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gautham Banasandra updated HDFS-16473: -- Description: The source files for *hdfs_stat* uses *getopt* for parsing the command line arguments. getopt is available only on Linux and thus, isn't cross platform. We need to replace getopt with *boost::program_options* to make this tool cross platform. (was: The source files for *hdfs_stat* uses *getopt* for parsing the command line arguments. getopt is available only on Linux and thus, isn't cross platform. We need to replace getopt with *boost::program_options* to make these tools cross platform.) > Make HDFS stat tool cross platform > -- > > Key: HDFS-16473 > URL: https://issues.apache.org/jira/browse/HDFS-16473 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client, libhdfs++, tools >Affects Versions: 3.4.0 > Environment: Centos 7, Centos 8, Debian 10, Ubuntu Focal >Reporter: Gautham Banasandra >Assignee: Gautham Banasandra >Priority: Major > Labels: libhdfscpp > > The source files for *hdfs_stat* uses *getopt* for parsing the command line > arguments. getopt is available only on Linux and thus, isn't cross platform. > We need to replace getopt with *boost::program_options* to make this tool > cross platform. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16457) Make fs.getspaceused.classname reconfigurable
[ https://issues.apache.org/jira/browse/HDFS-16457?focusedWorklogId=753739&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753739 ] ASF GitHub Bot logged work on HDFS-16457: - Author: ASF GitHub Bot Created on: 06/Apr/22 22:24 Start Date: 06/Apr/22 22:24 Worklog Time Spent: 10m Work Description: singer-bin commented on PR #4069: URL: https://github.com/apache/hadoop/pull/4069#issuecomment-1090872289 I ran it several times, and the error has nothing to do with my code. Can you help me, thank you. @tasanuma Issue Time Tracking --- Worklog Id: (was: 753739) Time Spent: 5h 20m (was: 5h 10m) > Make fs.getspaceused.classname reconfigurable > - > > Key: HDFS-16457 > URL: https://issues.apache.org/jira/browse/HDFS-16457 > Project: Hadoop HDFS > Issue Type: Improvement > Components: namenode >Affects Versions: 3.3.0 >Reporter: yanbin.zhang >Assignee: yanbin.zhang >Priority: Major > Labels: pull-request-available > Time Spent: 5h 20m > Remaining Estimate: 0h > > Now if we want to switch fs.getspaceused.classname we need to restart the > NameNode. It would be convenient if we can switch it at runtime. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-13522) RBF: Support observer node from Router-Based Federation
[
https://issues.apache.org/jira/browse/HDFS-13522?focusedWorklogId=753726&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753726
]
ASF GitHub Bot logged work on HDFS-13522:
-
Author: ASF GitHub Bot
Created on: 06/Apr/22 22:07
Start Date: 06/Apr/22 22:07
Worklog Time Spent: 10m
Work Description: simbadzina commented on code in PR #4127:
URL: https://github.com/apache/hadoop/pull/4127#discussion_r844460652
##
hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/NameNodeProxiesClient.java:
##
@@ -349,6 +349,18 @@ public static ClientProtocol
createProxyWithAlignmentContext(
boolean withRetries, AtomicBoolean fallbackToSimpleAuth,
AlignmentContext alignmentContext)
throws IOException {
+if (!conf.getBoolean(HdfsClientConfigKeys.DFS_OBSERVER_READ_ENABLE,
Review Comment:
Acknowledged. I'm still trying to verify the patch with a real cluster. Once
this is done, I can try splitting it up.
Issue Time Tracking
---
Worklog Id: (was: 753726)
Time Spent: 4h 40m (was: 4.5h)
> RBF: Support observer node from Router-Based Federation
> ---
>
> Key: HDFS-13522
> URL: https://issues.apache.org/jira/browse/HDFS-13522
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: federation, namenode
>Reporter: Erik Krogen
>Assignee: Simbarashe Dzinamarira
>Priority: Major
> Labels: pull-request-available
> Attachments: HDFS-13522.001.patch, HDFS-13522.002.patch,
> HDFS-13522_WIP.patch, RBF_ Observer support.pdf, Router+Observer RPC
> clogging.png, ShortTerm-Routers+Observer.png
>
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> Changes will need to occur to the router to support the new observer node.
> One such change will be to make the router understand the observer state,
> e.g. {{FederationNamenodeServiceState}}.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16457) Make fs.getspaceused.classname reconfigurable
[
https://issues.apache.org/jira/browse/HDFS-16457?focusedWorklogId=753661&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753661
]
ASF GitHub Bot logged work on HDFS-16457:
-
Author: ASF GitHub Bot
Created on: 06/Apr/22 20:22
Start Date: 06/Apr/22 20:22
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4069:
URL: https://github.com/apache/hadoop/pull/4069#issuecomment-1090745325
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 1s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 38m 49s | | trunk passed |
| +1 :green_heart: | compile | 1m 26s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 7s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 32s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 32s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 39s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 47s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 24m 6s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 20s | | the patch passed |
| +1 :green_heart: | compile | 1m 25s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 25s | | the patch passed |
| +1 :green_heart: | compile | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 53s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 24s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 58s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 35s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 18s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 29s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 406m 33s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4069/12/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 51s | | The patch does not
generate ASF License warnings. |
| | | 515m 47s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestObserverNode |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4069/12/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4069 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 1eb32ed85f5a 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 4f3341c0793a7a9e56dfef56f89b98fbb9eb79cc |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Resu
[jira] [Resolved] (HDFS-16530) setReplication debug log creates a new string even if debug is disabled
[
https://issues.apache.org/jira/browse/HDFS-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen O'Donnell resolved HDFS-16530.
--
Resolution: Fixed
> setReplication debug log creates a new string even if debug is disabled
> ---
>
> Key: HDFS-16530
> URL: https://issues.apache.org/jira/browse/HDFS-16530
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: Stephen O'Donnell
>Assignee: Stephen O'Donnell
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.2.4, 3.3.3
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In FSDirAttrOp, HDFS-14521 made a good change to move a noisy logger to debug:
> {code}
> if (oldBR > targetReplication) {
> FSDirectory.LOG.debug("Decreasing replication from {} to {} for {}",
> oldBR, targetReplication, iip.getPath());
> } else if (oldBR < targetReplication) {
> FSDirectory.LOG.debug("Increasing replication from {} to {} for {}",
> oldBR, targetReplication, iip.getPath());
> } else {
> FSDirectory.LOG.debug("Replication remains unchanged at {} for {}",
> oldBR, iip.getPath());
> }
> }
> {code}
> However the `iip.getPath()` method must be evaluated to pass the resulting
> string into the LOG.debug method, even if debug is not enabled:
> This code may form a new string where it does not need to:
> {code}
> public String getPath() {
> if (pathname == null) {
> pathname = DFSUtil.byteArray2PathString(path);
> }
> return pathname;
> }
> {code}
> We should wrap the entire logging block in `if LOG.debugEnabled()` to avoid
> any overhead when the logger is not enabled.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16530) setReplication debug log creates a new string even if debug is disabled
[
https://issues.apache.org/jira/browse/HDFS-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen O'Donnell updated HDFS-16530:
-
Fix Version/s: 3.2.4
> setReplication debug log creates a new string even if debug is disabled
> ---
>
> Key: HDFS-16530
> URL: https://issues.apache.org/jira/browse/HDFS-16530
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: Stephen O'Donnell
>Assignee: Stephen O'Donnell
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.2.4, 3.3.3
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In FSDirAttrOp, HDFS-14521 made a good change to move a noisy logger to debug:
> {code}
> if (oldBR > targetReplication) {
> FSDirectory.LOG.debug("Decreasing replication from {} to {} for {}",
> oldBR, targetReplication, iip.getPath());
> } else if (oldBR < targetReplication) {
> FSDirectory.LOG.debug("Increasing replication from {} to {} for {}",
> oldBR, targetReplication, iip.getPath());
> } else {
> FSDirectory.LOG.debug("Replication remains unchanged at {} for {}",
> oldBR, iip.getPath());
> }
> }
> {code}
> However the `iip.getPath()` method must be evaluated to pass the resulting
> string into the LOG.debug method, even if debug is not enabled:
> This code may form a new string where it does not need to:
> {code}
> public String getPath() {
> if (pathname == null) {
> pathname = DFSUtil.byteArray2PathString(path);
> }
> return pathname;
> }
> {code}
> We should wrap the entire logging block in `if LOG.debugEnabled()` to avoid
> any overhead when the logger is not enabled.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16457) Make fs.getspaceused.classname reconfigurable
[
https://issues.apache.org/jira/browse/HDFS-16457?focusedWorklogId=753314&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753314
]
ASF GitHub Bot logged work on HDFS-16457:
-
Author: ASF GitHub Bot
Created on: 06/Apr/22 11:30
Start Date: 06/Apr/22 11:30
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4069:
URL: https://github.com/apache/hadoop/pull/4069#issuecomment-1090161172
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 7s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 52s | | trunk passed |
| +1 :green_heart: | compile | 1m 29s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 22s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 9s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 37s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 19s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 43s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 4m 1s | | trunk passed |
| +1 :green_heart: | shadedclient | 26m 39s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 27m 0s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 26s | | the patch passed |
| +1 :green_heart: | compile | 1m 30s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 30s | | the patch passed |
| +1 :green_heart: | compile | 1m 24s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 24s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 1m 0s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 28s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 56s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 29s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 41s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 28s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 429m 7s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4069/11/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
| hadoop-hdfs in the patch passed. |
| +1 :green_heart: | asflicense | 0m 48s | | The patch does not
generate ASF License warnings. |
| | | 544m 14s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests |
hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4069/11/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4069 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 4acf8d6b5de4 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / e5052de0914c386ed3645486f288ef172dc49066 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
[jira] [Created] (HDFS-16532) HDFS Balancer does not move by bytes but by the original folder
ErJiuMi created HDFS-16532: -- Summary: HDFS Balancer does not move by bytes but by the original folder Key: HDFS-16532 URL: https://issues.apache.org/jira/browse/HDFS-16532 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs Reporter: ErJiuMi When using the HDFS cluster Balancer tool, is it possible to do Balance without destroying the original folder? -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16530) setReplication debug log creates a new string even if debug is disabled
[
https://issues.apache.org/jira/browse/HDFS-16530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stephen O'Donnell updated HDFS-16530:
-
Fix Version/s: 3.4.0
3.3.3
> setReplication debug log creates a new string even if debug is disabled
> ---
>
> Key: HDFS-16530
> URL: https://issues.apache.org/jira/browse/HDFS-16530
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: Stephen O'Donnell
>Assignee: Stephen O'Donnell
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.3
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In FSDirAttrOp, HDFS-14521 made a good change to move a noisy logger to debug:
> {code}
> if (oldBR > targetReplication) {
> FSDirectory.LOG.debug("Decreasing replication from {} to {} for {}",
> oldBR, targetReplication, iip.getPath());
> } else if (oldBR < targetReplication) {
> FSDirectory.LOG.debug("Increasing replication from {} to {} for {}",
> oldBR, targetReplication, iip.getPath());
> } else {
> FSDirectory.LOG.debug("Replication remains unchanged at {} for {}",
> oldBR, iip.getPath());
> }
> }
> {code}
> However the `iip.getPath()` method must be evaluated to pass the resulting
> string into the LOG.debug method, even if debug is not enabled:
> This code may form a new string where it does not need to:
> {code}
> public String getPath() {
> if (pathname == null) {
> pathname = DFSUtil.byteArray2PathString(path);
> }
> return pathname;
> }
> {code}
> We should wrap the entire logging block in `if LOG.debugEnabled()` to avoid
> any overhead when the logger is not enabled.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16457) Make fs.getspaceused.classname reconfigurable
[
https://issues.apache.org/jira/browse/HDFS-16457?focusedWorklogId=753292&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753292
]
ASF GitHub Bot logged work on HDFS-16457:
-
Author: ASF GitHub Bot
Created on: 06/Apr/22 10:24
Start Date: 06/Apr/22 10:24
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4069:
URL: https://github.com/apache/hadoop/pull/4069#issuecomment-1090105815
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 16m 59s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 39m 11s | | trunk passed |
| +1 :green_heart: | compile | 1m 27s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | compile | 1m 19s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 1m 6s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 30s | | trunk passed |
| +1 :green_heart: | javadoc | 1m 8s | | trunk passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 31s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 29s | | trunk passed |
| +1 :green_heart: | shadedclient | 23m 32s | | branch has no errors
when building and testing our client artifacts. |
| -0 :warning: | patch | 23m 51s | | Used diff version of patch file.
Binary files and potentially other changes not applied. Please rebase and
squash commits if necessary. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 22s | | the patch passed |
| +1 :green_heart: | compile | 1m 23s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javac | 1m 23s | | the patch passed |
| +1 :green_heart: | compile | 1m 18s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 1m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 53s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 20s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 55s | | the patch passed with JDK
Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04 |
| +1 :green_heart: | javadoc | 1m 30s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 3m 27s | | the patch passed |
| +1 :green_heart: | shadedclient | 23m 23s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 432m 12s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 50s | | The patch does not
generate ASF License warnings. |
| | | 556m 58s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4069/10/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4069 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux ebf438eb6fb8 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 63182b60b8d12c33abcbbce7bfc9b6d714411b39 |
| Default Java | Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14.1+1-Ubuntu-0ubuntu1.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4069/10/testReport/ |
| Max. process+thread count | 2783 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/ha
[jira] [Updated] (HDFS-16522) Set Http and Ipc ports for Datanodes in MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-16522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HDFS-16522: - Fix Version/s: 3.4.0 3.3.3 Resolution: Fixed Status: Resolved (was: Patch Available) Committed to trunk and branch-3.3. > Set Http and Ipc ports for Datanodes in MiniDFSCluster > -- > > Key: HDFS-16522 > URL: https://issues.apache.org/jira/browse/HDFS-16522 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.3 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > We should provide options to set Http and Ipc ports for Datanodes in > MiniDFSCluster. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-14478) Add libhdfs APIs for openFile
[
https://issues.apache.org/jira/browse/HDFS-14478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran reassigned HDFS-14478:
-
Assignee: Steve Loughran (was: Sahil Takiar)
> Add libhdfs APIs for openFile
> -
>
> Key: HDFS-14478
> URL: https://issues.apache.org/jira/browse/HDFS-14478
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client, libhdfs, native
>Reporter: Sahil Takiar
>Assignee: Steve Loughran
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> HADOOP-15229 added a "FileSystem builder-based openFile() API" that allows
> specifying configuration values for opening files (similar to HADOOP-14365).
> Support for {{openFile}} will be a little tricky as it is asynchronous and
> {{FutureDataInputStreamBuilder#build}} returns a {{CompletableFuture}}.
> At a high level, the API for {{openFile}} could look something like this:
> {code:java}
> hdfsFile hdfsOpenFile(hdfsFS fs, const char* path, int flags,
> int bufferSize, short replication, tSize blocksize);
> hdfsOpenFileBuilder *hdfsOpenFileBuilderAlloc(hdfsFS fs,
> const char *path);
> hdfsOpenFileBuilder *hdfsOpenFileBuilderMust(hdfsOpenFileBuilder *builder,
> const char *key, const char *value);
> hdfsOpenFileBuilder *hdfsOpenFileBuilderOpt(hdfsOpenFileBuilder *builder,
> const char *key, const char *value);
> hdfsOpenFileFuture *hdfsOpenFileBuilderBuild(hdfsOpenFileBuilder *builder);
> void hdfsOpenFileBuilderFree(hdfsOpenFileBuilder *builder);
> hdfsFile hdfsOpenFileFutureGet(hdfsOpenFileFuture *future);
> hdfsFile hdfsOpenFileFutureGetWithTimeout(hdfsOpenFileFuture *future,
> int64_t timeout, javaConcurrentTimeUnit timeUnit);
> int hdfsOpenFileFutureCancel(hdfsOpenFileFuture *future,
> int mayInterruptIfRunning);
> void hdfsOpenFileFutureFree(hdfsOpenFileFuture *future);
> {code}
> Instead of exposing all the functionality of {{CompleteableFuture}} libhdfs
> would just expose the functionality of {{Future}}.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16481) Provide support to set Http and Rpc ports in MiniJournalCluster
[ https://issues.apache.org/jira/browse/HDFS-16481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HDFS-16481: - Fix Version/s: 3.3.3 Backported to branch-3.3 because HDFS-16522 requires this. > Provide support to set Http and Rpc ports in MiniJournalCluster > --- > > Key: HDFS-16481 > URL: https://issues.apache.org/jira/browse/HDFS-16481 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 3.3.3 > > Time Spent: 6h 10m > Remaining Estimate: 0h > > We should provide support for clients to set Http and Rpc ports of > JournalNodes in MiniJournalCluster. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16522) Set Http and Ipc ports for Datanodes in MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-16522?focusedWorklogId=753273&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753273 ] ASF GitHub Bot logged work on HDFS-16522: - Author: ASF GitHub Bot Created on: 06/Apr/22 09:17 Start Date: 06/Apr/22 09:17 Worklog Time Spent: 10m Work Description: aajisaka merged PR #4108: URL: https://github.com/apache/hadoop/pull/4108 Issue Time Tracking --- Worklog Id: (was: 753273) Time Spent: 3h (was: 2h 50m) > Set Http and Ipc ports for Datanodes in MiniDFSCluster > -- > > Key: HDFS-16522 > URL: https://issues.apache.org/jira/browse/HDFS-16522 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 3h > Remaining Estimate: 0h > > We should provide options to set Http and Ipc ports for Datanodes in > MiniDFSCluster. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16522) Set Http and Ipc ports for Datanodes in MiniDFSCluster
[ https://issues.apache.org/jira/browse/HDFS-16522?focusedWorklogId=753274&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753274 ] ASF GitHub Bot logged work on HDFS-16522: - Author: ASF GitHub Bot Created on: 06/Apr/22 09:17 Start Date: 06/Apr/22 09:17 Worklog Time Spent: 10m Work Description: aajisaka commented on PR #4108: URL: https://github.com/apache/hadoop/pull/4108#issuecomment-1090044750 Merged. Thank you @virajjasani for your contribution! Issue Time Tracking --- Worklog Id: (was: 753274) Time Spent: 3h 10m (was: 3h) > Set Http and Ipc ports for Datanodes in MiniDFSCluster > -- > > Key: HDFS-16522 > URL: https://issues.apache.org/jira/browse/HDFS-16522 > Project: Hadoop HDFS > Issue Type: Task >Reporter: Viraj Jasani >Assignee: Viraj Jasani >Priority: Major > Labels: pull-request-available > Time Spent: 3h 10m > Remaining Estimate: 0h > > We should provide options to set Http and Ipc ports for Datanodes in > MiniDFSCluster. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-13522) RBF: Support observer node from Router-Based Federation
[
https://issues.apache.org/jira/browse/HDFS-13522?focusedWorklogId=753259&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753259
]
ASF GitHub Bot logged work on HDFS-13522:
-
Author: ASF GitHub Bot
Created on: 06/Apr/22 08:58
Start Date: 06/Apr/22 08:58
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on PR #4127:
URL: https://github.com/apache/hadoop/pull/4127#issuecomment-1090025395
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 49s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 11 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 15m 50s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 28m 21s | | trunk passed |
| +1 :green_heart: | compile | 24m 46s | | trunk passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 21m 12s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | checkstyle | 4m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 5m 9s | | trunk passed |
| +1 :green_heart: | javadoc | 3m 59s | | trunk passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 5m 12s | | trunk passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 10m 36s | | trunk passed |
| +1 :green_heart: | shadedclient | 27m 20s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 46s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 4m 11s | | the patch passed |
| +1 :green_heart: | compile | 25m 4s | | the patch passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 25m 4s | | the patch passed |
| +1 :green_heart: | compile | 21m 18s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | javac | 21m 18s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 3m 52s |
[/results-checkstyle-root.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4127/3/artifact/out/results-checkstyle-root.txt)
| root: The patch generated 3 new + 340 unchanged - 1 fixed = 343 total (was
341) |
| +1 :green_heart: | mvnsite | 5m 7s | | the patch passed |
| +1 :green_heart: | xml | 0m 2s | | The patch has no ill-formed XML
file. |
| +1 :green_heart: | javadoc | 3m 51s | | the patch passed with JDK
Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 5m 9s | | the patch passed with JDK
Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07 |
| +1 :green_heart: | spotbugs | 10m 52s | | the patch passed |
| +1 :green_heart: | shadedclient | 24m 18s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 47s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 2m 29s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | unit | 347m 9s | | hadoop-hdfs in the patch
passed. |
| -1 :x: | unit | 34m 20s |
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4127/3/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt)
| hadoop-hdfs-rbf in the patch passed. |
| +1 :green_heart: | asflicense | 1m 55s | | The patch does not
generate ASF License warnings. |
| | | 658m 18s | | |
| Reason | Tests |
|---:|:--|
| Failed junit tests | hadoop.hdfs.rbfbalance.TestRouterDistCpProcedure |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4127/3/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/4127 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell xml |
| uname | Linux 87
[jira] [Updated] (HDFS-16529) Remove unnecessary setObserverRead in TestConsistentReadsObserver
[ https://issues.apache.org/jira/browse/HDFS-16529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HDFS-16529: - Component/s: (was: s) > Remove unnecessary setObserverRead in TestConsistentReadsObserver > - > > Key: HDFS-16529 > URL: https://issues.apache.org/jira/browse/HDFS-16529 > Project: Hadoop HDFS > Issue Type: Improvement > Components: test >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Trivial > Labels: pull-request-available > Fix For: 3.4.0, 2.10.2, 3.2.4, 3.3.3 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16529) Remove unnecessary setObserverRead in TestConsistentReadsObserver
[ https://issues.apache.org/jira/browse/HDFS-16529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated HDFS-16529: - Component/s: s test > Remove unnecessary setObserverRead in TestConsistentReadsObserver > - > > Key: HDFS-16529 > URL: https://issues.apache.org/jira/browse/HDFS-16529 > Project: Hadoop HDFS > Issue Type: Improvement > Components: s, test >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Trivial > Labels: pull-request-available > Fix For: 3.4.0, 2.10.2, 3.2.4, 3.3.3 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-16529) Remove unnecessary setObserverRead in TestConsistentReadsObserver
[ https://issues.apache.org/jira/browse/HDFS-16529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka reassigned HDFS-16529: Assignee: wangzhaohui > Remove unnecessary setObserverRead in TestConsistentReadsObserver > - > > Key: HDFS-16529 > URL: https://issues.apache.org/jira/browse/HDFS-16529 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Assignee: wangzhaohui >Priority: Trivial > Labels: pull-request-available > Fix For: 3.4.0, 2.10.2, 3.2.4, 3.3.3 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-16529) Remove unnecessary setObserverRead in TestConsistentReadsObserver
[ https://issues.apache.org/jira/browse/HDFS-16529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka resolved HDFS-16529. -- Fix Version/s: 3.4.0 2.10.2 3.2.4 3.3.3 Resolution: Fixed Committed to trunk, branch-3.3, branch-3.2, and branch-2.10. Thank you [~wangzhaohui] for your contribution! > Remove unnecessary setObserverRead in TestConsistentReadsObserver > - > > Key: HDFS-16529 > URL: https://issues.apache.org/jira/browse/HDFS-16529 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Priority: Trivial > Labels: pull-request-available > Fix For: 3.4.0, 2.10.2, 3.2.4, 3.3.3 > > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16529) Remove unnecessary setObserverRead in TestConsistentReadsObserver
[ https://issues.apache.org/jira/browse/HDFS-16529?focusedWorklogId=753248&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753248 ] ASF GitHub Bot logged work on HDFS-16529: - Author: ASF GitHub Bot Created on: 06/Apr/22 08:29 Start Date: 06/Apr/22 08:29 Worklog Time Spent: 10m Work Description: aajisaka merged PR #4131: URL: https://github.com/apache/hadoop/pull/4131 Issue Time Tracking --- Worklog Id: (was: 753248) Time Spent: 0.5h (was: 20m) > Remove unnecessary setObserverRead in TestConsistentReadsObserver > - > > Key: HDFS-16529 > URL: https://issues.apache.org/jira/browse/HDFS-16529 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Priority: Trivial > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16529) Remove unnecessary setObserverRead in TestConsistentReadsObserver
[ https://issues.apache.org/jira/browse/HDFS-16529?focusedWorklogId=753249&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753249 ] ASF GitHub Bot logged work on HDFS-16529: - Author: ASF GitHub Bot Created on: 06/Apr/22 08:29 Start Date: 06/Apr/22 08:29 Worklog Time Spent: 10m Work Description: aajisaka commented on PR #4131: URL: https://github.com/apache/hadoop/pull/4131#issuecomment-1089991975 Thank you @wzhallright and @tomscut Issue Time Tracking --- Worklog Id: (was: 753249) Time Spent: 40m (was: 0.5h) > Remove unnecessary setObserverRead in TestConsistentReadsObserver > - > > Key: HDFS-16529 > URL: https://issues.apache.org/jira/browse/HDFS-16529 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: wangzhaohui >Priority: Trivial > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16530) setReplication debug log creates a new string even if debug is disabled
[
https://issues.apache.org/jira/browse/HDFS-16530?focusedWorklogId=753235&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753235
]
ASF GitHub Bot logged work on HDFS-16530:
-
Author: ASF GitHub Bot
Created on: 06/Apr/22 07:49
Start Date: 06/Apr/22 07:49
Worklog Time Spent: 10m
Work Description: sodonnel merged PR #4142:
URL: https://github.com/apache/hadoop/pull/4142
Issue Time Tracking
---
Worklog Id: (was: 753235)
Time Spent: 40m (was: 0.5h)
> setReplication debug log creates a new string even if debug is disabled
> ---
>
> Key: HDFS-16530
> URL: https://issues.apache.org/jira/browse/HDFS-16530
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: Stephen O'Donnell
>Assignee: Stephen O'Donnell
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In FSDirAttrOp, HDFS-14521 made a good change to move a noisy logger to debug:
> {code}
> if (oldBR > targetReplication) {
> FSDirectory.LOG.debug("Decreasing replication from {} to {} for {}",
> oldBR, targetReplication, iip.getPath());
> } else if (oldBR < targetReplication) {
> FSDirectory.LOG.debug("Increasing replication from {} to {} for {}",
> oldBR, targetReplication, iip.getPath());
> } else {
> FSDirectory.LOG.debug("Replication remains unchanged at {} for {}",
> oldBR, iip.getPath());
> }
> }
> {code}
> However the `iip.getPath()` method must be evaluated to pass the resulting
> string into the LOG.debug method, even if debug is not enabled:
> This code may form a new string where it does not need to:
> {code}
> public String getPath() {
> if (pathname == null) {
> pathname = DFSUtil.byteArray2PathString(path);
> }
> return pathname;
> }
> {code}
> We should wrap the entire logging block in `if LOG.debugEnabled()` to avoid
> any overhead when the logger is not enabled.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16530) setReplication debug log creates a new string even if debug is disabled
[
https://issues.apache.org/jira/browse/HDFS-16530?focusedWorklogId=753234&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753234
]
ASF GitHub Bot logged work on HDFS-16530:
-
Author: ASF GitHub Bot
Created on: 06/Apr/22 07:49
Start Date: 06/Apr/22 07:49
Worklog Time Spent: 10m
Work Description: sodonnel commented on PR #4142:
URL: https://github.com/apache/hadoop/pull/4142#issuecomment-1089941308
TestSeveralNamenodes passes locally and couldn't be related to this, so I
will go ahead and commit this one. Thanks for the review @aajisaka
Issue Time Tracking
---
Worklog Id: (was: 753234)
Time Spent: 0.5h (was: 20m)
> setReplication debug log creates a new string even if debug is disabled
> ---
>
> Key: HDFS-16530
> URL: https://issues.apache.org/jira/browse/HDFS-16530
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Reporter: Stephen O'Donnell
>Assignee: Stephen O'Donnell
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> In FSDirAttrOp, HDFS-14521 made a good change to move a noisy logger to debug:
> {code}
> if (oldBR > targetReplication) {
> FSDirectory.LOG.debug("Decreasing replication from {} to {} for {}",
> oldBR, targetReplication, iip.getPath());
> } else if (oldBR < targetReplication) {
> FSDirectory.LOG.debug("Increasing replication from {} to {} for {}",
> oldBR, targetReplication, iip.getPath());
> } else {
> FSDirectory.LOG.debug("Replication remains unchanged at {} for {}",
> oldBR, iip.getPath());
> }
> }
> {code}
> However the `iip.getPath()` method must be evaluated to pass the resulting
> string into the LOG.debug method, even if debug is not enabled:
> This code may form a new string where it does not need to:
> {code}
> public String getPath() {
> if (pathname == null) {
> pathname = DFSUtil.byteArray2PathString(path);
> }
> return pathname;
> }
> {code}
> We should wrap the entire logging block in `if LOG.debugEnabled()` to avoid
> any overhead when the logger is not enabled.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16384) Upgrade Netty to 4.1.72.Final
[ https://issues.apache.org/jira/browse/HDFS-16384?focusedWorklogId=753229&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-753229 ] ASF GitHub Bot logged work on HDFS-16384: - Author: ASF GitHub Bot Created on: 06/Apr/22 07:23 Start Date: 06/Apr/22 07:23 Worklog Time Spent: 10m Work Description: jojochuang commented on code in PR #4132: URL: https://github.com/apache/hadoop/pull/4132#discussion_r843561634 ## LICENSE-binary: ## @@ -261,7 +261,7 @@ io.grpc:grpc-protobuf:1.26.0 io.grpc:grpc-protobuf-lite:1.26.0 io.grpc:grpc-stub:1.26.0 io.netty:netty:3.10.6.Final -io.netty:netty-all:4.1.42.Final +io.netty:netty-all:4.1.72.Final io.netty:netty-buffer:4.1.27.Final Review Comment: Thanks for review. Almost forgot. Issue Time Tracking --- Worklog Id: (was: 753229) Time Spent: 3.5h (was: 3h 20m) > Upgrade Netty to 4.1.72.Final > - > > Key: HDFS-16384 > URL: https://issues.apache.org/jira/browse/HDFS-16384 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs-client >Affects Versions: 3.3.1 >Reporter: Tamas Penzes >Assignee: Tamas Penzes >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 3.5h > Remaining Estimate: 0h > > New fixes for netty, nothing else changed, just netty version bumped and two > more exclusion in hdfs-client because of new netty. > No new tests added as not needed. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
