[jira] [Commented] (HDFS-13892) Disk Balancer : Invalid exit code for disk balancer execute command
[
https://issues.apache.org/jira/browse/HDFS-13892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621469#comment-16621469
]
Hadoop QA commented on HDFS-13892:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
27m 54s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 10s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 42m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:4b8c2b1 |
| JIRA Issue | HDFS-13892 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12940529/HDFS-13892-002.patch |
| Optional Tests | dupname asflicense mvnsite |
| uname | Linux 85c989732542 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10
07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 6b5838e |
| maven | version: Apache Maven 3.3.9 |
| whitespace |
https://builds.apache.org/job/PreCommit-HDFS-Build/25106/artifact/out/whitespace-eol.txt
|
| Max. process+thread count | 406 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/25106/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Disk Balancer : Invalid exit code for disk balancer execute command
> ---
>
> Key: HDFS-13892
> URL: https://issues.apache.org/jira/browse/HDFS-13892
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: diskbalancer, documentation
>Reporter: Harshakiran Reddy
>Assignee: Ranith Sardar
>Priority: Major
> Labels: newbie
> Attachments: HDFS-13892-002.patch, HDFS-13892.patch
>
>
> {{scenario:-}}
> 1. Write some 5GB data with one DISK
> 2. Add one more non-empty Disk to above Datanode
> 3.Run the plan command for the above specific datanode
> 4. run the Execute command with the above plan file
> the above execute command not happened as per the datanode log
> {noformat}
> ERROR org.apache.hadoop.hdfs.server.datanode.DiskBalancer: Destination
> volume: file:/Test_Disk/DISK2/ does not have enough space to accommodate a
> block. Block Size: 268435456 Exiting from copyBlocks.
> {noformat}
> 5. see the exit code for execute command, it display the 0
> {{Expected Result :-}}
> 1. Exit code should be 1 why means execution was not happened
> 2. In this type of scenario In console print the that error message that
> time customer/user knows execute was not happened.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13892) Disk Balancer : Invalid exit code for disk balancer execute command
[
https://issues.apache.org/jira/browse/HDFS-13892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621441#comment-16621441
]
Ranith Sardar commented on HDFS-13892:
--
Hi [~anu], I have updated the patch. Please have a look.
> Disk Balancer : Invalid exit code for disk balancer execute command
> ---
>
> Key: HDFS-13892
> URL: https://issues.apache.org/jira/browse/HDFS-13892
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: diskbalancer, documentation
>Reporter: Harshakiran Reddy
>Assignee: Ranith Sardar
>Priority: Major
> Labels: newbie
> Attachments: HDFS-13892-002.patch, HDFS-13892.patch
>
>
> {{scenario:-}}
> 1. Write some 5GB data with one DISK
> 2. Add one more non-empty Disk to above Datanode
> 3.Run the plan command for the above specific datanode
> 4. run the Execute command with the above plan file
> the above execute command not happened as per the datanode log
> {noformat}
> ERROR org.apache.hadoop.hdfs.server.datanode.DiskBalancer: Destination
> volume: file:/Test_Disk/DISK2/ does not have enough space to accommodate a
> block. Block Size: 268435456 Exiting from copyBlocks.
> {noformat}
> 5. see the exit code for execute command, it display the 0
> {{Expected Result :-}}
> 1. Exit code should be 1 why means execution was not happened
> 2. In this type of scenario In console print the that error message that
> time customer/user knows execute was not happened.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13892) Disk Balancer : Invalid exit code for disk balancer execute command
[
https://issues.apache.org/jira/browse/HDFS-13892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621047#comment-16621047
]
Anu Engineer commented on HDFS-13892:
-
[~RANith] Thanks for the patch. It has some changes in {{TestRouterAdminCLI}},
which I think accidentally came from your workspace. Could you please take a
look and update the patch if needed?
> Disk Balancer : Invalid exit code for disk balancer execute command
> ---
>
> Key: HDFS-13892
> URL: https://issues.apache.org/jira/browse/HDFS-13892
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: diskbalancer, documentation
>Reporter: Harshakiran Reddy
>Assignee: Ranith Sardar
>Priority: Major
> Labels: newbie
> Attachments: HDFS-13892.patch
>
>
> {{scenario:-}}
> 1. Write some 5GB data with one DISK
> 2. Add one more non-empty Disk to above Datanode
> 3.Run the plan command for the above specific datanode
> 4. run the Execute command with the above plan file
> the above execute command not happened as per the datanode log
> {noformat}
> ERROR org.apache.hadoop.hdfs.server.datanode.DiskBalancer: Destination
> volume: file:/Test_Disk/DISK2/ does not have enough space to accommodate a
> block. Block Size: 268435456 Exiting from copyBlocks.
> {noformat}
> 5. see the exit code for execute command, it display the 0
> {{Expected Result :-}}
> 1. Exit code should be 1 why means execution was not happened
> 2. In this type of scenario In console print the that error message that
> time customer/user knows execute was not happened.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13892) Disk Balancer : Invalid exit code for disk balancer execute command
[
https://issues.apache.org/jira/browse/HDFS-13892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16621038#comment-16621038
]
Ranith Sardar commented on HDFS-13892:
--
[~Harsha1206], thank you for raising the jira. Hi [~anu], I have uploaded the
patch, please check it once.
> Disk Balancer : Invalid exit code for disk balancer execute command
> ---
>
> Key: HDFS-13892
> URL: https://issues.apache.org/jira/browse/HDFS-13892
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: diskbalancer, documentation
>Reporter: Harshakiran Reddy
>Assignee: Ranith Sardar
>Priority: Major
> Labels: newbie
> Attachments: HDFS-13892.patch
>
>
> {{scenario:-}}
> 1. Write some 5GB data with one DISK
> 2. Add one more non-empty Disk to above Datanode
> 3.Run the plan command for the above specific datanode
> 4. run the Execute command with the above plan file
> the above execute command not happened as per the datanode log
> {noformat}
> ERROR org.apache.hadoop.hdfs.server.datanode.DiskBalancer: Destination
> volume: file:/Test_Disk/DISK2/ does not have enough space to accommodate a
> block. Block Size: 268435456 Exiting from copyBlocks.
> {noformat}
> 5. see the exit code for execute command, it display the 0
> {{Expected Result :-}}
> 1. Exit code should be 1 why means execution was not happened
> 2. In this type of scenario In console print the that error message that
> time customer/user knows execute was not happened.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13892) Disk Balancer : Invalid exit code for disk balancer execute command
[
https://issues.apache.org/jira/browse/HDFS-13892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620578#comment-16620578
]
Ranith Sardar commented on HDFS-13892:
--
Hi, I would like to give a patch for this issue.
> Disk Balancer : Invalid exit code for disk balancer execute command
> ---
>
> Key: HDFS-13892
> URL: https://issues.apache.org/jira/browse/HDFS-13892
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: diskbalancer, documentation
>Reporter: Harshakiran Reddy
>Assignee: Ranith Sardar
>Priority: Major
> Labels: newbie
>
> {{scenario:-}}
> 1. Write some 5GB data with one DISK
> 2. Add one more non-empty Disk to above Datanode
> 3.Run the plan command for the above specific datanode
> 4. run the Execute command with the above plan file
> the above execute command not happened as per the datanode log
> {noformat}
> ERROR org.apache.hadoop.hdfs.server.datanode.DiskBalancer: Destination
> volume: file:/Test_Disk/DISK2/ does not have enough space to accommodate a
> block. Block Size: 268435456 Exiting from copyBlocks.
> {noformat}
> 5. see the exit code for execute command, it display the 0
> {{Expected Result :-}}
> 1. Exit code should be 1 why means execution was not happened
> 2. In this type of scenario In console print the that error message that
> time customer/user knows execute was not happened.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13892) Disk Balancer : Invalid exit code for disk balancer execute command
[
https://issues.apache.org/jira/browse/HDFS-13892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16620577#comment-16620577
]
Ranith Sardar commented on HDFS-13892:
--
Hi, I would like to give a patch for this issue.
> Disk Balancer : Invalid exit code for disk balancer execute command
> ---
>
> Key: HDFS-13892
> URL: https://issues.apache.org/jira/browse/HDFS-13892
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: diskbalancer, documentation
>Reporter: Harshakiran Reddy
>Assignee: Ranith Sardar
>Priority: Major
> Labels: newbie
>
> {{scenario:-}}
> 1. Write some 5GB data with one DISK
> 2. Add one more non-empty Disk to above Datanode
> 3.Run the plan command for the above specific datanode
> 4. run the Execute command with the above plan file
> the above execute command not happened as per the datanode log
> {noformat}
> ERROR org.apache.hadoop.hdfs.server.datanode.DiskBalancer: Destination
> volume: file:/Test_Disk/DISK2/ does not have enough space to accommodate a
> block. Block Size: 268435456 Exiting from copyBlocks.
> {noformat}
> 5. see the exit code for execute command, it display the 0
> {{Expected Result :-}}
> 1. Exit code should be 1 why means execution was not happened
> 2. In this type of scenario In console print the that error message that
> time customer/user knows execute was not happened.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13892) Disk Balancer : Invalid exit code for disk balancer execute command
[
https://issues.apache.org/jira/browse/HDFS-13892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16605352#comment-16605352
]
Harshakiran Reddy commented on HDFS-13892:
--
[~anu] Thanks for watching this issue and Good Explanation about Execute
command.
bq.please feel free to convert this Jira to a doc Jira, and one of us can get
this fixed.
Okay.
> Disk Balancer : Invalid exit code for disk balancer execute command
> ---
>
> Key: HDFS-13892
> URL: https://issues.apache.org/jira/browse/HDFS-13892
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: diskbalancer, documentation
>Reporter: Harshakiran Reddy
>Priority: Major
> Labels: newbie
>
> {{scenario:-}}
> 1. Write some 5GB data with one DISK
> 2. Add one more non-empty Disk to above Datanode
> 3.Run the plan command for the above specific datanode
> 4. run the Execute command with the above plan file
> the above execute command not happened as per the datanode log
> {noformat}
> ERROR org.apache.hadoop.hdfs.server.datanode.DiskBalancer: Destination
> volume: file:/Test_Disk/DISK2/ does not have enough space to accommodate a
> block. Block Size: 268435456 Exiting from copyBlocks.
> {noformat}
> 5. see the exit code for execute command, it display the 0
> {{Expected Result :-}}
> 1. Exit code should be 1 why means execution was not happened
> 2. In this type of scenario In console print the that error message that
> time customer/user knows execute was not happened.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13892) Disk Balancer : Invalid exit code for disk balancer execute command
[
https://issues.apache.org/jira/browse/HDFS-13892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16603237#comment-16603237
]
Anu Engineer commented on HDFS-13892:
-
Hi [~Harsha1206], Thank you for taking time to file this JIRA. I appreciate
it. From a high level, I do agree with your thought process. In an ideal world,
the execute should have returned an error code. However, disk balancer works
slightly differently, understanding what happens under the covers might help
you realize what is happening, though you may not agree with the behavior.
When DiskBalancer executes the plan, it is the beginning of an asynchronous
process that can take a long time. So execute command directly hands over the
disk balancer plan file to the data node and data node executes the plan.
Hence a return code from execute indicates merely that the command was
successful in handing over the plan file to data node to perform balancing at a
later point in time.
DiskBalancer also supports a command called query, that will give you
information about the current status, a failure during runtime will be
reflected in the query command, and as you rightly discovered in the data node
logs.
if you believe that Apache Hadoop user documentation does adequately explain
this process, please feel free to convert this Jira to a doc Jira, and one of
us can get this fixed.
if you would like to pursue the architecture of Disk balancer, there are two
documents – the proposal and the architecture and test plan in this Jira
https://issues.apache.org/jira/browse/HDFS-1312 which explains how disk
balancer works in great detail.
Thank you very much for taking time out to file the Jira and bringing this
issue to my attention.
> Disk Balancer : Invalid exit code for disk balancer execute command
> ---
>
> Key: HDFS-13892
> URL: https://issues.apache.org/jira/browse/HDFS-13892
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: diskbalancer
>Reporter: Harshakiran Reddy
>Priority: Major
>
> {{scenario:-}}
> 1. Write some 5GB data with one DISK
> 2. Add one more non-empty Disk to above Datanode
> 3.Run the plan command for the above specific datanode
> 4. run the Execute command with the above plan file
> the above execute command not happened as per the datanode log
> {noformat}
> ERROR org.apache.hadoop.hdfs.server.datanode.DiskBalancer: Destination
> volume: file:/Test_Disk/DISK2/ does not have enough space to accommodate a
> block. Block Size: 268435456 Exiting from copyBlocks.
> {noformat}
> 5. see the exit code for execute command, it display the 0
> {{Expected Result :-}}
> 1. Exit code should be 1 why means execution was not happened
> 2. In this type of scenario In console print the that error message that
> time customer/user knows execute was not happened.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
