[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[
https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403272#comment-16403272
]
Wei Yan commented on HDFS-13215:
{quote}I think the error for {{dfs.namenode.fs-limits.min-block-size}} has to
do with the fact that {{hdfs-site.xml}} has that value set to 0 which allows
any value.
{quote}
I add a separate hdfs-site.xml to hadoop-hdfs-rbf, like what hadoop-hdfs-httfs
does.
Upload a new patch [^HDFS-13215.007.patch]. I move webapps to hadoop-hdfs-rbf,
and verified it in a dev env, looks ok.
> RBF: Move Router to its own module
> --
>
> Key: HDFS-13215
> URL: https://issues.apache.org/jira/browse/HDFS-13215
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Íñigo Goiri
>Assignee: Wei Yan
>Priority: Major
> Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch,
> HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch,
> HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch
>
>
> We are splitting the HDFS client code base and potentially Router-based
> Federation is also independent enough to be in its own package.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated HDFS-13215: --- Attachment: HDFS-13215.007.patch > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch, HDFS-13215.007.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13304) Document: update the new ozone docker file location
[ https://issues.apache.org/jira/browse/HDFS-13304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-13304: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7240 Status: Resolved (was: Patch Available) [~xyao] Thanks for the contribution. I have committed this to the feature branch. > Document: update the new ozone docker file location > > > Key: HDFS-13304 > URL: https://issues.apache.org/jira/browse/HDFS-13304 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: documentation >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-13304-HDFS-7240.001.patch > > > The docker compose file has been moved from dev-support/compose/ozone to > hadoop-dist/target/compose/ozone, we need to update the document in > OzoneGettingStarted.md.vm. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12884) BlockUnderConstructionFeature.truncateBlock should be of type BlockInfo
[
https://issues.apache.org/jira/browse/HDFS-12884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403264#comment-16403264
]
genericqa commented on HDFS-12884:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 2s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 51s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 2 new + 227 unchanged - 0 fixed = 229 total (was 227) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 14s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
9s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0
unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}105m 31s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}160m 21s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs-project/hadoop-hdfs |
| | instanceof will always return true for all non-null values in
org.apache.hadoop.hdfs.server.namenode.INode$BlocksMapUpdateInfo.addDeleteBlock(BlockInfo),
since all org.apache.hadoop.hdfs.server.blockmanagement.BlockInfo are
instances of org.apache.hadoop.hdfs.server.blockmanagement.BlockInfo At
INode.java:for all non-null values in
org.apache.hadoop.hdfs.server.namenode.INode$BlocksMapUpdateInfo.addDeleteBlock(BlockInfo),
since all org.apache.hadoop.hdfs.server.blockmanagement.BlockInfo are
instances of org.apache.hadoop.hdfs.server.blockmanagement.BlockInfo At
INode.java:[line 1061] |
| Failed junit tests | hadoop.hdfs.TestAppendSnapshotTruncate |
| | hadoop.cli.TestHDFSCLI |
| | hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency |
| | hadoop.hdfs.TestReadStripedFileWithMissingBlocks |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy |
| | hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped |
|
[jira] [Updated] (HDFS-12884) BlockUnderConstructionFeature.truncateBlock should be of type BlockInfo
[
https://issues.apache.org/jira/browse/HDFS-12884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chencan updated HDFS-12884:
---
Status: Patch Available (was: Open)
Hi [~shv], I've submitted the first patch. Please help me see if it's what you
expect. Thanks.
> BlockUnderConstructionFeature.truncateBlock should be of type BlockInfo
> ---
>
> Key: HDFS-12884
> URL: https://issues.apache.org/jira/browse/HDFS-12884
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 2.7.4
>Reporter: Konstantin Shvachko
>Priority: Major
> Attachments: HDFS-12884.001.patch
>
>
> {{BlockUnderConstructionFeature.truncateBlock}} type should be changed to
> {{BlockInfo}} from {{Block}}. {{truncateBlock}} is always assigned as
> {{BlockInfo}}, so this will avoid unnecessary casts.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12884) BlockUnderConstructionFeature.truncateBlock should be of type BlockInfo
[
https://issues.apache.org/jira/browse/HDFS-12884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chencan updated HDFS-12884:
---
Attachment: HDFS-12884.001.patch
> BlockUnderConstructionFeature.truncateBlock should be of type BlockInfo
> ---
>
> Key: HDFS-12884
> URL: https://issues.apache.org/jira/browse/HDFS-12884
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
>Affects Versions: 2.7.4
>Reporter: Konstantin Shvachko
>Priority: Major
> Attachments: HDFS-12884.001.patch
>
>
> {{BlockUnderConstructionFeature.truncateBlock}} type should be changed to
> {{BlockInfo}} from {{Block}}. {{truncateBlock}} is always assigned as
> {{BlockInfo}}, so this will avoid unnecessary casts.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[
https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403209#comment-16403209
]
Íñigo Goiri commented on HDFS-13215:
I think the error for {{dfs.namenode.fs-limits.min-block-size}} has to do with
the fact that {{hdfs-site.xml}} has that value set to 0 which allows any value.
We need to figure how YARN takes the yarn-site.xml for all the subprojects.
> RBF: Move Router to its own module
> --
>
> Key: HDFS-13215
> URL: https://issues.apache.org/jira/browse/HDFS-13215
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Íñigo Goiri
>Assignee: Wei Yan
>Priority: Major
> Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch,
> HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch,
> HDFS-13215.005.patch, HDFS-13215.006.patch
>
>
> We are splitting the HDFS client code base and potentially Router-based
> Federation is also independent enough to be in its own package.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12977) Add stateId to RPC headers.
[
https://issues.apache.org/jira/browse/HDFS-12977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403177#comment-16403177
]
Konstantin Shvachko commented on HDFS-12977:
Great! One last thing. Out of those [checkstyle
warnings|https://builds.apache.org/job/PreCommit-HDFS-Build/23527/artifact/out/diff-checkstyle-root.txt]
from last build, the last one actually worth fixing:
{code:java}
./hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java:219:
volatile long lastSeenStateId;:17: Variable 'lastSeenStateId' must be private
and have accessor methods. [VisibilityModifier]{code}
> Add stateId to RPC headers.
> ---
>
> Key: HDFS-12977
> URL: https://issues.apache.org/jira/browse/HDFS-12977
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ipc, namenode
>Reporter: Konstantin Shvachko
>Assignee: Plamen Jeliazkov
>Priority: Major
> Attachments: HDFS_12977.trunk.001.patch, HDFS_12977.trunk.002.patch,
> HDFS_12977.trunk.003.patch, HDFS_12977.trunk.004.patch,
> HDFS_12977.trunk.005.patch, HDFS_12977.trunk.006.patch,
> HDFS_12977.trunk.007.patch, HDFS_12977.trunk.008.patch
>
>
> stateId is a new field in the RPC headers of NameNode proto calls.
> stateId is the journal transaction Id, which represents LastSeenId for the
> clients and LastWrittenId for NameNodes. See more in [reads from Standby
> design
> doc|https://issues.apache.org/jira/secure/attachment/12902925/ConsistentReadsFromStandbyNode.pdf].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-11337) [branch-2] Add instrumentation hooks around Datanode disk IO
[ https://issues.apache.org/jira/browse/HDFS-11337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] maobaolong updated HDFS-11337: -- Description: +underlined text+Cloned from HDFS-10958 to verify the branch-2 backport. (was: Cloned from HDFS-10958 to verify the branch-2 backport.) > [branch-2] Add instrumentation hooks around Datanode disk IO > > > Key: HDFS-11337 > URL: https://issues.apache.org/jira/browse/HDFS-11337 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Reporter: Xiaoyu Yao >Assignee: Arpit Agarwal >Priority: Major > Attachments: HDFS-11337-branch-2.01.patch, > HDFS-11337-branch-2.02.patch, HDFS-11337-branch-2.03.patch, > HDFS-11337-branch-2.04.patch, HDFS-11337-branch-2.05.patch, > HDFS-11337-branch-2.06.patch > > > +underlined text+Cloned from HDFS-10958 to verify the branch-2 backport. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[
https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403169#comment-16403169
]
Íñigo Goiri commented on HDFS-13215:
{quote}
The three testcases passed in my local env. You miss some patches?
{quote}
I was testing trunk with [^HDFS-13215.006.patch] on top.
{code}
[ERROR] Errors:
[ERROR] TestRouterAdminCLI.testCreateInvalidEntry:444 » NullPointerException
(this is probably something with Windows)
[ERROR] TestRouterQuota.testGetQuota:314 » Remote Specified block size is
less than configured minimum value (dfs.namenode.fs-limits.min-block-size): 512
< 1048576
[ERROR] TestRouterQuota.testQuotaUpdating:418 » Remote Specified block size
is less than configured minimum value (dfs.namenode.fs-limits.min-block-size):
512 < 1048576
[ERROR] TestRouterQuota.testStorageSpaceQuotaaExceed:181 » Remote Specified
block size is less than configured minimum value
(dfs.namenode.fs-limits.min-block-size): 512 < 1048576
[ERROR] TestRouterRpc.testErasureCoding:985 » Remote
org.apache.hadoop.ipc.RemoteException: NPE
[ERROR] TestRouterRpc.testProxyGetAdditionalDatanode:854 » Remote Specified
block size is less than configured minimum value
(dfs.namenode.fs-limits.min-block-size): 512 < 1048576
[ERROR] TestRouterRpcMultiDestination>TestRouterRpc.testErasureCoding:985 »
Specified block size is less than configured minimum value
(dfs.namenode.fs-limits.min-block-size): 1024 < 1048576
[ERROR]
TestRouterRpcMultiDestination>TestRouterRpc.testProxyGetAdditionalDatanode:854
» Specified block size is less than configured minimum value
(dfs.namenode.fs-limits.min-block-size): 1024 < 1048576
{code}
So it looks like the main issue is this dfs.namenode.fs-limits.min-block-size.
Maybe is taking the wrong config files?
> RBF: Move Router to its own module
> --
>
> Key: HDFS-13215
> URL: https://issues.apache.org/jira/browse/HDFS-13215
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Íñigo Goiri
>Assignee: Wei Yan
>Priority: Major
> Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch,
> HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch,
> HDFS-13215.005.patch, HDFS-13215.006.patch
>
>
> We are splitting the HDFS client code base and potentially Router-based
> Federation is also independent enough to be in its own package.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403153#comment-16403153 ] Wei Yan commented on HDFS-13215: The three testcases passed in my local env. You miss some patches? I'll move the webapps out. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12977) Add stateId to RPC headers.
[
https://issues.apache.org/jira/browse/HDFS-12977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403145#comment-16403145
]
genericqa commented on HDFS-12977:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 49s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
32s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 13m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 13m
3s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
3m 14s{color} | {color:orange} root: The patch generated 4 new + 670 unchanged
- 0 fixed = 674 total (was 670) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 6s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
58s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 46s{color}
| {color:red} hadoop-hdfs-client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 46s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}109m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-12977 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12914950/HDFS_12977.trunk.008.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle cc |
| uname | Linux 93522652dfaa 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403144#comment-16403144 ] Íñigo Goiri commented on HDFS-13215: We probably should move the webapps too. We need to check carefully what does the final dist do with it and if the references to the static HDFs js and css are still correct. There is also things like proto-web.xml. I'm also running the unit tests and I'm getting errors with: * TestRouterQuota * TestRouterRpc * TestRouterRpcMultiDestination > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403141#comment-16403141 ] Wei Yan commented on HDFS-13215: Good catch, [~elgoiri]. There is another dir: hadoop-hdfs/src/main/webapps/router, not sure whether we should move it out or just leave it there. I'm thinking, in the future, if we plan to separate hdfs components just like yarn, we could leave it there and seperate to another module (like hadoop-yarn-ui) which includes all HDFS UI related. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13296) GenericTestUtils#getTempPath and GenericTestUtils#getRandomizedTestDir generate paths with drive letter in Windows, and fail webhdfs related test cases
[ https://issues.apache.org/jira/browse/HDFS-13296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403140#comment-16403140 ] Íñigo Goiri commented on HDFS-13296: We went ahead and tested YARN, HDFS, Commons and tools. HDFS reduced by ~40 as I mentioned before and the rest are still as they use to be. We would like to put this into branch-2.9; should we commit to all of them (i.e., trunk, branch-3.1, branch-3.0, branch-2, branch-2.9)? Anyway, I think this is ready for review. +1 from me but as I was involved in the testing I'd like others to chime in. > GenericTestUtils#getTempPath and GenericTestUtils#getRandomizedTestDir > generate paths with drive letter in Windows, and fail webhdfs related test > cases > --- > > Key: HDFS-13296 > URL: https://issues.apache.org/jira/browse/HDFS-13296 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Xiao Liang >Assignee: Xiao Liang >Priority: Major > Labels: windows > Attachments: HDFS-13296.000.patch > > > In GenericTestUtils#getRandomizedTestDir, getAbsoluteFile is called and will > added drive letter to the path in windows, some test cases use the generated > path to send webhdfs request, which will fail due to the drive letter in the > URI like: "webhdfs://127.0.0.1:18334/D:/target/test/data/vUqZkOrBZa/test" > GenericTestUtils#getTempPath has the similar issue in Windows. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13304) Document: update the new ozone docker file location
[
https://issues.apache.org/jira/browse/HDFS-13304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403130#comment-16403130
]
genericqa commented on HDFS-13304:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} HDFS-7240 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
49s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
29m 37s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 31s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
21s{color} | {color:red} The patch generated 15 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 42m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:215a942 |
| JIRA Issue | HDFS-13304 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12914954/HDFS-13304-HDFS-7240.001.patch
|
| Optional Tests | asflicense mvnsite |
| uname | Linux d2ef314b9dcf 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | HDFS-7240 / fb9ba12 |
| maven | version: Apache Maven 3.3.9 |
| asflicense |
https://builds.apache.org/job/PreCommit-HDFS-Build/23528/artifact/out/patch-asflicense-problems.txt
|
| Max. process+thread count | 301 (vs. ulimit of 1) |
| modules | C: hadoop-ozone/ozone-manager U: hadoop-ozone/ozone-manager |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/23528/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Document: update the new ozone docker file location
>
>
> Key: HDFS-13304
> URL: https://issues.apache.org/jira/browse/HDFS-13304
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: documentation
>Reporter: Xiaoyu Yao
>Assignee: Xiaoyu Yao
>Priority: Major
> Attachments: HDFS-13304-HDFS-7240.001.patch
>
>
> The docker compose file has been moved from dev-support/compose/ozone to
> hadoop-dist/target/compose/ozone, we need to update the document in
> OzoneGettingStarted.md.vm.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403129#comment-16403129 ] Íñigo Goiri commented on HDFS-13215: We also need to add the hdfs-rb-default.xml to the pom.xml to copy it at the end. Is there anything we need to do for the documentation? > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13295) Namenode doesn't leave safemode if dfs.namenode.safemode.replication.min set < dfs.namenode.replication.min
[ https://issues.apache.org/jira/browse/HDFS-13295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403125#comment-16403125 ] Bharat Viswanadham edited comment on HDFS-13295 at 3/16/18 11:45 PM: - Hi [~nfraison.criteo] One question I have from the description, have you faced this issue on the cluster? Because when loading from edit logs, in blockInfo object storage will not be set, this will be set when block reports are received. So, I think we will not hit the issue. I have tried setting dfs.namenode.safemode.replication.min set < dfs.namenode.replication.min and then restarted my test cluster, not faced this issue. Let me know if i am missing something here? was (Author: bharatviswa): Hi [~nfraison.criteo] One question I have from the description, have you faced this issue on the cluster? Because when loading from edit logs, in blockInfo object storage will not be set, this will be set when block reports are received. So, I think we will not hit the issue. Let me know if i am missing something here? > Namenode doesn't leave safemode if dfs.namenode.safemode.replication.min set > < dfs.namenode.replication.min > --- > > Key: HDFS-13295 > URL: https://issues.apache.org/jira/browse/HDFS-13295 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode > Environment: CDH 5.11 with HDFS-8716 backported. > dfs.namenode.replication.min=2 > dfs.namenode.safemode.replication.min=1 > >Reporter: Nicolas Fraison >Assignee: Nicolas Fraison >Priority: Major > Attachments: HDFS-13295.patch > > > When we set dfs.namenode.safemode.replication.min < > dfs.namenode.replication.min from HDFS-8716 patch the number of replica for > which it will increase the safe block count > must be equal to dfs.namenode.safemode.replication.min in > `FSNamesystem.incrementSafeBlockCount` > When reading modification from edits, the replica number for new blocks is > set at min(numNodes, > dfs.namenode.replication.min) in BlockManager.completeBlock which is greater > than dfs.namenode.safemode.replication.min. > Due to that safe block count never reach number of available blocks and > namenode doesn't leave automatically the safemode -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13295) Namenode doesn't leave safemode if dfs.namenode.safemode.replication.min set < dfs.namenode.replication.min
[ https://issues.apache.org/jira/browse/HDFS-13295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403125#comment-16403125 ] Bharat Viswanadham commented on HDFS-13295: --- Hi [~nfraison.criteo] One question I have from the description, have you faced this issue on the cluster? Because when loading from edit logs, in blockInfo object storage will not be set, this will be set when block reports are received. So, I think we will not hit the issue. Let me know if i am missing something here? > Namenode doesn't leave safemode if dfs.namenode.safemode.replication.min set > < dfs.namenode.replication.min > --- > > Key: HDFS-13295 > URL: https://issues.apache.org/jira/browse/HDFS-13295 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode > Environment: CDH 5.11 with HDFS-8716 backported. > dfs.namenode.replication.min=2 > dfs.namenode.safemode.replication.min=1 > >Reporter: Nicolas Fraison >Assignee: Nicolas Fraison >Priority: Major > Attachments: HDFS-13295.patch > > > When we set dfs.namenode.safemode.replication.min < > dfs.namenode.replication.min from HDFS-8716 patch the number of replica for > which it will increase the safe block count > must be equal to dfs.namenode.safemode.replication.min in > `FSNamesystem.incrementSafeBlockCount` > When reading modification from edits, the replica number for new blocks is > set at min(numNodes, > dfs.namenode.replication.min) in BlockManager.completeBlock which is greater > than dfs.namenode.safemode.replication.min. > Due to that safe block count never reach number of available blocks and > namenode doesn't leave automatically the safemode -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403123#comment-16403123 ] Íñigo Goiri commented on HDFS-13215: This is not very promising with Yetus. I'll run locally, meanwhile we are missing in findbugsExcludeFile.xml: We need to remove it from the old one too. You may want to do a grep for router or federation. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13296) GenericTestUtils#getTempPath and GenericTestUtils#getRandomizedTestDir generate paths with drive letter in Windows, and fail webhdfs related test cases
[ https://issues.apache.org/jira/browse/HDFS-13296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403121#comment-16403121 ] Chris Douglas commented on HDFS-13296: -- bq. Chris Douglas, Subru Krishnan, any idea on how we can trigger HDFS unit tests and not only Commons? Sorry, not that I know of. Finding unit tests that use this utility class and running those manually is probably a sufficient spot check. We can fix regressions if they come up. > GenericTestUtils#getTempPath and GenericTestUtils#getRandomizedTestDir > generate paths with drive letter in Windows, and fail webhdfs related test > cases > --- > > Key: HDFS-13296 > URL: https://issues.apache.org/jira/browse/HDFS-13296 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Xiao Liang >Assignee: Xiao Liang >Priority: Major > Labels: windows > Attachments: HDFS-13296.000.patch > > > In GenericTestUtils#getRandomizedTestDir, getAbsoluteFile is called and will > added drive letter to the path in windows, some test cases use the generated > path to send webhdfs request, which will fail due to the drive letter in the > URI like: "webhdfs://127.0.0.1:18334/D:/target/test/data/vUqZkOrBZa/test" > GenericTestUtils#getTempPath has the similar issue in Windows. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-10419) Building HDFS on top of new storage layer (HDSL)
[
https://issues.apache.org/jira/browse/HDFS-10419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403116#comment-16403116
]
Sanjay Radia commented on HDFS-10419:
-
In the " [VOTE] Merging branch HDFS-7240 to trunk" thread [~andrew.wang] asked:
{quote}*Sanjay says*:
>- NN on top HDSL where the NN uses the new block layer (Both Daryn and Owen
>acknowledge the >benefit of the >>new block layer). We have two choices here
>** a) Evolve NN so that it can interact with both old and new block layer,
>** b) Fork and create new NN that works only with new block layer, the old
NN will continue to work with old >>block layer.
>There are trade-offs but clearly the 2nd option has least impact on the old
>HDFS code.
*Andrew asks*: Are you proposing that we pursue the 2nd option to integrate
HDSL with HDFS?
{quote}
Originally I would have preferred (a), but Owen made a strong case for (b) in
my discussions with his last week. I believe approach (a) or (b) will depend
strongly on what we want to do. For example if we do milestone-1 and get the 2x
scalability and decide to stop there then clearly go with option (a) - it will
require little refactoring and one can run old and new HDFS side-by-side. If
you are planning to follow up milestone-1 with say the caching the working set
of the namespace, then forking the NN code (ie option b) might be better, and
the new NN will have to keep pulling over features and bug fixes from the old
NN.. Konstantine has proposed other alternatives and we would evaluate (a) or
(b) for his alternative. I am not locked into any particular path or how we
would do it.
> Building HDFS on top of new storage layer (HDSL)
>
>
> Key: HDFS-10419
> URL: https://issues.apache.org/jira/browse/HDFS-10419
> Project: Hadoop HDFS
> Issue Type: New Feature
>Reporter: Jing Zhao
>Assignee: Jing Zhao
>Priority: Major
> Attachments: Evolving NN using new block-container layer.pdf
>
>
> In HDFS-7240, Ozone defines storage containers to store both the data and the
> metadata. The storage container layer provides an object storage interface
> and aims to manage data/metadata in a distributed manner. More details about
> storage containers can be found in the design doc in HDFS-7240.
> HDFS can adopt the storage containers to store and manage blocks. The general
> idea is:
> # Each block can be treated as an object and the block ID is the object's key.
> # Blocks will still be stored in DataNodes but as objects in storage
> containers.
> # The block management work can be separated out of the NameNode and will be
> handled by the storage container layer in a more distributed way. The
> NameNode will only manage the namespace (i.e., files and directories).
> # For each file, the NameNode only needs to record a list of block IDs which
> are used as keys to obtain real data from storage containers.
> # A new DFSClient implementation talks to both NameNode and the storage
> container layer to read/write.
> HDFS, especially the NameNode, can get much better scalability from this
> design. Currently the NameNode's heaviest workload comes from the block
> management, which includes maintaining the block-DataNode mapping, receiving
> full/incremental block reports, tracking block states (under/over/miss
> replicated), and joining every writing pipeline protocol to guarantee the
> data consistency. These work bring high memory footprint and make NameNode
> suffer from GC. HDFS-5477 already proposes to convert BlockManager as a
> service. If we can build HDFS on top of the storage container layer, we not
> only separate out the BlockManager from the NameNode, but also replace it
> with a new distributed management scheme.
> The storage container work is currently in progress in HDFS-7240, and the
> work proposed here is still in an experimental/exploring stage. We can do
> this experiment in a feature branch so that people with interests can be
> involved.
> A design doc will be uploaded later explaining more details.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403111#comment-16403111 ] Wei Yan commented on HDFS-13215: Strange Yetus didn't run any HDFS testcases.. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11043) TestWebHdfsTimeouts fails
[
https://issues.apache.org/jira/browse/HDFS-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403109#comment-16403109
]
genericqa commented on HDFS-11043:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 39s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 51s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}114m 29s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}179m 50s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
| | hadoop.hdfs.TestDFSClientRetries |
| | hadoop.hdfs.server.datanode.TestDataNodeUUID |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-11043 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12914923/HDFS-11043.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 54545fde12d9 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 08ff158 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/23524/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/23524/testReport/ |
| Max. process+thread count | 2438 (vs. ulimit of 1) |
| modules | C:
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[
https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403107#comment-16403107
]
genericqa commented on HDFS-13215:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
34s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 77 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 10m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 33s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-hdfs-project . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
31s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 12m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 12m 26s{color}
| {color:red} root generated 161 new + unchanged - 161 fixed = 1272 total
(was 1272) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 50s{color} | {color:orange} root: The patch generated 3 new + 431 unchanged
- 3 fixed = 434 total (was 434) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 10m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
11s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
8s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 30s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-hdfs-project . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 16m 6s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}141m 53s{color} |
{color:black} {color} |
\\
[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode
[
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403085#comment-16403085
]
Virajith Jalaparti commented on HDFS-13283:
---
Thanks for changing the enums to class [~lukmajercak]. I am ok with having the
various implementations in a single file, given they are small.
> Percentage based Reserved Space Calculation for DataNode
>
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be
> an issue in non-heterogeneous clusters where size of DNs can differ. The
> proposed solution is to allow percentage based configuration (and their
> combination):
> # ABSOLUTE
> ** based on absolute number of reserved space
> # PERCENTAGE
> ** based on percentage of total capacity in the storage
> # CONSERVATIVE
> ** calculates both of the above and takes the one that will yield more
> reserved space
> # AGGRESSIVE
> ** calculates 1. 2. and takes the one that will yield less reserved space
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13304) Document: update the new ozone docker file location
[ https://issues.apache.org/jira/browse/HDFS-13304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-13304: -- Status: Patch Available (was: Open) > Document: update the new ozone docker file location > > > Key: HDFS-13304 > URL: https://issues.apache.org/jira/browse/HDFS-13304 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: documentation >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao >Priority: Major > Attachments: HDFS-13304-HDFS-7240.001.patch > > > The docker compose file has been moved from dev-support/compose/ozone to > hadoop-dist/target/compose/ozone, we need to update the document in > OzoneGettingStarted.md.vm. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13304) Document: update the new ozone docker file location
[ https://issues.apache.org/jira/browse/HDFS-13304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16403066#comment-16403066 ] Anu Engineer commented on HDFS-13304: - +1. Thanks for updating the document. cc: [~elek] . I will commit this now. > Document: update the new ozone docker file location > > > Key: HDFS-13304 > URL: https://issues.apache.org/jira/browse/HDFS-13304 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: documentation >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao >Priority: Major > Attachments: HDFS-13304-HDFS-7240.001.patch > > > The docker compose file has been moved from dev-support/compose/ozone to > hadoop-dist/target/compose/ozone, we need to update the document in > OzoneGettingStarted.md.vm. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13305) Minor improvements to xml image output
Arpit Agarwal created HDFS-13305: Summary: Minor improvements to xml image output Key: HDFS-13305 URL: https://issues.apache.org/jira/browse/HDFS-13305 Project: Hadoop HDFS Issue Type: Improvement Components: tools Reporter: Arpit Agarwal Assignee: Arpit Agarwal Minor changes to xml image output in the snapshot diff and inode reference sections. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13305) Minor improvements to xml image output
[ https://issues.apache.org/jira/browse/HDFS-13305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-13305: - Attachment: (was: HDFS-13305.01.patch) > Minor improvements to xml image output > -- > > Key: HDFS-13305 > URL: https://issues.apache.org/jira/browse/HDFS-13305 > Project: Hadoop HDFS > Issue Type: Improvement > Components: tools >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal >Priority: Major > > Minor changes to xml image output in the snapshot diff and inode reference > sections. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13305) Minor improvements to xml image output
[ https://issues.apache.org/jira/browse/HDFS-13305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-13305: - Attachment: HDFS-13305.01.patch > Minor improvements to xml image output > -- > > Key: HDFS-13305 > URL: https://issues.apache.org/jira/browse/HDFS-13305 > Project: Hadoop HDFS > Issue Type: Improvement > Components: tools >Reporter: Arpit Agarwal >Assignee: Arpit Agarwal >Priority: Major > > Minor changes to xml image output in the snapshot diff and inode reference > sections. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13304) Document: update the new ozone docker file location
[ https://issues.apache.org/jira/browse/HDFS-13304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-13304: -- Attachment: HDFS-13304-HDFS-7240.001.patch > Document: update the new ozone docker file location > > > Key: HDFS-13304 > URL: https://issues.apache.org/jira/browse/HDFS-13304 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: documentation >Reporter: Xiaoyu Yao >Assignee: Xiaoyu Yao >Priority: Major > Attachments: HDFS-13304-HDFS-7240.001.patch > > > The docker compose file has been moved from dev-support/compose/ozone to > hadoop-dist/target/compose/ozone, we need to update the document in > OzoneGettingStarted.md.vm. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode
[
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402974#comment-16402974
]
genericqa commented on HDFS-13283:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 15s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 59s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 15 new + 449 unchanged - 0 fixed = 464 total (was 449) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 48s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 80m 39s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}140m 2s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.TestSafeModeWithStripedFile |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-13283 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12914924/HDFS-13283.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle xml |
| uname | Linux d4c2f387deda 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 08ff158 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/23523/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit |
[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode
[
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402646#comment-16402646
]
Lukas Majercak commented on HDFS-13283:
---
Considering the implementation is minimal, I'd rather keep them all in the same
file. What do you think [~virajith]? I'll wait for yetus to come with
checkstyle before making changes
> Percentage based Reserved Space Calculation for DataNode
>
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be
> an issue in non-heterogeneous clusters where size of DNs can differ. The
> proposed solution is to allow percentage based configuration (and their
> combination):
> # ABSOLUTE
> ** based on absolute number of reserved space
> # PERCENTAGE
> ** based on percentage of total capacity in the storage
> # CONSERVATIVE
> ** calculates both of the above and takes the one that will yield more
> reserved space
> # AGGRESSIVE
> ** calculates 1. 2. and takes the one that will yield less reserved space
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13304) Document: update the new ozone docker file location
Xiaoyu Yao created HDFS-13304: - Summary: Document: update the new ozone docker file location Key: HDFS-13304 URL: https://issues.apache.org/jira/browse/HDFS-13304 Project: Hadoop HDFS Issue Type: Sub-task Components: documentation Reporter: Xiaoyu Yao Assignee: Xiaoyu Yao The docker compose file has been moved from dev-support/compose/ozone to hadoop-dist/target/compose/ozone, we need to update the document in OzoneGettingStarted.md.vm. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12977) Add stateId to RPC headers.
[ https://issues.apache.org/jira/browse/HDFS-12977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402622#comment-16402622 ] Plamen Jeliazkov commented on HDFS-12977: - Hi [~vagarychen]! Thanks for taking a look at the patch! I absolutely agree. That's a great optimization! Thanks for the suggestion. I've updated my patch with your input. > Add stateId to RPC headers. > --- > > Key: HDFS-12977 > URL: https://issues.apache.org/jira/browse/HDFS-12977 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ipc, namenode >Reporter: Konstantin Shvachko >Assignee: Plamen Jeliazkov >Priority: Major > Attachments: HDFS_12977.trunk.001.patch, HDFS_12977.trunk.002.patch, > HDFS_12977.trunk.003.patch, HDFS_12977.trunk.004.patch, > HDFS_12977.trunk.005.patch, HDFS_12977.trunk.006.patch, > HDFS_12977.trunk.007.patch, HDFS_12977.trunk.008.patch > > > stateId is a new field in the RPC headers of NameNode proto calls. > stateId is the journal transaction Id, which represents LastSeenId for the > clients and LastWrittenId for NameNodes. See more in [reads from Standby > design > doc|https://issues.apache.org/jira/secure/attachment/12902925/ConsistentReadsFromStandbyNode.pdf]. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12977) Add stateId to RPC headers.
[ https://issues.apache.org/jira/browse/HDFS-12977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Plamen Jeliazkov updated HDFS-12977: Attachment: HDFS_12977.trunk.008.patch > Add stateId to RPC headers. > --- > > Key: HDFS-12977 > URL: https://issues.apache.org/jira/browse/HDFS-12977 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ipc, namenode >Reporter: Konstantin Shvachko >Assignee: Plamen Jeliazkov >Priority: Major > Attachments: HDFS_12977.trunk.001.patch, HDFS_12977.trunk.002.patch, > HDFS_12977.trunk.003.patch, HDFS_12977.trunk.004.patch, > HDFS_12977.trunk.005.patch, HDFS_12977.trunk.006.patch, > HDFS_12977.trunk.007.patch, HDFS_12977.trunk.008.patch > > > stateId is a new field in the RPC headers of NameNode proto calls. > stateId is the journal transaction Id, which represents LastSeenId for the > clients and LastWrittenId for NameNodes. See more in [reads from Standby > design > doc|https://issues.apache.org/jira/secure/attachment/12902925/ConsistentReadsFromStandbyNode.pdf]. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri resolved HDFS-12886. Resolution: Fixed > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.2, 3.2.0 > > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch, > HDFS-12886_branch-2.000.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402612#comment-16402612 ] Íñigo Goiri commented on HDFS-12886: Thanks [~lukmajercak] for [^HDFS-12886_branch-2.000.patch]. As Yetus for branch-2 is messed up, no point of on waiting. The build works locally. Thanks [~ywskycn] for catching this and sorry for the inconvenience. > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.2, 3.2.0 > > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch, > HDFS-12886_branch-2.000.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402603#comment-16402603 ] Lukas Majercak commented on HDFS-12886: --- Added patch for branch-2 > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.2, 3.2.0 > > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch, > HDFS-12886_branch-2.000.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-12886: -- Attachment: HDFS-12886_branch-2.000.patch > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.2, 3.2.0 > > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch, > HDFS-12886_branch-2.000.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11043) TestWebHdfsTimeouts fails
[
https://issues.apache.org/jira/browse/HDFS-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402601#comment-16402601
]
Chao Sun commented on HDFS-11043:
-
Thanks [~xyao]. I changed the {{CLIENTS_TO_CONSUME_BACKLOG}} to 512 + 1 in my
case and it also appear to be working (waited until 7000+ iterations), while it
failed after certain number of iterations without the change.
Also curious: with the change on {{CLIENTS_TO_CONSUME_BACKLOG}} do we still
need to change {{SHORT_SOCKET_TIMEOUT}} and {{TEST_TIMEOUT}}?
> TestWebHdfsTimeouts fails
> -
>
> Key: HDFS-11043
> URL: https://issues.apache.org/jira/browse/HDFS-11043
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 3.0.0-alpha1
>Reporter: Andrew Wang
>Assignee: Chao Sun
>Priority: Major
> Attachments: HDFS-11043.000.patch, HDFS-11043.001.patch,
> org.apache.hadoop.hdfs.web.TestWebHdfsTimeouts.txt
>
>
> I'm seeing reproducible test failures for TestWebHdfsTimeouts locally, at
> least on trunk.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402584#comment-16402584 ] Wei Yan commented on HDFS-13215: Thanks, [~elgoiri]. I run all RBF tests locally and they're fine. Not sure whether Yetus is ok. Working on branch-2 patch, close to go. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13302) Ozone: fix classpath of yarn components
[ https://issues.apache.org/jira/browse/HDFS-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-13302: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7240 Status: Resolved (was: Patch Available) > Ozone: fix classpath of yarn components > --- > > Key: HDFS-13302 > URL: https://issues.apache.org/jira/browse/HDFS-13302 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: HDFS-7240 >Affects Versions: HDFS-7240 >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-13302-HDFS-7240.001.patch > > > HDFS-13258 introduced a separated classpath for hdsl/cblock/ozone components. > The reason is the behaviour of the dist-layout-stitching. The internal copy > command in the dist-layout-stitching copies the jar files only if they don > not exist in any of the existing subfolder of the share/hadoop directory. > With the new separated classpath if some of the dependencies are already > copied to share/hadoop/hdsl or share/hadoop/ozone, they won't be copied to > the share/hadoop/yarn directory and won't be added to the classpath. (as the > hdsl copy is before the yarn copy commands) > Could be fixed easily with moving the ozone/cblock/hdsl related stuff to the > end of the dist-layout-stitching. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402580#comment-16402580 ] Íñigo Goiri commented on HDFS-12514: Is there a unit test we can add? Ideally we could reset it but mocking the exception is fine too. > Cancelled HedgedReads cause block to be marked as suspect on Windows > > > Key: HDFS-12514 > URL: https://issues.apache.org/jira/browse/HDFS-12514 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, hdfs-client >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Attachments: HDFS-12514-branch-2.000.patch, HDFS-12514.001.patch, > HDFS-12514.002.patch, HDFS-12514.003.patch > > > DFSClient with hedged reads enabled will often close previous spawned > connections if it successfully reads from one of them. This can result in > DataNode's BlockSender getting a socket exception and wrongly marking the > block as suspect and to be rescanned for errors. > This patch is aimed at adding windows specific network related exception > messages to be ignored in BlockSender.sendPacket. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13302) Ozone: fix classpath of yarn components
[ https://issues.apache.org/jira/browse/HDFS-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402581#comment-16402581 ] Anu Engineer commented on HDFS-13302: - [~elek] Thanks for the contribution. [~bharatviswa] Thanks for the review. The patch was not applying, since it was a single file I applied changs manually. Please check if they look correct and works as expected. > Ozone: fix classpath of yarn components > --- > > Key: HDFS-13302 > URL: https://issues.apache.org/jira/browse/HDFS-13302 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: HDFS-7240 >Affects Versions: HDFS-7240 >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-13302-HDFS-7240.001.patch > > > HDFS-13258 introduced a separated classpath for hdsl/cblock/ozone components. > The reason is the behaviour of the dist-layout-stitching. The internal copy > command in the dist-layout-stitching copies the jar files only if they don > not exist in any of the existing subfolder of the share/hadoop directory. > With the new separated classpath if some of the dependencies are already > copied to share/hadoop/hdsl or share/hadoop/ozone, they won't be copied to > the share/hadoop/yarn directory and won't be added to the classpath. (as the > hdsl copy is before the yarn copy commands) > Could be fixed easily with moving the ozone/cblock/hdsl related stuff to the > end of the dist-layout-stitching. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402576#comment-16402576 ] Íñigo Goiri commented on HDFS-13215: Posted message in HDFS-12615 about freezing commits. Do you know if the RBF tests will run now? We need to generate patches for each of the branches: * branch-2 * branch-2.9 * branch-3.0 * branch-3.1 * trunk > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12615) Router-based HDFS federation phase 2
[ https://issues.apache.org/jira/browse/HDFS-12615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402573#comment-16402573 ] Íñigo Goiri commented on HDFS-12615: We are freezing all commits to RBF until we get HDFS-13215 in. [~ywskycn], please update this thread once done. > Router-based HDFS federation phase 2 > > > Key: HDFS-12615 > URL: https://issues.apache.org/jira/browse/HDFS-12615 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri >Priority: Major > Labels: RBF > > This umbrella JIRA tracks set of improvements over the Router-based HDFS > federation (HDFS-10467). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13176) WebHdfs file path gets truncated when having semicolon (;) inside
[ https://issues.apache.org/jira/browse/HDFS-13176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsolt Venczel updated HDFS-13176: - Attachment: HDFS-13176-branch-2.03.patch > WebHdfs file path gets truncated when having semicolon (;) inside > - > > Key: HDFS-13176 > URL: https://issues.apache.org/jira/browse/HDFS-13176 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 3.0.0 >Reporter: Zsolt Venczel >Assignee: Zsolt Venczel >Priority: Major > Attachments: HDFS-13176-branch-2.01.patch, > HDFS-13176-branch-2.03.patch, HDFS-13176-branch-2.03.patch, > HDFS-13176-branch-2.03.patch, HDFS-13176-branch-2.03.patch, > HDFS-13176-branch-2.03.patch, HDFS-13176.01.patch, HDFS-13176.02.patch, > TestWebHdfsUrl.testWebHdfsSpecialCharacterFile.patch > > > Find attached a patch having a test case that tries to reproduce the problem. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Reopened] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri reopened HDFS-12886: > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.2, 3.2.0 > > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-13303) branch-2 build is broken due to BlockManager for HDFS-12886
[ https://issues.apache.org/jira/browse/HDFS-13303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri resolved HDFS-13303. Resolution: Fixed > branch-2 build is broken due to BlockManager for HDFS-12886 > --- > > Key: HDFS-13303 > URL: https://issues.apache.org/jira/browse/HDFS-13303 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Wei Yan >Assignee: Íñigo Goiri >Priority: Major > > I think this is because of HDFS-12886. cc [~elgoiri] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13277) Improve move to account for usage (number of files) to limit trash dir size
[
https://issues.apache.org/jira/browse/HDFS-13277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402562#comment-16402562
]
genericqa commented on HDFS-13277:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
33s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-12996 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
14s{color} | {color:green} HDFS-12996 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} HDFS-12996 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
47s{color} | {color:green} HDFS-12996 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green} HDFS-12996 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 16s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
48s{color} | {color:green} HDFS-12996 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} HDFS-12996 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 28s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
18s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs generated 3 new + 0
unchanged - 0 fixed = 3 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}123m 44s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}178m 39s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs-project/hadoop-hdfs |
| | Comparison of String objects using == or != in
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$ReplicaFileDeleteTask.moveFilesToReplicaTrash()
At FsDatasetAsyncDiskService.java:== or != in
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$ReplicaFileDeleteTask.moveFilesToReplicaTrash()
At FsDatasetAsyncDiskService.java:[line 346] |
| | Possible null pointer dereference of current in
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$ReplicaFileDeleteTask.moveFilesToReplicaTrash()
Dereferenced at FsDatasetAsyncDiskService.java:current in
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$ReplicaFileDeleteTask.moveFilesToReplicaTrash()
Dereferenced at FsDatasetAsyncDiskService.java:[line 352] |
| | Should
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$ReplicaTrashInfo
be a _static_ inner class? At
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402563#comment-16402563 ] Íñigo Goiri commented on HDFS-12886: branch-2 and branch-2.9 break the build. We need patches for those. > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.2, 3.2.0 > > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-13303) branch-2 build is broken due to BlockManager for HDFS-12886
[ https://issues.apache.org/jira/browse/HDFS-13303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri reassigned HDFS-13303: -- Assignee: Íñigo Goiri > branch-2 build is broken due to BlockManager for HDFS-12886 > --- > > Key: HDFS-13303 > URL: https://issues.apache.org/jira/browse/HDFS-13303 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Wei Yan >Assignee: Íñigo Goiri >Priority: Major > > I think this is because of HDFS-12886. cc [~elgoiri] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13303) branch-2 build is broken due to BlockManager for HDFS-12886
[ https://issues.apache.org/jira/browse/HDFS-13303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402561#comment-16402561 ] Íñigo Goiri commented on HDFS-13303: Actually, this JIRA should be just to track the revert (already done) and HDFS-12886 should have the branch-2 patch. > branch-2 build is broken due to BlockManager for HDFS-12886 > --- > > Key: HDFS-13303 > URL: https://issues.apache.org/jira/browse/HDFS-13303 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Wei Yan >Priority: Major > > I think this is because of HDFS-12886. cc [~elgoiri] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13303) branch-2 build is broken due to BlockManager for HDFS-12886
[ https://issues.apache.org/jira/browse/HDFS-13303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-13303: --- Summary: branch-2 build is broken due to BlockManager for HDFS-12886 (was: branch-2 build is broken due to BlockManager) > branch-2 build is broken due to BlockManager for HDFS-12886 > --- > > Key: HDFS-13303 > URL: https://issues.apache.org/jira/browse/HDFS-13303 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Wei Yan >Priority: Major > > I think this is because of HDFS-12886. cc [~elgoiri] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11043) TestWebHdfsTimeouts fails
[
https://issues.apache.org/jira/browse/HDFS-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402555#comment-16402555
]
Xiaoyu Yao commented on HDFS-11043:
---
{quote}I didn't do that. Will try it.
{quote}
IntelliJ has a feature to repeatedly run a unit test until fail. I guess you
can use a bash script to do that with Maven too.
{quote}do you have any idea why it failed after certain number of retries but
not always fail?
{quote}
/proc/sys/net/ipv4/tcp_max_syn_backlog is a system-wide limit, that's part of
the changes mentioned in the article about Kernel changes. Depending on how
fast the kernel clean up the half-open connections, we are not guaranteed to
get a repro unless the test itself guarantee to consume TCP connections greater
than tcp_max_syn_backlog. That's why I took 128 +1 in the patch. As you
mentioned, different images may have different default values for that, we
might need to adjust accordingly then.
> TestWebHdfsTimeouts fails
> -
>
> Key: HDFS-11043
> URL: https://issues.apache.org/jira/browse/HDFS-11043
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 3.0.0-alpha1
>Reporter: Andrew Wang
>Assignee: Chao Sun
>Priority: Major
> Attachments: HDFS-11043.000.patch, HDFS-11043.001.patch,
> org.apache.hadoop.hdfs.web.TestWebHdfsTimeouts.txt
>
>
> I'm seeing reproducible test failures for TestWebHdfsTimeouts locally, at
> least on trunk.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13303) branch-2 build is broken due to BlockManager
[ https://issues.apache.org/jira/browse/HDFS-13303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402556#comment-16402556 ] Íñigo Goiri commented on HDFS-13303: Thanks [~ywskycn]. Reverting the commits in branch-2 and branch-2.9. [~lukmajercak], do you mind posting branch-2 patches here? > branch-2 build is broken due to BlockManager > > > Key: HDFS-13303 > URL: https://issues.apache.org/jira/browse/HDFS-13303 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Wei Yan >Priority: Major > > I think this is because of HDFS-12886. cc [~elgoiri] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13302) Ozone: fix classpath of yarn components
[ https://issues.apache.org/jira/browse/HDFS-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402545#comment-16402545 ] Anu Engineer commented on HDFS-13302: - I will commit this patch shortly. > Ozone: fix classpath of yarn components > --- > > Key: HDFS-13302 > URL: https://issues.apache.org/jira/browse/HDFS-13302 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: HDFS-7240 >Affects Versions: HDFS-7240 >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Attachments: HDFS-13302-HDFS-7240.001.patch > > > HDFS-13258 introduced a separated classpath for hdsl/cblock/ozone components. > The reason is the behaviour of the dist-layout-stitching. The internal copy > command in the dist-layout-stitching copies the jar files only if they don > not exist in any of the existing subfolder of the share/hadoop directory. > With the new separated classpath if some of the dependencies are already > copied to share/hadoop/hdsl or share/hadoop/ozone, they won't be copied to > the share/hadoop/yarn directory and won't be added to the classpath. (as the > hdsl copy is before the yarn copy commands) > Could be fixed easily with moving the ozone/cblock/hdsl related stuff to the > end of the dist-layout-stitching. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13303) branch-2 build is broken due to BlockManager
Wei Yan created HDFS-13303: -- Summary: branch-2 build is broken due to BlockManager Key: HDFS-13303 URL: https://issues.apache.org/jira/browse/HDFS-13303 Project: Hadoop HDFS Issue Type: Bug Reporter: Wei Yan I think this is because of HDFS-12886. cc [~elgoiri] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13302) Ozone: fix classpath of yarn components
[ https://issues.apache.org/jira/browse/HDFS-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402525#comment-16402525 ] Bharat Viswanadham commented on HDFS-13302: --- +1. LGTM. Thanks, [~elek] for detailed offline discussion. > Ozone: fix classpath of yarn components > --- > > Key: HDFS-13302 > URL: https://issues.apache.org/jira/browse/HDFS-13302 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: HDFS-7240 >Affects Versions: HDFS-7240 >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Attachments: HDFS-13302-HDFS-7240.001.patch > > > HDFS-13258 introduced a separated classpath for hdsl/cblock/ozone components. > The reason is the behaviour of the dist-layout-stitching. The internal copy > command in the dist-layout-stitching copies the jar files only if they don > not exist in any of the existing subfolder of the share/hadoop directory. > With the new separated classpath if some of the dependencies are already > copied to share/hadoop/hdsl or share/hadoop/ozone, they won't be copied to > the share/hadoop/yarn directory and won't be added to the classpath. (as the > hdsl copy is before the yarn copy commands) > Could be fixed easily with moving the ozone/cblock/hdsl related stuff to the > end of the dist-layout-stitching. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13296) GenericTestUtils#getTempPath and GenericTestUtils#getRandomizedTestDir generate paths with drive letter in Windows, and fail webhdfs related test cases
[ https://issues.apache.org/jira/browse/HDFS-13296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402520#comment-16402520 ] Íñigo Goiri commented on HDFS-13296: This patch is able to reduce our failed unit tests in Windows for HDFS from 296 to 239 in 2.9. The results for YARN are the same and as Yetus says, there is no change for Commons. [~chris.douglas], [~subru], any idea on how we can trigger HDFS unit tests and not only Commons? [~giovanni.fumarola], review appreciated. > GenericTestUtils#getTempPath and GenericTestUtils#getRandomizedTestDir > generate paths with drive letter in Windows, and fail webhdfs related test > cases > --- > > Key: HDFS-13296 > URL: https://issues.apache.org/jira/browse/HDFS-13296 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Xiao Liang >Assignee: Xiao Liang >Priority: Major > Labels: windows > Attachments: HDFS-13296.000.patch > > > In GenericTestUtils#getRandomizedTestDir, getAbsoluteFile is called and will > added drive letter to the path in windows, some test cases use the generated > path to send webhdfs request, which will fail due to the drive letter in the > URI like: "webhdfs://127.0.0.1:18334/D:/target/test/data/vUqZkOrBZa/test" > GenericTestUtils#getTempPath has the similar issue in Windows. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13250) RBF: Router to manage requests across multiple subclusters
[
https://issues.apache.org/jira/browse/HDFS-13250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402519#comment-16402519
]
genericqa commented on HDFS-13250:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 54s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 39s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 31s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}119m 57s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}166m 3s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
| | hadoop.hdfs.web.TestWebHdfsTimeouts |
| | hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations |
| | hadoop.hdfs.server.federation.router.TestRouterAllResolver |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-13250 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12914904/HDFS-13250.003.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 8f8c4df08a01 4.4.0-64-generic #85-Ubuntu SMP Mon Feb 20
11:50:30 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 08ff158 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/23520/artifact/out/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit |
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402518#comment-16402518 ] Wei Yan commented on HDFS-13215: Rebase a new patch [^HDFS-13215.006.patch]. [~elgoiri] [~linyiqun] let's freeze the RBF commits to get this in. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei Yan updated HDFS-13215: --- Attachment: HDFS-13215.006.patch > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch, HDFS-13215.006.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11043) TestWebHdfsTimeouts fails
[ https://issues.apache.org/jira/browse/HDFS-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402516#comment-16402516 ] Chao Sun commented on HDFS-11043: - [~xyao]: oh I didn't do that. Will try it. Also: do you have any idea why it failed after certain number of retries but not always fail? BTW: feel free to take over the JIRA :) > TestWebHdfsTimeouts fails > - > > Key: HDFS-11043 > URL: https://issues.apache.org/jira/browse/HDFS-11043 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Chao Sun >Priority: Major > Attachments: HDFS-11043.000.patch, HDFS-11043.001.patch, > org.apache.hadoop.hdfs.web.TestWebHdfsTimeouts.txt > > > I'm seeing reproducible test failures for TestWebHdfsTimeouts locally, at > least on trunk. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13298) Ozone: Make ozone/hdsl/cblock modules turned off by default
[ https://issues.apache.org/jira/browse/HDFS-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-13298: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-7240 Status: Resolved (was: Patch Available) [~elek] Thanks for the contribution. [~bharatviswa] Thanks for the review and tests. I have committed this to the feature branch. > Ozone: Make ozone/hdsl/cblock modules turned off by default > --- > > Key: HDFS-13298 > URL: https://issues.apache.org/jira/browse/HDFS-13298 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Fix For: HDFS-7240 > > Attachments: HDFS-13298-HDFS-7240.001.patch > > > According to the > [proposal|https://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201803.mbox/%3CCAHfHakEoHTVFo9R3FoNTbYF-ovEEaCExtPqxhxv0UV0HXjhrhw%40mail.gmail.com%3E] > from [~owen.omalley] Hdsl/Ozone/Cblock projects could be activated by an > optional maven profile. > At HDFS-13258 we moved out Hdsl/Ozone/Cblock code out from hdfs/common > projects, this issue is about introducing the new profile to turn off/on the > hdsl compilation/packaging. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12977) Add stateId to RPC headers.
[
https://issues.apache.org/jira/browse/HDFS-12977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402512#comment-16402512
]
Chen Liang commented on HDFS-12977:
---
Thanks [~zero45] for working on this! patch looks pretty good to me. Just one
minor thing, looks {{ClientGCIContext#receiveResponseState}} will be called on
every single client RPC. Since there can be high volume of such calls, I think
it might worth considering to minimize the synchronization overhead here.
Instead of using synchronized, maybe we can leverage Java 8's
[{{LongAccumulator}}|https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/atomic/LongAccumulator.html],
such as:
{code}
LongAccumulator longAccumulator = new LongAccumulator(Long::max,
Long.MIN_VALUE);
{code}
Then in receiveResponseState, it's just:
{code}
long lastSeenStateId = header.getStateId();
lastSeenStateId.accumulate(lastSeenStateId);
{code}
> Add stateId to RPC headers.
> ---
>
> Key: HDFS-12977
> URL: https://issues.apache.org/jira/browse/HDFS-12977
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ipc, namenode
>Reporter: Konstantin Shvachko
>Assignee: Plamen Jeliazkov
>Priority: Major
> Attachments: HDFS_12977.trunk.001.patch, HDFS_12977.trunk.002.patch,
> HDFS_12977.trunk.003.patch, HDFS_12977.trunk.004.patch,
> HDFS_12977.trunk.005.patch, HDFS_12977.trunk.006.patch,
> HDFS_12977.trunk.007.patch
>
>
> stateId is a new field in the RPC headers of NameNode proto calls.
> stateId is the journal transaction Id, which represents LastSeenId for the
> clients and LastWrittenId for NameNodes. See more in [reads from Standby
> design
> doc|https://issues.apache.org/jira/secure/attachment/12902925/ConsistentReadsFromStandbyNode.pdf].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13284) Adjust criteria for LowRedundancyBlocks.QUEUE_VERY_LOW_REDUNDANCY
[
https://issues.apache.org/jira/browse/HDFS-13284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402507#comment-16402507
]
genericqa commented on HDFS-13284:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
30s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 55s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated
0 new + 58 unchanged - 3 fixed = 58 total (was 61) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 12s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}115m 6s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}171m 40s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations |
| | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-13284 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12914899/HDFS-13284.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 460b5053fb62 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 08ff158 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/23519/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/23519/testReport/ |
| Max. process+thread count | 2869 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
|
[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[
https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402508#comment-16402508
]
genericqa commented on HDFS-12514:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
32s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 4s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated
0 new + 37 unchanged - 1 fixed = 37 total (was 38) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 7s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}115m 31s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}171m 54s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.TestSafeModeWithStripedFileWithRandomECPolicy |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-12514 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12914901/HDFS-12514.003.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 216f8f28cc25 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 08ff158 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/23518/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/23518/testReport/ |
| Max. process+thread count | 2598 (vs. ulimit of 1) |
| modules | C:
[jira] [Commented] (HDFS-12977) Add stateId to RPC headers.
[
https://issues.apache.org/jira/browse/HDFS-12977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402485#comment-16402485
]
genericqa commented on HDFS-12977:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
39s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 5s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
32s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 12m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m
15s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 35s{color} | {color:orange} root: The patch generated 4 new + 670 unchanged
- 0 fixed = 674 total (was 670) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
8m 25s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
10s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
25s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}123m 9s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}221m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
| | hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
| | hadoop.hdfs.web.TestWebHdfsTimeouts |
| | hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations |
| | hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized |
| | hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-12977 |
| JIRA
[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode
[
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402476#comment-16402476
]
Íñigo Goiri commented on HDFS-13283:
I would probably move the implementations of ReservedSpaceCalculator to a
separate package and have each one in a file.
I also like this approach better than the previous one.
Nit: put the extends line split all together.
> Percentage based Reserved Space Calculation for DataNode
>
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be
> an issue in non-heterogeneous clusters where size of DNs can differ. The
> proposed solution is to allow percentage based configuration (and their
> combination):
> # ABSOLUTE
> ** based on absolute number of reserved space
> # PERCENTAGE
> ** based on percentage of total capacity in the storage
> # CONSERVATIVE
> ** calculates both of the above and takes the one that will yield more
> reserved space
> # AGGRESSIVE
> ** calculates 1. 2. and takes the one that will yield less reserved space
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13302) Ozone: fix classpath of yarn components
[
https://issues.apache.org/jira/browse/HDFS-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402475#comment-16402475
]
genericqa commented on HDFS-13302:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
0s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} HDFS-7240 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 0s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
12s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
8m 50s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:215a942 |
| JIRA Issue | HDFS-13302 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12914921/HDFS-13302-HDFS-7240.001.patch
|
| Optional Tests | asflicense shellcheck shelldocs |
| uname | Linux 679fe41d7d74 4.4.0-89-generic #112-Ubuntu SMP Mon Jul 31
19:38:41 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | HDFS-7240 / 1f74cb2 |
| maven | version: Apache Maven 3.3.9 |
| shellcheck | v0.4.6 |
| Max. process+thread count | 456 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/23522/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Ozone: fix classpath of yarn components
> ---
>
> Key: HDFS-13302
> URL: https://issues.apache.org/jira/browse/HDFS-13302
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: HDFS-7240
>Affects Versions: HDFS-7240
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Attachments: HDFS-13302-HDFS-7240.001.patch
>
>
> HDFS-13258 introduced a separated classpath for hdsl/cblock/ozone components.
> The reason is the behaviour of the dist-layout-stitching. The internal copy
> command in the dist-layout-stitching copies the jar files only if they don
> not exist in any of the existing subfolder of the share/hadoop directory.
> With the new separated classpath if some of the dependencies are already
> copied to share/hadoop/hdsl or share/hadoop/ozone, they won't be copied to
> the share/hadoop/yarn directory and won't be added to the classpath. (as the
> hdsl copy is before the yarn copy commands)
> Could be fixed easily with moving the ozone/cblock/hdsl related stuff to the
> end of the dist-layout-stitching.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-13298) Ozone: Make ozone/hdsl/cblock modules turned off by default
[ https://issues.apache.org/jira/browse/HDFS-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402449#comment-16402449 ] Bharat Viswanadham edited comment on HDFS-13298 at 3/16/18 8:26 PM: [~elek] Thank You for the patch. LGTM. +1. I have build with and with out profile and tested it. And able to start a hdsl/ozone cluster. was (Author: bharatviswa): [~elek] Thank You for the patch. LGTM. +1. I have installed and tested it. And able to start a hdsl/ozone cluster. > Ozone: Make ozone/hdsl/cblock modules turned off by default > --- > > Key: HDFS-13298 > URL: https://issues.apache.org/jira/browse/HDFS-13298 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Attachments: HDFS-13298-HDFS-7240.001.patch > > > According to the > [proposal|https://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201803.mbox/%3CCAHfHakEoHTVFo9R3FoNTbYF-ovEEaCExtPqxhxv0UV0HXjhrhw%40mail.gmail.com%3E] > from [~owen.omalley] Hdsl/Ozone/Cblock projects could be activated by an > optional maven profile. > At HDFS-13258 we moved out Hdsl/Ozone/Cblock code out from hdfs/common > projects, this issue is about introducing the new profile to turn off/on the > hdsl compilation/packaging. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13298) Ozone: Make ozone/hdsl/cblock modules turned off by default
[ https://issues.apache.org/jira/browse/HDFS-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402456#comment-16402456 ] Anu Engineer commented on HDFS-13298: - [~bharatviswa] Thanks for the review and testing it. I will commit this to the feature branch now. > Ozone: Make ozone/hdsl/cblock modules turned off by default > --- > > Key: HDFS-13298 > URL: https://issues.apache.org/jira/browse/HDFS-13298 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Attachments: HDFS-13298-HDFS-7240.001.patch > > > According to the > [proposal|https://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201803.mbox/%3CCAHfHakEoHTVFo9R3FoNTbYF-ovEEaCExtPqxhxv0UV0HXjhrhw%40mail.gmail.com%3E] > from [~owen.omalley] Hdsl/Ozone/Cblock projects could be activated by an > optional maven profile. > At HDFS-13258 we moved out Hdsl/Ozone/Cblock code out from hdfs/common > projects, this issue is about introducing the new profile to turn off/on the > hdsl compilation/packaging. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11043) TestWebHdfsTimeouts fails
[ https://issues.apache.org/jira/browse/HDFS-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402440#comment-16402440 ] Xiaoyu Yao commented on HDFS-11043: --- [~csun], did you try letting the test run until failed? It passed 20~50 times without changing anything on my Ubuntu box too. Let's find the settings /proc/sys/net/ipv4/tcp_max_syn_backlog for the Yetus base image and set the CLIENTS_TO_CONSUME_BACKLOG accordingly. I will post a patch to see if it works. > TestWebHdfsTimeouts fails > - > > Key: HDFS-11043 > URL: https://issues.apache.org/jira/browse/HDFS-11043 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Chao Sun >Priority: Major > Attachments: HDFS-11043.000.patch, > org.apache.hadoop.hdfs.web.TestWebHdfsTimeouts.txt > > > I'm seeing reproducible test failures for TestWebHdfsTimeouts locally, at > least on trunk. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode
[
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402448#comment-16402448
]
Lukas Majercak commented on HDFS-13283:
---
Thanks for the review and suggestion [~virajith]. I removed the enum and
changed the config to specify the class of the calculator to be used.
> Percentage based Reserved Space Calculation for DataNode
>
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be
> an issue in non-heterogeneous clusters where size of DNs can differ. The
> proposed solution is to allow percentage based configuration (and their
> combination):
> # ABSOLUTE
> ** based on absolute number of reserved space
> # PERCENTAGE
> ** based on percentage of total capacity in the storage
> # CONSERVATIVE
> ** calculates both of the above and takes the one that will yield more
> reserved space
> # AGGRESSIVE
> ** calculates 1. 2. and takes the one that will yield less reserved space
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13298) Ozone: Make ozone/hdsl/cblock modules turned off by default
[ https://issues.apache.org/jira/browse/HDFS-13298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402449#comment-16402449 ] Bharat Viswanadham commented on HDFS-13298: --- [~elek] Thank You for the patch. LGTM. +1. I have installed and tested it. And able to start a hdsl/ozone cluster. > Ozone: Make ozone/hdsl/cblock modules turned off by default > --- > > Key: HDFS-13298 > URL: https://issues.apache.org/jira/browse/HDFS-13298 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Attachments: HDFS-13298-HDFS-7240.001.patch > > > According to the > [proposal|https://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201803.mbox/%3CCAHfHakEoHTVFo9R3FoNTbYF-ovEEaCExtPqxhxv0UV0HXjhrhw%40mail.gmail.com%3E] > from [~owen.omalley] Hdsl/Ozone/Cblock projects could be activated by an > optional maven profile. > At HDFS-13258 we moved out Hdsl/Ozone/Cblock code out from hdfs/common > projects, this issue is about introducing the new profile to turn off/on the > hdsl compilation/packaging. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13283) Percentage based Reserved Space Calculation for DataNode
[
https://issues.apache.org/jira/browse/HDFS-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lukas Majercak updated HDFS-13283:
--
Attachment: HDFS-13283.001.patch
> Percentage based Reserved Space Calculation for DataNode
>
>
> Key: HDFS-13283
> URL: https://issues.apache.org/jira/browse/HDFS-13283
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: datanode, hdfs
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-13283.000.patch, HDFS-13283.001.patch
>
>
> Currently, the only way to configure reserved disk space for non-HDFS data on
> a DataNode is a constant value via {{dfs.datanode.du.reserved}}. This can be
> an issue in non-heterogeneous clusters where size of DNs can differ. The
> proposed solution is to allow percentage based configuration (and their
> combination):
> # ABSOLUTE
> ** based on absolute number of reserved space
> # PERCENTAGE
> ** based on percentage of total capacity in the storage
> # CONSERVATIVE
> ** calculates both of the above and takes the one that will yield more
> reserved space
> # AGGRESSIVE
> ** calculates 1. 2. and takes the one that will yield less reserved space
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13055) Aggregate usage statistics from datanodes
[
https://issues.apache.org/jira/browse/HDFS-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402443#comment-16402443
]
genericqa commented on HDFS-13055:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 58s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
42s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 2s{color} | {color:orange} hadoop-hdfs-project: The patch generated 11 new
+ 67 unchanged - 0 fixed = 78 total (was 67) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 42s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
39s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 87m 11s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}155m 53s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.TestDistributedFileSystem |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-13055 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12914897/HDFS-13055.008.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 4fb6e29c235a 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9
14:43:09 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git
[jira] [Updated] (HDFS-11043) TestWebHdfsTimeouts fails
[ https://issues.apache.org/jira/browse/HDFS-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-11043: -- Attachment: HDFS-11043.001.patch > TestWebHdfsTimeouts fails > - > > Key: HDFS-11043 > URL: https://issues.apache.org/jira/browse/HDFS-11043 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Chao Sun >Priority: Major > Attachments: HDFS-11043.000.patch, HDFS-11043.001.patch, > org.apache.hadoop.hdfs.web.TestWebHdfsTimeouts.txt > > > I'm seeing reproducible test failures for TestWebHdfsTimeouts locally, at > least on trunk. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402438#comment-16402438 ] Wei Yan commented on HDFS-13215: let me do a quick rebase. and then we can freeze commit. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[ https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402432#comment-16402432 ] Íñigo Goiri commented on HDFS-13215: As we committed new stuff, [^HDFS-13215.005.patch] is missing it. We would have to redo the patch. [~ywskycn] let us know when you wanna freeze related commits to avoid this. > RBF: Move Router to its own module > -- > > Key: HDFS-13215 > URL: https://issues.apache.org/jira/browse/HDFS-13215 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Wei Yan >Priority: Major > Attachments: HDFS-13215.000.patch, HDFS-13215.001.patch, > HDFS-13215.002.patch, HDFS-13215.003.patch, HDFS-13215.004.patch, > HDFS-13215.005.patch > > > We are splitting the HDFS client code base and potentially Router-based > Federation is also independent enough to be in its own package. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13302) Ozone: fix classpath of yarn components
[
https://issues.apache.org/jira/browse/HDFS-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Elek, Marton updated HDFS-13302:
Status: Patch Available (was: Open)
The easy workaround is implemented with moving the copy to a later position.
But long term a more mature solution may be required. For example if any of the
tools would use a dependency which is already used by the hdsl it won't be
included in the tools directory.
As of now it works well. I tested with both ozone/hdsl and yarn based tests:
{code}
==
Yarn
==
Yarn.Yarn Smoketest :: Smoketesting Yarn cluster
==
Daemons are running without error | PASS |
--
Calculate PI with mapreduce job | PASS |
--
Calculate word counts | PASS |
--
Yarn.Yarn Smoketest :: Smoketesting Yarn cluster | PASS |
3 critical tests, 3 passed, 0 failed
3 tests total, 3 passed, 0 failed
==
{code}
{code}
==
Robot.Simple Cluster Topology :: Smoke test to start cluster with docker-co...
==
Daemons are running without error | PASS |
--
Check if datanode is connected to the scm | PASS |
--
Scale it up to 5 datanodes| PASS |
--
Test rest interface | PASS |
--
Check webui static resources | PASS |
--
Start freon testing | PASS |
--
Robot.Simple Cluster Topology :: Smoke test to start cluster with ... | PASS |
6 critical tests, 6 passed, 0 failed
6 tests total, 6 passed, 0 failed
{code}
> Ozone: fix classpath of yarn components
> ---
>
> Key: HDFS-13302
> URL: https://issues.apache.org/jira/browse/HDFS-13302
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: HDFS-7240
>Affects Versions: HDFS-7240
>Reporter: Elek, Marton
>Assignee: Elek, Marton
>Priority: Major
> Attachments: HDFS-13302-HDFS-7240.001.patch
>
>
> HDFS-13258 introduced a separated classpath for hdsl/cblock/ozone components.
> The reason is the behaviour of the dist-layout-stitching. The internal copy
> command in the dist-layout-stitching copies the jar files only if they don
> not exist in any of the existing subfolder of the share/hadoop directory.
> With the new separated classpath if some of the dependencies are already
> copied to share/hadoop/hdsl or share/hadoop/ozone, they won't be copied to
> the share/hadoop/yarn directory and won't be added to the classpath. (as the
> hdsl copy is before the yarn copy commands)
> Could be fixed easily with moving the ozone/cblock/hdsl related stuff to the
> end of the dist-layout-stitching.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13302) Ozone: fix classpath of yarn components
[ https://issues.apache.org/jira/browse/HDFS-13302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elek, Marton updated HDFS-13302: Attachment: HDFS-13302-HDFS-7240.001.patch > Ozone: fix classpath of yarn components > --- > > Key: HDFS-13302 > URL: https://issues.apache.org/jira/browse/HDFS-13302 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: HDFS-7240 >Affects Versions: HDFS-7240 >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Attachments: HDFS-13302-HDFS-7240.001.patch > > > HDFS-13258 introduced a separated classpath for hdsl/cblock/ozone components. > The reason is the behaviour of the dist-layout-stitching. The internal copy > command in the dist-layout-stitching copies the jar files only if they don > not exist in any of the existing subfolder of the share/hadoop directory. > With the new separated classpath if some of the dependencies are already > copied to share/hadoop/hdsl or share/hadoop/ozone, they won't be copied to > the share/hadoop/yarn directory and won't be added to the classpath. (as the > hdsl copy is before the yarn copy commands) > Could be fixed easily with moving the ozone/cblock/hdsl related stuff to the > end of the dist-layout-stitching. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13215) RBF: Move Router to its own module
[
https://issues.apache.org/jira/browse/HDFS-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402374#comment-16402374
]
genericqa commented on HDFS-13215:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 73 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 11m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 20s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-hdfs-project . {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
16s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
29s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 1m
11s{color} | {color:red} hadoop-hdfs-project in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 5m
47s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 1m
58s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} cc {color} | {color:red} 1m 58s{color} |
{color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 1m 58s{color}
| {color:red} root in the patch failed. {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
3m 9s{color} | {color:orange} root: The patch generated 3 new + 431 unchanged
- 3 fixed = 434 total (was 434) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
51s{color} | {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
8s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
7s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 2m
51s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-hdfs-project . {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
30s{color} | {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m
47s{color} | {color:red} root in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 53m 38s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} |
[jira] [Commented] (HDFS-11043) TestWebHdfsTimeouts fails
[ https://issues.apache.org/jira/browse/HDFS-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402359#comment-16402359 ] Íñigo Goiri commented on HDFS-11043: Thanks [~xyao] for the findings, I would go for those fixes; let's post a patch. I can test this on Windows too and if it doesn't work, we can assume Linux. > TestWebHdfsTimeouts fails > - > > Key: HDFS-11043 > URL: https://issues.apache.org/jira/browse/HDFS-11043 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Chao Sun >Priority: Major > Attachments: HDFS-11043.000.patch, > org.apache.hadoop.hdfs.web.TestWebHdfsTimeouts.txt > > > I'm seeing reproducible test failures for TestWebHdfsTimeouts locally, at > least on trunk. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11043) TestWebHdfsTimeouts fails
[ https://issues.apache.org/jira/browse/HDFS-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402340#comment-16402340 ] Chao Sun commented on HDFS-11043: - [~xyao]: thanks for the findings! I also spent some time testing on my Ubuntu laptop, which has 512 in /proc/sys/net/ipv4/tcp_max_syn_backlog. However, the tests always succeeded without any change on the code. > TestWebHdfsTimeouts fails > - > > Key: HDFS-11043 > URL: https://issues.apache.org/jira/browse/HDFS-11043 > Project: Hadoop HDFS > Issue Type: Bug > Components: webhdfs >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Chao Sun >Priority: Major > Attachments: HDFS-11043.000.patch, > org.apache.hadoop.hdfs.web.TestWebHdfsTimeouts.txt > > > I'm seeing reproducible test failures for TestWebHdfsTimeouts locally, at > least on trunk. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11043) TestWebHdfsTimeouts fails
[
https://issues.apache.org/jira/browse/HDFS-11043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402324#comment-16402324
]
Xiaoyu Yao commented on HDFS-11043:
---
Thanks [~elgoiri] for the comments and [~csun] for sharing articles on this.
Looks like the /proc/sys/net/ipv4/tcp_max_syn_backlog in Ubuntu is 128. I
slightly tweak the CLIENTS_TO_CONSUME_BACKLOG to 129 along with two small
changes like below.
Now, the test can pass successfully in Ubuntu with repeated runs of 500+ times
on Ubuntu.
Without the change below, the test usually failed after a repeated run of 40~50
times on Ubuntu.
{code}
- private static final int CLIENTS_TO_CONSUME_BACKLOG = 100;
+ private static final int CLIENTS_TO_CONSUME_BACKLOG = 129;
private static final int CONNECTION_BACKLOG = 1;
- private static final int SHORT_SOCKET_TIMEOUT = 5;
- private static final int TEST_TIMEOUT = 1;
+ private static final int SHORT_SOCKET_TIMEOUT = 100;
+ private static final int TEST_TIMEOUT = 10;
{code}
> TestWebHdfsTimeouts fails
> -
>
> Key: HDFS-11043
> URL: https://issues.apache.org/jira/browse/HDFS-11043
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: webhdfs
>Affects Versions: 3.0.0-alpha1
>Reporter: Andrew Wang
>Assignee: Chao Sun
>Priority: Major
> Attachments: HDFS-11043.000.patch,
> org.apache.hadoop.hdfs.web.TestWebHdfsTimeouts.txt
>
>
> I'm seeing reproducible test failures for TestWebHdfsTimeouts locally, at
> least on trunk.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-13302) Ozone: fix classpath of yarn components
Elek, Marton created HDFS-13302: --- Summary: Ozone: fix classpath of yarn components Key: HDFS-13302 URL: https://issues.apache.org/jira/browse/HDFS-13302 Project: Hadoop HDFS Issue Type: Sub-task Components: HDFS-7240 Affects Versions: HDFS-7240 Reporter: Elek, Marton Assignee: Elek, Marton HDFS-13258 introduced a separated classpath for hdsl/cblock/ozone components. The reason is the behaviour of the dist-layout-stitching. The internal copy command in the dist-layout-stitching copies the jar files only if they don not exist in any of the existing subfolder of the share/hadoop directory. With the new separated classpath if some of the dependencies are already copied to share/hadoop/hdsl or share/hadoop/ozone, they won't be copied to the share/hadoop/yarn directory and won't be added to the classpath. (as the hdsl copy is before the yarn copy commands) Could be fixed easily with moving the ozone/cblock/hdsl related stuff to the end of the dist-layout-stitching. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13277) Improve move to account for usage (number of files) to limit trash dir size
[ https://issues.apache.org/jira/browse/HDFS-13277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402318#comment-16402318 ] Bharat Viswanadham commented on HDFS-13277: --- Rebased patch, as HDFS-13163 got committed. > Improve move to account for usage (number of files) to limit trash dir size > --- > > Key: HDFS-13277 > URL: https://issues.apache.org/jira/browse/HDFS-13277 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Attachments: HDFS-13277-HDFS-12996.00.patch, > HDFS-13277-HDFS-12996.01.patch > > > The trash subdirectory maximum entries. This puts an upper limit on the size > of subdirectories in replica-trash. Set this default value to > blockinvalidateLimit. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12996) DataNode Replica Trash
[ https://issues.apache.org/jira/browse/HDFS-12996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-12996: -- Attachment: (was: HDFS-13277-HDFS-12996.01.patch) > DataNode Replica Trash > -- > > Key: HDFS-12996 > URL: https://issues.apache.org/jira/browse/HDFS-12996 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: DataNode_Replica_Trash_Design_Doc.pdf > > > DataNode Replica Trash will allow administrators to recover from a recent > delete request that resulted in catastrophic loss of user data. This is > achieved by placing all invalidated blocks in a replica trash on the datanode > before completely purging them from the system. The design doc is attached > here. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13277) Improve move to account for usage (number of files) to limit trash dir size
[ https://issues.apache.org/jira/browse/HDFS-13277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-13277: -- Attachment: HDFS-13277-HDFS-12996.01.patch > Improve move to account for usage (number of files) to limit trash dir size > --- > > Key: HDFS-13277 > URL: https://issues.apache.org/jira/browse/HDFS-13277 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Bharat Viswanadham >Assignee: Bharat Viswanadham >Priority: Major > Attachments: HDFS-13277-HDFS-12996.00.patch, > HDFS-13277-HDFS-12996.01.patch > > > The trash subdirectory maximum entries. This puts an upper limit on the size > of subdirectories in replica-trash. Set this default value to > blockinvalidateLimit. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12996) DataNode Replica Trash
[ https://issues.apache.org/jira/browse/HDFS-12996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-12996: -- Attachment: HDFS-13277-HDFS-12996.01.patch > DataNode Replica Trash > -- > > Key: HDFS-12996 > URL: https://issues.apache.org/jira/browse/HDFS-12996 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: Hanisha Koneru >Assignee: Hanisha Koneru >Priority: Major > Attachments: DataNode_Replica_Trash_Design_Doc.pdf > > > DataNode Replica Trash will allow administrators to recover from a recent > delete request that resulted in catastrophic loss of user data. This is > achieved by placing all invalidated blocks in a replica trash on the datanode > before completely purging them from the system. The design doc is attached > here. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12886) Ignore minReplication for block recovery
[ https://issues.apache.org/jira/browse/HDFS-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402289#comment-16402289 ] Hudson commented on HDFS-12886: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13849 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13849/]) HDFS-12886. Ignore minReplication for block recovery. Contributed by (inigoiri: rev 08ff1586d5d3e39f546200f9e696f62ea4cf000d) * (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java * (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java > Ignore minReplication for block recovery > > > Key: HDFS-12886 > URL: https://issues.apache.org/jira/browse/HDFS-12886 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs, namenode >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Fix For: 3.1.0, 2.10.0, 2.9.1, 3.0.2, 3.2.0 > > Attachments: HDFS-12886.001.patch, HDFS-12886.002.patch > > > Ignore minReplication for blocks that went through recovery, and allow NN to > complete them and replicate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13250) RBF: Router to manage requests across multiple subclusters
[ https://issues.apache.org/jira/browse/HDFS-13250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402277#comment-16402277 ] Íñigo Goiri commented on HDFS-13250: Thanks [~linyiqun] for the comments. If we do what you propose, I think it can be simplified. I added [^HDFS-13250.003.patch] with what I think it can be. I also extended the unit tests a little but we may want to do some more check. > RBF: Router to manage requests across multiple subclusters > -- > > Key: HDFS-13250 > URL: https://issues.apache.org/jira/browse/HDFS-13250 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri >Priority: Major > Attachments: HDFS-13250.000.patch, HDFS-13250.001.patch, > HDFS-13250.002.patch, HDFS-13250.003.patch > > > HDFS-13124 introduces the concept of mount points spanning multiple > subclusters. The Router should distribute the requests across these > subclusters. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13250) RBF: Router to manage requests across multiple subclusters
[ https://issues.apache.org/jira/browse/HDFS-13250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Íñigo Goiri updated HDFS-13250: --- Attachment: HDFS-13250.003.patch > RBF: Router to manage requests across multiple subclusters > -- > > Key: HDFS-13250 > URL: https://issues.apache.org/jira/browse/HDFS-13250 > Project: Hadoop HDFS > Issue Type: Sub-task >Reporter: Íñigo Goiri >Assignee: Íñigo Goiri >Priority: Major > Attachments: HDFS-13250.000.patch, HDFS-13250.001.patch, > HDFS-13250.002.patch, HDFS-13250.003.patch > > > HDFS-13124 introduces the concept of mount points spanning multiple > subclusters. The Router should distribute the requests across these > subclusters. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402261#comment-16402261 ] Lukas Majercak commented on HDFS-12514: --- Reuploaded patch 003 to trigger a build. > Cancelled HedgedReads cause block to be marked as suspect on Windows > > > Key: HDFS-12514 > URL: https://issues.apache.org/jira/browse/HDFS-12514 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, hdfs-client >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Attachments: HDFS-12514-branch-2.000.patch, HDFS-12514.001.patch, > HDFS-12514.002.patch, HDFS-12514.003.patch > > > DFSClient with hedged reads enabled will often close previous spawned > connections if it successfully reads from one of them. This can result in > DataNode's BlockSender getting a socket exception and wrongly marking the > block as suspect and to be rescanned for errors. > This patch is aimed at adding windows specific network related exception > messages to be ignored in BlockSender.sendPacket. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-12514: -- Attachment: (was: HDFS-12514.003.patch) > Cancelled HedgedReads cause block to be marked as suspect on Windows > > > Key: HDFS-12514 > URL: https://issues.apache.org/jira/browse/HDFS-12514 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, hdfs-client >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Attachments: HDFS-12514-branch-2.000.patch, HDFS-12514.001.patch, > HDFS-12514.002.patch, HDFS-12514.003.patch > > > DFSClient with hedged reads enabled will often close previous spawned > connections if it successfully reads from one of them. This can result in > DataNode's BlockSender getting a socket exception and wrongly marking the > block as suspect and to be rescanned for errors. > This patch is aimed at adding windows specific network related exception > messages to be ignored in BlockSender.sendPacket. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lukas Majercak updated HDFS-12514: -- Attachment: HDFS-12514.003.patch > Cancelled HedgedReads cause block to be marked as suspect on Windows > > > Key: HDFS-12514 > URL: https://issues.apache.org/jira/browse/HDFS-12514 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, hdfs-client >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Attachments: HDFS-12514-branch-2.000.patch, HDFS-12514.001.patch, > HDFS-12514.002.patch, HDFS-12514.003.patch > > > DFSClient with hedged reads enabled will often close previous spawned > connections if it successfully reads from one of them. This can result in > DataNode's BlockSender getting a socket exception and wrongly marking the > block as suspect and to be rescanned for errors. > This patch is aimed at adding windows specific network related exception > messages to be ignored in BlockSender.sendPacket. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
