[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13649073#comment-13649073
]
Ted Yu commented on HDFS-4721:
--
Should this feature be backported to branch-1 ?
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13649155#comment-13649155
]
Varun Sharma commented on HDFS-4721:
I think so...
Speed up
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643622#comment-13643622
]
Hudson commented on HDFS-4721:
--
Integrated in Hadoop-Yarn-trunk #196 (See
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643653#comment-13643653
]
Hudson commented on HDFS-4721:
--
Integrated in Hadoop-Hdfs-trunk #1385 (See
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643670#comment-13643670
]
Hudson commented on HDFS-4721:
--
Integrated in Hadoop-Mapreduce-trunk #1412 (See
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643139#comment-13643139
]
Hadoop QA commented on HDFS-4721:
-
{color:green}+1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643246#comment-13643246
]
Hudson commented on HDFS-4721:
--
Integrated in Hadoop-trunk-Commit #3673 (See
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641509#comment-13641509
]
Varun Sharma commented on HDFS-4721:
[~szetszwo]
Sorry, I traced this down to a bug in
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641577#comment-13641577
]
Hadoop QA commented on HDFS-4721:
-
{color:green}+1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641884#comment-13641884
]
Varun Sharma commented on HDFS-4721:
[~szetszwo]
The tests are passing with the latest
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642050#comment-13642050
]
Tsz Wo (Nicholas), SZE commented on HDFS-4721:
--
That's great. Please feel
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13640619#comment-13640619
]
Varun Sharma commented on HDFS-4721:
[~szetszwo]
I had a question about lease recovery
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13640741#comment-13640741
]
Hadoop QA commented on HDFS-4721:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641075#comment-13641075
]
Tsz Wo (Nicholas), SZE commented on HDFS-4721:
--
... lease recovery can occur
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641082#comment-13641082
]
Tsz Wo (Nicholas), SZE commented on HDFS-4721:
--
Varun, it seems that the patch
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641117#comment-13641117
]
Varun Sharma commented on HDFS-4721:
Sure, I am checking on the unit tests.
Okay, I
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641127#comment-13641127
]
Tsz Wo (Nicholas), SZE commented on HDFS-4721:
--
At (e), is the file closed?
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641135#comment-13641135
]
Varun Sharma commented on HDFS-4721:
I dont think - if i grep the file name - all I
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641215#comment-13641215
]
Tsz Wo (Nicholas), SZE commented on HDFS-4721:
--
Could you also grep
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641217#comment-13641217
]
Tsz Wo (Nicholas), SZE commented on HDFS-4721:
--
BTW, what is the timestamp
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641221#comment-13641221
]
Varun Sharma commented on HDFS-4721:
Here are the remainder messages - looking at it
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641222#comment-13641222
]
Varun Sharma commented on HDFS-4721:
One more thing, currently we do it this way:
1)
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641237#comment-13641237
]
Varun Sharma commented on HDFS-4721:
Sorry I mean the directory containing the File is
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641265#comment-13641265
]
Tsz Wo (Nicholas), SZE commented on HDFS-4721:
--
Is there anything between
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641275#comment-13641275
]
Varun Sharma commented on HDFS-4721:
So there are three messages in the 1st log snippet
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641276#comment-13641276
]
Varun Sharma commented on HDFS-4721:
I have already pasted every single line for the
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638904#comment-13638904
]
Hadoop QA commented on HDFS-4721:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639190#comment-13639190
]
stack commented on HDFS-4721:
-
Patch looks nice and clean to me. +1
Minor Nit: Should this be
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639225#comment-13639225
]
Varun Sharma commented on HDFS-4721:
Thanks [~saint@gmail.com]
I personally prefer
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639235#comment-13639235
]
Nicolas Liochon commented on HDFS-4721:
---
Would it make sense, in
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639241#comment-13639241
]
stack commented on HDFS-4721:
-
[~varunsharma] I was suggesting the info logged not worth of
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639252#comment-13639252
]
Varun Sharma commented on HDFS-4721:
[~nkeywal]
From what I think we may really have a
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639323#comment-13639323
]
Nicolas Liochon commented on HDFS-4721:
---
May be there is something I haven't
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639330#comment-13639330
]
Varun Sharma commented on HDFS-4721:
Yep, this patch does not fix that behaviour on the
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639337#comment-13639337
]
Nicolas Liochon commented on HDFS-4721:
---
You can use a specific configuration for
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639342#comment-13639342
]
Varun Sharma commented on HDFS-4721:
Yeah, I don't want to combine the NN and the DN
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639360#comment-13639360
]
Nicolas Liochon commented on HDFS-4721:
---
bq. Btw, do you know why the build failed ?
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639362#comment-13639362
]
Tsz Wo (Nicholas), SZE commented on HDFS-4721:
--
For the v5 patch,
- We don't
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639413#comment-13639413
]
Varun Sharma commented on HDFS-4721:
Hi Nicholas,
Thanks for your comments. My
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639438#comment-13639438
]
Varun Sharma commented on HDFS-4721:
I have pasted the log above in the comments trail
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639629#comment-13639629
]
Varun Sharma commented on HDFS-4721:
I think additional opinions from other HDFS
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639656#comment-13639656
]
Tsz Wo (Nicholas), SZE commented on HDFS-4721:
--
Hi Varun,
I see your point
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639678#comment-13639678
]
Varun Sharma commented on HDFS-4721:
Thanks !
[~szetszwo]
I agree with all your
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639696#comment-13639696
]
Tsz Wo (Nicholas), SZE commented on HDFS-4721:
--
Sure, boolean array is okay
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639860#comment-13639860
]
Hadoop QA commented on HDFS-4721:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639862#comment-13639862
]
Varun Sharma commented on HDFS-4721:
[~szetszwo]
I just attached v6 with the changes
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639880#comment-13639880
]
Hadoop QA commented on HDFS-4721:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639892#comment-13639892
]
Tsz Wo (Nicholas), SZE commented on HDFS-4721:
--
- new ArrayList(..) needs
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639909#comment-13639909
]
Varun Sharma commented on HDFS-4721:
Fixed in v7 [~szetszwo]
Speed up
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639916#comment-13639916
]
Varun Sharma commented on HDFS-4721:
Fixed some naming in v8. Thanks
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639939#comment-13639939
]
Hadoop QA commented on HDFS-4721:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639944#comment-13639944
]
Varun Sharma commented on HDFS-4721:
Is this failing because its patching against trunk
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639972#comment-13639972
]
Tsz Wo (Nicholas), SZE commented on HDFS-4721:
--
Varun, you are right. Please
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639993#comment-13639993
]
Hadoop QA commented on HDFS-4721:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13637867#comment-13637867
]
Ted Yu commented on HDFS-4721:
--
In FSNamesystem#internalReleaseLease():
{code}
case
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638125#comment-13638125
]
Varun Sharma commented on HDFS-4721:
nicholas,
Good point. I will create a patch which
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638295#comment-13638295
]
Nicolas Liochon commented on HDFS-4721:
---
The algo as I understand it is:
namenode
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638457#comment-13638457
]
Varun Sharma commented on HDFS-4721:
Attached v2 which does the following:
1)
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638558#comment-13638558
]
Hadoop QA commented on HDFS-4721:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638561#comment-13638561
]
Tsz Wo (Nicholas), SZE commented on HDFS-4721:
--
I think we don't need the new
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638569#comment-13638569
]
Varun Sharma commented on HDFS-4721:
Hi Nicholas,
Thanks for taking a look...
In my
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638575#comment-13638575
]
Tsz Wo (Nicholas), SZE commented on HDFS-4721:
--
So, you think we should
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638578#comment-13638578
]
Varun Sharma commented on HDFS-4721:
Hi Nicholas,
I attached v3 which does that.
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638584#comment-13638584
]
Andrew Wang commented on HDFS-4721:
---
I took a look at the latest patch, some comments:
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638595#comment-13638595
]
Varun Sharma commented on HDFS-4721:
Thanks Andrew for taking a look !
I just attached
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638603#comment-13638603
]
Andrew Wang commented on HDFS-4721:
---
Regarding repeated recoveries, I had the same
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638614#comment-13638614
]
Varun Sharma commented on HDFS-4721:
Yeah, so currently, we just try to avoid trying
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638617#comment-13638617
]
Tsz Wo (Nicholas), SZE commented on HDFS-4721:
--
It is better to try all
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638679#comment-13638679
]
Varun Sharma commented on HDFS-4721:
Hi Nicholas,
I think thats a good point. Do you
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638684#comment-13638684
]
Ted Yu commented on HDFS-4721:
--
I think the actual failure scenarios in production cluster are
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638694#comment-13638694
]
Varun Sharma commented on HDFS-4721:
I think in 99 % of cases, when we see no heart
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638698#comment-13638698
]
Hadoop QA commented on HDFS-4721:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638723#comment-13638723
]
Varun Sharma commented on HDFS-4721:
Attached patch v4...
This one is a lot cleaner
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638726#comment-13638726
]
Varun Sharma commented on HDFS-4721:
Sorry, there was some issue with the previous
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638761#comment-13638761
]
Hadoop QA commented on HDFS-4721:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13637611#comment-13637611
]
Varun Sharma commented on HDFS-4721:
As a 2nd action item, it would also be nice to
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13637698#comment-13637698
]
Varun Sharma commented on HDFS-4721:
I attached a rough patch which
a) Avoids a stale
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13637708#comment-13637708
]
Ted Yu commented on HDFS-4721:
--
{code}
+ private volatile boolean avoidStaleNodesForRecovery
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13637731#comment-13637731
]
Hadoop QA commented on HDFS-4721:
-
{color:red}-1 overall{color}. Here are the results of
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13637743#comment-13637743
]
Varun Sharma commented on HDFS-4721:
[~ted_yu]
Thanks Ted !
I plan to have config
80 matches
Mail list logo
