[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13636941#comment-13636941
]
Varun Sharma commented on HDFS-3703:
I have a question.
What happens if a client tries
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13636959#comment-13636959
]
Varun Sharma commented on HDFS-3703:
I actually am seeing an interesting race condition
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13636998#comment-13636998
]
Varun Sharma commented on HDFS-3703:
Do you know if for a block which is not FINALIZED
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13637072#comment-13637072
]
Varun Sharma commented on HDFS-3703:
Thanks, Jing..
This holds for UNDER_RECOVERY bloc
Varun Sharma created HDFS-4721:
--
Summary: Speed up lease/block recovery when DN fails and a block
goes into recovery
Key: HDFS-4721
URL: https://issues.apache.org/jira/browse/HDFS-4721
Project: Hadoop HD
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13637611#comment-13637611
]
Varun Sharma commented on HDFS-4721:
As a 2nd action item, it would also be nice to hav
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-hadoop2.patch
> Speed up lease/block recovery when DN fails and a block goes into
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13637698#comment-13637698
]
Varun Sharma commented on HDFS-4721:
I attached a rough patch which
a) Avoids a stale n
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Fix Version/s: 2.0.4-alpha
Status: Patch Available (was: Open)
> Speed up lease/block
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13637743#comment-13637743
]
Varun Sharma commented on HDFS-4721:
[~ted_yu]
Thanks Ted !
I plan to have config par
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638125#comment-13638125
]
Varun Sharma commented on HDFS-4721:
nicholas,
Good point. I will create a patch which
[
https://issues.apache.org/jira/browse/HDFS-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638172#comment-13638172
]
Varun Sharma commented on HDFS-4724:
I guess we still need a client side timeout. Consi
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v2.patch
> Speed up lease/block recovery when DN fails and a block goes into reco
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638457#comment-13638457
]
Varun Sharma commented on HDFS-4721:
Attached v2 which does the following:
1) Introduc
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638569#comment-13638569
]
Varun Sharma commented on HDFS-4721:
Hi Nicholas,
Thanks for taking a look...
In my v
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v3.patch
> Speed up lease/block recovery when DN fails and a block goes into reco
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638578#comment-13638578
]
Varun Sharma commented on HDFS-4721:
Hi Nicholas,
I attached v3 which does that. Basic
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638595#comment-13638595
]
Varun Sharma commented on HDFS-4721:
Thanks Andrew for taking a look !
I just attached
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638614#comment-13638614
]
Varun Sharma commented on HDFS-4721:
Yeah, so currently, we just try to avoid trying th
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638679#comment-13638679
]
Varun Sharma commented on HDFS-4721:
Hi Nicholas,
I think thats a good point. Do you t
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638694#comment-13638694
]
Varun Sharma commented on HDFS-4721:
I think in 99 % of cases, when we see no heart bea
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v4.patch
> Speed up lease/block recovery when DN fails and a block goes into reco
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638723#comment-13638723
]
Varun Sharma commented on HDFS-4721:
Attached patch v4...
This one is a lot cleaner an
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: (was: 4721-v4.patch)
> Speed up lease/block recovery when DN fails and a block goe
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v4.patch
> Speed up lease/block recovery when DN fails and a block goes into reco
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13638726#comment-13638726
]
Varun Sharma commented on HDFS-4721:
Sorry, there was some issue with the previous atta
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v5.patch
> Speed up lease/block recovery when DN fails and a block goes into reco
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13639225#comment-13639225
]
Varun Sharma commented on HDFS-4721:
Thanks [[email protected]]
I personally prefer
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13639252#comment-13639252
]
Varun Sharma commented on HDFS-4721:
[~nkeywal]
>From what I think we may really have a
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13639330#comment-13639330
]
Varun Sharma commented on HDFS-4721:
Yep, this patch does not fix that behaviour on the
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13639342#comment-13639342
]
Varun Sharma commented on HDFS-4721:
Yeah, I don't want to combine the NN and the DN in
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13639413#comment-13639413
]
Varun Sharma commented on HDFS-4721:
Hi Nicholas,
Thanks for your comments. My respons
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13639438#comment-13639438
]
Varun Sharma commented on HDFS-4721:
I have pasted the log above in the comments trail
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13639629#comment-13639629
]
Varun Sharma commented on HDFS-4721:
I think additional opinions from other HDFS commit
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13639678#comment-13639678
]
Varun Sharma commented on HDFS-4721:
Thanks !
[~szetszwo]
I agree with all your comme
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v6.patch
> Speed up lease/block recovery when DN fails and a block goes into reco
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: (was: 4721-v6.patch)
> Speed up lease/block recovery when DN fails and a block goe
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v6.patch
> Speed up lease/block recovery when DN fails and a block goes into reco
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13639862#comment-13639862
]
Varun Sharma commented on HDFS-4721:
[~szetszwo]
I just attached v6 with the changes yo
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v7.patch
> Speed up lease/block recovery when DN fails and a block goes into reco
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13639909#comment-13639909
]
Varun Sharma commented on HDFS-4721:
Fixed in v7 [~szetszwo]
> Speed u
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v8.patch
> Speed up lease/block recovery when DN fails and a block goes into reco
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13639916#comment-13639916
]
Varun Sharma commented on HDFS-4721:
Fixed some naming in v8. Thanks (setChosenForRecov
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13639944#comment-13639944
]
Varun Sharma commented on HDFS-4721:
Is this failing because its patching against trunk
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-trunk.patch
> Speed up lease/block recovery when DN fails and a block goes into r
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-trunk-v2.patch
> Speed up lease/block recovery when DN fails and a block goes int
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13640619#comment-13640619
]
Varun Sharma commented on HDFS-4721:
[~szetszwo]
I had a question about lease recovery
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641117#comment-13641117
]
Varun Sharma commented on HDFS-4721:
Sure, I am checking on the unit tests.
Okay, I wa
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641135#comment-13641135
]
Varun Sharma commented on HDFS-4721:
I dont think - if i grep the file name - all I hav
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641161#comment-13641161
]
Varun Sharma commented on HDFS-4721:
Time t - recoveryLease called with false return -
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641221#comment-13641221
]
Varun Sharma commented on HDFS-4721:
Here are the remainder messages - looking at it th
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641222#comment-13641222
]
Varun Sharma commented on HDFS-4721:
One more thing, currently we do it this way:
1) C
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641237#comment-13641237
]
Varun Sharma commented on HDFS-4721:
Sorry I mean the directory containing the "File" i
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641275#comment-13641275
]
Varun Sharma commented on HDFS-4721:
So there are three messages in the 1st log snippet
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641276#comment-13641276
]
Varun Sharma commented on HDFS-4721:
I have already pasted every single line for the bl
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641509#comment-13641509
]
Varun Sharma commented on HDFS-4721:
[~szetszwo]
Sorry, I traced this down to a bug in
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-trunk-v3.patch
> Speed up lease/block recovery when DN fails and a block goes int
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13641884#comment-13641884
]
Varun Sharma commented on HDFS-4721:
[~szetszwo]
The tests are passing with the latest
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-branch2.patch
> Speed up lease/block recovery when DN fails and a block goes into
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-trunk-v4.patch
> Speed up lease/block recovery when DN fails and a block goes int
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: (was: 4721-hadoop2.patch)
> Speed up lease/block recovery when DN fails and a bloc
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13649155#comment-13649155
]
Varun Sharma commented on HDFS-4721:
I think so...
> Speed up lease/bl
Varun Sharma created HDFS-4960:
--
Summary: Unnecessary .meta seeks even when skip checksum is true
Key: HDFS-4960
URL: https://issues.apache.org/jira/browse/HDFS-4960
Project: Hadoop HDFS
Issue T
[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4960:
---
Attachment: 4960-trunk.patch
> Unnecessary .meta seeks even when skip checksum is true
> -
[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4960:
---
Attachment: 4960-branch2.patch
> Unnecessary .meta seeks even when skip checksum is true
> ---
[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4960:
---
Assignee: Varun Sharma
Status: Patch Available (was: Open)
> Unnecessary .meta seeks even w
[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4960:
---
Issue Type: Bug (was: Improvement)
> Unnecessary .meta seeks even when skip checksum is true
> --
[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13701677#comment-13701677
]
Varun Sharma commented on HDFS-4960:
Yep, that is how I confirmed it - strace now only
[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13702832#comment-13702832
]
Varun Sharma commented on HDFS-4960:
I don't know about the future plans of .meta - how
[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13703810#comment-13703810
]
Varun Sharma commented on HDFS-4960:
Actually, I am currently failing to max out the SS
[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13703840#comment-13703840
]
Varun Sharma commented on HDFS-4960:
IIRC, I ran with 10K or something...
71 matches
Mail list logo