[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13703810#comment-13703810
]
Varun Sharma commented on HDFS-4960:
Actually, I am currently failing to max out the
[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13703840#comment-13703840
]
Varun Sharma commented on HDFS-4960:
IIRC, I ran with 10K or something...
[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13702832#comment-13702832
]
Varun Sharma commented on HDFS-4960:
I don't know about the future plans of .meta -
Varun Sharma created HDFS-4960:
--
Summary: Unnecessary .meta seeks even when skip checksum is true
Key: HDFS-4960
URL: https://issues.apache.org/jira/browse/HDFS-4960
Project: Hadoop HDFS
Issue
[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4960:
---
Attachment: 4960-trunk.patch
Unnecessary .meta seeks even when skip checksum is true
[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4960:
---
Attachment: 4960-branch2.patch
Unnecessary .meta seeks even when skip checksum is true
[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4960:
---
Assignee: Varun Sharma
Status: Patch Available (was: Open)
Unnecessary .meta seeks even
[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4960:
---
Issue Type: Bug (was: Improvement)
Unnecessary .meta seeks even when skip checksum is true
[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13701677#comment-13701677
]
Varun Sharma commented on HDFS-4960:
Yep, that is how I confirmed it - strace now only
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13649155#comment-13649155
]
Varun Sharma commented on HDFS-4721:
I think so...
Speed up
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-branch2.patch
Speed up lease/block recovery when DN fails and a block goes into
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-trunk-v4.patch
Speed up lease/block recovery when DN fails and a block goes
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: (was: 4721-hadoop2.patch)
Speed up lease/block recovery when DN fails and a
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641509#comment-13641509
]
Varun Sharma commented on HDFS-4721:
[~szetszwo]
Sorry, I traced this down to a bug in
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-trunk-v3.patch
Speed up lease/block recovery when DN fails and a block goes
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641884#comment-13641884
]
Varun Sharma commented on HDFS-4721:
[~szetszwo]
The tests are passing with the latest
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-trunk-v2.patch
Speed up lease/block recovery when DN fails and a block goes
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13640619#comment-13640619
]
Varun Sharma commented on HDFS-4721:
[~szetszwo]
I had a question about lease recovery
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641117#comment-13641117
]
Varun Sharma commented on HDFS-4721:
Sure, I am checking on the unit tests.
Okay, I
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641135#comment-13641135
]
Varun Sharma commented on HDFS-4721:
I dont think - if i grep the file name - all I
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641221#comment-13641221
]
Varun Sharma commented on HDFS-4721:
Here are the remainder messages - looking at it
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641222#comment-13641222
]
Varun Sharma commented on HDFS-4721:
One more thing, currently we do it this way:
1)
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641237#comment-13641237
]
Varun Sharma commented on HDFS-4721:
Sorry I mean the directory containing the File is
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641275#comment-13641275
]
Varun Sharma commented on HDFS-4721:
So there are three messages in the 1st log snippet
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641276#comment-13641276
]
Varun Sharma commented on HDFS-4721:
I have already pasted every single line for the
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v5.patch
Speed up lease/block recovery when DN fails and a block goes into
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639225#comment-13639225
]
Varun Sharma commented on HDFS-4721:
Thanks [~saint@gmail.com]
I personally prefer
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639252#comment-13639252
]
Varun Sharma commented on HDFS-4721:
[~nkeywal]
From what I think we may really have a
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639330#comment-13639330
]
Varun Sharma commented on HDFS-4721:
Yep, this patch does not fix that behaviour on the
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639342#comment-13639342
]
Varun Sharma commented on HDFS-4721:
Yeah, I don't want to combine the NN and the DN
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639413#comment-13639413
]
Varun Sharma commented on HDFS-4721:
Hi Nicholas,
Thanks for your comments. My
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639438#comment-13639438
]
Varun Sharma commented on HDFS-4721:
I have pasted the log above in the comments trail
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639629#comment-13639629
]
Varun Sharma commented on HDFS-4721:
I think additional opinions from other HDFS
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639678#comment-13639678
]
Varun Sharma commented on HDFS-4721:
Thanks !
[~szetszwo]
I agree with all your
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v6.patch
Speed up lease/block recovery when DN fails and a block goes into
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: (was: 4721-v6.patch)
Speed up lease/block recovery when DN fails and a block
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v6.patch
Speed up lease/block recovery when DN fails and a block goes into
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639862#comment-13639862
]
Varun Sharma commented on HDFS-4721:
[~szetszwo]
I just attached v6 with the changes
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v7.patch
Speed up lease/block recovery when DN fails and a block goes into
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639909#comment-13639909
]
Varun Sharma commented on HDFS-4721:
Fixed in v7 [~szetszwo]
Speed up
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v8.patch
Speed up lease/block recovery when DN fails and a block goes into
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639916#comment-13639916
]
Varun Sharma commented on HDFS-4721:
Fixed some naming in v8. Thanks
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13639944#comment-13639944
]
Varun Sharma commented on HDFS-4721:
Is this failing because its patching against trunk
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-trunk.patch
Speed up lease/block recovery when DN fails and a block goes into
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638125#comment-13638125
]
Varun Sharma commented on HDFS-4721:
nicholas,
Good point. I will create a patch which
[
https://issues.apache.org/jira/browse/HDFS-4724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638172#comment-13638172
]
Varun Sharma commented on HDFS-4724:
I guess we still need a client side timeout.
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638457#comment-13638457
]
Varun Sharma commented on HDFS-4721:
Attached v2 which does the following:
1)
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638569#comment-13638569
]
Varun Sharma commented on HDFS-4721:
Hi Nicholas,
Thanks for taking a look...
In my
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v3.patch
Speed up lease/block recovery when DN fails and a block goes into
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638578#comment-13638578
]
Varun Sharma commented on HDFS-4721:
Hi Nicholas,
I attached v3 which does that.
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638595#comment-13638595
]
Varun Sharma commented on HDFS-4721:
Thanks Andrew for taking a look !
I just attached
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638614#comment-13638614
]
Varun Sharma commented on HDFS-4721:
Yeah, so currently, we just try to avoid trying
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638679#comment-13638679
]
Varun Sharma commented on HDFS-4721:
Hi Nicholas,
I think thats a good point. Do you
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638694#comment-13638694
]
Varun Sharma commented on HDFS-4721:
I think in 99 % of cases, when we see no heart
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v4.patch
Speed up lease/block recovery when DN fails and a block goes into
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638723#comment-13638723
]
Varun Sharma commented on HDFS-4721:
Attached patch v4...
This one is a lot cleaner
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: (was: 4721-v4.patch)
Speed up lease/block recovery when DN fails and a block
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-v4.patch
Speed up lease/block recovery when DN fails and a block goes into
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13638726#comment-13638726
]
Varun Sharma commented on HDFS-4721:
Sorry, there was some issue with the previous
Varun Sharma created HDFS-4721:
--
Summary: Speed up lease/block recovery when DN fails and a block
goes into recovery
Key: HDFS-4721
URL: https://issues.apache.org/jira/browse/HDFS-4721
Project: Hadoop
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13637611#comment-13637611
]
Varun Sharma commented on HDFS-4721:
As a 2nd action item, it would also be nice to
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Attachment: 4721-hadoop2.patch
Speed up lease/block recovery when DN fails and a block goes into
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13637698#comment-13637698
]
Varun Sharma commented on HDFS-4721:
I attached a rough patch which
a) Avoids a stale
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Sharma updated HDFS-4721:
---
Fix Version/s: 2.0.4-alpha
Status: Patch Available (was: Open)
Speed up lease/block
[
https://issues.apache.org/jira/browse/HDFS-4721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13637743#comment-13637743
]
Varun Sharma commented on HDFS-4721:
[~ted_yu]
Thanks Ted !
I plan to have config
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13636941#comment-13636941
]
Varun Sharma commented on HDFS-3703:
I have a question.
What happens if a client tries
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13636959#comment-13636959
]
Varun Sharma commented on HDFS-3703:
I actually am seeing an interesting race condition
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13636998#comment-13636998
]
Varun Sharma commented on HDFS-3703:
Do you know if for a block which is not FINALIZED
[
https://issues.apache.org/jira/browse/HDFS-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13637072#comment-13637072
]
Varun Sharma commented on HDFS-3703:
Thanks, Jing..
This holds for UNDER_RECOVERY
69 matches
Mail list logo