[jira] [Commented] (HDFS-14703) NameNode Fine-Grained Locking via Metadata Partitioning

[ https://issues.apache.org/jira/browse/HDFS-14703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17387540#comment-17387540 ] Daryn Sharp commented on HDFS-14703: I applaud this effort but I have concerns about real world

[jira] [Commented] (HDFS-16127) Improper pipeline close recovery causes a permanent write failure or data loss.

[ https://issues.apache.org/jira/browse/HDFS-16127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17382288#comment-17382288 ] Daryn Sharp commented on HDFS-16127: +1 this has become a source of very nasty problems. > Improper

[jira] [Commented] (HDFS-15915) Race condition with async edits logging due to updating txId outside of the namesystem log

[ https://issues.apache.org/jira/browse/HDFS-15915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17352565#comment-17352565 ] Daryn Sharp commented on HDFS-15915: I'm very nervous about this patch and need to thoroughly

[jira] [Updated] (HDFS-16000) HDFS : Rename performance optimization

[ https://issues.apache.org/jira/browse/HDFS-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-16000: --- Attachment: (was: image.png) > HDFS : Rename performance optimization >

[jira] [Updated] (HDFS-16000) HDFS : Rename performance optimization

[ https://issues.apache.org/jira/browse/HDFS-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-16000: --- Attachment: image.png > HDFS : Rename performance optimization >

[jira] [Commented] (HDFS-16000) HDFS : Rename performance optimization

[ https://issues.apache.org/jira/browse/HDFS-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17337540#comment-17337540 ] Daryn Sharp commented on HDFS-16000: Nice.  Quota optimization has been on my to-do list for years

[jira] [Commented] (HDFS-15869) Network issue while FSEditLogAsync is executing RpcEdit.logSyncNotify can cause the namenode to hang

[ https://issues.apache.org/jira/browse/HDFS-15869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17325331#comment-17325331 ] Daryn Sharp commented on HDFS-15869: This is again a fault injection only issue.  A non-blocking

[jira] [Commented] (HDFS-15957) The ignored IOException in the RPC response sent by FSEditLogAsync can cause the HDFS client to hang

[ https://issues.apache.org/jira/browse/HDFS-15957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17321069#comment-17321069 ] Daryn Sharp commented on HDFS-15957: I'll take a look today.  In practice, the only time sendResponse

[jira] [Commented] (HDFS-15953) DUMMY_CLIENT_ID's verification may be invalid

[ https://issues.apache.org/jira/browse/HDFS-15953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17315850#comment-17315850 ] Daryn Sharp commented on HDFS-15953: Unless something changed, DUMMY_CLIENT_ID is an internal

[jira] [Commented] (HDFS-15553) Improve NameNode RPC throughput with ReadWriteRpcCallQueue

[ https://issues.apache.org/jira/browse/HDFS-15553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17189294#comment-17189294 ] Daryn Sharp commented on HDFS-15553: I've only skimmed the jira.  The basic premise of re-ordering

[jira] [Commented] (HDFS-10756) Expose getTrashRoot to HTTPFS and WebHDFS

[ https://issues.apache.org/jira/browse/HDFS-10756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992923#comment-16992923 ] Daryn Sharp commented on HDFS-10756: Surprised nobody has discovered this will lead to an inevitable

[jira] [Commented] (HDFS-13351) Revert HDFS-11156 from branch-2/branch-2.8

[ https://issues.apache.org/jira/browse/HDFS-13351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16992911#comment-16992911 ] Daryn Sharp commented on HDFS-13351: Prepping for 2.10.  This was never reverted.  It's completely

[jira] [Updated] (HDFS-14277) [SBN read] Observer benchmark results

[ https://issues.apache.org/jira/browse/HDFS-14277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-14277: --- Affects Version/s: 2.10.0 Priority: Blocker (was: Major) > [SBN read] Observer

[jira] [Commented] (HDFS-14277) [SBN read] Observer benchmark results

[ https://issues.apache.org/jira/browse/HDFS-14277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16906483#comment-16906483 ] Daryn Sharp commented on HDFS-14277: Due to internal branch conflicts I had to take an interrupt to

[jira] [Commented] (HDFS-11246) FSNameSystem#logAuditEvent should be called outside the read or write locks

[ https://issues.apache.org/jira/browse/HDFS-11246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883544#comment-16883544 ] Daryn Sharp commented on HDFS-11246: Yes, yes, yes, I would love this to go in.  Users are

[jira] [Commented] (HDFS-14617) Improve fsimage load time by writing sub-sections to the fsimage index

[ https://issues.apache.org/jira/browse/HDFS-14617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883532#comment-16883532 ] Daryn Sharp commented on HDFS-14617: I'm definitely not opposed to startup gains because 1-2h startup

[jira] [Commented] (HDFS-11687) Add new public encryption APIs required by Hive

[ https://issues.apache.org/jira/browse/HDFS-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883523#comment-16883523 ] Daryn Sharp commented on HDFS-11687: Admittedly very late to comment but this is a bad patch.  Not

[jira] [Commented] (HDFS-14617) Improve fsimage load time by writing sub-sections to the fsimage index

[ https://issues.apache.org/jira/browse/HDFS-14617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16880666#comment-16880666 ] Daryn Sharp commented on HDFS-14617: Was asked to take a look at this.  I think this can be done with

[jira] [Created] (HDFS-14534) NetworkTopology's fair lock has poor performance

Daryn Sharp created HDFS-14534: -- Summary: NetworkTopology's fair lock has poor performance Key: HDFS-14534 URL: https://issues.apache.org/jira/browse/HDFS-14534 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-14533) Datanode short circuit cache can become blocked

[ https://issues.apache.org/jira/browse/HDFS-14533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16853340#comment-16853340 ] Daryn Sharp commented on HDFS-14533: On further inspection, the jammed threads were actually clients

[jira] [Created] (HDFS-14533) Datanode short circuit cache can become blocked

Daryn Sharp created HDFS-14533: -- Summary: Datanode short circuit cache can become blocked Key: HDFS-14533 URL: https://issues.apache.org/jira/browse/HDFS-14533 Project: Hadoop HDFS Issue Type:

[jira] [Created] (HDFS-14532) Datanode's BlockSender checksum buffer is too big

Daryn Sharp created HDFS-14532: -- Summary: Datanode's BlockSender checksum buffer is too big Key: HDFS-14532 URL: https://issues.apache.org/jira/browse/HDFS-14532 Project: Hadoop HDFS Issue

[jira] [Created] (HDFS-14531) Datanode's ScanInfo requires excessive memory

Daryn Sharp created HDFS-14531: -- Summary: Datanode's ScanInfo requires excessive memory Key: HDFS-14531 URL: https://issues.apache.org/jira/browse/HDFS-14531 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-14521) Suppress setReplication logging.

[ https://issues.apache.org/jira/browse/HDFS-14521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16852410#comment-16852410 ] Daryn Sharp commented on HDFS-14521: +1 > Suppress setReplication logging. >

[jira] [Commented] (HDFS-14525) JspHelper ignores hadoop.http.authentication.type

[ https://issues.apache.org/jira/browse/HDFS-14525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16852255#comment-16852255 ] Daryn Sharp commented on HDFS-14525: You actually want a secure cluster to accept anonymous users?  

[jira] [Commented] (HDFS-14521) Suppress setReplication logging while replaying edits

[ https://issues.apache.org/jira/browse/HDFS-14521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16852114#comment-16852114 ] Daryn Sharp commented on HDFS-14521: This is the only operation that logs in the fsdir operations,

[jira] [Commented] (HDFS-14437) Exception happened when rollEditLog expects empty EditsDoubleBuffer.bufCurrent but not

[ https://issues.apache.org/jira/browse/HDFS-14437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16832024#comment-16832024 ] Daryn Sharp commented on HDFS-14437: Unless I'm overlooking new details, I explained on the cited

[jira] [Commented] (HDFS-14117) RBF: We can only delete the files or dirs of one subcluster in a cluster with multiple subclusters when trash is enabled

[ https://issues.apache.org/jira/browse/HDFS-14117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819336#comment-16819336 ] Daryn Sharp commented on HDFS-14117: It's a big patch and caught my eye because we are adding support

[jira] [Commented] (HDFS-14390) Provide kerberos support for AliasMap service used by Provided storage

[ https://issues.apache.org/jira/browse/HDFS-14390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819252#comment-16819252 ] Daryn Sharp commented on HDFS-14390: Ok, so the kerberos annotation is effectively the only change.  

[jira] [Commented] (HDFS-13596) NN restart fails after RollingUpgrade from 2.x to 3.x

[ https://issues.apache.org/jira/browse/HDFS-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819198#comment-16819198 ] Daryn Sharp commented on HDFS-13596: Consider whether it would be simpler for

[jira] [Commented] (HDFS-14403) Cost-Based RPC FairCallQueue

[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819126#comment-16819126 ] Daryn Sharp commented on HDFS-14403: Fantastic.  I'd like a chance to review the patch so please

[jira] [Commented] (HDFS-14418) Remove redundant super user priveledge checks from namenode.

[ https://issues.apache.org/jira/browse/HDFS-14418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819087#comment-16819087 ] Daryn Sharp commented on HDFS-14418: Yes, double {{checkOperation}} is required to protect against

[jira] [Commented] (HDFS-14403) Cost-Based RPC FairCallQueue

[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16819075#comment-16819075 ] Daryn Sharp commented on HDFS-14403: I think abstracting the cost provider is a good idea.  I'm

[jira] [Commented] (HDFS-14423) Percent (%) characters no longer work in WebHDFS

[ https://issues.apache.org/jira/browse/HDFS-14423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815832#comment-16815832 ] Daryn Sharp commented on HDFS-14423: Those 2 referenced patches appear completely broken.  The DN is

[jira] [Commented] (HDFS-14403) Cost-Based RPC FairCallQueue

[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815793#comment-16815793 ] Daryn Sharp commented on HDFS-14403: {quote}if you enable the cost-based version, you still make use

[jira] [Updated] (HDFS-14403) Cost-Based RPC FairCallQueue

[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-14403: --- Attachment: HDFS-14403.branch-2.8.patch > Cost-Based RPC FairCallQueue >

[jira] [Commented] (HDFS-14403) Cost-Based RPC FairCallQueue

[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815511#comment-16815511 ] Daryn Sharp commented on HDFS-14403: Sorry for delay, been a little bit distracted.  We should

[jira] [Commented] (HDFS-14403) Cost-Based RPC FairCallQueue

[ https://issues.apache.org/jira/browse/HDFS-14403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813640#comment-16813640 ] Daryn Sharp commented on HDFS-14403: I've got a simpler and more robust patch I'll try to cleanup/put

[jira] [Commented] (HDFS-14419) Avoid repeated calls to the listOpenFiles function

[ https://issues.apache.org/jira/browse/HDFS-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813619#comment-16813619 ] Daryn Sharp commented on HDFS-14419: Seems like a more straightforward and durable approach may be to

[jira] [Commented] (HDFS-14390) Provide kerberos support for AliasMap service used by Provided storage

[ https://issues.apache.org/jira/browse/HDFS-14390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813586#comment-16813586 ] Daryn Sharp commented on HDFS-14390: Jason asked me to take a look.  Which change actually fixed the

[jira] [Commented] (HDFS-13972) RBF: Support for Delegation Token (WebHDFS)

[ https://issues.apache.org/jira/browse/HDFS-13972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813570#comment-16813570 ] Daryn Sharp commented on HDFS-13972: I've only skimmed the Jira based on activity.  I haven't checked

[jira] [Commented] (HDFS-13596) NN restart fails after RollingUpgrade from 2.x to 3.x

[ https://issues.apache.org/jira/browse/HDFS-13596?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813548#comment-16813548 ] Daryn Sharp commented on HDFS-13596: I'm really glad this has gained traction!  Otherwise 3.x hdfs

[jira] [Commented] (HDFS-14384) When lastLocatedBlock token expire, it will take 1~3s second to refetch it.

[ https://issues.apache.org/jira/browse/HDFS-14384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812842#comment-16812842 ] Daryn Sharp commented on HDFS-14384: The {{LocatedBlocks}} class appears fundamentally broken – a

[jira] [Commented] (HDFS-14418) Remove redundant super user priveledge checks from namenode.

[ https://issues.apache.org/jira/browse/HDFS-14418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812813#comment-16812813 ] Daryn Sharp commented on HDFS-14418: I agree the super user check should be done once and should be

[jira] [Commented] (HDFS-14406) Add per user RPC Processing time

[ https://issues.apache.org/jira/browse/HDFS-14406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16812451#comment-16812451 ] Daryn Sharp commented on HDFS-14406: I'd also prefer this be optional.  The overhead will provide

[jira] [Commented] (HDFS-3246) pRead equivalent for direct read path

[ https://issues.apache.org/jira/browse/HDFS-3246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16798429#comment-16798429 ] Daryn Sharp commented on HDFS-3246: --- I would suggest assigning the in/out buffers within the try block

[jira] [Commented] (HDFS-14355) Implement HDFS cache on SCM by using pure java mapped byte buffer

[ https://issues.apache.org/jira/browse/HDFS-14355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16798421#comment-16798421 ] Daryn Sharp commented on HDFS-14355: I quickly skimmed the patch. At a high level, a plugin design

[jira] [Commented] (HDFS-14320) Support skipTrash for WebHDFS

[ https://issues.apache.org/jira/browse/HDFS-14320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16798362#comment-16798362 ] Daryn Sharp commented on HDFS-14320: Also, this patch won't work with security enabled since the NN's

[jira] [Commented] (HDFS-14320) Support skipTrash for WebHDFS

[ https://issues.apache.org/jira/browse/HDFS-14320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16797605#comment-16797605 ] Daryn Sharp commented on HDFS-14320: # Trash is a FSShell concept.  I really question whether it

[jira] [Commented] (HDFS-12345) Scale testing HDFS NameNode with real metadata and workloads (Dynamometer)

[ https://issues.apache.org/jira/browse/HDFS-12345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16797591#comment-16797591 ] Daryn Sharp commented on HDFS-12345: I like the concept of Dynamometer but I don't have cycles to

[jira] [Commented] (HDFS-14186) blockreport storm slow down namenode restart seriously in large cluster

[ https://issues.apache.org/jira/browse/HDFS-14186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16750247#comment-16750247 ] Daryn Sharp commented on HDFS-14186: Kihwal asked me to take a look. Safe mode logic has

[jira] [Commented] (HDFS-12284) RBF: Support for Kerberos authentication

[ https://issues.apache.org/jira/browse/HDFS-12284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16660770#comment-16660770 ] Daryn Sharp commented on HDFS-12284: Skimmed the patch.  Appears to be ok.  Minor nit is duplication

[jira] [Commented] (HDFS-13988) Allow for Creating Temporary Files

[ https://issues.apache.org/jira/browse/HDFS-13988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16648213#comment-16648213 ] Daryn Sharp commented on HDFS-13988: I'm -1 if it's in the NN. The NN's primary role is maintaining

[jira] [Assigned] (HDFS-8893) DNs with failed volumes stop serving during rolling upgrade

[ https://issues.apache.org/jira/browse/HDFS-8893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp reassigned HDFS-8893: - Assignee: (was: Daryn Sharp) > DNs with failed volumes stop serving during rolling upgrade >

[jira] [Commented] (HDFS-8893) DNs with failed volumes stop serving during rolling upgrade

[ https://issues.apache.org/jira/browse/HDFS-8893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16648031#comment-16648031 ] Daryn Sharp commented on HDFS-8893: --- I recall trying to fix this but as is usual with hdfs, the simpler

[jira] [Commented] (HDFS-13951) HDFS DelegationTokenFetcher can't print non-HDFS tokens in a tokenfile

[ https://issues.apache.org/jira/browse/HDFS-13951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646710#comment-16646710 ] Daryn Sharp commented on HDFS-13951: Given the recent compatibility squabbles, in 2012

[jira] [Commented] (HDFS-12284) RBF: Support for Kerberos authentication

[ https://issues.apache.org/jira/browse/HDFS-12284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646612#comment-16646612 ] Daryn Sharp commented on HDFS-12284: Sorry I've been too busy to offline discuss the RBF.  I'll try

[jira] [Commented] (HDFS-13697) DFSClient should instantiate and cache KMSClientProvider using UGI at creation time for consistent UGI handling

[ https://issues.apache.org/jira/browse/HDFS-13697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645590#comment-16645590 ] Daryn Sharp commented on HDFS-13697: Let me dig out my interrupted patch because the posted patch has

[jira] [Commented] (HDFS-13951) HDFS DelegationTokenFetcher can't print non-HDFS tokens in a tokenfile

[ https://issues.apache.org/jira/browse/HDFS-13951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16635643#comment-16635643 ] Daryn Sharp commented on HDFS-13951: Please don't combine unrelated patches on another Jira.  Glad

[jira] [Commented] (HDFS-13818) Extend OIV to detect FSImage corruption

[ https://issues.apache.org/jira/browse/HDFS-13818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628998#comment-16628998 ] Daryn Sharp commented on HDFS-13818: I don't care to review the patch assuming it doesn't alter the

[jira] [Commented] (HDFS-13248) RBF: Namenode need to choose block location for the client

[ https://issues.apache.org/jira/browse/HDFS-13248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16628974#comment-16628974 ] Daryn Sharp commented on HDFS-13248: Re-reading this Jira more close, per much earlier discussion,

[jira] [Commented] (HDFS-13929) DFSClient leaks data streamer sockets

[ https://issues.apache.org/jira/browse/HDFS-13929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1663#comment-1663 ] Daryn Sharp commented on HDFS-13929: Not sure of exact version that introduced the issue. >

[jira] [Created] (HDFS-13929) DFSClient leaks data streamer sockets

Daryn Sharp created HDFS-13929: -- Summary: DFSClient leaks data streamer sockets Key: HDFS-13929 URL: https://issues.apache.org/jira/browse/HDFS-13929 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-13697) DFSClient should instantiate and cache KMSClientProvider using UGI at creation time for consistent UGI handling

[ https://issues.apache.org/jira/browse/HDFS-13697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16612380#comment-16612380 ] Daryn Sharp commented on HDFS-13697: Delayed because DelegationTokenAuthenticatedURL is fundamentally

[jira] [Commented] (HDFS-13697) DFSClient should instantiate and cache KMSClientProvider using UGI at creation time for consistent UGI handling

[ https://issues.apache.org/jira/browse/HDFS-13697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16610908#comment-16610908 ] Daryn Sharp commented on HDFS-13697: The fix is a little more involved.  DT auth url needs to look in

[jira] [Commented] (HDFS-13697) DFSClient should instantiate and cache KMSClientProvider using UGI at creation time for consistent UGI handling

[ https://issues.apache.org/jira/browse/HDFS-13697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16595499#comment-16595499 ] Daryn Sharp commented on HDFS-13697: I whipped up a patch to retain identity and avoid the x509

[jira] [Commented] (HDFS-13697) DFSClient should instantiate and cache KMSClientProvider using UGI at creation time for consistent UGI handling

[ https://issues.apache.org/jira/browse/HDFS-13697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16592036#comment-16592036 ] Daryn Sharp commented on HDFS-13697: {quote}If userA creates the KMSCP, then proxies as userB and

[jira] [Commented] (HDFS-13671) Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

[ https://issues.apache.org/jira/browse/HDFS-13671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580028#comment-16580028 ] Daryn Sharp commented on HDFS-13671: I think we all know the time complexity to update a large

[jira] [Commented] (HDFS-13051) dead lock occurs when rolleditlog rpc call happen and editPendingQ is full

[ https://issues.apache.org/jira/browse/HDFS-13051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16580011#comment-16580011 ] Daryn Sharp commented on HDFS-13051: No brave reviewers? > dead lock occurs when rolleditlog rpc

[jira] [Commented] (HDFS-13772) Erasure coding: Unnecessary NameNode Logs displaying for Enabling/Disabling Erasure coding policies which are already enabled/disabled

[ https://issues.apache.org/jira/browse/HDFS-13772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579931#comment-16579931 ] Daryn Sharp commented on HDFS-13772: Isn't modifying the method signature a binary incompatible

[jira] [Commented] (HDFS-12862) CacheDirective may invalidata,when NN restart or make a transition to Active.

[ https://issues.apache.org/jira/browse/HDFS-12862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16579903#comment-16579903 ] Daryn Sharp commented on HDFS-12862: Rather than rebuild the object, why not make the serialization

[jira] [Commented] (HDFS-13697) EDEK decrypt fails due to proxy user being lost because of empty AccessControllerContext

[ https://issues.apache.org/jira/browse/HDFS-13697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16537759#comment-16537759 ] Daryn Sharp commented on HDFS-13697: This is a concrete use case to illustrate how/why the kms

[jira] [Commented] (HDFS-13697) EDEK decrypt fails due to proxy user being lost because of empty AccessControllerContext

[ https://issues.apache.org/jira/browse/HDFS-13697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16537538#comment-16537538 ] Daryn Sharp commented on HDFS-13697: No, this is wrong, I've grumbled about the kms client and ugi in

[jira] [Commented] (HDFS-13676) TestEditLogRace fails on Windows

[ https://issues.apache.org/jira/browse/HDFS-13676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16514232#comment-16514232 ] Daryn Sharp commented on HDFS-13676: Hmm. I don’t intentionally submit patches with commented out

[jira] [Commented] (HDFS-13448) HDFS Block Placement - Ignore Locality for First Block Replica

[ https://issues.apache.org/jira/browse/HDFS-13448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16513084#comment-16513084 ] Daryn Sharp commented on HDFS-13448: {quote}They are generated by using existing test patterns on

[jira] [Commented] (HDFS-13473) DataNode update BlockKeys using mode PULL rather than PUSH from NameNode

[ https://issues.apache.org/jira/browse/HDFS-13473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16513074#comment-16513074 ] Daryn Sharp commented on HDFS-13473: Upon quick & cursory review, it generally looks good. # Doesn't

[jira] [Commented] (HDFS-11257) Evacuate DN when the remaining is negative

[ https://issues.apache.org/jira/browse/HDFS-11257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16512975#comment-16512975 ] Daryn Sharp commented on HDFS-11257: bq. Not dedicated HDFS clusters. We have some machines where DNs

[jira] [Commented] (HDFS-13671) Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

[ https://issues.apache.org/jira/browse/HDFS-13671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16511402#comment-16511402 ] Daryn Sharp commented on HDFS-13671: I'm not surprised. I fully expected having to revert when we

[jira] [Commented] (HDFS-13448) HDFS Block Placement - Ignore Locality for First Block Replica

[ https://issues.apache.org/jira/browse/HDFS-13448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500552#comment-16500552 ] Daryn Sharp commented on HDFS-13448: I'm fine with latest patch but is it really necessary to sleep

[jira] [Commented] (HDFS-13469) RBF: Support InodeID in the Router

[ https://issues.apache.org/jira/browse/HDFS-13469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500430#comment-16500430 ] Daryn Sharp commented on HDFS-13469: {quote}opaque handles to files/directories and use those for

[jira] [Commented] (HDFS-13643) Implement basic async rpc client

[ https://issues.apache.org/jira/browse/HDFS-13643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500258#comment-16500258 ] Daryn Sharp commented on HDFS-13643: bq. Its async. Its going to diverge, no? We'd like to make a

[jira] [Commented] (HDFS-13051) dead lock occurs when rolleditlog rpc call happen and editPendingQ is full

[ https://issues.apache.org/jira/browse/HDFS-13051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16500251#comment-16500251 ] Daryn Sharp commented on HDFS-13051: Tests failed due to unrelated "java.lang.OutOfMemoryError:

[jira] [Commented] (HDFS-13643) Implement basic async rpc client

[ https://issues.apache.org/jira/browse/HDFS-13643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16496721#comment-16496721 ] Daryn Sharp commented on HDFS-13643: It's a nice POC but not implementing security is a

[jira] [Commented] (HDFS-13281) Namenode#createFile should be /.reserved/raw/ aware.

[ https://issues.apache.org/jira/browse/HDFS-13281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479136#comment-16479136 ] Daryn Sharp commented on HDFS-13281: {quote}The argument is the test should verify the encrypted bytes

[jira] [Commented] (HDFS-13079) Provide a config to start namenode in safemode state upto a certain transaction id

[ https://issues.apache.org/jira/browse/HDFS-13079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472323#comment-16472323 ] Daryn Sharp commented on HDFS-13079: This appears to be a tricky to correctly wield "feature" (stop

[jira] [Commented] (HDFS-13339) Volume reference can't release when testVolFailureStatsPreservedOnNNRestart

[ https://issues.apache.org/jira/browse/HDFS-13339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472223#comment-16472223 ] Daryn Sharp commented on HDFS-13339: Creating new thread pool/factory instances is going to cause

[jira] [Commented] (HDFS-13489) Get base snapshotable path if exists for a given path

[ https://issues.apache.org/jira/browse/HDFS-13489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472166#comment-16472166 ] Daryn Sharp commented on HDFS-13489: On further inspection, this method appears to possibly be at odds

[jira] [Commented] (HDFS-13489) Get base snapshotable path if exists for a given path

[ https://issues.apache.org/jira/browse/HDFS-13489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472145#comment-16472145 ] Daryn Sharp commented on HDFS-13489: *Please* pay attention to security. Not not call

[jira] [Commented] (HDFS-13452) Some Potential NPE

[ https://issues.apache.org/jira/browse/HDFS-13452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470609#comment-16470609 ] Daryn Sharp commented on HDFS-13452: # Is it valid for {{representativeRecord}} to return null? If

[jira] [Commented] (HDFS-13448) HDFS Block Placement - Ignore Locality for First Block Replica

[ https://issues.apache.org/jira/browse/HDFS-13448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470559#comment-16470559 ] Daryn Sharp commented on HDFS-13448: Generally looks good! I'd suggest renaming {{NO_LOCALITY_WRITE}}

[jira] [Commented] (HDFS-13312) NameNode High Availability ZooKeeper based discovery rather than explicit nn1,nn2 configs

[ https://issues.apache.org/jira/browse/HDFS-13312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459764#comment-16459764 ] Daryn Sharp commented on HDFS-13312: bq. Yes, I don't expect this to work well for more than a couple

[jira] [Commented] (HDFS-13469) RBF: Support InodeID in the Router

[ https://issues.apache.org/jira/browse/HDFS-13469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459718#comment-16459718 ] Daryn Sharp commented on HDFS-13469: -1 on punting.  We can't close a serious oversight/regression in

[jira] [Commented] (HDFS-13111) Close recovery may incorrectly mark blocks corrupt

[ https://issues.apache.org/jira/browse/HDFS-13111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459059#comment-16459059 ] Daryn Sharp commented on HDFS-13111: Note that unassigned jiras are free for the taking.  Please. :)

[jira] [Commented] (HDFS-12136) BlockSender performance regression due to volume scanner edge case

[ https://issues.apache.org/jira/browse/HDFS-12136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459058#comment-16459058 ] Daryn Sharp commented on HDFS-12136: Did a cursory scan of HDFS-11187 and I'm not sure if it

[jira] [Commented] (HDFS-12136) BlockSender performance regression due to volume scanner edge case

[ https://issues.apache.org/jira/browse/HDFS-12136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459023#comment-16459023 ] Daryn Sharp commented on HDFS-12136: I'll study other Jira to see if it's a dup. > BlockSender

[jira] [Commented] (HDFS-13312) NameNode High Availability ZooKeeper based discovery rather than explicit nn1,nn2 configs

[ https://issues.apache.org/jira/browse/HDFS-13312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459021#comment-16459021 ] Daryn Sharp commented on HDFS-13312: You might want to talk to ZK devs.  An internal dev says ZK will

[jira] [Commented] (HDFS-13381) [SPS]: Use DFSUtilClient#makePathFromFileId() to prepare satisfier file path

[ https://issues.apache.org/jira/browse/HDFS-13381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16459012#comment-16459012 ] Daryn Sharp commented on HDFS-13381: {quote}{quote}More than one file collector implementation means

[jira] [Commented] (HDFS-10943) rollEditLog expects empty EditsDoubleBuffer.bufCurrent which is not guaranteed

[ https://issues.apache.org/jira/browse/HDFS-10943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446446#comment-16446446 ] Daryn Sharp commented on HDFS-10943: The quorum output stream will +always+ crash the NN if edits are

[jira] [Commented] (HDFS-10943) rollEditLog expects empty EditsDoubleBuffer.bufCurrent which is not guaranteed

[ https://issues.apache.org/jira/browse/HDFS-10943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16446329#comment-16446329 ] Daryn Sharp commented on HDFS-10943: This error should not be possible. The entire roll holds the FSN

[jira] [Commented] (HDFS-13469) RBF: Support InodeID in the Router

[ https://issues.apache.org/jira/browse/HDFS-13469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16443317#comment-16443317 ] Daryn Sharp commented on HDFS-13469: It's really as simple as:  the path /.reserved/.inodes/123 will

  1   2   3   4   5   6   7   8   9   10   >