[jira] [Commented] (HDFS-11921) Ozone: KSM: Unable to put keys with zero length
[ https://issues.apache.org/jira/browse/HDFS-11921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163194#comment-16163194 ] Anu Engineer commented on HDFS-11921: - bq. This is needed for OzoneFileSystem as well, where this API is needed for directories. Directories will have keys with no data. good to know. So you are saying that the behaviour specified by [~vagarychen] is all we need. bq.Right now, the behaviour is that if a length of 0 is given, key allocation will succeed but following writes will throw exception due to writing out of range. I am going to resolve this JIRA as fixed. > Ozone: KSM: Unable to put keys with zero length > --- > > Key: HDFS-11921 > URL: https://issues.apache.org/jira/browse/HDFS-11921 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Mukul Kumar Singh >Priority: Minor > Labels: OzonePostMerge > Attachments: HDFS-11921-HDFS-7240.001.patch > > > As part of working on HDFS-11909, I was trying to put zero length keys. I > found that put key refuses to do that. Here is the call trace, > bq. at ScmBlockLocationProtocolClientSideTranslatorPB.allocateBlock > we check if the block size is greater than 0, which makes sense since we > should not call into SCM to allocate a block of zero size. > However these 2 calls are invoked for creating the key, so that metadata for > key can be created, we should probably take care of this behavior here. > bq. ksm.KeyManagerImpl.allocateKey > bq. ksm.KeySpaceManager.allocateKey(KeySpaceManager.java:428) > Another way to fix this might be to just allocate a block with at least 1 > byte always, which might be easier than special casing code. > [~vagarychen] Would you like to fix this in the next patch you are working on > ? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11921) Ozone: KSM: Unable to put keys with zero length
[ https://issues.apache.org/jira/browse/HDFS-11921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16163163#comment-16163163 ] Mukul Kumar Singh commented on HDFS-11921: -- This is needed for OzoneFileSystem as well, where this API is needed for directories. Directories will have keys with no data. > Ozone: KSM: Unable to put keys with zero length > --- > > Key: HDFS-11921 > URL: https://issues.apache.org/jira/browse/HDFS-11921 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Priority: Minor > Labels: OzonePostMerge > Attachments: HDFS-11921-HDFS-7240.001.patch > > > As part of working on HDFS-11909, I was trying to put zero length keys. I > found that put key refuses to do that. Here is the call trace, > bq. at ScmBlockLocationProtocolClientSideTranslatorPB.allocateBlock > we check if the block size is greater than 0, which makes sense since we > should not call into SCM to allocate a block of zero size. > However these 2 calls are invoked for creating the key, so that metadata for > key can be created, we should probably take care of this behavior here. > bq. ksm.KeyManagerImpl.allocateKey > bq. ksm.KeySpaceManager.allocateKey(KeySpaceManager.java:428) > Another way to fix this might be to just allocate a block with at least 1 > byte always, which might be easier than special casing code. > [~vagarychen] Would you like to fix this in the next patch you are working on > ? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11921) Ozone: KSM: Unable to put keys with zero length
[ https://issues.apache.org/jira/browse/HDFS-11921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16149451#comment-16149451 ] Anu Engineer commented on HDFS-11921: - [~vagarychen] Thanks for bringing this up. I am debugging that state machine now. I will have a better answer once this path is fixed. > Ozone: KSM: Unable to put keys with zero length > --- > > Key: HDFS-11921 > URL: https://issues.apache.org/jira/browse/HDFS-11921 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer >Priority: Minor > Attachments: HDFS-11921-HDFS-7240.001.patch > > > As part of working on HDFS-11909, I was trying to put zero length keys. I > found that put key refuses to do that. Here is the call trace, > bq. at ScmBlockLocationProtocolClientSideTranslatorPB.allocateBlock > we check if the block size is greater than 0, which makes sense since we > should not call into SCM to allocate a block of zero size. > However these 2 calls are invoked for creating the key, so that metadata for > key can be created, we should probably take care of this behavior here. > bq. ksm.KeyManagerImpl.allocateKey > bq. ksm.KeySpaceManager.allocateKey(KeySpaceManager.java:428) > Another way to fix this might be to just allocate a block with at least 1 > byte always, which might be easier than special casing code. > [~vagarychen] Would you like to fix this in the next patch you are working on > ? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11921) Ozone: KSM: Unable to put keys with zero length
[ https://issues.apache.org/jira/browse/HDFS-11921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039377#comment-16039377 ] Chen Liang commented on HDFS-11921: --- Yes, I already changed that part in HDFS-11920 :) > Ozone: KSM: Unable to put keys with zero length > --- > > Key: HDFS-11921 > URL: https://issues.apache.org/jira/browse/HDFS-11921 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer >Priority: Minor > > As part of working on HDFS-11909, I was trying to put zero length keys. I > found that put key refuses to do that. Here is the call trace, > bq. at ScmBlockLocationProtocolClientSideTranslatorPB.allocateBlock > we check if the block size is greater than 0, which makes sense since we > should not call into SCM to allocate a block of zero size. > However these 2 calls are invoked for creating the key, so that metadata for > key can be created, we should probably take care of this behavior here. > bq. ksm.KeyManagerImpl.allocateKey > bq. ksm.KeySpaceManager.allocateKey(KeySpaceManager.java:428) > Another way to fix this might be to just allocate a block with at least 1 > byte always, which might be easier than special casing code. > [~vagarychen] Would you like to fix this in the next patch you are working on > ? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11921) Ozone: KSM: Unable to put keys with zero length
[ https://issues.apache.org/jira/browse/HDFS-11921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039361#comment-16039361 ] Anu Engineer commented on HDFS-11921: - Thanks, you will have to undo the HDFS-11796 change which remaps a zero block size to 1. > Ozone: KSM: Unable to put keys with zero length > --- > > Key: HDFS-11921 > URL: https://issues.apache.org/jira/browse/HDFS-11921 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer >Priority: Minor > > As part of working on HDFS-11909, I was trying to put zero length keys. I > found that put key refuses to do that. Here is the call trace, > bq. at ScmBlockLocationProtocolClientSideTranslatorPB.allocateBlock > we check if the block size is greater than 0, which makes sense since we > should not call into SCM to allocate a block of zero size. > However these 2 calls are invoked for creating the key, so that metadata for > key can be created, we should probably take care of this behavior here. > bq. ksm.KeyManagerImpl.allocateKey > bq. ksm.KeySpaceManager.allocateKey(KeySpaceManager.java:428) > Another way to fix this might be to just allocate a block with at least 1 > byte always, which might be easier than special casing code. > [~vagarychen] Would you like to fix this in the next patch you are working on > ? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11921) Ozone: KSM: Unable to put keys with zero length
[ https://issues.apache.org/jira/browse/HDFS-11921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039353#comment-16039353 ] Chen Liang commented on HDFS-11921: --- This will be fixed when HDFS-11920 gets in. HDFS-11920 changed the code that when allocating blocks for key, it allocates a list of blocks. In case where the total requested size is 0, the list will be empty and no blocks will be allocated at all. So it will not trigger the allocateBlock() with size = 0 exception code path. > Ozone: KSM: Unable to put keys with zero length > --- > > Key: HDFS-11921 > URL: https://issues.apache.org/jira/browse/HDFS-11921 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer >Priority: Minor > > As part of working on HDFS-11909, I was trying to put zero length keys. I > found that put key refuses to do that. Here is the call trace, > bq. at ScmBlockLocationProtocolClientSideTranslatorPB.allocateBlock > we check if the block size is greater than 0, which makes sense since we > should not call into SCM to allocate a block of zero size. > However these 2 calls are invoked for creating the key, so that metadata for > key can be created, we should probably take care of this behavior here. > bq. ksm.KeyManagerImpl.allocateKey > bq. ksm.KeySpaceManager.allocateKey(KeySpaceManager.java:428) > Another way to fix this might be to just allocate a block with at least 1 > byte always, which might be easier than special casing code. > [~vagarychen] Would you like to fix this in the next patch you are working on > ? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11921) Ozone: KSM: Unable to put keys with zero length
[ https://issues.apache.org/jira/browse/HDFS-11921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16035630#comment-16035630 ] Anu Engineer commented on HDFS-11921: - The fix for HDFS-11796 could have fixed this issue. We added a check where we always request at least a byte sized block from SCM. So keeping this JIRA open to validate this issue. > Ozone: KSM: Unable to put keys with zero length > --- > > Key: HDFS-11921 > URL: https://issues.apache.org/jira/browse/HDFS-11921 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Affects Versions: HDFS-7240 >Reporter: Anu Engineer >Assignee: Anu Engineer >Priority: Minor > > As part of working on HDFS-11909, I was trying to put zero length keys. I > found that put key refuses to do that. Here is the call trace, > bq. at ScmBlockLocationProtocolClientSideTranslatorPB.allocateBlock > we check if the block size is greater than 0, which makes sense since we > should not call into SCM to allocate a block of zero size. > However these 2 calls are invoked for creating the key, so that metadata for > key can be created, we should probably take care of this behavior here. > bq. ksm.KeyManagerImpl.allocateKey > bq. ksm.KeySpaceManager.allocateKey(KeySpaceManager.java:428) > Another way to fix this might be to just allocate a block with at least 1 > byte always, which might be easier than special casing code. > [~vagarychen] Would you like to fix this in the next patch you are working on > ? -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
