[jira] [Updated] (HBASE-9283) Struct and StructIterator should properly handle trailing nulls
[ https://issues.apache.org/jira/browse/HBASE-9283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-9283: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to 0.95 and trunk. Thanks for the patch Nick. > Struct and StructIterator should properly handle trailing nulls > --- > > Key: HBASE-9283 > URL: https://issues.apache.org/jira/browse/HBASE-9283 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.98.0, 0.96.0 > > Attachments: 0001-HBASE-9283-Struct-trailing-null-handling.patch, > 0001-HBASE-9283-Struct-trailing-null-handling.patch, > 0001-HBASE-9283-Struct-trailing-null-handling.patch > > > For a composite row key, Phoenix strips off trailing null columns values in > the row key. The reason this is important is that then new nullable row key > columns can be added to a schema without requiring any data upgrade to > existing rows. Otherwise, adding new row key columns to the end of a schema > becomes extremely cumbersome, as you'd need to delete all existing rows and > add them back with a row key that includes a null value. > Rather than Phoenix needing to modify the iteration code everywhere (as > [~ndimiduk] outlined here: > https://issues.apache.org/jira/browse/HBASE-8693?focusedCommentId=13744499&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13744499), > it'd be better if StructIterator handled this out-of-the-box. Otherwise, if > Phoenix has to specialize this, we'd lose the interop piece which is the > justification for switching our type system to this new one in the first > place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9283) Struct and StructIterator should properly handle trailing nulls
[ https://issues.apache.org/jira/browse/HBASE-9283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-9283: Status: Patch Available (was: Open) > Struct and StructIterator should properly handle trailing nulls > --- > > Key: HBASE-9283 > URL: https://issues.apache.org/jira/browse/HBASE-9283 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.98.0, 0.96.0 > > Attachments: 0001-HBASE-9283-Struct-trailing-null-handling.patch, > 0001-HBASE-9283-Struct-trailing-null-handling.patch, > 0001-HBASE-9283-Struct-trailing-null-handling.patch > > > For a composite row key, Phoenix strips off trailing null columns values in > the row key. The reason this is important is that then new nullable row key > columns can be added to a schema without requiring any data upgrade to > existing rows. Otherwise, adding new row key columns to the end of a schema > becomes extremely cumbersome, as you'd need to delete all existing rows and > add them back with a row key that includes a null value. > Rather than Phoenix needing to modify the iteration code everywhere (as > [~ndimiduk] outlined here: > https://issues.apache.org/jira/browse/HBASE-8693?focusedCommentId=13744499&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13744499), > it'd be better if StructIterator handled this out-of-the-box. Otherwise, if > Phoenix has to specialize this, we'd lose the interop piece which is the > justification for switching our type system to this new one in the first > place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9283) Struct and StructIterator should properly handle trailing nulls
[ https://issues.apache.org/jira/browse/HBASE-9283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-9283: Attachment: 0001-HBASE-9283-Struct-trailing-null-handling.patch Updated patch. More rigorous unit tests and updated documentation. Depends on HBASE-9348. > Struct and StructIterator should properly handle trailing nulls > --- > > Key: HBASE-9283 > URL: https://issues.apache.org/jira/browse/HBASE-9283 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.98.0, 0.96.0 > > Attachments: 0001-HBASE-9283-Struct-trailing-null-handling.patch, > 0001-HBASE-9283-Struct-trailing-null-handling.patch, > 0001-HBASE-9283-Struct-trailing-null-handling.patch > > > For a composite row key, Phoenix strips off trailing null columns values in > the row key. The reason this is important is that then new nullable row key > columns can be added to a schema without requiring any data upgrade to > existing rows. Otherwise, adding new row key columns to the end of a schema > becomes extremely cumbersome, as you'd need to delete all existing rows and > add them back with a row key that includes a null value. > Rather than Phoenix needing to modify the iteration code everywhere (as > [~ndimiduk] outlined here: > https://issues.apache.org/jira/browse/HBASE-8693?focusedCommentId=13744499&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13744499), > it'd be better if StructIterator handled this out-of-the-box. Otherwise, if > Phoenix has to specialize this, we'd lose the interop piece which is the > justification for switching our type system to this new one in the first > place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9283) Struct and StructIterator should properly handle trailing nulls
[ https://issues.apache.org/jira/browse/HBASE-9283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-9283: Attachment: 0001-HBASE-9283-Struct-trailing-null-handling.patch I realized I don't need to push this position checking down into the DataType implementations. This patch keeps the null extension logic localized to Struct and StructIterator. I'll post one more version tomorrow which updates the docs. > Struct and StructIterator should properly handle trailing nulls > --- > > Key: HBASE-9283 > URL: https://issues.apache.org/jira/browse/HBASE-9283 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.98.0, 0.96.0 > > Attachments: 0001-HBASE-9283-Struct-trailing-null-handling.patch, > 0001-HBASE-9283-Struct-trailing-null-handling.patch > > > For a composite row key, Phoenix strips off trailing null columns values in > the row key. The reason this is important is that then new nullable row key > columns can be added to a schema without requiring any data upgrade to > existing rows. Otherwise, adding new row key columns to the end of a schema > becomes extremely cumbersome, as you'd need to delete all existing rows and > add them back with a row key that includes a null value. > Rather than Phoenix needing to modify the iteration code everywhere (as > [~ndimiduk] outlined here: > https://issues.apache.org/jira/browse/HBASE-8693?focusedCommentId=13744499&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13744499), > it'd be better if StructIterator handled this out-of-the-box. Otherwise, if > Phoenix has to specialize this, we'd lose the interop piece which is the > justification for switching our type system to this new one in the first > place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9283) Struct and StructIterator should properly handle trailing nulls
[ https://issues.apache.org/jira/browse/HBASE-9283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-9283: Attachment: 0001-HBASE-9283-Struct-trailing-null-handling.patch Hi [~giacomotaylor]. Have a look at this patch. TestStructNullExtension is designed to demonstrate the behavior we discussed. Notice I had to change all the nullable types. The difficulty is in determining whether the last entry is an implied null or something like an empty RawByte instance. The solution I have here is to require that any nullable type check for src.getPosition() == src.getLength() and return null instead of error out. Without this change, TestStruct fails precisely because of this discrepancy. > Struct and StructIterator should properly handle trailing nulls > --- > > Key: HBASE-9283 > URL: https://issues.apache.org/jira/browse/HBASE-9283 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk >Assignee: Nick Dimiduk > Fix For: 0.98.0, 0.96.0 > > Attachments: 0001-HBASE-9283-Struct-trailing-null-handling.patch > > > For a composite row key, Phoenix strips off trailing null columns values in > the row key. The reason this is important is that then new nullable row key > columns can be added to a schema without requiring any data upgrade to > existing rows. Otherwise, adding new row key columns to the end of a schema > becomes extremely cumbersome, as you'd need to delete all existing rows and > add them back with a row key that includes a null value. > Rather than Phoenix needing to modify the iteration code everywhere (as > [~ndimiduk] outlined here: > https://issues.apache.org/jira/browse/HBASE-8693?focusedCommentId=13744499&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13744499), > it'd be better if StructIterator handled this out-of-the-box. Otherwise, if > Phoenix has to specialize this, we'd lose the interop piece which is the > justification for switching our type system to this new one in the first > place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9283) Struct and StructIterator should properly handle trailing nulls
[ https://issues.apache.org/jira/browse/HBASE-9283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-9283: Fix Version/s: 0.95.3 0.96.0 > Struct and StructIterator should properly handle trailing nulls > --- > > Key: HBASE-9283 > URL: https://issues.apache.org/jira/browse/HBASE-9283 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk > Fix For: 0.96.0, 0.95.3 > > > For a composite row key, Phoenix strips off trailing null columns values in > the row key. The reason this is important is that then new nullable row key > columns can be added to a schema without requiring any data upgrade to > existing rows. Otherwise, adding new row key columns to the end of a schema > becomes extremely cumbersome, as you'd need to delete all existing rows and > add them back with a row key that includes a null value. > Rather than Phoenix needing to modify the iteration code everywhere (as > [~ndimiduk] outlined here: > https://issues.apache.org/jira/browse/HBASE-8693?focusedCommentId=13744499&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13744499), > it'd be better if StructIterator handled this out-of-the-box. Otherwise, if > Phoenix has to specialize this, we'd lose the interop piece which is the > justification for switching our type system to this new one in the first > place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9283) Struct and StructIterator should properly handle trailing nulls
[ https://issues.apache.org/jira/browse/HBASE-9283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-9283: Affects Version/s: 0.95.2 > Struct and StructIterator should properly handle trailing nulls > --- > > Key: HBASE-9283 > URL: https://issues.apache.org/jira/browse/HBASE-9283 > Project: HBase > Issue Type: Bug >Affects Versions: 0.95.2 >Reporter: Nick Dimiduk > > For a composite row key, Phoenix strips off trailing null columns values in > the row key. The reason this is important is that then new nullable row key > columns can be added to a schema without requiring any data upgrade to > existing rows. Otherwise, adding new row key columns to the end of a schema > becomes extremely cumbersome, as you'd need to delete all existing rows and > add them back with a row key that includes a null value. > Rather than Phoenix needing to modify the iteration code everywhere (as > [~ndimiduk] outlined here: > https://issues.apache.org/jira/browse/HBASE-8693?focusedCommentId=13744499&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13744499), > it'd be better if StructIterator handled this out-of-the-box. Otherwise, if > Phoenix has to specialize this, we'd lose the interop piece which is the > justification for switching our type system to this new one in the first > place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9283) Struct and StructIterator should properly handle trailing nulls
[ https://issues.apache.org/jira/browse/HBASE-9283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated HBASE-9283: Description: For a composite row key, Phoenix strips off trailing null columns values in the row key. The reason this is important is that then new nullable row key columns can be added to a schema without requiring any data upgrade to existing rows. Otherwise, adding new row key columns to the end of a schema becomes extremely cumbersome, as you'd need to delete all existing rows and add them back with a row key that includes a null value. Rather than Phoenix needing to modify the iteration code everywhere (as [~ndimiduk] outlined here: https://issues.apache.org/jira/browse/HBASE-8693?focusedCommentId=13744499&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13744499), it'd be better if StructIterator handled this out-of-the-box. Otherwise, if Phoenix has to specialize this, we'd lose the interop piece which is the justification for switching our type system to this new one in the first place. was:For [~giacomotaylor] to fill in. > Struct and StructIterator should properly handle trailing nulls > --- > > Key: HBASE-9283 > URL: https://issues.apache.org/jira/browse/HBASE-9283 > Project: HBase > Issue Type: Bug >Reporter: Nick Dimiduk > > For a composite row key, Phoenix strips off trailing null columns values in > the row key. The reason this is important is that then new nullable row key > columns can be added to a schema without requiring any data upgrade to > existing rows. Otherwise, adding new row key columns to the end of a schema > becomes extremely cumbersome, as you'd need to delete all existing rows and > add them back with a row key that includes a null value. > Rather than Phoenix needing to modify the iteration code everywhere (as > [~ndimiduk] outlined here: > https://issues.apache.org/jira/browse/HBASE-8693?focusedCommentId=13744499&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13744499), > it'd be better if StructIterator handled this out-of-the-box. Otherwise, if > Phoenix has to specialize this, we'd lose the interop piece which is the > justification for switching our type system to this new one in the first > place. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-9283) Struct and StructIterator should properly handle trailing nulls
[ https://issues.apache.org/jira/browse/HBASE-9283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated HBASE-9283: Tags: Phoenix > Struct and StructIterator should properly handle trailing nulls > --- > > Key: HBASE-9283 > URL: https://issues.apache.org/jira/browse/HBASE-9283 > Project: HBase > Issue Type: Bug >Reporter: Nick Dimiduk > > For [~giacomotaylor] to fill in. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
