[
https://issues.apache.org/jira/browse/HBASE-30117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
JinHyuk Kim updated HBASE-30117:
--------------------------------
Description:
h2. Summary
The HBase shell {{create}} and {{alter}} commands put {{CONFIGURATION}} entries
into the wrong map for column family descriptors. This causes features like
{{ROWPREFIX_FIXED_LENGTH}} bloom filter to fail when configured via the shell.
h2. Problem
In HBASE-20819, {{set_descriptor_config}} in {{admin.rb}} was changed from
{{descriptor.setConfiguration(k, v)}} to {{descriptor.setValue(k, v)}} to
support {{TableDescriptorBuilder}} which does not have a {{setConfiguration}}
method. However, this change also affects
{{{}ColumnFamilyDescriptorBuilder{}}}, where {{setValue}} and
{{setConfiguration}} put values into different maps ({{{}values{}}} vs
{{{}configuration{}}}).
When we run:
{code:java}
create 'mytable', {NAME => 'f', BLOOMFILTER => 'ROWPREFIX_FIXED_LENGTH',
CONFIGURATION => {'RowPrefixBloomFilter.prefix_length' => '4'}}
{code}
The {{prefix_length}} is stored in the {{values}} map instead of the
{{configuration}} map. {{TableDescriptorChecker.checkBloomFilterType}} reads
from {{{}cfd.getConfiguration(){}}}, so it fails with...
{code:java}
Caused by: java.lang.IllegalArgumentException: Bloom filter type is
ROWPREFIX_FIXED_LENGTH, RowPrefixBloomFilter.prefix_length not specified.
at
org.apache.hadoop.hbase.util.BloomFilterUtil.getBloomFilterParam(BloomFilterUtil.java:262)
at
org.apache.hadoop.hbase.util.TableDescriptorChecker.lambda$checkBloomFilterType$6(TableDescriptorChecker.java:323)
... 11 more
{code}
The same gap exists in {{checkDateTieredCompactionForTimeRangeDataTiering}}: an
invalid combination of {{hbase.hstore.datatiering.type=TIME_RANGE}} with a
non-Date-Tiered store engine silently passes validation when set via the
shell's {{CONFIGURATION =>}}, because that check also reads only from
{{cfd.getConfiguration()}}.
{code:java}
# This should fail but succeed
create 'mytable2', {NAME => 'f', CONFIGURATION =>
{'hbase.hstore.datatiering.type' => 'TIME_RANGE', 'hbase.hstore.engine.class'
=> 'org.apache.hadoop.hbase.regionserver.DefaultStoreEngine'}}
{code}
h2. Fix
Follow the pattern from HBASE-29314 and extend it to the two CF-level
validators that were missed. Stack the values map alongside the configuration
map via {{CompoundConfiguration}}:
{code:java}
new CompoundConfiguration()
.addStringMap(cfd.getConfiguration())]
.addBytesMap(cfd.getValues());
{code}
Two call sites are updated. {{checkBloomFilterType}} and
{{checkDateTieredCompactionForTimeRangeDataTiering}}. No shell or DDL change
required; existing user DDL works unchanged.
was:
h2. Summary
The HBase shell {{create}} and {{alter}} commands put {{CONFIGURATION}} entries
into the wrong map for column family descriptors. This causes features like
{{ROWPREFIX_FIXED_LENGTH}} bloom filter to fail when configured via the shell.
h2. Problem
In HBASE-20819, {{set_descriptor_config}} in {{admin.rb}} was changed from
{{descriptor.setConfiguration(k, v)}} to {{descriptor.setValue(k, v)}} to
support {{TableDescriptorBuilder}} which does not have a {{setConfiguration}}
method. However, this change also affects
{{{}ColumnFamilyDescriptorBuilder{}}}, where {{setValue}} and
{{setConfiguration}} put values into different maps ({{{}values{}}} vs
{{{}configuration{}}}).
When we run:
{code:java}
create 'mytable', {NAME => 'f', BLOOMFILTER => 'ROWPREFIX_FIXED_LENGTH',
CONFIGURATION => {'RowPrefixBloomFilter.prefix_length' => '4'}}
{code}
The {{prefix_length}} is stored in the {{values}} map instead of the
{{configuration}} map. {{TableDescriptorChecker.checkBloomFilterType}} reads
from {{{}cfd.getConfiguration(){}}}, so it fails with...
{code:java}
Caused by: java.lang.IllegalArgumentException: Bloom filter type is
ROWPREFIX_FIXED_LENGTH, RowPrefixBloomFilter.prefix_length not specified.
at
org.apache.hadoop.hbase.util.BloomFilterUtil.getBloomFilterParam(BloomFilterUtil.java:262)
at
org.apache.hadoop.hbase.util.TableDescriptorChecker.lambda$checkBloomFilterType$6(TableDescriptorChecker.java:323)
... 11 more
{code}
h2. Fix
Call {{setConfiguration}} directly for column family descriptors in the {{cfd}}
method instead of using the shared {{set_descriptor_config}} method.
> TableDescriptorChecker should verify CF date-tiered and bloom filter
> configuration set via setValue
> ---------------------------------------------------------------------------------------------------
>
> Key: HBASE-30117
> URL: https://issues.apache.org/jira/browse/HBASE-30117
> Project: HBase
> Issue Type: Bug
> Components: shell
> Reporter: JinHyuk Kim
> Assignee: JinHyuk Kim
> Priority: Minor
> Labels: pull-request-available
>
> h2. Summary
> The HBase shell {{create}} and {{alter}} commands put {{CONFIGURATION}}
> entries into the wrong map for column family descriptors. This causes
> features like {{ROWPREFIX_FIXED_LENGTH}} bloom filter to fail when configured
> via the shell.
> h2. Problem
> In HBASE-20819, {{set_descriptor_config}} in {{admin.rb}} was changed from
> {{descriptor.setConfiguration(k, v)}} to {{descriptor.setValue(k, v)}} to
> support {{TableDescriptorBuilder}} which does not have a {{setConfiguration}}
> method. However, this change also affects
> {{{}ColumnFamilyDescriptorBuilder{}}}, where {{setValue}} and
> {{setConfiguration}} put values into different maps ({{{}values{}}} vs
> {{{}configuration{}}}).
> When we run:
> {code:java}
> create 'mytable', {NAME => 'f', BLOOMFILTER => 'ROWPREFIX_FIXED_LENGTH',
> CONFIGURATION => {'RowPrefixBloomFilter.prefix_length' => '4'}}
> {code}
> The {{prefix_length}} is stored in the {{values}} map instead of the
> {{configuration}} map. {{TableDescriptorChecker.checkBloomFilterType}} reads
> from {{{}cfd.getConfiguration(){}}}, so it fails with...
> {code:java}
> Caused by: java.lang.IllegalArgumentException: Bloom filter type is
> ROWPREFIX_FIXED_LENGTH, RowPrefixBloomFilter.prefix_length not specified.
> at
> org.apache.hadoop.hbase.util.BloomFilterUtil.getBloomFilterParam(BloomFilterUtil.java:262)
> at
> org.apache.hadoop.hbase.util.TableDescriptorChecker.lambda$checkBloomFilterType$6(TableDescriptorChecker.java:323)
> ... 11 more
> {code}
> The same gap exists in {{checkDateTieredCompactionForTimeRangeDataTiering}}:
> an invalid combination of {{hbase.hstore.datatiering.type=TIME_RANGE}} with a
> non-Date-Tiered store engine silently passes validation when set via the
> shell's {{CONFIGURATION =>}}, because that check also reads only from
> {{cfd.getConfiguration()}}.
> {code:java}
> # This should fail but succeed
> create 'mytable2', {NAME => 'f', CONFIGURATION =>
> {'hbase.hstore.datatiering.type' => 'TIME_RANGE',
> 'hbase.hstore.engine.class' =>
> 'org.apache.hadoop.hbase.regionserver.DefaultStoreEngine'}}
> {code}
> h2. Fix
> Follow the pattern from HBASE-29314 and extend it to the two CF-level
> validators that were missed. Stack the values map alongside the configuration
> map via {{CompoundConfiguration}}:
>
> {code:java}
> new CompoundConfiguration()
> .addStringMap(cfd.getConfiguration())]
> .addBytesMap(cfd.getValues());
> {code}
> Two call sites are updated. {{checkBloomFilterType}} and
> {{checkDateTieredCompactionForTimeRangeDataTiering}}. No shell or DDL change
> required; existing user DDL works unchanged.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
