[jira] [Updated] (HBASE-14326) HBase book: fix definition of max min size to compact
[
https://issues.apache.org/jira/browse/HBASE-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Misty Stanley-Jones updated HBASE-14326:
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Pushed to master. Thanks stack!
> HBase book: fix definition of max min size to compact
> -
>
> Key: HBASE-14326
> URL: https://issues.apache.org/jira/browse/HBASE-14326
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Vladimir Rodionov
>Assignee: Misty Stanley-Jones
> Fix For: 2.0.0
>
> Attachments: HBASE-14326.patch
>
>
> I think we need to change wording/definition of these config parameters in
> HBase book, they are misleading:
> {quote}
> hbase.hstore.compaction.min.size
> Description
> A StoreFile smaller than this size will always be eligible for minor
> compaction. HFiles this size or larger are evaluated by
> hbase.hstore.compaction.ratio to determine if they are eligible. Because this
> limit represents the "automatic include"limit for all StoreFiles smaller than
> this value, this value may need to be reduced in write-heavy environments
> where many StoreFiles in the 1-2 MB range are being flushed, because every
> StoreFile will be targeted for compaction and the resulting StoreFiles may
> still be under the minimum size and require further compaction. If this
> parameter is lowered, the ratio check is triggered more quickly. This
> addressed some issues seen in earlier versions of HBase but changing this
> parameter is no longer necessary in most situations. Default: 128 MB
> expressed in bytes.
> Default
> 134217728
> hbase.hstore.compaction.max.size
> Description
> A StoreFile larger than this size will be excluded from compaction. The
> effect of raising hbase.hstore.compaction.max.size is fewer, larger
> StoreFiles that do not get compacted often. If you feel that compaction is
> happening too often without much benefit, you can try raising this value.
> Default: the value of LONG.MAX_VALUE, expressed in bytes.
> hbase.hstore.compaction.ratio
> Description
> For minor compaction, this ratio is used to determine whether a given
> StoreFile which is larger than hbase.hstore.compaction.min.size is eligible
> for compaction. Its effect is to limit compaction of large StoreFiles. The
> value of hbase.hstore.compaction.ratio is expressed as a floating-point
> decimal. A large ratio, such as 10, will produce a single giant StoreFile.
> Conversely, a low value, such as .25, will produce behavior similar to the
> BigTable compaction algorithm, producing four StoreFiles. A moderate value of
> between 1.0 and 1.4 is recommended. When tuning this value, you are balancing
> write costs with read costs. Raising the value (to something like 1.4) will
> have more write costs, because you will compact larger StoreFiles. However,
> during reads, HBase will need to seek through fewer StoreFiles to accomplish
> the read. Consider this approach if you cannot take advantage of Bloom
> filters. Otherwise, you can lower this value to something like 1.0 to reduce
> the background cost of writes, and use Bloom filters to control the number of
> StoreFiles touched during reads. For most cases, the default value is
> appropriate.
> Default
> 1.2F
> {quote}
> For details, see HBASE-14263.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14326) HBase book: fix definition of max min size to compact
[
https://issues.apache.org/jira/browse/HBASE-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Misty Stanley-Jones updated HBASE-14326:
Attachment: HBASE-14326.patch
Hopefully this patch clarifies the docs.
> HBase book: fix definition of max min size to compact
> -
>
> Key: HBASE-14326
> URL: https://issues.apache.org/jira/browse/HBASE-14326
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Vladimir Rodionov
>Assignee: Misty Stanley-Jones
> Fix For: 2.0.0
>
> Attachments: HBASE-14326.patch
>
>
> I think we need to change wording/definition of these config parameters in
> HBase book, they are misleading:
> {quote}
> hbase.hstore.compaction.min.size
> Description
> A StoreFile smaller than this size will always be eligible for minor
> compaction. HFiles this size or larger are evaluated by
> hbase.hstore.compaction.ratio to determine if they are eligible. Because this
> limit represents the "automatic include"limit for all StoreFiles smaller than
> this value, this value may need to be reduced in write-heavy environments
> where many StoreFiles in the 1-2 MB range are being flushed, because every
> StoreFile will be targeted for compaction and the resulting StoreFiles may
> still be under the minimum size and require further compaction. If this
> parameter is lowered, the ratio check is triggered more quickly. This
> addressed some issues seen in earlier versions of HBase but changing this
> parameter is no longer necessary in most situations. Default: 128 MB
> expressed in bytes.
> Default
> 134217728
> hbase.hstore.compaction.max.size
> Description
> A StoreFile larger than this size will be excluded from compaction. The
> effect of raising hbase.hstore.compaction.max.size is fewer, larger
> StoreFiles that do not get compacted often. If you feel that compaction is
> happening too often without much benefit, you can try raising this value.
> Default: the value of LONG.MAX_VALUE, expressed in bytes.
> hbase.hstore.compaction.ratio
> Description
> For minor compaction, this ratio is used to determine whether a given
> StoreFile which is larger than hbase.hstore.compaction.min.size is eligible
> for compaction. Its effect is to limit compaction of large StoreFiles. The
> value of hbase.hstore.compaction.ratio is expressed as a floating-point
> decimal. A large ratio, such as 10, will produce a single giant StoreFile.
> Conversely, a low value, such as .25, will produce behavior similar to the
> BigTable compaction algorithm, producing four StoreFiles. A moderate value of
> between 1.0 and 1.4 is recommended. When tuning this value, you are balancing
> write costs with read costs. Raising the value (to something like 1.4) will
> have more write costs, because you will compact larger StoreFiles. However,
> during reads, HBase will need to seek through fewer StoreFiles to accomplish
> the read. Consider this approach if you cannot take advantage of Bloom
> filters. Otherwise, you can lower this value to something like 1.0 to reduce
> the background cost of writes, and use Bloom filters to control the number of
> StoreFiles touched during reads. For most cases, the default value is
> appropriate.
> Default
> 1.2F
> {quote}
> For details, see HBASE-14263.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14326) HBase book: fix definition of max min size to compact
[
https://issues.apache.org/jira/browse/HBASE-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Misty Stanley-Jones updated HBASE-14326:
Status: Patch Available (was: Open)
> HBase book: fix definition of max min size to compact
> -
>
> Key: HBASE-14326
> URL: https://issues.apache.org/jira/browse/HBASE-14326
> Project: HBase
> Issue Type: Bug
> Components: documentation
>Reporter: Vladimir Rodionov
>Assignee: Misty Stanley-Jones
> Fix For: 2.0.0
>
> Attachments: HBASE-14326.patch
>
>
> I think we need to change wording/definition of these config parameters in
> HBase book, they are misleading:
> {quote}
> hbase.hstore.compaction.min.size
> Description
> A StoreFile smaller than this size will always be eligible for minor
> compaction. HFiles this size or larger are evaluated by
> hbase.hstore.compaction.ratio to determine if they are eligible. Because this
> limit represents the "automatic include"limit for all StoreFiles smaller than
> this value, this value may need to be reduced in write-heavy environments
> where many StoreFiles in the 1-2 MB range are being flushed, because every
> StoreFile will be targeted for compaction and the resulting StoreFiles may
> still be under the minimum size and require further compaction. If this
> parameter is lowered, the ratio check is triggered more quickly. This
> addressed some issues seen in earlier versions of HBase but changing this
> parameter is no longer necessary in most situations. Default: 128 MB
> expressed in bytes.
> Default
> 134217728
> hbase.hstore.compaction.max.size
> Description
> A StoreFile larger than this size will be excluded from compaction. The
> effect of raising hbase.hstore.compaction.max.size is fewer, larger
> StoreFiles that do not get compacted often. If you feel that compaction is
> happening too often without much benefit, you can try raising this value.
> Default: the value of LONG.MAX_VALUE, expressed in bytes.
> hbase.hstore.compaction.ratio
> Description
> For minor compaction, this ratio is used to determine whether a given
> StoreFile which is larger than hbase.hstore.compaction.min.size is eligible
> for compaction. Its effect is to limit compaction of large StoreFiles. The
> value of hbase.hstore.compaction.ratio is expressed as a floating-point
> decimal. A large ratio, such as 10, will produce a single giant StoreFile.
> Conversely, a low value, such as .25, will produce behavior similar to the
> BigTable compaction algorithm, producing four StoreFiles. A moderate value of
> between 1.0 and 1.4 is recommended. When tuning this value, you are balancing
> write costs with read costs. Raising the value (to something like 1.4) will
> have more write costs, because you will compact larger StoreFiles. However,
> during reads, HBase will need to seek through fewer StoreFiles to accomplish
> the read. Consider this approach if you cannot take advantage of Bloom
> filters. Otherwise, you can lower this value to something like 1.0 to reduce
> the background cost of writes, and use Bloom filters to control the number of
> StoreFiles touched during reads. For most cases, the default value is
> appropriate.
> Default
> 1.2F
> {quote}
> For details, see HBASE-14263.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HBASE-14326) HBase book: fix definition of max min size to compact
[
https://issues.apache.org/jira/browse/HBASE-14326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladimir Rodionov updated HBASE-14326:
--
Description:
I think we need to change wording/definition of these config parameters in
HBase book, they are misleading:
{quote}
hbase.hstore.compaction.min.size
Description
A StoreFile smaller than this size will always be eligible for minor
compaction. HFiles this size or larger are evaluated by
hbase.hstore.compaction.ratio to determine if they are eligible. Because this
limit represents the automatic includelimit for all StoreFiles smaller than
this value, this value may need to be reduced in write-heavy environments where
many StoreFiles in the 1-2 MB range are being flushed, because every StoreFile
will be targeted for compaction and the resulting StoreFiles may still be under
the minimum size and require further compaction. If this parameter is lowered,
the ratio check is triggered more quickly. This addressed some issues seen in
earlier versions of HBase but changing this parameter is no longer necessary in
most situations. Default: 128 MB expressed in bytes.
Default
134217728
hbase.hstore.compaction.max.size
Description
A StoreFile larger than this size will be excluded from compaction. The effect
of raising hbase.hstore.compaction.max.size is fewer, larger StoreFiles that do
not get compacted often. If you feel that compaction is happening too often
without much benefit, you can try raising this value. Default: the value of
LONG.MAX_VALUE, expressed in bytes.
hbase.hstore.compaction.ratio
Description
For minor compaction, this ratio is used to determine whether a given StoreFile
which is larger than hbase.hstore.compaction.min.size is eligible for
compaction. Its effect is to limit compaction of large StoreFiles. The value of
hbase.hstore.compaction.ratio is expressed as a floating-point decimal. A large
ratio, such as 10, will produce a single giant StoreFile. Conversely, a low
value, such as .25, will produce behavior similar to the BigTable compaction
algorithm, producing four StoreFiles. A moderate value of between 1.0 and 1.4
is recommended. When tuning this value, you are balancing write costs with read
costs. Raising the value (to something like 1.4) will have more write costs,
because you will compact larger StoreFiles. However, during reads, HBase will
need to seek through fewer StoreFiles to accomplish the read. Consider this
approach if you cannot take advantage of Bloom filters. Otherwise, you can
lower this value to something like 1.0 to reduce the background cost of writes,
and use Bloom filters to control the number of StoreFiles touched during reads.
For most cases, the default value is appropriate.
Default
1.2F
{quote}
For details, see HBASE-14263.
was:
I think we need to change wording/definition of these config parameters in
HBase book, they are misleading:
{quote}
hbase.hstore.compaction.min.size
Description
A StoreFile smaller than this size will always be eligible for minor
compaction. HFiles this size or larger are evaluated by
hbase.hstore.compaction.ratio to determine if they are eligible. Because this
limit represents the automatic includelimit for all StoreFiles smaller than
this value, this value may need to be reduced in write-heavy environments where
many StoreFiles in the 1-2 MB range are being flushed, because every StoreFile
will be targeted for compaction and the resulting StoreFiles may still be under
the minimum size and require further compaction. If this parameter is lowered,
the ratio check is triggered more quickly. This addressed some issues seen in
earlier versions of HBase but changing this parameter is no longer necessary in
most situations. Default: 128 MB expressed in bytes.
Default
134217728
hbase.hstore.compaction.max.size
Description
A StoreFile larger than this size will be excluded from compaction. The effect
of raising hbase.hstore.compaction.max.size is fewer, larger StoreFiles that do
not get compacted often. If you feel that compaction is happening too often
without much benefit, you can try raising this value. Default: the value of
LONG.MAX_VALUE, expressed in bytes.
{quote}
For details, see HBASE-14263.
HBase book: fix definition of max min size to compact
-
Key: HBASE-14326
URL: https://issues.apache.org/jira/browse/HBASE-14326
Project: HBase
Issue Type: Bug
Components: documentation
Reporter: Vladimir Rodionov
Fix For: 2.0.0
I think we need to change wording/definition of these config parameters in
HBase book, they are misleading:
{quote}
hbase.hstore.compaction.min.size
Description
A StoreFile smaller than this size will always be eligible for minor
compaction. HFiles this size or larger are evaluated by
