updating docs from master
Project: http://git-wip-us.apache.org/repos/asf/hbase/repo Commit: http://git-wip-us.apache.org/repos/asf/hbase/commit/0dab4ad2 Tree: http://git-wip-us.apache.org/repos/asf/hbase/tree/0dab4ad2 Diff: http://git-wip-us.apache.org/repos/asf/hbase/diff/0dab4ad2 Branch: refs/heads/branch-1.1 Commit: 0dab4ad2a03fb64530582329b773fa337301a7a1 Parents: 7a823b4 Author: Nick Dimiduk <ndimi...@apache.org> Authored: Sat Jun 10 17:49:35 2017 -0700 Committer: Nick Dimiduk <ndimi...@apache.org> Committed: Sat Jun 10 17:49:35 2017 -0700 ---------------------------------------------------------------------- src/main/asciidoc/_chapters/external_apis.adoc | 9 ++- src/main/asciidoc/_chapters/faq.adoc | 3 + src/main/asciidoc/_chapters/hbase_mob.adoc | 4 +- src/main/asciidoc/_chapters/ops_mgt.adoc | 80 ++++++++++++++++++++- src/main/asciidoc/_chapters/orca.adoc | 6 +- src/main/asciidoc/_chapters/upgrading.adoc | 2 +- 6 files changed, 96 insertions(+), 8 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hbase/blob/0dab4ad2/src/main/asciidoc/_chapters/external_apis.adoc ---------------------------------------------------------------------- diff --git a/src/main/asciidoc/_chapters/external_apis.adoc b/src/main/asciidoc/_chapters/external_apis.adoc index 556c4e0..2f85461 100644 --- a/src/main/asciidoc/_chapters/external_apis.adoc +++ b/src/main/asciidoc/_chapters/external_apis.adoc @@ -225,14 +225,17 @@ creation or mutation, and `DELETE` for deletion. |Description |Example -|/_table_/_row_/_column:qualifier_/_timestamp_ +|/_table_/_row_ |GET -|Get the value of a single row. Values are Base-64 encoded. +|Get all columns of a single row. Values are Base-64 encoded. This requires the "Accept" request header with a type that can hold multiple columns (like xml, json or protobuf). |curl -vi -X GET \ -H "Accept: text/xml" \ "http://example.com:8000/users/row1"; -curl -vi -X GET \ +|/_table_/_row_/_column:qualifier_/_timestamp_ +|GET +|Get the value of a single column. Values are Base-64 encoded. +|curl -vi -X GET \ -H "Accept: text/xml" \ "http://example.com:8000/users/row1/cf:a/1458586888395"; http://git-wip-us.apache.org/repos/asf/hbase/blob/0dab4ad2/src/main/asciidoc/_chapters/faq.adoc ---------------------------------------------------------------------- diff --git a/src/main/asciidoc/_chapters/faq.adoc b/src/main/asciidoc/_chapters/faq.adoc index 7bffe0e..9034d4b 100644 --- a/src/main/asciidoc/_chapters/faq.adoc +++ b/src/main/asciidoc/_chapters/faq.adoc @@ -44,6 +44,9 @@ How can I find examples of NoSQL/HBase?:: What is the history of HBase?:: See <<hbase.history,hbase.history>>. +Why are the cells above 10MB not recommended for HBase?:: + Large cells don't fit well into HBase's approach to buffering data. First, the large cells bypass the MemStoreLAB when they are written. Then, they cannot be cached in the L2 block cache during read operations. Instead, HBase has to allocate on-heap memory for them each time. This can have a significant impact on the garbage collector within the RegionServer process. + === Upgrading How do I upgrade Maven-managed projects from HBase 0.94 to HBase 0.96+?:: In HBase 0.96, the project moved to a modular structure. Adjust your project's dependencies to rely upon the `hbase-client` module or another module as appropriate, rather than a single JAR. You can model your Maven dependency after one of the following, depending on your targeted version of HBase. See Section 3.5, âUpgrading from 0.94.x to 0.96.xâ or Section 3.3, âUpgrading from 0.96.x to 0.98.xâ for more information. http://git-wip-us.apache.org/repos/asf/hbase/blob/0dab4ad2/src/main/asciidoc/_chapters/hbase_mob.adoc ---------------------------------------------------------------------- diff --git a/src/main/asciidoc/_chapters/hbase_mob.adoc b/src/main/asciidoc/_chapters/hbase_mob.adoc index bdf077a..5da0343 100644 --- a/src/main/asciidoc/_chapters/hbase_mob.adoc +++ b/src/main/asciidoc/_chapters/hbase_mob.adoc @@ -36,7 +36,7 @@ read and write paths are optimized for values smaller than 100KB in size. When HBase deals with large numbers of objects over this threshold, referred to here as medium objects, or MOBs, performance is degraded due to write amplification caused by splits and compactions. When using MOBs, ideally your objects will be between -100KB and 10MB. HBase ***FIX_VERSION_NUMBER*** adds support +100KB and 10MB (see the <<faq>>). HBase ***FIX_VERSION_NUMBER*** adds support for better managing large numbers of MOBs while maintaining performance, consistency, and low operational overhead. MOB support is provided by the work done in link:https://issues.apache.org/jira/browse/HBASE-11339[HBASE-11339]. To @@ -155,7 +155,7 @@ family as the second argument. and take a compaction type as the third argument. ---- hbase> compact 't1', 'c1â, âMOBâ -hbase> major_compact_mob 't1', 'c1â, âMOBâ +hbase> major_compact 't1', 'c1â, âMOBâ ---- These commands are also available via `Admin.compact` and http://git-wip-us.apache.org/repos/asf/hbase/blob/0dab4ad2/src/main/asciidoc/_chapters/ops_mgt.adoc ---------------------------------------------------------------------- diff --git a/src/main/asciidoc/_chapters/ops_mgt.adoc b/src/main/asciidoc/_chapters/ops_mgt.adoc index e4c077f..b26e44b 100644 --- a/src/main/asciidoc/_chapters/ops_mgt.adoc +++ b/src/main/asciidoc/_chapters/ops_mgt.adoc @@ -1705,7 +1705,7 @@ handling multiple workloads: [[quota]] === Quotas -HBASE-11598 introduces quotas, which allow you to throttle requests based on +HBASE-11598 introduces RPC quotas, which allow you to throttle requests based on the following limits: . <<request-quotas,The number or size of requests(read, write, or read+write) in a given timeframe>> @@ -1885,6 +1885,84 @@ at the same time and that fewer scans can be executed at the same time. A value `0.9` will give more queue/handlers to scans, so the number of scans executed will increase and the number of gets will decrease. +[[space-quotas]] +=== Space Quotas + +link:https://issues.apache.org/jira/browse/HBASE-16961[HBASE-16961] introduces a new type of +quotas for HBase to leverage: filesystem quotas. These "space" quotas limit the amount of space +on the filesystem that HBase namespaces and tables can consume. If a user, malicious or ignorant, +has the ability to write data into HBase, with enough time, that user can effectively crash HBase +(or worse HDFS) by consuming all available space. When there is no filesystem space available, +HBase crashes because it can no longer create/sync data to the write-ahead log. + +This feature allows a for a limit to be set on the size of a table or namespace. When a space quota is set +on a namespace, the quota's limit applies to the sum of usage of all tables in that namespace. +When a table with a quota exists in a namespace with a quota, the table quota takes priority +over the namespace quota. This allows for a scenario where a large limit can be placed on +a collection of tables, but a single table in that collection can have a fine-grained limit set. + +The existing `set_quota` and `list_quota` HBase shell commands can be used to interact with +space quotas. Space quotas are quotas with a `TYPE` of `SPACE` and have `LIMIT` and `POLICY` +attributes. The `LIMIT` is a string that refers to the amount of space on the filesystem +that the quota subject (e.g. the table or namespace) may consume. For example, valid values +of `LIMIT` are `'10G'`, `'2T'`, or `'256M'`. The `POLICY` refers to the action that HBase will +take when the quota subject's usage exceeds the `LIMIT`. The following are valid `POLICY` values. + +* `NO_INSERTS` - No new data may be written (e.g. `Put`, `Increment`, `Append`). +* `NO_WRITES` - Same as `NO_INSERTS` but `Deletes` are also disallowed. +* `NO_WRITES_COMPACTIONS` - Same as `NO_WRITES` but compactions are also disallowed. +* `DISABLE` - The table(s) are disabled, preventing all read/write access. + +.Setting simple space quotas +---- +# Sets a quota on the table 't1' with a limit of 1GB, disallowing Puts/Increments/Appends when the table exceeds 1GB +hbase> set_quota TYPE => SPACE, TABLE => 't1', LIMIT => '1G', POLICY => NO_INSERTS + +# Sets a quota on the namespace 'ns1' with a limit of 50TB, disallowing Puts/Increments/Appends/Deletes +hbase> set_quota TYPE => SPACE, NAMESPACE => 'ns1', LIMIT => '50T', POLICY => NO_WRITES + +# Sets a quota on the table 't3' with a limit of 2TB, disallowing any writes and compactions when the table exceeds 2TB. +hbase> set_quota TYPE => SPACE, TABLE => 't3', LIMIT => '2T', POLICY => NO_WRITES_COMPACTIONS + +# Sets a quota on the table 't2' with a limit of 50GB, disabling the table when it exceeds 50GB +hbase> set_quota TYPE => SPACE, TABLE => 't2', LIMIT => '50G', POLICY => DISABLE +---- + +Consider the following scenario to set up quotas on a namespace, overriding the quota on tables in that namespace + +.Table and Namespace space quotas +---- +hbase> create_namespace 'ns1' +hbase> create 'ns1:t1' +hbase> create 'ns1:t2' +hbase> create 'ns1:t3' +hbase> set_quota TYPE => SPACE, NAMESPACE => 'ns1', LIMIT => '100T', POLICY => NO_INSERTS +hbase> set_quota TYPE => SPACE, TABLE => 'ns1:t2', LIMIT => '200G', POLICY => NO_WRITES +hbase> set_quota TYPE => SPACE, TABLE => 'ns1:t3', LIMIT => '20T', POLICY => NO_WRITES +---- + +In the above scenario, the tables in the namespace `ns1` will not be allowed to consume more than +100TB of space on the filesystem among each other. The table 'ns1:t2' is only allowed to be 200GB in size, and will +disallow all writes when the usage exceeds this limit. The table 'ns1:t3' is allowed to grow to 20TB in size +and also will disallow all writes then the usage exceeds this limit. Because there is no table quota +on 'ns1:t1', this table can grow up to 100TB, but only if 'ns1:t2' and 'ns1:t3' have a usage of zero bytes. +Practically, it's limit is 100TB less the current usage of 'ns1:t2' and 'ns1:t3'. + +[[ops.space.quota.deletion]] +=== Disabling Automatic Space Quota Deletion + +By default, if a table or namespace is deleted that has a space quota, the quota itself is +also deleted. In some cases, it may be desirable for the space quota to not be automatically deleted. +In these cases, the user may configure the system to not delete any space quota automatically via hbase-site.xml. + +[source,java] +---- + + <property> + <name>hbase.master.quota.observer.ignore</name> + <value>true</value> + </property> +---- [[ops.backup]] == HBase Backup http://git-wip-us.apache.org/repos/asf/hbase/blob/0dab4ad2/src/main/asciidoc/_chapters/orca.adoc ---------------------------------------------------------------------- diff --git a/src/main/asciidoc/_chapters/orca.adoc b/src/main/asciidoc/_chapters/orca.adoc index 1816b1a..0ba65ba 100644 --- a/src/main/asciidoc/_chapters/orca.adoc +++ b/src/main/asciidoc/_chapters/orca.adoc @@ -28,11 +28,15 @@ :icons: font :experimental: -.Apache HBase Orca +.Apache HBase Orca, HBase Colors, & Font image::jumping-orca_rotated_25percent.png[] link:https://issues.apache.org/jira/browse/HBASE-4920[An Orca is the Apache HBase mascot.] See NOTICES.txt. Our Orca logo we got here: http://www.vectorfree.com/jumping-orca It is licensed Creative Commons Attribution 3.0. See https://creativecommons.org/licenses/by/3.0/us/ We changed the logo by stripping the colored background, inverting it and then rotating it some. +The 'official' HBase color is "International Orange (Engineering)", the color of the link:https://en.wikipedia.org/wiki/International_orange[Golden Gate bridge] in San Francisco and for space suits used by NASA. + +Our 'font' is link:http://www.dafont.com/bitsumishi.font[Bitsumishi]. + :numbered: http://git-wip-us.apache.org/repos/asf/hbase/blob/0dab4ad2/src/main/asciidoc/_chapters/upgrading.adoc ---------------------------------------------------------------------- diff --git a/src/main/asciidoc/_chapters/upgrading.adoc b/src/main/asciidoc/_chapters/upgrading.adoc index 7210040..9984cf5 100644 --- a/src/main/asciidoc/_chapters/upgrading.adoc +++ b/src/main/asciidoc/_chapters/upgrading.adoc @@ -127,7 +127,7 @@ for warning about incompatible changes). All effort will be made to provide a de HBase has a lot of API points, but for the compatibility matrix above, we differentiate between Client API, Limited Private API, and Private API. HBase uses a version of link:https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html[Hadoop's Interface classification]. HBase's Interface classification classes can be found link:https://hbase.apache.org/apidocs/org/apache/hadoop/hbase/classification/package-summary.html[here]. -* InterfaceAudience: captures the intended audience, possible values are Public (for end users and external projects), LimitedPrivate (for other Projects, Coprocessors or other plugin points), and Private (for internal use). +* InterfaceAudience: captures the intended audience, possible values are Public (for end users and external projects), LimitedPrivate (for other Projects, Coprocessors or other plugin points), and Private (for internal use). Notice that, you may find that the classes which are declared as IA.Private are used as parameter or return value for the interfaces which are declared as IA.LimitedPrivate. This is possible. You should treat the IA.Private object as a monolithic object, which means you can use it as a parameter to call other methods, or return it, but you should never try to access its methods or fields. * InterfaceStability: describes what types of interface changes are permitted. Possible values are Stable, Evolving, Unstable, and Deprecated. Notice that this annotation is only valid for classes which are marked as IA.LimitedPrivate. The stability of IA.Public classes is only related to the upgrade type(major, minor or patch). And for IA.Private classes, there is no guarantee on the stability between releases. Refer to the Compatibility Matrix above for more details. [[hbase.client.api]]
