[CONF] Apache Camel > hbase
Christian Mueller edited the page:
hbase
...
Code Block
xml
${in.body.id}
${in.body.value}
opertaionoperation=CamelHBasePut&family=myfamily&qualifier=myqualifier"/>
...
Code Block
xml
${in.body.id}
${in.body.id}
${in.body.value}
${in.body.othervalue}
opertaionoperation=CamelHBasePut&family=myfamily&qualifier=myqualifier&family2=myfamily&qualifier2=myqualifier2"/>
[CONF] Apache Camel > hbase
willem jiang edited the page:
hbase
Comment: CAMEL-7296
...
Wiki Markup
{div:class=confluenceTableSmall}
|| Name || Default Value || Description ||
| {{initialDelay}} | {{1000}} | Milliseconds before the first polling starts. |
| {{delay}} | {{500}} | Milliseconds before the next poll. |
| {{useFixedDelay}} | {{true}} | Controls if fixed delay or fixed rate is used. See [ScheduledExecutorService|http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/ScheduledExecutorService.html] in JDK for details. |
| timeUnit | {{TimeUnit.MILLISECONDS}} | time unit for {{initialDelay}} and {{delay}} options. |
| {{runLoggingLevel}} | {{TRACE}} | *Camel 2.8:* The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that. |
| {{operation}} | {{CamelHBasePut}} | The HBase operation to perform. *Supported values*: {{CamelHBasePut}}, {{CamelHBaseGet}}, {{CamelHBaseDelete}}, and {{CamelHBaseScan}}. |
| {{maxResults}} | {{100}} | The maximum number of rows to scan. *Supported operations:* {{CamelHBaseScan}}. |
| {{mappingStrategyName}} | {{header}} | The strategy to use for mapping Camel messages to HBase columns. Supported values: {{header}}, or {{body}}. |
| {{mappingStrategyClassName}} | {{null}} | The class name of a custom mapping strategy implementation. |
| {{filters}} | {{null}} | A list of filters. *Supported operations*: {{CamelHBaseScan}} |
| {{remove}} | {{true}} | If the option is true, Camel HBase Consumer will remove the rows which it processes.|
{div}
Header mapping options:
...
View Online · Like · View Changes
Stop watching space · Manage Notifications
This message was sent by Atlassian Confluence 5.0.3, Team Collaboration Software
[CONF] Apache Camel > hbase
hbase
Page edited by Christian Mueller
Changes (1)
...
*Available as of Camel 2.10*
This component provides an idemptotent repository, producers and consumers for Apache HBase. [Apache HBase|http://hbase.apache.org/].
Maven users will need to add the following dependency to their {{pom.xml}} for this component:
...
Full Content
HBase Component
Available as of Camel 2.10
This component provides an idemptotent repository, producers and consumers for Apache HBase.
Maven users will need to add the following dependency to their pom.xml for this component:
org.apache.camel
camel-hbase
x.x.x
Apache HBase Overview
HBase is an open-source, distributed, versioned, column-oriented store modeled after Google's Bigtable: A Distributed Storage System for Structured Data. You can use HBase when you need random, realtime read/write access to your Big Data. More information at Apache HBase.
Camel and HBase
When using a datasotre inside a camel route, there is always the chalenge of specifying how the camel message will stored to the datastore. In document based stores things are more easy as the message body can be directly mapped to a document. In relational databases an ORM solution can be used to map properties to columns etc. In column based stores things are more challenging as there is no standard way to perform that kind of mapping.
HBase adds two additional challenges:
HBase groups columns into families, so just mapping a property to a column using a name convention is just not enough.
HBase doesn't have the notion of type, which means that it stores everything as byte[] and doesn't know if the byte[] represents a String, a Number, a serialized Java object or just binary data.
To overcome these challenges, camel-hbase makes use of the message headers to specify the mapping of the message to HBase columns. It also provides the ability to use some camel-hbase provided classes that model HBase data and can be easily convert to and from xml/json etc.
Finally it provides the ability to the user to implement and use his own mapping strategy.
Regardless of the mapping strategy camel-hbase will convert a message into an org.apache.camel.component.hbase.model.HBaseData object and use that object for its internal operations.
Configuring the component
The HBase component can be provided a custom HBaseConfiguration object as a property or it can create an HBase configuration object on its own based on the HBase related resources that are found on classpath.
If no configuration object is provided to the component, the component will create one. The created configuration will search the class path for an hbase-site.xml file, from which it will draw the configuration. You can find more information about how to configure HBase clients at: HBase client configuration and dependencies
HBase Producer
As mentioned above camel provides produers endpoints for HBase. This allows you to store, delete, retrieve or query data from HBase using your camel routes.
hbase://table[?options]
where table is the table name.
The supported operations are:
Put
Get
Delete
Scan
Supported URI options on producer
Name
Default Value
Description
operation
CamelHBasePut
The HBase operation to perform. Supported values: CamelHBasePut, CamelHBaseGet, CamelHBaseDelete, and CamelHBaseScan.
maxResults
100
The maximum number of rows to scan.Supported operations: CamelHBaseScan.
mappingStrategyName
header
The strategy to use for mapping Camel messages to HBase columns. Supported values: header, or body.
mappingStrategyClassName
null
The class name of a custom mapping strategy implementation.
filters
null
A list of filters. Supported operations: CamelHBaseScan.
Header mapping options:
Name
Default Value
Description
rowId
The id of the row. This has limited use as the row usually changes per Exchange.
rowType
String
The type to covert row id to. Supported operations: CamelHBaseScan.
family
The column family. Supports a number suffix for referring to more than one columns
qualifier
The column qualifier. Supports a number suffix for referring to more than one columns
value
The value. Supports a number suffix for referring to more than one columns
valueType
String
The value type. Supports a number suffix for referring to more than one columns. Supported operations: CamelHBaseGet, and CamelHBaseScan.
Put Operations.
HBase is a column based store, which allows you to store data into a specific column of a specific row. Columns are grouped into families, so in order to specify a column you need to specify th
[CONF] Apache Camel > hbase
hbase
Page edited by Claus Ibsen
Changes (2)
...
{div:class=confluenceTableSmall} || Name || Default Value || Description ||
| {{initialDelay}} | {{1000}} | Milliseconds before the first polling starts. | | {{delay}} | {{500}} | Milliseconds before the next poll. | | {{useFixedDelay}} | {{true}} | Controls if fixed delay or fixed rate is used. See [ScheduledExecutorService|http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/ScheduledExecutorService.html] in JDK for details. | | timeUnit | {{TimeUnit.MILLISECONDS}} | time unit for {{initialDelay}} and {{delay}} options. | | {{runLoggingLevel}} | {{TRACE}} | *Camel 2.8:* The consumer logs a start/complete log line when it polls. This option allows you to configure the logging level for that. |
| {{operation}} | {{CamelHBasePut}} | The HBase operation to perform. *Supported values*: {{CamelHBasePut}}, {{CamelHBaseGet}}, {{CamelHBaseDelete}}, and {{CamelHBaseScan}}. | | {{maxResults}} | {{100}} | The maximum number of rows to scan. *Supported operations:* {{CamelHBaseScan}}. |
...
h3. See also
- [Polling Consumer]
- [Apache HBase|http://hbase.apache.org]
Full Content
HBase Component
Available as of Camel 2.10
This component provides an idemptotent repository, producers and consumers for Apache HBase.
Maven users will need to add the following dependency to their pom.xml for this component:
org.apache.camel
camel-hbase
x.x.x
Apache HBase Overview
HBase is an open-source, distributed, versioned, column-oriented store modeled after Google's Bigtable: A Distributed Storage System for Structured Data. You can use HBase when you need random, realtime read/write access to your Big Data. More information at Apache HBase.
Camel and HBase
When using a datasotre inside a camel route, there is always the chalenge of specifying how the camel message will stored to the datastore. In document based stores things are more easy as the message body can be directly mapped to a document. In relational databases an ORM solution can be used to map properties to columns etc. In column based stores things are more challenging as there is no standard way to perform that kind of mapping.
HBase adds two additional challenges:
HBase groups columns into families, so just mapping a property to a column using a name convention is just not enough.
HBase doesn't have the notion of type, which means that it stores everything as byte[] and doesn't know if the byte[] represents a String, a Number, a serialized Java object or just binary data.
To overcome these challenges, camel-hbase makes use of the message headers to specify the mapping of the message to HBase columns. It also provides the ability to use some camel-hbase provided classes that model HBase data and can be easily convert to and from xml/json etc.
Finally it provides the ability to the user to implement and use his own mapping strategy.
Regardless of the mapping strategy camel-hbase will convert a message into an org.apache.camel.component.hbase.model.HBaseData object and use that object for its internal operations.
Configuring the component
The HBase component can be provided a custom HBaseConfiguration object as a property or it can create an HBase configuration object on its own based on the HBase related resources that are found on classpath.
"hbase" class="org.apache.camel.component.hbase.HBaseComponent">
"configuration" ref="config"/>
If no configuration object is provided to the component, the component will create one. The created configuration will search the class path for an hbase-site.xml file, from which it will draw the configuration. You can find more information about how to configure HBase clients at: HBase client configuration and dependencies
HBase Producer
As mentioned above camel provides produers endpoints for HBase. This allows you to store, delete, retrieve or query data from HBase using your camel routes.
hbase://table[?options]
where table is the table name.
The supported operations are:
Put
Get
Delete
Scan
Supported URI options on producer
Name
Default Value
Description
operation
CamelHBasePut
The HBase operation to perform. Supported values: CamelHBasePut, CamelHBaseGet, CamelHBaseDelete, and CamelHBaseScan.
maxResults
100
The maximum number of rows to scan.Supported operations: CamelHBaseScan.
mappingStrategyName
header
The strategy to use for mapping Camel messages to HBase columns. Supported values: header, or body.
mappingStrategyClassName
null
The class name of a custom mapping strategy implementation.
filters
null
A list of filters. Supported operations
[CONF] Apache Camel > hbase
hbase
Page edited by Claus Ibsen
Changes (14)
...
h3. URI format
h3. HBase Producer
As mentioned above camel provides produers endpoints for HBase. This allows you to store, delete, retrieve or query data from HBase using your camel routes.
...
where *table* is the table name.
h4. Producer
The supported operations are: - Put
...
- Scan
h4. Supported URI options on producer
{div:class=confluenceTableSmall}
...
{code}
hbase:[table]
hbase://table[?options]
{code}
...
The example above will create a model object that is consisted of the specified fields and the scan results will populate the model object with values. Finally the mapping strategy will be used to map this model to the camel message.
h4. Supported URI options on consumer
{div:class=confluenceTableSmall} || Name || Default Value || Description ||
| {{operation}} | CamelHBasePut | The HBase operation to perform. *Supported values*: CamelHbasePut, CamelHBaseGet, CamelHBaseDelete, CamelHBaseScan | | {{maxResults}} | 100 | The maximum number of rows to scan. (CamelHBaseScanOnly)|
| {{operation}} | {{CamelHBasePut}} | The HBase operation to perform. *Supported values*: {{CamelHBasePut}}, {{CamelHBaseGet}}, {{CamelHBaseDelete}}, and {{CamelHBaseScan}}. | | {{maxResults}} | {{100}} | The maximum number of rows to scan. *Supported operations:* {{CamelHBaseScan}}. |
| {{mappingStrategyName}} | {{header}} | The strategy to use for mapping Camel messages to HBase columns. Supported values: header, body| {{header}}, or {{body}}. |
| {{mappingStrategyClassName}} | {{null}} | The class name of a custom mapping strategy implementation. |
| {{filters}} | {{null}} | A list of filters. *Supported operations*: {{CamelHBaseScan}} |
{div}
...
HBaseConfiguration configuration = HBaseConfiguration.create(); HBaseIdempotentRepository repository = new HBaseIdempotentRepository(configuration, tableName, family, qualifier);
from("direct:in").idempotentConsumer(header("messageId"), repository).to("log:out);
from("direct:in") .idempotentConsumer(header("messageId"), repository) .to("log:out);
{code}
...
Full Content
HBase Component
Available as of Camel 2.10
This component provides an idemptotent repository, producers and consumers for Apache HBase.
Maven users will need to add the following dependency to their pom.xml for this component:
org.apache.camel
camel-hbase
x.x.x
Apache HBase Overview
HBase is an open-source, distributed, versioned, column-oriented store modeled after Google's Bigtable: A Distributed Storage System for Structured Data. You can use HBase when you need random, realtime read/write access to your Big Data. More information at Apache HBase.
Camel and HBase
When using a datasotre inside a camel route, there is always the chalenge of specifying how the camel message will stored to the datastore. In document based stores things are more easy as the message body can be directly mapped to a document. In relational databases an ORM solution can be used to map properties to columns etc. In column based stores things are more challenging as there is no standard way to perform that kind of mapping.
HBase adds two additional challenges:
HBase groups columns into families, so just mapping a property to a column using a name convention is just not enough.
HBase doesn't have the notion of type, which means that it stores everything as byte[] and doesn't know if the byte[] represents a String, a Number, a serialized Java object or just binary data.
To overcome these challenges, camel-hbase makes use of the message headers to specify the mapping of the message to HBase columns. It also provides the ability to use some camel-hbase provided classes that model HBase data and can be easily convert to and from xml/json etc.
Finally it provides the ability to the user to implement and use his own mapping strategy.
Regardless of the mapping strategy camel-hbase will convert a message into an org.apache.camel.component.hbase.model.HBaseData object and use that object for its internal operations.
Configuring the component
The HBase component can be provided a custom HBaseConfiguration object as a property or it can create an HBase configuration object on its own based on the HBase related resources that are found on classpath.
"hbase" class="org.apache.camel.component.hbase.HBaseCom
[CONF] Apache Camel > hbase
hbase
Page edited by Claus Ibsen
Changes (15)
...
${in.body.value}
uri="hbase:mytable?opertaion=CamelHBasePut&family=myfamily&qualifier=myqualifier"/>
{code}
...
${in.body.othervalue}
uri="hbase:mytable?opertaion=CamelHBasePut&family=myfamily&qualifier=myqualifier&family2=myfamily&qualifier2=myqualifier2"/>
{code}
...
${in.body.id}
uri="hbase:mytable?opertaion=CamelHBaseGet&family=myfamily&qualifier=myqualifier&valueType=java.lang.Long"/>
...
${in.body.id}
uri="hbase:mytable?opertaion=CamelHBaseDelete"/>
{code}
...
uri="hbase:mytable?opertaion=CamelHBaseScan&family=myfamily&qualifier=myqualifier&valueType=java.lang.Long&rowType=java.lang.String"/>
...
country
uri="hbase:mytable?opertaion=CamelHBaseScan&filters=#myFilterList"/>
...
| {{mappingStrategyClassName}} | | The class name of a custom mapping strategy implementation| | {{filters}} | | A list of filters. *Supported operations*: CamelHBaseScan |
{div}
Header mapping options:
{div:class=confluenceTableSmall}
|| Name || Default Value || Description || | {{rowId}} | | The id of the row. This has limited use as the row usually changes per Exchange.|
...
| {{value}} | | The value. Supports a number suffix for referring to more than one columns | | {{rowModel}} | String | An instance of org.apache.camel.component.hbase.model.HBaseRow which describes how each row should be modeled |
{div}
If the role of the rowModel is not clear, it allows you to construct the HBaseRow modle programmatically instead of "describing" it with uri options (such as family, qualifier, type etc).
...
To put the value "myvalue" into HBase row "myrow" and column "myfamily:mycolum" the message should contain the following headers:
{div:class=confluenceTableSmall}
|| Header || Value || | CamelHBaseRowId | myrow |
...
| CamelHBaseQualifier | myqualifier | | CamelHBaseValue | myvalue |
{div}
To put more values for different columns and / or different rows you can specify additional headers suffixed with the index of the headers, e.g:
{div:class=confluenceTableSmall}
|| Header || Value || | CamelHBaseRowId | myrow |
...
| CamelHBaseQualifier2 | myqualifier | | CamelHBaseValue2 | myvalue2 |
{div}
In the case of retrieval operations such as get or scan you can also specify for each column the type that you want the data to be converted to. For exampe:
{div:class=confluenceTableSmall}
|| Header || Value || | CamelHBaseFamily | myfamily | | CamelHBaseQualifier | myqualifier | | CamelHBaseValueType | Long |
{div}
Please note that in order to avoid boilerplate headers that are considered constant for all messages, you can also specify them as part of the endpoint uri, as you will see below.
...
Full Content
HBase Component
Available as of Camel 2.10
This component provides an idemptotent repository, producers and consumers for Apache HBase.
Maven users will need to add the following dependency to their pom.xml for this component:
org.apache.camel
camel-hbase
x.x.x
Apache HBase Overview
HBase is an open-source, distributed, versioned, column-oriented store modeled after Google's Bigtable: A Distributed Storage System for Structured Data. You can use HBase when you need random, realtime read/write access to your Big Data. More information at Apache HBase.
Camel and HBase
When using a datasotre inside a camel route, there is always the chalenge of specifying how the camel message will stored to the datastore. In document based stores things are more easy as the message body can be directly mapped to a document. In relational databases an ORM solution can be used to map properties to columns etc. In column based stores things are more challenging as there is no standard way to perform that kind of mapping.
HBase adds two additional challenges:
HBase groups columns into families, so just mapping a property to a column using a name co
[CONF] Apache Camel > hbase
hbase
Page added by Ioannis Canellos
HBase Component
Available as of Camel 2.10
This component provides an idemptotent repository, producers and consumers for Apache HBase.
Maven users will need to add the following dependency to their pom.xml for this component:
org.apache.camel
camel-hbase
x.x.x
Apache HBase Overview
HBase is an open-source, distributed, versioned, column-oriented store modeled after Google's Bigtable: A Distributed Storage System for Structured Data. You can use HBase when you need random, realtime read/write access to your Big Data. More information at Apache HBase.
Camel and HBase
When using a datasotre inside a camel route, there is always the chalenge of specifying how the camel message will stored to the datastore. In document based stores things are more easy as the message body can be directly mapped to a document. In relational databases an ORM solution can be used to map properties to columns etc. In column based stores things are more challenging as there is no standard way to perform that kind of mapping.
HBase adds two additional challenges:
HBase groups columns into families, so just mapping a property to a column using a name convention is just not enough.
HBase doesn't have the notion of type, which means that it stores everything as byte[] and doesn't know if the byte[] represents a String, a Number, a serialized Java object or just binary data.
To overcome these challenges, camel-hbase makes use of the message headers to specify the mapping of the message to HBase columns. It also provides the ability to use some camel-hbase provided classes that model HBase data and can be easily convert to and from xml/json etc.
Finally it provides the ability to the user to implement and use his own mapping strategy.
Regardless of the mapping strategy camel-hbase will convert a message into an org.apache.camel.component.hbase.model.HBaseData object and use that object for its internal operations.
Configuring the component
The HBase component can be provided a custom HBaseConfiguration object as a property or it can create an HBase configuration object on its own based on the HBase related resources that are found on classpath.
"hbase" class="org.apache.camel.component.hbase.HBaseComponent">
"configuration" ref="config"/>
If no configuration object is provided to the component, the component will create one. The created configuration will search the class path for an hbase-site.xml file, from which it will draw the configuration. You can find more information about how to configure HBase clients at: HBase client configuration and dependencies
HBase Producer
As mentioned above camel provides produers endpoints for HBase. This allows you to store, delete, retrieve or query data from HBase using your camel routes.
hbase:[table]
The supported operations are:
Put
Get
Delete
Scan
Supported URI options
Name
Default Value
Description
operation
CamelHBasePut
The HBase operation to perform. Supported values: CamelHbasePut, CamelHBaseGet, CamelHBaseDelete, CamelHBaseScan
maxResults
100
The maximum number of rows to scan.Supported operations: CamelHBaseScan
mappingStrategyName
header
The strategy to use for mapping Camel messages to HBase columns. Supported values: header, body
mappingStrategyClassName
The class name of a custom mapping strategy implementation
filters
A list of filters. Supported operations: CamelHBaseScan
Header mapping options:
Name
Default Value
Description
rowId
The id of the row. This has limited use as the row usually changes per Exchange.
rowType
String
The type to covert row id to. Supported operations: CamelHBaseScan
family
The column family. Supports a number suffix for referring to more than one columns
qualifier
The column qualifier. Supports a number suffix for referring to more than one columns
value
The value. Supports a number suffix for referring to more than one columns
valueType
String
The value type. Supported operations: CamelHBaseGet & CamelHBaseScan. Supports a number suffix for referring to more than one columns
Put Operations.
HBase is a column based store, which allows you to store data into a specific column of a specific row. Columns are grouped into families, so in order to specify a column you need to specify the column family and the qualifier of that column. To store data into a specific column you need to specify both the column and the row.
The simplest scenario for storing data into HBase from a camel route, would be to store part of the message body to specified HBase column.
"direct:in"/>
"CamelHBaseRowId">
${in.body.id}
"CamelHBaseValue">
${in.body.value}
