Hello Srinivasan, We need create HBaseStorage. Created https://issues.apache.org/jira/browse/LENS-612 for tracking. HBaseStorage needs to be implemented on the similar lines of https://github.com/apache/incubator-lens/blob/master/lens-storage-db/src/main/java/org/apache/lens/storage/db/DBStorage.java
Once we have HBaseStorage created, you can create your tables like the following, with HBaseStorageHandler and its properties: Create your storage like the following ( https://github.com/apache/incubator-lens/blob/master/lens-examples/src/main/resources/db-storage.xml ) : ---- <x_storage classname="org.apache.lens.storage.db.DBStorage" name="mydb" xmlns="uri:lens:cube:0.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="uri:lens:cube:0.1 cube-0.1.xsd "> <properties/> </x_storage> ---- And tables similar to https://github.com/apache/incubator-lens/blob/master/lens-examples/src/main/resources/city_subset.xml ---- <x_dimension_table dimension_name="city" table_name="city_subset" weight="100.0" xmlns="uri:lens:cube:0.1" xmlns:xsi=" http://www.w3.org/2001/XMLSchema-instance"; xsi:schemaLocation="uri:lens:cube:0.1 cube-0.1.xsd "> <columns> <column comment="ID" name="id" type="INT"/> <column comment="name" name="name" type="STRING"/> </columns> <storage_tables> <storage_table> <storage_name>mydb</storage_name> <table_desc external="true" field_delimiter="," table_location="/tmp/db-storage.db" storage_handler_name="org.apache.lens.storage.db.DBStorageHandler"> <table_parameters> <property name="lens.metastore.native.db.name" value="default"/> </table_parameters> </table_desc> </storage_table> </storage_tables> </x_dimension_table> ---- Writing HBaseStorage class should be simple, do you want to give it a try, as no special handling is required for HBase ? On Tue, Jun 16, 2015 at 6:52 PM, Srinivasan Hariharan02 < [email protected]> wrote: > Thanks Amareshwari. This partly answers my question. My need is, I have > an external dim table in hive which uses Hbase storage handler. So the > data for this table is located in Hbase, not directly under hdfs file > location. So how can I create dimension for these kind of tables in Lens > without moving the data to HDFS. > > > > *Regards,* > > *Srinivasan Hariharan* > > *Mob +91-9940395830 <%2B91-9940395830>* > > > > *From:* amareshwarisr . [mailto:[email protected]] > *Sent:* Tuesday, June 16, 2015 6:16 PM > *To:* [email protected] > *Subject:* Re: Cube Creation for Hive Tables > > > > Srinivasan Hariharan, > > > > I will try to explain how city table is created in examples. Let me see if > that helps you? > > > > If you see > https://github.com/apache/incubator-lens/blob/master/lens-examples/src/main/resources/city.xml > : > > > > ---- > > > > <x_dimension name="city" xmlns="uri:lens:cube:0.1" xmlns:xsi=" > http://www.w3.org/2001/XMLSchema-instance"; > xsi:schemaLocation="uri:lens:cube:0.1 cube-0.1.xsd "> > > <attributes> > > <dim_attribute name="id" type="INT"/> > > <dim_attribute name="name" type="STRING"/> > > <dim_attribute name="POI" type="ARRAY<STRING>" > description="Point of interests"/> > > <dim_attribute name="population" type="BIGINT" /> > > </attributes> > > <properties> > > <property name="dimension.city.timed.dimension" value="dt"/> > > </properties> > > </x_dimension> > > ---- > > > > Above of definition of city defines dimension city and its attribute. This > is only logical table, not linked to any physical table. > > > > Then we have > https://github.com/apache/incubator-lens/blob/master/lens-examples/src/main/resources/city_table.xml > - Which defines the dimtable associated with dimension city > > > > ---- > > <x_dimension_table dimension_name="city" table_name="city_table" > weight="10.0" xmlns="uri:lens:cube:0.1" xmlns:xsi=" > http://www.w3.org/2001/XMLSchema-instance"; > xsi:schemaLocation="uri:lens:cube:0.1 cube-0.1.xsd "> > > <columns> > > <column comment="ID" name="id" type="INT"/> > > <column comment="name" name="name" type="STRING"/> > > <column comment="Point of interests" name="POI" > type="ARRAY<STRING>"/> > > <column comment="city population" name="population" type="BIGINT"/> > > </columns> > > <properties> > > <property name="city.prop" value="d1"/> > > </properties> > > <storage_tables> > > <storage_table> > > <update_periods> > > <update_period>HOURLY</update_period> > > </update_periods> > > <storage_name>local</storage_name> > > <table_desc external="true" field_delimiter="," > collection_delimiter=":" table_location="/tmp/examples/city"> > > <part_cols> > > <column comment="Time column" name="dt" type="STRING"/> > > </part_cols> > > <time_part_cols>dt</time_part_cols> > > </table_desc> > > </storage_table> > > </storage_tables> > > </x_dimension_table> > > ---- > > > > Here the above definition will be mapped to table on HDFS - which is Text > formatted with delimiters specified above. > > > > Are you looking for something which uses existing tables to create facts > and dimensions - https://issues.apache.org/jira/browse/LENS-340 ? Right > now, that feature is not there. Until then you have to map lens created > table as external table to your existing table. > > > > For ex: > > You have Hive table Table1 : col1, col2; > > > > Create Dimension1 : col1, col2 as dim attributes - give schema for them. > > Create DimTable1 : col1, col2 associated with Dimension1. > > > > Then you should be good. > > > > Let us know if above answers your query. Or let us know if you are looking > for something else. > > > > Thanks > > Amareshwari > > > > > > > > On Tue, Jun 16, 2015 at 1:02 PM, Srinivasan Hariharan02 < > [email protected]> wrote: > > Hi, > > > > I have all my Dim tables in Hive, I want to create cube on top of these > tables using lens. Can anyone provide how ca I start doing that. I have > checked the examples in the resources directory under client folder. It > doesn’t help much for creation of storage dim, facts from hive tables. Help > is much appreciated. > > > > *Regards,* > > *Srinivasan Hariharan* > > *Mob +91-9940395830 <%2B91-9940395830>* > > > > > > **************** CAUTION - Disclaimer ***************** > This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely > for the use of the addressee(s). If you are not the intended recipient, please > notify the sender by e-mail and delete the original message. Further, you are > not > to copy, disclose, or distribute this e-mail or its contents to any other > person and > any such actions are unlawful. This e-mail may contain viruses. Infosys has > taken > every reasonable precaution to minimize this risk, but is not liable for any > damage > you may sustain as a result of any virus in this e-mail. You should carry out > your > own virus checks before opening the e-mail or attachment. Infosys reserves the > right to monitor and review the content of all messages sent to or from this > e-mail > address. Messages sent to or from this e-mail address may be stored on the > Infosys e-mail system. > ***INFOSYS******** End of Disclaimer ********INFOSYS*** > >
