This is data in mysql table: +----+------+------+-------------+-------+ | id | name | age | email | title | +----+------+------+-------------+-------+ | 1 | aaaa | 20 | [email protected] | A | | 2 | bbbb | 21 | [email protected] | B | | 3 | cccc | 21 | [email protected] | A | +----+------+------+-------------+-------+
and, I want data in hbase: ROW COLUMN+CELL 1 column=baseinfo:age, timestamp=1369041765578, value=20 1 column=baseinfo:name, timestamp=1369041765578, value=aaaa 1 column=workinfo:email, timestamp=1369041967157,value= [email protected] 1 column=workinfo:title, timestamp=1369041967157, value=A 2 column=baseinfo:age, timestamp=1369041778521, value=21 2 column=baseinfo:name, timestamp=1369041778521, value=bbbb 2 column=workinfo:email, timestamp=1369041970526, value= [email protected] 2 column=workinfo:title, timestamp=1369041970526, value=B 3 column=baseinfo:age, timestamp=1369041778521, value=21 3 column=baseinfo:name, timestamp=1369041778521, value=cccc 3 column=workinfo:email, timestamp=1369041970526, value= [email protected] 3 column=workinfo:title, timestamp=1369041970526, value=A 3 row(s) in 1.1040 seconds I use two commonds: #name,age to baseinfo sqoop import --connect jdbc:mysql://h2/test --username hiveuser --password password \ --query 'SELECT id,name,age FROM testtable WHERE $CONDITIONS' \ --hbase-create-table --hbase-table hbasetable \ --column-family baseinfo --hbase-row-key id --split-by id #email,title to workinfo sqoop import --connect jdbc:mysql://h2/test --username hiveuser --password password \ --query 'SELECT id,email,title FROM testtable WHERE $CONDITIONS' \ --hbase-create-table --hbase-table hbasetable \ --column-family workinfo --hbase-row-key id --split-by id Anyone have more simple, easy-understand way? On Tue, May 21, 2013 at 12:55 AM, Jarek Jarcec Cecho <[email protected]>wrote: > Hi Robber, > Sqoop won't import rows where all columns that should be inserted into > cell contains NULL. Would you mind taking a look if in your case the name > and age are NULL? Another possibility to set the property > sqoop.hbase.add.row.key. You might find more information about this > property in SQOOP-870 [1] or in our user guide [1]. > > Jarcec > > Links: > 1: https://issues.apache.org/jira/browse/SQOOP-870 > 2: > http://sqoop.apache.org/docs/1.4.3/SqoopUserGuide.html#_additional_import_configuration_properties > > On Mon, May 20, 2013 at 11:02:56PM +0800, Robber Phex wrote: > > I think I can do it: > > > > HBase MySQL Columns > > > > row key <----------- id > > baseinfo <----------- name > > `-<--- age > > > > But, I use following command import zero record. > > > > sqoop import --connect jdbc:mysql://h2/test --username hiveuser > --password > > password --table testtable --hbase-table hbasetable --column-family > > baseinfo --columns "name,age" --hbase-row-key id > > > > Or,you means: > > > > sqoop import --connect jdbc:mysql://h2/test --username hiveuser > --password > > password --table testtable --hbase-table hbasetable --column-family name > > --hbase-row-key id > > > > sqoop import --connect jdbc:mysql://h2/test --username hiveuser > --password > > password --table testtable --hbase-table hbasetable --column-family age > > --hbase-row-key id > > > > Thanks. > > > > On Mon, May 20, 2013 at 10:10 PM, Jarek Jarcec Cecho <[email protected] > >wrote: > > > > > Hi Robber, > > > Sqoop currently supports importing only into single column family. You > can > > > import data twice using two different column families in each case and > > > parameter --columns to filter only relevant columns for given column > family. > > > > > > Jarcec > > > > > > On Mon, May 20, 2013 at 09:42:14PM +0800, Robber Phex wrote: > > > > I have a table in mysql like this: > > > > +-------+-------------+------+-----+---------+-------+ > > > > | Field | Type | Null | Key | Default | Extra | > > > > +-------+-------------+------+-----+---------+-------+ > > > > | id | int(11) | NO | PRI | NULL | | > > > > | name | varchar(45) | YES | | NULL | | > > > > | age | int(11) | YES | | NULL | | > > > > | email | varchar(20) | YES | | NULL | | > > > > | title | varchar(10) | YES | | NULL | | > > > > +-------+-------------+------+-----+---------+-------+ > > > > I want to export it to hbase by sqoop. > > > > I execute this: > > > > sqoop import --connect jdbc:mysql://h2/test --username testu > --password > > > > testp \ > > > > --table testtable --hbase-create-table --hbase-table hbasetable \ > > > > --column-family name --column-family age --column-family email > > > > --column-family title \ > > > > --hbase-row-key id > > > > > > > > I get full data in hbase, but I want custom row-family. Exactly I > need > > > > transformation like this: > > > > > > > > HBase MySQL Columns > > > > > > > > row key <----------- id > > > > baseinfo <----------- name > > > > `-<--- age > > > > workinfo <----------- email > > > > `-<--- title > > > > > > > > Anyone have any idea? > > > > > > > > -- > > > > Regards, > > > > RobberPhex > > > > > > > > About me: http://about.me/RobberPhex > > > > > > > > > > > -- > > Regards, > > RobberPhex > > > > About me: http://about.me/RobberPhex > -- Regards, RobberPhex About me: http://about.me/RobberPhex
