Re: no data in external table
Hi, I upgraded to hadoop-1.0.3, but still have the same results. I did as you suggested. When I do "select * from hbase_table_2" it does not output any job information. When I do "select value from hbase_table_2" it outputs job information. In both cases there is no data returned. Thanks. Alex. -Original Message- From: kulkarni.swarnim To: user Sent: Thu, Oct 4, 2012 3:16 pm Subject: Re: no data in external table Can you try creating a table like this: CREATE EXTERNAL TABLE hbase_table_2(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name" = "xyz"); Now do a select * from hbase_table_2; Do you see any data now? On Thu, Oct 4, 2012 at 5:10 PM, wrote: Hi, In the hbase table I do not see column qualifier, only family. For testing connection to hbase I also created a table using CREATE TABLE hbase_table_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name" = "xyz"); I see xyz table in hbase. then I added a row in hbase using put 'xyz', 'row1', 'cf1', 'abc' Then in hive I did: select * from hbase_table_1; No results are returned, but scan xys in hbase returns 1 row. Thanks. Alex. -Original Message- From: kulkarni.swarnim To: user Sent: Thu, Oct 4, 2012 3:00 pm Subject: Re: no data in external table > "hbase.columns.mapping" = ":key,mtdt:string,il:string,ol:string" This doesn't look right. The mapping should be of form COLUMN_FAMILY:COLUMN_QUALIFIER. In this case it seems to be COLUMN_FAMILY:TYPE which is not right. On Thu, Oct 4, 2012 at 3:25 PM, wrote: Hi, In hive shell I did create external table myextrenaltable (key string, metadata string, inlinks string, outlinks string) stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' with serdeproperties ("hbase.columns.mapping" = ":key,mtdt:string,il:string,ol:string") tblproperties ("hbase.table.name" = "myextrenaltable"); In tasktracker log I do not see anything relevant to hbase. In jobdetails page I see a few successful jobs. in hive shell I see Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_201210031146_0016, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201210031146_0016 Kill Command = /home/dev/hadoop-0.20.2/bin/../bin/hadoop job -Dmapred.job.tracker=localhost:9001 -kill job_201210031146_0016 Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0 2012-10-04 13:19:06,581 Stage-1 map = 0%, reduce = 0% 2012-10-04 13:19:12,629 Stage-1 map = 100%, reduce = 0% 2012-10-04 13:19:15,657 Stage-1 map = 100%, reduce = 100% Ended Job = job_201210031146_0016 MapReduce Jobs Launched: Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 SUCCESS Total MapReduce CPU Time Spent: 0 msec OK Time taken: 17.47 seconds Thanks in advance. Alex. -Original Message- From: Ted Yu To: user Sent: Thu, Oct 4, 2012 11:33 am Subject: Re: no data in external table Can you tell us how you created mapping for the existing table ? In task log, do you see any connection attempt to HBase ? Cheers On Thu, Oct 4, 2012 at 11:30 AM, wrote: Hello, I use hive-0.9.0 with hadoop-0.20.2 and hbase -0.92.1. I have created external table, mapping it to an existing table in hbase. When I do "select * from myextrenaltable" it returns no results, although scan in hbase shows data, and I do not see any errors in jobtracker log. Any ideas how to debug this issue. Thanks. Alex. -- Swarnim -- Swarnim
Re: no data in external table
Can you try creating a table like this: CREATE EXTERNAL TABLE hbase_table_2(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name" = "xyz"); Now do a select * from hbase_table_2; Do you see any data now? On Thu, Oct 4, 2012 at 5:10 PM, wrote: > Hi, > > In the hbase table I do not see column qualifier, only family. > For testing connection to hbase I also created a table using > > CREATE TABLE hbase_table_1(key int, value string) > STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' > WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") > TBLPROPERTIES ("hbase.table.name" = "xyz"); > > I see xyz table in hbase. then I added a row in hbase using put 'xyz', > 'row1', 'cf1', 'abc' > > > Then in hive I did: select * from hbase_table_1; > No results are returned, but scan xys in hbase returns 1 row. > > Thanks. > > Alex. > > -Original Message- > From: kulkarni.swarnim > To: user > Sent: Thu, Oct 4, 2012 3:00 pm > Subject: Re: no data in external table > > "hbase.columns.mapping" = ":key,mtdt:string,il:string,ol:string" > > This doesn't look right. The mapping should be of form > COLUMN_FAMILY:COLUMN_QUALIFIER. In this case it seems to be > COLUMN_FAMILY:TYPE which is not right. > > On Thu, Oct 4, 2012 at 3:25 PM, wrote: > >> Hi, >> >> In hive shell I did >> >> create external table myextrenaltable (key string, metadata string, >> inlinks string, outlinks string) stored by >> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' >> with serdeproperties ("hbase.columns.mapping" = >> ":key,mtdt:string,il:string,ol:string") >> tblproperties ("hbase.table.name" = "myextrenaltable"); >> >> In tasktracker log I do not see anything relevant to hbase. In jobdetails >> page I see a few successful jobs. in hive shell I see >> >> Total MapReduce jobs = 1 >> Launching Job 1 out of 1 >> Number of reduce tasks is set to 0 since there's no reduce operator >> Starting Job = job_201210031146_0016, Tracking URL = >> http://localhost:50030/jobdetails.jsp?jobid=job_201210031146_0016 >> Kill Command = /home/dev/hadoop-0.20.2/bin/../bin/hadoop job >> -Dmapred.job.tracker=localhost:9001 -kill job_201210031146_0016 >> Hadoop job information for Stage-1: number of mappers: 1; number of >> reducers: 0 >> 2012-10-04 13:19:06,581 Stage-1 map = 0%, reduce = 0% >> 2012-10-04 13:19:12,629 Stage-1 map = 100%, reduce = 0% >> 2012-10-04 13:19:15,657 Stage-1 map = 100%, reduce = 100% >> Ended Job = job_201210031146_0016 >> MapReduce Jobs Launched: >> Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 SUCCESS >> Total MapReduce CPU Time Spent: 0 msec >> OK >> Time taken: 17.47 seconds >> >> >> >> Thanks in advance. >> Alex. >> >> -Original Message- >> From: Ted Yu >> To: user >> Sent: Thu, Oct 4, 2012 11:33 am >> Subject: Re: no data in external table >> >> Can you tell us how you created mapping for the existing table ? >> >> In task log, do you see any connection attempt to HBase ? >> >> Cheers >> >> On Thu, Oct 4, 2012 at 11:30 AM, wrote: >> >>> Hello, >>> >>> I use hive-0.9.0 with hadoop-0.20.2 and hbase -0.92.1. I have created >>> external table, mapping it to an existing table in hbase. When I do "select >>> * from myextrenaltable" it returns no results, although scan in hbase shows >>> data, and I do not see any errors in jobtracker log. >>> >>> Any ideas how to debug this issue. >>> >>> Thanks. >>> Alex. >>> >> >> > > > -- > Swarnim > -- Swarnim
Re: no data in external table
Hi, In the hbase table I do not see column qualifier, only family. For testing connection to hbase I also created a table using CREATE TABLE hbase_table_1(key int, value string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:val") TBLPROPERTIES ("hbase.table.name" = "xyz"); I see xyz table in hbase. then I added a row in hbase using put 'xyz', 'row1', 'cf1', 'abc' Then in hive I did: select * from hbase_table_1; No results are returned, but scan xys in hbase returns 1 row. Thanks. Alex. -Original Message----- From: kulkarni.swarnim To: user Sent: Thu, Oct 4, 2012 3:00 pm Subject: Re: no data in external table > "hbase.columns.mapping" = ":key,mtdt:string,il:string,ol:string" This doesn't look right. The mapping should be of form COLUMN_FAMILY:COLUMN_QUALIFIER. In this case it seems to be COLUMN_FAMILY:TYPE which is not right. On Thu, Oct 4, 2012 at 3:25 PM, wrote: Hi, In hive shell I did create external table myextrenaltable (key string, metadata string, inlinks string, outlinks string) stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' with serdeproperties ("hbase.columns.mapping" = ":key,mtdt:string,il:string,ol:string") tblproperties ("hbase.table.name" = "myextrenaltable"); In tasktracker log I do not see anything relevant to hbase. In jobdetails page I see a few successful jobs. in hive shell I see Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_201210031146_0016, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201210031146_0016 Kill Command = /home/dev/hadoop-0.20.2/bin/../bin/hadoop job -Dmapred.job.tracker=localhost:9001 -kill job_201210031146_0016 Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0 2012-10-04 13:19:06,581 Stage-1 map = 0%, reduce = 0% 2012-10-04 13:19:12,629 Stage-1 map = 100%, reduce = 0% 2012-10-04 13:19:15,657 Stage-1 map = 100%, reduce = 100% Ended Job = job_201210031146_0016 MapReduce Jobs Launched: Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 SUCCESS Total MapReduce CPU Time Spent: 0 msec OK Time taken: 17.47 seconds Thanks in advance. Alex. -Original Message- From: Ted Yu To: user Sent: Thu, Oct 4, 2012 11:33 am Subject: Re: no data in external table Can you tell us how you created mapping for the existing table ? In task log, do you see any connection attempt to HBase ? Cheers On Thu, Oct 4, 2012 at 11:30 AM, wrote: Hello, I use hive-0.9.0 with hadoop-0.20.2 and hbase -0.92.1. I have created external table, mapping it to an existing table in hbase. When I do "select * from myextrenaltable" it returns no results, although scan in hbase shows data, and I do not see any errors in jobtracker log. Any ideas how to debug this issue. Thanks. Alex. -- Swarnim
Re: no data in external table
> "hbase.columns.mapping" = ":key,mtdt:string,il:string,ol:string" This doesn't look right. The mapping should be of form COLUMN_FAMILY:COLUMN_QUALIFIER. In this case it seems to be COLUMN_FAMILY:TYPE which is not right. On Thu, Oct 4, 2012 at 3:25 PM, wrote: > Hi, > > In hive shell I did > > create external table myextrenaltable (key string, metadata string, > inlinks string, outlinks string) stored by > 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' > with serdeproperties ("hbase.columns.mapping" = > ":key,mtdt:string,il:string,ol:string") > tblproperties ("hbase.table.name" = "myextrenaltable"); > > In tasktracker log I do not see anything relevant to hbase. In jobdetails > page I see a few successful jobs. in hive shell I see > > Total MapReduce jobs = 1 > Launching Job 1 out of 1 > Number of reduce tasks is set to 0 since there's no reduce operator > Starting Job = job_201210031146_0016, Tracking URL = > http://localhost:50030/jobdetails.jsp?jobid=job_201210031146_0016 > Kill Command = /home/dev/hadoop-0.20.2/bin/../bin/hadoop job > -Dmapred.job.tracker=localhost:9001 -kill job_201210031146_0016 > Hadoop job information for Stage-1: number of mappers: 1; number of > reducers: 0 > 2012-10-04 13:19:06,581 Stage-1 map = 0%, reduce = 0% > 2012-10-04 13:19:12,629 Stage-1 map = 100%, reduce = 0% > 2012-10-04 13:19:15,657 Stage-1 map = 100%, reduce = 100% > Ended Job = job_201210031146_0016 > MapReduce Jobs Launched: > Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 SUCCESS > Total MapReduce CPU Time Spent: 0 msec > OK > Time taken: 17.47 seconds > > > > Thanks in advance. > Alex. > > -Original Message- > From: Ted Yu > To: user > Sent: Thu, Oct 4, 2012 11:33 am > Subject: Re: no data in external table > > Can you tell us how you created mapping for the existing table ? > > In task log, do you see any connection attempt to HBase ? > > Cheers > > On Thu, Oct 4, 2012 at 11:30 AM, wrote: > >> Hello, >> >> I use hive-0.9.0 with hadoop-0.20.2 and hbase -0.92.1. I have created >> external table, mapping it to an existing table in hbase. When I do "select >> * from myextrenaltable" it returns no results, although scan in hbase shows >> data, and I do not see any errors in jobtracker log. >> >> Any ideas how to debug this issue. >> >> Thanks. >> Alex. >> > > -- Swarnim
Re: no data in external table
Hi, In hive shell I did create external table myextrenaltable (key string, metadata string, inlinks string, outlinks string) stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' with serdeproperties ("hbase.columns.mapping" = ":key,mtdt:string,il:string,ol:string") tblproperties ("hbase.table.name" = "myextrenaltable"); In tasktracker log I do not see anything relevant to hbase. In jobdetails page I see a few successful jobs. in hive shell I see Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks is set to 0 since there's no reduce operator Starting Job = job_201210031146_0016, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201210031146_0016 Kill Command = /home/dev/hadoop-0.20.2/bin/../bin/hadoop job -Dmapred.job.tracker=localhost:9001 -kill job_201210031146_0016 Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0 2012-10-04 13:19:06,581 Stage-1 map = 0%, reduce = 0% 2012-10-04 13:19:12,629 Stage-1 map = 100%, reduce = 0% 2012-10-04 13:19:15,657 Stage-1 map = 100%, reduce = 100% Ended Job = job_201210031146_0016 MapReduce Jobs Launched: Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 SUCCESS Total MapReduce CPU Time Spent: 0 msec OK Time taken: 17.47 seconds Thanks in advance. Alex. -Original Message- From: Ted Yu To: user Sent: Thu, Oct 4, 2012 11:33 am Subject: Re: no data in external table Can you tell us how you created mapping for the existing table ? In task log, do you see any connection attempt to HBase ? Cheers On Thu, Oct 4, 2012 at 11:30 AM, wrote: Hello, I use hive-0.9.0 with hadoop-0.20.2 and hbase -0.92.1. I have created external table, mapping it to an existing table in hbase. When I do "select * from myextrenaltable" it returns no results, although scan in hbase shows data, and I do not see any errors in jobtracker log. Any ideas how to debug this issue. Thanks. Alex.
Re: no data in external table
Can you tell us how you created mapping for the existing table ? In task log, do you see any connection attempt to HBase ? Cheers On Thu, Oct 4, 2012 at 11:30 AM, wrote: > Hello, > > I use hive-0.9.0 with hadoop-0.20.2 and hbase -0.92.1. I have created > external table, mapping it to an existing table in hbase. When I do "select > * from myextrenaltable" it returns no results, although scan in hbase shows > data, and I do not see any errors in jobtracker log. > > Any ideas how to debug this issue. > > Thanks. > Alex. >
no data in external table
Hello, I use hive-0.9.0 with hadoop-0.20.2 and hbase -0.92.1. I have created external table, mapping it to an existing table in hbase. When I do "select * from myextrenaltable" it returns no results, although scan in hbase shows data, and I do not see any errors in jobtracker log. Any ideas how to debug this issue. Thanks. Alex.