Set variable in hive query

2015-06-05 Thread Ravisankar Mani
Hi everyone, I assign values for set variable. Please refer following query SET Row_Number_0 = 0; SELECT criticid AS criticid, sum(rating) as rating, ${hiveconf:Row_Number_0}= ${hiveconf:Row_Number_0} + 1 AS R_N_1 FROM GROUP BY recommend_ratings2.criticid

Re: Reading Json files in hive

2015-06-05 Thread Wollert, Fabian
Hey Ayazur, please see here for doc's on how to read JSON in Hive: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-get_json_object Cheers Fabian 2015-06-05 16:30 GMT+02:00 Ayazur Rehman rehman.ayazu...@gmail.com: Hi, Can someone tell me how to read Json

Reading Json files in hive

2015-06-05 Thread Ayazur Rehman
Hi, Can someone tell me how to read Json file in hive My Hive version is 0.14 the schema is something like this- {columnA: some_string, columnB: 227958.0, columnC: [0.0, 0.0, 0.0, 0.0], columnD: {some_string:10.0}} If there is documentation about it, please let me know as well -- Thanking

RE: Reading Json files in hive

2015-06-05 Thread Gary Clark
CREATE EXTERNAL TABLE IF NOT EXISTS sampledata ( id string, type string, label string, realpoweroutput string, realpowerdemand string, reactivepoweroutput string, kvarpowerdemand string, averagesoc string,

Re: Reading Json files in hive

2015-06-05 Thread DU DU
You can use json_tuple which has better performance as I remember as follows. Your table should have single column to keep each line of Json string. select * from json_table lateral view json_tuple(json_table_cloumn, 'columnA') val as columnA ... On Fri, Jun 5, 2015 at 10:38 AM, Wollert, Fabian

Re: HBase and Hive integration

2015-06-05 Thread Sean Busbey
+user@hive -user@hbase to bcc Hi! This question is better handled by the hive user list, so I've copied them in and moved the hbase user list to bcc. On Fri, Jun 5, 2015 at 12:54 PM, Buntu Dev buntu...@gmail.com wrote: Hi - Newbie question: I got Hive and HBase on different clusters and say

Re: Reading Json files in hive

2015-06-05 Thread Ayazur Rehman
What is the meaning of fix column...does it mean non-nested columns? So I decided to use the UDF ... But I am still struggling with a command to create a table Is it necessary to use String datatype...even if I have integersand how do I specify the nested data typeslike the following

Re: Reading Json files in hive

2015-06-05 Thread DU DU
Fix column here means you have to define the table using SerDe in advance. If you have new columns added to the json, the table has to be changed. Create a table only have a column, such as value. Each row is line of json. Then use the jason tuple UDF to parcer the json on the fly On Fri, Jun 5,

Re: Reading Json files in hive

2015-06-05 Thread DU DU
Using Json serde requires you have fix column. In your case, you should consider using HIVE UDF mentioned above On Fri, Jun 5, 2015 at 11:13 AM, Wollert, Fabian fabian.woll...@zalando.de wrote: you can either pursuit the approaches from the others (reading the whole table with JSON Input

Re: Reading Json files in hive

2015-06-05 Thread Wollert, Fabian
you can either pursuit the approaches from the others (reading the whole table with JSON Input Format) or you read the File with normal Input Format and STRING Datatype and then apply the function i mentioned afterwards. Both Approaches work, i usually use mine. I don't know though which one is

Understanding Hive Outer Join Behavior

2015-06-05 Thread Jim Green
Sharing one article about how to understand Hive Outer Join Behavior http://www.openkb.info/2015/06/understanding-hive-outer-join-behavior.html . -- Thanks, www.openkb.info (Open KnowledgeBase for Hadoop/Database/OS/Network/Tool)

Re: Reading Json files in hive

2015-06-05 Thread Ayazur Rehman
Hi Gary, Thanks for the response. I tried creating a simple json table - create table jsonTable(id string, type string) row format serde 'org.apache.hadoop.hive.contrib.serde2.JsonSerde'; And I got the following error- Cannot validate serde: org.apache.hadoop.hive.contrib.serde2.JsonSerde Do

RE: Reading Json files in hive

2015-06-05 Thread Gary Clark
Yes you will need to import the jar file. I don’t think it comes with the hive by default. From: Ayazur Rehman [mailto:rehman.ayazu...@gmail.com] Sent: Friday, June 05, 2015 9:58 AM To: user@hive.apache.org Subject: Re: Reading Json files in hive Hi Gary, Thanks for the response. I tried

Re: Reading Json files in hive

2015-06-05 Thread Ayazur Rehman
Hi Fabian, The documentation only points to how to run some queries with json format table and I could not find the documentation for creating a Json Table. Did I miss something in the documentation? I apologize if my question was not clear. Thanks On Fri, Jun 5, 2015 at 10:38 AM, Wollert,