Hello Sam,
Please find attached PIG script for the same. You may find the necessary
jars below.
http://mvnrepository.com/artifact/com.twitter.elephantbird/elephant-bird-pig
Note: Same functionality can be achieved in Hive as well.
Thanks and Regards
Nishant Aggarwal, PMP
Cell No:- +91 99588
Hi,
> If you have any tutorial for extracting data from complex nested json
>arrays (as the example given in my previous email), please send it.
90% of working with the real world is cleansing bad data. People
under-sell hive's flexibility in situations like this.
This is what I do
hive> comp
a' turns out to be an array datatype you just need to troubleshoot
the explode()
From: Sam Joe [mailto:games2013@gmail.com]
Sent: Tuesday, October 27, 2015 7:37 PM
To: user@hive.apache.org
Subject: Re: Using json_tuple for Nested json Arrays
Hi Ryan,
I think tr3.media a complex jso
Thanks Nishant! Will try using Pig json loader too to achieve this
requirement. If you have any tutorial for extracting data from complex
nested json arrays (as the example given in my previous email), please send
it.
Appreciate your help!
Thanks,
Joel
On Tue, Oct 27, 2015 at 10:20 PM, Nishant A
Hello Sam,
You can easily achieve this by using elephant-bird.jars in pig. We are also
caturing tweets via flume and filter them using pig and elephant-jars. You
can find the related jars over internet.
Cheers,
Nishant Aggarwal
On 28 Oct 2015 00:50, "Sam Joe" wrote:
> Hi,
>
> Is it possible to u
o what is
> it?
>
>
>
> *From:* Sam Joe [mailto:games2013@gmail.com]
> *Sent:* Tuesday, October 27, 2015 2:56 PM
>
> *To:* user@hive.apache.org
> *Subject:* Re: Using json_tuple for Nested json Arrays
>
>
>
> Hi Ryan,
>
>
>
> The statement returns nul
a map as a parameter
Apparently, hive doesn't think tr3.media is an array or map..so what is it?
From: Sam Joe [mailto:games2013@gmail.com]
Sent: Tuesday, October 27, 2015 2:56 PM
To: user@hive.apache.org
Subject: Re: Using json_tuple for Nested json Arrays
Hi Ryan,
The statement returns nu
":"crop","h":150},"small":{"w":340,"resize":"fit","h":255},"large":{"w":1024,"resize":"fit","h":768},"medium":{"w":600,"resize":"fit&
ctober 27, 2015 2:25 PM
To: user@hive.apache.org
Subject: Re: Using json_tuple for Nested json Arrays
Hi Ryan,
The simple query is running fine as shown below:
hive> SELECT tr2.id<http://tr2.id>, tr2.possibly_sensitive
> FROM tweets_raw tr1
> LATERAL VIEW json_tupl
>
> SELECT get_json_object(text_col, '$.id') as id FROM tweets_raw limit 10;
>
>
>
> You should also be able to use json_tuple(), but start simple
>
>
>
> *From:* Sam Joe [mailto:games2013....@gmail.com]
> *Sent:* Tuesday, October 27, 2015 1:43 PM
>
&
weets_raw limit 10;
You should also be able to use json_tuple(), but start simple
From: Sam Joe [mailto:games2013@gmail.com]
Sent: Tuesday, October 27, 2015 1:43 PM
To: user@hive.apache.org
Subject: Re: Using json_tuple for Nested json Arrays
Hi Ryan,
Thanks for your reply! I d
as in these examples:
>
>
> http://mechanics.flite.com/blog/2014/04/16/using-explode-and-lateral-view-in-hive/
>
>
> http://stackoverflow.com/questions/28716165/how-to-query-struct-array-with-hive-get-json-object
>
>
>
>
>
> *From:* Sam Joe [mailto:games2013....@gmai
, October 27, 2015 1:29 PM
To: user@hive.apache.org
Subject: Re: Using json_tuple for Nested json Arrays
I tried using EXPLODE function on the nested json array but it doesn't work and
throws following error:
FAILED: UDFArgumentException explode() takes an array or a map as a parameter
Thanks,
I tried using EXPLODE function on the nested json array but it doesn't work
and throws following error:
FAILED: UDFArgumentException explode() takes an array or a map as a
parameter
Thanks,
Joel
On Tue, Oct 27, 2015 at 3:20 PM, Sam Joe wrote:
> Hi,
>
> Is it possible to use json_tuple functio
Hi,
Is it possible to use json_tuple function to extract data from json arrays
(nested too). I am trying to process json data as string and avoid using
serdes since user data may be malformed.
Please see a sample json data given below:
{
"filter_level": "low",
"retweeted": false,
"in_reply_t
15 matches
Mail list logo