Re: Reading json file.

2013-08-29 Thread jamal sasha
Umm.. I am trying .. but somehow i am not able to get my head around this: a = load 'sample_json.json' using JsonLoader('id:chararray,categories:[chararray], hostt:{ (variable_a: {(first:int,last:int)})}, ns:[chararray],rep:chararray '); But i get this error: org.codehaus.jackson.JsonParseExcepti

Re: Reading json file.

2013-08-29 Thread Shahab Yunus
Have you seen these? http://pig.apache.org/docs/r0.11.0/api/org/apache/pig/builtin/JsonStorage.html http://hortonworks.com/blog/jsonize-anything-in-pig-with-tojson/ Regards, Shahab On Thu, Aug 29, 2013 at 6:19 PM, jamal sasha wrote: > Hi, > > I have json file in follwoing format: > { "_id" :

Reading json file.

2013-08-29 Thread jamal sasha
Hi, I have json file in follwoing format: { "_id" : "foo.com", "categories" : [], "h1" : { "bar==" : { "first" : 1281916800, "last" : 1316995200 }, "foo==" : { "first" : 1281916800, "last" : 1316995200 } }, "name2" : [ "foobarl.com", "foobar2.com" ], "rep" : null } So, how do i parse this json in

Re: Pig GROUP operator - Data is shuffled and wind up together for the same grouping key

2013-08-29 Thread Prashant Kommireddi
Hi Viswa, All records with the same key ending up in the same reducer is expected. Can you provide us with your script and a sample input/output if you are seeing something different? On Thursday, August 29, 2013, Viswanathan J wrote: > Hi, > > I'm using pig version 0.11.0 > > While using GROUP

Pig GROUP operator - Data is shuffled and wind up together for the same grouping key

2013-08-29 Thread Viswanathan J
Hi, I'm using pig version 0.11.0 While using GROUP operator in Pig all the data is shuffled, so that rows in different partitions that have the same grouping key wind up together and got wrong results for grouping. While storing the result data, it is share work between multiple calculations. H

Bug in ILLUSTRATE operator

2013-08-29 Thread Mike Sukmanowsky
Was going to file in JIRA, but wanted to reach out here first to see if I'm just going crazy. When using 0.11.2-SNAPSHOT I'm seeing errors only when using ILLUSTRATE (dump and describe seem to work fine). See gist for script and error https://gist.github.com/msukmanowsky/ede8c74577004f744b58 Any

Re: Reading Kafka directly from Pig?

2013-08-29 Thread Mohammad Tariq
Great job. +1 Warm Regards, Tariq cloudfront.blogspot.com On Wed, Aug 7, 2013 at 8:27 PM, Russell Jurney wrote: > Cool stuff, a Pig Kafka UDF. > > Russell Jurney http://datasyndrome.com > > Begin forwarded message: > > *From:* David Arthur > *Date:* August 7, 2013, 7:41:30 AM PDT > *To:* us...

Avro to Tuples during UnitTest

2013-08-29 Thread Serega Sheypak
Hi, we have itegration test java utility which helps to test pig scripts. We often develop different UDFs in java and I would like to create unit tests for them. Right now they are tested with pig scripts during integration tests. I have a pack of prepared avro files with etalon input for my java

Re: Misplaced pigsample_123456.... file fails the pig job !

2013-08-29 Thread Ruslan Al-Fakikh
Which hadoop distro are you using? I've heard Hortonworks has a windows-compatible hadoop. On Wed, Aug 28, 2013 at 2:36 PM, Darpan R wrote: > Hi folks, > I am facing a wiered issue. > I am running PIG 0.11 on windows7/64 bit machine with latest version of > cygwin. > > I am a weblog which I wan