Query on Pig on Spark (Spork)

> > Hi, As a beginner ,I have below queries on Spork(Pig on Spark). I have cloned git clone https://github.com/apache/pig -b spark . 1.On which version of Pig and Spark , Spork is being built. 2. I followed the steps mentioned in https://issues.apache.org/ jira/browse/PIG-4059 and try to run

Re: Query On Pig

Hi Nivetha, I don't know if I understood it correctly, but I think you can just use FLATTEN to flatten your bag. It would become something like: A = ... B = FOREACH A GENERATE FLATTEN(YourUDF(input)) AS (description, name, no, trans_no); and then you can access your data normally by just using

Re: Query On Pig

Hi Rodrigo Ferreira, Thanks a lot It worked. On 18 July 2014 15:18, Rodrigo Ferreira web...@gmail.com wrote: Hi Nivetha, I don't know if I understood it correctly, but I think you can just use FLATTEN to flatten your bag. It would become something like: A = ... B = FOREACH A

Re: Query on Pig

Hi, Thanks for replying.Can you please explain how mapreduce operator works in pig On 5 July 2014 10:35, Darpan R darpa...@gmail.com wrote: Looks like Classpath problem :java.lang.RuntimeException: java.lang.ClassNotFoundException: Class WordCount$Map not found Can you make sure your

Re: Query on Pig

Hi, We are calling external map reduce program inside our pig script to perform a specific task. Lets take the example crawling process. -- Load the all seed urls into the relation crawldata. *crawldata = load 'baseurls' using PigStorage( pageid: chararray, pageurl:chararray)* normalizedata =

Re: Query on Pig

Nivetha, Did you create the jar manually ? Seems that Map is an inner class in *WordCount* Class. Did you add *WordCount$Map.class* to the jar along with *WordCount.class* ? -- Suraj Nayak On Saturday 05 July 2014 10:35 AM, Darpan R wrote: Looks like Classpath problem

Re: Query on Pig

Looks like Classpath problem :java.lang.RuntimeException: java.lang.ClassNotFoundException: Class WordCount$Map not found Can you make sure your jar is in the class path ? On 4 July 2014 11:19, Nivetha K nivethak3...@gmail.com wrote: Hi, I am currently working with Pig. I get struck

Query on Pig

Hi, I am currently working with Pig. I get struck with following script. A = load 'sample.txt'; B = MAPREDUCE '/home/training/simp.jar' Store A into 'inputDir' Load 'outputDir' as (word:chararray, count: int) `WordCount inputDir outputDir`; dump B; Error : 2014-07-04 11:17:57,811 [main]

Re: Query On Pig

i. Equals can be mimicked by specifying both = and = (i.e. -lte=123 -gte=123) ii. What do you mean by taking a partial rowkey? the lte and gte are partial matches. On Mon, Jun 30, 2014 at 10:24 PM, Nivetha K nivethak3...@gmail.com wrote: Hi, Iam working with Pig. I need to know some

Re: Query On Pig

(i) There is no direct way to take exact match (ii) Partial row key means consider my rowkeys are 123456,123678,123678,124789,124789.. i need to take the rowkeys starts with 123 On 1 July 2014 11:36, Pradeep Gollakota pradeep...@gmail.com wrote: i. Equals can be mimicked

Re: Query On Pig

i. That's correct. ii. If the key partial match is at the beginning of the row key, then what you're looking for is the -gte and -lt/-lte flags. If you want to start with 123, you just specify -gte 123 -lt 124. This would have the same affect as a partial starts with match. If what you're looking

Query On Pig

Hi, Iam working with Pig. I need to know some information on HBaseStorage. B = LOAD 'hbase://sample' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('details:* details1:* details2:*','-loadKey true -lte=123') as (id:chararray,m1:map[],m2:map[],m3:map[]); (i) like lte ((ie) Less

Query on Pig

Hi, I am trying to parse XML using Pig. It is working fine with simple XML but with nested XML tags I am getting problem. I tried with some code i displayed it below which does not work fine for nested tags. If there is any solution please suggest me. input xml : students student