Re: Pig question

2013-05-07 Thread abhishek
Coveney, Thanks for the reply Got the answer using nested foreach. a = load 'data' using PigStorage(','); b = foreach a { c = substring(col1,0,4); generate 332 as x, c; } Sent from my iPhone On May 7, 2013, at 4:33 AM, Jonathan Coveney jcove...@gmail.com wrote: cdh-user to bcc Your

pig question

2013-04-27 Thread jamal sasha
Hi, I have data of format id1,id2, value 1 , abc, 2993 1, dhu, 9284 1,dus,2389 2, acs,29392 and so on For each id1, I want to find the maximum value and then divide value by max_value so in example above: 1,abc, 2993/9284 1,dhu ,9284/9284 1,dus, 2389/9284 2,acs, 29392/max_value_for_this id

Re: pig question

2013-04-27 Thread Russell Jurney
values = LOAD 'my_path' AS (id1:int, id2:chararray, value:int); overall = FOREACH (GROUP values BY id1) GENERATE group AS id1, value/MAX(value) as div_max; Russell Jurney http://datasyndrome.com On Apr 27, 2013, at 2:32 AM, jamal sasha jamalsha...@gmail.com wrote: Hi, I have data of format

Pig question.

2012-10-03 Thread jamal sasha
Hi, I have a table in format: Id: int, amount: float, true_date: chararray, time:chararray, state:chararray Fortunately, there are only two states in my db. So if I have a state as “CA” then add +1 to datetime If state is “MA”, then add +5 to datetime And then save the results. Also a

Re: Pig question.

2012-10-03 Thread TianYi Zhu
Hi Jamal, you can write a UDF convert time between different time zones with following utilities, java.text.DateFormat; java.text.SimpleDateFormat; java.util.Date; java.util.TimeZone; Thanks, TianYi On Thu, Oct 4, 2012 at 12:53 AM, jamal sasha jamalsha...@gmail.com wrote: Hi, I have a

Re: Pig question.

2012-10-03 Thread Russell Jurney
0) convert dates to ISO format via CustomFormatToISO 1) convert dates to unix time longs via ISOToUnix 2) use foreach/generate with ternary operator to add/subtract hours based on the value of the other field 3) convert dates back to ISO format with UnixToISO Call it a day. Russell Jurney