Load_cfl = LOAD '/user/hdfs/file.txt' USING PigStorage('|') as (code:chararray,book_code:int,currency_code:chararray,start_ date:datetime,end_date:datetime,type:chararray,amount:double );
....... cfl_grp = GROUP Load_cfl BY (*flatn_col*); I don't see a column "*flatn_col" *in cfl_grp. Typo? On Mon, Jan 25, 2016 at 6:12 PM, Divya Gehlot <divya.htco...@gmail.com> wrote: > Hi, > The file Groupby_condition.txt looks like below where data is seperated by > | > > Column1|Y > Column2|N > Column3|Y > > On 26 January 2016 at 07:17, Prashant Kommireddi <prkommire...@apache.org> > wrote: > >> You are flattening a scalar column "groupBy". What does the file >> "GroupBy_Condition" look like? I'm guessing you want to load that as a >> bag/tuple before flattening, but you're reading as a chararray instead. >> >> On Monday, January 25, 2016, Divya Gehlot <divya.htco...@gmail.com> >> wrote: >> >> > Hi, >> > I have two files >> > Group_condition.txt >> > Colun1|Y >> > Column2|N >> > Column3|Y >> > >> > Load_cfl = LOAD '/user/hdfs/file.txt' USING PigStorage('|') as >> > >> > >> (code:chararray,book_code:int,currency_code:chararray,start_date:datetime,end_date:datetime,type:chararray,amount:double >> > ); >> > Load_GroupBy = LOAD '/user/hdfs/Groupby_Condition.txt' USING >> > PigStorage('|') as (groupBy:chararray , condition :chararray); >> > filter_col = FILTER Load_GroupBy BY (condition == 'Y'); >> > flatn_col = FOREACH filter_col GENERATE FLATTEN(groupBy); >> > tuple_col = FOREACH flatn_col GENERATE TOTUPLE(groupBy); >> > cfl_grp = GROUP Load_cfl BY (flatn_col); >> > >> > getting error : >> > >> > > Invalid scalar projection: flatn_col : A column needs to be projected >> > from >> > > a relation for it to be used as a scalar >> > >> > >> > >> > What am I missing in above ? >> > Would really appreciate the help. >> > >> > Thanks, >> > Divya >> > >> > >