I had anticipated it would throw a similar error with this suggestion as the last one... and it did. My fields are declared as INT, just to re-iterate. I don't think they can be compared to regexes. Here is the error:
ERROR 1037: <file LeadSales.pig, line 19, column 29> Operands of Regex can be CharArray only :(Name: Regex Type: null Uid: null) org.apache.pig.impl.logicalLayer.validators.TypeCheckerException: ERROR 1037: <file LeadSales.pig, line 19, column 29> Operands of Regex can be CharArray only :(Name: Regex Type: null Uid: null) Thanks. On Thu, Feb 18, 2016 at 5:24 AM, Chandeep Singh <c...@chandeep.com> wrote: > Since you integers in this field can you try matching to a regular > expression? > > Something like: X matches '\\d+' > > > On Feb 18, 2016, at 12:55 AM, Parth Sawant <parth.sawan...@gmail.com> > wrote: > > > > Hi Chandeep. I tried that already but it gave me the following error: > > > > ERROR 1039: > > <file LeadSales.pig, line 19, column 27> In alias X, incompatible > > types in NotEqual Operator left hand side:int right hand > > side:chararray. > > > > The error makes sense cause the fields I have are INT type and hence > > cannot be compared to a chararray. > > > > > > Thanks for the prompt response though. > > > > > > > > On Feb 17, 2016 16:32, "Chandeep Singh" <c...@chandeep.com> wrote: > > > > Try adding != '' along with IS NOT NULL. > >> > >>> On Feb 18, 2016, at 12:26 AM, Parth Sawant <parth.sawan...@gmail.com> > >> wrote: > >>> > >>> I'm trying to Filter some null fields in Pig using 'IS NOT NULL' . For > >> some > >>> reason the null data values persist. > >>> For eg: the following filter on storing it's contents, contains null > >> values > >>> for ABC and PQR. > >>> > >>> X = FILTER D BY (ABC IS NOT NULL) AND (ABC==1) AND (PQR==1) AND (PQR IS > >> NOT > >>> NULL) ; > >>> > >>> > >>> Can someone help with this? > >>> > >>> Thanks > >>> > >>> Parth S > >> > >> > >
