Nice, it works, thanks.
On 5/17/10 1:49 PM, "Dmitriy Ryaboy" wrote:
> double-escape the slash.
>
> grunt> data1 = FOREACH data GENERATE id, name, (gender=='\\N'?'U':gender) AS
> gender;
> grunt> dump data1
> (7001L,Test0,U)
> (7002L,Test1,U)
> (7003L,Test3,U)
> (7004L,Test4,U)
>
>
> On Mon,
double-escape the slash.
grunt> data1 = FOREACH data GENERATE id, name, (gender=='\\N'?'U':gender) AS
gender;
grunt> dump data1
(7001L,Test0,U)
(7002L,Test1,U)
(7003L,Test3,U)
(7004L,Test4,U)
On Mon, May 17, 2010 at 1:44 PM, Syed Wasti wrote:
> Attached is the data file, just in case, below is
Attached is the data file, just in case, below is the data and the script,
this should give you all you want and to your last question, I am using Mac.
7001Test001\N
7002Test101\N
7003Test312\N
7004Test422\N
grunt> data = LOAD 'data'
There must be some noise in your input that is getting interpreted
differently by Hive and Pig. Loading a bunch of newlines does generate
nulls, so I am not sure what's happening there. Are you loading using
PigStorage? Default delimiters? Can you upload a sample file and script that
reproduces the
Have tried both ways foo is null OR foo == '\n', doesn't work in pig.
Why would null values be saved as \N in a file ? Is there a reason, is this
hive or hadoop way which pig cant understand ?
On 5/17/10 11:53 AM, "Dmitriy Ryaboy" wrote:
> Arguably, that's a Hive bug. What does hive do if you *
Arguably, that's a Hive bug. What does hive do if you *want* to have a \n as
a value?
For your case, I think it's as simple as foreach rel generate ( foo is null
OR foo == '\n' ? 'U' : foo);
-D
On Mon, May 17, 2010 at 11:42 AM, Syed Wasti wrote:
> Well Dmitriy, my bad, I was looking at the dat
Well Dmitriy, my bad, I was looking at the data through a hive query and it
shows as NULL, but when I looked into the flat file all the NULL values are
are seen as \N.
Hive is able to understand \N as NULL but pig is not... How can I resolve
this ?
On 5/16/10 4:33 PM, "Dmitriy Ryaboy" wrote:
>
Is this suppose to be working on pig 5 ?
On 5/16/10 4:47 PM, "Syed Wasti" wrote:
> Hmmm,,, not really, it doesn't show the \n if I don't use the bincond operator
> on that column...
>
>
> On 5/16/10 4:33 PM, "Dmitriy Ryaboy" wrote:
>
>> In that case, maybe it's the data, and what you think
Hmmm,,, not really, it doesn't show the \n if I don't use the bincond
operator on that column...
On 5/16/10 4:33 PM, "Dmitriy Ryaboy" wrote:
> In that case, maybe it's the data, and what you think is null is actually
> '\n' ?
>
> -D
>
> On Sun, May 16, 2010 at 4:07 PM, Syed Wasti wrote:
>
>
In that case, maybe it's the data, and what you think is null is actually
'\n' ?
-D
On Sun, May 16, 2010 at 4:07 PM, Syed Wasti wrote:
> Doing absolutely the same thing and I am using pig 6 too.
> Tried with the fake data on both local and mapreduce modes, works fine.
> But on my script against
Doing absolutely the same thing and I am using pig 6 too.
Tried with the fake data on both local and mapreduce modes, works fine.
But on my script against actual data in mapreduce mode, it fails to do the
same thing, places \N instead of U.
grunt> rel1 = LOAD '/user/swasti/data' USING PigStorage('
So what I am saying is, check that you are not inserting some weird
non-ascii quotes in your actual script.
I just ran this on Pig 6, it worked:
grunt> data = load 'tmp/nulltest' using PigStorage() as (num);
grunt> dump data;
(1)
(2)
()
(3)
grunt> find_nulls = foreach data generate ( num is null ?
Hmm not sure why, I used quotes in this mail, let me rewrite,
SQL(U is within single quotes): NVL(city,U) city
Pig(U is within single quotes): (city is null?U:city) AS city
On 5/16/10 1:36 PM, "Dmitriy Ryaboy" wrote:
> Syed,
> The samples you pasted include all kinds of extraneous characters.
Syed,
The samples you pasted include all kinds of extraneous characters. Are you
sure your script is properly encoded?
On Sun, May 16, 2010 at 1:16 PM, Syed Wasti wrote:
> I am trying the SQL ³NVL(city, ŒU¹) city² in pig I am using the bincond
> operator, ³(city is null?'U': city) AS city², whi
I am trying the SQL ³NVL(city, U¹) city² in pig I am using the bincond
operator, ³(city is null?'U': city) AS city², which is of chararray type,
the result file shows \N¹ instead of U. Any ideas ?
15 matches
Mail list logo
