Re: Hive QL - NOT IN, NOT EXIST

2013-05-06 Thread Stephen Boesch
on left outer join? Peter -- Date: Sun, 5 May 2013 21:44:37 -0700 Subject: Re: Hive QL - NOT IN, NOT EXIST From: java...@gmail.com To: user@hive.apache.org @Peter Does the query plan demonstrate that the 3Meg row table is being map-joined and the 400M table

RE: Hive QL - NOT IN, NOT EXIST

2013-05-06 Thread Peter Chu
Thanks Stephen, Will start a cluster today to see if it helps. Peter Date: Mon, 6 May 2013 00:05:45 -0700 Subject: Re: Hive QL - NOT IN, NOT EXIST From: java...@gmail.com To: user@hive.apache.org Hi Peter, Looks like mapjoin does not work with outer join so streamtable is instead a possible

Hive QL - NOT IN, NOT EXIST

2013-05-05 Thread Peter Chu
Hi, I am trying to write a hive query to find the equivalent of NOT IN / NOT EXIST in SQL. However, Hive does not support this. It does have Left Semi Join which serves as IN but NOT does not support. I am wondering if there is any way to do this without resorting to using left outer join and

Re: Hive QL - NOT IN, NOT EXIST

2013-05-05 Thread Michael Malak
--- On Sun, 5/5/13, Peter Chu pete@outlook.com wrote: I am wondering if there is any way to do this without resorting to using left outer join and finding nulls. I have found this to be an acceptable substitute. Is it not working for you?

RE: Hive QL - NOT IN, NOT EXIST

2013-05-05 Thread Peter Chu
: michaelma...@yahoo.com Subject: Re: Hive QL - NOT IN, NOT EXIST To: user@hive.apache.org --- On Sun, 5/5/13, Peter Chu pete@outlook.com wrote: I am wondering if there is any way to do this without resorting to using left outer join and finding nulls. I have found

Re: Hive QL - NOT IN, NOT EXIST

2013-05-05 Thread Stephen Boesch
NOT IN (SELECT uuid FROM message); Date: Sun, 5 May 2013 20:25:15 -0700 From: michaelma...@yahoo.com Subject: Re: Hive QL - NOT IN, NOT EXIST To: user@hive.apache.org --- On Sun, 5/5/13, Peter Chu pete@outlook.com wrote: I am wondering if there is any way to do this without resorting

RE: Hive QL - NOT IN, NOT EXIST

2013-05-05 Thread Peter Chu
Thanks, Stephen, I do not quite understand what you mean by Stream, specifically Stream the 400M message records through the in-memory maps.Can you please elaborate. Also, can you use MAPJOIN on left outer join? Peter Date: Sun, 5 May 2013 21:44:37 -0700 Subject: Re: Hive QL - NOT IN, NOT EXIST