Now I assign 100M memory to hive, you consider that can support how many 'OR' string?
2010/8/5 Edward Capriolo <edlinuxg...@gmail.com> > On Wed, Aug 4, 2010 at 12:15 PM, lei liu <liulei...@gmail.com> wrote: > > Hello Edward Capriolo, > > > > Thank you for your reply. Are you sure that if you string enough 'or' > > together (say 8000) the query parser which uses java beans serialization > > will OOM? How many memory you assign to hive? > > > > 2010/8/4 Edward Capriolo <edlinuxg...@gmail.com> > >> > >> On Wed, Aug 4, 2010 at 6:10 AM, lei liu <liulei...@gmail.com> wrote: > >> > Because my company reuire we use 0.4.1 version, the version don't > >> > support IN > >> > clause. I want to use the OR clause(example:where id=1 or id=2 or > id=3) > >> > to > >> > implement the IN clause(example: id in(1,2,3) ). I know it will be > >> > slower > >> > especially when the list after "in" is very long. Could anybody can > >> > tell me > >> > why is slow when use OR clause to implement In clause? > >> > > >> > > >> > Thanks, > >> > > >> > > >> > LiuLei > >> > > >> > >> I can not imagine the performance difference between 'or' or 'in' > >> would be that great but I never benchmarked it. The big looming > >> problems is that if you string enough 'or' together (say 8000) the > >> query parser which uses java beans serialization will OOM. > >> > >> Edward > > > > > > That is exactly what I am saying. I tested with 4GB and 8GB. I am not > exactly sure how many OR's you can get away with for your memory size, > but some upper limit exists currently. Most people never hit it. (I > did because my middle name is "edge case" ) >
