forgot to mention, to determine maximum value of elements in an IN clause it's important to know how many elements each of the M queries actually returns and the size of the resultsets for each; while the number of elements in IN clause can be up to X , the number of rows retrieved will be up to Y, and the MAX for X will depend on Y
On Wed, May 28, 2014 at 1:53 PM, alex kamil <[email protected]> wrote: > Dmitry, > we've tried something similar, on a table with a few hundreds of VARCHAR > columns getting >30k items in IN clause was starting to exceed 60 sec, if > I remember correctly, > it basically becomes an IO bottleneck getting the huge resultsets back to > the client > > this was a workaround we used before JOINs became available in phoenix - > pulled a subset of data to the client using a batch of M queries with N > elements each, and did a join on client side in HSQLDB, which was messy and > inefficient > with phoenix 3.0 we switched to using joins on server side and avoid this > scenario completely > > Alex > > > > > On Wed, May 28, 2014 at 1:27 PM, Dmitry Goldenberg <[email protected] > > wrote: > >> Hi, >> >> I was wondering that the maximum value of elements in an IN clause is, in >> Phoenix. Also, from the performance standpoint, if I were to batch up the >> items into M statements of N elements per In clause and execute multiple >> statements to avoid ever hitting the max, what may be an optimal value for >> N, considering that I only have a few relatively small column values (int, >> varchar, timestamp) per statement? >> >> Thanks. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Phoenix HBase User" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > >
