I changed the description in the ticket. If anyone has comments about the implementation details, feel free to add them there.
-Val On Wed, Feb 10, 2016 at 2:30 PM, Dmitriy Setrakyan <dsetrak...@apache.org> wrote: > I like the new method on IgniteCache API: > > <R> QueryCursor<R> query(Query<E> qry, Transformer<E, R> transformer); > > I do agree that it makes little sense for the SqlFieldsQuery, but we can > simply throw an unsupported exception here. For all other queries it does > make sense. > > Val, can you put together all the thoughts expressed in this thread into a > ticket? > > D. > > On Thu, Feb 4, 2016 at 4:14 PM, Andrey Gura <ag...@gridgain.com> wrote: > > > Val, > > > > From my point of view, special query class that allows transforming is > > confusing. Two points about it: > > > > 1. ScanTransformQuery will duplicate ScanQuery code with all drawbacks > that > > we have. Moreover, any fix for ScanQuery should be repeated for > > ScanTransformQuery. It will lead to bugs. DRY. > > > > 2. If some users want to do transformations for SqlQuery we will > introduce > > SqlTransformQuery. Right? At this point see previous item. > > > > Transformation logic is some kind of strategy. IMHO, the most convenient > > API should get transformation logic as some function provided by user. > > > > On Thu, Feb 4, 2016 at 11:05 PM, Valentin Kulichenko < > > valentin.kuliche...@gmail.com> wrote: > > > > > Agree with Sergi. Mixing SQL with Java code transformers is confusing. > In > > > rare case when it's really required, user can implement a custom > > function. > > > > > > I copy-pasted the API to the ticket [1]. Please provide any additional > > > comments there. > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-2546 > > > > > > -Val > > > > > > On Thu, Feb 4, 2016 at 10:08 AM, Andrey Gura <ag...@gridgain.com> > wrote: > > > > > > > Sergi, > > > > > > > > > > > > > What you are going to transform remotely here? > > > > > > > > > > > > I'm not going. I believe :) > > > > > > > > Just hypothetical use case: You have one SqlFieldsQuery but different > > > > requirements for returned values. For one case you have to return > some > > > > string fields as is and for another case you have to trim string to > 32 > > > > characters. Of course you still can trim strings locally but > > transformers > > > > allow you do it remotely. > > > > > > > > Anyway, this solution can be usefull for rest query types. > > > > > > > > On Thu, Feb 4, 2016 at 8:54 PM, Sergi Vladykin < > > sergi.vlady...@gmail.com > > > > > > > > wrote: > > > > > > > > > The whole point of Transformer is to do remote transform, how will > > this > > > > > work with SqlFieldsQuery? What you are going to transform remotely > > > here? > > > > I > > > > > believe all the transformations must happen at SQL level here. > > > > > > > > > > Sergi > > > > > > > > > > > > > > > > > > > > 2016-02-04 20:10 GMT+03:00 Andrey Gura <ag...@gridgain.com>: > > > > > > > > > > > SqlQuery, TextQuery and SpiQuery are similar to ScanQuery because > > all > > > > of > > > > > > them also defined as Query<Cache.Entry<>>. > > > > > > > > > > > > It can be usefull to have one query SqlQuery that can return > > > different > > > > > > result that will be produced from cache entry by transformer. > > > > > > > > > > > > Actually only SqlFieldsQuery has different definition. So > > > transformers > > > > > can > > > > > > be applied to any type of query (including SqlFieldsQuery, I > > > believe). > > > > > > > > > > > > On Thu, Feb 4, 2016 at 7:42 PM, Sergi Vladykin < > > > > sergi.vlady...@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > I don't like the idea of having additional method > *query(Query<E> > > > > qry, > > > > > > > Transformer<E, R> transfomer); *because I don't see how these > > > > > > transformers > > > > > > > will work for example with SQL, but this API makes you think > that > > > > > > > transformers are supported for all the query types. > > > > > > > > > > > > > > Sergi > > > > > > > > > > > > > > 2016-02-04 16:46 GMT+03:00 Andrey Gura <ag...@gridgain.com>: > > > > > > > > > > > > > > > Val, > > > > > > > > > > > > > > > > can we introduce new method into IgnteCache API? > > > > > > > > > > > > > > > > Now we have method: public <R> QueryCursor<R> query(Query<R> > > > qry); > > > > > > > > > > > > > > > > New method will be something like this: <R> QueryCursor<R> > > > > > > query(Query<E> > > > > > > > > qry, Transformer<E, R> transfomer); > > > > > > > > > > > > > > > > It allows provide transformers for all query types and > chnages > > > will > > > > > be > > > > > > > > related only with query cursor functionality. > > > > > > > > > > > > > > > > Will it work? > > > > > > > > > > > > > > > > On Thu, Feb 4, 2016 at 11:13 AM, Andrey Kornev < > > > > > > andrewkor...@hotmail.com > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Another perhaps bigger problem with running queries > > (including > > > > scan > > > > > > > > > queries) using closures was discussed at length on the @dev > > not > > > > so > > > > > > long > > > > > > > > > ago. It has to do with partitions migration due to cluster > > > > topology > > > > > > > > changes > > > > > > > > > which may result in the query returning incomplete result. > > And > > > > > while > > > > > > it > > > > > > > > is > > > > > > > > > possible to solve this problem for the scan queries by > using > > > some > > > > > > > clever > > > > > > > > > tricks, all bets are off with the SQL queries.Andrey > > > > > > > > > _____________________________ > > > > > > > > > From: Valentin Kulichenko <valentin.kuliche...@gmail.com> > > > > > > > > > Sent: Thursday, February 4, 2016 6:29 AM > > > > > > > > > Subject: Re: Transformers in SCAN queries > > > > > > > > > To: <dev@ignite.apache.org> > > > > > > > > > > > > > > > > > > > > > > > > > > > Dmitry, > > > > > > > > > > > > > > > > > > The main difference in my view is that you lose pagination > > > when > > > > > > > sending > > > > > > > > > results from servers to client. What if one wants to > iterate > > > > > through > > > > > > > all > > > > > > > > > entries in cache? > > > > > > > > > > > > > > > > > > On Wed, Feb 3, 2016 at 9:47 PM, Dmitriy Setrakyan < > > > > > > > > dsetrak...@apache.org> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Valentin, > > > > > > > > > > > > > > > > > > > > Wouldn’t the same effect be achieved by broadcasting a > > > closure > > > > > to > > > > > > > the > > > > > > > > > > cluster and executing scan-query on every node locally? > > > > > > > > > > > > > > > > > > > > D. > > > > > > > > > > > > > > > > > > > > On Wed, Feb 3, 2016 at 9:17 PM, Valentin Kulichenko < > > > > > > > > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > > > > > > > I keep getting requests from our users to add optional > > > > > > > transformers > > > > > > > > to > > > > > > > > > > SCAN > > > > > > > > > > > queries. This will allow to iterate through cache, but > > do > > > > not > > > > > > > > transfer > > > > > > > > > > > whole key-value pairs across networks (e.g., get only > > > keys). > > > > > The > > > > > > > > > feature > > > > > > > > > > > looks useful and I created a ticket [1]. > > > > > > > > > > > > > > > > > > > > > > I am struggling with the design now. The problem is > > that I > > > > > > wanted > > > > > > > to > > > > > > > > > > extend > > > > > > > > > > > existing ScanQuery object for this, but this seems to > be > > > > > > > impossible > > > > > > > > > > because > > > > > > > > > > > it already extends Query<Cache.Entry<K, V>> and thus > can > > > > > iterate > > > > > > > > only > > > > > > > > > > > through entries. > > > > > > > > > > > > > > > > > > > > > > The only option I see now is to create a separate > query > > > > type, > > > > > > > > > copy-paste > > > > > > > > > > > everything from ScanQuery and add *mandatory* > > transformer. > > > > > > > Something > > > > > > > > > like > > > > > > > > > > > this: > > > > > > > > > > > > > > > > > > > > > > ScanTransformQuery<K, V, R> extends Query<R> { > > > > > > > > > > > IgniteBiPredicate<K, V> filter; > > > > > > > > > > > IgniteClosure<Cache.Entry<K, V>, R> transformer; > > > > > > > > > > > int part; > > > > > > > > > > > ... > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > > Thoughts? Does anyone has other ideas? > > > > > > > > > > > > > > > > > > > > > > [1] > > https://issues.apache.org/jira/browse/IGNITE-2546 > > > > > > > > > > > > > > > > > > > > > > -Val > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Andrey Gura > > > > > > > > GridGain Systems, Inc. > > > > > > > > www.gridgain.com > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Andrey Gura > > > > > > GridGain Systems, Inc. > > > > > > www.gridgain.com > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Andrey Gura > > > > GridGain Systems, Inc. > > > > www.gridgain.com > > > > > > > > > > > > > > > -- > > Andrey Gura > > GridGain Systems, Inc. > > www.gridgain.com > > >
