Re: [DISCUSS] IEP-71 Public API for secondary index search

Hi! The patch merged (IndexQuery with required index name). A lot of thanks to Andrew Mashenkov, Taras Ledkov and Alex Plekhanov for their time and reviews. By the way, I'd like to up the discussion about the optionality of the indexName param for creating IndexQuery. I believe that it's a useful

Re: [DISCUSS] IEP-71 Public API for secondary index search

Hi! It's a PR re-review request. I've reworked PR [1]. The major changes are: 1. indexName is a required param for building IndexQuery. 2. the order of criteria fields doesn't make sense - now it checks only that criteria is a prefix for index fields. 3. IndexQuery can accept a value type (only v

Re: [DISCUSS] IEP-71 Public API for secondary index search

Hi! Thanks all for your comments! 1. I'm OK on the first step to make the indexName param required and not check fields order for this case. So, I'll remove code with search index by fields from current PR. 2. But I propose to create a ticket for possible improvement - IndexQuery should be creat

Re: [DISCUSS] IEP-71 Public API for secondary index search

Prefer 1 from Teras' response. Specifying index name is preferred. I've seen customers do idx(A,B) and idx(B,A) where semantics change between the two. Regards, Courtney Robinson Founder and CEO, Hypi Tel: ++44 208 123 2413 (GMT+0) https://hypi.io On Thu, Aug

Re: [DISCUSS] IEP-71 Public API for secondary index search

Hi, My proposal: 1. Don't search index by criteria, specify the index name always (preferred). OR 2. Search index by criteria without check the order of criteriones. Use the Set of criterions instead of the ordered collection. In the strange case when the both index exist (a, b) and (b, a) -

Re: [DISCUSS] IEP-71 Public API for secondary index search

> But lt("b", 1) AND lt("a", 2) is equivalent to lt("a", 1) AND lt("b", 2) according with criteria API and javadoc of the 'IndexQuery#setCriteria' method. I updated javadoc, Andrey suggested writing this in javadocs, so now there is a note about field order. > If the user uses logical criteria in

Re: [DISCUSS] IEP-71 Public API for secondary index search

There are some thoughts about strict field order: 1. Index (A, B) is not equivalent to index (B, A). Some queries may have different performance on such indexes, and users have to specify the right index. What if both indexes exist? 2. We should avoid cases when a user uses in query only field B fo

Re: [DISCUSS] IEP-71 Public API for secondary index search

1. I suppose, that the next step is to implement the api for manually creating index. I think that user wants to create index that will speed up his criteria base queries, so he or she will use the same criteria to define the index. So no problem at all 2. We should print warning or throws exceptio

Re: [DISCUSS] IEP-71 Public API for secondary index search

Hi, It is an usability nightmare to make user write index name in all cases. I don't see any difference between specifying the index name and specifying the index fields in the right order. Do you see? Let's there is the index: idx_A_B ON TBL (A, B) Is it OK that the query like below doesn't

Re: [DISCUSS] IEP-71 Public API for secondary index search

@Ivan Daschinsky By the way, users are forced to specify index condition in order that match column order in the index. On Thu, Aug 26, 2021 at 3:24 PM Ivan Daschinsky wrote: > I am against to make user write index name. It is quite simple and > straightforward algorithm to match index to fiel

Re: [DISCUSS] IEP-71 Public API for secondary index search

I am against to make user write index name. It is quite simple and straightforward algorithm to match index to field names, so it is strange to compare it to sql engine optimizer. It is an usability nightmare to make user write index name in all cases. чт, 26 авг. 2021 г., 14:42 Maksim Timonin :

Re: [DISCUSS] IEP-71 Public API for secondary index search

Hi, Igniters! There is a discussion about how to specify an index to query with an IndexQuery [1]. Currently my PR provides 2 ways to specify index: 1. With a table and index name; 2. With a table and list of index fields (without index name). In this case IndexQueryProcessor tries to find an inde

Re: [DISCUSS] IEP-71 Public API for secondary index search

Hi, all! It's a gentle reminder. There is a PR for the new Index API [1]. It was approved by Alex Plekhanov. Does anybody want to review this API too? If there won't be objections we're going to merge it Monday, 16th of August. Thanks! [1] https://github.com/apache/ignite/pull/9118 On Fri, May

Re: [DISCUSS] IEP-71 Public API for secondary index search

Andrey, hi! Some updates, there. I've submitted a PR for IndexQuery [1]. There is an issue about lazy page loading, that is also related to Text query ticket IGNITE-12291. CacheQueries already have pending pages functionality, it's done with multiple sending GridCacheQueryRequest. There was an i

Re: [DISCUSS] IEP-71 Public API for secondary index search

Andrey, Thanks! I picked it. On Mon, Apr 12, 2021 at 1:51 PM Maksim Timonin wrote: > Stephen, > > I don't see a reason to replace or deprecate IndexingSpi. I'm not sure how > smbd uses it, but it works now. > > On Mon, Apr 12, 2021 at 1:42 PM Stephen Darlington < > stephen.darling...@gridgain.c

Re: [DISCUSS] IEP-71 Public API for secondary index search

Stephen, I don't see a reason to replace or deprecate IndexingSpi. I'm not sure how smbd uses it, but it works now. On Mon, Apr 12, 2021 at 1:42 PM Stephen Darlington < stephen.darling...@gridgain.com> wrote: > Is this a replacement for IndexingSpi? Put bluntly, do we deprecate (and > remove) it

Re: [DISCUSS] IEP-71 Public API for secondary index search

Is this a replacement for IndexingSpi? Put bluntly, do we deprecate (and remove) it? Or do you see them as complimentary? > On 12 Apr 2021, at 11:29, Maksim Timonin wrote: > > Hi Stephen! > > Please have a look at the QueryProcessing paragraph [1]. I've described > why IndexingSpi doesn't fit

Re: [DISCUSS] IEP-71 Public API for secondary index search

Maksim, Am I right, that you mean this ticket [1] *IGNITE-12291 Create controllable > paged query requests / responses for TextQuery similar to current SQL > result processing*, when talked about incorrect limit work for > TextQueries? > Yes, sure, that's it. On Mon, Apr 12, 2021 at 12:15 PM Maks

Re: [DISCUSS] IEP-71 Public API for secondary index search

Hi Stephen! Please have a look at the QueryProcessing paragraph [1]. I've described why IndexingSpi doesn't fit us well. [1] https://cwiki.apache.org/confluence/display/IGNITE/IEP-71+Public+API+for+secondary+index+search#IEP71PublicAPIforsecondaryindexsearch-2)QueryProcessing On Mon, Apr 12, 202

Re: [DISCUSS] IEP-71 Public API for secondary index search

How does this fit with the current IndexingSpi? Superficially they appear to do very similar things? Regards, Stephen > On 6 Apr 2021, at 14:13, Maksim Timonin wrote: > > Hi, Igniters! > > I'd like to propose a new feature - opportunity to query and create indexes > from public API. > > It w

Re: [DISCUSS] IEP-71 Public API for secondary index search

Hi, Andrey! Am I right, that you mean this ticket [1] *IGNITE-12291 Create controllable paged query requests / responses for TextQuery similar to current SQL result processing*, when talked about incorrect limit work for TextQueries? [1] https://issues.apache.org/jira/browse/IGNITE-12291 On Thu

Re: [DISCUSS] IEP-71 Public API for secondary index search

Hi, Andrey! >> ScanQuery, TextQuery and partially SQL query share the same infrastructure I think I understand what you mean. I debug query processing and now agree that it's a nice idea to try to reuse the infrastructure of scan and text queries. Also as I can see there already Reducer functional

Re: [DISCUSS] IEP-71 Public API for secondary index search

Maksim, The ScanQuery API provides a filter as > param that for case of index query should be splitted on such conditions. > It looks like a non-trivial task. > ScanQuery, TextQuery and partially SQL query share the same infrastructure. I've thought we could extend, improve and reuse some ScanQuer

Re: [DISCUSS] IEP-71 Public API for secondary index search

Hi, Andrey! Thanks for the review and your comments! >> Is it possible to extend ScanQuery functionality to pass index condition I investigated this way and see some issues: 1. Querying of indexes is not a scan actually. It's a tree traverse (predicate operation is an exclusion, other operations

Re: [DISCUSS] IEP-71 Public API for secondary index search

Hi Maksim, Nice idea, I'd like to see this feature in Ignite. The motivation is clear to me, it would be nice to have fast scans and omit SQL overhead on planning, parsing and etc in some simple use-cases. I've left few minor comments to the IEP, but I have the next questions which answer I faile

[DISCUSS] IEP-71 Public API for secondary index search

Hi, Igniters! I'd like to propose a new feature - opportunity to query and create indexes from public API. It will help in some cases, where: 1. SQL is not applicable by design of user application; 2. Where IndexScan is preferable than ScanQuery for performance reasons; 3. Functional indexes are