Hi, Afaiu your proposal would mix 2 different concerns: a) the query plan, which should be guided by the cost to execute a query and b) how up-to-date an index is. I do not see why b) should interfere with a) unless you have a situation where an async index is so far behind that it should not be used at all.
From your original mail: Sometimes we could have the same query better served by a property index, or traversing for example. Could you please specify “better” in the proposal above? Thanks! Michael On 04/11/15 17:37, "Davide Giannella" <[email protected]<mailto:[email protected]>> wrote: On 04/11/2015 00:49, Ian Boston wrote: ... Going down the property index route, for a DocumentMK instance will bloat the DocumentStore further. That already consumes 60% of a production repository and like many in DB inverted indexes is not an efficient storage structure. It's probably ok for TarMK. Traversals are a problem for production. They will create random outages under any sort of concurrent load. --- If the way the indexing was performed is changed, it could make the index NRT or real time depending on your point of view. eg. Local indexes, each Oak index in the cluster becoming a shard with replication to cover instance unavailability. No more indexing cycles, soft commits with each instance using a FS Directory and a update queue replacing the async indexing queue. Query by map reduce. It might have to copy on write to seed new instances where the number of instances falls below 3. I didn't mean to replace the lucene indexes with property or traversing. The index definitions remain the same. It's only on the cost of the plan that Lucene itself is aware of when the last successfull indexing and if lagging behind, corrects its own cost accordingly. Something like saying: hey, I'm normally ok in serving this query but I know I may be slightly/highly out of date and therefore I'm giving room to any other potential index that could serve the query. So if there won't be any other indexes to serve the query, lucene will still do it. Davide
