Re: Regarding the Secondary Index write path

2018-10-31 Thread Abhishek Talluri
Thanks for confirming on the WALEdit Vincent. I did see there were multiple comments saying // skip this mutation if we aren't enabling indexing but there isn't a way to make it skip these indexing steps or is there a variable that needs to be set on a table level to make it skip indexing. IMO,

Re: Regarding the Secondary Index write path

2018-10-30 Thread Vincent Poon
Looks like you're right, the Indexer is loaded for all base tables. I got confused with the case where the Indexer coproc is not loaded for *index* tables. I wonder what the overhead is like for having that. It does seem from the code like loading it was intentional, as there are lines like this

Re: Regarding the Secondary Index write path

2018-10-30 Thread Abhishek Talluri
Thanks Vincent. Just to clarify, if I create a table through Phoenix and check the describe on the hbase table, I see that the Indexer co-processor is loaded with the table. Is there some code that loads only the required co-processors when there is an Observer event? Coming back to the sequence

Re: Regarding the Secondary Index write path

2018-10-30 Thread Vincent Poon
If the table has no indexes, the Indexer coprocessor won't be loaded. As for your original question, the answer is a little nuanced. For Phoenix 4.14+: The index updates are calculated in preBatchMutate, and the index updates are written in postBatchMutate. So from HRegion#doMiniBatchMutation,

Re: Regarding the Secondary Index write path

2018-10-30 Thread Abhishek Talluri
Thanks Geoffrey for confirming that. Will go through that presentation. I have a follow-up question though, Let’s say if a table does not have any indexes on it, will these co-processors still be triggered and try to calculate index updates, since these are loaded by default for any table that is

Re: Regarding the Secondary Index write path

2018-10-30 Thread Geoffrey Jacoby
Abhishek, You might want to check out Vincent Poon's excellent presentation at this year's PhoenixCon about recent changes over the past couple of years to the index pipeline. https://www.youtube.com/watch?v=VBONDM7sD40 One of those changes is the one you observed. Global mutable index writes