[
https://issues.apache.org/jira/browse/OAK-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496015#comment-14496015
]
Davide Giannella commented on OAK-2749:
---------------------------------------
[~chetanm]
{quote}
Did not understood about forcing new executor requirement. I think we just need
to add a new scheduled job (possibly provide a config to get such a job enabled)
{quote}
I forced a new executor so that it would be sure the "slow" part won't
affect the "fast" one. Didn't get how to add a new scheduled job
yet. Do you mind to provide it against my fork (pull request?)? I've
just merged the latest trunk in it.
{quote}
Further we would also need to take care on how to do the switch of an existing
index. Alex Parvulescu mentioned that if we start a new async index we would
need to take care of checkpoint state also. So we switch an existing index to
say 'async-slow' then we would also need to bootstrap the checkpoint state
otherwise it would trigger a full reindex.
{quote}
True. Didn't think about it. I think it's more a task that could be
taken care by some groovy script in the oak shell. Otherwise we'll
need some commit hook and in case of changes on the {{async}} aspect
will take care of the checkpoints.
{quote}
Also I am not sure if we would should allow such task to run in parallel or
ensure at a time one indexer is running (so as to not put load)
{quote}
As far as I understood, one indexer running at time is the current
implementation. That's why we could experience some delays on updating
the async indexes as in big repositories with frequent updates and big
documents the full-text could take some time to index.
Running the full-text on a separate thread in parallel will allow the
propery indexes (Lucene etc) to complete faster. The side effect is of
course more load on the system but I don't have any meaning on how to
quantify it now.
> Provide a "different lane" for slow indexers in async indexing
> --------------------------------------------------------------
>
> Key: OAK-2749
> URL: https://issues.apache.org/jira/browse/OAK-2749
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
> Reporter: Davide Giannella
> Assignee: Davide Giannella
> Fix For: 1.3.0
>
> Attachments: OAK-2749-rc1.diff
>
>
> In case of big repositories, asynchronous index like Lucene Property,
> could lag behind as slow indexes, for example Full Text, are taken
> care in the same thread pool.
> Provide a separate thread pool in which such indexes could be
> registered.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
