[jira] [Commented] (OAK-2718) NodeStateSolrServersObserver performs complete diff synchronously causing slowness in dispatch
[ https://issues.apache.org/jira/browse/OAK-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14483525#comment-14483525 ] Tommaso Teofili commented on OAK-2718: -- sure, it can be done, as I said this would turn the option of using a persisted Solr configuration off, but I guess it's an acceptable compromise. > NodeStateSolrServersObserver performs complete diff synchronously causing > slowness in dispatch > -- > > Key: OAK-2718 > URL: https://issues.apache.org/jira/browse/OAK-2718 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-solr >Reporter: Chetan Mehrotra >Assignee: Tommaso Teofili > Fix For: 1.2 > > > {{NodeStateSolrServersObserver}} is enabled by default and performs diff > synchronously. Further it performs complete diff which might take time and > would cause the dispatch thread to slowdown. > This would cause issues at least with {{DocumentNodeStore}} as there the > dispatch is done as part of background read and that call is time sensitive. > As a fix the diff should performed asynchronously and also be selective. A > similar fix was done for Lucene index as part of OAK-2570 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2718) NodeStateSolrServersObserver performs complete diff synchronously causing slowness in dispatch
[ https://issues.apache.org/jira/browse/OAK-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14483345#comment-14483345 ] Chetan Mehrotra commented on OAK-2718: -- [~teofili] Can we also disable it by default and let it be registered only if some OSGi config is provided. That can be simply achieved by marking the component with ConfigurationPolicy REQUIRED. As most of the deployments would not have Solr configured this should be fine and not cause much inconvenience. An observer doing the complete diff would still add to the cost. Later once we make it precise we can let it be enabled by default > NodeStateSolrServersObserver performs complete diff synchronously causing > slowness in dispatch > -- > > Key: OAK-2718 > URL: https://issues.apache.org/jira/browse/OAK-2718 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-solr >Reporter: Chetan Mehrotra >Assignee: Tommaso Teofili > Fix For: 1.2 > > > {{NodeStateSolrServersObserver}} is enabled by default and performs diff > synchronously. Further it performs complete diff which might take time and > would cause the dispatch thread to slowdown. > This would cause issues at least with {{DocumentNodeStore}} as there the > dispatch is done as part of background read and that call is time sensitive. > As a fix the diff should performed asynchronously and also be selective. A > similar fix was done for Lucene index as part of OAK-2570 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2718) NodeStateSolrServersObserver performs complete diff synchronously causing slowness in dispatch
[ https://issues.apache.org/jira/browse/OAK-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14483228#comment-14483228 ] Tommaso Teofili commented on OAK-2718: -- in r1671853 I've made the {{NodeStateSolrServersObserver}} be wrapped by a {{BackgroundObserver}} in {{NodeStateSolrServersObserverService}} to make the observer work asynchronously. > NodeStateSolrServersObserver performs complete diff synchronously causing > slowness in dispatch > -- > > Key: OAK-2718 > URL: https://issues.apache.org/jira/browse/OAK-2718 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-solr >Reporter: Chetan Mehrotra >Assignee: Tommaso Teofili > Fix For: 1.2 > > > {{NodeStateSolrServersObserver}} is enabled by default and performs diff > synchronously. Further it performs complete diff which might take time and > would cause the dispatch thread to slowdown. > This would cause issues at least with {{DocumentNodeStore}} as there the > dispatch is done as part of background read and that call is time sensitive. > As a fix the diff should performed asynchronously and also be selective. A > similar fix was done for Lucene index as part of OAK-2570 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2718) NodeStateSolrServersObserver performs complete diff synchronously causing slowness in dispatch
[ https://issues.apache.org/jira/browse/OAK-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14482778#comment-14482778 ] Tommaso Teofili commented on OAK-2718: -- bq. Enable the observer via OSGi config i.e. it should work only if configured that could be done, but would make it possible to use persisted configurations for Solr index (OAK-2526) only upon configuration. bq. Wrap the observor with BackgroundObserver. See fix done for OAK-2570 for similar issue sure {quote} For proper fix (which might take some time) i.e. diff only selective part we can take approach similar to one in Lucene. The IndexTracker tracks which all Lucene index are in use and then diff is only performed for those paths. This keeps the diff logic short and precise. {quote} while working on OAK-2526 I had thought of such an approach however it would have meant reinventing / recreating kind of the same code you have been writing for the Lucene index, that would've required quite some changes in the way the Solr index works so I decided not to go that path. However I really think that we would really benefit from abstracting the generic part of the Lucene index in oak-core (definitions, aggregates, rules, etc.) to allow writing of implementation specific bits only for Lucene and Solr. That would allow to reduce the amount of per index maintenance and eventually have generic features implemented once instead of having to code them for each implementation. I am aware that this may require some time, but I think it's 100% worth the effort, at least for full text indexes. > NodeStateSolrServersObserver performs complete diff synchronously causing > slowness in dispatch > -- > > Key: OAK-2718 > URL: https://issues.apache.org/jira/browse/OAK-2718 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-solr >Reporter: Chetan Mehrotra >Assignee: Tommaso Teofili > Fix For: 1.2 > > > {{NodeStateSolrServersObserver}} is enabled by default and performs diff > synchronously. Further it performs complete diff which might take time and > would cause the dispatch thread to slowdown. > This would cause issues at least with {{DocumentNodeStore}} as there the > dispatch is done as part of background read and that call is time sensitive. > As a fix the diff should performed asynchronously and also be selective. A > similar fix was done for Lucene index as part of OAK-2570 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2718) NodeStateSolrServersObserver performs complete diff synchronously causing slowness in dispatch
[ https://issues.apache.org/jira/browse/OAK-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14394183#comment-14394183 ] Chetan Mehrotra commented on OAK-2718: -- [~teofili] I can think of following options. For 1.2 * Enable the observer via OSGi config i.e. it should work only if configured * Wrap the observor with BackgroundObserver. See fix done for OAK-2570 for similar issue For proper fix (which might take some time) i.e. diff only selective part we can take approach similar to one in Lucene. The {{IndexTracker}} tracks which all Lucene index are in use and then diff is only performed for those paths. This keeps the diff logic short and precise. > NodeStateSolrServersObserver performs complete diff synchronously causing > slowness in dispatch > -- > > Key: OAK-2718 > URL: https://issues.apache.org/jira/browse/OAK-2718 > Project: Jackrabbit Oak > Issue Type: Bug > Components: oak-solr >Reporter: Chetan Mehrotra >Assignee: Tommaso Teofili > Fix For: 1.2 > > > {{NodeStateSolrServersObserver}} is enabled by default and performs diff > synchronously. Further it performs complete diff which might take time and > would cause the dispatch thread to slowdown. > This would cause issues at least with {{DocumentNodeStore}} as there the > dispatch is done as part of background read and that call is time sensitive. > As a fix the diff should performed asynchronously and also be selective. A > similar fix was done for Lucene index as part of OAK-2570 -- This message was sent by Atlassian JIRA (v6.3.4#6332)