[jira] [Updated] (SOLR-6845) figure out why suggester causes slow startup - even when not used
[
https://issues.apache.org/jira/browse/SOLR-6845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomás Fernández Löbbe updated SOLR-6845:
Attachment: SOLR-6845.patch
Here is a new patch. I saw that we were trying to load the suggester twice,
once on the Suggester init and once in the searcher listener. I removed the
load from the searcher listener.
I also changed the default of the "buildOnStartup", instead of being always
"false", it defaults to true if no suggester dictionary exists (if the storeDir
is not set or if it's set but the suggester was never built). This is
compatible with the behavior in 4.10.
Unfortunately I didn't have time to put this change in 5.0, there are no major
compatibility issues, the only one I see is that "reload" no longer builds if
the dictionary is not present. I think this is correct and should be the
behavior of 6.x, for 5.x I'll make the reload action build if no dictionary is
present.
> figure out why suggester causes slow startup - even when not used
> -
>
> Key: SOLR-6845
> URL: https://issues.apache.org/jira/browse/SOLR-6845
> Project: Solr
> Issue Type: Bug
>Reporter: Hoss Man
> Attachments: SOLR-6845.patch, SOLR-6845.patch, SOLR-6845.patch
>
>
> SOLR-6679 was filed to track the investigation into the following problem...
> {panel}
> The stock solrconfig provides a bad experience with a large index... start up
> Solr and it will spin at 100% CPU for minutes, unresponsive, while it
> apparently builds a suggester index.
> ...
> This is what I did:
> 1) indexed 10M very small docs (only takes a few minutes).
> 2) shut down Solr
> 3) start up Solr and watch it be unresponsive for over 4 minutes!
> I didn't even use any of the fields specified in the suggester config and I
> never called the suggest request handler.
> {panel}
> ..but ultimately focused on removing/disabling the suggester from the sample
> configs.
> Opening this new issue to focus on actually trying to identify the root
> problem & fix it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6845) figure out why suggester causes slow startup - even when not used
[
https://issues.apache.org/jira/browse/SOLR-6845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomás Fernández Löbbe updated SOLR-6845:
Attachment: SOLR-6845.patch
In this patch, I added a “buildOnStartup” for the suggester that defaults to
false. If this is not set, the suggester will load a dictionary if it exists,
but won’t create it if it doesn’t.
"buildOnStartup" will also build the suggester in case of a core reload. Users
should be aware that in both, “SolrCloud mode” and with a “master-slave” setup,
Solr may trigger a core reload internally, and if “buildOnStartup” is set, the
core reload will build the suggester (if it's not being stored). Unlike with
the current code a core reload won’t trigger a “buildOnCommit” event.
A side note is that, even if “useColdSearcher” is set to “false”, in case of a
core reload the suggester may be built after the first searcher is registered,
the reason for this is that it is built using the second searcher created in
the core reload process.
> figure out why suggester causes slow startup - even when not used
> -
>
> Key: SOLR-6845
> URL: https://issues.apache.org/jira/browse/SOLR-6845
> Project: Solr
> Issue Type: Bug
>Reporter: Hoss Man
> Attachments: SOLR-6845.patch, SOLR-6845.patch
>
>
> SOLR-6679 was filed to track the investigation into the following problem...
> {panel}
> The stock solrconfig provides a bad experience with a large index... start up
> Solr and it will spin at 100% CPU for minutes, unresponsive, while it
> apparently builds a suggester index.
> ...
> This is what I did:
> 1) indexed 10M very small docs (only takes a few minutes).
> 2) shut down Solr
> 3) start up Solr and watch it be unresponsive for over 4 minutes!
> I didn't even use any of the fields specified in the suggester config and I
> never called the suggest request handler.
> {panel}
> ..but ultimately focused on removing/disabling the suggester from the sample
> configs.
> Opening this new issue to focus on actually trying to identify the root
> problem & fix it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-6845) figure out why suggester causes slow startup - even when not used
[
https://issues.apache.org/jira/browse/SOLR-6845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tomás Fernández Löbbe updated SOLR-6845:
Attachment: SOLR-6845.patch
Here is a patch to add buildOnStartup. With buildOnStartup=false and
buildOnCommit=true, the suggester is loaded if it already exist, but not built
if it doesn't on startup.
The problem I see is that in regular core reload, searchers are open in a way
that the "newSearcherListeners" are invoked, this means that even if
buildOnStartup=false and buildOnCommit=true, a core reload would build the
suggester if not found
> figure out why suggester causes slow startup - even when not used
> -
>
> Key: SOLR-6845
> URL: https://issues.apache.org/jira/browse/SOLR-6845
> Project: Solr
> Issue Type: Bug
>Reporter: Hoss Man
> Attachments: SOLR-6845.patch
>
>
> SOLR-6679 was filed to track the investigation into the following problem...
> {panel}
> The stock solrconfig provides a bad experience with a large index... start up
> Solr and it will spin at 100% CPU for minutes, unresponsive, while it
> apparently builds a suggester index.
> ...
> This is what I did:
> 1) indexed 10M very small docs (only takes a few minutes).
> 2) shut down Solr
> 3) start up Solr and watch it be unresponsive for over 4 minutes!
> I didn't even use any of the fields specified in the suggester config and I
> never called the suggest request handler.
> {panel}
> ..but ultimately focused on removing/disabling the suggester from the sample
> configs.
> Opening this new issue to focus on actually trying to identify the root
> problem & fix it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
