[jira] [Commented] (SOLR-10816) Change uniqueKey to use docValues and not stored field
[ https://issues.apache.org/jira/browse/SOLR-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16678057#comment-16678057 ] Uwe Schindler commented on SOLR-10816: -- Hi Erick, thanks for reminding me and thanks for fixing SOLR-12625. I agree, the best would be to make the ID field stored and doc values enabled. This should be done anyways, as the ID field is most ofen of type "StrField", which has docvalues enabled by default for recent schema versions. But Maybe we should enforce some properties for "internal fields" like 'id', '_version' and similar. > Change uniqueKey to use docValues and not stored field > -- > > Key: SOLR-10816 > URL: https://issues.apache.org/jira/browse/SOLR-10816 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Varun Thacker >Priority: Major > > This issue is about the performance improvements you can get by avoiding > decompression during the first phase of a distributed search where only id > and score is needed. > The improvements will be noticed for users if the docs are large or have lots > of fields in them. > For users who don't have this scenario it shouldn't slow things done by any > noticeable amounts? > We should default the unique key field to use docValuues='true' and > stored='false' > Links to the discussion that lead to this idea: > - > https://issues.apache.org/jira/browse/SOLR-5478?focusedCommentId=16036951&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16036951 > - > http://mail-archives.apache.org/mod_mbox/lucene-dev/201706.mbox/%3C008201d2ddf9%2429435740%247bca05c0%24%40thetaphi.de%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10816) Change uniqueKey to use docValues and not stored field
[ https://issues.apache.org/jira/browse/SOLR-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677488#comment-16677488 ] Erick Erickson commented on SOLR-10816: --- This is probably obsolete at this point in at least the following way: Given the work on SOLR-12625, we shouldn't be paying the performance penalty for first-pass fetching of the doc ID. that does not address Uwe's comments about not storing the uniqueKey, so it's probably best to store it _and_ make it docValues. > Change uniqueKey to use docValues and not stored field > -- > > Key: SOLR-10816 > URL: https://issues.apache.org/jira/browse/SOLR-10816 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Varun Thacker >Priority: Major > > This issue is about the performance improvements you can get by avoiding > decompression during the first phase of a distributed search where only id > and score is needed. > The improvements will be noticed for users if the docs are large or have lots > of fields in them. > For users who don't have this scenario it shouldn't slow things done by any > noticeable amounts? > We should default the unique key field to use docValuues='true' and > stored='false' > Links to the discussion that lead to this idea: > - > https://issues.apache.org/jira/browse/SOLR-5478?focusedCommentId=16036951&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16036951 > - > http://mail-archives.apache.org/mod_mbox/lucene-dev/201706.mbox/%3C008201d2ddf9%2429435740%247bca05c0%24%40thetaphi.de%3E -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10816) Change uniqueKey to use docValues and not stored field
[ https://issues.apache.org/jira/browse/SOLR-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16042463#comment-16042463 ] Uwe Schindler commented on SOLR-10816: -- We have found out that there are some problems with changing the ID field to stored=false: - It currently prints a warning that distributed search does not work. This warning is obsolete. - MoreLikeThis breaks if the ID field is not stored. I think this is a bug that should be fixed! Morelikethis should use the same mechanism to fetch vield values like the rest of solr, and not rely on stored fields. I think this is some missing part in the "docvalues as stord fields" refactoring. > Change uniqueKey to use docValues and not stored field > -- > > Key: SOLR-10816 > URL: https://issues.apache.org/jira/browse/SOLR-10816 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Varun Thacker > > This issue is about the performance improvements you can get by avoiding > decompression during the first phase of a distributed search where only id > and score is needed. > The improvements will be noticed for users if the docs are large or have lots > of fields in them. > For users who don't have this scenario it shouldn't slow things done by any > noticeable amounts? > We should default the unique key field to use docValuues='true' and > stored='false' > Links to the discussion that lead to this idea: > - > https://issues.apache.org/jira/browse/SOLR-5478?focusedCommentId=16036951&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16036951 > - > http://mail-archives.apache.org/mod_mbox/lucene-dev/201706.mbox/%3C008201d2ddf9%2429435740%247bca05c0%24%40thetaphi.de%3E -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10816) Change uniqueKey to use docValues and not stored field
[ https://issues.apache.org/jira/browse/SOLR-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037374#comment-16037374 ] Erick Erickson commented on SOLR-10816: --- Uwe: Well, _I_ knew what I meant ;)... Right, I hacked in some code to pull the ID from the index on first pass rather than get it from stored (decompressed) values and saw about a 10% improvement in round-trip time. Note that this was all done locally so network overhead was minimal. > Change uniqueKey to use docValues and not stored field > -- > > Key: SOLR-10816 > URL: https://issues.apache.org/jira/browse/SOLR-10816 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Varun Thacker > > This issue is about the performance improvements you can get by avoiding > decompression during the first phase of a distributed search where only id > and score is needed. > The improvements will be noticed for users if the docs are large or have lots > of fields in them. > For users who don't have this scenario it shouldn't slow things done by any > noticeable amounts? > We should default the unique key field to use docValuues='true' and > stored='false' > Links to the discussion that lead to this idea: > - > https://issues.apache.org/jira/browse/SOLR-5478?focusedCommentId=16036951&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16036951 > - > http://mail-archives.apache.org/mod_mbox/lucene-dev/201706.mbox/%3C008201d2ddf9%2429435740%247bca05c0%24%40thetaphi.de%3E -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10816) Change uniqueKey to use docValues and not stored field
[ https://issues.apache.org/jira/browse/SOLR-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037312#comment-16037312 ] Uwe Schindler commented on SOLR-10816: -- bq. got about a 10% penalty Erick: The penalty is with stored fields, right? That means it is always a win! Sorry, I don't get your comment, maybe I misunderstood. > Change uniqueKey to use docValues and not stored field > -- > > Key: SOLR-10816 > URL: https://issues.apache.org/jira/browse/SOLR-10816 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Varun Thacker > > This issue is about the performance improvements you can get by avoiding > decompression during the first phase of a distributed search where only id > and score is needed. > The improvements will be noticed for users if the docs are large or have lots > of fields in them. > For users who don't have this scenario it shouldn't slow things done by any > noticeable amounts? > We should default the unique key field to use docValuues='true' and > stored='false' > Links to the discussion that lead to this idea: > - > https://issues.apache.org/jira/browse/SOLR-5478?focusedCommentId=16036951&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16036951 > - > http://mail-archives.apache.org/mod_mbox/lucene-dev/201706.mbox/%3C008201d2ddf9%2429435740%247bca05c0%24%40thetaphi.de%3E -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-10816) Change uniqueKey to use docValues and not stored field
[ https://issues.apache.org/jira/browse/SOLR-10816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037281#comment-16037281 ] Erick Erickson commented on SOLR-10816: --- bq: For users who don't have this scenario it shouldn't slow things done by any noticeable amounts? I did some crude timings in SOLR-6888 and got about a 10% penalty even on very small documents. The minimum block size is 16K so. > Change uniqueKey to use docValues and not stored field > -- > > Key: SOLR-10816 > URL: https://issues.apache.org/jira/browse/SOLR-10816 > Project: Solr > Issue Type: Improvement > Security Level: Public(Default Security Level. Issues are Public) >Reporter: Varun Thacker > > This issue is about the performance improvements you can get by avoiding > decompression during the first phase of a distributed search where only id > and score is needed. > The improvements will be noticed for users if the docs are large or have lots > of fields in them. > For users who don't have this scenario it shouldn't slow things done by any > noticeable amounts? > We should default the unique key field to use docValuues='true' and > stored='false' > Links to the discussion that lead to this idea: > - > https://issues.apache.org/jira/browse/SOLR-5478?focusedCommentId=16036951&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16036951 > - > http://mail-archives.apache.org/mod_mbox/lucene-dev/201706.mbox/%3C008201d2ddf9%2429435740%247bca05c0%24%40thetaphi.de%3E -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
