[jira] [Commented] (SOLR-7996) Evaluate moving SolrIndexSearcher creation logic to a factory
[ https://issues.apache.org/jira/browse/SOLR-7996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15243273#comment-15243273 ] Jamie Johnson commented on SOLR-7996: - [~tomasflobbe], it looks like some of what was proposed in 5621 could have helped with this though like you say it is far more ambitious. Having a way to better control the creation of the SolrIndexSearcher and ultimately things like what stored field visitor to use would be really beneficial. > Evaluate moving SolrIndexSearcher creation logic to a factory > - > > Key: SOLR-7996 > URL: https://issues.apache.org/jira/browse/SOLR-7996 > Project: Solr > Issue Type: Improvement >Reporter: Tomás Fernández Löbbe > > Moving this logic away from SolrCore is already a win, plus it should make it > easier to unit test and extend for advanced use cases. > See discussion here: http://search-lucene.com/m/eHNlWNCtoeLwQp -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8466) Add support for UnInvertedField based faceting to FacetComponent
[ https://issues.apache.org/jira/browse/SOLR-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15114311#comment-15114311 ] Jamie Johnson commented on SOLR-8466: - I like the approach as it's much less invasive. In regards to testing isn't this just a matter of piggy backing on the existing distributed faceting/exclusion tests to add uif as a method? I'm assuming those already exist? > Add support for UnInvertedField based faceting to FacetComponent > > > Key: SOLR-8466 > URL: https://issues.apache.org/jira/browse/SOLR-8466 > Project: Solr > Issue Type: New Feature > Components: Facet Module, faceting >Affects Versions: 5.0, 5.1, 5.2, 5.2.1, 5.3, 5.3.1, 5.4 >Reporter: Jamie Johnson >Assignee: Mikhail Khludnev > Attachments: 8466.diff, SOLR-8466.patch > > > The new JSON Faceting API supports building an UnInvertedField to do faceting > which would be beneficial to add to Solr FacetComponent. This would provide > additional options to implementors to choose the appropriate method of > faceting for their particular use case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-8466) Add support for UnInvertedField based faceting to FacetComponent
[ https://issues.apache.org/jira/browse/SOLR-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113485#comment-15113485 ] Jamie Johnson edited comment on SOLR-8466 at 1/23/16 2:28 AM: -- [~mkhludnev], I'm assuming you're referring to splitting out the FacetContext, FacetParser, FacetTopParser, FacetQueryParser, FacetFieldParser and FacetRangeParser? If so FacetContext, FacetParser, FacetTopParser all are used by SimpleFacets. It felt a little strange to leave the others (FacetQueryParser, FacetFieldParser and FacetRangeParser) as protected within FacetRequest after moving the others out but I could move them back if that makes more sense. was (Author: jej2003): I'm assuming you're referring to splitting out the FacetContext, FacetParser, FacetTopParser, FacetQueryParser, FacetFieldParser and FacetRangeParser? If so FacetContext, FacetParser, FacetTopParser all are used by SimpleFacets. It felt a little strange to leave the others (FacetQueryParser, FacetFieldParser and FacetRangeParser) as protected within FacetRequest after moving the others out but I could move them back if that makes more sense. Updated patch name in the mean time > Add support for UnInvertedField based faceting to FacetComponent > > > Key: SOLR-8466 > URL: https://issues.apache.org/jira/browse/SOLR-8466 > Project: Solr > Issue Type: New Feature > Components: Facet Module, faceting >Affects Versions: 5.0, 5.1, 5.2, 5.2.1, 5.3, 5.3.1, 5.4 >Reporter: Jamie Johnson >Assignee: Mikhail Khludnev > Attachments: 8466.diff > > > The new JSON Faceting API supports building an UnInvertedField to do faceting > which would be beneficial to add to Solr FacetComponent. This would provide > additional options to implementors to choose the appropriate method of > faceting for their particular use case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8466) Add support for UnInvertedField based faceting to FacetComponent
[ https://issues.apache.org/jira/browse/SOLR-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113485#comment-15113485 ] Jamie Johnson commented on SOLR-8466: - I'm assuming you're referring to splitting out the FacetContext, FacetParser, FacetTopParser, FacetQueryParser, FacetFieldParser and FacetRangeParser? If so FacetContext, FacetParser, FacetTopParser all are used by SimpleFacets. It felt a little strange to leave the others (FacetQueryParser, FacetFieldParser and FacetRangeParser) as protected within FacetRequest after moving the others out but I could move them back if that makes more sense. Updated patch name in the mean time > Add support for UnInvertedField based faceting to FacetComponent > > > Key: SOLR-8466 > URL: https://issues.apache.org/jira/browse/SOLR-8466 > Project: Solr > Issue Type: New Feature > Components: Facet Module, faceting >Affects Versions: 5.0, 5.1, 5.2, 5.2.1, 5.3, 5.3.1, 5.4 >Reporter: Jamie Johnson >Assignee: Mikhail Khludnev > Attachments: 8466.diff > > > The new JSON Faceting API supports building an UnInvertedField to do faceting > which would be beneficial to add to Solr FacetComponent. This would provide > additional options to implementors to choose the appropriate method of > faceting for their particular use case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-8466) Add support for UnInvertedField based faceting to FacetComponent
[ https://issues.apache.org/jira/browse/SOLR-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jamie Johnson updated SOLR-8466: Attachment: 8466.diff updated patch name > Add support for UnInvertedField based faceting to FacetComponent > > > Key: SOLR-8466 > URL: https://issues.apache.org/jira/browse/SOLR-8466 > Project: Solr > Issue Type: New Feature > Components: Facet Module, faceting >Affects Versions: 5.0, 5.1, 5.2, 5.2.1, 5.3, 5.3.1, 5.4 >Reporter: Jamie Johnson >Assignee: Mikhail Khludnev > Attachments: 8466.diff > > > The new JSON Faceting API supports building an UnInvertedField to do faceting > which would be beneficial to add to Solr FacetComponent. This would provide > additional options to implementors to choose the appropriate method of > faceting for their particular use case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-8466) Add support for UnInvertedField based faceting to FacetComponent
[ https://issues.apache.org/jira/browse/SOLR-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jamie Johnson updated SOLR-8466: Attachment: (was: patch.diff) > Add support for UnInvertedField based faceting to FacetComponent > > > Key: SOLR-8466 > URL: https://issues.apache.org/jira/browse/SOLR-8466 > Project: Solr > Issue Type: New Feature > Components: Facet Module, faceting >Affects Versions: 5.0, 5.1, 5.2, 5.2.1, 5.3, 5.3.1, 5.4 >Reporter: Jamie Johnson >Assignee: Mikhail Khludnev > Attachments: 8466.diff > > > The new JSON Faceting API supports building an UnInvertedField to do faceting > which would be beneficial to add to Solr FacetComponent. This would provide > additional options to implementors to choose the appropriate method of > faceting for their particular use case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8466) Add support for UnInvertedField based faceting to FacetComponent
[ https://issues.apache.org/jira/browse/SOLR-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15080581#comment-15080581 ] Jamie Johnson commented on SOLR-8466: - Correct, the patch adds facet.method=uif but I'd really like someone who is familiar with the JSON Faceting API to give this a go and make sure things look right. > Add support for UnInvertedField based faceting to FacetComponent > > > Key: SOLR-8466 > URL: https://issues.apache.org/jira/browse/SOLR-8466 > Project: Solr > Issue Type: New Feature > Components: Facet Module, faceting >Affects Versions: 5.0, 5.1, 5.2, 5.2.1, 5.3, 5.3.1, 5.4 >Reporter: Jamie Johnson > Attachments: patch.diff > > > The new JSON Faceting API supports building an UnInvertedField to do faceting > which would be beneficial to add to Solr FacetComponent. This would provide > additional options to implementors to choose the appropriate method of > faceting for their particular use case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-8466) Add support for UnInvertedField based faceting to FacetComponent
[ https://issues.apache.org/jira/browse/SOLR-8466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jamie Johnson updated SOLR-8466: Attachment: patch.diff Initial implementation to add support for UnInvertedField based faceting to FacetComponent > Add support for UnInvertedField based faceting to FacetComponent > > > Key: SOLR-8466 > URL: https://issues.apache.org/jira/browse/SOLR-8466 > Project: Solr > Issue Type: New Feature > Components: Facet Module, faceting >Affects Versions: 5.0, 5.1, 5.2, 5.2.1, 5.3, 5.3.1, 5.4 >Reporter: Jamie Johnson > Attachments: patch.diff > > > The new JSON Faceting API supports building an UnInvertedField to do faceting > which would be beneficial to add to Solr FacetComponent. This would provide > additional options to implementors to choose the appropriate method of > faceting for their particular use case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-8466) Add support for UnInvertedField based faceting to FacetComponent
Jamie Johnson created SOLR-8466: --- Summary: Add support for UnInvertedField based faceting to FacetComponent Key: SOLR-8466 URL: https://issues.apache.org/jira/browse/SOLR-8466 Project: Solr Issue Type: New Feature Components: Facet Module, faceting Affects Versions: 5.4, 5.3.1, 5.3, 5.2.1, 5.2, 5.1, 5.0 Reporter: Jamie Johnson The new JSON Faceting API supports building an UnInvertedField to do faceting which would be beneficial to add to Solr FacetComponent. This would provide additional options to implementors to choose the appropriate method of faceting for their particular use case. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-8096) Major faceting performance regressions
[ https://issues.apache.org/jira/browse/SOLR-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jamie Johnson updated SOLR-8096: Attachment: simple_facets.diff > Major faceting performance regressions > -- > > Key: SOLR-8096 > URL: https://issues.apache.org/jira/browse/SOLR-8096 > Project: Solr > Issue Type: Bug >Affects Versions: 5.0, 5.1, 5.2, 5.3, Trunk >Reporter: Yonik Seeley >Priority: Critical > Attachments: simple_facets.diff > > > Use of the highly optimized faceting that Solr had for multi-valued fields > over relatively static indexes was removed as part of LUCENE-5666, causing > severe performance regressions. > Here are some quick benchmarks to gauge the damage, on a 5M document index, > with each field having between 0 and 5 values per document. *Higher numbers > represent worse 5x performance*. > Solr 5.4_dev faceting time as a percent of Solr 4.10.3 faceting time > ||...|| Percent of index being faceted > ||num_unique_values|| 10% || 50% || 90% || > |10 | 351.17% | 1587.08% | 3057.28% | > |100 | 158.10% | 203.61% | 1421.93% | > |1000 | 143.78% | 168.01% | 1325.87% | > |1| 137.98% | 175.31% | 1233.97% | > |10 | 142.98% | 159.42% | 1252.45% | > |100 | 255.15% | 165.17% | 1236.75% | > For example, a field with 1000 unique values in the whole index, faceting > with 5x took 143% of the 4x time, when ~10% of the docs in the index were > faceted. > One user who brought the performance problem to our attention: > http://markmail.org/message/ekmqh4ocbkwxv3we > "faceting is unusable slow since upgrade to 5.3.0" (from 4.10.3) > The disabling of the UnInvertedField algorithm was previously discovered in > SOLR-7190, but we didn't know just how bad the problem was at that time. > edit: removed "secret" adverb by request -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-8096) Major faceting performance regressions
[ https://issues.apache.org/jira/browse/SOLR-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jamie Johnson updated SOLR-8096: Attachment: (was: simple_facets.diff) > Major faceting performance regressions > -- > > Key: SOLR-8096 > URL: https://issues.apache.org/jira/browse/SOLR-8096 > Project: Solr > Issue Type: Bug >Affects Versions: 5.0, 5.1, 5.2, 5.3, Trunk >Reporter: Yonik Seeley >Priority: Critical > > Use of the highly optimized faceting that Solr had for multi-valued fields > over relatively static indexes was removed as part of LUCENE-5666, causing > severe performance regressions. > Here are some quick benchmarks to gauge the damage, on a 5M document index, > with each field having between 0 and 5 values per document. *Higher numbers > represent worse 5x performance*. > Solr 5.4_dev faceting time as a percent of Solr 4.10.3 faceting time > ||...|| Percent of index being faceted > ||num_unique_values|| 10% || 50% || 90% || > |10 | 351.17% | 1587.08% | 3057.28% | > |100 | 158.10% | 203.61% | 1421.93% | > |1000 | 143.78% | 168.01% | 1325.87% | > |1| 137.98% | 175.31% | 1233.97% | > |10 | 142.98% | 159.42% | 1252.45% | > |100 | 255.15% | 165.17% | 1236.75% | > For example, a field with 1000 unique values in the whole index, faceting > with 5x took 143% of the 4x time, when ~10% of the docs in the index were > faceted. > One user who brought the performance problem to our attention: > http://markmail.org/message/ekmqh4ocbkwxv3we > "faceting is unusable slow since upgrade to 5.3.0" (from 4.10.3) > The disabling of the UnInvertedField algorithm was previously discovered in > SOLR-7190, but we didn't know just how bad the problem was at that time. > edit: removed "secret" adverb by request -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-8096) Major faceting performance regressions
[ https://issues.apache.org/jira/browse/SOLR-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jamie Johnson updated SOLR-8096: Attachment: simple_facets.diff > Major faceting performance regressions > -- > > Key: SOLR-8096 > URL: https://issues.apache.org/jira/browse/SOLR-8096 > Project: Solr > Issue Type: Bug >Affects Versions: 5.0, 5.1, 5.2, 5.3, Trunk >Reporter: Yonik Seeley >Priority: Critical > Attachments: simple_facets.diff > > > Use of the highly optimized faceting that Solr had for multi-valued fields > over relatively static indexes was removed as part of LUCENE-5666, causing > severe performance regressions. > Here are some quick benchmarks to gauge the damage, on a 5M document index, > with each field having between 0 and 5 values per document. *Higher numbers > represent worse 5x performance*. > Solr 5.4_dev faceting time as a percent of Solr 4.10.3 faceting time > ||...|| Percent of index being faceted > ||num_unique_values|| 10% || 50% || 90% || > |10 | 351.17% | 1587.08% | 3057.28% | > |100 | 158.10% | 203.61% | 1421.93% | > |1000 | 143.78% | 168.01% | 1325.87% | > |1| 137.98% | 175.31% | 1233.97% | > |10 | 142.98% | 159.42% | 1252.45% | > |100 | 255.15% | 165.17% | 1236.75% | > For example, a field with 1000 unique values in the whole index, faceting > with 5x took 143% of the 4x time, when ~10% of the docs in the index were > faceted. > One user who brought the performance problem to our attention: > http://markmail.org/message/ekmqh4ocbkwxv3we > "faceting is unusable slow since upgrade to 5.3.0" (from 4.10.3) > The disabling of the UnInvertedField algorithm was previously discovered in > SOLR-7190, but we didn't know just how bad the problem was at that time. > edit: removed "secret" adverb by request -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8096) Major faceting performance regressions
[ https://issues.apache.org/jira/browse/SOLR-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15065639#comment-15065639 ] Jamie Johnson commented on SOLR-8096: - Sorry if this is the incorrect place for this, but I took a stab at trying to implement supporting uninverted field based facets in SimpleFacets. i am not sure it's 100% there but I think it's close. The basic approach was to leverage as much as possible from the JSON Faceting API since that is the only consumer of the UIF that I could find. This meant I had to make some classes public that were previously package protected (Perhaps moving SimpleFacets into the facet package would have been better?). Additionally, I had to make FacetProcessor aware of Grouping so that the docset would be adjusted appropriately for grouping requests with truncate set to true. Also I did this by adding DV as a new FacetMethod and made it so this is what triggers using DocValues vs FC which currently triggers it. Perhaps it would be more appropriate to add a new FacetMethod named UIF and leave FC alone? I'm open to suggestions here. Last significant difference from the 4.10.4 implementation is I didn't attempt to use the Lucene FieldCache at all since it was made package protected. The 4.10.4 implementation used that in cases, but this should be inline with what JSON Facets is doing. The commits are attached as a patch to this ticket (I'm happy to spawn off a new ticket if it's more appropriate) and also available at https://github.com/jej2003/lucene-solr > Major faceting performance regressions > -- > > Key: SOLR-8096 > URL: https://issues.apache.org/jira/browse/SOLR-8096 > Project: Solr > Issue Type: Bug >Affects Versions: 5.0, 5.1, 5.2, 5.3, Trunk >Reporter: Yonik Seeley >Priority: Critical > > Use of the highly optimized faceting that Solr had for multi-valued fields > over relatively static indexes was removed as part of LUCENE-5666, causing > severe performance regressions. > Here are some quick benchmarks to gauge the damage, on a 5M document index, > with each field having between 0 and 5 values per document. *Higher numbers > represent worse 5x performance*. > Solr 5.4_dev faceting time as a percent of Solr 4.10.3 faceting time > ||...|| Percent of index being faceted > ||num_unique_values|| 10% || 50% || 90% || > |10 | 351.17% | 1587.08% | 3057.28% | > |100 | 158.10% | 203.61% | 1421.93% | > |1000 | 143.78% | 168.01% | 1325.87% | > |1| 137.98% | 175.31% | 1233.97% | > |10 | 142.98% | 159.42% | 1252.45% | > |100 | 255.15% | 165.17% | 1236.75% | > For example, a field with 1000 unique values in the whole index, faceting > with 5x took 143% of the 4x time, when ~10% of the docs in the index were > faceted. > One user who brought the performance problem to our attention: > http://markmail.org/message/ekmqh4ocbkwxv3we > "faceting is unusable slow since upgrade to 5.3.0" (from 4.10.3) > The disabling of the UnInvertedField algorithm was previously discovered in > SOLR-7190, but we didn't know just how bad the problem was at that time. > edit: removed "secret" adverb by request -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-8096) Major faceting performance regressions
[ https://issues.apache.org/jira/browse/SOLR-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15065185#comment-15065185 ] Jamie Johnson edited comment on SOLR-8096 at 12/19/15 5:12 AM: --- While some (all?) of the performance issues are addressed, would it not still be useful to add an option to support either faceting approach? I understand the benefits of DocValues but we have a case where the facets need to be calculated based on an access level the user has. Simply storing in a separate field is not an option because the access controls are complex. Given that the JSON Facet API allows developers to choose the faceting method it would seem reasonable to provide similar functionality here, no? Perhaps support the original implementation as the approach when method is fc and add a dv method to support docvalues. This would be inline with the new JSON API I believe, though from the looks of things it is not a trivial patch since the SimpleFacets seems pretty out of sync with the new faceting approach required in regards to using the UnInvertedField was (Author: jej2003): While some (all?) of the performance issues are addressed, would it not still be useful to add an option to support either faceting approach? I understand the benefits of DocValues but we have a case where the facets need to be calculated based on an access level the user has. Simply storing in a separate field is not an option because the access controls are complex. Given that the JSON Facet API allows developers to choose the faceting method it would seem reasonable to provide similar functionality here, no? It would seem a fairly trivial patch to support the original implementation as the approach when method is fc and add a dv method to support docvalues. This would be inline with the new JSON API I believe. > Major faceting performance regressions > -- > > Key: SOLR-8096 > URL: https://issues.apache.org/jira/browse/SOLR-8096 > Project: Solr > Issue Type: Bug >Affects Versions: 5.0, 5.1, 5.2, 5.3, Trunk >Reporter: Yonik Seeley >Priority: Critical > > Use of the highly optimized faceting that Solr had for multi-valued fields > over relatively static indexes was removed as part of LUCENE-5666, causing > severe performance regressions. > Here are some quick benchmarks to gauge the damage, on a 5M document index, > with each field having between 0 and 5 values per document. *Higher numbers > represent worse 5x performance*. > Solr 5.4_dev faceting time as a percent of Solr 4.10.3 faceting time > ||...|| Percent of index being faceted > ||num_unique_values|| 10% || 50% || 90% || > |10 | 351.17% | 1587.08% | 3057.28% | > |100 | 158.10% | 203.61% | 1421.93% | > |1000 | 143.78% | 168.01% | 1325.87% | > |1| 137.98% | 175.31% | 1233.97% | > |10 | 142.98% | 159.42% | 1252.45% | > |100 | 255.15% | 165.17% | 1236.75% | > For example, a field with 1000 unique values in the whole index, faceting > with 5x took 143% of the 4x time, when ~10% of the docs in the index were > faceted. > One user who brought the performance problem to our attention: > http://markmail.org/message/ekmqh4ocbkwxv3we > "faceting is unusable slow since upgrade to 5.3.0" (from 4.10.3) > The disabling of the UnInvertedField algorithm was previously discovered in > SOLR-7190, but we didn't know just how bad the problem was at that time. > edit: removed "secret" adverb by request -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-8096) Major faceting performance regressions
[ https://issues.apache.org/jira/browse/SOLR-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15065185#comment-15065185 ] Jamie Johnson edited comment on SOLR-8096 at 12/19/15 3:31 AM: --- While some (all?) of the performance issues are addressed, would it not still be useful to add an option to support either faceting approach? I understand the benefits of DocValues but we have a case where the facets need to be calculated based on an access level the user has. Simply storing in a separate field is not an option because the access controls are complex. Given that the JSON Facet API allows developers to choose the faceting method it would seem reasonable to provide similar functionality here, no? It would seem a fairly trivial patch to support the original implementation as the approach when method is fc and add a dv method to support docvalues. This would be inline with the new JSON API I believe. was (Author: jej2003): While some (all?) of the performance issues are addressed, would it not still be useful to add an option to support either faceting approach? I understand the benefits of DocValues but we have a case where the facets need to be calculated based on an access level the user has. Simply storing in a separate field is not an option because the access controls are complex. Given that the JSON Facet API allows developers to choose the faceting method it would seem reasonable to provide similar functionality here, no? > Major faceting performance regressions > -- > > Key: SOLR-8096 > URL: https://issues.apache.org/jira/browse/SOLR-8096 > Project: Solr > Issue Type: Bug >Affects Versions: 5.0, 5.1, 5.2, 5.3, Trunk >Reporter: Yonik Seeley >Priority: Critical > > Use of the highly optimized faceting that Solr had for multi-valued fields > over relatively static indexes was removed as part of LUCENE-5666, causing > severe performance regressions. > Here are some quick benchmarks to gauge the damage, on a 5M document index, > with each field having between 0 and 5 values per document. *Higher numbers > represent worse 5x performance*. > Solr 5.4_dev faceting time as a percent of Solr 4.10.3 faceting time > ||...|| Percent of index being faceted > ||num_unique_values|| 10% || 50% || 90% || > |10 | 351.17% | 1587.08% | 3057.28% | > |100 | 158.10% | 203.61% | 1421.93% | > |1000 | 143.78% | 168.01% | 1325.87% | > |1| 137.98% | 175.31% | 1233.97% | > |10 | 142.98% | 159.42% | 1252.45% | > |100 | 255.15% | 165.17% | 1236.75% | > For example, a field with 1000 unique values in the whole index, faceting > with 5x took 143% of the 4x time, when ~10% of the docs in the index were > faceted. > One user who brought the performance problem to our attention: > http://markmail.org/message/ekmqh4ocbkwxv3we > "faceting is unusable slow since upgrade to 5.3.0" (from 4.10.3) > The disabling of the UnInvertedField algorithm was previously discovered in > SOLR-7190, but we didn't know just how bad the problem was at that time. > edit: removed "secret" adverb by request -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8096) Major faceting performance regressions
[ https://issues.apache.org/jira/browse/SOLR-8096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15065185#comment-15065185 ] Jamie Johnson commented on SOLR-8096: - While some (all?) of the performance issues are addressed, would it not still be useful to add an option to support either faceting approach? I understand the benefits of DocValues but we have a case where the facets need to be calculated based on an access level the user has. Simply storing in a separate field is not an option because the access controls are complex. Given that the JSON Facet API allows developers to choose the faceting method it would seem reasonable to provide similar functionality here, no? > Major faceting performance regressions > -- > > Key: SOLR-8096 > URL: https://issues.apache.org/jira/browse/SOLR-8096 > Project: Solr > Issue Type: Bug >Affects Versions: 5.0, 5.1, 5.2, 5.3, Trunk >Reporter: Yonik Seeley >Priority: Critical > > Use of the highly optimized faceting that Solr had for multi-valued fields > over relatively static indexes was removed as part of LUCENE-5666, causing > severe performance regressions. > Here are some quick benchmarks to gauge the damage, on a 5M document index, > with each field having between 0 and 5 values per document. *Higher numbers > represent worse 5x performance*. > Solr 5.4_dev faceting time as a percent of Solr 4.10.3 faceting time > ||...|| Percent of index being faceted > ||num_unique_values|| 10% || 50% || 90% || > |10 | 351.17% | 1587.08% | 3057.28% | > |100 | 158.10% | 203.61% | 1421.93% | > |1000 | 143.78% | 168.01% | 1325.87% | > |1| 137.98% | 175.31% | 1233.97% | > |10 | 142.98% | 159.42% | 1252.45% | > |100 | 255.15% | 165.17% | 1236.75% | > For example, a field with 1000 unique values in the whole index, faceting > with 5x took 143% of the 4x time, when ~10% of the docs in the index were > faceted. > One user who brought the performance problem to our attention: > http://markmail.org/message/ekmqh4ocbkwxv3we > "faceting is unusable slow since upgrade to 5.3.0" (from 4.10.3) > The disabling of the UnInvertedField algorithm was previously discovered in > SOLR-7190, but we didn't know just how bad the problem was at that time. > edit: removed "secret" adverb by request -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2394) Factories for cache creation
[ https://issues.apache.org/jira/browse/LUCENE-2394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14716453#comment-14716453 ] Jamie Johnson commented on LUCENE-2394: --- Having the ability to do this would solve an issue stopping me from moving to Solr 5, are there any plans to tackle this? > Factories for cache creation > > > Key: LUCENE-2394 > URL: https://issues.apache.org/jira/browse/LUCENE-2394 > Project: Lucene - Core > Issue Type: Improvement > Components: core/search >Reporter: Oswaldo Dantas > Fix For: 4.9, Trunk > > Attachments: ASF.LICENSE.NOT.GRANTED--factoriesPatch.patch > > > Hello all, > I've seen the LUCENE-831 (Complete overhaul of FieldCache API/Implementation) > targeted for version 3.1 and I think that maybe, before this overhaul, it > would be good to have a more cirurgical change, that would need smaller > effort in new unit tests, without behavior changes and almost no performance > impact. > One way to achieve that is inserting strategically positioned calls to a > factory structure that would allow every already developed code to continue > working without changes, at the same time giving the opportunity to put > alternative factories to work. > Focusing on the cache idea (not specifically the FieldCache, that has it's > own specific responsabilities, but in the key/value structure that will > ultimately hold the cached objects) i've done the small change contained in > the patch I'm attaching to this. > It has default implementations that encapsulate what was being originally > used in FieldCache, so all current test cases passes, and creates the > possibility to create a EHCacheFactory or InfinispanCacheFactory, or even > MyOwnCachingStructureFactory. > With this, it would be easy to take advantage of the features provided by > this kind of project in a uniform way and rapidly allowing new possibilities > in scalability and tuning. > The code in the patch is small (16kb file is small if compared to the > hundreds of kbs in other patchs) and even though it doesn't have javadoc > right now (sorry) I hope it can be easly understood. So, if Lucene > maintainers see that this contribution could be used (in a 2.9.n+1 and > 3.0.n+1 and maybe influencing future versions) we could put some more effort > in it, documenting, adding necessary unit tests and maybe contributing other > factory implementations. > What do you think? -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-7975) Support payloads on primitive types
Jamie Johnson created SOLR-7975: --- Summary: Support payloads on primitive types Key: SOLR-7975 URL: https://issues.apache.org/jira/browse/SOLR-7975 Project: Solr Issue Type: New Feature Components: clients - java, Server Reporter: Jamie Johnson Currently payloads are supported through the use of an analysis chain, this limits the ability to provide payloads on primitive fields like Trie, Bool, etc without copying these classes and adding the ability in custom code. It would be great if payloads could be added to these field types in a pluggable way similar to what is supported for non primitive types, perhaps through extending the base primitive implementations. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-7190) Remove unused UninvertedField
[ https://issues.apache.org/jira/browse/SOLR-7190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14711312#comment-14711312 ] Jamie Johnson commented on SOLR-7190: - While I understand that DocValuesFacets is more performant at query time I am wondering if there is an option to support both methods given that the current implementation pushes the caches deep down into lucene without any real option for extension. The UnInvertedField allowed me to defer the creation of this cache until query time and then ultimately control what exactly went into the cache by specifying a custom cache implementation I was able to create a separate cache entry for different access levels. > Remove unused UninvertedField > - > > Key: SOLR-7190 > URL: https://issues.apache.org/jira/browse/SOLR-7190 > Project: Solr > Issue Type: Task >Reporter: Shalin Shekhar Mangar >Priority: Minor > Fix For: 5.2, Trunk > > > I was surprised to find that UninvertedField is no longer used in Solr. The > only references to UninvertedField is from the fieldValueCache inside > SolrIndexSearcher and that itself is not used anywhere in SolrIndexSearcher > except for initialization and regeneration. I can't trace when Solr stopped > using this class but in any case, we should remove it. > In a related note, Lucene's DocTermOrds has a copy of the class level > javadocs of UninvertedField (which extends DocTermOrds). This was done in in > LUCENE-5666. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6706) Support Payload scoring for all SpanQueries
[ https://issues.apache.org/jira/browse/LUCENE-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14652052#comment-14652052 ] Jamie Johnson commented on LUCENE-6706: --- Thanks Alan, I really appreciate the quick turn around on this. > Support Payload scoring for all SpanQueries > --- > > Key: LUCENE-6706 > URL: https://issues.apache.org/jira/browse/LUCENE-6706 > Project: Lucene - Core > Issue Type: New Feature > Components: core/query/scoring >Affects Versions: 5.2.1 >Reporter: Jamie Johnson >Assignee: Alan Woodward >Priority: Minor > Fix For: 5.3 > > Attachments: LUCENE-6706.patch, PayloadSpanOrQuery.java > > > I need a way to have payloads influence the score of SpanOrQuery's. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-6706) Support Payload scoring for SpanOrQuery
[ https://issues.apache.org/jira/browse/LUCENE-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14647454#comment-14647454 ] Jamie Johnson edited comment on LUCENE-6706 at 7/30/15 10:47 AM: - Sounds reasonable to me. I had considered making larger changes like what you are suggesting but decided to get the functionality working with as minimal changes to the other classes. That said I think your suggestion should be the path forward as it provides support to the span family of queries. was (Author: jej2003): Sounds reasonable to me. I had considered making larger changes like what you are suggesting but decided to get the functionality working with as minimal changes to the other classes. That said I think your suggestion should be the party forward as it provides support to the span family of queries. > Support Payload scoring for SpanOrQuery > --- > > Key: LUCENE-6706 > URL: https://issues.apache.org/jira/browse/LUCENE-6706 > Project: Lucene - Core > Issue Type: New Feature > Components: core/query/scoring >Affects Versions: 5.2.1 >Reporter: Jamie Johnson >Assignee: Alan Woodward >Priority: Minor > Attachments: PayloadSpanOrQuery.java > > > I need a way to have payloads influence the score of SpanOrQuery's. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-6706) Support Payload scoring for SpanOrQuery
[ https://issues.apache.org/jira/browse/LUCENE-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14647454#comment-14647454 ] Jamie Johnson commented on LUCENE-6706: --- Sounds reasonable to me. I had considered making larger changes like what you are suggesting but decided to get the functionality working with as minimal changes to the other classes. That said I think your suggestion should be the party forward as it provides support to the span family of queries. > Support Payload scoring for SpanOrQuery > --- > > Key: LUCENE-6706 > URL: https://issues.apache.org/jira/browse/LUCENE-6706 > Project: Lucene - Core > Issue Type: New Feature > Components: core/query/scoring >Affects Versions: 5.2.1 >Reporter: Jamie Johnson >Assignee: Alan Woodward >Priority: Minor > Attachments: PayloadSpanOrQuery.java > > > I need a way to have payloads influence the score of SpanOrQuery's. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-6706) Support Payload scoring for SpanOrQuery
[ https://issues.apache.org/jira/browse/LUCENE-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jamie Johnson updated LUCENE-6706: -- Attachment: PayloadSpanOrQuery.java > Support Payload scoring for SpanOrQuery > --- > > Key: LUCENE-6706 > URL: https://issues.apache.org/jira/browse/LUCENE-6706 > Project: Lucene - Core > Issue Type: New Feature > Components: core/query/scoring >Affects Versions: 5.2.1 >Reporter: Jamie Johnson >Priority: Minor > Attachments: PayloadSpanOrQuery.java > > > I need a way to have payloads influence the score of SpanOrQuery's. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-6706) Support Payload scoring for SpanOrQuery
Jamie Johnson created LUCENE-6706: - Summary: Support Payload scoring for SpanOrQuery Key: LUCENE-6706 URL: https://issues.apache.org/jira/browse/LUCENE-6706 Project: Lucene - Core Issue Type: New Feature Components: core/query/scoring Affects Versions: 5.2.1 Reporter: Jamie Johnson Priority: Minor I need a way to have payloads influence the score of SpanOrQuery's. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-5511) Provide a more customizable explain
Jamie Johnson created SOLR-5511: --- Summary: Provide a more customizable explain Key: SOLR-5511 URL: https://issues.apache.org/jira/browse/SOLR-5511 Project: Solr Issue Type: New Feature Components: SearchComponents - other Affects Versions: 4.5.1 Reporter: Jamie Johnson It would be great if there was the capability to choose the items we want returned when using explain. For instance there are cases where tf is needed, but fieldnorm, idf, etc are not. I'm not sure if this requires any additional or less processing but could certainly save some on the transfer and make the clients life easier in only getting items they are interested in. -- This message was sent by Atlassian JIRA (v6.1#6144) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3598) Provide option to allow aliased field to be included in query for EDisMax QParser
[ https://issues.apache.org/jira/browse/SOLR-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13412962#comment-13412962 ] Jamie Johnson commented on SOLR-3598: - Just to make sure I understand you're saying create a pseudo field which we use for querying the actual fields? so basically pseudofield=realfield1,realfield2,realfield3 > Provide option to allow aliased field to be included in query for EDisMax > QParser > - > > Key: SOLR-3598 > URL: https://issues.apache.org/jira/browse/SOLR-3598 > Project: Solr > Issue Type: New Feature > Components: query parsers >Affects Versions: 3.6, 4.0-ALPHA >Reporter: Jamie Johnson >Priority: Minor > Attachments: alias.patch > > > I currently have a situation where I'd like the original field included in > the query, for instance I have several fields with differing granularity, > name, firstname and lastname. Some of my sources differentiate between these > so I can fill out firstname and lastname, while others don't and I need to > just place this information in the name field. When querying I'd like to be > able to say name:Jamie and have it translated to name:Jamie first_name:Jamie > last_name:Jamie. In order to do this it creates an alias cycle and the > EDisMax Query parser throws an exception about it. Ideally there would be an > option to include the original field as part of the query to support this use > case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3598) Provide option to allow aliased field to be included in query for EDisMax QParser
[ https://issues.apache.org/jira/browse/SOLR-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jamie Johnson updated SOLR-3598: Attachment: alias.patch simple patch which supports this. > Provide option to allow aliased field to be included in query for EDisMax > QParser > - > > Key: SOLR-3598 > URL: https://issues.apache.org/jira/browse/SOLR-3598 > Project: Solr > Issue Type: New Feature > Components: query parsers >Affects Versions: 3.6, 4.0, 4.0-ALPHA >Reporter: Jamie Johnson >Priority: Minor > Attachments: alias.patch > > > I currently have a situation where I'd like the original field included in > the query, for instance I have several fields with differing granularity, > name, firstname and lastname. Some of my sources differentiate between these > so I can fill out firstname and lastname, while others don't and I need to > just place this information in the name field. When querying I'd like to be > able to say name:Jamie and have it translated to name:Jamie first_name:Jamie > last_name:Jamie. In order to do this it creates an alias cycle and the > EDisMax Query parser throws an exception about it. Ideally there would be an > option to include the original field as part of the query to support this use > case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-3598) Provide option to allow aliased field to be included in query for EDisMax QParser
[ https://issues.apache.org/jira/browse/SOLR-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13407716#comment-13407716 ] Jamie Johnson edited comment on SOLR-3598 at 7/6/12 5:30 AM: - simple patch which supports this. I didn't run any exhaustive tests but the aliasing piece works. was (Author: jej2003): simple patch which supports this. > Provide option to allow aliased field to be included in query for EDisMax > QParser > - > > Key: SOLR-3598 > URL: https://issues.apache.org/jira/browse/SOLR-3598 > Project: Solr > Issue Type: New Feature > Components: query parsers >Affects Versions: 3.6, 4.0, 4.0-ALPHA >Reporter: Jamie Johnson >Priority: Minor > Attachments: alias.patch > > > I currently have a situation where I'd like the original field included in > the query, for instance I have several fields with differing granularity, > name, firstname and lastname. Some of my sources differentiate between these > so I can fill out firstname and lastname, while others don't and I need to > just place this information in the name field. When querying I'd like to be > able to say name:Jamie and have it translated to name:Jamie first_name:Jamie > last_name:Jamie. In order to do this it creates an alias cycle and the > EDisMax Query parser throws an exception about it. Ideally there would be an > option to include the original field as part of the query to support this use > case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-3598) Provide option to allow aliased field to be included in query for EDisMax QParser
Jamie Johnson created SOLR-3598: --- Summary: Provide option to allow aliased field to be included in query for EDisMax QParser Key: SOLR-3598 URL: https://issues.apache.org/jira/browse/SOLR-3598 Project: Solr Issue Type: New Feature Components: query parsers Affects Versions: 3.6, 4.0, 4.0-ALPHA Reporter: Jamie Johnson Priority: Minor I currently have a situation where I'd like the original field included in the query, for instance I have several fields with differing granularity, name, firstname and lastname. Some of my sources differentiate between these so I can fill out firstname and lastname, while others don't and I need to just place this information in the name field. When querying I'd like to be able to say name:Jamie and have it translated to name:Jamie first_name:Jamie last_name:Jamie. In order to do this it creates an alias cycle and the EDisMax Query parser throws an exception about it. Ideally there would be an option to include the original field as part of the query to support this use case. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1397) It should be possible to highlight external text
[
https://issues.apache.org/jira/browse/SOLR-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13274673#comment-13274673
]
Jamie Johnson commented on SOLR-1397:
-
Attached is a first patch at adding the External Highlighter. I have not had a
chance to write tests for this as of yet, but it's just meant to be a starting
point. There were some changes to the DefaultHighlighter, so my changes didn't
apply cleanly out of the box, but hopefully I've caught everything.
To add an external provider just add this to the highlighter.
{code:title=solrconfig.xml}
value
{code}
> It should be possible to highlight external text
>
>
> Key: SOLR-1397
> URL: https://issues.apache.org/jira/browse/SOLR-1397
> Project: Solr
> Issue Type: New Feature
> Components: highlighter
>Reporter: Anders Melchiorsen
> Attachments: DefaultSolrHighlighter.java, ExternalHighlighter.patch,
> SolrExternalFieldProvider.java, SolrHighlighter.java,
> TestExternalFieldProvider.java
>
>
> Many sites don't store text in Lucene/Solr and so need a way to highlight
> text stored in a database (or some equivalent).
> As a workaround, FieldAnalysisRequestHandler can provide offsets from
> external text, but it does not support wildcard queries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-1397) It should be possible to highlight external text
[ https://issues.apache.org/jira/browse/SOLR-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jamie Johnson updated SOLR-1397: Attachment: ExternalHighlighter.patch > It should be possible to highlight external text > > > Key: SOLR-1397 > URL: https://issues.apache.org/jira/browse/SOLR-1397 > Project: Solr > Issue Type: New Feature > Components: highlighter >Reporter: Anders Melchiorsen > Attachments: DefaultSolrHighlighter.java, ExternalHighlighter.patch, > SolrExternalFieldProvider.java, SolrHighlighter.java, > TestExternalFieldProvider.java > > > Many sites don't store text in Lucene/Solr and so need a way to highlight > text stored in a database (or some equivalent). > As a workaround, FieldAnalysisRequestHandler can provide offsets from > external text, but it does not support wildcard queries. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3427) Faceting under some conditions throws NPE
[ https://issues.apache.org/jira/browse/SOLR-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266811#comment-13266811 ] Jamie Johnson commented on SOLR-3427: - Thanks Yonik! I will pull the nightly build tomorrow morning to try this out on our environment, appreciate all the help! > Faceting under some conditions throws NPE > - > > Key: SOLR-3427 > URL: https://issues.apache.org/jira/browse/SOLR-3427 > Project: Solr > Issue Type: Bug > Components: SearchComponents - other >Affects Versions: 4.0 >Reporter: Jamie Johnson > Fix For: 4.0 > > > Under some circumstances faceting in Solr throws the following NPE > May 1, 2012 8:48:52 PM org.apache.solr.common.SolrException log > SEVERE: java.lang.NullPointerException >at org.apache.lucene.index.DocTermOrds.lookupTerm(DocTermOrds.java:807) >at > org.apache.solr.request.UnInvertedField.getTermValue(UnInvertedField.java:636) >at > org.apache.solr.request.UnInvertedField.getCounts(UnInvertedField.java:411) >at > org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:300) >at > org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:396) >at > org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:205) >at > org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:81) >at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:204) >at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) >at org.apache.solr.core.SolrCore.execute(SolrCore.java:1550) >at > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:442) >at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:263) >at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337) >at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484) >at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) >at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) >at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233) >at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065) >at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413) >at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) >at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999) >at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) >at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250) >at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) >at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) >at org.eclipse.jetty.server.Server.handle(Server.java:351) >at > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454) >at > org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47) >at > org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900) >at > org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954) >at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:857) >at > org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) >at > org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66) >at > org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254) >at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599) >at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534) >at java.lang.Thread.run(Thread.java:662) > I've noticed this happening after doing deletes. When I've seen this issue > before an optimize has made the issue go away. I believe this may also be > related to the describe here: > http://stackoverflow.com/questions/10124055/solr-faceted-search-throws-nullpointerexception-with-http-500-status > I have been unable to reproduce this in a test but this has come up at least > 3 times in our environments. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://
[jira] [Commented] (SOLR-3427) Faceting under some conditions throws NPE
[ https://issues.apache.org/jira/browse/SOLR-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266597#comment-13266597 ] Jamie Johnson commented on SOLR-3427: - Great to hear Yonik, sorry I couldn't have given you a working example of the issue, but glad you were able to duplicate it. Please let me know if there is anything else I can provide. > Faceting under some conditions throws NPE > - > > Key: SOLR-3427 > URL: https://issues.apache.org/jira/browse/SOLR-3427 > Project: Solr > Issue Type: Bug > Components: SearchComponents - other >Affects Versions: 4.0 >Reporter: Jamie Johnson > > Under some circumstances faceting in Solr throws the following NPE > May 1, 2012 8:48:52 PM org.apache.solr.common.SolrException log > SEVERE: java.lang.NullPointerException >at org.apache.lucene.index.DocTermOrds.lookupTerm(DocTermOrds.java:807) >at > org.apache.solr.request.UnInvertedField.getTermValue(UnInvertedField.java:636) >at > org.apache.solr.request.UnInvertedField.getCounts(UnInvertedField.java:411) >at > org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:300) >at > org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:396) >at > org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:205) >at > org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:81) >at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:204) >at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) >at org.apache.solr.core.SolrCore.execute(SolrCore.java:1550) >at > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:442) >at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:263) >at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337) >at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484) >at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) >at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) >at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233) >at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065) >at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413) >at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) >at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999) >at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) >at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250) >at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) >at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) >at org.eclipse.jetty.server.Server.handle(Server.java:351) >at > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454) >at > org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47) >at > org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900) >at > org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954) >at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:857) >at > org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) >at > org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66) >at > org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254) >at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599) >at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534) >at java.lang.Thread.run(Thread.java:662) > I've noticed this happening after doing deletes. When I've seen this issue > before an optimize has made the issue go away. I believe this may also be > related to the describe here: > http://stackoverflow.com/questions/10124055/solr-faceted-search-throws-nullpointerexception-with-http-500-status > I have been unable to reproduce this in a test but this has come up at least > 3 times in our environments. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact
[jira] [Commented] (SOLR-3427) Faceting under some conditions throws NPE
[ https://issues.apache.org/jira/browse/SOLR-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266271#comment-13266271 ] Jamie Johnson commented on SOLR-3427: - I believe the issue is happening on a multivalued field where the document which contributed that field has been deleted. If I can narrow it down to the field that is causing the issue in my index what information can I provide to you that would give you more insight into what is happening? > Faceting under some conditions throws NPE > - > > Key: SOLR-3427 > URL: https://issues.apache.org/jira/browse/SOLR-3427 > Project: Solr > Issue Type: Bug > Components: SearchComponents - other >Affects Versions: 4.0 >Reporter: Jamie Johnson > > Under some circumstances faceting in Solr throws the following NPE > May 1, 2012 8:48:52 PM org.apache.solr.common.SolrException log > SEVERE: java.lang.NullPointerException >at org.apache.lucene.index.DocTermOrds.lookupTerm(DocTermOrds.java:807) >at > org.apache.solr.request.UnInvertedField.getTermValue(UnInvertedField.java:636) >at > org.apache.solr.request.UnInvertedField.getCounts(UnInvertedField.java:411) >at > org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:300) >at > org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:396) >at > org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:205) >at > org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:81) >at > org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:204) >at > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) >at org.apache.solr.core.SolrCore.execute(SolrCore.java:1550) >at > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:442) >at > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:263) >at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337) >at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484) >at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) >at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) >at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233) >at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065) >at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413) >at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) >at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999) >at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) >at > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250) >at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) >at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) >at org.eclipse.jetty.server.Server.handle(Server.java:351) >at > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454) >at > org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47) >at > org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900) >at > org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954) >at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:857) >at > org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) >at > org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66) >at > org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254) >at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599) >at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534) >at java.lang.Thread.run(Thread.java:662) > I've noticed this happening after doing deletes. When I've seen this issue > before an optimize has made the issue go away. I believe this may also be > related to the describe here: > http://stackoverflow.com/questions/10124055/solr-faceted-search-throws-nullpointerexception-with-http-500-status > I have been unable to reproduce this in a test but this has come up at least > 3 times in our environments
[jira] [Created] (SOLR-3427) Faceting under some conditions throws NPE
Jamie Johnson created SOLR-3427: --- Summary: Faceting under some conditions throws NPE Key: SOLR-3427 URL: https://issues.apache.org/jira/browse/SOLR-3427 Project: Solr Issue Type: Bug Components: SearchComponents - other Affects Versions: 4.0 Reporter: Jamie Johnson Under some circumstances faceting in Solr throws the following NPE May 1, 2012 8:48:52 PM org.apache.solr.common.SolrException log SEVERE: java.lang.NullPointerException at org.apache.lucene.index.DocTermOrds.lookupTerm(DocTermOrds.java:807) at org.apache.solr.request.UnInvertedField.getTermValue(UnInvertedField.java:636) at org.apache.solr.request.UnInvertedField.getCounts(UnInvertedField.java:411) at org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:300) at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:396) at org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:205) at org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:81) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:204) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1550) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:442) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:263) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) at org.eclipse.jetty.server.Server.handle(Server.java:351) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:857) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534) at java.lang.Thread.run(Thread.java:662) I've noticed this happening after doing deletes. When I've seen this issue before an optimize has made the issue go away. I believe this may also be related to the describe here: http://stackoverflow.com/questions/10124055/solr-faceted-search-throws-nullpointerexception-with-http-500-status I have been unable to reproduce this in a test but this has come up at least 3 times in our environments. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3401) Solr Core Admin view is not visible unless multiple cores already exist
[ https://issues.apache.org/jira/browse/SOLR-3401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jamie Johnson updated SOLR-3401: Description: The new web gui does not show the Core Admin view unless there are already multiples cores defined. We should show the view in instances when we're running in single core mode as well. (was: The new web gui does not show the Core Admin view unless there are already multiples cores defined.) > Solr Core Admin view is not visible unless multiple cores already exist > --- > > Key: SOLR-3401 > URL: https://issues.apache.org/jira/browse/SOLR-3401 > Project: Solr > Issue Type: Improvement > Components: web gui >Affects Versions: 4.0 >Reporter: Jamie Johnson >Priority: Minor > > The new web gui does not show the Core Admin view unless there are already > multiples cores defined. We should show the view in instances when we're > running in single core mode as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-3401) Solr Core Admin view is not visible unless multiple cores already exist
Jamie Johnson created SOLR-3401: --- Summary: Solr Core Admin view is not visible unless multiple cores already exist Key: SOLR-3401 URL: https://issues.apache.org/jira/browse/SOLR-3401 Project: Solr Issue Type: Improvement Components: web gui Affects Versions: 4.0 Reporter: Jamie Johnson Priority: Minor The new web gui does not show the Core Admin view unless there are already multiples cores defined. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-788) MoreLikeThis should support distributed search
[ https://issues.apache.org/jira/browse/SOLR-788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258747#comment-13258747 ] Jamie Johnson commented on SOLR-788: I unfortunately have not and don't think I'll have the time to do so in the near future. The patch was updated to trunk not too long ago so may not be too difficult to revive assuming the original patch worked as expected > MoreLikeThis should support distributed search > -- > > Key: SOLR-788 > URL: https://issues.apache.org/jira/browse/SOLR-788 > Project: Solr > Issue Type: Improvement > Components: MoreLikeThis >Reporter: Grant Ingersoll >Assignee: Mark Miller >Priority: Minor > Fix For: 4.0 > > Attachments: AlternateDistributedMLT.patch, MLT.patch, MLT.patch, > MoreLikeThisComponentTest.patch, SolrMoreLikeThisPatch.txt > > > The MoreLikeThis component should support distributed processing. > See SOLR-303. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2341) Shard distribution policy
[ https://issues.apache.org/jira/browse/SOLR-2341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066579#comment-13066579 ] Jamie Johnson commented on SOLR-2341: - Where would this be plugged into Solr? The provided patches don't seem to modify any existing solr files so I'm having difficulty understanding where this fits. > Shard distribution policy > - > > Key: SOLR-2341 > URL: https://issues.apache.org/jira/browse/SOLR-2341 > Project: Solr > Issue Type: New Feature > Components: SolrCloud >Reporter: William Mayor >Priority: Minor > Fix For: 4.0 > > Attachments: SOLR-2341.patch, SOLR-2341.patch > > > A first crack at creating policies to be used for determining to which of a > list of shards a document should go. See discussion on "Distributed Indexing" > on dev-list. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1397) It should be possible to highlight external text
[ https://issues.apache.org/jira/browse/SOLR-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066343#comment-13066343 ] Jamie Johnson commented on SOLR-1397: - David, I looked at SOLR-1954 but after applying the patch to trunk the offsets that are returned span the full length of the field + the highlight tags, any idea why? > It should be possible to highlight external text > > > Key: SOLR-1397 > URL: https://issues.apache.org/jira/browse/SOLR-1397 > Project: Solr > Issue Type: New Feature > Components: highlighter >Reporter: Anders Melchiorsen > Attachments: DefaultSolrHighlighter.java, > SolrExternalFieldProvider.java, SolrHighlighter.java, > TestExternalFieldProvider.java > > > Many sites don't store text in Lucene/Solr and so need a way to highlight > text stored in a database (or some equivalent). > As a workaround, FieldAnalysisRequestHandler can provide offsets from > external text, but it does not support wildcard queries. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Issue Comment Edited] (SOLR-1954) Highlighter component should expose snippet character offsets and the score.
[
https://issues.apache.org/jira/browse/SOLR-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066207#comment-13066207
]
Jamie Johnson edited comment on SOLR-1954 at 7/15/11 9:12 PM:
--
I know this has been awhile, but I had a need for something like this and while
I implemented (and added it to SOLR-1397) I figured I'd try this out instead as
well. After applying the patch I got the following response
Test subject message
0
29
seems that the startPos is always 0 and endPos is the length of the field
including the highlighting start/end tags. Is this expected?
was (Author: jej2003):
I know this has been awhile, but I had a need for something like this and
while I implemented (and added it to SOLR-1397) I figured I'd try this out
instead as well. After applying the patch I got the following response
Test subject message
0
29
seems that the endPos is the length of the field including the highlighting
start/end tags. Is this expected?
> Highlighter component should expose snippet character offsets and the score.
>
>
> Key: SOLR-1954
> URL: https://issues.apache.org/jira/browse/SOLR-1954
> Project: Solr
> Issue Type: New Feature
> Components: highlighter
>Reporter: David Smiley
>Priority: Minor
> Attachments: SOLR-1954_start_and_end_offsets.patch
>
>
> The Highlighter Component does not currently expose the snippet character
> offsets nor the score. There is a TODO in DefaultSolrHighlighter indicating
> the intention to add this eventually. This information is needed when doing
> highlighting on external content. The data is there so its pretty easy to
> output it in some way. The challenge is deciding on the output and its
> ramifications on backwards compatibility. The current highlighter component
> response structure doesn't lend itself to adding any new data, unfortunately.
> I wish the original implementer had some foresight. Unfortunately all the
> highlighting tests assume this structure. Here is a snippet of the current
> response structure in Solr's sample data searching for "sdram" for reference:
> {code:xml}
>
>
>
> CORSAIR ValueSelect 1GB 184-Pin DDR SDRAM
> Unbuffered DDR 400 (PC 3200) System Memory - Retail
>
>
>
> {code}
> Perhaps as a little hack, we introduce a pseudo field called
> text_startCharOffset which is the concatenation of the matching field and
> "_startCharOffset". This would be an array of ints. Likewise, there would
> be another array for endCharOffset and score.
> Thoughts?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1954) Highlighter component should expose snippet character offsets and the score.
[
https://issues.apache.org/jira/browse/SOLR-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066207#comment-13066207
]
Jamie Johnson commented on SOLR-1954:
-
I know this has been awhile, but I had a need for something like this and while
I implemented (and added it to SOLR-1397) I figured I'd try this out instead as
well. After applying the patch I got the following response
Test subject message
0
29
seems that the endPos is the length of the field including the highlighting
start/end tags. Is this expected?
> Highlighter component should expose snippet character offsets and the score.
>
>
> Key: SOLR-1954
> URL: https://issues.apache.org/jira/browse/SOLR-1954
> Project: Solr
> Issue Type: New Feature
> Components: highlighter
>Reporter: David Smiley
>Priority: Minor
> Attachments: SOLR-1954_start_and_end_offsets.patch
>
>
> The Highlighter Component does not currently expose the snippet character
> offsets nor the score. There is a TODO in DefaultSolrHighlighter indicating
> the intention to add this eventually. This information is needed when doing
> highlighting on external content. The data is there so its pretty easy to
> output it in some way. The challenge is deciding on the output and its
> ramifications on backwards compatibility. The current highlighter component
> response structure doesn't lend itself to adding any new data, unfortunately.
> I wish the original implementer had some foresight. Unfortunately all the
> highlighting tests assume this structure. Here is a snippet of the current
> response structure in Solr's sample data searching for "sdram" for reference:
> {code:xml}
>
>
>
> CORSAIR ValueSelect 1GB 184-Pin DDR SDRAM
> Unbuffered DDR 400 (PC 3200) System Memory - Retail
>
>
>
> {code}
> Perhaps as a little hack, we introduce a pseudo field called
> text_startCharOffset which is the concatenation of the matching field and
> "_startCharOffset". This would be an array of ints. Likewise, there would
> be another array for endCharOffset and score.
> Thoughts?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-1397) It should be possible to highlight external text
[ https://issues.apache.org/jira/browse/SOLR-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jamie Johnson updated SOLR-1397: Attachment: TestExternalFieldProvider.java SolrExternalFieldProvider.java SolrHighlighter.java DefaultSolrHighlighter.java Modified classes to support External Fields. The test class provided external field provider is very simple and always returns the same values, this was fine for my test since my test data always had the same value. > It should be possible to highlight external text > > > Key: SOLR-1397 > URL: https://issues.apache.org/jira/browse/SOLR-1397 > Project: Solr > Issue Type: New Feature > Components: highlighter >Reporter: Anders Melchiorsen > Attachments: DefaultSolrHighlighter.java, > SolrExternalFieldProvider.java, SolrHighlighter.java, > TestExternalFieldProvider.java > > > Many sites don't store text in Lucene/Solr and so need a way to highlight > text stored in a database (or some equivalent). > As a workaround, FieldAnalysisRequestHandler can provide offsets from > external text, but it does not support wildcard queries. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1397) It should be possible to highlight external text
[ https://issues.apache.org/jira/browse/SOLR-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13066162#comment-13066162 ] Jamie Johnson commented on SOLR-1397: - I had a need for this as well and have put together an implementation that works for my use case. I've attached my implementation to this JIRA, I didn't know how to create a patch, but if someone has those details I'll do so. > It should be possible to highlight external text > > > Key: SOLR-1397 > URL: https://issues.apache.org/jira/browse/SOLR-1397 > Project: Solr > Issue Type: New Feature > Components: highlighter >Reporter: Anders Melchiorsen > > Many sites don't store text in Lucene/Solr and so need a way to highlight > text stored in a database (or some equivalent). > As a workaround, FieldAnalysisRequestHandler can provide offsets from > external text, but it does not support wildcard queries. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-2643) Allow multiple field aliases in ExtendedDisMaxQParser
Allow multiple field aliases in ExtendedDisMaxQParser - Key: SOLR-2643 URL: https://issues.apache.org/jira/browse/SOLR-2643 Project: Solr Issue Type: Improvement Components: search Affects Versions: 4.0 Reporter: Jamie Johnson The original DisMaxQParser seems to have support for handling multiple aliases so someone could do query rewrite on more than just the default field. If the ExtendedDisMaxQParser supported this and exposed this capability we'd be able to build more powerful rewrite capabilities such that could reduce the size of an index. For instance say we have a scenario where we have 3 fields first_name, last_name and name. In this situation we don't completely control the input, we may have first_name and last_name or just name. In this case given 2 documents as follows: Doc 1 first_name: John last_name: Doe Doc 2 name: Jane Doe if the user did a query on name:Doe we would be able to rewrite the query to return both documents such that the query would be name:Doe OR first_name:Doe OR last_name:Doe -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-2587) SolrCloud should allow for partial results should a shard be unavailable
SolrCloud should allow for partial results should a shard be unavailable Key: SOLR-2587 URL: https://issues.apache.org/jira/browse/SOLR-2587 Project: Solr Issue Type: Improvement Components: SolrCloud Affects Versions: 3.2 Reporter: Jamie Johnson When executing a query against a SolrCloud there are instances where it would be beneficial to allow the results to still be returned should some shards be unreachable. Additionally, the response should somehow indicate to the caller that this is a partial result set. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Issue Comment Edited] (SOLR-1395) Integrate Katta
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13034887#comment-13034887 ] Jamie Johnson edited comment on SOLR-1395 at 5/17/11 9:23 PM: -- I think I have most of this running, but I still have a disconnect. I've done the following: 1. Patched 2. Compiled 3. Run web application with additional request handler added to solrconfig.mxl 4. Started katta 5. Started DeployableSolrKattaServer Now if I execute a query (http://localhost:8983/solr/select/?q=*%3A*&version=2.2&start=0&rows=10&indent=on&distrib=true) I get net.sf.katta.util.KattaException: No shards for indices: [*], which makes perfect sense since I have no indices deployed. As a simple test I deployed an index that comes stock with katta (bin/katta addIndex testIndex src/test/testIndexA 2), and execute my query again and I get no results (which also makes sense since that index does not match my solr config). All of that being said what is the process for publishing a core to katta? Is there a way to use the standard http methods to add to the index (using something like java -jar post.jar *.xml)? If not how is it done? Any insight into this would be greatly appreciated. was (Author: jej2003): Is there any updated documentation for how to do this? I've attempted to run through the patching process but the exact steps are not clear since the versions have changed significantly. > Integrate Katta > --- > > Key: SOLR-1395 > URL: https://issues.apache.org/jira/browse/SOLR-1395 > Project: Solr > Issue Type: New Feature >Affects Versions: 1.4 >Reporter: Jason Rutherglen >Priority: Minor > Fix For: 3.2 > > Attachments: SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, > back-end.log, front-end.log, hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, > katta-solrcores.jpg, katta.node.properties, katta.zk.properties, > log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, > solr-1395-1431-katta0.6.patch, solr-1395-1431-katta0.6.patch, > solr-1395-1431.patch, solr-1395-katta-0.6.2-1.patch, > solr-1395-katta-0.6.2-2.patch, solr-1395-katta-0.6.2-3.patch, > solr-1395-katta-0.6.2.patch, test-katta-core-0.6-dev.jar, > zkclient-0.1-dev.jar, zookeeper-3.2.1.jar > > Original Estimate: 336h > Remaining Estimate: 336h > > We'll integrate Katta into Solr so that: > * Distributed search uses Hadoop RPC > * Shard/SolrCore distribution and management > * Zookeeper based failover > * Indexes may be built using Hadoop -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1395) Integrate Katta
[ https://issues.apache.org/jira/browse/SOLR-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13034887#comment-13034887 ] Jamie Johnson commented on SOLR-1395: - Is there any updated documentation for how to do this? I've attempted to run through the patching process but the exact steps are not clear since the versions have changed significantly. > Integrate Katta > --- > > Key: SOLR-1395 > URL: https://issues.apache.org/jira/browse/SOLR-1395 > Project: Solr > Issue Type: New Feature >Affects Versions: 1.4 >Reporter: Jason Rutherglen >Priority: Minor > Fix For: 3.2 > > Attachments: SOLR-1395.patch, SOLR-1395.patch, SOLR-1395.patch, > back-end.log, front-end.log, hadoop-core-0.19.0.jar, katta-core-0.6-dev.jar, > katta-solrcores.jpg, katta.node.properties, katta.zk.properties, > log4j-1.2.13.jar, solr-1395-1431-3.patch, solr-1395-1431-4.patch, > solr-1395-1431-katta0.6.patch, solr-1395-1431-katta0.6.patch, > solr-1395-1431.patch, solr-1395-katta-0.6.2-1.patch, > solr-1395-katta-0.6.2-2.patch, solr-1395-katta-0.6.2-3.patch, > solr-1395-katta-0.6.2.patch, test-katta-core-0.6-dev.jar, > zkclient-0.1-dev.jar, zookeeper-3.2.1.jar > > Original Estimate: 336h > Remaining Estimate: 336h > > We'll integrate Katta into Solr so that: > * Distributed search uses Hadoop RPC > * Shard/SolrCore distribution and management > * Zookeeper based failover > * Indexes may be built using Hadoop -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
