[JENKINS] Lucene-Solr-Tests-4.x-java7 - Build # 668 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-java7/668/ All tests passed Build Log: [...truncated 19533 lines...] BUILD FAILED /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/build.xml:62: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build.xml:523: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/common-build.xml:1961: Can't get https://issues.apache.org/jira/rest/api/2/project/LUCENE to /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-java7/lucene/build/docs/changes/jiraVersionList.json Total time: 20 minutes 46 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Recording test results Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.8.0-ea-b58) - Build # 2577 - Failure!
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux/2577/ Java: 32bit/jdk1.8.0-ea-b58 -server -XX:+UseSerialGC All tests passed Build Log: [...truncated 17857 lines...] [javadoc] Generating Javadoc [javadoc] Javadoc execution [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.7 [javadoc] Loading source files for package org.apache.lucene... [javadoc] Loading source files for package org.apache.lucene.analysis... [javadoc] Loading source files for package org.apache.lucene.analysis.tokenattributes... [javadoc] Loading source files for package org.apache.lucene.codecs... [javadoc] Loading source files for package org.apache.lucene.codecs.compressing... [javadoc] Loading source files for package org.apache.lucene.codecs.lucene3x... [javadoc] Loading source files for package org.apache.lucene.codecs.lucene40... [javadoc] Loading source files for package org.apache.lucene.codecs.lucene40.values... [javadoc] Loading source files for package org.apache.lucene.codecs.lucene41... [javadoc] Loading source files for package org.apache.lucene.codecs.perfield... [javadoc] Loading source files for package org.apache.lucene.document... [javadoc] Loading source files for package org.apache.lucene.index... [javadoc] Loading source files for package org.apache.lucene.search... [javadoc] Loading source files for package org.apache.lucene.search.payloads... [javadoc] Loading source files for package org.apache.lucene.search.similarities... [javadoc] Loading source files for package org.apache.lucene.search.spans... [javadoc] Loading source files for package org.apache.lucene.store... [javadoc] Loading source files for package org.apache.lucene.util... [javadoc] Loading source files for package org.apache.lucene.util.automaton... [javadoc] Loading source files for package org.apache.lucene.util.fst... [javadoc] Loading source files for package org.apache.lucene.util.mutable... [javadoc] Loading source files for package org.apache.lucene.util.packed... [javadoc] Constructing Javadoc information... [javadoc] Standard Doclet version 1.8.0-ea [javadoc] Building tree for all the packages and classes... [javadoc] Building index for all the packages and classes... [javadoc] Building index for all classes... [javadoc] Generating /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/docs/core/help-doc.html... [javadoc] 1 warning [...truncated 44 lines...] [javadoc] Generating Javadoc [javadoc] Javadoc execution [javadoc] Loading source files for package org.apache.lucene.analysis.ar... [javadoc] warning: [options] bootstrap class path not set in conjunction with -source 1.7 [javadoc] Loading source files for package org.apache.lucene.analysis.bg... [javadoc] Loading source files for package org.apache.lucene.analysis.br... [javadoc] Loading source files for package org.apache.lucene.analysis.ca... [javadoc] Loading source files for package org.apache.lucene.analysis.charfilter... [javadoc] Loading source files for package org.apache.lucene.analysis.cjk... [javadoc] Loading source files for package org.apache.lucene.analysis.cn... [javadoc] Loading source files for package org.apache.lucene.analysis.commongrams... [javadoc] Loading source files for package org.apache.lucene.analysis.compound... [javadoc] Loading source files for package org.apache.lucene.analysis.compound.hyphenation... [javadoc] Loading source files for package org.apache.lucene.analysis.core... [javadoc] Loading source files for package org.apache.lucene.analysis.cz... [javadoc] Loading source files for package org.apache.lucene.analysis.da... [javadoc] Loading source files for package org.apache.lucene.analysis.de... [javadoc] Loading source files for package org.apache.lucene.analysis.el... [javadoc] Loading source files for package org.apache.lucene.analysis.en... [javadoc] Loading source files for package org.apache.lucene.analysis.es... [javadoc] Loading source files for package org.apache.lucene.analysis.eu... [javadoc] Loading source files for package org.apache.lucene.analysis.fa... [javadoc] Loading source files for package org.apache.lucene.analysis.fi... [javadoc] Loading source files for package org.apache.lucene.analysis.fr... [javadoc] Loading source files for package org.apache.lucene.analysis.ga... [javadoc] Loading source files for package org.apache.lucene.analysis.gl... [javadoc] Loading source files for package org.apache.lucene.analysis.hi... [javadoc] Loading source files for package org.apache.lucene.analysis.hu... [javadoc] Loading source files for package org.apache.lucene.analysis.hunspell... [javadoc] Loading source files for package org.apache.lucene.analysis.hy... [javadoc] Loading source files for package org.apache.lucene.analysis.id... [javadoc] Loading source files for package org.apache.lucene.analysis.in... [javadoc] Loading
[jira] [Updated] (LUCENE-4258) Incremental Field Updates through Stacked Segments
[ https://issues.apache.org/jira/browse/LUCENE-4258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sivan Yogev updated LUCENE-4258: Attachment: LUCENE-4258.r1410593.patch New patch implementing some previously missing parts, with preliminary code to enable field replacements. Incremental Field Updates through Stacked Segments -- Key: LUCENE-4258 URL: https://issues.apache.org/jira/browse/LUCENE-4258 Project: Lucene - Core Issue Type: Improvement Components: core/index Reporter: Sivan Yogev Attachments: IncrementalFieldUpdates.odp, LUCENE-4258-API-changes.patch, LUCENE-4258-inner-changes.patch, LUCENE-4258.r1402630.patch, LUCENE-4258.r1410593.patch Original Estimate: 2,520h Remaining Estimate: 2,520h Shai and I would like to start working on the proposal to Incremental Field Updates outlined here (http://markmail.org/message/zhrdxxpfk6qvdaex). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-4.x-Windows (32bit/jdk1.7.0_09) - Build # 1707 - Failure!
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Windows/1707/ Java: 32bit/jdk1.7.0_09 -server -XX:+UseConcMarkSweepGC All tests passed Build Log: [...truncated 23561 lines...] changes-to-html: [mkdir] Created dir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\docs\changes [get] Getting: https://issues.apache.org/jira/rest/api/2/project/SOLR [get] To: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build\docs\changes\jiraVersionList.json [exec] Bareword found where operator expected at (eval 1) line 3, near titleSystem [exec] (Missing operator before System?) [exec] Having no space between pattern and following word is deprecated at (eval 1) line 8. [exec] ERROR eval'ing munged JSON string ||html [exec] head [exec] titleSystem Maintenance/title [exec] /head [exec] body [exec] h1Maintenance in progress/h1 [exec] pThis system is currently down for maintenance. More details may be [exec] available from the a href='http://monitoring.apache.org/status/' [exec] ASF Public Network Status/a page./p [exec] /body [exec] /html [exec] ||: Can't find string terminator ' anywhere before EOF at (eval 1) line 8. [exec] BUILD FAILED C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\build.xml:62: The following error occurred while executing this line: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\solr\build.xml:501: The following error occurred while executing this line: C:\Users\JenkinsSlave\workspace\Lucene-Solr-4.x-Windows\lucene\common-build.xml:1963: exec returned: 255 Total time: 27 minutes 28 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Recording test results Description set: Java: 32bit/jdk1.7.0_09 -server -XX:+UseConcMarkSweepGC Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Linux (32bit/jdk1.7.0_09) - Build # 2587 - Failure!
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Linux/2587/ Java: 32bit/jdk1.7.0_09 -client -XX:+UseG1GC All tests passed Build Log: [...truncated 19693 lines...] changes-to-html: [mkdir] Created dir: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/docs/changes [get] Getting: https://issues.apache.org/jira/rest/api/2/project/LUCENE [get] To: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build/docs/changes/jiraVersionList.json [exec] Bareword found where operator expected at (eval 1) line 3, near titleSystem [exec] (Missing operator before System?) [exec] Having no space between pattern and following word is deprecated at (eval 1) line 8. [exec] ERROR eval'ing munged JSON string ||html [exec] head [exec] titleSystem Maintenance/title [exec] /head [exec] body [exec] h1Maintenance in progress/h1 [exec] pThis system is currently down for maintenance. More details may be [exec] available from the a href='http://monitoring.apache.org/status/' [exec] ASF Public Network Status/a page./p [exec] /body [exec] /html [exec] ||: Can't find string terminator ' anywhere before EOF at (eval 1) line 8. [exec] BUILD FAILED /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:62: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build.xml:524: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1964: exec returned: 255 Total time: 17 minutes 2 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Recording test results Description set: Java: 32bit/jdk1.7.0_09 -client -XX:+UseG1GC Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-Tests-trunk-java7 - Build # 3420 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-trunk-java7/3420/ All tests passed Build Log: [...truncated 19703 lines...] changes-to-html: [mkdir] Created dir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/docs/changes [get] Getting: https://issues.apache.org/jira/rest/api/2/project/LUCENE [get] To: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build/docs/changes/jiraVersionList.json [exec] Bareword found where operator expected at (eval 1) line 3, near titleSystem [exec] (Missing operator before System?) [exec] ERROR eval'ing munged JSON string ||html [exec] head [exec] titleSystem Maintenance/title [exec] /head [exec] body [exec] h1Maintenance in progress/h1 [exec] pThis system is currently down for maintenance. More details may be [exec] available from the a href='http://monitoring.apache.org/status/' [exec] ASF Public Network Status/a page./p [exec] /body [exec] /html [exec] ||: Can't find string terminator ' anywhere before EOF at (eval 1) line 8. [exec] BUILD FAILED /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/build.xml:62: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/build.xml:524: The following error occurred while executing this line: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-trunk-java7/lucene/common-build.xml:1964: exec returned: 255 Total time: 22 minutes 2 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Recording test results Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.7.0_09) - Build # 1712 - Failure!
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Windows/1712/ Java: 32bit/jdk1.7.0_09 -server -XX:+UseSerialGC All tests passed Build Log: [...truncated 19686 lines...] changes-to-html: [mkdir] Created dir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\docs\changes [get] Getting: https://issues.apache.org/jira/rest/api/2/project/LUCENE [get] To: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\docs\changes\jiraVersionList.json [exec] Bareword found where operator expected at (eval 1) line 3, near titleSystem [exec] (Missing operator before System?) [exec] Having no space between pattern and following word is deprecated at (eval 1) line 8. [exec] ERROR eval'ing munged JSON string ||html [exec] head [exec] titleSystem Maintenance/title [exec] /head [exec] body [exec] h1Maintenance in progress/h1 [exec] pThis system is currently down for maintenance. More details may be [exec] available from the a href='http://monitoring.apache.org/status/' [exec] ASF Public Network Status/a page./p [exec] /body [exec] /html [exec] ||: Can't find string terminator ' anywhere before EOF at (eval 1) line 8. [exec] BUILD FAILED C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:62: The following error occurred while executing this line: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build.xml:524: The following error occurred while executing this line: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1964: exec returned: 255 Total time: 22 minutes 28 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Recording test results Description set: Java: 32bit/jdk1.7.0_09 -server -XX:+UseSerialGC Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-4.x-Linux (32bit/jdk1.7.0_09) - Build # 2578 - Still Failing!
Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux/2578/ Java: 32bit/jdk1.7.0_09 -client -XX:+UseConcMarkSweepGC All tests passed Build Log: [...truncated 19504 lines...] changes-to-html: [mkdir] Created dir: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/docs/changes [get] Getting: https://issues.apache.org/jira/rest/api/2/project/LUCENE [get] To: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build/docs/changes/jiraVersionList.json [exec] Bareword found where operator expected at (eval 1) line 3, near titleSystem [exec] (Missing operator before System?) [exec] Having no space between pattern and following word is deprecated at (eval 1) line 8. [exec] ERROR eval'ing munged JSON string ||html [exec] head [exec] titleSystem Maintenance/title [exec] /head [exec] body [exec] h1Maintenance in progress/h1 [exec] pThis system is currently down for maintenance. More details may be [exec] available from the a href='http://monitoring.apache.org/status/' [exec] ASF Public Network Status/a page./p [exec] /body [exec] /html [exec] ||: Can't find string terminator ' anywhere before EOF at (eval 1) line 8. [exec] BUILD FAILED /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:62: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build.xml:523: The following error occurred while executing this line: /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1963: exec returned: 255 Total time: 19 minutes 29 seconds Build step 'Invoke Ant' marked build as failure Archiving artifacts Recording test results Description set: Java: 32bit/jdk1.7.0_09 -client -XX:+UseConcMarkSweepGC Email was triggered for: Failure Sending email for trigger: Failure - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-4091) Suggester Component exception null:String cannot be cast to Float
ALP YILANCIOGLU created SOLR-4091: - Summary: Suggester Component exception null:String cannot be cast to Float Key: SOLR-4091 URL: https://issues.apache.org/jira/browse/SOLR-4091 Project: Solr Issue Type: Bug Components: clients - java Environment: Apache Solr 4.0 - Tomcat 7.0- Java 1.6 Reporter: ALP YILANCIOGLU Priority: Critical I am new to apache solr, i deployed the war ( solr 4.0 ) file to tomcat 7.0 and than i set the paths for the cores. i started the server .. and every thing worked fine after that i am setting the suggester component to one of the cores, by adding the following lines to it. bu unfortunetly i am getting an exception on start up of the server when i look at the admin core page.. which is : null:org.apache.solr.common.SolrException: java.lang.String cannot be cast to java.lang.Float what should i do, (i found some patches but they are for solr 3.x i am using 4.0 ) searchComponent name=suggester class=solr.SpellCheckComponent lst name=spellchecker str name=namesuggester/str str name=classnameorg.apache.solr.spelling.suggest.Suggester/str str name=loopupImplorg.apache.solr.spelling.suggest.tst.TSTLookup/str str name=fieldname/str str name=threshold2/str /lst /searchComponent requestHandler class=org.apache.solr.handler.component.SearchHandler name=/suggester lst name=defaults str name=spellchecktrue/str str name=spellcheck.dictionarysuggester/str str name=spellcheck.count10/str /lst arr name=components strsuggester/str /arr /requestHandler -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-4091) Suggester Component exception null:String cannot be cast to Float
[ https://issues.apache.org/jira/browse/SOLR-4091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ALP YILANCIOGLU resolved SOLR-4091. --- Resolution: Not A Problem By changing to a correct representation threshold to a float instead of Str =) float name=threshold0.005/float Suggester Component exception null:String cannot be cast to Float - Key: SOLR-4091 URL: https://issues.apache.org/jira/browse/SOLR-4091 Project: Solr Issue Type: Bug Components: clients - java Environment: Apache Solr 4.0 - Tomcat 7.0- Java 1.6 Reporter: ALP YILANCIOGLU Priority: Critical I am new to apache solr, i deployed the war ( solr 4.0 ) file to tomcat 7.0 and than i set the paths for the cores. i started the server .. and every thing worked fine after that i am setting the suggester component to one of the cores, by adding the following lines to it. bu unfortunetly i am getting an exception on start up of the server when i look at the admin core page.. which is : null:org.apache.solr.common.SolrException: java.lang.String cannot be cast to java.lang.Float what should i do, (i found some patches but they are for solr 3.x i am using 4.0 ) searchComponent name=suggester class=solr.SpellCheckComponent lst name=spellchecker str name=namesuggester/str str name=classnameorg.apache.solr.spelling.suggest.Suggester/str str name=loopupImplorg.apache.solr.spelling.suggest.tst.TSTLookup/str str name=fieldname/str str name=threshold2/str /lst /searchComponent requestHandler class=org.apache.solr.handler.component.SearchHandler name=/suggester lst name=defaults str name=spellchecktrue/str str name=spellcheck.dictionarysuggester/str str name=spellcheck.count10/str /lst arr name=components strsuggester/str /arr /requestHandler -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4091) Suggester Component exception null:String cannot be cast to Float
[ https://issues.apache.org/jira/browse/SOLR-4091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499803#comment-13499803 ] Erick Erickson commented on SOLR-4091: -- Alp: Thanks for closing this out. In future, it's best to raise this kind of issue on the user's list first until you're _sure_ it's a problem with Solr/Lucene then raise a JIRA... You can subscribe here: http://lucene.apache.org/solr/discussion.html Best Erick Suggester Component exception null:String cannot be cast to Float - Key: SOLR-4091 URL: https://issues.apache.org/jira/browse/SOLR-4091 Project: Solr Issue Type: Bug Components: clients - java Environment: Apache Solr 4.0 - Tomcat 7.0- Java 1.6 Reporter: ALP YILANCIOGLU Priority: Critical I am new to apache solr, i deployed the war ( solr 4.0 ) file to tomcat 7.0 and than i set the paths for the cores. i started the server .. and every thing worked fine after that i am setting the suggester component to one of the cores, by adding the following lines to it. bu unfortunetly i am getting an exception on start up of the server when i look at the admin core page.. which is : null:org.apache.solr.common.SolrException: java.lang.String cannot be cast to java.lang.Float what should i do, (i found some patches but they are for solr 3.x i am using 4.0 ) searchComponent name=suggester class=solr.SpellCheckComponent lst name=spellchecker str name=namesuggester/str str name=classnameorg.apache.solr.spelling.suggest.Suggester/str str name=loopupImplorg.apache.solr.spelling.suggest.tst.TSTLookup/str str name=fieldname/str str name=threshold2/str /lst /searchComponent requestHandler class=org.apache.solr.handler.component.SearchHandler name=/suggester lst name=defaults str name=spellchecktrue/str str name=spellcheck.dictionarysuggester/str str name=spellcheck.count10/str /lst arr name=components strsuggester/str /arr /requestHandler -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-4084) Add factory for FuzzySuggester
[ https://issues.apache.org/jira/browse/SOLR-4084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved SOLR-4084. --- Resolution: Fixed Fix Version/s: 5.0 4.1 Thanks Areek! Add factory for FuzzySuggester -- Key: SOLR-4084 URL: https://issues.apache.org/jira/browse/SOLR-4084 Project: Solr Issue Type: New Feature Components: spellchecker Reporter: Robert Muir Fix For: 4.1, 5.0 Attachments: SOLR-4084.patch, SOLR-4084.patch Should look a lot like the Analyzing one. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-3959) csv output is invalid csv if there is a currency field
[ https://issues.apache.org/jira/browse/SOLR-3959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir resolved SOLR-3959. --- Resolution: Fixed Fix Version/s: 5.0 4.1 csv output is invalid csv if there is a currency field -- Key: SOLR-3959 URL: https://issues.apache.org/jira/browse/SOLR-3959 Project: Solr Issue Type: Bug Affects Versions: 4.0 Reporter: Robert Muir Fix For: 4.1, 5.0 Attachments: SOLR-3959.patch, SOLR-3959.patch, SOLR-3959.patch Like in the example. http://localhost:8983/solr/collection1/select?q=*%3A*fl=price_cwt=csv -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4560) Support Filtering Segments During Merge
[
https://issues.apache.org/jira/browse/LUCENE-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499871#comment-13499871
]
Shai Erera commented on LUCENE-4560:
Hi Tim. While in general I'm not against the idea (and I think that in general
have some more control during the merge stage is needed), may I ask why can't
you e.g. do this code (borrowing from your patch):
{code}
IndexWriter writer = new IndexWriter(newDirectory);
writer.addIndexes(new RemoveFieldReader(oldReader));
{code}
That will accomplish, I believe, exactly what you want, no?
The benefits to your approach is that the filtering is done in-place, i.e. no
need to add to a new directory, then switch old/new dirs. But it also may
inadvertently add bugs, e.g. if someone mistakenly decided to remove a field,
or worse, removes the wrong field ... w/ the addIndexes approach, you can do
the process offline, investigate the result index and once you're contend with
it, make the switch.
I can see the benefits in both approaches, but I think that the addIndexes
approach is safer, as it's not 'online' and does not change the source
directory. I'm not sure how 'online' this process needs to be though. How often
do you remove fields, or change index options? That's a fairly serious decision
IMO, and should be done w/ care and lots of testing. Doing that in-place may be
dangerous.
About the patch, it's very simple and clean, which is a good thing ! I would
make RemoveFieldReader extend FilterAtomicReader, to save you some lines of
code, even though it's just a test class.
If you do (and others agree) want to continue w/ the online filtering approach,
perhaps, instead of introducing a MergedSegmentFilter, we could make
SegmentMerger pluggable, with few extension points that allow you to allocate
your own AtomicReader ... just a thought, I know it's not directly related to
this issue, but if we're going to open segment merging up for some serious
hacking, let's do it w/ all intentions :).
Support Filtering Segments During Merge
---
Key: LUCENE-4560
URL: https://issues.apache.org/jira/browse/LUCENE-4560
Project: Lucene - Core
Issue Type: Improvement
Reporter: Tim Smith
Attachments: LUCENE-4560.patch
Spun off from LUCENE-4557
It is desirable to be able to filter segments during merge.
Most often, full reindex of content is not possible.
Merging segments can sometimes have negative consequences when fields are
have different options (most restrictive option is forced during merge)
Being able to filter segments during merges will allow gradually migrating
indexed data to new index settings, support pruning/enhancing existing data
gradually
Use Cases:
* Migrate IndexOptions for fields (See LUCENE-4557)
* Gradually Remove index fields no longer used
* Migrate indexed sort fields to DocValues
* Support converting data types for indexed data
* and so on
patch will be forthcoming
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4560) Support Filtering Segments During Merge
[ https://issues.apache.org/jira/browse/LUCENE-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499876#comment-13499876 ] Shai Erera commented on LUCENE-4560: Thinking about this some more, I really don't thing it's a 'gradual' thing that you do to the index: * Depending on the state of the index, this migration may not happen at all to some segments, typically very large segments and are not picked for merge anymore. So what will happen is that you'll have code in your app that will never be invoked after some time ... not a good sign to me. * I won't want to have code in my app that lives there forever. Rather, I'd like to make a decision to remove field 'foo', run the process which removes it once, and be done with it, moving the code to some tools area that is never run again. ** With your approach, RemoveFieldReader will not go away, unless you can guarantee it ran on all segments, which is like forcing forceMerge(1) to run (note, it may not do what you want, per MP settings !), which is really like addIndexes ** Worse, today it's RemoveFieldReader, and tomorrow it will turn into RemoveFieldAndMigrateIndexOptionsReader, because as I wrote above, you cannot stop running that code if you cannot ensure that all segments have been migrated. So I'm beginning to think that this process should not be an incremental/gradual/online thing, but rather an addIndexes type of process, that you run once, and know that you're done with it, until the next time where you need to rewrite the index, w/o actually re-indexing the content. BTW, did you take a look at LUCENE-2632? It is about adding a FilteringCodec which filters the data that it writes/reads. Could it help you here? If so, I think that it has better chances to get committed, than the approach in this issue (Codecs are already an extension point...). Support Filtering Segments During Merge --- Key: LUCENE-4560 URL: https://issues.apache.org/jira/browse/LUCENE-4560 Project: Lucene - Core Issue Type: Improvement Reporter: Tim Smith Attachments: LUCENE-4560.patch Spun off from LUCENE-4557 It is desirable to be able to filter segments during merge. Most often, full reindex of content is not possible. Merging segments can sometimes have negative consequences when fields are have different options (most restrictive option is forced during merge) Being able to filter segments during merges will allow gradually migrating indexed data to new index settings, support pruning/enhancing existing data gradually Use Cases: * Migrate IndexOptions for fields (See LUCENE-4557) * Gradually Remove index fields no longer used * Migrate indexed sort fields to DocValues * Support converting data types for indexed data * and so on patch will be forthcoming -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS-MAVEN] Lucene-Solr-Maven-4.x #157: POMs out of sync
On Nov 17, 2012, at 4:16 PM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: Curious that Maven is seeing these failures while the Ant build isn't. I think it does. Or it did. It's been eerily quiet -- I agree -- but I haven't been in sync with the latest changes so I don't know what you guys did to shut jenkins up a bit. You're right. Robert Muir just reminded me on #lucene IRC that he disabled all Solr tests on Jenkins, on November 9th: trunk: r1407438 branch_4x: r1407439 Steve - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-4562) Pair-wise FST key comparator by ords
David Smiley created LUCENE-4562: Summary: Pair-wise FST key comparator by ords Key: LUCENE-4562 URL: https://issues.apache.org/jira/browse/LUCENE-4562 Project: Lucene - Core Issue Type: New Feature Components: core/FSTs Reporter: David Smiley Priority: Minor It would be useful to have an FST utility method to do a compare() operation between a key in one FST and a key in a second FST, by ords instead of the keys. So the input is the ord for FST1 and an ord for FST2 and the output is -1, 0, 1. The result is the same as if you were to do a Util.getByOutput for both ords against their respective FSTs then compare the resulting byte arrays. The point of this is to speedup LUCENE-3729 further, which impact sorting across segments. I would be surprised if it doesn't have applicability to other problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-4092) Test failure in org.apache.solr.core.TestConfig.testDefaults31
Shawn Heisey created SOLR-4092: -- Summary: Test failure in org.apache.solr.core.TestConfig.testDefaults31 Key: SOLR-4092 URL: https://issues.apache.org/jira/browse/SOLR-4092 Project: Solr Issue Type: Bug Affects Versions: 4.1, 5.0 Reporter: Shawn Heisey Fix For: 4.1, 5.0 Checkout of branch_4x on 2012/11/18 at about 19:00 UTC. [junit4:junit4] FAILURE 0.21s | TestConfig.testDefaults31 [junit4:junit4] Throwable #1: java.lang.AssertionError: default ramBufferSizeMB should be 16 [junit4:junit4]at __randomizedtesting.SeedInfo.seed([25295D05EC822830:3C3BA9C90D30C561]:0) [junit4:junit4]at org.junit.Assert.fail(Assert.java:93) [junit4:junit4]at org.junit.Assert.assertTrue(Assert.java:43) [junit4:junit4]at org.apache.solr.core.TestConfig.testDefaults31(TestConfig.java:152) snip -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4092) Test failure in org.apache.solr.core.TestConfig.testDefaults31
[ https://issues.apache.org/jira/browse/SOLR-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499886#comment-13499886 ] Shawn Heisey commented on SOLR-4092: This test loads up a config with luceneMatchVersion set to LUCENE_31, then checks what ramBufferSizeMB is set to. LUCENE_31 should have a value of 16, but it has picked up the new default of 100, and the test fails. Test failure in org.apache.solr.core.TestConfig.testDefaults31 -- Key: SOLR-4092 URL: https://issues.apache.org/jira/browse/SOLR-4092 Project: Solr Issue Type: Bug Affects Versions: 4.1, 5.0 Reporter: Shawn Heisey Fix For: 4.1, 5.0 Checkout of branch_4x on 2012/11/18 at about 19:00 UTC. [junit4:junit4] FAILURE 0.21s | TestConfig.testDefaults31 [junit4:junit4] Throwable #1: java.lang.AssertionError: default ramBufferSizeMB should be 16 [junit4:junit4]at __randomizedtesting.SeedInfo.seed([25295D05EC822830:3C3BA9C90D30C561]:0) [junit4:junit4]at org.junit.Assert.fail(Assert.java:93) [junit4:junit4]at org.junit.Assert.assertTrue(Assert.java:43) [junit4:junit4]at org.apache.solr.core.TestConfig.testDefaults31(TestConfig.java:152) snip -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3729) Allow using FST to hold terms data in DocValues.BYTES_*_SORTED
[ https://issues.apache.org/jira/browse/LUCENE-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499887#comment-13499887 ] David Smiley commented on LUCENE-3729: -- See LUCENE-4562 which should improve sorting performance across segments. I also wonder if the ord's might be accessible to the higher level APIs so that those APIs could work off of ord's (int) instead of terms (byteref). Maybe this is too radical, or I'm out of my league on understanding this stuff and I'm making no sense. Allow using FST to hold terms data in DocValues.BYTES_*_SORTED -- Key: LUCENE-3729 URL: https://issues.apache.org/jira/browse/LUCENE-3729 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Labels: gsoc2012, lucene-gsoc-11 Attachments: LUCENE-3729.patch, LUCENE-3729.patch, LUCENE-3729.patch, LUCENE-3729.patch, LUCENE-3729.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4092) Test failure in org.apache.solr.core.TestConfig.testDefaults31
[ https://issues.apache.org/jira/browse/SOLR-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Heisey updated SOLR-4092: --- Affects Version/s: (was: 5.0) Test failure in org.apache.solr.core.TestConfig.testDefaults31 -- Key: SOLR-4092 URL: https://issues.apache.org/jira/browse/SOLR-4092 Project: Solr Issue Type: Bug Affects Versions: 4.1 Reporter: Shawn Heisey Fix For: 4.1 Checkout of branch_4x on 2012/11/18 at about 19:00 UTC. [junit4:junit4] FAILURE 0.21s | TestConfig.testDefaults31 [junit4:junit4] Throwable #1: java.lang.AssertionError: default ramBufferSizeMB should be 16 [junit4:junit4]at __randomizedtesting.SeedInfo.seed([25295D05EC822830:3C3BA9C90D30C561]:0) [junit4:junit4]at org.junit.Assert.fail(Assert.java:93) [junit4:junit4]at org.junit.Assert.assertTrue(Assert.java:43) [junit4:junit4]at org.apache.solr.core.TestConfig.testDefaults31(TestConfig.java:152) snip -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4092) Test failure in org.apache.solr.core.TestConfig.testDefaults31
[ https://issues.apache.org/jira/browse/SOLR-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Heisey updated SOLR-4092: --- Fix Version/s: (was: 5.0) Test failure in org.apache.solr.core.TestConfig.testDefaults31 -- Key: SOLR-4092 URL: https://issues.apache.org/jira/browse/SOLR-4092 Project: Solr Issue Type: Bug Affects Versions: 4.1 Reporter: Shawn Heisey Fix For: 4.1 Checkout of branch_4x on 2012/11/18 at about 19:00 UTC. [junit4:junit4] FAILURE 0.21s | TestConfig.testDefaults31 [junit4:junit4] Throwable #1: java.lang.AssertionError: default ramBufferSizeMB should be 16 [junit4:junit4]at __randomizedtesting.SeedInfo.seed([25295D05EC822830:3C3BA9C90D30C561]:0) [junit4:junit4]at org.junit.Assert.fail(Assert.java:93) [junit4:junit4]at org.junit.Assert.assertTrue(Assert.java:43) [junit4:junit4]at org.apache.solr.core.TestConfig.testDefaults31(TestConfig.java:152) snip -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4092) Test failure in org.apache.solr.core.TestConfig.testDefaults31
[ https://issues.apache.org/jira/browse/SOLR-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499890#comment-13499890 ] Shawn Heisey commented on SOLR-4092: Originally, the code that set the default ramBufferSizeMB used 32 for LUCENE_36 or later, and 16 for LUCENE_35 and earlier. Now it just sets it to a blanket default of 100. Should the ramBufferSizeMB test be removed from testDefaults31, or should the code set it to 100 for LUCENE_41 or higher, 32 for LUCENE36 and LUCENE_40, and 16 for LUCENE_35 and earlier? Trunk doesn't have testDefaults31 at all, so it passes. Test failure in org.apache.solr.core.TestConfig.testDefaults31 -- Key: SOLR-4092 URL: https://issues.apache.org/jira/browse/SOLR-4092 Project: Solr Issue Type: Bug Affects Versions: 4.1 Reporter: Shawn Heisey Fix For: 4.1 Checkout of branch_4x on 2012/11/18 at about 19:00 UTC. [junit4:junit4] FAILURE 0.21s | TestConfig.testDefaults31 [junit4:junit4] Throwable #1: java.lang.AssertionError: default ramBufferSizeMB should be 16 [junit4:junit4]at __randomizedtesting.SeedInfo.seed([25295D05EC822830:3C3BA9C90D30C561]:0) [junit4:junit4]at org.junit.Assert.fail(Assert.java:93) [junit4:junit4]at org.junit.Assert.assertTrue(Assert.java:43) [junit4:junit4]at org.apache.solr.core.TestConfig.testDefaults31(TestConfig.java:152) snip -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4092) Test failure in org.apache.solr.core.TestConfig.testDefaults31
[ https://issues.apache.org/jira/browse/SOLR-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shawn Heisey updated SOLR-4092: --- Attachment: SOLR-4092-test.patch SOLR-4092-code.patch Patches for both possible solutions to this problem. Pick one. :) Test failure in org.apache.solr.core.TestConfig.testDefaults31 -- Key: SOLR-4092 URL: https://issues.apache.org/jira/browse/SOLR-4092 Project: Solr Issue Type: Bug Affects Versions: 4.1 Reporter: Shawn Heisey Fix For: 4.1 Attachments: SOLR-4092-code.patch, SOLR-4092-test.patch Checkout of branch_4x on 2012/11/18 at about 19:00 UTC. [junit4:junit4] FAILURE 0.21s | TestConfig.testDefaults31 [junit4:junit4] Throwable #1: java.lang.AssertionError: default ramBufferSizeMB should be 16 [junit4:junit4]at __randomizedtesting.SeedInfo.seed([25295D05EC822830:3C3BA9C90D30C561]:0) [junit4:junit4]at org.junit.Assert.fail(Assert.java:93) [junit4:junit4]at org.junit.Assert.assertTrue(Assert.java:43) [junit4:junit4]at org.apache.solr.core.TestConfig.testDefaults31(TestConfig.java:152) snip -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4092) Test failure in org.apache.solr.core.TestConfig.testDefaults31
[ https://issues.apache.org/jira/browse/SOLR-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499906#comment-13499906 ] Shawn Heisey commented on SOLR-4092: In order to be affected by this, the user must be running running Solr 4.1 with no ramBufferSizeMB in their config. With the Solr upgrade from 3.x, Solr's memory footprint will be drastically altered no matter what happens with the default ramBufferSizeMB. In many cases that footprint will be significantly smaller than it was before, so any change from this increase may be in the noise. The user may be surprised by new index segments that are suddenly much larger, but since it comes with better performance, I doubt there would be any major fallout. My bias is to change the test, and leave ramBufferSizeMB at 100 for all luceneMatchVersion values. Or you could appeal to binary perfection and make it 128. :) Test failure in org.apache.solr.core.TestConfig.testDefaults31 -- Key: SOLR-4092 URL: https://issues.apache.org/jira/browse/SOLR-4092 Project: Solr Issue Type: Bug Affects Versions: 4.1 Reporter: Shawn Heisey Fix For: 4.1 Attachments: SOLR-4092-code.patch, SOLR-4092-test.patch Checkout of branch_4x on 2012/11/18 at about 19:00 UTC. [junit4:junit4] FAILURE 0.21s | TestConfig.testDefaults31 [junit4:junit4] Throwable #1: java.lang.AssertionError: default ramBufferSizeMB should be 16 [junit4:junit4]at __randomizedtesting.SeedInfo.seed([25295D05EC822830:3C3BA9C90D30C561]:0) [junit4:junit4]at org.junit.Assert.fail(Assert.java:93) [junit4:junit4]at org.junit.Assert.assertTrue(Assert.java:43) [junit4:junit4]at org.apache.solr.core.TestConfig.testDefaults31(TestConfig.java:152) snip -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4092) Test failure in org.apache.solr.core.TestConfig.testDefaults31
[ https://issues.apache.org/jira/browse/SOLR-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499926#comment-13499926 ] Yonik Seeley commented on SOLR-4092: Thanks Shawn, I hadn't realized I broke 4x (since I didn't realize that solr tests had been disabled for all jenkins instances that report to the mailing lists). bq. My bias is to change the test, and leave ramBufferSizeMB at 100 for all luceneMatchVersion values. Agree, that was my intent. Test failure in org.apache.solr.core.TestConfig.testDefaults31 -- Key: SOLR-4092 URL: https://issues.apache.org/jira/browse/SOLR-4092 Project: Solr Issue Type: Bug Affects Versions: 4.1 Reporter: Shawn Heisey Fix For: 4.1 Attachments: SOLR-4092-code.patch, SOLR-4092-test.patch Checkout of branch_4x on 2012/11/18 at about 19:00 UTC. [junit4:junit4] FAILURE 0.21s | TestConfig.testDefaults31 [junit4:junit4] Throwable #1: java.lang.AssertionError: default ramBufferSizeMB should be 16 [junit4:junit4]at __randomizedtesting.SeedInfo.seed([25295D05EC822830:3C3BA9C90D30C561]:0) [junit4:junit4]at org.junit.Assert.fail(Assert.java:93) [junit4:junit4]at org.junit.Assert.assertTrue(Assert.java:43) [junit4:junit4]at org.apache.solr.core.TestConfig.testDefaults31(TestConfig.java:152) snip -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-4092) Test failure in org.apache.solr.core.TestConfig.testDefaults31
[ https://issues.apache.org/jira/browse/SOLR-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yonik Seeley resolved SOLR-4092. Resolution: Fixed Committed: http://svn.apache.org/viewvc?rev=1411000view=rev Test failure in org.apache.solr.core.TestConfig.testDefaults31 -- Key: SOLR-4092 URL: https://issues.apache.org/jira/browse/SOLR-4092 Project: Solr Issue Type: Bug Affects Versions: 4.1 Reporter: Shawn Heisey Fix For: 4.1 Attachments: SOLR-4092-code.patch, SOLR-4092-test.patch Checkout of branch_4x on 2012/11/18 at about 19:00 UTC. [junit4:junit4] FAILURE 0.21s | TestConfig.testDefaults31 [junit4:junit4] Throwable #1: java.lang.AssertionError: default ramBufferSizeMB should be 16 [junit4:junit4]at __randomizedtesting.SeedInfo.seed([25295D05EC822830:3C3BA9C90D30C561]:0) [junit4:junit4]at org.junit.Assert.fail(Assert.java:93) [junit4:junit4]at org.junit.Assert.assertTrue(Assert.java:43) [junit4:junit4]at org.apache.solr.core.TestConfig.testDefaults31(TestConfig.java:152) snip -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS-MAVEN] Lucene-Solr-Maven-4.x #157: POMs out of sync
On Nov 17, 2012, at 4:16 PM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: Not that I wasn't persistent enough, I just can't spend a week on it. :( I spent a long time getting something decent up. The VM is now a treasured artifact that I have backed up. Even installing java was annoying on FreeBSD. Getting a UI at a decent resolution in virtualbox? I think I spent at least an entire weekend working on it and didn't even get to using the resolution of my monitor until the following week. Lot's of googling and trying things and getting bit and going back and googling and now I try and port, and now I try the package. Bah. I think that weekend was also already after some previous time on failed attempts. Twisty and turvy enough that I didn't even end up recording all my steps. FreeBSD is cmd line only as far as I'm concerned if I ever lose my treasured Gnome2/GuestAdditions/Eclipse/Java image file. - Mark - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4562) Pair-wise FST key comparator by ords
[ https://issues.apache.org/jira/browse/LUCENE-4562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499964#comment-13499964 ] Michael McCandless commented on LUCENE-4562: That's a cool idea! In the patch in LUCENE-3729 I just do the full lookup and then compare ... but making it incremental (this issue) is a great idea. Pair-wise FST key comparator by ords Key: LUCENE-4562 URL: https://issues.apache.org/jira/browse/LUCENE-4562 Project: Lucene - Core Issue Type: New Feature Components: core/FSTs Reporter: David Smiley Priority: Minor It would be useful to have an FST utility method to do a compare() operation between a key in one FST and a key in a second FST, by ords instead of the keys. So the input is the ord for FST1 and an ord for FST2 and the output is -1, 0, 1. The result is the same as if you were to do a Util.getByOutput for both ords against their respective FSTs then compare the resulting byte arrays. The point of this is to speedup LUCENE-3729 further, which impact sorting across segments. I would be surprised if it doesn't have applicability to other problems. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-4093) localParams syntax for standard query parser
Yonik Seeley created SOLR-4093:
--
Summary: localParams syntax for standard query parser
Key: SOLR-4093
URL: https://issues.apache.org/jira/browse/SOLR-4093
Project: Solr
Issue Type: New Feature
Reporter: Yonik Seeley
Instead of
{code}
_query_:{!geodist d=10 p=20.5,30.2}
{code}
one could directly use
{code}
{!geodist d=10 p=20.5,30.2}
{code}
references: http://wiki.apache.org/solr/LocalParams
First raised as LUCENE-4271, but moved to this Solr issue due to concerns about
Solr specific features in Lucene.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3729) Allow using FST to hold terms data in DocValues.BYTES_*_SORTED
[ https://issues.apache.org/jira/browse/LUCENE-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499965#comment-13499965 ] Michael McCandless commented on LUCENE-3729: bq. I also wonder if the ord's might be accessible to the higher level APIs so that those APIs could work off of ord's (int) instead of terms (byteref). The challenge is the ords are not global (ie not a total order): they are only ordered for the terms within one segment, unless your index is single segment, or you're working with a top-level reader. Allow using FST to hold terms data in DocValues.BYTES_*_SORTED -- Key: LUCENE-3729 URL: https://issues.apache.org/jira/browse/LUCENE-3729 Project: Lucene - Core Issue Type: Improvement Reporter: Michael McCandless Assignee: Michael McCandless Labels: gsoc2012, lucene-gsoc-11 Attachments: LUCENE-3729.patch, LUCENE-3729.patch, LUCENE-3729.patch, LUCENE-3729.patch, LUCENE-3729.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-4093) localParams syntax for standard query parser
[
https://issues.apache.org/jira/browse/SOLR-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yonik Seeley updated SOLR-4093:
---
Attachment: SOLR-4093.patch
Here's a patch that adds a Solr specific parser under o.a.s.parser and
implements the localParams functionality.
I've also used this opportunity to migrate from ParseException to SyntaxError
in the bulk of Solr code - we shouldn't be relying on an implementation detail
of one parser for what should be a generic exception.
localParams syntax for standard query parser
Key: SOLR-4093
URL: https://issues.apache.org/jira/browse/SOLR-4093
Project: Solr
Issue Type: New Feature
Reporter: Yonik Seeley
Attachments: SOLR-4093.patch
Instead of
{code}
_query_:{!geodist d=10 p=20.5,30.2}
{code}
one could directly use
{code}
{!geodist d=10 p=20.5,30.2}
{code}
references: http://wiki.apache.org/solr/LocalParams
First raised as LUCENE-4271, but moved to this Solr issue due to concerns
about Solr specific features in Lucene.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-4556) FuzzyTermsEnum creates tons of objects
[ https://issues.apache.org/jira/browse/LUCENE-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael McCandless updated LUCENE-4556: --- Attachment: LUCENE-4556.patch I'm attaching a possible alternate way to reduce objects ... it's only just a start ... I created a new LightAutomaton class (I'm not wed to that name!) which places a severe append only restriction on how you are allowed to build up the FSA: you must add all transitions for a given state before adding another state's transitions. It operates with only int state, and stores all transitions in a private int[]. This is a big restriction, but I think a number of our FSA ops would work fine with this. I'm pretty sure building the LevA, and doing the UTF32-UTF8 conversion would work fine append-only... In the patch, I added Automaton.toLightAutomaton to convert from heavy to LightAutomaton, and then fixed CompiledAutomaton (and its consumers) to use that. Tests pass. I think it shouldn't be too hard to cut over the Lev building to this too ... but wanted to get feedback first. Simon, it'd be great if you could try this patch on your benchmark since I can't reproduce the too-heavy GC in my benchmark ... I'm particularly curious whether the 50% time spent in GC you see is due to 1) creating too many objects vs 2) holding onto those objects for too long (in CompiledAutomaton, while the query runs...). So this patch would test whether it's case 2). FuzzyTermsEnum creates tons of objects -- Key: LUCENE-4556 URL: https://issues.apache.org/jira/browse/LUCENE-4556 Project: Lucene - Core Issue Type: Improvement Components: core/search, modules/spellchecker Affects Versions: 4.0 Reporter: Simon Willnauer Assignee: Simon Willnauer Priority: Critical Fix For: 4.1, 5.0 Attachments: LUCENE-4556.patch, LUCENE-4556.patch I ran into this problem in production using the DirectSpellchecker. The number of objects created by the spellchecker shoot through the roof very very quickly. We ran about 130 queries and ended up with 2M transitions / states. We spend 50% of the time in GC just because of transitions. Other parts of the system behave just fine here. I talked quickly to robert and gave a POC a shot providing a LevenshteinAutomaton#toRunAutomaton(prefix, n) method to optimize this case and build a array based strucuture converted into UTF-8 directly instead of going through the object based APIs. This involved quite a bit of changes but they are all package private at this point. I have a patch that still has a fair set of nocommits but its shows that its possible and IMO worth the trouble to make this really useable in production. All tests pass with the patch - its a start -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4560) Support Filtering Segments During Merge
[ https://issues.apache.org/jira/browse/LUCENE-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1346#comment-1346 ] Tim Smith commented on LUCENE-4560: --- My base requirement here is that this be an online process. As such, the add indexes approach is really not useful as i see it, especially as it requires 2x disk space, as well as completely new index directories, it does not play well with upgrading a user's existing index. what i see as needed is the ability to gradually migrate indexes such that any individual segment is itself consistent. currently, merging of indexes can result in loss of indexed data or otherwise break consistency, as in LUCENE-4557 it is 100% ok if all segments have not been processed as i can identify each segment's settings at index open/search time, and optionally filter/search/read segments differently. It is true that once you start using this SegmentMergeFilter, you pretty much have to keep using it forever. I don't see this as an issue as when dealing with supporting old indexes, you constantly have to support migration of data that was indexed using old code. For instance, as time goes on, my MergeSegmentFilter will do more, supporting migrating more and more old index formats/config settings to the latest indexing format/settings. At quick glance, FilteringCodec looks like it applies to writing new content, not reading existing indexes? Doesn't seem quite like that would do the trick here. I would need some way to have the index writer wrap the codec for existing segments in order to inject my custom filtering that would apply during merging. That would be logically identical to the patch provided, however would potentially result in a much more complex patch. Support Filtering Segments During Merge --- Key: LUCENE-4560 URL: https://issues.apache.org/jira/browse/LUCENE-4560 Project: Lucene - Core Issue Type: Improvement Reporter: Tim Smith Attachments: LUCENE-4560.patch Spun off from LUCENE-4557 It is desirable to be able to filter segments during merge. Most often, full reindex of content is not possible. Merging segments can sometimes have negative consequences when fields are have different options (most restrictive option is forced during merge) Being able to filter segments during merges will allow gradually migrating indexed data to new index settings, support pruning/enhancing existing data gradually Use Cases: * Migrate IndexOptions for fields (See LUCENE-4557) * Gradually Remove index fields no longer used * Migrate indexed sort fields to DocValues * Support converting data types for indexed data * and so on patch will be forthcoming -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (SOLR-4093) localParams syntax for standard query parser
[
https://issues.apache.org/jira/browse/SOLR-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500016#comment-13500016
]
David Smiley edited comment on SOLR-4093 at 11/19/12 4:52 AM:
--
Nice! It'll be good to see the old \_query\_ hack gone.
Curious; how did you come up with the JavaCC parser? Did you copy and modify
it from Lucene's?
was (Author: dsmiley):
Nice! It'll be good to see the old _query_ hack gone.
Curious; how did you come up with the JavaCC parser? Did you copy and modify
it from Lucene's?
localParams syntax for standard query parser
Key: SOLR-4093
URL: https://issues.apache.org/jira/browse/SOLR-4093
Project: Solr
Issue Type: New Feature
Reporter: Yonik Seeley
Attachments: SOLR-4093.patch
Instead of
{code}
_query_:{!geodist d=10 p=20.5,30.2}
{code}
one could directly use
{code}
{!geodist d=10 p=20.5,30.2}
{code}
references: http://wiki.apache.org/solr/LocalParams
First raised as LUCENE-4271, but moved to this Solr issue due to concerns
about Solr specific features in Lucene.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-4093) localParams syntax for standard query parser
[
https://issues.apache.org/jira/browse/SOLR-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500016#comment-13500016
]
David Smiley commented on SOLR-4093:
Nice! It'll be good to see the old _query_ hack gone.
Curious; how did you come up with the JavaCC parser? Did you copy and modify
it from Lucene's?
localParams syntax for standard query parser
Key: SOLR-4093
URL: https://issues.apache.org/jira/browse/SOLR-4093
Project: Solr
Issue Type: New Feature
Reporter: Yonik Seeley
Attachments: SOLR-4093.patch
Instead of
{code}
_query_:{!geodist d=10 p=20.5,30.2}
{code}
one could directly use
{code}
{!geodist d=10 p=20.5,30.2}
{code}
references: http://wiki.apache.org/solr/LocalParams
First raised as LUCENE-4271, but moved to this Solr issue due to concerns
about Solr specific features in Lucene.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4560) Support Filtering Segments During Merge
[
https://issues.apache.org/jira/browse/LUCENE-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500050#comment-13500050
]
Shai Erera commented on LUCENE-4560:
I don't see why you need to gradually migrate segments, rather than a one-off
thing that you do when you decide to change the format of the index.
Is it really important to do this gradually? Or if there was a tool which
allowed you to do an in-place upgrade of certain segments, that would be
something to consider?
For instance, you can do something similar to:
{code}
Directory dir; // directory with indexed documents
IndexWriter writer; // your instance of IndexWriter
IndexReader r = YourIndexReader.open(dir);
writer.deleteAll();
writer.addIndexes(r);
writer.commit();
{code}
This is all transactional, so until you commit, searches don't see any of this
work.
Note however that while it's seemingly done in-place, you need to copy all the
segments, even if they don't need to be upgraded.
I guess that I just can't think of a good reason to do a gradual upgrade of
segments. Whenever I had to upgrade old indexes, it was done as a one-off
process and that's it. E.g. IndexUpgrader is such a tool -- upgrades the index
in place.
Having said that, if others think that it might be useful to let one extend
e.g. IndexWriter by providing different instances than SegmentReader
(hard-coded in IW), then I prefer that route to the MergedSegmentFilter. Today
it's SegmentMerger, tomorrow it will be something else. If we want to handle
it, let's handle it from the root. SegmentMerger itself really has nothing to
do with filtering readers. That way, you could write something like
IndexUpgrader (or UpgradeMergePolicy) and upgrade the index as a one-off
process, in place, touching only needed segments.
Support Filtering Segments During Merge
---
Key: LUCENE-4560
URL: https://issues.apache.org/jira/browse/LUCENE-4560
Project: Lucene - Core
Issue Type: Improvement
Reporter: Tim Smith
Attachments: LUCENE-4560.patch
Spun off from LUCENE-4557
It is desirable to be able to filter segments during merge.
Most often, full reindex of content is not possible.
Merging segments can sometimes have negative consequences when fields are
have different options (most restrictive option is forced during merge)
Being able to filter segments during merges will allow gradually migrating
indexed data to new index settings, support pruning/enhancing existing data
gradually
Use Cases:
* Migrate IndexOptions for fields (See LUCENE-4557)
* Gradually Remove index fields no longer used
* Migrate indexed sort fields to DocValues
* Support converting data types for indexed data
* and so on
patch will be forthcoming
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4560) Support Filtering Segments During Merge
[ https://issues.apache.org/jira/browse/LUCENE-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500073#comment-13500073 ] Uwe Schindler commented on LUCENE-4560: --- We had something similar in the past (called PayloadProcessor), which was removed completely in 4.0 (without replacement). The reason was, that the stuff can be implemented inside a FilterAtomicReader and used with IW#addIndexes(IndexReader...). I agree with Shai, that this should be enough for most cases, especially as gradually merging segments can corrumpt your index if you have an error. If you really want to merge in-place: Your patch has nice ideas from my perspective, only the wrapping should be done in the MP and not on IndexWriter level (the number of settings in IWConfig is already too big). So the main thing that needs to be done here is: - Move the AtomicReader instances into MergePolicy.OneMerge - As a result, you need to implement a custom wrapper-MergePolicy like UpgradeIndexMergePolicy, that wraps the AtomicReaders when creating the MergePolicy.OneMerge instances. Another possible approach *without modification in Lucene core* is: - open IndexWriter - get NRT Reader and wrap with one or more FilterAtomicReader - addIndexes the filtered segments - delete the old segments manually (e.g. by deleting all documents) - start final maybeMerge() - commit Uwe Support Filtering Segments During Merge --- Key: LUCENE-4560 URL: https://issues.apache.org/jira/browse/LUCENE-4560 Project: Lucene - Core Issue Type: Improvement Reporter: Tim Smith Attachments: LUCENE-4560.patch Spun off from LUCENE-4557 It is desirable to be able to filter segments during merge. Most often, full reindex of content is not possible. Merging segments can sometimes have negative consequences when fields are have different options (most restrictive option is forced during merge) Being able to filter segments during merges will allow gradually migrating indexed data to new index settings, support pruning/enhancing existing data gradually Use Cases: * Migrate IndexOptions for fields (See LUCENE-4557) * Gradually Remove index fields no longer used * Migrate indexed sort fields to DocValues * Support converting data types for indexed data * and so on patch will be forthcoming -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-4560) Support Filtering Segments During Merge
[ https://issues.apache.org/jira/browse/LUCENE-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500073#comment-13500073 ] Uwe Schindler edited comment on LUCENE-4560 at 11/19/12 7:43 AM: - We had something similar in the past (called PayloadProcessor), which was removed completely in 4.0 (without replacement). The reason was, that the stuff can be implemented inside a FilterAtomicReader and used with IW#addIndexes(IndexReader...). I agree with Shai, that this should be enough for most cases, especially as gradually merging segments can corrumpt your index if you have an error. If you really want to merge in-place: Your patch has nice ideas from my perspective, only the wrapping should be done in the MP and not on IndexWriter level (the number of settings in IWConfig is already too big). So the main thing that needs to be done here is: - Move the AtomicReader instances into MergePolicy.OneMerge - As a result, you need to implement a custom wrapper-MergePolicy like UpgradeIndexMergePolicy, that wraps the AtomicReaders when creating the MergePolicy.OneMerge instances. Another possible approach *without modification in Lucene core* is: - open IndexWriter - get NRT Reader and wrap with one or more FilterAtomicReader - delete the old segments manually (e.g. by deleting all documents) - addIndexes the filtered segments - start final maybeMerge() - commit Uwe was (Author: thetaphi): We had something similar in the past (called PayloadProcessor), which was removed completely in 4.0 (without replacement). The reason was, that the stuff can be implemented inside a FilterAtomicReader and used with IW#addIndexes(IndexReader...). I agree with Shai, that this should be enough for most cases, especially as gradually merging segments can corrumpt your index if you have an error. If you really want to merge in-place: Your patch has nice ideas from my perspective, only the wrapping should be done in the MP and not on IndexWriter level (the number of settings in IWConfig is already too big). So the main thing that needs to be done here is: - Move the AtomicReader instances into MergePolicy.OneMerge - As a result, you need to implement a custom wrapper-MergePolicy like UpgradeIndexMergePolicy, that wraps the AtomicReaders when creating the MergePolicy.OneMerge instances. Another possible approach *without modification in Lucene core* is: - open IndexWriter - get NRT Reader and wrap with one or more FilterAtomicReader - addIndexes the filtered segments - delete the old segments manually (e.g. by deleting all documents) - start final maybeMerge() - commit Uwe Support Filtering Segments During Merge --- Key: LUCENE-4560 URL: https://issues.apache.org/jira/browse/LUCENE-4560 Project: Lucene - Core Issue Type: Improvement Reporter: Tim Smith Attachments: LUCENE-4560.patch Spun off from LUCENE-4557 It is desirable to be able to filter segments during merge. Most often, full reindex of content is not possible. Merging segments can sometimes have negative consequences when fields are have different options (most restrictive option is forced during merge) Being able to filter segments during merges will allow gradually migrating indexed data to new index settings, support pruning/enhancing existing data gradually Use Cases: * Migrate IndexOptions for fields (See LUCENE-4557) * Gradually Remove index fields no longer used * Migrate indexed sort fields to DocValues * Support converting data types for indexed data * and so on patch will be forthcoming -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Comment Edited] (LUCENE-4560) Support Filtering Segments During Merge
[ https://issues.apache.org/jira/browse/LUCENE-4560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13500073#comment-13500073 ] Uwe Schindler edited comment on LUCENE-4560 at 11/19/12 7:48 AM: - We had something similar in the past (called PayloadProcessor), which was removed completely in 4.0 (without replacement). The reason was, that the stuff can be implemented inside a FilterAtomicReader and used with IW#addIndexes(IndexReader...). I agree with Shai, that this should be enough for most cases, especially as gradually merging segments can corrumpt your index if you have an error. If you really want to merge in-place: Your patch has nice ideas from my perspective, only the wrapping should be done in the MP and not on IndexWriter level (the number of settings in IWConfig is already too big). So the main thing that needs to be done here is: - Move the AtomicReader instances into MergePolicy.OneMerge - As a result, you need to implement a custom wrapper-MergePolicy like UpgradeIndexMergePolicy, that wraps the AtomicReaders when creating the MergePolicy.OneMerge instances. Another possible approach *without modification in Lucene core* is: - open IndexWriter - get NRT Reader and wrap with one or more FilterAtomicReader, or leave as-is, if no upgrade is needed. - delete the old segments manually (e.g. by deleting all documents) - addIndexes the filtered segments (optionally one-by-one, so it will not merge all atomic readers into one new segment) - commit Uwe was (Author: thetaphi): We had something similar in the past (called PayloadProcessor), which was removed completely in 4.0 (without replacement). The reason was, that the stuff can be implemented inside a FilterAtomicReader and used with IW#addIndexes(IndexReader...). I agree with Shai, that this should be enough for most cases, especially as gradually merging segments can corrumpt your index if you have an error. If you really want to merge in-place: Your patch has nice ideas from my perspective, only the wrapping should be done in the MP and not on IndexWriter level (the number of settings in IWConfig is already too big). So the main thing that needs to be done here is: - Move the AtomicReader instances into MergePolicy.OneMerge - As a result, you need to implement a custom wrapper-MergePolicy like UpgradeIndexMergePolicy, that wraps the AtomicReaders when creating the MergePolicy.OneMerge instances. Another possible approach *without modification in Lucene core* is: - open IndexWriter - get NRT Reader and wrap with one or more FilterAtomicReader - delete the old segments manually (e.g. by deleting all documents) - addIndexes the filtered segments - start final maybeMerge() - commit Uwe Support Filtering Segments During Merge --- Key: LUCENE-4560 URL: https://issues.apache.org/jira/browse/LUCENE-4560 Project: Lucene - Core Issue Type: Improvement Reporter: Tim Smith Attachments: LUCENE-4560.patch Spun off from LUCENE-4557 It is desirable to be able to filter segments during merge. Most often, full reindex of content is not possible. Merging segments can sometimes have negative consequences when fields are have different options (most restrictive option is forced during merge) Being able to filter segments during merges will allow gradually migrating indexed data to new index settings, support pruning/enhancing existing data gradually Use Cases: * Migrate IndexOptions for fields (See LUCENE-4557) * Gradually Remove index fields no longer used * Migrate indexed sort fields to DocValues * Support converting data types for indexed data * and so on patch will be forthcoming -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
