[jira] Commented: (SOLR-586) Maven - Solr Artifact Publishing
[ https://issues.apache.org/jira/browse/SOLR-586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622449#action_12622449 ] Shalin Shekhar Mangar commented on SOLR-586: Committed revision 685781. Thanks Ryan! Maven - Solr Artifact Publishing Key: SOLR-586 URL: https://issues.apache.org/jira/browse/SOLR-586 Project: Solr Issue Type: New Feature Components: clients - java, contrib - DataImportHandler Affects Versions: 1.3 Reporter: Spencer Crissman Assignee: Shalin Shekhar Mangar Priority: Minor Fix For: 1.3 Attachments: SOLR-586-20080811-craigmcc.zip, SOLR-586-nits.patch, SOLR-586.patch, SOLR-586.patch, SOLR-586.patch, solr-common.pom.xml, solr-dih.pom.xml, solr-server.pom.xml, solr2mvn.sh, solrj.pom.xml I know there is an issue open (SOLR-19) for getting a solr build going under Maven. This issue differs from that in that it does not concern the build process of the solr project, but rather simple dependency management for maven projects that depend on the solr artifacts. I've outlined a way to easily incorporate solrj + dependencies into your own maven projects, in hopes that others doing this find it useful. This issue's purpose is twofold: 1) Let others know the process. 2) Open the idea of whether this can be streamlined/incorporated into the standard build in some manner. Depending on Solrj in a Maven Project 1) Build a 1.3 snapshot. 1.1) Check out the code from http://svn.apache.org/repos/asf/lucene/solr/ 1.2) Build using ant dist 2) Install the artifacts into your maven repo, using the included pom files. 2.1) Move to your dist/apache-solr-1.3-dev/dist directory. 2.2) Copy the attached pom files into this directory. 2.3) Install solr-common into your repo. ex) mvn install:install-file -Dfile=apache-solr-common-1.3-dev.jar -DpomFile=solr-common.pom.xml 2.4) Install solrj into your repo. ex) mvn install:install-file -Dfile=apache-solr-solrj-1.3-dev.jar -DpomFile=solrj.pom.xml 3) Use Solrj in your existing Maven projects by including it as a dependency in your own pom.xml dependency groupIdorg.apache.lucene.solr/groupId artifactIdsolrj/artifactId version1.3-SNAPSHOT/version /dependency So given the above process, it seems like it would be relatively simple to standardize this process by: 1) Including the solr-common and solrj pom files w/ the dist. 2) Automating the periodic installation of the artifacts to a central repo, such as the ibiblio repo. If those steps were performed, then creating a (maven) project based on solrj would be super simple: just #3 from above. Since most custom developments are probably for the clients, it seems like simplifying this would be a nice step to take. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-216) Improvements to solr.py
[ https://issues.apache.org/jira/browse/SOLR-216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622470#action_12622470 ] Dariusz Suchojad commented on SOLR-216: --- I just checked solr.py with 1.3 (r685786) and, except for the one mentioned above, all tests pass correctly. However, I would like to add some more tests before merging features from other places, replacing xml.sax with etree/lxml or use some other Python 2.5 niceties. One thing I still don't know what to do about though is batch updating, I can't get it working right now, and I'm still not sure what it is good for, hence it's hard for me to come up with some good test cases here before trying to fix it :-) Improvements to solr.py --- Key: SOLR-216 URL: https://issues.apache.org/jira/browse/SOLR-216 Project: Solr Issue Type: Improvement Components: clients - python Affects Versions: 1.2 Reporter: Jason Cater Assignee: Mike Klaas Priority: Trivial Attachments: solr-solrpy-r5.patch, solr.py, solr.py, solr.py, solr.py, test_all.py I've taken the original solr.py code and extended it to include higher-level functions. * Requires python 2.3+ * Supports SSL (https://) schema * Conforms (mostly) to PEP 8 -- the Python Style Guide * Provides a high-level results object with implicit data type conversion * Supports batching of update commands -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-698) SpellCheckComponent should use the queryString set in the ResponseBuilder
[ https://issues.apache.org/jira/browse/SOLR-698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622480#action_12622480 ] Shalin Shekhar Mangar commented on SOLR-698: ResponseBuilder#setQueryString can be used by other components to change the query itself before it is passed down to SpellCheckComponent. Moreover all other components such as FacetComponent, QueryComponent and DebugComponent use the ResponseBuilder#getQueryString. I guess SpellCheckComponent should also do the same. SpellCheckComponent should use the queryString set in the ResponseBuilder - Key: SOLR-698 URL: https://issues.apache.org/jira/browse/SOLR-698 Project: Solr Issue Type: Improvement Components: spellchecker Reporter: Steven Broadbridge Attachments: SOLR-698.patch SpellCheckComponent should check if a previous component has set the queryString property on ResponseBuilder before using the q parameter. This will allow it to be used with query components besides QueryComponent. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-698) SpellCheckComponent should use the queryString set in the ResponseBuilder
[ https://issues.apache.org/jira/browse/SOLR-698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-698: --- Fix Version/s: 1.3 Assignee: Shalin Shekhar Mangar Priority: Minor (was: Major) Affects Version/s: 1.3 SpellCheckComponent should use the queryString set in the ResponseBuilder - Key: SOLR-698 URL: https://issues.apache.org/jira/browse/SOLR-698 Project: Solr Issue Type: Improvement Components: spellchecker Affects Versions: 1.3 Reporter: Steven Broadbridge Assignee: Shalin Shekhar Mangar Priority: Minor Fix For: 1.3 Attachments: SOLR-698.patch SpellCheckComponent should check if a previous component has set the queryString property on ResponseBuilder before using the q parameter. This will allow it to be used with query components besides QueryComponent. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-698) SpellCheckComponent should use the queryString set in the ResponseBuilder
[ https://issues.apache.org/jira/browse/SOLR-698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar resolved SOLR-698. Resolution: Fixed Committed revision 685830. Thanks Steven! SpellCheckComponent should use the queryString set in the ResponseBuilder - Key: SOLR-698 URL: https://issues.apache.org/jira/browse/SOLR-698 Project: Solr Issue Type: Improvement Components: spellchecker Affects Versions: 1.3 Reporter: Steven Broadbridge Assignee: Shalin Shekhar Mangar Priority: Minor Fix For: 1.3 Attachments: SOLR-698.patch SpellCheckComponent should check if a previous component has set the queryString property on ResponseBuilder before using the q parameter. This will allow it to be used with query components besides QueryComponent. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (SOLR-374) use IndexReader.reopen
Yeah, it may not be so simple as I assumed eh? Really appreciate the tips, I'll spend some real time going over things rather than relying so much on the tests. - Mark On Aug 13, 2008, at 11:31 PM, Yonik Seeley (JIRA) [EMAIL PROTECTED] wrote: [ https://issues.apache.org/jira/browse/SOLR-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622430#action_12622430 ] Yonik Seeley commented on SOLR-374: --- You've involved yourself in one of the more complicated methods in Solr ;-) - Latest patch has a new race condition: _searcher.incref() may be called after a final _searcher.decref() closes the searcher/reader. - we shouldn't need to check if _searcher==null or not... there may be searchers open that have not yet been registered. - if the reader from the *newest* searcher is equal to it's reopen, you return the registered searcher (which may actually be different from the newest searcher) - returning a RefCountedSolrIndexSearcher immediately can expose it before it was supposed to be used (before warming has completed, etc) - returning a RefCountedSolrIndexSearcher is not always the right thing to do - it depends on the parameters to the function. There are really two different optimizations here: 1) call IndexReader.reopen() and share parts of the most recently opened IndexReader 2) if the IndexReader didn't change, avoid going through warming, autowarming, etc and just reuse the same searcher use IndexReader.reopen -- Key: SOLR-374 URL: https://issues.apache.org/jira/browse/SOLR-374 Project: Solr Issue Type: Improvement Reporter: Yonik Seeley Attachments: SOLR-374.patch, SOLR-374.patch Take advantage of IndexReader.reopen(): LUCENE-743 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-606) spellcheck.colate doesn't handle multiple tokens properly
[ https://issues.apache.org/jira/browse/SOLR-606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Oestreicher updated SOLR-606: Attachment: handler.component.SpellCheckComponent-collate-patch.txt I recently ran into this exact issue and I found the problem. The collation is created by replacing the misspelled tokens with the suggestions using a StringBuilder: {noformat} for (IteratorMap.EntryToken, String bestIter = best.entrySet().iterator(); bestIter.hasNext();) { Map.EntryToken, String entry = bestIter.next(); Token tok = entry.getKey(); collation.replace(tok.startOffset(), tok.endOffset(), entry.getValue()); } {noformat} As you can see it's just replacing the relevant tokens in the original query. However, if the length of a suggestion doesn't equal the length of the original token, all offsets used after that replacement are no longer valid thus randomly yielding incorrect results. I fixed that by keeping track of that difference and adding it to the token offsets. For this to work I had to change the HashMap to a LinkedHashMap since this solution depends on the iteration order of the Tokens to correspond to their occurrence in the string. spellcheck.colate doesn't handle multiple tokens properly - Key: SOLR-606 URL: https://issues.apache.org/jira/browse/SOLR-606 Project: Solr Issue Type: Bug Components: spellchecker Affects Versions: 1.3 Environment: tomcat Reporter: Geoffrey Young Assignee: Grant Ingersoll Priority: Minor Attachments: handler.component.SpellCheckComponent-collate-patch.txt, SOLR-606.patch originally posted as part of SOLR-572: https://issues.apache.org/jira/browse/SOLR-572?focusedCommentId=12608487#action_12608487 the new spellcheck.collate feature seems to exhibit some strange behaviors when handed a query with multiple tokens. {noformat} { responseHeader:{ params:{ q:redbull air show}}, spellcheck:{ suggestions:[ redbull,[ suggestion,[redbelly]], show,[ suggestion,[shot]], collation,redbelly airshotw]}} {noformat} in this case, note the fields are incorrectly concatenated (no space between tokens, left over 'w' from input string) {noformat} { responseHeader:{ params:{ q:redbull air show, spellcheck.q:redbull air show}}, spellcheck:{ suggestions:[ redbull air show,[ suggestion,[redbull singers]], collation,redbull singersredbull air show]}} {noformat} this is slightly different - the suggestions are still concatenated without a space, but the collation is way off. --Geoff -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: failed tests because of wrong locale
Hi Stefan, Do you mind opening a jira issue and attaching these patches there? For NumberFormatTransformer, I guess we should add another attribute for locale which should be used for parsing. The reason is that the data you are processing may be coming from a different locale than the system locale. What do you think? On Thu, Aug 14, 2008 at 2:33 AM, Stefan Oestreicher [EMAIL PROTECTED] wrote: Hi, I just ran the unit tests on my system (uses a german locale) and the DateFieldTest and LegacyDateFieldTest failed because the expected decimal format used . as separator but the actual result was formatted using the , separator. This happens because the ISO8601CanonicalDateFormat class which is defined inside the DateField class doesn't specify the US locale when creating the formatter used for milliseconds. I attached a patch which fixes that problem (schema.DateField-locale.patch.txt). TestNumberFormatTransformer in contrib/dataimporthandler also failed because handler.dataimport.NumberFormatTransformer uses a NumberFormatter which relies on the system locale. But I think in this case it's intentional, so I modified the test case to use the grouping separator of the system locale (handler.dataimport.TestNumberFormatTransformer-locale.patch.txt). After applying those changes all tests succeeded. kind regards, Stefan ### Eclipse Workspace Patch 1.0 #P solr Index: contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java === --- contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java (revision 685675) +++ contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java (working copy) @@ -19,6 +19,7 @@ import org.junit.Assert; import org.junit.Test; +import java.text.DecimalFormatSymbols; import java.util.ArrayList; import java.util.List; import java.util.Map; @@ -32,6 +33,9 @@ * @since solr 1.3 */ public class TestNumberFormatTransformer { + private static final char GRP_SEP = new DecimalFormatSymbols() +.getGroupingSeparator(); + @Test @SuppressWarnings(unchecked) public void testTransformRow_SingleNumber() { @@ -40,7 +44,7 @@ NumberFormatTransformer.FORMAT_STYLE, NumberFormatTransformer.NUMBER)); Context c = AbstractDataImportHandlerTest.getContext(null, null, null, 0, l, null); -Map m = AbstractDataImportHandlerTest.createMap(num, 123,567); +Map m = AbstractDataImportHandlerTest.createMap(num, 123+GRP_SEP+567); new NumberFormatTransformer().transformRow(m, c); Assert.assertEquals(new Long(123567), m.get(num)); } @@ -56,8 +60,8 @@ NumberFormatTransformer.FORMAT_STYLE, NumberFormatTransformer.NUMBER)); List inputs = new ArrayList(); -inputs.add(123,567); -inputs.add(245,678); +inputs.add(123+GRP_SEP+567); +inputs.add(245+GRP_SEP+678); Map row = AbstractDataImportHandlerTest.createMap(inputs, inputs); VariableResolverImpl resolver = new VariableResolverImpl(); ### Eclipse Workspace Patch 1.0 #P solr Index: src/java/org/apache/solr/schema/DateField.java === --- src/java/org/apache/solr/schema/DateField.java (revision 685675) +++ src/java/org/apache/solr/schema/DateField.java (working copy) @@ -31,6 +31,7 @@ import java.util.Date; import java.util.TimeZone; import java.util.Locale; +import java.text.DecimalFormatSymbols; import java.text.SimpleDateFormat; import java.text.DateFormat; import java.text.NumberFormat; @@ -245,7 +246,8 @@ protected NumberFormat millisParser = NumberFormat.getIntegerInstance(Locale.US); -protected NumberFormat millisFormat = new DecimalFormat(.###); +protected NumberFormat millisFormat = new DecimalFormat(.###, + new DecimalFormatSymbols(Locale.US)); public ISO8601CanonicalDateFormat() { super(-MM-dd'T'HH:mm:ss, Locale.US); @@ -295,7 +297,8 @@ ISO8601CanonicalDateFormat c = (ISO8601CanonicalDateFormat) super.clone(); c.millisParser = NumberFormat.getIntegerInstance(Locale.US); - c.millisFormat = new DecimalFormat(.###); + c.millisFormat = new DecimalFormat(.###, +new DecimalFormatSymbols(Locale.US)); return c; } } -- Regards, Shalin Shekhar Mangar.
[jira] Created: (SOLR-700) test failures because of wrong/unexpected locale
test failures because of wrong/unexpected locale Key: SOLR-700 URL: https://issues.apache.org/jira/browse/SOLR-700 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Affects Versions: 1.3 Reporter: Stefan Oestreicher Priority: Minor Attachments: handler.dataimport.TestNumberFormatTransformer-locale.patch.txt, schema.DateField-locale.patch.txt The following unit tests fail if they're run with a german locale: org.apache.solr.schema.LegacyDateFieldTest org.apache.solr.schema.DateFieldTest org.apache.solr.handler.dataimport.TestNumberFormatTransformer The DateField tests fail because the expected decimal format uses . as separator but the actual result is formatted using the , separator. This happens because the ISO8601CanonicalDateFormat class which is defined inside the DateField class doesn't specify the US locale when creating the formatter used for milliseconds. Patch: schema.DateField-locale.patch.txt TestNumberFormatTransformer failed because handler.dataimport.NumberFormatTransformer uses a NumberFormatter which relies on the system locale. But I think in this case it's intentional, so I modified the test case to use the grouping separator of the system locale. Patch: handler.dataimport.TestNumberFormatTransformer-locale.patch.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-700) test failures because of wrong/unexpected locale
[ https://issues.apache.org/jira/browse/SOLR-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Oestreicher updated SOLR-700: Attachment: handler.dataimport.TestNumberFormatTransformer-locale.patch.txt test failures because of wrong/unexpected locale Key: SOLR-700 URL: https://issues.apache.org/jira/browse/SOLR-700 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Affects Versions: 1.3 Reporter: Stefan Oestreicher Priority: Minor Attachments: handler.dataimport.TestNumberFormatTransformer-locale.patch.txt, schema.DateField-locale.patch.txt The following unit tests fail if they're run with a german locale: org.apache.solr.schema.LegacyDateFieldTest org.apache.solr.schema.DateFieldTest org.apache.solr.handler.dataimport.TestNumberFormatTransformer The DateField tests fail because the expected decimal format uses . as separator but the actual result is formatted using the , separator. This happens because the ISO8601CanonicalDateFormat class which is defined inside the DateField class doesn't specify the US locale when creating the formatter used for milliseconds. Patch: schema.DateField-locale.patch.txt TestNumberFormatTransformer failed because handler.dataimport.NumberFormatTransformer uses a NumberFormatter which relies on the system locale. But I think in this case it's intentional, so I modified the test case to use the grouping separator of the system locale. Patch: handler.dataimport.TestNumberFormatTransformer-locale.patch.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-700) test failures because of wrong/unexpected locale
[ https://issues.apache.org/jira/browse/SOLR-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Oestreicher updated SOLR-700: Attachment: schema.DateField-locale.patch.txt test failures because of wrong/unexpected locale Key: SOLR-700 URL: https://issues.apache.org/jira/browse/SOLR-700 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Affects Versions: 1.3 Reporter: Stefan Oestreicher Priority: Minor Attachments: handler.dataimport.TestNumberFormatTransformer-locale.patch.txt, schema.DateField-locale.patch.txt The following unit tests fail if they're run with a german locale: org.apache.solr.schema.LegacyDateFieldTest org.apache.solr.schema.DateFieldTest org.apache.solr.handler.dataimport.TestNumberFormatTransformer The DateField tests fail because the expected decimal format uses . as separator but the actual result is formatted using the , separator. This happens because the ISO8601CanonicalDateFormat class which is defined inside the DateField class doesn't specify the US locale when creating the formatter used for milliseconds. Patch: schema.DateField-locale.patch.txt TestNumberFormatTransformer failed because handler.dataimport.NumberFormatTransformer uses a NumberFormatter which relies on the system locale. But I think in this case it's intentional, so I modified the test case to use the grouping separator of the system locale. Patch: handler.dataimport.TestNumberFormatTransformer-locale.patch.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: failed tests because of wrong locale
Shalin Shekhar Mangar schrieb: Hi Stefan, Do you mind opening a jira issue and attaching these patches there? not at all: https://issues.apache.org/jira/browse/SOLR-700 For NumberFormatTransformer, I guess we should add another attribute for locale which should be used for parsing. The reason is that the data you are processing may be coming from a different locale than the system locale. What do you think? I totally agree. On Thu, Aug 14, 2008 at 2:33 AM, Stefan Oestreicher [EMAIL PROTECTED] wrote: Hi, I just ran the unit tests on my system (uses a german locale) and the DateFieldTest and LegacyDateFieldTest failed because the expected decimal format used . as separator but the actual result was formatted using the , separator. This happens because the ISO8601CanonicalDateFormat class which is defined inside the DateField class doesn't specify the US locale when creating the formatter used for milliseconds. I attached a patch which fixes that problem (schema.DateField-locale.patch.txt). TestNumberFormatTransformer in contrib/dataimporthandler also failed because handler.dataimport.NumberFormatTransformer uses a NumberFormatter which relies on the system locale. But I think in this case it's intentional, so I modified the test case to use the grouping separator of the system locale (handler.dataimport.TestNumberFormatTransformer-locale.patch.txt). After applying those changes all tests succeeded. kind regards, Stefan ### Eclipse Workspace Patch 1.0 #P solr Index: contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java === --- contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java (revision 685675) +++ contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java (working copy) @@ -19,6 +19,7 @@ import org.junit.Assert; import org.junit.Test; +import java.text.DecimalFormatSymbols; import java.util.ArrayList; import java.util.List; import java.util.Map; @@ -32,6 +33,9 @@ * @since solr 1.3 */ public class TestNumberFormatTransformer { + private static final char GRP_SEP = new DecimalFormatSymbols() +.getGroupingSeparator(); + @Test @SuppressWarnings(unchecked) public void testTransformRow_SingleNumber() { @@ -40,7 +44,7 @@ NumberFormatTransformer.FORMAT_STYLE, NumberFormatTransformer.NUMBER)); Context c = AbstractDataImportHandlerTest.getContext(null, null, null, 0, l, null); -Map m = AbstractDataImportHandlerTest.createMap(num, 123,567); +Map m = AbstractDataImportHandlerTest.createMap(num, 123+GRP_SEP+567); new NumberFormatTransformer().transformRow(m, c); Assert.assertEquals(new Long(123567), m.get(num)); } @@ -56,8 +60,8 @@ NumberFormatTransformer.FORMAT_STYLE, NumberFormatTransformer.NUMBER)); List inputs = new ArrayList(); -inputs.add(123,567); -inputs.add(245,678); +inputs.add(123+GRP_SEP+567); +inputs.add(245+GRP_SEP+678); Map row = AbstractDataImportHandlerTest.createMap(inputs, inputs); VariableResolverImpl resolver = new VariableResolverImpl(); ### Eclipse Workspace Patch 1.0 #P solr Index: src/java/org/apache/solr/schema/DateField.java === --- src/java/org/apache/solr/schema/DateField.java (revision 685675) +++ src/java/org/apache/solr/schema/DateField.java (working copy) @@ -31,6 +31,7 @@ import java.util.Date; import java.util.TimeZone; import java.util.Locale; +import java.text.DecimalFormatSymbols; import java.text.SimpleDateFormat; import java.text.DateFormat; import java.text.NumberFormat; @@ -245,7 +246,8 @@ protected NumberFormat millisParser = NumberFormat.getIntegerInstance(Locale.US); -protected NumberFormat millisFormat = new DecimalFormat(.###); +protected NumberFormat millisFormat = new DecimalFormat(.###, + new DecimalFormatSymbols(Locale.US)); public ISO8601CanonicalDateFormat() { super(-MM-dd'T'HH:mm:ss, Locale.US); @@ -295,7 +297,8 @@ ISO8601CanonicalDateFormat c = (ISO8601CanonicalDateFormat) super.clone(); c.millisParser = NumberFormat.getIntegerInstance(Locale.US); - c.millisFormat = new DecimalFormat(.###); + c.millisFormat = new DecimalFormat(.###, +new DecimalFormatSymbols(Locale.US)); return c; } }
Re: Solr changes date format?
: If a European locale was used when the seconds portion of the date : was formatted, it would use a comma for the radix point. But nothing in DateField relies on the default locale -- we don't even use DateFormatter's for milliseconds anymore because of the trailing zero problem uh ... hmmm, okay wait a minute, maybe the problem is the *DecimalFormat* ... it looks like it probably pays attention to the default Locale. crap. JuaNiK: do you know what the default Locale of your jvm is? Can you try running the Solr unit tests (specificly DateFieldTest) and (assuming it fails) file a bug with the results? Hoss, the default Locale of my jvm is: user.country : ES user.language : es i just ran DateFieldTest and it fails as Stefan Oestreicher said in a recent message. I was applying exactly the same solution than him but over DateField.java -Hoss
[jira] Updated: (SOLR-700) test failures because of wrong/unexpected locale
[ https://issues.apache.org/jira/browse/SOLR-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Oestreicher updated SOLR-700: Attachment: handler.dataimport.NumberFormatTransformer-locale.patch..txt based on Shalins suggestion to add an attribute for the locale I prepared another patch implementing those changes. test failures because of wrong/unexpected locale Key: SOLR-700 URL: https://issues.apache.org/jira/browse/SOLR-700 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Affects Versions: 1.3 Reporter: Stefan Oestreicher Priority: Minor Attachments: handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.TestNumberFormatTransformer-locale.patch.txt, schema.DateField-locale.patch.txt The following unit tests fail if they're run with a german locale: org.apache.solr.schema.LegacyDateFieldTest org.apache.solr.schema.DateFieldTest org.apache.solr.handler.dataimport.TestNumberFormatTransformer The DateField tests fail because the expected decimal format uses . as separator but the actual result is formatted using the , separator. This happens because the ISO8601CanonicalDateFormat class which is defined inside the DateField class doesn't specify the US locale when creating the formatter used for milliseconds. Patch: schema.DateField-locale.patch.txt TestNumberFormatTransformer failed because handler.dataimport.NumberFormatTransformer uses a NumberFormatter which relies on the system locale. But I think in this case it's intentional, so I modified the test case to use the grouping separator of the system locale. Patch: handler.dataimport.TestNumberFormatTransformer-locale.patch.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-700) test failures because of wrong/unexpected locale
[ https://issues.apache.org/jira/browse/SOLR-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622530#action_12622530 ] Shalin Shekhar Mangar commented on SOLR-700: Thanks Stefan. I was thinking about having a locale attribute on each field element in data-config.xml. We should modify the NumberFormatTransformer#transformRow method to read each field's locale attribute. Every field's attributes are exposed as a map, so we should have: {code} // Each locale must be of format lang:country e.g. en-US String locale = fld.get(locale); // Create a locale object based on language and country and pass it to the process method {code} That way locale will be configurable on a per-field basis and no EntityProcessor would need to be modified. test failures because of wrong/unexpected locale Key: SOLR-700 URL: https://issues.apache.org/jira/browse/SOLR-700 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Affects Versions: 1.3 Reporter: Stefan Oestreicher Priority: Minor Attachments: handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.TestNumberFormatTransformer-locale.patch.txt, schema.DateField-locale.patch.txt The following unit tests fail if they're run with a german locale: org.apache.solr.schema.LegacyDateFieldTest org.apache.solr.schema.DateFieldTest org.apache.solr.handler.dataimport.TestNumberFormatTransformer The DateField tests fail because the expected decimal format uses . as separator but the actual result is formatted using the , separator. This happens because the ISO8601CanonicalDateFormat class which is defined inside the DateField class doesn't specify the US locale when creating the formatter used for milliseconds. Patch: schema.DateField-locale.patch.txt TestNumberFormatTransformer failed because handler.dataimport.NumberFormatTransformer uses a NumberFormatter which relies on the system locale. But I think in this case it's intentional, so I modified the test case to use the grouping separator of the system locale. Patch: handler.dataimport.TestNumberFormatTransformer-locale.patch.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-700) test failures because of wrong/unexpected locale
[ https://issues.apache.org/jira/browse/SOLR-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Oestreicher updated SOLR-700: Attachment: handler.dataimport.NumberFormatTransformer-locale.patch..txt I see, that's certainly more flexible. I modified the patch. test failures because of wrong/unexpected locale Key: SOLR-700 URL: https://issues.apache.org/jira/browse/SOLR-700 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Affects Versions: 1.3 Reporter: Stefan Oestreicher Priority: Minor Attachments: handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.TestNumberFormatTransformer-locale.patch.txt, schema.DateField-locale.patch.txt The following unit tests fail if they're run with a german locale: org.apache.solr.schema.LegacyDateFieldTest org.apache.solr.schema.DateFieldTest org.apache.solr.handler.dataimport.TestNumberFormatTransformer The DateField tests fail because the expected decimal format uses . as separator but the actual result is formatted using the , separator. This happens because the ISO8601CanonicalDateFormat class which is defined inside the DateField class doesn't specify the US locale when creating the formatter used for milliseconds. Patch: schema.DateField-locale.patch.txt TestNumberFormatTransformer failed because handler.dataimport.NumberFormatTransformer uses a NumberFormatter which relies on the system locale. But I think in this case it's intentional, so I modified the test case to use the grouping separator of the system locale. Patch: handler.dataimport.TestNumberFormatTransformer-locale.patch.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-700) test failures because of wrong/unexpected locale
[ https://issues.apache.org/jira/browse/SOLR-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Oestreicher updated SOLR-700: Attachment: handler.dataimport.NumberFormatTransformer-locale.patch..txt minor update using the LOCALE constant instead of literal String. test failures because of wrong/unexpected locale Key: SOLR-700 URL: https://issues.apache.org/jira/browse/SOLR-700 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Affects Versions: 1.3 Reporter: Stefan Oestreicher Priority: Minor Attachments: handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.TestNumberFormatTransformer-locale.patch.txt, schema.DateField-locale.patch.txt The following unit tests fail if they're run with a german locale: org.apache.solr.schema.LegacyDateFieldTest org.apache.solr.schema.DateFieldTest org.apache.solr.handler.dataimport.TestNumberFormatTransformer The DateField tests fail because the expected decimal format uses . as separator but the actual result is formatted using the , separator. This happens because the ISO8601CanonicalDateFormat class which is defined inside the DateField class doesn't specify the US locale when creating the formatter used for milliseconds. Patch: schema.DateField-locale.patch.txt TestNumberFormatTransformer failed because handler.dataimport.NumberFormatTransformer uses a NumberFormatter which relies on the system locale. But I think in this case it's intentional, so I modified the test case to use the grouping separator of the system locale. Patch: handler.dataimport.TestNumberFormatTransformer-locale.patch.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-700) test failures because of wrong/unexpected locale
[ https://issues.apache.org/jira/browse/SOLR-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Oestreicher updated SOLR-700: Attachment: handler.dataimport.NumberFormatTransformer-locale.patch..txt setting the locale to the default locale just to have it changed afterwards if the parameter exists is not very nice I guess, so I changed that as well. Furthermore the user should probably be notified if he supplies an invalid locale, but I'm unsure how to proceed in this case, so I leave this for now as it is. test failures because of wrong/unexpected locale Key: SOLR-700 URL: https://issues.apache.org/jira/browse/SOLR-700 Project: Solr Issue Type: Bug Components: contrib - DataImportHandler Affects Versions: 1.3 Reporter: Stefan Oestreicher Priority: Minor Attachments: handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.TestNumberFormatTransformer-locale.patch.txt, schema.DateField-locale.patch.txt The following unit tests fail if they're run with a german locale: org.apache.solr.schema.LegacyDateFieldTest org.apache.solr.schema.DateFieldTest org.apache.solr.handler.dataimport.TestNumberFormatTransformer The DateField tests fail because the expected decimal format uses . as separator but the actual result is formatted using the , separator. This happens because the ISO8601CanonicalDateFormat class which is defined inside the DateField class doesn't specify the US locale when creating the formatter used for milliseconds. Patch: schema.DateField-locale.patch.txt TestNumberFormatTransformer failed because handler.dataimport.NumberFormatTransformer uses a NumberFormatter which relies on the system locale. But I think in this case it's intentional, so I modified the test case to use the grouping separator of the system locale. Patch: handler.dataimport.TestNumberFormatTransformer-locale.patch.txt -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henri Biestro updated SOLR-647: --- Attachment: solr-647.patch My apologies to all for cluttering the issue. New simplified version based on Yonik's example for trunk 685913 (post solr-695 commit), dont let the patch size fool you: CoreContainer.getCore() CoreContainer.getAdminCore() now return an incref-ed (opened) core; core.close() must be called when these 2 have been used. SolrCore.Reference is gone. SolrCore.open() SolrCore.close() method are kept because we can retrieve cores in 3 close-unprotected ways: 1 - a call to CoreDescriptor.getCore(), descriptors that can be retrieved through CoreContainer.getCoreDescriptors()) 2 - a list of close-unprotected cores through CoreContainer.getCores(). 3 - SolrCore.getCore() - which is deprecated The first 2 can be used in a user-defined filter/servlet after the SolrDispatchFilter falls through the filter-chain, the CoreContainer being set as an attribute of the request (org.apache.solr.CoreContainer). Because of this, we are not always synchronized by the CoreContainer#cores when we incref/decref. We can thus try to open() a core which is closed and cant use a simple refCount.incrementAndGet(). Thus the refCount.get()/refCount.compareAndSet() pattern in both open close. The TestHarness is modified to always create a CoreContainer that contains the unnamed core so testCoreMT uses CoreContainer.getCore(). Also touched some tests that were using SolrCore.getCore() when they can use the TestHarness core. Do SolrCore.close() in a refcounted way --- Key: SOLR-647 URL: https://issues.apache.org/jira/browse/SOLR-647 Project: Solr Issue Type: Bug Affects Versions: 1.3 Reporter: Noble Paul Assignee: Grant Ingersoll Fix For: 1.3 Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch The method _SolrCore.close()_ directly closes the core . It can cause Exceptions for in-flight requests. The _close()_ method should just do a decrement on refcount and the actual close must happen when the last request being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622575#action_12622575 ] Noble Paul commented on SOLR-647: - I fail to see the need for this code in SolrCore#close() {code:java} int count; do { count = refCount.get(); if (count = 0) return; } while (!refCount.compareAndSet(count, count - 1)); {code} If you do refCount.decrementAndGet() it should be fine. similarly in SolrCore$open() also you can do a refCount.incrementAndGet() and will be fine. What you have written is a duplication of code in AtomicInteger Even the RefCounted class does increment and decrement in the same way Do SolrCore.close() in a refcounted way --- Key: SOLR-647 URL: https://issues.apache.org/jira/browse/SOLR-647 Project: Solr Issue Type: Bug Affects Versions: 1.3 Reporter: Noble Paul Assignee: Grant Ingersoll Fix For: 1.3 Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch The method _SolrCore.close()_ directly closes the core . It can cause Exceptions for in-flight requests. The _close()_ method should just do a decrement on refcount and the actual close must happen when the last request being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-697) Lucene TimeLimitCollector
[ https://issues.apache.org/jira/browse/SOLR-697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otis Gospodnetic resolved SOLR-697. --- Resolution: Invalid Lucene TimeLimitCollector - Key: SOLR-697 URL: https://issues.apache.org/jira/browse/SOLR-697 Project: Solr Issue Type: Bug Components: update Affects Versions: 1.3 Reporter: Wayne Graham Fix For: 1.3 When instantiating a SolrCore using the Solr API, a Lucene exception is thrown {{ org.apache.solr.core.SolrResourceLoader createClassLoader INFO: Reusing parent classloader Exception in thread main java.lang.NoClassDefFoundError: org/apache/lucene/search/TimeLimitedCollector$TimeExceededException at org.apache.solr.core.SolrConfig.init(SolrConfig.java:132) at org.apache.solr.core.SolrConfig.init(SolrConfig.java:101) ... }} Updating the lucene-core-2.4-dev.jar that is included in the nightly to a more current snapshot resolves the problem (tested with Lucene r685576). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henri Biestro updated SOLR-647: --- Attachment: solr-647.patch Based on Ryan's comment, simplified the patch to strictly focus on the matter at hand. Do SolrCore.close() in a refcounted way --- Key: SOLR-647 URL: https://issues.apache.org/jira/browse/SOLR-647 Project: Solr Issue Type: Bug Affects Versions: 1.3 Reporter: Noble Paul Assignee: Grant Ingersoll Fix For: 1.3 Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch The method _SolrCore.close()_ directly closes the core . It can cause Exceptions for in-flight requests. The _close()_ method should just do a decrement on refcount and the actual close must happen when the last request being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622575#action_12622575 ] noble.paul edited comment on SOLR-647 at 8/14/08 8:36 AM: -- I fail to see the need for this code in SolrCore#close() {code:java} int count; do { count = refCount.get(); if (count = 0) return; } while (!refCount.compareAndSet(count, count - 1)); {code} If you do refCount.decrementAndGet() it should be fine. similarly in SolrCore$open() also you can do a refCount.incrementAndGet() and will be fine. What you have written is a duplication of code in AtomicInteger Even the RefCounted class does increment and decrement in the same way Yonik's patch does it simply and I do not see anything wrong with that. was (Author: noble.paul): I fail to see the need for this code in SolrCore#close() {code:java} int count; do { count = refCount.get(); if (count = 0) return; } while (!refCount.compareAndSet(count, count - 1)); {code} If you do refCount.decrementAndGet() it should be fine. similarly in SolrCore$open() also you can do a refCount.incrementAndGet() and will be fine. What you have written is a duplication of code in AtomicInteger Even the RefCounted class does increment and decrement in the same way Do SolrCore.close() in a refcounted way --- Key: SOLR-647 URL: https://issues.apache.org/jira/browse/SOLR-647 Project: Solr Issue Type: Bug Affects Versions: 1.3 Reporter: Noble Paul Assignee: Grant Ingersoll Fix For: 1.3 Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch The method _SolrCore.close()_ directly closes the core . It can cause Exceptions for in-flight requests. The _close()_ method should just do a decrement on refcount and the actual close must happen when the last request being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622583#action_12622583 ] Henri Biestro commented on SOLR-647: About the SolrCore.open() close(): If we were to do an incrementAndGet(), we could end up opening a closed core since we can not always be under the CoreContainer#cores synchronized protection. We thus must check the refcount is not =0 first. The close could use a decrementAndGet() but the current code ensures the count will never go 0. Itn both cases, it is the test in between the get() compareAndSet() that makes the whole difference with {in,de]crementAndGet. Do SolrCore.close() in a refcounted way --- Key: SOLR-647 URL: https://issues.apache.org/jira/browse/SOLR-647 Project: Solr Issue Type: Bug Affects Versions: 1.3 Reporter: Noble Paul Assignee: Grant Ingersoll Fix For: 1.3 Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch The method _SolrCore.close()_ directly closes the core . It can cause Exceptions for in-flight requests. The _close()_ method should just do a decrement on refcount and the actual close must happen when the last request being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622583#action_12622583 ] henrib edited comment on SOLR-647 at 8/14/08 8:46 AM: - About the SolrCore.open() close(): If we were to do an incrementAndGet(), we could end up opening a closed core; We thus must check the refcount is not =0 first. The close could use a decrementAndGet() but the current code ensures the count will never go 0 and is symmetrical to open. In both cases, it is the test in between the get() compareAndSet() that makes the whole difference with {in,de]crementAndGet. My understanding of Yonik's version is that, as a premise, opening a core is always performed under the CoreContainer#cores synchronized protection; as I explained in a previous comment, the assumption can not be strictly met. was (Author: henrib): About the SolrCore.open() close(): If we were to do an incrementAndGet(), we could end up opening a closed core since we can not always be under the CoreContainer#cores synchronized protection. We thus must check the refcount is not =0 first. The close could use a decrementAndGet() but the current code ensures the count will never go 0. Itn both cases, it is the test in between the get() compareAndSet() that makes the whole difference with {in,de]crementAndGet. Do SolrCore.close() in a refcounted way --- Key: SOLR-647 URL: https://issues.apache.org/jira/browse/SOLR-647 Project: Solr Issue Type: Bug Affects Versions: 1.3 Reporter: Noble Paul Assignee: Grant Ingersoll Fix For: 1.3 Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch The method _SolrCore.close()_ directly closes the core . It can cause Exceptions for in-flight requests. The _close()_ method should just do a decrement on refcount and the actual close must happen when the last request being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622586#action_12622586 ] Yonik Seeley commented on SOLR-647: --- bq. he close could use a decrementAndGet() but the current code ensures the count will never go 0. I don't think we should cover this error case up since it could lead to premature closure in other cases. I think we should do a simple decrementAndGet() followed by a logged error if the current count is less than zero. Do SolrCore.close() in a refcounted way --- Key: SOLR-647 URL: https://issues.apache.org/jira/browse/SOLR-647 Project: Solr Issue Type: Bug Affects Versions: 1.3 Reporter: Noble Paul Assignee: Grant Ingersoll Fix For: 1.3 Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch The method _SolrCore.close()_ directly closes the core . It can cause Exceptions for in-flight requests. The _close()_ method should just do a decrement on refcount and the actual close must happen when the last request being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-700) test failures because of wrong/unexpected locale
[ https://issues.apache.org/jira/browse/SOLR-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-700: --- Fix Version/s: 1.3 Assignee: Shalin Shekhar Mangar Description: NumberFormatTransformer uses a NumberFormatter which relies on the system locale. This makes it impossible to use NumberFormatTransformer with data whose locale does not match the system locale. TestNumberFormatTransformer fails on some locales for similar reasons because the grouping symbol differs in different locales. This issue adds a locale attribute for NumberFormatTransformer which allows the user to specify the locale which should be used for formatting. The locale must be specified as land-country e.g. en-US {code:xml} field column=myNumber formatStyle=number locale=de-DE / {code} was: The following unit tests fail if they're run with a german locale: org.apache.solr.schema.LegacyDateFieldTest org.apache.solr.schema.DateFieldTest org.apache.solr.handler.dataimport.TestNumberFormatTransformer The DateField tests fail because the expected decimal format uses . as separator but the actual result is formatted using the , separator. This happens because the ISO8601CanonicalDateFormat class which is defined inside the DateField class doesn't specify the US locale when creating the formatter used for milliseconds. Patch: schema.DateField-locale.patch.txt TestNumberFormatTransformer failed because handler.dataimport.NumberFormatTransformer uses a NumberFormatter which relies on the system locale. But I think in this case it's intentional, so I modified the test case to use the grouping separator of the system locale. Patch: handler.dataimport.TestNumberFormatTransformer-locale.patch.txt Issue Type: Improvement (was: Bug) test failures because of wrong/unexpected locale Key: SOLR-700 URL: https://issues.apache.org/jira/browse/SOLR-700 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Affects Versions: 1.3 Reporter: Stefan Oestreicher Assignee: Shalin Shekhar Mangar Priority: Minor Fix For: 1.3 Attachments: handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.TestNumberFormatTransformer-locale.patch.txt, schema.DateField-locale.patch.txt NumberFormatTransformer uses a NumberFormatter which relies on the system locale. This makes it impossible to use NumberFormatTransformer with data whose locale does not match the system locale. TestNumberFormatTransformer fails on some locales for similar reasons because the grouping symbol differs in different locales. This issue adds a locale attribute for NumberFormatTransformer which allows the user to specify the locale which should be used for formatting. The locale must be specified as land-country e.g. en-US {code:xml} field column=myNumber formatStyle=number locale=de-DE / {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622592#action_12622592 ] Henri Biestro commented on SOLR-647: Yonik; About the close(), you're right: the refcount 0 being an error, shouldn't we go all the way and throw a runtime exception ? Are you in agreement about open()? Do SolrCore.close() in a refcounted way --- Key: SOLR-647 URL: https://issues.apache.org/jira/browse/SOLR-647 Project: Solr Issue Type: Bug Affects Versions: 1.3 Reporter: Noble Paul Assignee: Grant Ingersoll Fix For: 1.3 Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch The method _SolrCore.close()_ directly closes the core . It can cause Exceptions for in-flight requests. The _close()_ method should just do a decrement on refcount and the actual close must happen when the last request being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-700) NumberFormatTransformer should have configurable locales
[ https://issues.apache.org/jira/browse/SOLR-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-700: --- Summary: NumberFormatTransformer should have configurable locales (was: test failures because of wrong/unexpected locale) NumberFormatTransformer should have configurable locales Key: SOLR-700 URL: https://issues.apache.org/jira/browse/SOLR-700 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Affects Versions: 1.3 Reporter: Stefan Oestreicher Assignee: Shalin Shekhar Mangar Priority: Minor Fix For: 1.3 Attachments: handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.TestNumberFormatTransformer-locale.patch.txt, schema.DateField-locale.patch.txt NumberFormatTransformer uses a NumberFormatter which relies on the system locale. This makes it impossible to use NumberFormatTransformer with data whose locale does not match the system locale. TestNumberFormatTransformer fails on some locales for similar reasons because the grouping symbol differs in different locales. This issue adds a locale attribute for NumberFormatTransformer which allows the user to specify the locale which should be used for formatting. The locale must be specified as land-country e.g. en-US {code:xml} field column=myNumber formatStyle=number locale=de-DE / {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-700) NumberFormatTransformer should have configurable locales
[ https://issues.apache.org/jira/browse/SOLR-700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-700: --- Attachment: SOLR-700.patch A new patch (SOLR-700.patch) based on Stefan's work with the following changes: # Changed the locale parsing code to use pre-compiled regex. # Throw an exception if an invalid value (which does not match the regex) is specified. Interestingly, the java.util.Locale class's docs do not say anything about invalid language or countries. # Modified TestNumberFormatTransformer to demonstrate the fix and enhancement. I will commit this shortly. NumberFormatTransformer should have configurable locales Key: SOLR-700 URL: https://issues.apache.org/jira/browse/SOLR-700 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Affects Versions: 1.3 Reporter: Stefan Oestreicher Assignee: Shalin Shekhar Mangar Priority: Minor Fix For: 1.3 Attachments: handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.TestNumberFormatTransformer-locale.patch.txt, schema.DateField-locale.patch.txt, SOLR-700.patch NumberFormatTransformer uses a NumberFormatter which relies on the system locale. This makes it impossible to use NumberFormatTransformer with data whose locale does not match the system locale. TestNumberFormatTransformer fails on some locales for similar reasons because the grouping symbol differs in different locales. This issue adds a locale attribute for NumberFormatTransformer which allows the user to specify the locale which should be used for formatting. The locale must be specified as land-country e.g. en-US {code:xml} field column=myNumber formatStyle=number locale=de-DE / {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-700) NumberFormatTransformer should have configurable locales
[ https://issues.apache.org/jira/browse/SOLR-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622603#action_12622603 ] Stefan Oestreicher commented on SOLR-700: - Nice. I also stumbled upon the fact that the java docs don't say anything about invalid locales. I quickly tested that and interestingly the DateFormatSymbols instance returned . as decimal and , as grouping separator. I would've expected the default locale (which is de-AT in my case) to be used but obviously it did not. In any case the getISO3Country method of the Locale class (among others) throws an exception if it's invoked on an invalid locale. Maybe it would be best to check all available locales explicitly instead of relying on the regex, possibly by constructing a static HashMap of them?! NumberFormatTransformer should have configurable locales Key: SOLR-700 URL: https://issues.apache.org/jira/browse/SOLR-700 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Affects Versions: 1.3 Reporter: Stefan Oestreicher Assignee: Shalin Shekhar Mangar Priority: Minor Fix For: 1.3 Attachments: handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.TestNumberFormatTransformer-locale.patch.txt, schema.DateField-locale.patch.txt, SOLR-700.patch NumberFormatTransformer uses a NumberFormatter which relies on the system locale. This makes it impossible to use NumberFormatTransformer with data whose locale does not match the system locale. TestNumberFormatTransformer fails on some locales for similar reasons because the grouping symbol differs in different locales. This issue adds a locale attribute for NumberFormatTransformer which allows the user to specify the locale which should be used for formatting. The locale must be specified as land-country e.g. en-US {code:xml} field column=myNumber formatStyle=number locale=de-DE / {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henri Biestro updated SOLR-647: --- Attachment: solr-647.patch updated to reflect Yonik's last suggestion. fixed test attempting to close closed core. (throwing an exception was useful for quick detection). Do SolrCore.close() in a refcounted way --- Key: SOLR-647 URL: https://issues.apache.org/jira/browse/SOLR-647 Project: Solr Issue Type: Bug Affects Versions: 1.3 Reporter: Noble Paul Assignee: Grant Ingersoll Fix For: 1.3 Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch The method _SolrCore.close()_ directly closes the core . It can cause Exceptions for in-flight requests. The _close()_ method should just do a decrement on refcount and the actual close must happen when the last request being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622613#action_12622613 ] Yonik Seeley commented on SOLR-647: --- bq. Are you in agreement about open()? Yes, if cores obtained through other methods need to be open for some reason, then open() would need to be called. I'm not sure if we have any cases like that. Do SolrCore.close() in a refcounted way --- Key: SOLR-647 URL: https://issues.apache.org/jira/browse/SOLR-647 Project: Solr Issue Type: Bug Affects Versions: 1.3 Reporter: Noble Paul Assignee: Grant Ingersoll Fix For: 1.3 Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch The method _SolrCore.close()_ directly closes the core . It can cause Exceptions for in-flight requests. The _close()_ method should just do a decrement on refcount and the actual close must happen when the last request being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622618#action_12622618 ] Yonik Seeley commented on SOLR-647: --- bq. refcount 0 is an error . but should it throw an exception? I am not too sure. logging a severe error should be fine. because that should not be a reason to crash a server Right... on one hand an exception brings more visibility, but on the other hand it's a recoverable error that's not necessarily tied to the request that would get the exception. bq. even open() does not need a compareAndSet. just do a refCount.incrementAndget(). if count =1 return null .because it is already closed or being closed . And then if another thread calls open() at the same time, count will be 2 and it would incorrectly return a closing core. If open() is needed, Henri has the right impl there. Do SolrCore.close() in a refcounted way --- Key: SOLR-647 URL: https://issues.apache.org/jira/browse/SOLR-647 Project: Solr Issue Type: Bug Affects Versions: 1.3 Reporter: Noble Paul Assignee: Grant Ingersoll Fix For: 1.3 Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch The method _SolrCore.close()_ directly closes the core . It can cause Exceptions for in-flight requests. The _close()_ method should just do a decrement on refcount and the actual close must happen when the last request being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-702) Long lists of synonyms seem not to work
Long lists of synonyms seem not to work --- Key: SOLR-702 URL: https://issues.apache.org/jira/browse/SOLR-702 Project: Solr Issue Type: Bug Components: search Affects Versions: 1.3 Environment: I'm using Java 1.6 running on OS X on a Macbook Pro with a 64 bit processor. Reporter: Matthew Runo Attachments: Synonyms2.txt I've been having issues with synonyms apparently not working. Shorter lists work just fine, while longer lists do not appear to work. I'll attach my schema, config, and synonyms file for testing. As a quick example... this works... zutanoapparel = zutano But this does not... aadias, aadidas, aaidas, adadas, adaddas, adaddis, adadias, adadis, adaidas, adaies, addedas, addedis, addidaas, addidads, addidais, addidas, addidascom, addiddas, addides, addidis, adeadas, adedas, adeddas, adedias, adiada, adiadas, adiadis, adiads, adida, adidaas, adidas1, adidass, adidaz, adidda, adiddas, adiddias, adidias, adidis, adiidas, aditas, adudas, afidas, aididas, wwwadidascom = adidas This works... liumiani, loomiani, lumaini, lumanai, lumani, lumiami, lumian, lumiana, lumianai, lumiari, luminani, lumini, luminiani = lumiani But this does not... clegerie, cleregie, clergerie, clergie, robertclaregie, robert claregie, robertclargeries, robert clargeries, robertclegerie, robert clegerie, robertcleregie, robert cleregie, robertclergeic, robert clergeic, robertclergerie, robertclergi, robert clergi, robertclergie, robert clergie, robertclergoe, robert clergoe, robertclerige, robert clerige, robertclerterie, robert clerterie = Robert Clergerie -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-702) Long lists of synonyms seem not to work
[ https://issues.apache.org/jira/browse/SOLR-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Runo updated SOLR-702: -- Attachment: Synonyms2.txt My synonyms file. Some of them work, some do not. Long lists of synonyms seem not to work --- Key: SOLR-702 URL: https://issues.apache.org/jira/browse/SOLR-702 Project: Solr Issue Type: Bug Components: search Affects Versions: 1.3 Environment: I'm using Java 1.6 running on OS X on a Macbook Pro with a 64 bit processor. Reporter: Matthew Runo Attachments: Synonyms2.txt I've been having issues with synonyms apparently not working. Shorter lists work just fine, while longer lists do not appear to work. I'll attach my schema, config, and synonyms file for testing. As a quick example... this works... zutanoapparel = zutano But this does not... aadias, aadidas, aaidas, adadas, adaddas, adaddis, adadias, adadis, adaidas, adaies, addedas, addedis, addidaas, addidads, addidais, addidas, addidascom, addiddas, addides, addidis, adeadas, adedas, adeddas, adedias, adiada, adiadas, adiadis, adiads, adida, adidaas, adidas1, adidass, adidaz, adidda, adiddas, adiddias, adidias, adidis, adiidas, aditas, adudas, afidas, aididas, wwwadidascom = adidas This works... liumiani, loomiani, lumaini, lumanai, lumani, lumiami, lumian, lumiana, lumianai, lumiari, luminani, lumini, luminiani = lumiani But this does not... clegerie, cleregie, clergerie, clergie, robertclaregie, robert claregie, robertclargeries, robert clargeries, robertclegerie, robert clegerie, robertcleregie, robert cleregie, robertclergeic, robert clergeic, robertclergerie, robertclergi, robert clergi, robertclergie, robert clergie, robertclergoe, robert clergoe, robertclerige, robert clerige, robertclerterie, robert clerterie = Robert Clergerie -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-702) Long lists of synonyms seem not to work
[ https://issues.apache.org/jira/browse/SOLR-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Runo updated SOLR-702: -- Attachment: schema.xml Long lists of synonyms seem not to work --- Key: SOLR-702 URL: https://issues.apache.org/jira/browse/SOLR-702 Project: Solr Issue Type: Bug Components: search Affects Versions: 1.3 Environment: I'm using Java 1.6 running on OS X on a Macbook Pro with a 64 bit processor. Reporter: Matthew Runo Attachments: schema.xml, solrconfig.xml, Synonyms2.txt I've been having issues with synonyms apparently not working. Shorter lists work just fine, while longer lists do not appear to work. I'll attach my schema, config, and synonyms file for testing. As a quick example... this works... zutanoapparel = zutano But this does not... aadias, aadidas, aaidas, adadas, adaddas, adaddis, adadias, adadis, adaidas, adaies, addedas, addedis, addidaas, addidads, addidais, addidas, addidascom, addiddas, addides, addidis, adeadas, adedas, adeddas, adedias, adiada, adiadas, adiadis, adiads, adida, adidaas, adidas1, adidass, adidaz, adidda, adiddas, adiddias, adidias, adidis, adiidas, aditas, adudas, afidas, aididas, wwwadidascom = adidas This works... liumiani, loomiani, lumaini, lumanai, lumani, lumiami, lumian, lumiana, lumianai, lumiari, luminani, lumini, luminiani = lumiani But this does not... clegerie, cleregie, clergerie, clergie, robertclaregie, robert claregie, robertclargeries, robert clargeries, robertclegerie, robert clegerie, robertcleregie, robert cleregie, robertclergeic, robert clergeic, robertclergerie, robertclergi, robert clergi, robertclergie, robert clergie, robertclergoe, robert clergoe, robertclerige, robert clerige, robertclerterie, robert clerterie = Robert Clergerie -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-702) Long lists of synonyms seem not to work
[ https://issues.apache.org/jira/browse/SOLR-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Runo updated SOLR-702: -- Attachment: solrconfig.xml Long lists of synonyms seem not to work --- Key: SOLR-702 URL: https://issues.apache.org/jira/browse/SOLR-702 Project: Solr Issue Type: Bug Components: search Affects Versions: 1.3 Environment: I'm using Java 1.6 running on OS X on a Macbook Pro with a 64 bit processor. Reporter: Matthew Runo Attachments: schema.xml, solrconfig.xml, Synonyms2.txt I've been having issues with synonyms apparently not working. Shorter lists work just fine, while longer lists do not appear to work. I'll attach my schema, config, and synonyms file for testing. As a quick example... this works... zutanoapparel = zutano But this does not... aadias, aadidas, aaidas, adadas, adaddas, adaddis, adadias, adadis, adaidas, adaies, addedas, addedis, addidaas, addidads, addidais, addidas, addidascom, addiddas, addides, addidis, adeadas, adedas, adeddas, adedias, adiada, adiadas, adiadis, adiads, adida, adidaas, adidas1, adidass, adidaz, adidda, adiddas, adiddias, adidias, adidis, adiidas, aditas, adudas, afidas, aididas, wwwadidascom = adidas This works... liumiani, loomiani, lumaini, lumanai, lumani, lumiami, lumian, lumiana, lumianai, lumiari, luminani, lumini, luminiani = lumiani But this does not... clegerie, cleregie, clergerie, clergie, robertclaregie, robert claregie, robertclargeries, robert clargeries, robertclegerie, robert clegerie, robertcleregie, robert cleregie, robertclergeic, robert clergeic, robertclergerie, robertclergi, robert clergi, robertclergie, robert clergie, robertclergoe, robert clergoe, robertclerige, robert clerige, robertclerterie, robert clerterie = Robert Clergerie -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-606) spellcheck.colate doesn't handle multiple tokens properly
[ https://issues.apache.org/jira/browse/SOLR-606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Grant Ingersoll resolved SOLR-606. -- Resolution: Fixed Fix Version/s: 1.3 Committed revision 685983. Also added in a unit test that caused it to fail w/o the patch. spellcheck.colate doesn't handle multiple tokens properly - Key: SOLR-606 URL: https://issues.apache.org/jira/browse/SOLR-606 Project: Solr Issue Type: Bug Components: spellchecker Affects Versions: 1.3 Environment: tomcat Reporter: Geoffrey Young Assignee: Grant Ingersoll Priority: Minor Fix For: 1.3 Attachments: handler.component.SpellCheckComponent-collate-patch.txt, SOLR-606.patch originally posted as part of SOLR-572: https://issues.apache.org/jira/browse/SOLR-572?focusedCommentId=12608487#action_12608487 the new spellcheck.collate feature seems to exhibit some strange behaviors when handed a query with multiple tokens. {noformat} { responseHeader:{ params:{ q:redbull air show}}, spellcheck:{ suggestions:[ redbull,[ suggestion,[redbelly]], show,[ suggestion,[shot]], collation,redbelly airshotw]}} {noformat} in this case, note the fields are incorrectly concatenated (no space between tokens, left over 'w' from input string) {noformat} { responseHeader:{ params:{ q:redbull air show, spellcheck.q:redbull air show}}, spellcheck:{ suggestions:[ redbull air show,[ suggestion,[redbull singers]], collation,redbull singersredbull air show]}} {noformat} this is slightly different - the suggestions are still concatenated without a space, but the collation is way off. --Geoff -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622632#action_12622632 ] Noble Paul commented on SOLR-647: - The CoreContainer#create(CoreDescriptor dcore) must close the old core after creating the new one Do SolrCore.close() in a refcounted way --- Key: SOLR-647 URL: https://issues.apache.org/jira/browse/SOLR-647 Project: Solr Issue Type: Bug Affects Versions: 1.3 Reporter: Noble Paul Assignee: Grant Ingersoll Fix For: 1.3 Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch The method _SolrCore.close()_ directly closes the core . It can cause Exceptions for in-flight requests. The _close()_ method should just do a decrement on refcount and the actual close must happen when the last request being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-545) remove MultiCore default core / cleanup DispatchHandler
[ https://issues.apache.org/jira/browse/SOLR-545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McKinley resolved SOLR-545. Resolution: Fixed remove MultiCore default core / cleanup DispatchHandler - Key: SOLR-545 URL: https://issues.apache.org/jira/browse/SOLR-545 Project: Solr Issue Type: Bug Affects Versions: 1.3 Reporter: Ryan McKinley Assignee: Ryan McKinley Fix For: 1.3 Attachments: SOLR-545-path-access.patch, solr-545.patch, solr-545.patch, solr-545.patch, solr-545.patch MultiCore should require a core name in the URL. If the core name is missing, there should be a 404, not a valid core. That is: http://localhost:8983/solr/select?q=*:* should return 404. While we are at it, we should cleanup the DispatchHandler. Perhaps the best approach is to treat single core as multicore with only one core? As is the tangle of potential paths is ugly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-695) Make single core solr equivolent to multicore with a single configured core.
[ https://issues.apache.org/jira/browse/SOLR-695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan McKinley resolved SOLR-695. Resolution: Fixed Make single core solr equivolent to multicore with a single configured core. Key: SOLR-695 URL: https://issues.apache.org/jira/browse/SOLR-695 Project: Solr Issue Type: Improvement Affects Versions: 1.3 Reporter: Ryan McKinley Assignee: Ryan McKinley Fix For: 1.3 Attachments: SOLR-695-no-singlecore.patch, SOLR-695-refactor-multicore.patch Multicore vs single core configuration has caused a lot of confusion. To limit this confusion, I think we should: 1. rename MultiCore to CoreContainer (SOLR-689) 2. always register CoreContainer and include all valid cores within it. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-700) NumberFormatTransformer should have configurable locales
[ https://issues.apache.org/jira/browse/SOLR-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622650#action_12622650 ] Shalin Shekhar Mangar commented on SOLR-700: bq. In any case the getISO3Country method of the Locale class (among others) throws an exception if it's invoked on an invalid locale. Maybe it would be best to check all available locales explicitly instead of relying on the regex, possibly by constructing a static HashMap of them?! We don't need to be paranoid about this because if such a Locale is used for parsing, the NumberFormat#getInstance will throw a MissingResourceException for the lack of corresponding resource bundle. Reading through the docs, another area I'm becoming concerned about is partial parsing of the string. Pehaps we should use ParsePosition and verify that the whole string was actually used. See the section on validation at http://www.ibm.com/developerworks/java/library/j-numberformat/ NumberFormatTransformer should have configurable locales Key: SOLR-700 URL: https://issues.apache.org/jira/browse/SOLR-700 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Affects Versions: 1.3 Reporter: Stefan Oestreicher Assignee: Shalin Shekhar Mangar Priority: Minor Fix For: 1.3 Attachments: handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.TestNumberFormatTransformer-locale.patch.txt, schema.DateField-locale.patch.txt, SOLR-700.patch NumberFormatTransformer uses a NumberFormatter which relies on the system locale. This makes it impossible to use NumberFormatTransformer with data whose locale does not match the system locale. TestNumberFormatTransformer fails on some locales for similar reasons because the grouping symbol differs in different locales. This issue adds a locale attribute for NumberFormatTransformer which allows the user to specify the locale which should be used for formatting. The locale must be specified as land-country e.g. en-US {code:xml} field column=myNumber formatStyle=number locale=de-DE / {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-374) use IndexReader.reopen
[ https://issues.apache.org/jira/browse/SOLR-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622655#action_12622655 ] Yonik Seeley commented on SOLR-374: --- Looking pretty good now... but there is a reference leak. decref() should always be called for any RefCounted instances (preferably in a finally block) use IndexReader.reopen -- Key: SOLR-374 URL: https://issues.apache.org/jira/browse/SOLR-374 Project: Solr Issue Type: Improvement Reporter: Yonik Seeley Attachments: SOLR-374.patch, SOLR-374.patch, SOLR-374.patch Take advantage of IndexReader.reopen(): LUCENE-743 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-586) Maven - Solr Artifact Publishing
[ https://issues.apache.org/jira/browse/SOLR-586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622660#action_12622660 ] Ryan McKinley commented on SOLR-586: I'm running into a funny issue. $ ant clean $ ant generate-maven-artifacts everything works great now run: ant generate-maven-artifacts again. The second time, I get: {panel} BUILD FAILED /Users/ryan/Documents/workspace/APACHE/s3/build.xml:645: A tar file cannot include itself {panel} not really a big deal, but figured I'd point it out... Maven - Solr Artifact Publishing Key: SOLR-586 URL: https://issues.apache.org/jira/browse/SOLR-586 Project: Solr Issue Type: New Feature Components: clients - java, contrib - DataImportHandler Affects Versions: 1.3 Reporter: Spencer Crissman Assignee: Shalin Shekhar Mangar Priority: Minor Fix For: 1.3 Attachments: SOLR-586-20080811-craigmcc.zip, SOLR-586-nits.patch, SOLR-586.patch, SOLR-586.patch, SOLR-586.patch, solr-common.pom.xml, solr-dih.pom.xml, solr-server.pom.xml, solr2mvn.sh, solrj.pom.xml I know there is an issue open (SOLR-19) for getting a solr build going under Maven. This issue differs from that in that it does not concern the build process of the solr project, but rather simple dependency management for maven projects that depend on the solr artifacts. I've outlined a way to easily incorporate solrj + dependencies into your own maven projects, in hopes that others doing this find it useful. This issue's purpose is twofold: 1) Let others know the process. 2) Open the idea of whether this can be streamlined/incorporated into the standard build in some manner. Depending on Solrj in a Maven Project 1) Build a 1.3 snapshot. 1.1) Check out the code from http://svn.apache.org/repos/asf/lucene/solr/ 1.2) Build using ant dist 2) Install the artifacts into your maven repo, using the included pom files. 2.1) Move to your dist/apache-solr-1.3-dev/dist directory. 2.2) Copy the attached pom files into this directory. 2.3) Install solr-common into your repo. ex) mvn install:install-file -Dfile=apache-solr-common-1.3-dev.jar -DpomFile=solr-common.pom.xml 2.4) Install solrj into your repo. ex) mvn install:install-file -Dfile=apache-solr-solrj-1.3-dev.jar -DpomFile=solrj.pom.xml 3) Use Solrj in your existing Maven projects by including it as a dependency in your own pom.xml dependency groupIdorg.apache.lucene.solr/groupId artifactIdsolrj/artifactId version1.3-SNAPSHOT/version /dependency So given the above process, it seems like it would be relatively simple to standardize this process by: 1) Including the solr-common and solrj pom files w/ the dist. 2) Automating the periodic installation of the artifacts to a central repo, such as the ibiblio repo. If those steps were performed, then creating a (maven) project based on solrj would be super simple: just #3 from above. Since most custom developments are probably for the clients, it seems like simplifying this would be a nice step to take. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-433) MultiCore and SpellChecker replication
[ https://issues.apache.org/jira/browse/SOLR-433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otis Gospodnetic updated SOLR-433: -- Fix Version/s: 1.4 MultiCore and SpellChecker replication -- Key: SOLR-433 URL: https://issues.apache.org/jira/browse/SOLR-433 Project: Solr Issue Type: Improvement Components: replication, spellchecker Affects Versions: 1.3 Reporter: Otis Gospodnetic Fix For: 1.4 Attachments: RunExecutableListener.patch, SOLR-433.patch, solr-433.patch, SOLR-433_unified.patch, spellindexfix.patch With MultiCore functionality coming along, it looks like we'll need to be able to: A) snapshot each core's index directory, and B) replicate any and all cores' complete data directories, not just their index directories. Pulled from the spellchecker and multi-core index replication thread - http://markmail.org/message/pj2rjzegifd6zm7m Otis: I think that makes sense - distribute everything for a given core, not just its index. And the spellchecker could then also have its data dir (and only index/ underneath really) and be replicated in the same fashion. Right? Ryan: Yes, that was my thought. If an arbitrary directory could be distributed, then you could have /path/to/dist/index/... /path/to/dist/spelling-index/... /path/to/dist/foo and that would all get put into a snapshot. This would also let you put multiple cores within a single distribution: /path/to/dist/core0/index/... /path/to/dist/core0/spelling-index/... /path/to/dist/core0/foo /path/to/dist/core1/index/... /path/to/dist/core1/spelling-index/... /path/to/dist/core1/foo -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-586) Maven - Solr Artifact Publishing
[ https://issues.apache.org/jira/browse/SOLR-586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622665#action_12622665 ] Shalin Shekhar Mangar commented on SOLR-586: Nothing to do with generate-maven-artifacts target. Run ant package twice and the same thing happens. Simply deleting the tar/zip before package runs will fix this. I shall make the change in the package target. Maven - Solr Artifact Publishing Key: SOLR-586 URL: https://issues.apache.org/jira/browse/SOLR-586 Project: Solr Issue Type: New Feature Components: clients - java, contrib - DataImportHandler Affects Versions: 1.3 Reporter: Spencer Crissman Assignee: Shalin Shekhar Mangar Priority: Minor Fix For: 1.3 Attachments: SOLR-586-20080811-craigmcc.zip, SOLR-586-nits.patch, SOLR-586.patch, SOLR-586.patch, SOLR-586.patch, solr-common.pom.xml, solr-dih.pom.xml, solr-server.pom.xml, solr2mvn.sh, solrj.pom.xml I know there is an issue open (SOLR-19) for getting a solr build going under Maven. This issue differs from that in that it does not concern the build process of the solr project, but rather simple dependency management for maven projects that depend on the solr artifacts. I've outlined a way to easily incorporate solrj + dependencies into your own maven projects, in hopes that others doing this find it useful. This issue's purpose is twofold: 1) Let others know the process. 2) Open the idea of whether this can be streamlined/incorporated into the standard build in some manner. Depending on Solrj in a Maven Project 1) Build a 1.3 snapshot. 1.1) Check out the code from http://svn.apache.org/repos/asf/lucene/solr/ 1.2) Build using ant dist 2) Install the artifacts into your maven repo, using the included pom files. 2.1) Move to your dist/apache-solr-1.3-dev/dist directory. 2.2) Copy the attached pom files into this directory. 2.3) Install solr-common into your repo. ex) mvn install:install-file -Dfile=apache-solr-common-1.3-dev.jar -DpomFile=solr-common.pom.xml 2.4) Install solrj into your repo. ex) mvn install:install-file -Dfile=apache-solr-solrj-1.3-dev.jar -DpomFile=solrj.pom.xml 3) Use Solrj in your existing Maven projects by including it as a dependency in your own pom.xml dependency groupIdorg.apache.lucene.solr/groupId artifactIdsolrj/artifactId version1.3-SNAPSHOT/version /dependency So given the above process, it seems like it would be relatively simple to standardize this process by: 1) Including the solr-common and solrj pom files w/ the dist. 2) Automating the periodic installation of the artifacts to a central repo, such as the ibiblio repo. If those steps were performed, then creating a (maven) project based on solrj would be super simple: just #3 from above. Since most custom developments are probably for the clients, it seems like simplifying this would be a nice step to take. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: CHANGES.txt
On a related note, I can see a CHANGES.txt inside client/java/solrj folder. However, it has not been updated with many changes that have took place in Solrj. I wasn't even aware of it's existence until recently. Two options: 1. We merge it with the main changelog file and delete it 2. We gather all SolrJ changes from the main changelog and put it here If we decide to go for the 2nd route, is a separate changelog needed for DIH as well? On Sat, Aug 9, 2008 at 8:29 AM, Ryan McKinley [EMAIL PROTECTED] wrote: Hello- Otis pointed out (via direct email) that I have not added many notes in CHANGES.txt recently. I may be getting slopy, but I also remember some discussion about how to use changes.txt a while back and figure we should revist it to make sure everything necessary is in there before 1.3. My understanding is that CHANGES.txt is for folks following what has happened between releases -- it does not need to include internal modifications if there is no impact on the user. For example, SOLR-493 is not there since it fixes an issue introduced since 1.2 -- using CHANGES.txt to say what happened in SOLR-142 had some side-effects that we now have cleared up seems overly complicated for anyone following. Likewise the recent changes/modifictions to the UpdateRequestProcessor framework. Is this understanding correct? Are there more (or fewer) things we should include in CHANGES.txt? ryan -- Regards, Shalin Shekhar Mangar.
[jira] Commented: (SOLR-474) audit docs for Spellchecker
[ https://issues.apache.org/jira/browse/SOLR-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622675#action_12622675 ] Otis Gospodnetic commented on SOLR-474: --- I went through the classes in o.a.s.spell package and things look OK there. Not absolutely all public methods have javadocs, but I think there is enough. The wiki page seems pretty thorough, but I can't verify if absolutely everything there is 100% correct and up to date without running the Spellchecker component myself, which I'm currently unable to do. I'd mark this as resolved and update the Wiki when/if we/others find errors or omissions. Mike? audit docs for Spellchecker --- Key: SOLR-474 URL: https://issues.apache.org/jira/browse/SOLR-474 Project: Solr Issue Type: Task Affects Versions: 1.3 Reporter: Hoss Man Assignee: Mike Klaas Fix For: 1.3 according to this troubling comment from Mike, the spellchecker handler javadocs (and wiki) may not reflect reality... http://www.nabble.com/spellcheckhandler-to14627712.html#a14627712 {quote} Multi-word spell checking is available only with extendedResults=true, and only in trunk. I believe that the current javadocs are incorrect on this point. {quote} we should audit/fix this before 1.3 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: CHANGES.txt
I wasn't aware of it either :) I like the idea of keeping it separate, mostly because I fear the core CHANGES.txt growing super big and thus harder for people to use, especially as contrib/ starts growing. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Shalin Shekhar Mangar [EMAIL PROTECTED] To: solr-dev@lucene.apache.org Sent: Thursday, August 14, 2008 4:27:22 PM Subject: Re: CHANGES.txt On a related note, I can see a CHANGES.txt inside client/java/solrj folder. However, it has not been updated with many changes that have took place in Solrj. I wasn't even aware of it's existence until recently. Two options: 1. We merge it with the main changelog file and delete it 2. We gather all SolrJ changes from the main changelog and put it here If we decide to go for the 2nd route, is a separate changelog needed for DIH as well? On Sat, Aug 9, 2008 at 8:29 AM, Ryan McKinley wrote: Hello- Otis pointed out (via direct email) that I have not added many notes in CHANGES.txt recently. I may be getting slopy, but I also remember some discussion about how to use changes.txt a while back and figure we should revist it to make sure everything necessary is in there before 1.3. My understanding is that CHANGES.txt is for folks following what has happened between releases -- it does not need to include internal modifications if there is no impact on the user. For example, SOLR-493 is not there since it fixes an issue introduced since 1.2 -- using CHANGES.txt to say what happened in SOLR-142 had some side-effects that we now have cleared up seems overly complicated for anyone following. Likewise the recent changes/modifictions to the UpdateRequestProcessor framework. Is this understanding correct? Are there more (or fewer) things we should include in CHANGES.txt? ryan -- Regards, Shalin Shekhar Mangar.
[jira] Commented: (SOLR-474) audit docs for Spellchecker
[ https://issues.apache.org/jira/browse/SOLR-474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622677#action_12622677 ] Mike Klaas commented on SOLR-474: - The issue is more wikidocs vs. behaviour. I apologize I haven't gotten to this yet--I've been suffering from RSI the last month or so and it has been difficult to get it non-work computer time. I'll take a look today. audit docs for Spellchecker --- Key: SOLR-474 URL: https://issues.apache.org/jira/browse/SOLR-474 Project: Solr Issue Type: Task Affects Versions: 1.3 Reporter: Hoss Man Assignee: Mike Klaas Fix For: 1.3 according to this troubling comment from Mike, the spellchecker handler javadocs (and wiki) may not reflect reality... http://www.nabble.com/spellcheckhandler-to14627712.html#a14627712 {quote} Multi-word spell checking is available only with extendedResults=true, and only in trunk. I believe that the current javadocs are incorrect on this point. {quote} we should audit/fix this before 1.3 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: CHANGES.txt
I think an independent location for solrj changes is a good thing. They are not relevant for anyone using any other client... the ruby client has its own changelog. Solrj is a bit strange though since it is used within the SearchComponents. Since solrj has not yet been released, the coverage in CHANGES.txt it a bit limited -- i don't think it is important for to follow all the internal changes. Re independent change log for DIH? that sounds good to me. ryan On Aug 14, 2008, at 4:27 PM, Shalin Shekhar Mangar wrote: On a related note, I can see a CHANGES.txt inside client/java/solrj folder. However, it has not been updated with many changes that have took place in Solrj. I wasn't even aware of it's existence until recently. Two options: 1. We merge it with the main changelog file and delete it 2. We gather all SolrJ changes from the main changelog and put it here If we decide to go for the 2nd route, is a separate changelog needed for DIH as well? On Sat, Aug 9, 2008 at 8:29 AM, Ryan McKinley [EMAIL PROTECTED] wrote: Hello- Otis pointed out (via direct email) that I have not added many notes in CHANGES.txt recently. I may be getting slopy, but I also remember some discussion about how to use changes.txt a while back and figure we should revist it to make sure everything necessary is in there before 1.3. My understanding is that CHANGES.txt is for folks following what has happened between releases -- it does not need to include internal modifications if there is no impact on the user. For example, SOLR-493 is not there since it fixes an issue introduced since 1.2 -- using CHANGES.txt to say what happened in SOLR-142 had some side-effects that we now have cleared up seems overly complicated for anyone following. Likewise the recent changes/modifictions to the UpdateRequestProcessor framework. Is this understanding correct? Are there more (or fewer) things we should include in CHANGES.txt? ryan -- Regards, Shalin Shekhar Mangar.
[jira] Updated: (SOLR-284) Parsing Rich Document Types
[ https://issues.apache.org/jira/browse/SOLR-284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otis Gospodnetic updated SOLR-284: -- Fix Version/s: 1.4 Parsing Rich Document Types --- Key: SOLR-284 URL: https://issues.apache.org/jira/browse/SOLR-284 Project: Solr Issue Type: New Feature Components: update Reporter: Eric Pugh Fix For: 1.4 Attachments: libs.zip, rich.patch, rich.patch, rich.patch, rich.patch, rich.patch, source.zip, test-files.zip, test-files.zip, test.zip I have developed a RichDocumentRequestHandler based on the CSVRequestHandler that supports streaming a PDF, Word, Powerpoint, Excel, or PDF document into Solr. There is a wiki page with information here: http://wiki.apache.org/solr/UpdateRichDocuments -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-586) Maven - Solr Artifact Publishing
[ https://issues.apache.org/jira/browse/SOLR-586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622679#action_12622679 ] Ryan McKinley commented on SOLR-586: the included pom is missing a reference to commons-fileupload. This is needed in the core and embedded solrj {noformat} Index: client/java/solrj/solr-solrj-pom.xml.template === --- client/java/solrj/solr-solrj-pom.xml.template (revision 686022) +++ client/java/solrj/solr-solrj-pom.xml.template (working copy) @@ -66,6 +66,11 @@ artifactIdcommons-logging/artifactId version1.0.4/version /dependency +dependency + groupIdcommons-fileupload/groupId + artifactIdcommons-fileupload/artifactId + version1.2/version +/dependency !-- Stax -- dependency {noformat} Maven - Solr Artifact Publishing Key: SOLR-586 URL: https://issues.apache.org/jira/browse/SOLR-586 Project: Solr Issue Type: New Feature Components: clients - java, contrib - DataImportHandler Affects Versions: 1.3 Reporter: Spencer Crissman Assignee: Shalin Shekhar Mangar Priority: Minor Fix For: 1.3 Attachments: SOLR-586-20080811-craigmcc.zip, SOLR-586-nits.patch, SOLR-586.patch, SOLR-586.patch, SOLR-586.patch, solr-common.pom.xml, solr-dih.pom.xml, solr-server.pom.xml, solr2mvn.sh, solrj.pom.xml I know there is an issue open (SOLR-19) for getting a solr build going under Maven. This issue differs from that in that it does not concern the build process of the solr project, but rather simple dependency management for maven projects that depend on the solr artifacts. I've outlined a way to easily incorporate solrj + dependencies into your own maven projects, in hopes that others doing this find it useful. This issue's purpose is twofold: 1) Let others know the process. 2) Open the idea of whether this can be streamlined/incorporated into the standard build in some manner. Depending on Solrj in a Maven Project 1) Build a 1.3 snapshot. 1.1) Check out the code from http://svn.apache.org/repos/asf/lucene/solr/ 1.2) Build using ant dist 2) Install the artifacts into your maven repo, using the included pom files. 2.1) Move to your dist/apache-solr-1.3-dev/dist directory. 2.2) Copy the attached pom files into this directory. 2.3) Install solr-common into your repo. ex) mvn install:install-file -Dfile=apache-solr-common-1.3-dev.jar -DpomFile=solr-common.pom.xml 2.4) Install solrj into your repo. ex) mvn install:install-file -Dfile=apache-solr-solrj-1.3-dev.jar -DpomFile=solrj.pom.xml 3) Use Solrj in your existing Maven projects by including it as a dependency in your own pom.xml dependency groupIdorg.apache.lucene.solr/groupId artifactIdsolrj/artifactId version1.3-SNAPSHOT/version /dependency So given the above process, it seems like it would be relatively simple to standardize this process by: 1) Including the solr-common and solrj pom files w/ the dist. 2) Automating the periodic installation of the artifacts to a central repo, such as the ibiblio repo. If those steps were performed, then creating a (maven) project based on solrj would be super simple: just #3 from above. Since most custom developments are probably for the clients, it seems like simplifying this would be a nice step to take. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-374) use IndexReader.reopen
[ https://issues.apache.org/jira/browse/SOLR-374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Miller updated SOLR-374: - Attachment: SOLR-374.patch Okay. Not sure how kosher taking ownership of the Reader form SolrIndexSearcher is, but it seems the thing to do then. use IndexReader.reopen -- Key: SOLR-374 URL: https://issues.apache.org/jira/browse/SOLR-374 Project: Solr Issue Type: Improvement Reporter: Yonik Seeley Attachments: SOLR-374.patch, SOLR-374.patch, SOLR-374.patch, SOLR-374.patch Take advantage of IndexReader.reopen(): LUCENE-743 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-586) Maven - Solr Artifact Publishing
[ https://issues.apache.org/jira/browse/SOLR-586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622688#action_12622688 ] Shalin Shekhar Mangar commented on SOLR-586: Committed revision 686039. Sorry, my bad. Thanks Ryan! Maven - Solr Artifact Publishing Key: SOLR-586 URL: https://issues.apache.org/jira/browse/SOLR-586 Project: Solr Issue Type: New Feature Components: clients - java, contrib - DataImportHandler Affects Versions: 1.3 Reporter: Spencer Crissman Assignee: Shalin Shekhar Mangar Priority: Minor Fix For: 1.3 Attachments: SOLR-586-20080811-craigmcc.zip, SOLR-586-nits.patch, SOLR-586.patch, SOLR-586.patch, SOLR-586.patch, solr-common.pom.xml, solr-dih.pom.xml, solr-server.pom.xml, solr2mvn.sh, solrj.pom.xml I know there is an issue open (SOLR-19) for getting a solr build going under Maven. This issue differs from that in that it does not concern the build process of the solr project, but rather simple dependency management for maven projects that depend on the solr artifacts. I've outlined a way to easily incorporate solrj + dependencies into your own maven projects, in hopes that others doing this find it useful. This issue's purpose is twofold: 1) Let others know the process. 2) Open the idea of whether this can be streamlined/incorporated into the standard build in some manner. Depending on Solrj in a Maven Project 1) Build a 1.3 snapshot. 1.1) Check out the code from http://svn.apache.org/repos/asf/lucene/solr/ 1.2) Build using ant dist 2) Install the artifacts into your maven repo, using the included pom files. 2.1) Move to your dist/apache-solr-1.3-dev/dist directory. 2.2) Copy the attached pom files into this directory. 2.3) Install solr-common into your repo. ex) mvn install:install-file -Dfile=apache-solr-common-1.3-dev.jar -DpomFile=solr-common.pom.xml 2.4) Install solrj into your repo. ex) mvn install:install-file -Dfile=apache-solr-solrj-1.3-dev.jar -DpomFile=solrj.pom.xml 3) Use Solrj in your existing Maven projects by including it as a dependency in your own pom.xml dependency groupIdorg.apache.lucene.solr/groupId artifactIdsolrj/artifactId version1.3-SNAPSHOT/version /dependency So given the above process, it seems like it would be relatively simple to standardize this process by: 1) Including the solr-common and solrj pom files w/ the dist. 2) Automating the periodic installation of the artifacts to a central repo, such as the ibiblio repo. If those steps were performed, then creating a (maven) project based on solrj would be super simple: just #3 from above. Since most custom developments are probably for the clients, it seems like simplifying this would be a nice step to take. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-575) Highlighting spans should merge across phrase query
[ https://issues.apache.org/jira/browse/SOLR-575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otis Gospodnetic updated SOLR-575: -- Priority: Minor (was: Major) Brian, perhaps something as simple as the java equivalent of s//span span//g might work? Highlighting spans should merge across phrase query --- Key: SOLR-575 URL: https://issues.apache.org/jira/browse/SOLR-575 Project: Solr Issue Type: Improvement Components: highlighter Affects Versions: 1.2 Reporter: Brian Whitman Priority: Minor Somewhat related to but separate from SOLR-553, It would be nice if the highlighter component joined the formatter tags across an entire PhraseQuery. e.g. Lights (Live) : spanI/span spanLove/span spanYou/span But spanI've/span spanChosen/span spanDarkness/span : should really be Lights (Live) : spanI Love You But I've Chosen Darkness/span : assuming the query that generated these fragments was I Love You But I've Chosen Darkness I assume there's issues with stopwords (the But in the name was not formatted) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-575) Highlighting spans should merge across phrase query
[ https://issues.apache.org/jira/browse/SOLR-575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622695#action_12622695 ] Brian Whitman commented on SOLR-575: Sure, post-processing is somewhat easy, except for stopwords (note the But in the example) -- it's just one of those quality-of-life concerns :) Highlighting spans should merge across phrase query --- Key: SOLR-575 URL: https://issues.apache.org/jira/browse/SOLR-575 Project: Solr Issue Type: Improvement Components: highlighter Affects Versions: 1.2 Reporter: Brian Whitman Priority: Minor Somewhat related to but separate from SOLR-553, It would be nice if the highlighter component joined the formatter tags across an entire PhraseQuery. e.g. Lights (Live) : spanI/span spanLove/span spanYou/span But spanI've/span spanChosen/span spanDarkness/span : should really be Lights (Live) : spanI Love You But I've Chosen Darkness/span : assuming the query that generated these fragments was I Love You But I've Chosen Darkness I assume there's issues with stopwords (the But in the name was not formatted) -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-374) use IndexReader.reopen
[ https://issues.apache.org/jira/browse/SOLR-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622696#action_12622696 ] Mark Miller commented on SOLR-374: -- Hmmm...probably need a searcher lock around taking reader ownership use IndexReader.reopen -- Key: SOLR-374 URL: https://issues.apache.org/jira/browse/SOLR-374 Project: Solr Issue Type: Improvement Reporter: Yonik Seeley Attachments: SOLR-374.patch, SOLR-374.patch, SOLR-374.patch, SOLR-374.patch Take advantage of IndexReader.reopen(): LUCENE-743 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (SOLR-374) use IndexReader.reopen
[ https://issues.apache.org/jira/browse/SOLR-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622696#action_12622696 ] [EMAIL PROTECTED] edited comment on SOLR-374 at 8/14/08 2:04 PM: --- Hmmm...probably need a searcher lock around taking reader ownership or not...the incref will keep it from being closed. NM. was (Author: [EMAIL PROTECTED]): Hmmm...probably need a searcher lock around taking reader ownership use IndexReader.reopen -- Key: SOLR-374 URL: https://issues.apache.org/jira/browse/SOLR-374 Project: Solr Issue Type: Improvement Reporter: Yonik Seeley Attachments: SOLR-374.patch, SOLR-374.patch, SOLR-374.patch, SOLR-374.patch Take advantage of IndexReader.reopen(): LUCENE-743 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-589) DisMaxRequestHandler crashes on badly formed query strings ( combinations of - and + )
[ https://issues.apache.org/jira/browse/SOLR-589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otis Gospodnetic resolved SOLR-589. --- Resolution: Fixed Assignee: Otis Gospodnetic Thanks Sean! SendingCHANGES.txt Sendingsrc/java/org/apache/solr/search/DisMaxQParserPlugin.java Sendingsrc/java/org/apache/solr/util/SolrPluginUtils.java Sendingsrc/test/org/apache/solr/util/SolrPluginUtilsTest.java Transmitting file data Committed revision 686042. DisMaxRequestHandler crashes on badly formed query strings ( combinations of - and + ) -- Key: SOLR-589 URL: https://issues.apache.org/jira/browse/SOLR-589 Project: Solr Issue Type: Bug Components: search Affects Versions: 1.2 Environment: all platforms Reporter: bram de jong Assignee: Otis Gospodnetic Priority: Minor Attachments: SOLR-589.patch, SOLR-589.patch, SOLR-589.patch The DisMaxRequestHandler parser crashes on strings which contain double dashes or various combinations of - and + like: chocolate cookie - chocolate -+cookie chocolate --cookie chocolate - - cookie Originally found by me: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/200806.mbox/browser And verified by Sean Tim: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/200806.mbox/browser -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Resolved: (SOLR-93) scripts don't work cleanly on FreeBSD
[ https://issues.apache.org/jira/browse/SOLR-93?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Otis Gospodnetic resolved SOLR-93. -- Resolution: Fixed Assignee: Otis Gospodnetic OK, closing. If you encounter other OS-related problems, feel free to submit a patch. Of course, SOLR-561 is coming in Solr 1.4! :) scripts don't work cleanly on FreeBSD - Key: SOLR-93 URL: https://issues.apache.org/jira/browse/SOLR-93 Project: Solr Issue Type: Bug Components: replication Affects Versions: 1.1.0 Reporter: Hoss Man Assignee: Otis Gospodnetic As first reported by liuping.james in this thread... http://www.nabble.com/SolrDistribution-is-tested-by-FreeBSD--tf2869098.html#a8019145 ...the various distribution scripts have issues using the FreeBSD version of bash ... more specificly, they don't run cleanly via bash script-name on FreeBSD systems. notes from when i verified this... -- I just tried this out on the only FreeBSD system i have access to (minotaur.apache.org) and found a few things... 1) /bin/bash doesn't exist 2) /bin/sh is true sh, not bash 3) bash is installed in /usr/local/bin/bash ...i'm not sure how standard these are among FreeBSD systems. I also verified that rsyncd-enable and rsyncd-disable do work if you run them explicitly through bash, but for some reason you must use the -u option, and even then it logs a straing warning about scripts.conf... [EMAIL PROTECTED] bash solr/bin/rsyncd-enable -h /home/hossman/tmp/apache-solr-1.1.0-incubating/example/solr/conf/scripts.conf: l: command not found solr/bin/rsyncd-enable: illegal option -- h usage: rsyncd-enable [-u username] [-v] -u specify user to sudo to before running script -v increase verbosity -V output debugging info [EMAIL PROTECTED] ls solr/logs/ [EMAIL PROTECTED] bash solr/bin/rsyncd-enable /home/hossman/tmp/apache-solr-1.1.0-incubating/example/solr/conf/scripts.conf: l: command not found !udo: no passwd entry for [EMAIL PROTECTED] ls solr/logs/ [EMAIL PROTECTED] bash solr/bin/rsyncd-enable -u hossman /home/hossman/tmp/apache-solr-1.1.0-incubating/example/solr/conf/scripts.conf: l: command not found [EMAIL PROTECTED] ls solr/logs/ rsyncd-enabled rsyncd.log ...so above and beyond the sh vs bash and patch issues .. there does seem to be a compatibility issue. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: CHANGES.txt
Ok, just to summarize: We will maintain separate changelogs for both SolrJ and DIH. We should also make sure these changelogs are included in the nightly tarballs. Even if they are internal changes, no harm in documenting them in a single place. Besides it is the only place where we give credit to our contributors in a proper fashion. So should we move all SolrJ and DIH related changes to their respective changelogs? I share Otis's concern on the inflating main changelog file. On Fri, Aug 15, 2008 at 2:11 AM, Ryan McKinley [EMAIL PROTECTED] wrote: I think an independent location for solrj changes is a good thing. They are not relevant for anyone using any other client... the ruby client has its own changelog. Solrj is a bit strange though since it is used within the SearchComponents. Since solrj has not yet been released, the coverage in CHANGES.txt it a bit limited -- i don't think it is important for to follow all the internal changes. Re independent change log for DIH? that sounds good to me. ryan -- Regards, Shalin Shekhar Mangar.
[jira] Commented: (SOLR-374) use IndexReader.reopen
[ https://issues.apache.org/jira/browse/SOLR-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622704#action_12622704 ] Yonik Seeley commented on SOLR-374: --- bq. or not...the incref will keep it from being closed. NM. Right. I think all you need to add is a decref() use IndexReader.reopen -- Key: SOLR-374 URL: https://issues.apache.org/jira/browse/SOLR-374 Project: Solr Issue Type: Improvement Reporter: Yonik Seeley Attachments: SOLR-374.patch, SOLR-374.patch, SOLR-374.patch, SOLR-374.patch Take advantage of IndexReader.reopen(): LUCENE-743 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-683) Distributed Search / Shards Deadlock
[ https://issues.apache.org/jira/browse/SOLR-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622706#action_12622706 ] Otis Gospodnetic commented on SOLR-683: --- Hm, hard to tell from sparse Jetty javadocs. http://docs.codehaus.org/display/JETTY/Configuring+Connectors states: acceptQueueSize Number of connection requests that can be queued up before the operating system starts to send rejections. Sounds more like the latter than the former. That is, it sounds like Jetty itself might accept connections until the OS starts complaining. Hm, either way this doesn't help if one has an actual deadlock, like the one you described in the 1-thread-example, does it? Distributed Search / Shards Deadlock Key: SOLR-683 URL: https://issues.apache.org/jira/browse/SOLR-683 Project: Solr Issue Type: Bug Components: search Affects Versions: 1.3 Environment: Linux jre1.6.0_05 8GB RAM 2 x 2 core AMD 2.4 Ghz 2 x 140GB disk Reporter: Cameron Assignee: Yonik Seeley Fix For: 1.3 Attachments: locked.log, SOLR-683.patch Per this discussion: http://www.nabble.com/Distributed-Search-Strategy---Shards-td18882112.html Solr seems to lock up when running distributed search on three servers, with all three using shards of each other. Thread dump attached. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-683) Distributed Search / Shards Deadlock
[ https://issues.apache.org/jira/browse/SOLR-683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622705#action_12622705 ] Mark Miller commented on SOLR-683: -- When jetty runs out of handler threads, does it not accept new connections, or does it accept the connection and wait for a thread to become free to handle it? Not sure if this is still the case, but I believe Jetty did just use the standard socket backlog queue and set it by default to the number of service threads - so you can have that many threadless requests queued up. Dunno if they changed that recently or not. - Mark Distributed Search / Shards Deadlock Key: SOLR-683 URL: https://issues.apache.org/jira/browse/SOLR-683 Project: Solr Issue Type: Bug Components: search Affects Versions: 1.3 Environment: Linux jre1.6.0_05 8GB RAM 2 x 2 core AMD 2.4 Ghz 2 x 140GB disk Reporter: Cameron Assignee: Yonik Seeley Fix For: 1.3 Attachments: locked.log, SOLR-683.patch Per this discussion: http://www.nabble.com/Distributed-Search-Strategy---Shards-td18882112.html Solr seems to lock up when running distributed search on three servers, with all three using shards of each other. Thread dump attached. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: CHANGES.txt
I wouldn't worry about moving of already recorded changes (i.e. cleanup). I'd just do the right thing from now on. Getting 1.3 out next week sounds much more interesting to me. :) Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Shalin Shekhar Mangar [EMAIL PROTECTED] To: solr-dev@lucene.apache.org Sent: Thursday, August 14, 2008 5:16:07 PM Subject: Re: CHANGES.txt Ok, just to summarize: We will maintain separate changelogs for both SolrJ and DIH. We should also make sure these changelogs are included in the nightly tarballs. Even if they are internal changes, no harm in documenting them in a single place. Besides it is the only place where we give credit to our contributors in a proper fashion. So should we move all SolrJ and DIH related changes to their respective changelogs? I share Otis's concern on the inflating main changelog file. On Fri, Aug 15, 2008 at 2:11 AM, Ryan McKinley wrote: I think an independent location for solrj changes is a good thing. They are not relevant for anyone using any other client... the ruby client has its own changelog. Solrj is a bit strange though since it is used within the SearchComponents. Since solrj has not yet been released, the coverage in CHANGES.txt it a bit limited -- i don't think it is important for to follow all the internal changes. Re independent change log for DIH? that sounds good to me. ryan -- Regards, Shalin Shekhar Mangar.
[jira] Updated: (SOLR-589) DisMaxRequestHandler crashes on badly formed query strings ( combinations of - and + )
[ https://issues.apache.org/jira/browse/SOLR-589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shalin Shekhar Mangar updated SOLR-589: --- Affects Version/s: 1.3 Fix Version/s: 1.3 Updating fix version to 1.3 DisMaxRequestHandler crashes on badly formed query strings ( combinations of - and + ) -- Key: SOLR-589 URL: https://issues.apache.org/jira/browse/SOLR-589 Project: Solr Issue Type: Bug Components: search Affects Versions: 1.2, 1.3 Environment: all platforms Reporter: bram de jong Assignee: Otis Gospodnetic Priority: Minor Fix For: 1.3 Attachments: SOLR-589.patch, SOLR-589.patch, SOLR-589.patch The DisMaxRequestHandler parser crashes on strings which contain double dashes or various combinations of - and + like: chocolate cookie - chocolate -+cookie chocolate --cookie chocolate - - cookie Originally found by me: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/200806.mbox/browser And verified by Sean Tim: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/200806.mbox/browser -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-700) NumberFormatTransformer should have configurable locales
[ https://issues.apache.org/jira/browse/SOLR-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622709#action_12622709 ] Stefan Oestreicher commented on SOLR-700: - bq. We don't need to be paranoid about this [...] Ok. bq. Reading through the docs, another area I'm becoming concerned about is partial parsing of the string I see. Interestingly this behaviour of accepting invalid strings seems to only apply to the generic formatter. I wrote some tests and one of them uses a percent formatter (I also tested currency) and that throws a ParseException. Fixing that seems to be even easier than shown in the example :) I already have a patch ready but I guess I should open another issue for this (seems pretty seperate to me)? NumberFormatTransformer should have configurable locales Key: SOLR-700 URL: https://issues.apache.org/jira/browse/SOLR-700 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Affects Versions: 1.3 Reporter: Stefan Oestreicher Assignee: Shalin Shekhar Mangar Priority: Minor Fix For: 1.3 Attachments: handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.TestNumberFormatTransformer-locale.patch.txt, schema.DateField-locale.patch.txt, SOLR-700.patch NumberFormatTransformer uses a NumberFormatter which relies on the system locale. This makes it impossible to use NumberFormatTransformer with data whose locale does not match the system locale. TestNumberFormatTransformer fails on some locales for similar reasons because the grouping symbol differs in different locales. This issue adds a locale attribute for NumberFormatTransformer which allows the user to specify the locale which should be used for formatting. The locale must be specified as land-country e.g. en-US {code:xml} field column=myNumber formatStyle=number locale=de-DE / {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-700) NumberFormatTransformer should have configurable locales
[ https://issues.apache.org/jira/browse/SOLR-700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622711#action_12622711 ] Shalin Shekhar Mangar commented on SOLR-700: bq. Fixing that seems to be even easier than shown in the example I already have a patch ready but I guess I should open another issue for this (seems pretty seperate to me)? By all means, go ahead :) I will commit this one now. NumberFormatTransformer should have configurable locales Key: SOLR-700 URL: https://issues.apache.org/jira/browse/SOLR-700 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Affects Versions: 1.3 Reporter: Stefan Oestreicher Assignee: Shalin Shekhar Mangar Priority: Minor Fix For: 1.3 Attachments: handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.NumberFormatTransformer-locale.patch..txt, handler.dataimport.TestNumberFormatTransformer-locale.patch.txt, schema.DateField-locale.patch.txt, SOLR-700.patch NumberFormatTransformer uses a NumberFormatter which relies on the system locale. This makes it impossible to use NumberFormatTransformer with data whose locale does not match the system locale. TestNumberFormatTransformer fails on some locales for similar reasons because the grouping symbol differs in different locales. This issue adds a locale attribute for NumberFormatTransformer which allows the user to specify the locale which should be used for formatting. The locale must be specified as land-country e.g. en-US {code:xml} field column=myNumber formatStyle=number locale=de-DE / {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-703) Make UUIDField.toInternal use UUID.fromString
[ https://issues.apache.org/jira/browse/SOLR-703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dariusz Suchojad updated SOLR-703: -- Attachment: SOLR-703-code.patch Code. Make UUIDField.toInternal use UUID.fromString - Key: SOLR-703 URL: https://issues.apache.org/jira/browse/SOLR-703 Project: Solr Issue Type: Improvement Affects Versions: 1.3 Reporter: Dariusz Suchojad Priority: Minor Attachments: SOLR-703-code.patch I thought I'd just mention that java.util.UUID supports parsing of strings containing UUIDs while still choking on anything UUID-like or simply plain wrong, as seen in attached patch and unittests. It won't accept Windows GUIDs with curly brackets {12345678-1234-5678-1234-567812345678}, but then again, neither did the previous code. Perhaps it should though? Considering that it will fail when someone starts pulling data directly from MSSQL (http://wiki.apache.org/solr/DataImportHandler)? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-703) Make UUIDField.toInternal use UUID.fromString
Make UUIDField.toInternal use UUID.fromString - Key: SOLR-703 URL: https://issues.apache.org/jira/browse/SOLR-703 Project: Solr Issue Type: Improvement Affects Versions: 1.3 Reporter: Dariusz Suchojad Priority: Minor Attachments: SOLR-703-code.patch I thought I'd just mention that java.util.UUID supports parsing of strings containing UUIDs while still choking on anything UUID-like or simply plain wrong, as seen in attached patch and unittests. It won't accept Windows GUIDs with curly brackets {12345678-1234-5678-1234-567812345678}, but then again, neither did the previous code. Perhaps it should though? Considering that it will fail when someone starts pulling data directly from MSSQL (http://wiki.apache.org/solr/DataImportHandler)? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-703) Make UUIDField.toInternal use UUID.fromString
[ https://issues.apache.org/jira/browse/SOLR-703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dariusz Suchojad updated SOLR-703: -- Attachment: SOLR-703-unittests.patch Unittests. Make UUIDField.toInternal use UUID.fromString - Key: SOLR-703 URL: https://issues.apache.org/jira/browse/SOLR-703 Project: Solr Issue Type: Improvement Affects Versions: 1.3 Reporter: Dariusz Suchojad Priority: Minor Attachments: SOLR-703-code.patch, SOLR-703-unittests.patch I thought I'd just mention that java.util.UUID supports parsing of strings containing UUIDs while still choking on anything UUID-like or simply plain wrong, as seen in attached patch and unittests. It won't accept Windows GUIDs with curly brackets {12345678-1234-5678-1234-567812345678}, but then again, neither did the previous code. Perhaps it should though? Considering that it will fail when someone starts pulling data directly from MSSQL (http://wiki.apache.org/solr/DataImportHandler)? -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-374) use IndexReader.reopen
[ https://issues.apache.org/jira/browse/SOLR-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622717#action_12622717 ] Yonik Seeley commented on SOLR-374: --- I missed the last patch (I wish JIRA defaulted to All). It seems like that if reopen() returns us the same reader, we should just incRef it... (or is that in a slightly later version of Lucene?) Trying to steal the reader instead seems hard to get right (seems like another thread could try to open another searcher, but our searcher doesn't have it and neither does the old one, so your exception might be triggered.) use IndexReader.reopen -- Key: SOLR-374 URL: https://issues.apache.org/jira/browse/SOLR-374 Project: Solr Issue Type: Improvement Reporter: Yonik Seeley Attachments: SOLR-374.patch, SOLR-374.patch, SOLR-374.patch, SOLR-374.patch Take advantage of IndexReader.reopen(): LUCENE-743 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-374) use IndexReader.reopen
[ https://issues.apache.org/jira/browse/SOLR-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622724#action_12622724 ] Mark Miller commented on SOLR-374: -- Man...nothing is ever simple :) A search lock around the ownership change would solve that right? The incref on the Reader is way cleaner though - from what I can tell solr Lucene is a bit too old though. Worth it to wait I think - much better than a sync. use IndexReader.reopen -- Key: SOLR-374 URL: https://issues.apache.org/jira/browse/SOLR-374 Project: Solr Issue Type: Improvement Reporter: Yonik Seeley Attachments: SOLR-374.patch, SOLR-374.patch, SOLR-374.patch, SOLR-374.patch Take advantage of IndexReader.reopen(): LUCENE-743 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-701) DateField implicitly uses default locale to format milliseconds
[ https://issues.apache.org/jira/browse/SOLR-701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Oestreicher updated SOLR-701: Description: The ISO8601CanonicalDateFormat class which is defined inside the DateField class doesn't specify the US locale when creating the formatter used for milliseconds. Because of that the following unit tests fail when run with another locale than english: org.apache.solr.schema.LegacyDateFieldTest org.apache.solr.schema.DateFieldTest The attached patch resolves this issue. was: The following unit tests fail if they're run with a locale using , as decimal separator (e.g. german): org.apache.solr.schema.LegacyDateFieldTest org.apache.solr.schema.DateFieldTest org.apache.solr.handler.dataimport.TestNumberFormatTransformer The DateField tests fail because the expected decimal format uses . as separator but the actual result is formatted using the , separator. This happens because the ISO8601CanonicalDateFormat class which is defined inside the DateField class doesn't specify the US locale when creating the formatter used for milliseconds. Patch: schema.DateField-locale.patch.txt see SOLR-700 for a resolution to the TestNumberFormatTransformer failure. Affects Version/s: (was: 1.3) Summary: DateField implicitly uses default locale to format milliseconds (was: test failures because system locale is used) clarify subject and description DateField implicitly uses default locale to format milliseconds --- Key: SOLR-701 URL: https://issues.apache.org/jira/browse/SOLR-701 Project: Solr Issue Type: Bug Reporter: Stefan Oestreicher Priority: Minor Attachments: schema.DateField-locale.patch.txt The ISO8601CanonicalDateFormat class which is defined inside the DateField class doesn't specify the US locale when creating the formatter used for milliseconds. Because of that the following unit tests fail when run with another locale than english: org.apache.solr.schema.LegacyDateFieldTest org.apache.solr.schema.DateFieldTest The attached patch resolves this issue. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (SOLR-491) highlight doesn't work with range search
[ https://issues.apache.org/jira/browse/SOLR-491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12622734#action_12622734 ] Mark Miller commented on SOLR-491: -- No, it won't work. There is an option to highlight ConstantScoreRange, but by default it is not on. Looking though, there is a bug where I am not respecting that setting, so it is on :) But it really shouldn't be. So it could be turned on in solr, but it can bring problems, the most immediate being that you can't rewrite the query first - which means you can't highlight fuzzy/wildcard multiterm queries. - Mark highlight doesn't work with range search Key: SOLR-491 URL: https://issues.apache.org/jira/browse/SOLR-491 Project: Solr Issue Type: Bug Components: highlighter Affects Versions: 1.3 Environment: windows xp sp2 jboss4.0.5 Reporter: Xuesong Luo Priority: Minor I need to do range search on an integer field, which is defined as type sint. It works fine without highlight. However if I turn on highlight, I got the following error: 2008-02-25 16:54:53,524 ERROR [STDERR] Feb 25, 2008 4:54:53 PM org.apache.solr.core.SolrCore execute INFO: [xluo] /select/rows=10start=0hl.fl=bookCountindent=onq=bookCount:5hl=trueversion=2.2 0 0 2008-02-25 16:54:53,524 ERROR [STDERR] Feb 25, 2008 4:54:53 PM org.apache.solr.common.SolrException log SEVERE: java.lang.NumberFormatException: For input string: at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) at java.lang.Long.parseLong(Long.java:403) at java.lang.Long.parseLong(Long.java:461) at org.apache.solr.util.NumberUtils.long2sortableStr(NumberUtils.java:52) at org.apache.solr.schema.SortableLongField.toInternal(SortableLongField.java:49) at org.apache.solr.schema.FieldType$DefaultAnalyzer$1.next(FieldType.java:315) at org.apache.solr.highlight.TokenOrderingFilter.next(SolrHighlighter.java:439) at org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:226) I also tried range strange on data field, got similr error too when use highlight. I posted the problem at solr user list, here is what Hoss said: -- I'm not sure if i really understand what it would mean to highlight a numeric field, hilighting a range query probably won't ever work because of the way range queries are implemented in Solr ... but at the very least there should be a better error message in this case. (and the case of a simple single value numeric lookup should probably work) -- The reason I need to highlight the numeric or data field is I have to loop through the search result to apply role permission check on those fields. If the searcher doesn't have permission to see the numeric/date field of the user in the search result list, that field should be set to null when returned. If the search doesn't have permission on all matching fields, then the whole record should not be returned. How can I find out which field is the matching field if searching on multiple fields? The only easy way I can think about is if the field is highlighted, it's a matching field. http://www.mail-archive.com/[EMAIL PROTECTED]/msg09239.html -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-647) Do SolrCore.close() in a refcounted way
[ https://issues.apache.org/jira/browse/SOLR-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henri Biestro updated SOLR-647: --- Attachment: solr-647.patch bq. Yes, if cores obtained through other methods need to be open for some reason, then open() would need to be called. bq. I'm not sure if we have any cases like that. No, we don't have any cases today. It could have been useful to someone implementing another filter or servlet. Since the same effect can be obtained with: {code} CoreDescriptor dcore; CoreContainer container; ... SolrCore opened = container.getCore(dcore.getName()); {code} open() can just be incrementAndGet(). (finally!:-)) bq.LOG.severe(XYZ happened. Please report this problem on [EMAIL PROTECTED]) Seems to be the consensus; updated. bq.The CoreContainer#create(CoreDescriptor dcore) must close the old core after creating the new one Correct, fixed. Do SolrCore.close() in a refcounted way --- Key: SOLR-647 URL: https://issues.apache.org/jira/browse/SOLR-647 Project: Solr Issue Type: Bug Affects Versions: 1.3 Reporter: Noble Paul Assignee: Grant Ingersoll Fix For: 1.3 Attachments: refcount_example.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, solr-647.patch, SOLR-647.patch, SOLR-647.patch The method _SolrCore.close()_ directly closes the core . It can cause Exceptions for in-flight requests. The _close()_ method should just do a decrement on refcount and the actual close must happen when the last request being processed by that core instance is completed -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-704) improve NumberFormatTransformers handling of invalid input
improve NumberFormatTransformers handling of invalid input -- Key: SOLR-704 URL: https://issues.apache.org/jira/browse/SOLR-704 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Reporter: Stefan Oestreicher Priority: Minor As described in http://www.ibm.com/developerworks/java/library/j-numberformat/ instances of the NumberFormat class don't throw ParseExceptions in all cases of invalid input. This affects the org.apache.solr.handler.dataimport.NumberFormatTransformer class which relies on NumberFormat. I will attach a patch to demonstrate and fix this issue shortly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (SOLR-704) improve NumberFormatTransformers handling of invalid input
[ https://issues.apache.org/jira/browse/SOLR-704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Oestreicher updated SOLR-704: Attachment: SOLR-704.patch This is a diff against r686094 improve NumberFormatTransformers handling of invalid input -- Key: SOLR-704 URL: https://issues.apache.org/jira/browse/SOLR-704 Project: Solr Issue Type: Improvement Components: contrib - DataImportHandler Reporter: Stefan Oestreicher Priority: Minor Attachments: SOLR-704.patch As described in http://www.ibm.com/developerworks/java/library/j-numberformat/ instances of the NumberFormat class don't throw ParseExceptions in all cases of invalid input. This affects the org.apache.solr.handler.dataimport.NumberFormatTransformer class which relies on NumberFormat. I will attach a patch to demonstrate and fix this issue shortly. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: svn commit: r685577 - in /lucene/solr/trunk: client/java/solrj/src/org/apache/solr/client/solrj/embedded/ client/java/solrj/test/org/apache/solr/client/solrj/ client/java/solrj/test/org/apache/sol
Here's a _possible_ patch: Index: src/webapp/src/org/apache/solr/servlet/SolrDispatchFilter.java === --- src/webapp/src/org/apache/solr/servlet/ SolrDispatchFilter.java (revision 686097) +++ src/webapp/src/org/apache/solr/servlet/ SolrDispatchFilter.java (working copy) @@ -64,7 +64,9 @@ try { // web.xml configuration init.setPathPrefix(config.getInitParameter( path-prefix )); + this.pathPrefix = init.getPathPrefix(); init.setSolrConfigFilename(config.getInitParameter(solrconfig- filename)); + this.solrConfigFilename = init.getSolrConfigFilename(); this.cores = init.initialize(); abortOnConfigurationError = init.isAbortOnConfigurationError(); -Grant On Aug 14, 2008, at 8:56 PM, Grant Ingersoll wrote: Hey Ryan, Something's messed up with this commit in regards to the SolrDispatchFilter. It appears the pathPrefix is no longer being set. It gets set on the CoreContainer.Initializer(), but then the code later checks to see if the SolrDispatchFilter.pathPrefix is set and the path prefix on the init is ignored (not that it could use it, since it is out of scope anyway). Not sure how you want to handle setting the pathPrefix in the SDF. It's easy enough to do, just not sure on the relation between the Initializer and the member variables in SDF. -Grant On Aug 13, 2008, at 10:57 AM, [EMAIL PROTECTED] wrote: Author: ryan Date: Wed Aug 13 07:57:20 2008 New Revision: 685577 URL: http://svn.apache.org/viewvc?rev=685577view=rev Log: SOLR-695 -- refactoring MultiCore to CoreContainer Added: lucene/solr/trunk/src/java/org/apache/solr/core/CoreContainer.java - copied, changed from r685549, lucene/solr/trunk/src/java/org/ apache/solr/core/MultiCore.java lucene/solr/trunk/src/java/org/apache/solr/handler/admin/ CoreAdminHandler.java - copied, changed from r685549, lucene/solr/trunk/src/java/org/ apache/solr/handler/admin/MultiCoreHandler.java Removed: lucene/solr/trunk/src/java/org/apache/solr/core/MultiCore.java lucene/solr/trunk/src/java/org/apache/solr/handler/admin/ MultiCoreHandler.java Modified: lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/ solrj/embedded/EmbeddedSolrServer.java lucene/solr/trunk/client/java/solrj/test/org/apache/solr/client/ solrj/MultiCoreExampleTestBase.java lucene/solr/trunk/client/java/solrj/test/org/apache/solr/client/ solrj/embedded/JettyWebappTest.java lucene/solr/trunk/example/solr/conf/schema.xml lucene/solr/trunk/src/java/org/apache/solr/core/CoreDescriptor.java lucene/solr/trunk/src/java/org/apache/solr/core/SolrCore.java lucene/solr/trunk/src/webapp/src/org/apache/solr/servlet/ SolrDispatchFilter.java lucene/solr/trunk/src/webapp/web/admin/index.jsp lucene/solr/trunk/src/webapp/web/index.jsp Modified: lucene/solr/trunk/client/java/solrj/src/org/apache/solr/ client/solrj/embedded/EmbeddedSolrServer.java URL: http://svn.apache.org/viewvc/lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/solrj/embedded/EmbeddedSolrServer.java?rev=685577r1=685576r2=685577view=diff = = = = = = = = = = --- lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/ solrj/embedded/EmbeddedSolrServer.java (original) +++ lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/ solrj/embedded/EmbeddedSolrServer.java Wed Aug 13 07:57:20 2008 @@ -34,7 +34,7 @@ import org.apache.solr.common.params.ModifiableSolrParams; import org.apache.solr.common.params.SolrParams; import org.apache.solr.common.util.NamedList; -import org.apache.solr.core.MultiCore; +import org.apache.solr.core.CoreContainer; import org.apache.solr.core.SolrCore; import org.apache.solr.request.BinaryResponseWriter; import org.apache.solr.request.QueryResponseWriter; @@ -55,9 +55,9 @@ public class EmbeddedSolrServer extends SolrServer { - protected final MultiCore multicore; // either multicore + protected final CoreContainer multicore; // either multicore protected final SolrCore core; // or single core - protected final String coreName; // use MultiCore registry + protected final String coreName; // use CoreContainer registry private final SolrRequestParsers _parser; @@ -73,10 +73,10 @@ _parser = new SolrRequestParsers( null ); } - public EmbeddedSolrServer( MultiCore multicore, String coreName ) + public EmbeddedSolrServer( CoreContainer multicore, String coreName ) { if ( multicore == null ) { - throw new NullPointerException(MultiCore instance required); + throw new NullPointerException(CoreContainer instance required); } this.core = null; this.multicore = multicore; Modified: lucene/solr/trunk/client/java/solrj/test/org/apache/solr/ client/solrj/MultiCoreExampleTestBase.java URL:
Re: svn commit: r685577 - in /lucene/solr/trunk: client/java/solrj/src/org/apache/solr/client/solrj/embedded/ client/java/solrj/test/org/apache/solr/client/solrj/ client/java/solrj/test/org/apache/sol
thanks grant -- just committed a slightly different one :) On Aug 14, 2008, at 9:06 PM, Grant Ingersoll wrote: Here's a _possible_ patch: Index: src/webapp/src/org/apache/solr/servlet/SolrDispatchFilter.java === --- src/webapp/src/org/apache/solr/servlet/ SolrDispatchFilter.java (revision 686097) +++ src/webapp/src/org/apache/solr/servlet/ SolrDispatchFilter.java (working copy) @@ -64,7 +64,9 @@ try { // web.xml configuration init.setPathPrefix(config.getInitParameter( path-prefix )); + this.pathPrefix = init.getPathPrefix(); init.setSolrConfigFilename(config.getInitParameter(solrconfig- filename)); + this.solrConfigFilename = init.getSolrConfigFilename(); this.cores = init.initialize(); abortOnConfigurationError = init.isAbortOnConfigurationError(); -Grant On Aug 14, 2008, at 8:56 PM, Grant Ingersoll wrote: Hey Ryan, Something's messed up with this commit in regards to the SolrDispatchFilter. It appears the pathPrefix is no longer being set. It gets set on the CoreContainer.Initializer(), but then the code later checks to see if the SolrDispatchFilter.pathPrefix is set and the path prefix on the init is ignored (not that it could use it, since it is out of scope anyway). Not sure how you want to handle setting the pathPrefix in the SDF. It's easy enough to do, just not sure on the relation between the Initializer and the member variables in SDF. -Grant On Aug 13, 2008, at 10:57 AM, [EMAIL PROTECTED] wrote: Author: ryan Date: Wed Aug 13 07:57:20 2008 New Revision: 685577 URL: http://svn.apache.org/viewvc?rev=685577view=rev Log: SOLR-695 -- refactoring MultiCore to CoreContainer Added: lucene/solr/trunk/src/java/org/apache/solr/core/CoreContainer.java - copied, changed from r685549, lucene/solr/trunk/src/java/org/ apache/solr/core/MultiCore.java lucene/solr/trunk/src/java/org/apache/solr/handler/admin/ CoreAdminHandler.java - copied, changed from r685549, lucene/solr/trunk/src/java/org/ apache/solr/handler/admin/MultiCoreHandler.java Removed: lucene/solr/trunk/src/java/org/apache/solr/core/MultiCore.java lucene/solr/trunk/src/java/org/apache/solr/handler/admin/ MultiCoreHandler.java Modified: lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/ solrj/embedded/EmbeddedSolrServer.java lucene/solr/trunk/client/java/solrj/test/org/apache/solr/client/ solrj/MultiCoreExampleTestBase.java lucene/solr/trunk/client/java/solrj/test/org/apache/solr/client/ solrj/embedded/JettyWebappTest.java lucene/solr/trunk/example/solr/conf/schema.xml lucene/solr/trunk/src/java/org/apache/solr/core/CoreDescriptor.java lucene/solr/trunk/src/java/org/apache/solr/core/SolrCore.java lucene/solr/trunk/src/webapp/src/org/apache/solr/servlet/ SolrDispatchFilter.java lucene/solr/trunk/src/webapp/web/admin/index.jsp lucene/solr/trunk/src/webapp/web/index.jsp Modified: lucene/solr/trunk/client/java/solrj/src/org/apache/solr/ client/solrj/embedded/EmbeddedSolrServer.java URL: http://svn.apache.org/viewvc/lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/solrj/embedded/EmbeddedSolrServer.java?rev=685577r1=685576r2=685577view=diff = = = = = = = = = = --- lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/ solrj/embedded/EmbeddedSolrServer.java (original) +++ lucene/solr/trunk/client/java/solrj/src/org/apache/solr/client/ solrj/embedded/EmbeddedSolrServer.java Wed Aug 13 07:57:20 2008 @@ -34,7 +34,7 @@ import org.apache.solr.common.params.ModifiableSolrParams; import org.apache.solr.common.params.SolrParams; import org.apache.solr.common.util.NamedList; -import org.apache.solr.core.MultiCore; +import org.apache.solr.core.CoreContainer; import org.apache.solr.core.SolrCore; import org.apache.solr.request.BinaryResponseWriter; import org.apache.solr.request.QueryResponseWriter; @@ -55,9 +55,9 @@ public class EmbeddedSolrServer extends SolrServer { - protected final MultiCore multicore; // either multicore + protected final CoreContainer multicore; // either multicore protected final SolrCore core; // or single core - protected final String coreName; // use MultiCore registry + protected final String coreName; // use CoreContainer registry private final SolrRequestParsers _parser; @@ -73,10 +73,10 @@ _parser = new SolrRequestParsers( null ); } - public EmbeddedSolrServer( MultiCore multicore, String coreName ) + public EmbeddedSolrServer( CoreContainer multicore, String coreName ) { if ( multicore == null ) { - throw new NullPointerException(MultiCore instance required); + throw new NullPointerException(CoreContainer instance required); } this.core = null; this.multicore = multicore; Modified: lucene/solr/trunk/client/java/solrj/test/org/apache/solr/ client/solrj/MultiCoreExampleTestBase.java
Re: CHANGES.txt
the ruby client has its own changelog. Oops. I wasn't aware of it's existence. But it's name is CHANGES.yml, not CHANGES.txt, and the contents look like gem update info maintained by Erik. If Solrj and DIH have their own CHANGES.txt, should solr-ruby have its CHANGES.txt? Koji Ryan McKinley wrote: I think an independent location for solrj changes is a good thing. They are not relevant for anyone using any other client... the ruby client has its own changelog. Solrj is a bit strange though since it is used within the SearchComponents. Since solrj has not yet been released, the coverage in CHANGES.txt it a bit limited -- i don't think it is important for to follow all the internal changes. Re independent change log for DIH? that sounds good to me. ryan
[jira] Resolved: (SOLR-474) audit docs for Spellchecker
[ https://issues.apache.org/jira/browse/SOLR-474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Klaas resolved SOLR-474. - Resolution: Fixed I've verified the behaviour and updated the wiki page accordingly. audit docs for Spellchecker --- Key: SOLR-474 URL: https://issues.apache.org/jira/browse/SOLR-474 Project: Solr Issue Type: Task Affects Versions: 1.3 Reporter: Hoss Man Assignee: Mike Klaas Fix For: 1.3 according to this troubling comment from Mike, the spellchecker handler javadocs (and wiki) may not reflect reality... http://www.nabble.com/spellcheckhandler-to14627712.html#a14627712 {quote} Multi-word spell checking is available only with extendedResults=true, and only in trunk. I believe that the current javadocs are incorrect on this point. {quote} we should audit/fix this before 1.3 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
Re: contrived use of extends to access static members
: I truly believe SolrPluginUtils.parseFieldBoosts(foo)) is the straight : foward way to call a static method in SolrPluginUtils, extending the like i said: submit a patch and i'll commit it ... i don't mind changing it to the full class name, i just dont' wnat to use a static import. -Hoss