RE: [Lucene.Net] Patches for signed assembly and other minor stuff
Itamar, The best way to have changes incorporated in to the trunk is to create JIRA issues. You will find the Lucene.Net JIRA at https://issues.apache.org/jira/browse/LUCENENET. Scott -Original Message- From: Itamar Syn-Hershko [mailto:ita...@code972.com] Sent: Monday, March 28, 2011 4:10 PM To: lucene-net-dev@lucene.apache.org Subject: [Lucene.Net] Patches for signed assembly and other minor stuff Hi all, We use a local fork of Lucene.NET 2.9.2, and have made made several small changes we'd like to see incorporated to the main branches (2_9_2 and trunk). I'm attaching the patches with this message. The changes are for making the AlreadyClosedException serializable, CLS compliance, and for signing the assembly. Thanks, Itamar.
[Lucene.Net] [jira] [Created] (LUCENENET-407) Signing the assembly
Signing the assembly Key: LUCENENET-407 URL: https://issues.apache.org/jira/browse/LUCENENET-407 Project: Lucene.Net Issue Type: Improvement Components: Lucene.Net Core Affects Versions: Lucene.Net 2.9.2, Lucene.Net 2.9.4, Lucene.Net 3.x Reporter: Itamar Syn-Hershko Fix For: Lucene.Net 2.9.2, Lucene.Net 2.9.4, Lucene.Net 3.x For our usage of Lucene.NET we need the assembly to be signed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Lucene.Net] [jira] [Updated] (LUCENENET-407) Signing the assembly
[ https://issues.apache.org/jira/browse/LUCENENET-407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Itamar Syn-Hershko updated LUCENENET-407: - Attachment: signing.patch Lucene.NET.snk snk file and a patch to apply signing Signing the assembly Key: LUCENENET-407 URL: https://issues.apache.org/jira/browse/LUCENENET-407 Project: Lucene.Net Issue Type: Improvement Components: Lucene.Net Core Affects Versions: Lucene.Net 2.9.2, Lucene.Net 2.9.4, Lucene.Net 3.x Reporter: Itamar Syn-Hershko Fix For: Lucene.Net 2.9.2, Lucene.Net 2.9.4, Lucene.Net 3.x Attachments: Lucene.NET.snk, signing.patch For our usage of Lucene.NET we need the assembly to be signed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[Lucene.Net] [jira] [Created] (LUCENENET-408) Mark assembly as CLS compliant; make AlreadyClosedException serializable
Mark assembly as CLS compliant; make AlreadyClosedException serializable Key: LUCENENET-408 URL: https://issues.apache.org/jira/browse/LUCENENET-408 Project: Lucene.Net Issue Type: Improvement Components: Lucene.Net Core Reporter: Itamar Syn-Hershko Priority: Minor Fix For: Lucene.Net 2.9.2, Lucene.Net 2.9.4, Lucene.Net 3.x * Making sure that the AlreadyClosedException exception can be serialized * Making sure that Lucene is marked as cls compliant -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [Lucene.Net] Patches for signed assembly and other minor stuff
Just did - and we'd like to see this in 2_9_2 and 2_9_4 as well (these are quite trivial). Thanks. https://issues.apache.org/jira/browse/LUCENENET-407 https://issues.apache.org/jira/browse/LUCENENET-408 https://issues.apache.org/jira/browse/LUCENENET-408 On 28/03/2011 22:28, Scott Lombard wrote: Itamar, The best way to have changes incorporated in to the trunk is to create JIRA issues. You will find the Lucene.Net JIRA at https://issues.apache.org/jira/browse/LUCENENET. Scott -Original Message- From: Itamar Syn-Hershko [mailto:ita...@code972.com] Sent: Monday, March 28, 2011 4:10 PM To: lucene-net-dev@lucene.apache.org Subject: [Lucene.Net] Patches for signed assembly and other minor stuff Hi all, We use a local fork of Lucene.NET 2.9.2, and have made made several small changes we'd like to see incorporated to the main branches (2_9_2 and trunk). I'm attaching the patches with this message. The changes are for making the AlreadyClosedException serializable, CLS compliance, and for signing the assembly. Thanks, Itamar.
[Lucene.Net] [jira] [Updated] (LUCENENET-408) Mark assembly as CLS compliant; make AlreadyClosedException serializable
[ https://issues.apache.org/jira/browse/LUCENENET-408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Itamar Syn-Hershko updated LUCENENET-408: - Attachment: commit-40f81a3.patch Attaching patch Mark assembly as CLS compliant; make AlreadyClosedException serializable Key: LUCENENET-408 URL: https://issues.apache.org/jira/browse/LUCENENET-408 Project: Lucene.Net Issue Type: Improvement Components: Lucene.Net Core Reporter: Itamar Syn-Hershko Priority: Minor Fix For: Lucene.Net 2.9.2, Lucene.Net 2.9.4, Lucene.Net 3.x Attachments: commit-40f81a3.patch * Making sure that the AlreadyClosedException exception can be serialized * Making sure that Lucene is marked as cls compliant -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
Search in both database and documents
i just to do a search in both database and documents(pdf,doc etc). how to make schema for both database and documents fields. i m unable to do.
'ant validate-lucene' fails
Hi When I run 'ant validate-lucene' on the latest 3x, I get this: Starting on dir: lucene\lib WARNING: There may be missing NOTICE files in: lucene\lib. Note, not all files require a NOTICE. Jar file count: 3 Notice Count: 2 Invalid license: LICENSE for ant-junit-LICENSE.txt Exception in thread main java.lang.NullPointerException at org.apache.lucene.validation.DependencyChecker.main(DependencyChecker.java:124) I noticed there are several LICENSE.txt files under lucene/lib: * ant-LICENSE-ASL.txt * junit-LICENSE-CPL.txt * ant-junit-LICENSE.txt The last one has a different format, and I believe that's what causing the exception -- there is no type in the name. Am I the only one that encounters this failure? Do we need that file at all (there are LICENSE files for both Ant and JUnit already). Shai
[HUDSON] Lucene-Solr-tests-only-3.x - Build # 6438 - Failure
Build: https://hudson.apache.org/hudson/job/Lucene-Solr-tests-only-3.x/6438/ 1 tests failed. REGRESSION: org.apache.lucene.collation.TestCollationKeyAnalyzer.testThreadSafe Error Message: Java heap space Stack Trace: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2894) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:117) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:589) at java.lang.StringBuffer.append(StringBuffer.java:337) at java.text.RuleBasedCollator.getCollationKey(RuleBasedCollator.java:617) at org.apache.lucene.collation.CollationKeyFilter.incrementToken(CollationKeyFilter.java:93) at org.apache.lucene.collation.CollationTestBase.assertThreadSafe(CollationTestBase.java:304) at org.apache.lucene.collation.TestCollationKeyAnalyzer.testThreadSafe(TestCollationKeyAnalyzer.java:89) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1076) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1008) Build Log (for compile errors): [...truncated 5558 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
how to update one field of document
how to update fields of existing document through http or curl methods.
[HUDSON] Lucene-Solr-tests-only-3.x - Build # 6440 - Failure
Build: https://hudson.apache.org/hudson/job/Lucene-Solr-tests-only-3.x/6440/ 1 tests failed. REGRESSION: org.apache.lucene.collation.TestCollationKeyAnalyzer.testThreadSafe Error Message: Java heap space Stack Trace: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2894) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:117) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:589) at java.lang.StringBuffer.append(StringBuffer.java:337) at java.text.RuleBasedCollator.getCollationKey(RuleBasedCollator.java:617) at org.apache.lucene.collation.CollationKeyFilter.incrementToken(CollationKeyFilter.java:93) at org.apache.lucene.collation.CollationTestBase.assertThreadSafe(CollationTestBase.java:304) at org.apache.lucene.collation.TestCollationKeyAnalyzer.testThreadSafe(TestCollationKeyAnalyzer.java:89) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1076) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1008) Build Log (for compile errors): [...truncated 5558 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Test failure on 3x
Turns out this is not easily reproducible. While I am able to reproduce it
*many times* on my machine, it's not *always*. I did notice a strange
behavior -- when the test fails, the stack trace points to what seems to be
an incorrect line. I don't know why, even though I 'ant clean' and 'svn up'
etc.
Anyway, I ported some debug messages from trunk to 3x, and with that, I see
that the failure is on the line assertTrue(docCount lowerBound) in
checkInvariants. Below is the stacktrace (note that it points at an
incorrect line #222 which is not the assert that fails):
java.lang.AssertionError: docCount=10 lowerBound=100 i=7 segmentCount=11
index=_1e(3.2):Cv1081 _1z(3.2):Cv1064 _2k(3.2):Cv1425 _35(3.2):Cv1510
_3q(3.2):cv190 _64(3.2):cv550 _5f(3.2):cv10 _5g(3.2):cv10 _61(3.2):cv190
_62(3.2):cv10 _63(3.2):cv10
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.assertTrue(Assert.java:43)
at
org.apache.lucene.index.TestIndexWriterMergePolicy.checkInvariants(TestIndexWriterMergePolicy.java:222)
at
org.apache.lucene.index.TestIndexWriterMergePolicy.testMaxBufferedDocsChange(TestIndexWriterMergePolicy.java:168)
I believe that the fact it's not always reproducible is related to CMS. I
ran the test with -Dtests.iter=10 and it failed a couple of times with CMS,
but if I switch to SMS and run with -Dtests.iter=100 it succeeds.
The test includes this code:
{code}
writer.commit();
writer.waitForMerges();
writer.commit();
{code}
It's clear that the test wants to want for CMS to finish all its merges.
However, according to waitForMerges' javadocs: It is guaranteed that any
merges started *prior* to calling this method will have completed once this
method completes.
What if a merge starts after waitForMerges is called (due to cascaded
merges) -- will waitForMerges still wait for it? To check that I tried the
following:
1) I added another call to waitForMerges following the second commit(), but
that didn't help (out of 100 runs, I had 3 failures).
2) I add this call instead: ((ConcurrentMergeScheduler)
writer.getConfig().getMergeScheduler()).sync(); but it didn't help either (9
out of 100 failures)
3) Tried to close the writer and open it, still failures.
So I'm not sure that the problem is related to waitForMerges, however it
clearly has something to do with CMS, because all iterations pass with SMS.
Shai
On Mon, Mar 28, 2011 at 7:00 AM, Shai Erera ser...@gmail.com wrote:
I ran tests on 3x today and hit this:
[junit] Testsuite: org.apache.lucene.index.TestIndexWriterMergePolicy
[junit] Testcase:
testMaxBufferedDocsChange(org.apache.lucene.index.TestIndexWriterMergePolicy):
FAILED
[junit]
[junit] junit.framework.AssertionFailedError:
[junit] at
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1076)
[junit] at
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1008)
[junit] at
org.apache.lucene.index.TestIndexWriterMergePolicy.checkInvariants(TestIndexWriterMergePolicy.java:221)
[junit] at
org.apache.lucene.index.TestIndexWriterMergePolicy.testMaxBufferedDocsChange(TestIndexWriterMergePolicy.java:168)
[junit]
[junit]
[junit] Tests run: 6, Failures: 1, Errors: 0, Time elapsed: 2.777 sec
[junit]
[junit] - Standard Error -
[junit] NOTE: reproduce with: ant test
-Dtestcase=TestIndexWriterMergePolicy -Dtestmethod=testMaxBufferedDocsChange
-Dtests.seed=-4805406970232115944:2175520643927646734
[junit] NOTE: test params are: locale=el_CY_PREEURO,
timezone=America/Dominica
[junit] NOTE: all tests run in this JVM:
[junit] [TestCharArrayMap, TestLengthFilter, TestTeeSinkTokenFilter,
TestBinaryDocument, TestByteSlices, TestDocumentWriter,
TestIndexReaderClone, TestIndexWriterMergePolicy]
[junit] NOTE: Windows 7 6.1 build 7600 amd64/IBM Corporation 1.6.0
(64-bit)/cpus=2,threads=3,free=4734480,total=13416448
I was able to reproduce it with the 'reproduce' line only on 3x. Also
reproducible if it's the only test that's running. I can look at it in 3-4
hours from now, so am posting this to the list in case someone gets to it
before me.
Shai
Re: building Lucene from sources without Solr sources from svn ?
the real question is: why is this fixed only in the 3.1 branch? how did our 3.1 branch and 3.x/trunk get so different? I don't like that any work done to get 3.1 out the door is going to be lost and have to be re-done. On Sun, Mar 27, 2011 at 5:26 PM, Andi Vajda va...@osafoundation.org wrote: Hi, It seems that at this time, the HEAD of branch_3x can no longer be conveniently checked out with Lucene sources only. If I check out http://svn.apache.org/repos/asf/lucene/dev/branches/branch_3x/lucene I'm not getting the common-build.xml file from above that the Lucene one depends on. If I checkout branch_3x, I get Solr sources as well. Is this intentional or an oversight ? Andi.. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[HUDSON] Lucene-Solr-tests-only-trunk - Build # 6456 - Failure
Build: https://hudson.apache.org/hudson/job/Lucene-Solr-tests-only-trunk/6456/ 15 tests failed. REGRESSION: org.apache.lucene.index.TestIndexReaderReopen.testThreadSafety Error Message: org/apache/lucene/util/LuceneTestCase$TwoLongs Stack Trace: java.lang.NoClassDefFoundError: org/apache/lucene/util/LuceneTestCase$TwoLongs at org.apache.lucene.util.LuceneTestCase.reportAdditionalFailureInfo(LuceneTestCase.java:1112) at org.apache.lucene.util.LuceneTestCase$1.failed(LuceneTestCase.java:439) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1215) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1147) Caused by: java.lang.ClassNotFoundException: org.apache.lucene.util.LuceneTestCase$TwoLongs at java.net.URLClassLoader$1.run(URLClassLoader.java:214) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) Caused by: java.io.FileNotFoundException: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/lucene/build/classes/test-framework/org/apache/lucene/util/LuceneTestCase$TwoLongs.class (Too many open files in system) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.init(FileInputStream.java:137) at sun.misc.URLClassPath$FileLoader$1.getInputStream(URLClassPath.java:1005) at sun.misc.Resource.cachedInputStream(Resource.java:77) at sun.misc.Resource.getByteBuffer(Resource.java:160) at java.net.URLClassLoader.defineClass(URLClassLoader.java:266) at java.net.URLClassLoader.access$000(URLClassLoader.java:73) at java.net.URLClassLoader$1.run(URLClassLoader.java:212) REGRESSION: org.apache.lucene.index.TestIndexWriter.testBackgroundOptimize Error Message: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/lucene/build/test/7/test7562124135320648790tmp/_17_0.pyl (Too many open files in system) Stack Trace: java.io.FileNotFoundException: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/lucene/build/test/7/test7562124135320648790tmp/_17_0.pyl (Too many open files in system) at java.io.RandomAccessFile.open(Native Method) at java.io.RandomAccessFile.init(RandomAccessFile.java:233) at org.apache.lucene.store.FSDirectory$FSIndexOutput.init(FSDirectory.java:448) at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:312) at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:347) at org.apache.lucene.index.codecs.sep.SepPostingsWriterImpl.init(SepPostingsWriterImpl.java:122) at org.apache.lucene.index.codecs.mockintblock.MockVariableIntBlockCodec.fieldsConsumer(MockVariableIntBlockCodec.java:138) at org.apache.lucene.index.PerFieldCodecWrapper$FieldsWriter.init(PerFieldCodecWrapper.java:64) at org.apache.lucene.index.PerFieldCodecWrapper.fieldsConsumer(PerFieldCodecWrapper.java:54) at org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:78) at org.apache.lucene.index.TermsHash.flush(TermsHash.java:103) at org.apache.lucene.index.DocInverter.flush(DocInverter.java:65) at org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:55) at org.apache.lucene.index.DocumentsWriter.flush(DocumentsWriter.java:567) at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:2487) at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:2452) at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1211) at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1180) at org.apache.lucene.index.TestIndexWriter.testBackgroundOptimize(TestIndexWriter.java:1030) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1215) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1147) REGRESSION: org.apache.lucene.index.TestIndexWriter.testExpungeDeletes3 Error Message: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/lucene/build/test/7/test8030532957701118561tmp/_16_0.tib (Too many open files in system) Stack Trace: java.io.FileNotFoundException: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-trunk/checkout/lucene/build/test/7/test8030532957701118561tmp/_16_0.tib (Too many open files in system) at java.io.RandomAccessFile.open(Native Method) at java.io.RandomAccessFile.init(RandomAccessFile.java:233) at
[jira] [Updated] (LUCENE-2996) addIndexes(IndexReader) incorrectly applies existing deletes
[ https://issues.apache.org/jira/browse/LUCENE-2996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-2996: --- Attachment: LUCENE-2996.patch Patch adds a test to TestAddIndexes and the trivial fix to IndexWriter (against 3x). I will port to trunk and 3.1 branch after a review. addIndexes(IndexReader) incorrectly applies existing deletes Key: LUCENE-2996 URL: https://issues.apache.org/jira/browse/LUCENE-2996 Project: Lucene - Java Issue Type: Bug Components: Index Reporter: Shai Erera Assignee: Shai Erera Fix For: 3.1.1, 3.2, 4.0 Attachments: LUCENE-2996.patch If you perform these operations: # deleteDocuments(Term) for all the new documents # addIndexes(IndexReader) # commit Then addIndexes applies the previous deletes on the incoming indexes as well, which is incorrect. If you call addIndexes(Directory) instead, the deletes are applied beforehand, as they should. The solution, as Mike indicated here: http://osdir.com/ml/general/2011-03/msg20876.html is to add *flush(false,true)* to addIndexes(IndexReader). I will create a patch with a matching unit test shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Search in both database and documents
Please, as asked previously, use the users list for this type of question. This list is intended for developers working on the Lucene and Solr code. The correct address is: solr-u...@lucene.apache.org See: http://lucene.apache.org/solr/mailing_lists.html Best Erick On Mon, Mar 28, 2011 at 3:51 AM, Deepak Singh deep...@praumtech.com wrote: i just to do a search in both database and documents(pdf,doc etc). how to make schema for both database and documents fields. i m unable to do. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2996) addIndexes(IndexReader) incorrectly applies existing deletes
[ https://issues.apache.org/jira/browse/LUCENE-2996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012028#comment-13012028 ] Simon Willnauer commented on LUCENE-2996: - Shai, patch looks ok. shouldn't the test use newIndexWriterConfig(TEST_VERSION_CURRENT, new MockAnalyzer()) instead of new IndexWriterConfig(...) addIndexes(IndexReader) incorrectly applies existing deletes Key: LUCENE-2996 URL: https://issues.apache.org/jira/browse/LUCENE-2996 Project: Lucene - Java Issue Type: Bug Components: Index Reporter: Shai Erera Assignee: Shai Erera Fix For: 3.1.1, 3.2, 4.0 Attachments: LUCENE-2996.patch If you perform these operations: # deleteDocuments(Term) for all the new documents # addIndexes(IndexReader) # commit Then addIndexes applies the previous deletes on the incoming indexes as well, which is incorrect. If you call addIndexes(Directory) instead, the deletes are applied beforehand, as they should. The solution, as Mike indicated here: http://osdir.com/ml/general/2011-03/msg20876.html is to add *flush(false,true)* to addIndexes(IndexReader). I will create a patch with a matching unit test shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: building Lucene from sources without Solr sources from svn ?
I don't think it is lost, it's likely my mistake in adding a top level common-build for the validation stuff. We can change it out. On Mar 28, 2011, at 8:27 AM, Robert Muir wrote: the real question is: why is this fixed only in the 3.1 branch? how did our 3.1 branch and 3.x/trunk get so different? I don't like that any work done to get 3.1 out the door is going to be lost and have to be re-done. On Sun, Mar 27, 2011 at 5:26 PM, Andi Vajda va...@osafoundation.org wrote: Hi, It seems that at this time, the HEAD of branch_3x can no longer be conveniently checked out with Lucene sources only. If I check out http://svn.apache.org/repos/asf/lucene/dev/branches/branch_3x/lucene I'm not getting the common-build.xml file from above that the Lucene one depends on. If I checkout branch_3x, I get Solr sources as well. Is this intentional or an oversight ? Andi.. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-2996) addIndexes(IndexReader) incorrectly applies existing deletes
[ https://issues.apache.org/jira/browse/LUCENE-2996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shai Erera updated LUCENE-2996: --- Attachment: LUCENE-2996.patch bq. shouldn't the test use newIndexWriterConfig(TEST_VERSION_CURRENT, new MockAnalyzer()) instead of new IndexWriterConfig(...) Good catch ! it was a remnant from the time this test was a static main(). addIndexes(IndexReader) incorrectly applies existing deletes Key: LUCENE-2996 URL: https://issues.apache.org/jira/browse/LUCENE-2996 Project: Lucene - Java Issue Type: Bug Components: Index Reporter: Shai Erera Assignee: Shai Erera Fix For: 3.1.1, 3.2, 4.0 Attachments: LUCENE-2996.patch, LUCENE-2996.patch If you perform these operations: # deleteDocuments(Term) for all the new documents # addIndexes(IndexReader) # commit Then addIndexes applies the previous deletes on the incoming indexes as well, which is incorrect. If you call addIndexes(Directory) instead, the deletes are applied beforehand, as they should. The solution, as Mike indicated here: http://osdir.com/ml/general/2011-03/msg20876.html is to add *flush(false,true)* to addIndexes(IndexReader). I will create a patch with a matching unit test shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2436) move uimaConfig to under the uima's update processor in solrconfig.xml
[
https://issues.apache.org/jira/browse/SOLR-2436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Koji Sekiguchi updated SOLR-2436:
-
Attachment: SOLR-2436.patch
Hi Tommaso,
bq. However I think it should be good if it was possible to alternatively get
rid of the uimaConfig file defining each parameter inside the Processor with
Solr elements (str/lst/int etc.) as well.
I've done this in the attached patch. Please review it. I'm not confident about
the part:
{code}
lst name=fieldMappings
lst name=mapping
str name=typeorg.apache.uima.SentenceAnnotation/str
str name=featurecoveredText/str
str name=fieldsentence/str
/lst
/lst
{code}
the structure is appropriate or not.
move uimaConfig to under the uima's update processor in solrconfig.xml
--
Key: SOLR-2436
URL: https://issues.apache.org/jira/browse/SOLR-2436
Project: Solr
Issue Type: Improvement
Affects Versions: 3.1
Reporter: Koji Sekiguchi
Priority: Minor
Attachments: SOLR-2436.patch, SOLR-2436.patch, SOLR-2436.patch,
SOLR-2436_2.patch
Solr contrib UIMA has its config just beneath config. I think it should
move to uima's update processor tag.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-1144) replication hang
[ https://issues.apache.org/jira/browse/SOLR-1144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012047#comment-13012047 ] Vadim Kisselmann commented on SOLR-1144: I have Solr running on one master and two slaves (load balanced) via Solr 1.4.1 native replication. If the load is low, both slaves replicate with around 100MB/s from master. After a couple of hours the replication slows down to 100KB/s. So the problem is still there. I tested it with both Jetty and Tomcat. It looks like that aggressive JVM-Options can delay the problem, but then it starts anyway. My Index is about 100GB, i use 10GB for JVM, 24GB total. The slaves polls every 5 minutes. replication hang Key: SOLR-1144 URL: https://issues.apache.org/jira/browse/SOLR-1144 Project: Solr Issue Type: Bug Reporter: Yonik Seeley Assignee: Noble Paul Fix For: 1.4 Attachments: stacktrace-master.txt, stacktrace-slave-1.txt, stacktrace-slave-2.txt It seems that replication can sometimes hang. http://www.lucidimagination.com/search/document/403305a3fda18599 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[HUDSON] Lucene-Solr-tests-only-3.x - Build # 6446 - Failure
Build: https://hudson.apache.org/hudson/job/Lucene-Solr-tests-only-3.x/6446/ 1 tests failed. REGRESSION: org.apache.lucene.index.TestIndexWriterMergePolicy.testMaxBufferedDocsChange Error Message: docCount=10 lowerBound=10 i=5 segmentCount=11 index=_1e(3.2):Cv1081 _1z(3.2):Cv1064 _2k(3.2):Cv1425 _35(3.2):Cv1510 _62(3.2):Cv730 _5e(3.2):cv10 _5p(3.2):cv100 _60(3.2):cv100 _61(3.2):cv10 _63(3.2):cv10 _64(3.2):cv10 Stack Trace: junit.framework.AssertionFailedError: docCount=10 lowerBound=10 i=5 segmentCount=11 index=_1e(3.2):Cv1081 _1z(3.2):Cv1064 _2k(3.2):Cv1425 _35(3.2):Cv1510 _62(3.2):Cv730 _5e(3.2):cv10 _5p(3.2):cv100 _60(3.2):cv100 _61(3.2):cv10 _63(3.2):cv10 _64(3.2):cv10 at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1076) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1008) at org.apache.lucene.index.TestIndexWriterMergePolicy.checkInvariants(TestIndexWriterMergePolicy.java:236) at org.apache.lucene.index.TestIndexWriterMergePolicy.testMaxBufferedDocsChange(TestIndexWriterMergePolicy.java:168) Build Log (for compile errors): [...truncated 5567 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2573) Tiered flushing of DWPTs by RAM with low/high water marks
[
https://issues.apache.org/jira/browse/LUCENE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012069#comment-13012069
]
Simon Willnauer commented on LUCENE-2573:
-
bq. How come we lost 'assert !bufferedDeletesStream.any();' inIndexWriter.java?
so this is tricky here. Since we are flushing concurrently this could false
fail. The same assertion is in bufferedDeletesStream.prune(segmentInfos); which
is synced. But another thread could sneak in between the prune and the any()
check updating / deleting a document this could false fail. Or do I miss
something here?
bq. Maybe, for stalling, instead of triggering by max RAM, we can take
this simple approach: if the number of flushing DWPTs ever exceeds one
plus number of active DWPTs, then we stall (and resume once it's below
again).
Awesome idea mike! I will do that!
{quote}
We should fix DefaultFlushPolicy to first pull the relevant config
from IWC (eg maxBufferedDocs), then check if that config is -1 or
not, etc., because IWC's config can be changed at any time (live)
so we may read eg 1 at first and then -1 the second time.
{quote}
What you mean is we should check if we flush by Ram, DocCount etc. and only if
so we check the live values for Ram, DocCount etc.?
{quote}
Hmm deletes are actually tricky, because somehow the FlushPolicy
needs access to the global deletes count (and also the to per-DWPT
deletes count). If a given DWPT has 0 buffered docs, then indeed the
buffered deletes in its pool doesn't matter. But, we do need to respect
the buffered deletes in the global pool...
{quote}
I think it does not make sense to check both the global count and the DWPT
count against the same value. If we have a DWPT that exceeds it we also exceed
globally or could it happen that a DWPT has more deletes than the global pool?
Further if we observe the global pool and we exceed the limit do we flush all
as written on the IWC documentation?
once we sort this out I upload a new patch with javadoc etc for flush policy.
we seem to be close here man!
Tiered flushing of DWPTs by RAM with low/high water marks
-
Key: LUCENE-2573
URL: https://issues.apache.org/jira/browse/LUCENE-2573
Project: Lucene - Java
Issue Type: Improvement
Components: Index
Reporter: Michael Busch
Assignee: Simon Willnauer
Priority: Minor
Fix For: Realtime Branch
Attachments: LUCENE-2573.patch, LUCENE-2573.patch, LUCENE-2573.patch,
LUCENE-2573.patch, LUCENE-2573.patch, LUCENE-2573.patch, LUCENE-2573.patch,
LUCENE-2573.patch
Now that we have DocumentsWriterPerThreads we need to track total consumed
RAM across all DWPTs.
A flushing strategy idea that was discussed in LUCENE-2324 was to use a
tiered approach:
- Flush the first DWPT at a low water mark (e.g. at 90% of allowed RAM)
- Flush all DWPTs at a high water mark (e.g. at 110%)
- Use linear steps in between high and low watermark: E.g. when 5 DWPTs are
used, flush at 90%, 95%, 100%, 105% and 110%.
Should we allow the user to configure the low and high water mark values
explicitly using total values (e.g. low water mark at 120MB, high water mark
at 140MB)? Or shall we keep for simplicity the single setRAMBufferSizeMB()
config method and use something like 90% and 110% for the water marks?
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Test failure on 3x, TestLocaleMethods
My machine is hot at picking seeds today :). [junit] [junit] junit.framework.AssertionFailedError: [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1076) [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1008) [junit] at org.apache.lucene.search.TestLocaleMethods.testSort(TestLocaleMethods.java:84) [junit] [junit] [junit] Testcase: testSort2(org.apache.lucene.search.TestLocaleMethods):FAILED [junit] [junit] junit.framework.AssertionFailedError: [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1076) [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1008) [junit] at org.apache.lucene.search.TestLocaleMethods.testSort2(TestLocaleMethods.java:100) [junit] [junit] [junit] Tests run: 4, Failures: 2, Errors: 0, Time elapsed: 2.03 sec [junit] [junit] - Standard Error - [junit] NOTE: reproduce with: ant test -Dtestcase=TestLocaleMethods -Dtestmethod=testSort -Dtests.seed=-6178492448073106006:6179706939192629911 [junit] NOTE: reproduce with: ant test -Dtestcase=TestLocaleMethods -Dtestmethod=testSort2 -Dtests.seed=-6178492448073106006:7745709510566497976 [junit] NOTE: test params are: locale=tr_TR, timezone=GMT0 [junit] NOTE: all tests run in this JVM: [junit] [TestSearch, TestAnalyzers, TestISOLatin1AccentFilter, TestStandardAnalyzer, TestTermAttributeImpl, TestAddIndexes, TestDeletionPolicy, TestIndexFileDeleter, TestIndexWriterDelete, TestIndexWriterWithThreads, TestNoDeletionPolicy, TestParallelReaderEmptyIndex, TestSegmentInfo, TestStressIndexing, TestTransactions, TestBoolean2, TestComplexExplanationsOfNonMatches, TestElevationComparator, TestLocaleMethods] [junit] NOTE: Windows 7 6.1 build 7600 amd64/IBM Corporation 1.6.0 (64-bit)/cpus=2,threads=4,free=27719824,total=53872640 [junit] - --- The first repro line fails both testSort and testSort2. It fails w/ an IBM JRE, but passes w/ Sun's. The default locale picked by the test is tr_TR. Will investigate, though the test itself looks 'safe', so it must be something behind the scenes that uses the default locale on us. Shai
RE: building Lucene from sources without Solr sources from svn ?
Hi, If we we have to rebuild the artifacts, should we add Shai/Mike's addIndexes() fix, too? Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Grant Ingersoll [mailto:gsing...@apache.org] Sent: Monday, March 28, 2011 2:51 PM To: dev@lucene.apache.org Cc: Andi Vajda Subject: Re: building Lucene from sources without Solr sources from svn ? I don't think it is lost, it's likely my mistake in adding a top level common-build for the validation stuff. We can change it out. On Mar 28, 2011, at 8:27 AM, Robert Muir wrote: the real question is: why is this fixed only in the 3.1 branch? how did our 3.1 branch and 3.x/trunk get so different? I don't like that any work done to get 3.1 out the door is going to be lost and have to be re-done. On Sun, Mar 27, 2011 at 5:26 PM, Andi Vajda va...@osafoundation.org wrote: Hi, It seems that at this time, the HEAD of branch_3x can no longer be conveniently checked out with Lucene sources only. If I check out http://svn.apache.org/repos/asf/lucene/dev/branches/branch_3x/lucene I'm not getting the common-build.xml file from above that the Lucene one depends on. If I checkout branch_3x, I get Solr sources as well. Is this intentional or an oversight ? Andi.. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2445) unknown handler: standard
[ https://issues.apache.org/jira/browse/SOLR-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Sekiguchi updated SOLR-2445: - Fix Version/s: 4.0 3.2 Assignee: Koji Sekiguchi Will commit soon. unknown handler: standard - Key: SOLR-2445 URL: https://issues.apache.org/jira/browse/SOLR-2445 Project: Solr Issue Type: Bug Affects Versions: 1.4.1, 3.1, 3.2, 4.0 Reporter: Koji Sekiguchi Assignee: Koji Sekiguchi Priority: Minor Fix For: 3.2, 4.0 Attachments: SOLR-2445.patch To reproduce the problem using example config, go form.jsp, use standard for qt (it is default) then click Search. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: building Lucene from sources without Solr sources from svn ?
On Mon, Mar 28, 2011 at 10:54 AM, Uwe Schindler u...@thetaphi.de wrote: Hi, If we we have to rebuild the artifacts, should we add Shai/Mike's addIndexes() fix, too? 3.1 branch is fine with regards to this issue, thats why I raised my question... it seems only the 3.1 release branch was fixed for this but trunk and branch_3x are broken. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Test failure on 3x, TestLocaleMethods
FYI this is a relatively new test, and I think its a bug in IBM's jre. I found similar problems in ICU and i know IBM JRE's intl impl is different than sun's (maybe a newer icu version?) I worked around these in the icu collation tests, and we might have to do the same here... we can't use totally random unicode strings or we might hit some bugs: http://bugs.icu-project.org/trac/ticket/8060 http://bugs.icu-project.org/trac/ticket/7732 On Mon, Mar 28, 2011 at 10:46 AM, Shai Erera ser...@gmail.com wrote: My machine is hot at picking seeds today :). [junit] [junit] junit.framework.AssertionFailedError: [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1076) [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1008) [junit] at org.apache.lucene.search.TestLocaleMethods.testSort(TestLocaleMethods.java:84) [junit] [junit] [junit] Testcase: testSort2(org.apache.lucene.search.TestLocaleMethods): FAILED [junit] [junit] junit.framework.AssertionFailedError: [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1076) [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1008) [junit] at org.apache.lucene.search.TestLocaleMethods.testSort2(TestLocaleMethods.java:100) [junit] [junit] [junit] Tests run: 4, Failures: 2, Errors: 0, Time elapsed: 2.03 sec [junit] [junit] - Standard Error - [junit] NOTE: reproduce with: ant test -Dtestcase=TestLocaleMethods -Dtestmethod=testSort -Dtests.seed=-6178492448073106006:6179706939192629911 [junit] NOTE: reproduce with: ant test -Dtestcase=TestLocaleMethods -Dtestmethod=testSort2 -Dtests.seed=-6178492448073106006:7745709510566497976 [junit] NOTE: test params are: locale=tr_TR, timezone=GMT0 [junit] NOTE: all tests run in this JVM: [junit] [TestSearch, TestAnalyzers, TestISOLatin1AccentFilter, TestStandardAnalyzer, TestTermAttributeImpl, TestAddIndexes, TestDeletionPolicy, TestIndexFileDeleter, TestIndexWriterDelete, TestIndexWriterWithThreads, TestNoDeletionPolicy, TestParallelReaderEmptyIndex, TestSegmentInfo, TestStressIndexing, TestTransactions, TestBoolean2, TestComplexExplanationsOfNonMatches, TestElevationComparator, TestLocaleMethods] [junit] NOTE: Windows 7 6.1 build 7600 amd64/IBM Corporation 1.6.0 (64-bit)/cpus=2,threads=4,free=27719824,total=53872640 [junit] - --- The first repro line fails both testSort and testSort2. It fails w/ an IBM JRE, but passes w/ Sun's. The default locale picked by the test is tr_TR. Will investigate, though the test itself looks 'safe', so it must be something behind the scenes that uses the default locale on us. Shai - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: building Lucene from sources without Solr sources from svn ?
If you're talking about LUCENE-2996, then note that I haven't checked in the code yet. If you're going to rebuild the artifacts off of branches/lucene_solr_3_1, I can check in the code there now. Shai On Mon, Mar 28, 2011 at 5:04 PM, Robert Muir rcm...@gmail.com wrote: On Mon, Mar 28, 2011 at 10:54 AM, Uwe Schindler u...@thetaphi.de wrote: Hi, If we we have to rebuild the artifacts, should we add Shai/Mike's addIndexes() fix, too? 3.1 branch is fine with regards to this issue, thats why I raised my question... it seems only the 3.1 release branch was fixed for this but trunk and branch_3x are broken. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: building Lucene from sources without Solr sources from svn ?
OK, sorry, I misunderstood the thread. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Robert Muir [mailto:rcm...@gmail.com] Sent: Monday, March 28, 2011 5:05 PM To: dev@lucene.apache.org Subject: Re: building Lucene from sources without Solr sources from svn ? On Mon, Mar 28, 2011 at 10:54 AM, Uwe Schindler u...@thetaphi.de wrote: Hi, If we we have to rebuild the artifacts, should we add Shai/Mike's addIndexes() fix, too? 3.1 branch is fine with regards to this issue, thats why I raised my question... it seems only the 3.1 release branch was fixed for this but trunk and branch_3x are broken. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (SOLR-2445) unknown handler: standard
[ https://issues.apache.org/jira/browse/SOLR-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Sekiguchi resolved SOLR-2445. -- Resolution: Fixed trunk: Committed revision 1086276. 3x: Committed revision 1086289. unknown handler: standard - Key: SOLR-2445 URL: https://issues.apache.org/jira/browse/SOLR-2445 Project: Solr Issue Type: Bug Affects Versions: 1.4.1, 3.1, 3.2, 4.0 Reporter: Koji Sekiguchi Assignee: Koji Sekiguchi Priority: Minor Fix For: 3.2, 4.0 Attachments: SOLR-2445.patch To reproduce the problem using example config, go form.jsp, use standard for qt (it is default) then click Search. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2996) addIndexes(IndexReader) incorrectly applies existing deletes
[ https://issues.apache.org/jira/browse/LUCENE-2996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012116#comment-13012116 ] Shai Erera commented on LUCENE-2996: Committed revision 1086275 (3x). Committed revision 1086288 (trunk). I don't know if it's ok to commit to branches/lucene_solr_3_1 just yet, so I'll keep this open for now. addIndexes(IndexReader) incorrectly applies existing deletes Key: LUCENE-2996 URL: https://issues.apache.org/jira/browse/LUCENE-2996 Project: Lucene - Java Issue Type: Bug Components: Index Reporter: Shai Erera Assignee: Shai Erera Fix For: 3.1.1, 3.2, 4.0 Attachments: LUCENE-2996.patch, LUCENE-2996.patch If you perform these operations: # deleteDocuments(Term) for all the new documents # addIndexes(IndexReader) # commit Then addIndexes applies the previous deletes on the incoming indexes as well, which is incorrect. If you call addIndexes(Directory) instead, the deletes are applied beforehand, as they should. The solution, as Mike indicated here: http://osdir.com/ml/general/2011-03/msg20876.html is to add *flush(false,true)* to addIndexes(IndexReader). I will create a patch with a matching unit test shortly. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Test failure on 3x
Shai,
This has also been failing on trunk, but I can't repro it, so it's
awesome you can!
Can you turn on IW's infoStream and get the failure to happen and post
the results?
I agree this is likely a CMS thing...
Mike
http://blog.mikemccandless.com
On Mon, Mar 28, 2011 at 8:15 AM, Shai Erera ser...@gmail.com wrote:
Turns out this is not easily reproducible. While I am able to reproduce it
*many times* on my machine, it's not *always*. I did notice a strange
behavior -- when the test fails, the stack trace points to what seems to be
an incorrect line. I don't know why, even though I 'ant clean' and 'svn up'
etc.
Anyway, I ported some debug messages from trunk to 3x, and with that, I see
that the failure is on the line assertTrue(docCount lowerBound) in
checkInvariants. Below is the stacktrace (note that it points at an
incorrect line #222 which is not the assert that fails):
java.lang.AssertionError: docCount=10 lowerBound=100 i=7 segmentCount=11
index=_1e(3.2):Cv1081 _1z(3.2):Cv1064 _2k(3.2):Cv1425 _35(3.2):Cv1510
_3q(3.2):cv190 _64(3.2):cv550 _5f(3.2):cv10 _5g(3.2):cv10 _61(3.2):cv190
_62(3.2):cv10 _63(3.2):cv10
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.assertTrue(Assert.java:43)
at
org.apache.lucene.index.TestIndexWriterMergePolicy.checkInvariants(TestIndexWriterMergePolicy.java:222)
at
org.apache.lucene.index.TestIndexWriterMergePolicy.testMaxBufferedDocsChange(TestIndexWriterMergePolicy.java:168)
I believe that the fact it's not always reproducible is related to CMS. I
ran the test with -Dtests.iter=10 and it failed a couple of times with CMS,
but if I switch to SMS and run with -Dtests.iter=100 it succeeds.
The test includes this code:
{code}
writer.commit();
writer.waitForMerges();
writer.commit();
{code}
It's clear that the test wants to want for CMS to finish all its merges.
However, according to waitForMerges' javadocs: It is guaranteed that any
merges started *prior* to calling this method will have completed once this
method completes.
What if a merge starts after waitForMerges is called (due to cascaded
merges) -- will waitForMerges still wait for it? To check that I tried the
following:
1) I added another call to waitForMerges following the second commit(), but
that didn't help (out of 100 runs, I had 3 failures).
2) I add this call instead: ((ConcurrentMergeScheduler)
writer.getConfig().getMergeScheduler()).sync(); but it didn't help either (9
out of 100 failures)
3) Tried to close the writer and open it, still failures.
So I'm not sure that the problem is related to waitForMerges, however it
clearly has something to do with CMS, because all iterations pass with SMS.
Shai
On Mon, Mar 28, 2011 at 7:00 AM, Shai Erera ser...@gmail.com wrote:
I ran tests on 3x today and hit this:
[junit] Testsuite: org.apache.lucene.index.TestIndexWriterMergePolicy
[junit] Testcase:
testMaxBufferedDocsChange(org.apache.lucene.index.TestIndexWriterMergePolicy):
FAILED
[junit]
[junit] junit.framework.AssertionFailedError:
[junit] at
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1076)
[junit] at
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1008)
[junit] at
org.apache.lucene.index.TestIndexWriterMergePolicy.checkInvariants(TestIndexWriterMergePolicy.java:221)
[junit] at
org.apache.lucene.index.TestIndexWriterMergePolicy.testMaxBufferedDocsChange(TestIndexWriterMergePolicy.java:168)
[junit]
[junit]
[junit] Tests run: 6, Failures: 1, Errors: 0, Time elapsed: 2.777 sec
[junit]
[junit] - Standard Error -
[junit] NOTE: reproduce with: ant test
-Dtestcase=TestIndexWriterMergePolicy -Dtestmethod=testMaxBufferedDocsChange
-Dtests.seed=-4805406970232115944:2175520643927646734
[junit] NOTE: test params are: locale=el_CY_PREEURO,
timezone=America/Dominica
[junit] NOTE: all tests run in this JVM:
[junit] [TestCharArrayMap, TestLengthFilter, TestTeeSinkTokenFilter,
TestBinaryDocument, TestByteSlices, TestDocumentWriter,
TestIndexReaderClone, TestIndexWriterMergePolicy]
[junit] NOTE: Windows 7 6.1 build 7600 amd64/IBM Corporation 1.6.0
(64-bit)/cpus=2,threads=3,free=4734480,total=13416448
I was able to reproduce it with the 'reproduce' line only on 3x. Also
reproducible if it's the only test that's running. I can look at it in 3-4
hours from now, so am posting this to the list in case someone gets to it
before me.
Shai
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Test failure on 3x, TestLocaleMethods
Update: I switched to the newest IBM JRE I have available, but the test still fails with that seed. Shai On Mon, Mar 28, 2011 at 5:20 PM, Robert Muir rcm...@gmail.com wrote: FYI this is a relatively new test, and I think its a bug in IBM's jre. I found similar problems in ICU and i know IBM JRE's intl impl is different than sun's (maybe a newer icu version?) I worked around these in the icu collation tests, and we might have to do the same here... we can't use totally random unicode strings or we might hit some bugs: http://bugs.icu-project.org/trac/ticket/8060 http://bugs.icu-project.org/trac/ticket/7732 On Mon, Mar 28, 2011 at 10:46 AM, Shai Erera ser...@gmail.com wrote: My machine is hot at picking seeds today :). [junit] [junit] junit.framework.AssertionFailedError: [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1076) [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1008) [junit] at org.apache.lucene.search.TestLocaleMethods.testSort(TestLocaleMethods.java:84) [junit] [junit] [junit] Testcase: testSort2(org.apache.lucene.search.TestLocaleMethods):FAILED [junit] [junit] junit.framework.AssertionFailedError: [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1076) [junit] at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1008) [junit] at org.apache.lucene.search.TestLocaleMethods.testSort2(TestLocaleMethods.java:100) [junit] [junit] [junit] Tests run: 4, Failures: 2, Errors: 0, Time elapsed: 2.03 sec [junit] [junit] - Standard Error - [junit] NOTE: reproduce with: ant test -Dtestcase=TestLocaleMethods -Dtestmethod=testSort -Dtests.seed=-6178492448073106006:6179706939192629911 [junit] NOTE: reproduce with: ant test -Dtestcase=TestLocaleMethods -Dtestmethod=testSort2 -Dtests.seed=-6178492448073106006:7745709510566497976 [junit] NOTE: test params are: locale=tr_TR, timezone=GMT0 [junit] NOTE: all tests run in this JVM: [junit] [TestSearch, TestAnalyzers, TestISOLatin1AccentFilter, TestStandardAnalyzer, TestTermAttributeImpl, TestAddIndexes, TestDeletionPolicy, TestIndexFileDeleter, TestIndexWriterDelete, TestIndexWriterWithThreads, TestNoDeletionPolicy, TestParallelReaderEmptyIndex, TestSegmentInfo, TestStressIndexing, TestTransactions, TestBoolean2, TestComplexExplanationsOfNonMatches, TestElevationComparator, TestLocaleMethods] [junit] NOTE: Windows 7 6.1 build 7600 amd64/IBM Corporation 1.6.0 (64-bit)/cpus=2,threads=4,free=27719824,total=53872640 [junit] - --- The first repro line fails both testSort and testSort2. It fails w/ an IBM JRE, but passes w/ Sun's. The default locale picked by the test is tr_TR. Will investigate, though the test itself looks 'safe', so it must be something behind the scenes that uses the default locale on us. Shai - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: building Lucene from sources without Solr sources from svn ?
I would rather not rebuild for L-2996. This issue has a known workaround. As for the sources issue that Andi brought up, it never effected 3.1 b/c it doesn't have the validation stuff. I'd like to stick w/ the artifacts we have. On Mar 28, 2011, at 11:33 AM, Shai Erera wrote: If you're talking about LUCENE-2996, then note that I haven't checked in the code yet. If you're going to rebuild the artifacts off of branches/lucene_solr_3_1, I can check in the code there now. Shai On Mon, Mar 28, 2011 at 5:04 PM, Robert Muir rcm...@gmail.com wrote: On Mon, Mar 28, 2011 at 10:54 AM, Uwe Schindler u...@thetaphi.de wrote: Hi, If we we have to rebuild the artifacts, should we add Shai/Mike's addIndexes() fix, too? 3.1 branch is fine with regards to this issue, thats why I raised my question... it seems only the 3.1 release branch was fixed for this but trunk and branch_3x are broken. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[HUDSON] Lucene-Solr-tests-only-3.x - Build # 6454 - Failure
Build: https://hudson.apache.org/hudson/job/Lucene-Solr-tests-only-3.x/6454/ 1 tests failed. REGRESSION: org.apache.lucene.collation.TestCollationKeyAnalyzer.testThreadSafe Error Message: Java heap space Stack Trace: java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2894) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:117) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:589) at java.lang.StringBuffer.append(StringBuffer.java:337) at java.text.RuleBasedCollator.getCollationKey(RuleBasedCollator.java:617) at org.apache.lucene.collation.CollationKeyFilter.incrementToken(CollationKeyFilter.java:93) at org.apache.lucene.collation.CollationTestBase.assertThreadSafe(CollationTestBase.java:304) at org.apache.lucene.collation.TestCollationKeyAnalyzer.testThreadSafe(TestCollationKeyAnalyzer.java:89) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1076) at org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1008) Build Log (for compile errors): [...truncated 5563 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [VOTE] Lucene 3.1.0 RC3
My comments, mostly minor: apache-solr-3.1.0.zip: * README.TXT - Mixes Unix- and Windows-style syntaxes without further comments Propose to change e.g. %JAVA_HOME%\bin - $JAVA_HOME/bin to be consistent and add a comment about Windows * CHANGES.TXT - The headline Upgrading from Solr 1.4 should be Upgrading from Solr 1.4.1? * docs/who.html is old - refers to the old Solr committers list at http://lucene.apache.org/solr/who * docs/features.html - Does not list CSV among response writers * CHANGES.TXT - No mention of SOLR-1889 (dismax respecting q.op) - I thought this was backported in 3.1? apache-solr-3.1.0-src.tgz: * After building, http://localhost:8983/solr/admin/registry.jsp shows Solr Implementation Version: 3.1-SNAPSHOT Is that correct, or should it be 3.1.0-SNAPSHOT or something else? Last comment: I know we've intentionally bundled slf4j-jdk14-1.5.5.jar in the war to help newbies get up and running. But I find myself re-packaging the war for every customer when adapting to their choice of logger framework, which is counter-productive. Would it not be sufficient to have the jdk-logging binding in example/lib to let the example and tutorial still work OOTB but as soon as you deploy solr.war to production you're forced to explicitly decide what logging to use. -- Jan Høydahl, search solution architect Cominvent AS - www.cominvent.com On 27. mars 2011, at 04.07, Grant Ingersoll wrote: Artifacts are at http://people.apache.org/~gsingers/staging_area/rc3/. Please vote as you see appropriate. Vote closes on March 29th. I've also updated the Release To Do for both Lucene and Solr and it is hopefully a lot easier now to produce the artifacts as more of it is automated (including uploading to staging area). - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [VOTE] Lucene 3.1.0 RC3
Last comment: I know we've intentionally bundled slf4j-jdk14-1.5.5.jar in the war to help newbies get up and running. But I find myself re-packaging the war for every customer when adapting to their choice of logger framework, which is counter-productive. Would it not be sufficient to have the jdk-logging binding in example/lib to let the example and tutorial still work OOTB but as soon as you deploy solr.war to production you're forced to explicitly decide what logging to use. I agree, but don't think we should change it for 3.1 How about we make a different issue for that? ryan - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (LUCENE-2997) PayloadQueryParser addition
PayloadQueryParser addition --- Key: LUCENE-2997 URL: https://issues.apache.org/jira/browse/LUCENE-2997 Project: Lucene - Java Issue Type: New Feature Components: contrib/*, QueryParser Affects Versions: 3.0.1 Environment: n/a Reporter: Edward Drapkin Fix For: 3.0.4 I recently needed to deploy payloads for my search system and ran into a small wall: there was no query parser available for use with Payloads. I through this one together, extending out of the new modular QueryParser structure. Attached is the class file. I didn't know what package this would belong in (whether with the query parser or with the rest of the payload functionality in contrib/analyzers), so it's in the default package for now. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-2997) PayloadQueryParser addition
[ https://issues.apache.org/jira/browse/LUCENE-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Drapkin updated LUCENE-2997: --- Attachment: PayloadQueryParser.java PayloadQueryParser implementation. PayloadQueryParser addition --- Key: LUCENE-2997 URL: https://issues.apache.org/jira/browse/LUCENE-2997 Project: Lucene - Java Issue Type: New Feature Components: contrib/*, QueryParser Affects Versions: 3.0.1 Environment: n/a Reporter: Edward Drapkin Labels: features, patch Fix For: 3.0.4 Attachments: PayloadQueryParser.java Original Estimate: 0h Remaining Estimate: 0h I recently needed to deploy payloads for my search system and ran into a small wall: there was no query parser available for use with Payloads. I through this one together, extending out of the new modular QueryParser structure. Attached is the class file. I didn't know what package this would belong in (whether with the query parser or with the rest of the payload functionality in contrib/analyzers), so it's in the default package for now. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-2997) PayloadQueryParser addition
[ https://issues.apache.org/jira/browse/LUCENE-2997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Edward Drapkin updated LUCENE-2997: --- Description: I recently needed to deploy payloads for my search system and ran into a small wall: there was no query parser available for use with Payloads. I through this one together, extending out of the new modular QueryParser structure. Attached is the class file. I didn't know what package this would belong in (whether with the query parser or with the rest of the payload functionality in contrib/analyzers), so it's in the default package for now. I know this is a little, simple thing, but it seemed like something that should probably be included. was: I recently needed to deploy payloads for my search system and ran into a small wall: there was no query parser available for use with Payloads. I through this one together, extending out of the new modular QueryParser structure. Attached is the class file. I didn't know what package this would belong in (whether with the query parser or with the rest of the payload functionality in contrib/analyzers), so it's in the default package for now. PayloadQueryParser addition --- Key: LUCENE-2997 URL: https://issues.apache.org/jira/browse/LUCENE-2997 Project: Lucene - Java Issue Type: New Feature Components: contrib/*, QueryParser Affects Versions: 3.0.1 Environment: n/a Reporter: Edward Drapkin Labels: features, patch Fix For: 3.0.4 Attachments: PayloadQueryParser.java Original Estimate: 0h Remaining Estimate: 0h I recently needed to deploy payloads for my search system and ran into a small wall: there was no query parser available for use with Payloads. I through this one together, extending out of the new modular QueryParser structure. Attached is the class file. I didn't know what package this would belong in (whether with the query parser or with the rest of the payload functionality in contrib/analyzers), so it's in the default package for now. I know this is a little, simple thing, but it seemed like something that should probably be included. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: add an afterFilter to IndexSearcher.search()
: Optimizing cases where filters might be more expensive than the main query ;-) : But you must have a real use case... that inspired this idea? Where : are apps/Solr typically using such expensive filters? i have no idea what usecase prompted yonik to start thinking about this, but the first example that came up in my mind is the semi-ubiquitous 'ACL' question that people always want to implement using a completley external authorization system. If you have no choice but to query an remote system to find out if a document should be included in a particular result set, odds are you don't want to pay the price of asking it only to find out afterwards that it doesn't matter anyway, because it doesn't metch a required query clause. -Hoss - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Reopened] (SOLR-2445) unknown handler: standard
[ https://issues.apache.org/jira/browse/SOLR-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hoss Man reopened SOLR-2445: wait a minute ... there's no requirement that a handler named standard exist, nor is there any reason why solr should create a handler with that name automagically for you. (we already create one anonymous handler instance) All the docs have ever said is that *if* you declare a handler named standard then it will be used if no qt is specified, and no handler is marked default=true http://wiki.apache.org/solr/SolrRequestHandler I deliberately removed the example usage of name=standard from the example solrconfig.xml to simplify the confusion that people have about the way name=standard and default=true interact. If the problem here is that form.jsp defaults to listing standard for the qt param, then the fix is to change form.jsp and leave that box blank. unknown handler: standard - Key: SOLR-2445 URL: https://issues.apache.org/jira/browse/SOLR-2445 Project: Solr Issue Type: Bug Affects Versions: 1.4.1, 3.1, 3.2, 4.0 Reporter: Koji Sekiguchi Assignee: Koji Sekiguchi Priority: Minor Fix For: 3.2, 4.0 Attachments: SOLR-2445.patch To reproduce the problem using example config, go form.jsp, use standard for qt (it is default) then click Search. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2445) unknown handler: standard
[ https://issues.apache.org/jira/browse/SOLR-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012290#comment-13012290 ] Koji Sekiguchi commented on SOLR-2445: -- bq. If the problem here is that form.jsp defaults to listing standard for the qt param, then the fix is to change form.jsp and leave that box blank. No problem for me to fix form.jsp. unknown handler: standard - Key: SOLR-2445 URL: https://issues.apache.org/jira/browse/SOLR-2445 Project: Solr Issue Type: Bug Affects Versions: 1.4.1, 3.1, 3.2, 4.0 Reporter: Koji Sekiguchi Assignee: Koji Sekiguchi Priority: Minor Fix For: 3.2, 4.0 Attachments: SOLR-2445.patch To reproduce the problem using example config, go form.jsp, use standard for qt (it is default) then click Search. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2445) unknown handler: standard
[
https://issues.apache.org/jira/browse/SOLR-2445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012298#comment-13012298
]
Koji Sekiguchi commented on SOLR-2445:
--
bq. there's no requirement that a handler named standard exist, nor is there
any reason why solr should create a handler with that name automagically for
you.
I didn't think so, because I saw the standard name that is defined public
static final:
{code:title=RequestHandlers.java}
public static final String DEFAULT_HANDLER_NAME=standard;
{code}
unknown handler: standard
-
Key: SOLR-2445
URL: https://issues.apache.org/jira/browse/SOLR-2445
Project: Solr
Issue Type: Bug
Affects Versions: 1.4.1, 3.1, 3.2, 4.0
Reporter: Koji Sekiguchi
Assignee: Koji Sekiguchi
Priority: Minor
Fix For: 3.2, 4.0
Attachments: SOLR-2445.patch
To reproduce the problem using example config, go form.jsp, use standard for
qt (it is default) then click Search.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-2417) Allow explain info directly to response documents
: On IRC, yonik suggested that the explain format should mimic follow what : the debugQuery parameter would use. : : I'm don't really agree -- long term I would even suggest dropping the : explain section from debug and letting you specify it as an inline : parameter. those seem like orthoginal issues: 1) (ryan) deprecate/remove the explain section from debug and tell people to use the _explain_ psuedo field instead 2) (ryan) add options to the _explain_ psuedofield to let you pick an explanation style inline (ie: _explain:nl_) 3) (yonik) make the _explain_ psuedofield respect the debug.explain.structured param (or something like it) #1 #2 don't preclude #3 ... if i always want to get the nl mode explanations, it would be nice to be able to hardcode something a param the defaults section for my handler so that adding the _explain_ psuedofield just made it happen. -Hoss - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-2417) Allow explain info directly to response documents
On Mon, Mar 28, 2011 at 9:36 PM, Chris Hostetter hossman_luc...@fucit.org wrote: : On IRC, yonik suggested that the explain format should mimic follow what : the debugQuery parameter would use. : : I'm don't really agree -- long term I would even suggest dropping the : explain section from debug and letting you specify it as an inline : parameter. those seem like orthoginal issues: 1) (ryan) deprecate/remove the explain section from debug and tell people to use the _explain_ psuedo field instead 2) (ryan) add options to the _explain_ psuedofield to let you pick an explanation style inline (ie: _explain:nl_) 3) (yonik) make the _explain_ psuedofield respect the debug.explain.structured param (or something like it) #1 #2 don't preclude #3 ... if i always want to get the nl mode explanations, it would be nice to be able to hardcode something a param the defaults section for my handler so that adding the _explain_ psuedofield just made it happen. As is, it is not defined by a param, but via solrconfig: transformer name=explain class=org.apache.solr.response.transform.ExplainAugmenterFactory str name=argsnl/str /transformer As is, Transformers do not have access to SolrParams -- i think that is OK, but we should discuss TransformerFactory could easily be changed to take SolrParams or SolrQueryRequest. ryan - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2444) support wildcards in fl parameter, improve DocTransformer parsing
[ https://issues.apache.org/jira/browse/SOLR-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012317#comment-13012317 ] Yonik Seeley commented on SOLR-2444: It's not even clear to me why invoking a transformer would even be part of fl. fl=name,_docid_ makes sense because the user is asking for the _docid_ field back - the fact that it's a transformer is an implementation detail. If one wants to add a transformer that can do *anything* to a document, it feels like that should be specified elsewhere, and not in the field list? support wildcards in fl parameter, improve DocTransformer parsing - Key: SOLR-2444 URL: https://issues.apache.org/jira/browse/SOLR-2444 Project: Solr Issue Type: New Feature Reporter: Ryan McKinley Attachments: SOLR-2444-fl-parsing.patch, SOLR-2444-fl-parsing.patch The ReturnFields parsing needs to be improved. It should also support wildcards -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2444) support wildcards in fl parameter, improve DocTransformer parsing
[ https://issues.apache.org/jira/browse/SOLR-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012323#comment-13012323 ] Yonik Seeley commented on SOLR-2444: In the case where some fields may come from a DB, all of the clients definitely shouldn't be exposed to that mapping. The goal should be to have those fields look like any other fields, with the clients isolated from any mapping changes. support wildcards in fl parameter, improve DocTransformer parsing - Key: SOLR-2444 URL: https://issues.apache.org/jira/browse/SOLR-2444 Project: Solr Issue Type: New Feature Reporter: Ryan McKinley Attachments: SOLR-2444-fl-parsing.patch, SOLR-2444-fl-parsing.patch The ReturnFields parsing needs to be improved. It should also support wildcards -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [jira] [Commented] (SOLR-2417) Allow explain info directly to response documents
On Mon, Mar 28, 2011 at 9:36 PM, Chris Hostetter hossman_luc...@fucit.org wrote: : On IRC, yonik suggested that the explain format should mimic follow what : the debugQuery parameter would use. : : I'm don't really agree -- long term I would even suggest dropping the : explain section from debug and letting you specify it as an inline : parameter. those seem like orthoginal issues: 1) (ryan) deprecate/remove the explain section from debug and tell people to use the _explain_ psuedo field instead 2) (ryan) add options to the _explain_ psuedofield to let you pick an explanation style inline (ie: _explain:nl_) 3) (yonik) make the _explain_ psuedofield respect the debug.explain.structured param (or something like it) #1 #2 don't preclude #3 ... if i always want to get the nl mode explanations, it would be nice to be able to hardcode something a param the defaults section for my handler so that adding the _explain_ psuedofield just made it happen. I think we both agreed that _explain_ was a good thing. The issue was more about this: if explain has different formatting, how should it be controlled? I pointed out there was already a parameter to control this. Ryan had some alternate syntax proposals to allow passing parameters to transformers (and I was on the fence due to syntax proliferation - we already have localParams). And I'm still on the fence - _explain_ alone does not justify a whole new syntax IMO... so we may need more usecase examples to figure out what problem we're actually trying to solve. -Yonik http://www.lucenerevolution.org -- Lucene/Solr User Conference, May 25-26, San Francisco - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-2444) support wildcards in fl parameter, improve DocTransformer parsing
[ https://issues.apache.org/jira/browse/SOLR-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012327#comment-13012327 ] Ryan McKinley commented on SOLR-2444: - That seems fine -- if you don't want people to add a transformer as in the fl parameter, don't register the factory in solrconfig.xml, and add it to ReturnFields in a Component/Handler/whatever The Component/Handler/Whatever could be configured in solrconfig.xml with whatever it needs to make/edit the Transformer support wildcards in fl parameter, improve DocTransformer parsing - Key: SOLR-2444 URL: https://issues.apache.org/jira/browse/SOLR-2444 Project: Solr Issue Type: New Feature Reporter: Ryan McKinley Attachments: SOLR-2444-fl-parsing.patch, SOLR-2444-fl-parsing.patch The ReturnFields parsing needs to be improved. It should also support wildcards -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-2979) Simplify configuration API of contrib Query Parser
[
https://issues.apache.org/jira/browse/LUCENE-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13012331#comment-13012331
]
Phillipe Ramalho commented on LUCENE-2979:
--
{quote}
Don't forget to describe how you plan make the old and new API work together.
{quote}
I will do, thanks for reminding me ;)
Simplify configuration API of contrib Query Parser
--
Key: LUCENE-2979
URL: https://issues.apache.org/jira/browse/LUCENE-2979
Project: Lucene - Java
Issue Type: Improvement
Components: contrib/*
Affects Versions: 2.9, 3.0
Reporter: Adriano Crestani
Assignee: Adriano Crestani
Labels: api-change, gsoc, gsoc2011, lucene-gsoc-11, mentor
Fix For: 3.2, 4.0
The current configuration API is very complicated and inherit the concept
used by Attribute API to store token information in token streams. However,
the requirements for both (QP config and token stream) are not the same, so
they shouldn't be using the same thing.
I propose to simplify QP config and make it less scary for people intending
to use contrib QP. The task is not difficult, it will just require a lot of
code change and figure out the best way to do it. That's why it's a good
candidate for a GSoC project.
I would like to hear good proposals about how to make the API more friendly
and less scaring :)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
