[jira] [Created] (LUCENENET-488) Can't open IndexReader, get OutOFMemory Exception
Steven created LUCENENET-488: Summary: Can't open IndexReader, get OutOFMemory Exception Key: LUCENENET-488 URL: https://issues.apache.org/jira/browse/LUCENENET-488 Project: Lucene.Net Issue Type: Bug Components: Lucene.Net Core Affects Versions: Lucene.Net 2.9.4g Environment: Windows server 2008R2 Reporter: Steven Have build a large database with ~1Bn records (2 items per document) it has size 200GB on disk. I managed to write the indexe by chunking into 100,000 blocks as I ended up with some threading issues (another bug submission). Anyway the index is built but I can't open it and get a memory exception (process explorer gets to 1.5GB allocated before it dies but not sure how reliable that is, but do know there is plenty more RAM left on the box). Stack trace below: System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown. at Lucene.Net.Index.TermInfosReader..ctor(Directory dir, String seg, FieldInf os fis, Int32 readBufferSize, Int32 indexDivisor) at Lucene.Net.Index.SegmentReader.CoreReaders..ctor(SegmentReader origInstanc e, Directory dir, SegmentInfo si, Int32 readBufferSize, Int32 termsIndexDivisor) at Lucene.Net.Index.SegmentReader.Get(Boolean readOnly, Directory dir, Segmen tInfo si, Int32 readBufferSize, Boolean doOpenStores, Int32 termInfosIndexDiviso r) at Lucene.Net.Index.SegmentReader.Get(Boolean readOnly, SegmentInfo si, Int32 termInfosIndexDivisor) at Lucene.Net.Index.DirectoryReader..ctor(Directory directory, SegmentInfos s is, IndexDeletionPolicy deletionPolicy, Boolean readOnly, Int32 termInfosIndexDi visor) at Lucene.Net.Index.DirectoryReader.c__DisplayClass1.Openb__0(String segm entFileName) at Lucene.Net.Index.SegmentInfos.FindSegmentsFile.Run(IndexCommit commit) at Lucene.Net.Index.DirectoryReader.Open(Directory directory, IndexDeletionPo licy deletionPolicy, IndexCommit commit, Boolean readOnly, Int32 termInfosIndexD ivisor) at Lucene.Net.Index.IndexReader.Open(String path, Boolean readOnly) at Lucene.Net.Demo.SearchFiles.Main(String[] args) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: including external code under apache 2.0
On 2012-04-30, Itamar Syn-Hershko wrote: ICLA signed and sent and recorded by the secretary. Your name appears in this list http://people.apache.org/committer-index.html#unlistedclas as well. Many thanks Stefan
[JENKINS] Lucene-Solr-tests-only-trunk-java7 - Build # 2390 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk-java7/2390/ 1 tests failed. FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandler Error Message: ERROR: SolrIndexSearcher opens=74 closes=73 Stack Trace: java.lang.AssertionError: ERROR: SolrIndexSearcher opens=74 closes=73 at __randomizedtesting.SeedInfo.seed([6E989BE3D16133D1]:0) at org.junit.Assert.fail(Assert.java:93) at org.apache.solr.SolrTestCaseJ4.endTrackingSearchers(SolrTestCaseJ4.java:212) at org.apache.solr.SolrTestCaseJ4.afterClassSolrTestCase(SolrTestCaseJ4.java:101) at sun.reflect.GeneratedMethodAccessor17.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1913) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:741) at org.apache.lucene.util.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:63) at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:75) at org.apache.lucene.util.StoreClassNameRule$1.evaluate(StoreClassNameRule.java:38) at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:69) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:604) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:550) Build Log (for compile errors): [...truncated 11318 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3178) Versioning - optimistic locking
[ https://issues.apache.org/jira/browse/SOLR-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265685#comment-13265685 ] Per Steffensen commented on SOLR-3178: -- Agree, people have different preferences on how to use Solr, and therefore a.o. how the exact semantics of update should be. My solution is also better prepared for that, since I offer the semanticsMode property in updatehandler (DirectUpdateHandler2) where you can state if you want strict versioning (consistency mode) or the good old semantics (classic mode) or a mix (classic-consistency-mode mode). Guess we will always need to do leader-first-then-replica I cases where we want to do strict versioning control, but in classic mode (or a new future mode) we could certainly consider doing leader-and-replica-in-parallel. Will provide new patch today or tomorrow. Versioning - optimistic locking --- Key: SOLR-3178 URL: https://issues.apache.org/jira/browse/SOLR-3178 Project: Solr Issue Type: New Feature Components: update Affects Versions: 3.5 Environment: All Reporter: Per Steffensen Assignee: Per Steffensen Labels: RDBMS, insert, locking, nosql, optimistic, uniqueKey, update, versioning Fix For: 4.0 Attachments: SOLR-3178.patch, SOLR_3173_3178_3382_plus.patch Original Estimate: 168h Remaining Estimate: 168h In order increase the ability of Solr to be used as a NoSql database (lots of concurrent inserts, updates, deletes and queries in the entire lifetime of the index) instead of just a search index (first: everything indexed (in one thread), after: only queries), I would like Solr to support versioning to be used for optimistic locking. When my intent (see SOLR-3173) is to update an existing document, I will need to provide a version-number equal to the version number I got when I fetched the existing document for update plus one. If this provided version-number does not correspond to the newest version-number of that document at the time of update plus one, I will get a VersionConflict error. If it does correspond the document will be updated with the new one, so that the newest version-number of that document is NOW one higher than before the update. Correct but efficient concurrency handling. When my intent (see SOLR-3173) is to insert a new document, the version number provided will not be used - instead a version-number 0 will be used. According to SOLR-3173 insert will only succeed if a document with the same value on uniqueKey-field does not already exist. In general when talking about different versions of the same document, of course we need to be able to identify when a document is the same - that, per definition, is when the values of the uniqueKey-fields are equal. The functionality provided by this issue is only really meaningfull when you run with updateLog activated. This issue might be solved more or less at the same time as SOLR-3173, and only one single SVN patch might be given to cover both issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3178) Versioning - optimistic locking
[ https://issues.apache.org/jira/browse/SOLR-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265686#comment-13265686 ] Per Steffensen commented on SOLR-3178: -- Fire and forget will also be much cooler with a real async interface. We will provide that very soon - see SOLR-3383. Versioning - optimistic locking --- Key: SOLR-3178 URL: https://issues.apache.org/jira/browse/SOLR-3178 Project: Solr Issue Type: New Feature Components: update Affects Versions: 3.5 Environment: All Reporter: Per Steffensen Assignee: Per Steffensen Labels: RDBMS, insert, locking, nosql, optimistic, uniqueKey, update, versioning Fix For: 4.0 Attachments: SOLR-3178.patch, SOLR_3173_3178_3382_plus.patch Original Estimate: 168h Remaining Estimate: 168h In order increase the ability of Solr to be used as a NoSql database (lots of concurrent inserts, updates, deletes and queries in the entire lifetime of the index) instead of just a search index (first: everything indexed (in one thread), after: only queries), I would like Solr to support versioning to be used for optimistic locking. When my intent (see SOLR-3173) is to update an existing document, I will need to provide a version-number equal to the version number I got when I fetched the existing document for update plus one. If this provided version-number does not correspond to the newest version-number of that document at the time of update plus one, I will get a VersionConflict error. If it does correspond the document will be updated with the new one, so that the newest version-number of that document is NOW one higher than before the update. Correct but efficient concurrency handling. When my intent (see SOLR-3173) is to insert a new document, the version number provided will not be used - instead a version-number 0 will be used. According to SOLR-3173 insert will only succeed if a document with the same value on uniqueKey-field does not already exist. In general when talking about different versions of the same document, of course we need to be able to identify when a document is the same - that, per definition, is when the values of the uniqueKey-fields are equal. The functionality provided by this issue is only really meaningfull when you run with updateLog activated. This issue might be solved more or less at the same time as SOLR-3173, and only one single SVN patch might be given to cover both issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Issue Comment Edited] (SOLR-3178) Versioning - optimistic locking
[ https://issues.apache.org/jira/browse/SOLR-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265686#comment-13265686 ] Per Steffensen edited comment on SOLR-3178 at 5/1/12 6:47 AM: -- Fire and forget will also be much cooler with a real async client interface. We will provide that very soon - see SOLR-3383. was (Author: steff1193): Fire and forget will also be much cooler with a real async interface. We will provide that very soon - see SOLR-3383. Versioning - optimistic locking --- Key: SOLR-3178 URL: https://issues.apache.org/jira/browse/SOLR-3178 Project: Solr Issue Type: New Feature Components: update Affects Versions: 3.5 Environment: All Reporter: Per Steffensen Assignee: Per Steffensen Labels: RDBMS, insert, locking, nosql, optimistic, uniqueKey, update, versioning Fix For: 4.0 Attachments: SOLR-3178.patch, SOLR_3173_3178_3382_plus.patch Original Estimate: 168h Remaining Estimate: 168h In order increase the ability of Solr to be used as a NoSql database (lots of concurrent inserts, updates, deletes and queries in the entire lifetime of the index) instead of just a search index (first: everything indexed (in one thread), after: only queries), I would like Solr to support versioning to be used for optimistic locking. When my intent (see SOLR-3173) is to update an existing document, I will need to provide a version-number equal to the version number I got when I fetched the existing document for update plus one. If this provided version-number does not correspond to the newest version-number of that document at the time of update plus one, I will get a VersionConflict error. If it does correspond the document will be updated with the new one, so that the newest version-number of that document is NOW one higher than before the update. Correct but efficient concurrency handling. When my intent (see SOLR-3173) is to insert a new document, the version number provided will not be used - instead a version-number 0 will be used. According to SOLR-3173 insert will only succeed if a document with the same value on uniqueKey-field does not already exist. In general when talking about different versions of the same document, of course we need to be able to identify when a document is the same - that, per definition, is when the values of the uniqueKey-fields are equal. The functionality provided by this issue is only really meaningfull when you run with updateLog activated. This issue might be solved more or less at the same time as SOLR-3173, and only one single SVN patch might be given to cover both issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk-java7 - Build # 2391 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk-java7/2391/ 1 tests failed. REGRESSION: org.apache.solr.cloud.RecoveryZkTest.testDistribSearch Error Message: Thread threw an uncaught exception, thread: Thread[Lucene Merge Thread #1,6,] Stack Trace: java.lang.RuntimeException: Thread threw an uncaught exception, thread: Thread[Lucene Merge Thread #1,6,] at com.carrotsearch.randomizedtesting.RunnerThreadGroup.processUncaught(RunnerThreadGroup.java:96) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:848) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:668) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:687) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:723) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:734) at org.apache.lucene.util.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:63) at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:75) at org.apache.lucene.util.StoreClassNameRule$1.evaluate(StoreClassNameRule.java:38) at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:69) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:604) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:550) Caused by: org.apache.lucene.index.MergePolicy$MergeException: org.apache.lucene.store.AlreadyClosedException: this Directory is closed at __randomizedtesting.SeedInfo.seed([8BA910D0B66DC53E]:0) at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:507) at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:480) Caused by: org.apache.lucene.store.AlreadyClosedException: this Directory is closed at org.apache.lucene.store.Directory.ensureOpen(Directory.java:244) at org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:241) at org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:345) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3019) at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:382) at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:451) Build Log (for compile errors): [...truncated 12184 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3424) PhoneticFilterFactory threadsafety bug
[
https://issues.apache.org/jira/browse/SOLR-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265696#comment-13265696
]
Uwe Schindler commented on SOLR-3424:
-
Hi David,
looks good, but there is a bug:
The refactoring of the REGISTRY static map was an anonymous inner class before
(new HashMap() {{ hashmap ctor code }}; (please note double parens). Now it is
assigned to a static field, but the initializer code is an inline ctor of the
factory, means the initialization runs on every instantiation.
- add *static* before the anonymous ctor:
{code:java}
private static final MapString, Class? extends Encoder registry = new
HashMapString, Class? extends Encoder(6);
static {
registry.put(...
{code}
- or leave the initializer as it was before (anonymous HashMap subclass with
overridden ctor).
in resolve encoder maybe remove the clazz variable and directly return the
result of forName(). I do't like non-final variables initialized with null
because that prevents compilation failures on missing initialization and null
could be returned - especially with lots of try-catch blocks.
PhoneticFilterFactory threadsafety bug
--
Key: SOLR-3424
URL: https://issues.apache.org/jira/browse/SOLR-3424
Project: Solr
Issue Type: Bug
Components: Schema and Analysis
Affects Versions: 3.6, 4.0
Reporter: David Smiley
Assignee: David Smiley
Priority: Minor
Fix For: 4.0
Attachments: SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch,
SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch
PhoneticFilterFactory has a static HashMap registry mapping an encoder name
to an implementation. There is a ReentrantLock used when the map is modified
(when the encoder config specifies a class name). However, this map, which
can be accessed by multiple indexing threads, isn't guarded on any of the
reads, which isn't just the common path but also the error messages which
dump the registry into the error message.
I realize the likelihood of a problem is extremely slim, but a bug's a bug.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3424) PhoneticFilterFactory threadsafety bug
[ https://issues.apache.org/jira/browse/SOLR-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated SOLR-3424: Attachment: SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch Attached is a patch that fixes the initialization bug and improves the reflection code. *One thing that came into my mind when I looked at the code of PhoneticFilter:* I did not find out if Encoders are threadsafe or not. If they aren't or we are not sure (the documentation states nothing), we should create the Encoder class instance in the TokenStream create() method. TokenStreams itsself are only used per thread (iterator pattern), and the Analyzer reuse ensures that we have a separate instance for each thread. PhoneticFilterFactory threadsafety bug -- Key: SOLR-3424 URL: https://issues.apache.org/jira/browse/SOLR-3424 Project: Solr Issue Type: Bug Components: Schema and Analysis Affects Versions: 3.6, 4.0 Reporter: David Smiley Assignee: David Smiley Priority: Minor Fix For: 4.0 Attachments: SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch, SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch, SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch PhoneticFilterFactory has a static HashMap registry mapping an encoder name to an implementation. There is a ReentrantLock used when the map is modified (when the encoder config specifies a class name). However, this map, which can be accessed by multiple indexing threads, isn't guarded on any of the reads, which isn't just the common path but also the error messages which dump the registry into the error message. I realize the likelihood of a problem is extremely slim, but a bug's a bug. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3424) PhoneticFilterFactory threadsafety bug
[ https://issues.apache.org/jira/browse/SOLR-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uwe Schindler updated SOLR-3424: Attachment: SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch Updated patch, I added a testcase for the reflection without package name: - I use Caverphone2 as package less name, which is not in the REGISTRY. The returned class should be Caverphone2 - I cross-check with the REGISTRY name Caverphone, which should also return the new Caverphone2 encoder, but without reflection PhoneticFilterFactory threadsafety bug -- Key: SOLR-3424 URL: https://issues.apache.org/jira/browse/SOLR-3424 Project: Solr Issue Type: Bug Components: Schema and Analysis Affects Versions: 3.6, 4.0 Reporter: David Smiley Assignee: David Smiley Priority: Minor Fix For: 4.0 Attachments: SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch, SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch, SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch, SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch PhoneticFilterFactory has a static HashMap registry mapping an encoder name to an implementation. There is a ReentrantLock used when the map is modified (when the encoder config specifies a class name). However, this map, which can be accessed by multiple indexing threads, isn't guarded on any of the reads, which isn't just the common path but also the error messages which dump the registry into the error message. I realize the likelihood of a problem is extremely slim, but a bug's a bug. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3178) Versioning - optimistic locking
[ https://issues.apache.org/jira/browse/SOLR-3178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265768#comment-13265768 ] Yonik Seeley commented on SOLR-3178: bq. since I offer the semanticsMode property in updatehandler Seems like this should be up to the client... why wouldn't update semantics be handled on a per-request basis? Versioning - optimistic locking --- Key: SOLR-3178 URL: https://issues.apache.org/jira/browse/SOLR-3178 Project: Solr Issue Type: New Feature Components: update Affects Versions: 3.5 Environment: All Reporter: Per Steffensen Assignee: Per Steffensen Labels: RDBMS, insert, locking, nosql, optimistic, uniqueKey, update, versioning Fix For: 4.0 Attachments: SOLR-3178.patch, SOLR_3173_3178_3382_plus.patch Original Estimate: 168h Remaining Estimate: 168h In order increase the ability of Solr to be used as a NoSql database (lots of concurrent inserts, updates, deletes and queries in the entire lifetime of the index) instead of just a search index (first: everything indexed (in one thread), after: only queries), I would like Solr to support versioning to be used for optimistic locking. When my intent (see SOLR-3173) is to update an existing document, I will need to provide a version-number equal to the version number I got when I fetched the existing document for update plus one. If this provided version-number does not correspond to the newest version-number of that document at the time of update plus one, I will get a VersionConflict error. If it does correspond the document will be updated with the new one, so that the newest version-number of that document is NOW one higher than before the update. Correct but efficient concurrency handling. When my intent (see SOLR-3173) is to insert a new document, the version number provided will not be used - instead a version-number 0 will be used. According to SOLR-3173 insert will only succeed if a document with the same value on uniqueKey-field does not already exist. In general when talking about different versions of the same document, of course we need to be able to identify when a document is the same - that, per definition, is when the values of the uniqueKey-fields are equal. The functionality provided by this issue is only really meaningfull when you run with updateLog activated. This issue might be solved more or less at the same time as SOLR-3173, and only one single SVN patch might be given to cover both issues. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: hudson hung
Hmm, as a test, I tried adding @Timeout(millis=10) to LuceneTestCase, ie 100 seconds, which I think should not trigger on any core tests today. That should then apply to all subclasses of LuceneTestCase right? Ie all tests will be aborted after 100 seconds... But, something is wrong: I get lots of quick (ie much less than 100 seconds) failures like this: [junit4] Suite: org.apache.lucene.util.TestRamUsageEstimatorOnWildAnimals [junit4] ERROR 0.00s J2 | TestRamUsageEstimatorOnWildAnimals (suite) [junit4] Throwable #1: java.lang.RuntimeException: Interrupted while waiting for worker? Weird. [junit4]at __randomizedtesting.SeedInfo.seed([329191043FC42BEA]:0) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner.runAndWait(RandomizedRunner.java:786) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner.access$900(RandomizedRunner.java:131) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:689) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:723) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:734) [junit4]at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:75) [junit4]at org.apache.lucene.util.StoreClassNameRule$1.evaluate(StoreClassNameRule.java:38) [junit4]at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:69) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:604) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:131) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:550) [junit4] Caused by: java.lang.InterruptedException [junit4]at java.lang.Object.wait(Native Method) [junit4]at java.lang.Thread.join(Thread.java:1266) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner.runAndWait(RandomizedRunner.java:776) [junit4]... 10 more [junit4] [junit4] Completed on J2 in 0.34s, 1 test, 1 error FAILURES! [junit4] [junit4] May 01, 2012 9:02:18 AM com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus dispatch [junit4] SEVERE: Could not dispatch event: com.carrotsearch.ant.tasks.junit4.events.aggregated.AggregatedSuiteResultEvent@6637b753 to handler [wrapper public void com.carrotsearch.ant.tasks.junit4.listeners.TextReport.onSuiteResult(com.carrotsearch.ant.tasks.junit4.events.aggregated.AggregatedSuiteResultEvent)] [junit4] java.lang.reflect.InvocationTargetException [junit4] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit4] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [junit4] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [junit4] at java.lang.reflect.Method.invoke(Method.java:601) [junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:68) [junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45) [junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatch(EventBus.java:313) [junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatchQueuedEvents(EventBus.java:296) [junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.post(EventBus.java:264) [junit4] at com.carrotsearch.ant.tasks.junit4.events.aggregated.AggregatingListener.receiveSuiteEnd(AggregatingListener.java:130) [junit4] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit4] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [junit4] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [junit4] at java.lang.reflect.Method.invoke(Method.java:601) [junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:68) [junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45) [junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatch(EventBus.java:313) [junit4] at
[jira] [Updated] (SOLR-3425) CloudSolrServer can't create cores when using the zkHost based constructor
[ https://issues.apache.org/jira/browse/SOLR-3425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tommaso Teofili updated SOLR-3425: -- Attachment: SOLR-3425-test.patch test case: setup a 2 nodes cluster as example A in SolrCloud wiki page (http://wiki.apache.org/solr/SolrCloud) and run the attached test. The testLBServerCoreCreation() test should pass while the testZKHostCoreCreation() should fail CloudSolrServer can't create cores when using the zkHost based constructor -- Key: SOLR-3425 URL: https://issues.apache.org/jira/browse/SOLR-3425 Project: Solr Issue Type: Improvement Components: SolrCloud Reporter: Tommaso Teofili Priority: Minor Fix For: 4.0 Attachments: SOLR-3425-test.patch When programmatically creating cores with a running SolrCloud instance the CloudSolrServer uses the slices nodes information to feed the underlying LBHttpSolrServer so it fails to create cores as there aren't any slices for any new collection (as it's still to be created). This happens when using the CloudSolrServer constructor which takes the ZK host as only parameter while it can be avoided by using the constructor which also takes the list of Solr URLs and the underlying LBHttpSolrServer is actually used for making the core creation request. However it'd be good to use the ZK host live nodes information to automatically issue a core creation command on one of the underlying Solr hosts without having to specify the full list of URLs beforehand. The scenario is when one wants to create a collection with N shards so the client sends N core creation requests for the same collection thus the SolrCloud stuff should just take care of choosing the host where to issue the core creation request and update the cluster state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4025) ReferenceManager.maybeRefresh should allow the caller to block
[
https://issues.apache.org/jira/browse/LUCENE-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265800#comment-13265800
]
Michael McCandless commented on LUCENE-4025:
Approach patch look good!
Maybe change javadocs to state that you must call maybeRefresh or
maybeRefreshBlocking periodically (now each one states you must call that
one). Also maybe say if another thread is currently refreshing, this method
blocks until that thread completes.
ReferenceManager.maybeRefresh should allow the caller to block
--
Key: LUCENE-4025
URL: https://issues.apache.org/jira/browse/LUCENE-4025
Project: Lucene - Java
Issue Type: Improvement
Components: core/search
Reporter: Shai Erera
Priority: Minor
Fix For: 4.0
Attachments: LUCENE-4025.patch
ReferenceManager.maybeRefresh() returns a boolean today, specifying whether
the maybeRefresh logic was executed by the caller or not. If it's false, it
means that another thread is currently refreshing and the call returns
immediately.
I think that that's inconvenient to the caller. I.e., if you wanted to do
something like:
{code}
writer.commit();
searcherManager.maybeRefresh();
searcherManager.acquire();
{code}
It'd be better if you could guarantee that when the maybeRefresh() call
returned, the follow on acquire() will return a refreshed IndexSearcher. Even
if you omit the commit instruction, it'd be good if you can guarantee that.
I don't quite see the benefit of having the caller thread not block if
there's another thread currently refreshing. In, I believe, most cases, you'd
anyway have just one thread calling maybeRefresh(). Even if not, the only
benefit of not blocking is if you have commit() followed by maybeRefresh()
logic done by some threads, while other threads acquire searchers - maybe
then you wouldn't care if another thread is currently doing the refresh?
Actually, I tend to think that not blocking is buggy? I mean, what if two
threads do commit() + maybeRefresh(). The first thread finishes commit,
enters maybeRefresh(), acquires the lock and reopens the Reader. Then the
second thread does its commit(), enters maybeRefresh, fails to obtain the
lock and exits. Its changes won't be exposed by SM until the next
maybeRefresh() is called.
So it looks to me like current logic may be buggy in that sense?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3425) CloudSolrServer can't create cores when using the zkHost based constructor
[ https://issues.apache.org/jira/browse/SOLR-3425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265805#comment-13265805 ] Mark Miller commented on SOLR-3425: --- I'd like to improve collection creation outside of Solrj as well - but I think it also makes sense to improve this here as well. Do you have a proposal yet? Perhaps we should just add new collection management api? Trying to wrap this stuff in a SolrCore's world gets kind of ugly. CloudSolrServer can't create cores when using the zkHost based constructor -- Key: SOLR-3425 URL: https://issues.apache.org/jira/browse/SOLR-3425 Project: Solr Issue Type: Improvement Components: SolrCloud Reporter: Tommaso Teofili Priority: Minor Fix For: 4.0 Attachments: SOLR-3425-test.patch When programmatically creating cores with a running SolrCloud instance the CloudSolrServer uses the slices nodes information to feed the underlying LBHttpSolrServer so it fails to create cores as there aren't any slices for any new collection (as it's still to be created). This happens when using the CloudSolrServer constructor which takes the ZK host as only parameter while it can be avoided by using the constructor which also takes the list of Solr URLs and the underlying LBHttpSolrServer is actually used for making the core creation request. However it'd be good to use the ZK host live nodes information to automatically issue a core creation command on one of the underlying Solr hosts without having to specify the full list of URLs beforehand. The scenario is when one wants to create a collection with N shards so the client sends N core creation requests for the same collection thus the SolrCloud stuff should just take care of choosing the host where to issue the core creation request and update the cluster state. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3296) Explore alternatives to Commons CSV
[ https://issues.apache.org/jira/browse/SOLR-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265809#comment-13265809 ] Steven Rowe commented on SOLR-3296: --- Chris, I don't see org.noggit:noggit up on Maven Central yet, so I guess your request has hit a snag - do you know what's happening? Explore alternatives to Commons CSV --- Key: SOLR-3296 URL: https://issues.apache.org/jira/browse/SOLR-3296 Project: Solr Issue Type: Improvement Components: Build Reporter: Chris Male Attachments: SOLR-3295-CSV-tests.patch, SOLR-3296_noggit.patch, pom.xml, pom.xml In LUCENE-3930 we're implementing some less than ideal solutions to make available the unreleased version of commons-csv. We could remove these solutions if we didn't rely on this lib. So I think we should explore alternatives. I think [opencsv|http://opencsv.sourceforge.net/] is an alternative to consider, I've used it in many commercial projects. Bizarrely Commons-CSV's website says that Opencsv uses a BSD license, but this isn't the case, OpenCSV uses ASL2. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: hudson hung
I tried to enforce a Timeout using this code, added to my top-level test case (didn't try yet on LTC): @Rule public static Timeout globalTimeout = new Timeout(10); // 100 seconds This comes straight from JUnit. I don't have the randomizedtesting jar in my project, so I couldn't add the @Timeout annotation. When I set it to a low value (20 ms), many tests failed as expected. However, on one occasion, a test took 135 seconds, as reported by JUnit in eclipse, and still didn't fail ! I don't know what it means though -- the test usually doesn't take more than 15 seconds to run, so perhaps the computer was busy, and therefore even the timeout thread didn't get the chance to fail ... or perhaps the time reported by eclipse is inaccurate. Anyway, perhaps try with that (just add it to LTC anywhere). Shai On Tue, May 1, 2012 at 4:05 PM, Michael McCandless luc...@mikemccandless.com wrote: Hmm, as a test, I tried adding @Timeout(millis=10) to LuceneTestCase, ie 100 seconds, which I think should not trigger on any core tests today. That should then apply to all subclasses of LuceneTestCase right? Ie all tests will be aborted after 100 seconds... But, something is wrong: I get lots of quick (ie much less than 100 seconds) failures like this: [junit4] Suite: org.apache.lucene.util.TestRamUsageEstimatorOnWildAnimals [junit4] ERROR 0.00s J2 | TestRamUsageEstimatorOnWildAnimals (suite) [junit4] Throwable #1: java.lang.RuntimeException: Interrupted while waiting for worker? Weird. [junit4]at __randomizedtesting.SeedInfo.seed([329191043FC42BEA]:0) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner.runAndWait(RandomizedRunner.java:786) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner.access$900(RandomizedRunner.java:131) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:689) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:723) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:734) [junit4]at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:75) [junit4]at org.apache.lucene.util.StoreClassNameRule$1.evaluate(StoreClassNameRule.java:38) [junit4]at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:69) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:604) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:131) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:550) [junit4] Caused by: java.lang.InterruptedException [junit4]at java.lang.Object.wait(Native Method) [junit4]at java.lang.Thread.join(Thread.java:1266) [junit4]at com.carrotsearch.randomizedtesting.RandomizedRunner.runAndWait(RandomizedRunner.java:776) [junit4]... 10 more [junit4] [junit4] Completed on J2 in 0.34s, 1 test, 1 error FAILURES! [junit4] [junit4] May 01, 2012 9:02:18 AM com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus dispatch [junit4] SEVERE: Could not dispatch event: com.carrotsearch.ant.tasks.junit4.events.aggregated.AggregatedSuiteResultEvent@6637b753 to handler [wrapper public void com.carrotsearch.ant.tasks.junit4.listeners.TextReport.onSuiteResult(com.carrotsearch.ant.tasks.junit4.events.aggregated.AggregatedSuiteResultEvent)] [junit4] java.lang.reflect.InvocationTargetException [junit4] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [junit4] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [junit4] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [junit4] at java.lang.reflect.Method.invoke(Method.java:601) [junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventHandler.handleEvent(EventHandler.java:68) [junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.SynchronizedEventHandler.handleEvent(SynchronizedEventHandler.java:45) [junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatch(EventBus.java:313) [junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.dispatchQueuedEvents(EventBus.java:296) [junit4] at com.carrotsearch.ant.tasks.junit4.dependencies.com.google.common.eventbus.EventBus.post(EventBus.java:264) [junit4] at
[jira] [Updated] (LUCENE-4025) ReferenceManager.maybeRefresh should allow the caller to block
[
https://issues.apache.org/jira/browse/LUCENE-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shai Erera updated LUCENE-4025:
---
Attachment: LUCENE-4025.patch
Thanks Mike. I updated the javadoc as you suggest and added a CHANGES entry.
I think this is ready to commit.
ReferenceManager.maybeRefresh should allow the caller to block
--
Key: LUCENE-4025
URL: https://issues.apache.org/jira/browse/LUCENE-4025
Project: Lucene - Java
Issue Type: Improvement
Components: core/search
Reporter: Shai Erera
Priority: Minor
Fix For: 4.0
Attachments: LUCENE-4025.patch, LUCENE-4025.patch
ReferenceManager.maybeRefresh() returns a boolean today, specifying whether
the maybeRefresh logic was executed by the caller or not. If it's false, it
means that another thread is currently refreshing and the call returns
immediately.
I think that that's inconvenient to the caller. I.e., if you wanted to do
something like:
{code}
writer.commit();
searcherManager.maybeRefresh();
searcherManager.acquire();
{code}
It'd be better if you could guarantee that when the maybeRefresh() call
returned, the follow on acquire() will return a refreshed IndexSearcher. Even
if you omit the commit instruction, it'd be good if you can guarantee that.
I don't quite see the benefit of having the caller thread not block if
there's another thread currently refreshing. In, I believe, most cases, you'd
anyway have just one thread calling maybeRefresh(). Even if not, the only
benefit of not blocking is if you have commit() followed by maybeRefresh()
logic done by some threads, while other threads acquire searchers - maybe
then you wouldn't care if another thread is currently doing the refresh?
Actually, I tend to think that not blocking is buggy? I mean, what if two
threads do commit() + maybeRefresh(). The first thread finishes commit,
enters maybeRefresh(), acquires the lock and reopens the Reader. Then the
second thread does its commit(), enters maybeRefresh, fails to obtain the
lock and exits. Its changes won't be exposed by SM until the next
maybeRefresh() is called.
So it looks to me like current logic may be buggy in that sense?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3425) CloudSolrServer can't create cores when using the zkHost based constructor
[
https://issues.apache.org/jira/browse/SOLR-3425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265841#comment-13265841
]
Tommaso Teofili commented on SOLR-3425:
---
bq. Perhaps we should just add new collection management api?
I think so, that would also help SolrCloud users understand the mind shift from
cores to collections.
For this particular thing the dummy fix (not tested widely but still all the
tests pass) could be to add the following lines for filling the urlList
variable:
{code}
// enable automatic distributed core creation
if (request instanceof CoreAdminRequest.Create) {
for (String liveNodeHost : zkStateReader.getCloudState().getLiveNodes()) {
urlList.add(new
StringBuilder(http://;).append(liveNodeHost.replaceAll(_solr,
/solr/)).toString());
}
}
else {
for (Slice slice : slices.values()) {
...
{code}
however i don't like it very much. I think adding proper APIs would be just
better.
CloudSolrServer can't create cores when using the zkHost based constructor
--
Key: SOLR-3425
URL: https://issues.apache.org/jira/browse/SOLR-3425
Project: Solr
Issue Type: Improvement
Components: SolrCloud
Reporter: Tommaso Teofili
Priority: Minor
Fix For: 4.0
Attachments: SOLR-3425-test.patch
When programmatically creating cores with a running SolrCloud instance the
CloudSolrServer uses the slices nodes information to feed the underlying
LBHttpSolrServer so it fails to create cores as there aren't any slices for
any new collection (as it's still to be created).
This happens when using the CloudSolrServer constructor which takes the ZK
host as only parameter while it can be avoided by using the constructor which
also takes the list of Solr URLs and the underlying LBHttpSolrServer is
actually used for making the core creation request.
However it'd be good to use the ZK host live nodes information to
automatically issue a core creation command on one of the underlying Solr
hosts without having to specify the full list of URLs beforehand.
The scenario is when one wants to create a collection with N shards so the
client sends N core creation requests for the same collection thus the
SolrCloud stuff should just take care of choosing the host where to issue the
core creation request and update the cluster state.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4025) ReferenceManager.maybeRefresh should allow the caller to block
[
https://issues.apache.org/jira/browse/LUCENE-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265842#comment-13265842
]
Michael McCandless commented on LUCENE-4025:
+1, looks good. Thanks Shai!
ReferenceManager.maybeRefresh should allow the caller to block
--
Key: LUCENE-4025
URL: https://issues.apache.org/jira/browse/LUCENE-4025
Project: Lucene - Java
Issue Type: Improvement
Components: core/search
Reporter: Shai Erera
Priority: Minor
Fix For: 4.0
Attachments: LUCENE-4025.patch, LUCENE-4025.patch
ReferenceManager.maybeRefresh() returns a boolean today, specifying whether
the maybeRefresh logic was executed by the caller or not. If it's false, it
means that another thread is currently refreshing and the call returns
immediately.
I think that that's inconvenient to the caller. I.e., if you wanted to do
something like:
{code}
writer.commit();
searcherManager.maybeRefresh();
searcherManager.acquire();
{code}
It'd be better if you could guarantee that when the maybeRefresh() call
returned, the follow on acquire() will return a refreshed IndexSearcher. Even
if you omit the commit instruction, it'd be good if you can guarantee that.
I don't quite see the benefit of having the caller thread not block if
there's another thread currently refreshing. In, I believe, most cases, you'd
anyway have just one thread calling maybeRefresh(). Even if not, the only
benefit of not blocking is if you have commit() followed by maybeRefresh()
logic done by some threads, while other threads acquire searchers - maybe
then you wouldn't care if another thread is currently doing the refresh?
Actually, I tend to think that not blocking is buggy? I mean, what if two
threads do commit() + maybeRefresh(). The first thread finishes commit,
enters maybeRefresh(), acquires the lock and reopens the Reader. Then the
second thread does its commit(), enters maybeRefresh, fails to obtain the
lock and exits. Its changes won't be exposed by SM until the next
maybeRefresh() is called.
So it looks to me like current logic may be buggy in that sense?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
report-text doesn't support the useSimpleNames attribute
Hi I svn up and ran 'ant test-core', but hit this strange error: lucene\build.xml:50: The following error occurred while executing this line: lucene\common-build.xml:1018: The following error occurred while executing this line: lucene\common-build.xml:744: report-text doesn't support the useSimpleNames attribute I have Ant 1.8.2. I tried to run the tests with an Oracle JVM (the above failure was from an IBM JVM), and hit this: lucene\build.xml:50: The following error occurred while executing this line: lucene\common-build.xml:1018: The following error occurred while executing this line: lucene\common-build.xml:744: report-text doesn't support the maxClassNameColumns attribute Each JVM complains about a different attribute ! This is the Oracle Java 6 that I ran with: java version 1.6.0_31 Java(TM) SE Runtime Environment (build 1.6.0_31-b05) Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode) Has anyone run into this? Shai
[jira] [Created] (SOLR-3426) CoreContainer goes looking for nonexistent Log4j class
Benson Margulies created SOLR-3426:
--
Summary: CoreContainer goes looking for nonexistent Log4j class
Key: SOLR-3426
URL: https://issues.apache.org/jira/browse/SOLR-3426
Project: Solr
Issue Type: Bug
Affects Versions: 4.0
Reporter: Benson Margulies
The code quoted here from CoreContainer.java ends up looking for a class named
Log4j instead of Log4jWatcher. Neither name exists in the tree.
// Now load the framework
if(fname!=null) {
if(JUL.equalsIgnoreCase(fname)) {
logging = new JulWatcher(slf4jImpl);
}
//else if( Log4j.equals(fname) ) {
// logging = new Log4jWatcher(slf4jImpl);
//}
else {
try {
logging = loader.newInstance(fname, LogWatcher.class);
}
catch (Throwable e) {
log.warn(Unable to load LogWatcher, e);
}
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Resolved] (LUCENE-4025) ReferenceManager.maybeRefresh should allow the caller to block
[
https://issues.apache.org/jira/browse/LUCENE-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shai Erera resolved LUCENE-4025.
Resolution: Fixed
Lucene Fields: New,Patch Available (was: New)
Committed rev 1332699.
I also added randomly calling it from TestSearcherManager.
Thanks Mike for the review !
ReferenceManager.maybeRefresh should allow the caller to block
--
Key: LUCENE-4025
URL: https://issues.apache.org/jira/browse/LUCENE-4025
Project: Lucene - Java
Issue Type: Improvement
Components: core/search
Reporter: Shai Erera
Priority: Minor
Fix For: 4.0
Attachments: LUCENE-4025.patch, LUCENE-4025.patch
ReferenceManager.maybeRefresh() returns a boolean today, specifying whether
the maybeRefresh logic was executed by the caller or not. If it's false, it
means that another thread is currently refreshing and the call returns
immediately.
I think that that's inconvenient to the caller. I.e., if you wanted to do
something like:
{code}
writer.commit();
searcherManager.maybeRefresh();
searcherManager.acquire();
{code}
It'd be better if you could guarantee that when the maybeRefresh() call
returned, the follow on acquire() will return a refreshed IndexSearcher. Even
if you omit the commit instruction, it'd be good if you can guarantee that.
I don't quite see the benefit of having the caller thread not block if
there's another thread currently refreshing. In, I believe, most cases, you'd
anyway have just one thread calling maybeRefresh(). Even if not, the only
benefit of not blocking is if you have commit() followed by maybeRefresh()
logic done by some threads, while other threads acquire searchers - maybe
then you wouldn't care if another thread is currently doing the refresh?
Actually, I tend to think that not blocking is buggy? I mean, what if two
threads do commit() + maybeRefresh(). The first thread finishes commit,
enters maybeRefresh(), acquires the lock and reopens the Reader. Then the
second thread does its commit(), enters maybeRefresh, fails to obtain the
lock and exits. Its changes won't be exposed by SM until the next
maybeRefresh() is called.
So it looks to me like current logic may be buggy in that sense?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3424) PhoneticFilterFactory threadsafety bug
[ https://issues.apache.org/jira/browse/SOLR-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265862#comment-13265862 ] David Smiley commented on SOLR-3424: Uwe: Excellent catch RE my static bug, and I agree on your improvement RE clazz null. I tried to ascertain the thread-safety status of Commons-Codec and unfortunately I don't think we can assume it's thread-safe. I sent an email to their list about this just now: http://apache-commons.680414.n4.nabble.com/Thread-safety-of-Commons-Codec-Encoder-tt4600956.html So I agree that we'll have to insatiate one of these in our create() method instead of caching it. PhoneticFilterFactory threadsafety bug -- Key: SOLR-3424 URL: https://issues.apache.org/jira/browse/SOLR-3424 Project: Solr Issue Type: Bug Components: Schema and Analysis Affects Versions: 3.6, 4.0 Reporter: David Smiley Assignee: David Smiley Priority: Minor Fix For: 4.0 Attachments: SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch, SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch, SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch, SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch PhoneticFilterFactory has a static HashMap registry mapping an encoder name to an implementation. There is a ReentrantLock used when the map is modified (when the encoder config specifies a class name). However, this map, which can be accessed by multiple indexing threads, isn't guarded on any of the reads, which isn't just the common path but also the error messages which dump the registry into the error message. I realize the likelihood of a problem is extremely slim, but a bug's a bug. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk-java7 - Build # 2395 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk-java7/2395/ 1 tests failed. REGRESSION: org.apache.lucene.search.TestSearcherManager.testSearcherManager Error Message: Stack Trace: java.lang.AssertionError at __randomizedtesting.SeedInfo.seed([4E8F23F3DAA59805:42A06B496CACF9F8]:0) at org.junit.Assert.fail(Assert.java:92) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertTrue(Assert.java:54) at org.apache.lucene.search.TestSearcherManager.getFinalSearcher(TestSearcherManager.java:63) at org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.runTest(ThreadedIndexingAndSearchingTestCase.java:505) at org.apache.lucene.search.TestSearcherManager.testSearcherManager(TestSearcherManager.java:55) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1913) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:805) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:866) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:880) at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:770) at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:692) at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:69) at org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(LuceneTestCase.java:625) at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:75) at org.apache.lucene.util.LuceneTestCase$SaveThreadAndTestNameRule$1.evaluate(LuceneTestCase.java:664) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:812) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:668) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:687) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:723) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:734) at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:75) at org.apache.lucene.util.StoreClassNameRule$1.evaluate(StoreClassNameRule.java:38) at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:69) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:604) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:550) Build Log (for compile errors): [...truncated 1629 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk - Build # 13657 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/13657/ 1 tests failed. REGRESSION: org.apache.lucene.search.TestSearcherManager.testSearcherManager Error Message: Stack Trace: java.lang.AssertionError at __randomizedtesting.SeedInfo.seed([7BE3A3BA9711FD3E:77CCEB0021189CC3]:0) at org.junit.Assert.fail(Assert.java:92) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertTrue(Assert.java:54) at org.apache.lucene.search.TestSearcherManager.getFinalSearcher(TestSearcherManager.java:63) at org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.runTest(ThreadedIndexingAndSearchingTestCase.java:505) at org.apache.lucene.search.TestSearcherManager.testSearcherManager(TestSearcherManager.java:55) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1913) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:805) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:866) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:880) at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:770) at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:692) at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:69) at org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(LuceneTestCase.java:625) at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:75) at org.apache.lucene.util.LuceneTestCase$SaveThreadAndTestNameRule$1.evaluate(LuceneTestCase.java:664) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:812) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:668) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:687) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:723) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:734) at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:75) at org.apache.lucene.util.StoreClassNameRule$1.evaluate(StoreClassNameRule.java:38) at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:69) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:604) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:550) Build Log (for compile errors): [...truncated 1601 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-Solr-tests-only-trunk - Build # 13658 - Still Failing
Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/13658/ 1 tests failed. FAILED: org.apache.lucene.search.TestSearcherManager.testSearcherManager Error Message: Stack Trace: java.lang.AssertionError at __randomizedtesting.SeedInfo.seed([1DADD745DCD7932:DF595CEEBC418CF]:0) at org.junit.Assert.fail(Assert.java:92) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertTrue(Assert.java:54) at org.apache.lucene.search.TestSearcherManager.getFinalSearcher(TestSearcherManager.java:63) at org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.runTest(ThreadedIndexingAndSearchingTestCase.java:505) at org.apache.lucene.search.TestSearcherManager.testSearcherManager(TestSearcherManager.java:55) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1913) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:805) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:866) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:880) at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:770) at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:692) at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:69) at org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(LuceneTestCase.java:625) at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:75) at org.apache.lucene.util.LuceneTestCase$SaveThreadAndTestNameRule$1.evaluate(LuceneTestCase.java:664) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:812) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:668) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:687) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:723) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:734) at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:75) at org.apache.lucene.util.StoreClassNameRule$1.evaluate(StoreClassNameRule.java:38) at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:69) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:604) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:550) Build Log (for compile errors): [...truncated 1601 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-tests-only-trunk - Build # 13658 - Still Failing
My bad. Mispelled unlock with lock in the finally part ! (so I ended up locking twice). Will add a deterministic test commit the fix Sorry ! Shai On Tue, May 1, 2012 at 6:46 PM, Apache Jenkins Server jenk...@builds.apache.org wrote: Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/13658/ 1 tests failed. FAILED: org.apache.lucene.search.TestSearcherManager.testSearcherManager Error Message: Stack Trace: java.lang.AssertionError at __randomizedtesting.SeedInfo.seed([1DADD745DCD7932:DF595CEEBC418CF]:0) at org.junit.Assert.fail(Assert.java:92) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertTrue(Assert.java:54) at org.apache.lucene.search.TestSearcherManager.getFinalSearcher(TestSearcherManager.java:63) at org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.runTest(ThreadedIndexingAndSearchingTestCase.java:505) at org.apache.lucene.search.TestSearcherManager.testSearcherManager(TestSearcherManager.java:55) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1913) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:805) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:866) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:880) at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:770) at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:692) at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:69) at org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(LuceneTestCase.java:625) at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:75) at org.apache.lucene.util.LuceneTestCase$SaveThreadAndTestNameRule$1.evaluate(LuceneTestCase.java:664) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:812) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:668) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:687) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:723) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:734) at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:75) at org.apache.lucene.util.StoreClassNameRule$1.evaluate(StoreClassNameRule.java:38) at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:69) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:604) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:550) Build Log (for compile errors): [...truncated 1601 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3426) CoreContainer goes looking for nonexistent Log4j class
[
https://issues.apache.org/jira/browse/SOLR-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265880#comment-13265880
]
Ryan McKinley commented on SOLR-3426:
-
right -- if you are using Log4j, the LogWatcher stuff will not work.
In the current behavior, you should get a warning in the logs and when you try
the logging UI it will give a warning that it did not load.
See: SOLR-3399 and SOLR-3358 that look at ways to capture log4j events.
CoreContainer goes looking for nonexistent Log4j class
--
Key: SOLR-3426
URL: https://issues.apache.org/jira/browse/SOLR-3426
Project: Solr
Issue Type: Bug
Affects Versions: 4.0
Reporter: Benson Margulies
The code quoted here from CoreContainer.java ends up looking for a class
named Log4j instead of Log4jWatcher. Neither name exists in the tree.
// Now load the framework
if(fname!=null) {
if(JUL.equalsIgnoreCase(fname)) {
logging = new JulWatcher(slf4jImpl);
}
//else if( Log4j.equals(fname) ) {
// logging = new Log4jWatcher(slf4jImpl);
//}
else {
try {
logging = loader.newInstance(fname, LogWatcher.class);
}
catch (Throwable e) {
log.warn(Unable to load LogWatcher, e);
}
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
Re: [JENKINS] Lucene-Solr-tests-only-trunk - Build # 13658 - Still Failing
Committed fix in r1332733. On Tue, May 1, 2012 at 7:07 PM, Shai Erera ser...@gmail.com wrote: My bad. Mispelled unlock with lock in the finally part ! (so I ended up locking twice). Will add a deterministic test commit the fix Sorry ! Shai On Tue, May 1, 2012 at 6:46 PM, Apache Jenkins Server jenk...@builds.apache.org wrote: Build: https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/13658/ 1 tests failed. FAILED: org.apache.lucene.search.TestSearcherManager.testSearcherManager Error Message: Stack Trace: java.lang.AssertionError at __randomizedtesting.SeedInfo.seed([1DADD745DCD7932:DF595CEEBC418CF]:0) at org.junit.Assert.fail(Assert.java:92) at org.junit.Assert.assertTrue(Assert.java:43) at org.junit.Assert.assertTrue(Assert.java:54) at org.apache.lucene.search.TestSearcherManager.getFinalSearcher(TestSearcherManager.java:63) at org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase.runTest(ThreadedIndexingAndSearchingTestCase.java:505) at org.apache.lucene.search.TestSearcherManager.testSearcherManager(TestSearcherManager.java:55) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1913) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:805) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:866) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:880) at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:770) at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:692) at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:69) at org.apache.lucene.util.LuceneTestCase$TestResultInterceptorRule$1.evaluate(LuceneTestCase.java:625) at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:75) at org.apache.lucene.util.LuceneTestCase$SaveThreadAndTestNameRule$1.evaluate(LuceneTestCase.java:664) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:812) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:668) at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:687) at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:723) at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:734) at org.apache.lucene.util.UncaughtExceptionsRule$1.evaluate(UncaughtExceptionsRule.java:75) at org.apache.lucene.util.StoreClassNameRule$1.evaluate(StoreClassNameRule.java:38) at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:69) at com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:604) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:550) Build Log (for compile errors): [...truncated 1601 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: LUCENE-4012
I left a comment on the issue, I think this looks worth pursuing. Since we removed the java serialization support, this would be a nice alternative to have (in addition to the xml queryparser). On Mon, Apr 30, 2012 at 11:26 AM, Benson Margulies bimargul...@gmail.com wrote: I'd be ever so grateful for feedback on what I've got here so far. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3830) MappingCharFilter could be improved by switching to an FST.
[ https://issues.apache.org/jira/browse/LUCENE-3830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265888#comment-13265888 ] Robert Muir commented on LUCENE-3830: - I only glanced at the Schulz/Mihov paper, but it seemed like in order for it to work you have to add extra transitions (loops) which would make the FST infinite? (could be totally wrong here) Maybe there is some creative way we could do this, but thats just in addition to the fact the paper is ... like their other papers and would take some work to decode anyway :) MappingCharFilter could be improved by switching to an FST. --- Key: LUCENE-3830 URL: https://issues.apache.org/jira/browse/LUCENE-3830 Project: Lucene - Java Issue Type: Improvement Reporter: Dawid Weiss Assignee: Michael McCandless Priority: Minor Labels: gsoc2012, lucene-gsoc-12 Fix For: 4.0 Attachments: LUCENE-3830.patch MappingCharFilter stores an overly complex tree-like structure for matching input patterns. The input is a union of fixed strings mapped to a set of fixed strings; an fst matcher would be ideal here and provide both memory and speed improvement I bet. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3426) CoreContainer goes looking for nonexistent Log4j class
[
https://issues.apache.org/jira/browse/SOLR-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benson Margulies updated SOLR-3426:
---
Attachment: solr-3426.patch
here is a temporary fix that removes all the noisy backtracing from the log.
CoreContainer goes looking for nonexistent Log4j class
--
Key: SOLR-3426
URL: https://issues.apache.org/jira/browse/SOLR-3426
Project: Solr
Issue Type: Bug
Affects Versions: 4.0
Reporter: Benson Margulies
Attachments: solr-3426.patch
The code quoted here from CoreContainer.java ends up looking for a class
named Log4j instead of Log4jWatcher. Neither name exists in the tree.
// Now load the framework
if(fname!=null) {
if(JUL.equalsIgnoreCase(fname)) {
logging = new JulWatcher(slf4jImpl);
}
//else if( Log4j.equals(fname) ) {
// logging = new Log4jWatcher(slf4jImpl);
//}
else {
try {
logging = loader.newInstance(fname, LogWatcher.class);
}
catch (Throwable e) {
log.warn(Unable to load LogWatcher, e);
}
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3426) CoreContainer goes looking for nonexistent Log4j class
[
https://issues.apache.org/jira/browse/SOLR-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265900#comment-13265900
]
Benson Margulies commented on SOLR-3426:
To be clear, the 'warning' is a screenful of backtraces, and that's pretty
scary and annoying.
CoreContainer goes looking for nonexistent Log4j class
--
Key: SOLR-3426
URL: https://issues.apache.org/jira/browse/SOLR-3426
Project: Solr
Issue Type: Bug
Affects Versions: 4.0
Reporter: Benson Margulies
Attachments: solr-3426.patch
The code quoted here from CoreContainer.java ends up looking for a class
named Log4j instead of Log4jWatcher. Neither name exists in the tree.
// Now load the framework
if(fname!=null) {
if(JUL.equalsIgnoreCase(fname)) {
logging = new JulWatcher(slf4jImpl);
}
//else if( Log4j.equals(fname) ) {
// logging = new Log4jWatcher(slf4jImpl);
//}
else {
try {
logging = loader.newInstance(fname, LogWatcher.class);
}
catch (Throwable e) {
log.warn(Unable to load LogWatcher, e);
}
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4012) Make all query classes serializable with Jackson, and provide a trivial query parser to consume them
[ https://issues.apache.org/jira/browse/LUCENE-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265901#comment-13265901 ] Benson Margulies commented on LUCENE-4012: -- Right. The tiny patch is an example of the sort of change, and I think it's always good to have 'getters'. In fact, I don't even need that patch for the code to work, but i thought it was a good idea. I'll continue to set this up. Let me know when/how/if you want it shaped to go into the tree somewhere. Make all query classes serializable with Jackson, and provide a trivial query parser to consume them Key: LUCENE-4012 URL: https://issues.apache.org/jira/browse/LUCENE-4012 Project: Lucene - Java Issue Type: New Feature Components: core/queryparser Affects Versions: 4.0 Reporter: Benson Margulies Attachments: bq.patch I started off on LUCENE-4004 wanting to use DisjunctionMaxQuery via a parser. However, this wasn't really because I thought that human beans should be improvisationally composing such thing. My real goal was to concoct a query tree over *here*, and then serialize it to send to Solr over *there*. It occurs to me that if the Xml parser is pretty good for this, JSON would be better. It further occurs to me that the query classes may already all work with Jackson, and, if they don't, the required tweaks will be quite small. By allowing Jackson to write out class names as needed, you get the ability to serialize *any* query, so long as the other side has the classes in class path. A trifle verbose, but not as verbose as XML, and furthermore squishable (though not in a URL) via SMILE or BSON. So, the goal of this JIRA is to accumulate tweaks to the query classes to make them more 'bean pattern'. An alternative would be Jackson annotations. However, I suspect that folks would be happier to minimize the level of coupling here; in the extreme, the trivial parser could live in contrib if no one wants a dependency, even optional, on Jackson itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3426) Verbose error message when Log4j is enabled
[
https://issues.apache.org/jira/browse/SOLR-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan McKinley updated SOLR-3426:
Priority: Trivial (was: Major)
Assignee: Ryan McKinley
Issue Type: Improvement (was: Bug)
Summary: Verbose error message when Log4j is enabled (was:
CoreContainer goes looking for nonexistent Log4j class)
Verbose error message when Log4j is enabled
---
Key: SOLR-3426
URL: https://issues.apache.org/jira/browse/SOLR-3426
Project: Solr
Issue Type: Improvement
Affects Versions: 4.0
Reporter: Benson Margulies
Assignee: Ryan McKinley
Priority: Trivial
Attachments: solr-3426.patch
The code quoted here from CoreContainer.java ends up looking for a class
named Log4j instead of Log4jWatcher. Neither name exists in the tree.
// Now load the framework
if(fname!=null) {
if(JUL.equalsIgnoreCase(fname)) {
logging = new JulWatcher(slf4jImpl);
}
//else if( Log4j.equals(fname) ) {
// logging = new Log4jWatcher(slf4jImpl);
//}
else {
try {
logging = loader.newInstance(fname, LogWatcher.class);
}
catch (Throwable e) {
log.warn(Unable to load LogWatcher, e);
}
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENENET-488) Can't open IndexReader, get OutOFMemory Exception
[ https://issues.apache.org/jira/browse/LUCENENET-488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265916#comment-13265916 ] Simon Svensson commented on LUCENENET-488: -- The following may be off since I don't know the inner technical workings of Lucene.Net. All terms in your index is read into an in-memory index when opening an IndexReader. The termInfosIndexDivisor tells the IndexReader instance to read every n-th term into this index. The default value, 1, will cause every term to be loaded into memory. Using termIndexIndexDivisor=2 means that you'll read every second term into memory, theoretically halving the required memory size. Your value, 10, would only consume a tenth of the memory compared to termIndexDivisor=1. This comes to a price; as 9 out of 10 terms are not cached in memory they take longer time to retrieve. This is done in many cases, like a new TermQuery(f, test). It needs to seek to the indexed term, then iterate forward until it matches the correct term. This could be, if teargas was the indexed term; teargas technicians tegument teleconference temporal tenotomy teocalli terbium test. Instead of being able to directly seek to the term, we now seek to a term before, and iterate the list for another 8 terms. (It would still go faster than the time it took for me to find odd example words...) I've never measured this, but I doubt that low numbers will cause much trouble. Any term except teargas would need to read the term information from disk, and this disk read will [probably] end up in the file system cache. I can see a problem if you have numbers high enough causing a second disk read, but at what value of termInfosIndexDivisor this happens is system-dependent. The size of the disk reads, the amount of data per term, etc, would affect this. I guess you could use a low-level monitoring tool (Process Monitor?) to see every read if you really want to find the perfect number. I believe this bug report can be closed as invalid; it was a case of default values that did not work out for 200 GiB indexes. Do you agree on this, Steven? Can't open IndexReader, get OutOFMemory Exception - Key: LUCENENET-488 URL: https://issues.apache.org/jira/browse/LUCENENET-488 Project: Lucene.Net Issue Type: Bug Components: Lucene.Net Core Affects Versions: Lucene.Net 2.9.4g Environment: Windows server 2008R2 Reporter: Steven Have build a large database with ~1Bn records (2 items per document) it has size 200GB on disk. I managed to write the indexe by chunking into 100,000 blocks as I ended up with some threading issues (another bug submission). Anyway the index is built but I can't open it and get a memory exception (process explorer gets to 1.5GB allocated before it dies but not sure how reliable that is, but do know there is plenty more RAM left on the box). Stack trace below: System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown. at Lucene.Net.Index.TermInfosReader..ctor(Directory dir, String seg, FieldInf os fis, Int32 readBufferSize, Int32 indexDivisor) at Lucene.Net.Index.SegmentReader.CoreReaders..ctor(SegmentReader origInstanc e, Directory dir, SegmentInfo si, Int32 readBufferSize, Int32 termsIndexDivisor) at Lucene.Net.Index.SegmentReader.Get(Boolean readOnly, Directory dir, Segmen tInfo si, Int32 readBufferSize, Boolean doOpenStores, Int32 termInfosIndexDiviso r) at Lucene.Net.Index.SegmentReader.Get(Boolean readOnly, SegmentInfo si, Int32 termInfosIndexDivisor) at Lucene.Net.Index.DirectoryReader..ctor(Directory directory, SegmentInfos s is, IndexDeletionPolicy deletionPolicy, Boolean readOnly, Int32 termInfosIndexDi visor) at Lucene.Net.Index.DirectoryReader.c__DisplayClass1.Openb__0(String segm entFileName) at Lucene.Net.Index.SegmentInfos.FindSegmentsFile.Run(IndexCommit commit) at Lucene.Net.Index.DirectoryReader.Open(Directory directory, IndexDeletionPo licy deletionPolicy, IndexCommit commit, Boolean readOnly, Int32 termInfosIndexD ivisor) at Lucene.Net.Index.IndexReader.Open(String path, Boolean readOnly) at Lucene.Net.Demo.SearchFiles.Main(String[] args) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (SOLR-3426) Verbose error message when Log4j is enabled
[
https://issues.apache.org/jira/browse/SOLR-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan McKinley resolved SOLR-3426.
-
Resolution: Fixed
removed the error trace in revision: 1332752
Verbose error message when Log4j is enabled
---
Key: SOLR-3426
URL: https://issues.apache.org/jira/browse/SOLR-3426
Project: Solr
Issue Type: Improvement
Affects Versions: 4.0
Reporter: Benson Margulies
Assignee: Ryan McKinley
Priority: Trivial
Attachments: solr-3426.patch
The code quoted here from CoreContainer.java ends up looking for a class
named Log4j instead of Log4jWatcher. Neither name exists in the tree.
// Now load the framework
if(fname!=null) {
if(JUL.equalsIgnoreCase(fname)) {
logging = new JulWatcher(slf4jImpl);
}
//else if( Log4j.equals(fname) ) {
// logging = new Log4jWatcher(slf4jImpl);
//}
else {
try {
logging = loader.newInstance(fname, LogWatcher.class);
}
catch (Throwable e) {
log.warn(Unable to load LogWatcher, e);
}
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-4012) Make all query classes serializable with Jackson, and provide a trivial query parser to consume them
[ https://issues.apache.org/jira/browse/LUCENE-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265919#comment-13265919 ] Robert Muir commented on LUCENE-4012: - I think if it works for you, just iterate in your github and ping the issue when you make progress? Otherwise we worry too much about where/how the code sits when that isn't really so important at this stage. As far as final integration, I think there are a number of ways to do it but its really unrelated to making progress here. One suggestion might be to split queryparser/ module like analyzers/ so we have: * classic/ [including things based on it: complexPhrase, ext, analyzing] * flexible/ [including precedence which is based on it] * xml/ * json/ This could probably make things less confusing, as currently queryparser/ is a mix of different frameworks with different dependencies (e.g. xml depends on queries/ and sandbox/, but the others dont, and json will depend on jackson and maybe other stuff, etc, etc). And then finally, probably a followup issue to do solr integration (i'm more fuzzy on that). Make all query classes serializable with Jackson, and provide a trivial query parser to consume them Key: LUCENE-4012 URL: https://issues.apache.org/jira/browse/LUCENE-4012 Project: Lucene - Java Issue Type: New Feature Components: core/queryparser Affects Versions: 4.0 Reporter: Benson Margulies Attachments: bq.patch I started off on LUCENE-4004 wanting to use DisjunctionMaxQuery via a parser. However, this wasn't really because I thought that human beans should be improvisationally composing such thing. My real goal was to concoct a query tree over *here*, and then serialize it to send to Solr over *there*. It occurs to me that if the Xml parser is pretty good for this, JSON would be better. It further occurs to me that the query classes may already all work with Jackson, and, if they don't, the required tweaks will be quite small. By allowing Jackson to write out class names as needed, you get the ability to serialize *any* query, so long as the other side has the classes in class path. A trifle verbose, but not as verbose as XML, and furthermore squishable (though not in a URL) via SMILE or BSON. So, the goal of this JIRA is to accumulate tweaks to the query classes to make them more 'bean pattern'. An alternative would be Jackson annotations. However, I suspect that folks would be happier to minimize the level of coupling here; in the extreme, the trivial parser could live in contrib if no one wants a dependency, even optional, on Jackson itself. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3892) Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, Simple9/16/64, etc.)
[ https://issues.apache.org/jira/browse/LUCENE-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265950#comment-13265950 ] Han Jiang commented on LUCENE-3892: --- A postings format named VSEncoding also seems promising! It is available here: http://integerencoding.isti.cnr.it/ And license compatible: https://github.com/maropu/integer_encoding_library/blob/master/LICENSE Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, Simple9/16/64, etc.) - Key: LUCENE-3892 URL: https://issues.apache.org/jira/browse/LUCENE-3892 Project: Lucene - Java Issue Type: Improvement Reporter: Michael McCandless Labels: gsoc2012, lucene-gsoc-12 Fix For: 4.0 Attachments: LUCENE-3892_settings.patch, LUCENE-3892_settings.patch On the flex branch we explored a number of possible intblock encodings, but for whatever reason never brought them to completion. There are still a number of issues opened with patches in different states. Initial results (based on prototype) were excellent (see http://blog.mikemccandless.com/2010/08/lucene-performance-with-pfordelta-codec.html ). I think this would make a good GSoC project. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (LUCENE-3830) MappingCharFilter could be improved by switching to an FST.
[ https://issues.apache.org/jira/browse/LUCENE-3830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13265964#comment-13265964 ] Michael McCandless commented on LUCENE-3830: bq. Just laughing a little bit to stumble on this: Only 23 pages! bq. Javadoc is probably wrong (sequence of bytes?). Thanks Dawid, I'll fix. bq. I'd probably implement it based on the stack of currently active tokens moving through the fst – this way you know when you have a greedy first longest match simply by looking at the set of active tokens when a match fires instead of restarting from each position. Hmm I don't quite understand ... can you describe more? If there's something simple in between what MappingCharFilter (and my patch) does today (restart on each position), and the 23 page paper, that would be nice :) MappingCharFilter could be improved by switching to an FST. --- Key: LUCENE-3830 URL: https://issues.apache.org/jira/browse/LUCENE-3830 Project: Lucene - Java Issue Type: Improvement Reporter: Dawid Weiss Assignee: Michael McCandless Priority: Minor Labels: gsoc2012, lucene-gsoc-12 Fix For: 4.0 Attachments: LUCENE-3830.patch MappingCharFilter stores an overly complex tree-like structure for matching input patterns. The input is a union of fixed strings mapped to a set of fixed strings; an fst matcher would be ideal here and provide both memory and speed improvement I bet. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3406) Support grouped range and query facets.
[ https://issues.apache.org/jira/browse/SOLR-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266046#comment-13266046 ] David commented on SOLR-3406: - will this be committed and released in the alpha? Support grouped range and query facets. --- Key: SOLR-3406 URL: https://issues.apache.org/jira/browse/SOLR-3406 Project: Solr Issue Type: New Feature Reporter: David Priority: Critical Fix For: 4.0 Attachments: SOLR-2898-backport.patch, SOLR-3406.patch, SOLR-3406.patch Original Estimate: 504h Remaining Estimate: 504h Need the ability to support grouped range and query facets. Grouped facet fields have already been implemented in SOLR-2898 but we still need the ability to compute grouped range and query facets. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-2794) change the default of hl.phraseLimit to 5000
[ https://issues.apache.org/jira/browse/SOLR-2794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zarni updated SOLR-2794: Attachment: SOLR-2794.patch change the default of hl.phraseLimit to 5000 Key: SOLR-2794 URL: https://issues.apache.org/jira/browse/SOLR-2794 Project: Solr Issue Type: Improvement Components: highlighter Affects Versions: 3.4, 4.0 Reporter: Koji Sekiguchi Priority: Trivial Attachments: SOLR-2794.patch David Smiley suggested that the default of hl.phraseLimit parameter should be 5000 (5000 was suggested by Mike). See LUCENE-3234. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Created] (SOLR-3427) Faceting under some conditions throws NPE
Jamie Johnson created SOLR-3427: --- Summary: Faceting under some conditions throws NPE Key: SOLR-3427 URL: https://issues.apache.org/jira/browse/SOLR-3427 Project: Solr Issue Type: Bug Components: SearchComponents - other Affects Versions: 4.0 Reporter: Jamie Johnson Under some circumstances faceting in Solr throws the following NPE May 1, 2012 8:48:52 PM org.apache.solr.common.SolrException log SEVERE: java.lang.NullPointerException at org.apache.lucene.index.DocTermOrds.lookupTerm(DocTermOrds.java:807) at org.apache.solr.request.UnInvertedField.getTermValue(UnInvertedField.java:636) at org.apache.solr.request.UnInvertedField.getCounts(UnInvertedField.java:411) at org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:300) at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:396) at org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:205) at org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:81) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:204) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1550) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:442) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:263) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) at org.eclipse.jetty.server.Server.handle(Server.java:351) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:857) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534) at java.lang.Thread.run(Thread.java:662) I've noticed this happening after doing deletes. When I've seen this issue before an optimize has made the issue go away. I believe this may also be related to the describe here: http://stackoverflow.com/questions/10124055/solr-faceted-search-throws-nullpointerexception-with-http-500-status I have been unable to reproduce this in a test but this has come up at least 3 times in our environments. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3296) Explore alternatives to Commons CSV
[
https://issues.apache.org/jira/browse/SOLR-3296?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266253#comment-13266253
]
Chris Male commented on SOLR-3296:
--
Funny you ask,
When I submitted the bundle I received the same 'Staging Completed'
notification as I did when I submitted langdetect. A relevant snippet from the
email:
{noformat}
The following artifacts have been staged to the Central Bundles-102
(u:MYUSERNAME, a:122.59.251.231) repository.
{noformat}
with all the appropriate artifacts listed.
Just today I received a 'Staging Repository Dropped' notification with only the
following information:
{noformat}
The Central Bundles-102 (u:MYUSERNAME, a:122.59.251.231) staging repository has
been dropped.
{noformat}
When langdetect was accepted, I received a 'Promotion Completed' email, so I
think this is a bad sign but I've received no information about why it was
rejected and don't know how to proceed further.
Explore alternatives to Commons CSV
---
Key: SOLR-3296
URL: https://issues.apache.org/jira/browse/SOLR-3296
Project: Solr
Issue Type: Improvement
Components: Build
Reporter: Chris Male
Attachments: SOLR-3295-CSV-tests.patch, SOLR-3296_noggit.patch,
pom.xml, pom.xml
In LUCENE-3930 we're implementing some less than ideal solutions to make
available the unreleased version of commons-csv. We could remove these
solutions if we didn't rely on this lib. So I think we should explore
alternatives.
I think [opencsv|http://opencsv.sourceforge.net/] is an alternative to
consider, I've used it in many commercial projects. Bizarrely Commons-CSV's
website says that Opencsv uses a BSD license, but this isn't the case,
OpenCSV uses ASL2.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
RE: svn commit: r1332886 - /lucene/dev/trunk/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/TestUAX29URLEmailTokenizer.java
Thanks, Robert. - Steve
-Original Message-
From: rm...@apache.org [mailto:rm...@apache.org]
Sent: Tuesday, May 01, 2012 8:11 PM
To: comm...@lucene.apache.org
Subject: svn commit: r1332886 -
/lucene/dev/trunk/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/TestUAX29URLEmailTokenizer.java
Author: rmuir
Date: Wed May 2 00:10:55 2012
New Revision: 1332886
URL: http://svn.apache.org/viewvc?rev=1332886view=rev
Log:
LUCENE-3880: add backwards test
Modified:
lucene/dev/trunk/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/TestUAX29URLEmailTokenizer.java
Modified:
lucene/dev/trunk/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/TestUAX29URLEmailTokenizer.java
URL:
http://svn.apache.org/viewvc/lucene/dev/trunk/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/TestUAX29URLEmailTokenizer.java?rev=1332886r1=1332885r2=1332886view=diff
==
---
lucene/dev/trunk/lucene/analysis/common/src/test/org/apache/lucene/analysis/core/TestUAX29URLEmailTokenizer.java
(original)
+++ lucene/dev/trunk/lucene/analysis/common/src/test/org/apache/lucene/a
+++ nalysis/core/TestUAX29URLEmailTokenizer.java Wed May 2 00:10:55
+++ 2012
@@ -470,6 +470,21 @@ public class TestUAX29URLEmailTokenizer
checkOneTerm(a, 壹ã, 壹); // ideographic Bug
checkOneTerm(a, ìã, ìã); // hangul Works
}
+
+ // LUCENE-3880
+ /** @deprecated remove this and sophisticated backwards layer in 5.0
+ */ @Deprecated public void testMailtoBackwards() throws Exception {
+Analyzer a = new Analyzer() {
+ @Override
+ protected TokenStreamComponents createComponents(String fieldName,
Reader reader) {
+Tokenizer tokenizer = new UAX29URLEmailTokenizer(Version.LUCENE_34,
reader);
+return new TokenStreamComponents(tokenizer);
+ }
+};
+assertAnalyzesTo(a, mailto:t...@example.org;,
+new String[] { mailto:test;, example.org }); }
/** blast some random strings through the analyzer */
public void testRandomStrings() throws Exception {
[jira] [Created] (LUCENE-4027) 3.x codec should have its own skipper
Robert Muir created LUCENE-4027: --- Summary: 3.x codec should have its own skipper Key: LUCENE-4027 URL: https://issues.apache.org/jira/browse/LUCENE-4027 Project: Lucene - Java Issue Type: Bug Reporter: Robert Muir Currently it uses the 4.0 impl..., worse the PreFlexRW writes with 4.0 impl too. This is dangerous. we changed the format even to support offsets (though 3.x should be unaffacted by that), but just in general I don't even trust that testbackwardscompatibility has big enough indexes to ever use any skipdata to detect if we screw up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-4027) 3.x codec should have its own skipper
[ https://issues.apache.org/jira/browse/LUCENE-4027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Muir updated LUCENE-4027: Attachment: LUCENE-4027.patch 3.x codec should have its own skipper - Key: LUCENE-4027 URL: https://issues.apache.org/jira/browse/LUCENE-4027 Project: Lucene - Java Issue Type: Bug Reporter: Robert Muir Attachments: LUCENE-4027.patch Currently it uses the 4.0 impl..., worse the PreFlexRW writes with 4.0 impl too. This is dangerous. we changed the format even to support offsets (though 3.x should be unaffacted by that), but just in general I don't even trust that testbackwardscompatibility has big enough indexes to ever use any skipdata to detect if we screw up. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-3427) Faceting under some conditions throws NPE
[ https://issues.apache.org/jira/browse/SOLR-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266271#comment-13266271 ] Jamie Johnson commented on SOLR-3427: - I believe the issue is happening on a multivalued field where the document which contributed that field has been deleted. If I can narrow it down to the field that is causing the issue in my index what information can I provide to you that would give you more insight into what is happening? Faceting under some conditions throws NPE - Key: SOLR-3427 URL: https://issues.apache.org/jira/browse/SOLR-3427 Project: Solr Issue Type: Bug Components: SearchComponents - other Affects Versions: 4.0 Reporter: Jamie Johnson Under some circumstances faceting in Solr throws the following NPE May 1, 2012 8:48:52 PM org.apache.solr.common.SolrException log SEVERE: java.lang.NullPointerException at org.apache.lucene.index.DocTermOrds.lookupTerm(DocTermOrds.java:807) at org.apache.solr.request.UnInvertedField.getTermValue(UnInvertedField.java:636) at org.apache.solr.request.UnInvertedField.getCounts(UnInvertedField.java:411) at org.apache.solr.request.SimpleFacets.getTermCounts(SimpleFacets.java:300) at org.apache.solr.request.SimpleFacets.getFacetFieldCounts(SimpleFacets.java:396) at org.apache.solr.request.SimpleFacets.getFacetCounts(SimpleFacets.java:205) at org.apache.solr.handler.component.FacetComponent.process(FacetComponent.java:81) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:204) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1550) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:442) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:263) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) at org.eclipse.jetty.server.Server.handle(Server.java:351) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47) at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:857) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534) at java.lang.Thread.run(Thread.java:662) I've noticed this happening after doing deletes. When I've seen this issue before an optimize has made the issue go away. I believe this may also be related to the describe here: http://stackoverflow.com/questions/10124055/solr-faceted-search-throws-nullpointerexception-with-http-500-status I have been unable to reproduce this in a test but this has come up at least 3 times in our environments. -- This message is automatically generated by JIRA. If you think it was sent
[jira] [Commented] (LUCENE-2510) migrate solr analysis factories to analyzers module
[ https://issues.apache.org/jira/browse/LUCENE-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13266280#comment-13266280 ] Chris Male commented on LUCENE-2510: Actually I think I've been a little naive with backwards compatibility here. I think I need to ensure that any user created Factory implementations continue to work and existing schemas continue to load. Otherwise upgrading to Solr 4 is going to be an epic hassle. migrate solr analysis factories to analyzers module --- Key: LUCENE-2510 URL: https://issues.apache.org/jira/browse/LUCENE-2510 Project: Lucene - Java Issue Type: Task Components: modules/analysis Affects Versions: 4.0 Reporter: Robert Muir Fix For: 4.0 Attachments: LUCENE-2510-parent-classes.patch, LUCENE-2510.patch, LUCENE-2510.patch, LUCENE-2510.patch In LUCENE-2413 all TokenStreams were consolidated into the analyzers module. This is a good step, but I think the next step is to put the Solr factories into the analyzers module, too. This would make analyzers artifacts plugins to both lucene and solr, with benefits such as: * users could use the old analyzers module with solr, too. This is a good step to use real library versions instead of Version for backwards compat. * analyzers modules such as smartcn and icu, that aren't currently available to solr users due to large file sizes or dependencies, would be simple optional plugins to solr and easily available to users that want them. Rough sketch in this thread: http://www.lucidimagination.com/search/document/3465a0e55ba94d58/solr_and_analyzers_module Practically, I havent looked much and don't really have a plan for how this will work yet, so ideas are very welcome. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-2510) migrate solr analysis factories to analyzers module
[
https://issues.apache.org/jira/browse/LUCENE-2510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Male updated LUCENE-2510:
---
Attachment: LUCENE-2510-resourceloader-bw.patch
Patch which does some backwards compat work for ResourceLoader/Aware.
Deprecated versions of both are added in their old locations and they extend
from those in the new locations. {{SolrResourceLoader}} implements the
deprecated ResourceLoader so it can continue to be used with Factories
expecting the old ResourceLoader. It also includes a check for plugins using
the old ResourceLoaderAware and logs a warning recommending that they are
upgraded.
migrate solr analysis factories to analyzers module
---
Key: LUCENE-2510
URL: https://issues.apache.org/jira/browse/LUCENE-2510
Project: Lucene - Java
Issue Type: Task
Components: modules/analysis
Affects Versions: 4.0
Reporter: Robert Muir
Fix For: 4.0
Attachments: LUCENE-2510-parent-classes.patch,
LUCENE-2510-resourceloader-bw.patch, LUCENE-2510.patch, LUCENE-2510.patch,
LUCENE-2510.patch
In LUCENE-2413 all TokenStreams were consolidated into the analyzers module.
This is a good step, but I think the next step is to put the Solr factories
into the analyzers module, too.
This would make analyzers artifacts plugins to both lucene and solr, with
benefits such as:
* users could use the old analyzers module with solr, too. This is a good
step to use real library versions instead of Version for backwards compat.
* analyzers modules such as smartcn and icu, that aren't currently available
to solr users due to large file sizes or dependencies, would be simple
optional plugins to solr and easily available to users that want them.
Rough sketch in this thread:
http://www.lucidimagination.com/search/document/3465a0e55ba94d58/solr_and_analyzers_module
Practically, I havent looked much and don't really have a plan for how this
will work yet, so ideas are very welcome.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[JENKINS] Lucene-trunk - Build # 1917 - Failure
Build: https://builds.apache.org/job/Lucene-trunk/1917/ 1 tests failed. REGRESSION: org.apache.lucene.index.TestNorms.testNormsNotPresent Error Message: Java heap space Stack Trace: java.lang.OutOfMemoryError: Java heap space at __randomizedtesting.SeedInfo.seed([DD43C1EE3741DA4F:EB6A38A1F62C0B06]:0) at org.apache.lucene.util.ArrayUtil.grow(ArrayUtil.java:338) at org.apache.lucene.util.fst.FST$BytesWriter.setPosWrite(FST.java:1213) at org.apache.lucene.util.fst.FST.pack(FST.java:1701) at org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish(MemoryPostingsFormat.java:270) at org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:207) at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:65) at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:356) at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:115) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3382) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3004) at org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeScheduler.java:37) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1644) at org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1638) at org.apache.lucene.index.IndexWriter.prepareCommit(IndexWriter.java:2465) at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2548) at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2530) at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2514) at org.apache.lucene.index.RandomIndexWriter.commit(RandomIndexWriter.java:324) at org.apache.lucene.index.TestNorms.buildIndex(TestNorms.java:195) at org.apache.lucene.index.TestNorms.testNormsNotPresent(TestNorms.java:124) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1913) at com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(RandomizedRunner.java:131) at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:805) at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:866) at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:880) at org.apache.lucene.util.LuceneTestCase$SubclassSetupTeardownRule$1.evaluate(LuceneTestCase.java:770) at org.apache.lucene.util.LuceneTestCase$InternalSetupTeardownRule$1.evaluate(LuceneTestCase.java:692) at org.apache.lucene.util.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:69) Build Log (for compile errors): [...truncated 18389 lines...] - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (LUCENE-3973) Incorporate PMD / FindBugs
[
https://issues.apache.org/jira/browse/LUCENE-3973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Male updated LUCENE-3973:
---
Attachment: LUCENE-3973.patch
Updated patch which includes:
- Moved listing of the rules to their own ruleset file, which will allow us to
customize them more and so I don't have to keep making changes to
common-build.xml
- Added full support for solr and top-level.
Incorporate PMD / FindBugs
--
Key: LUCENE-3973
URL: https://issues.apache.org/jira/browse/LUCENE-3973
Project: Lucene - Java
Issue Type: Improvement
Components: general/build
Reporter: Chris Male
Attachments: LUCENE-3973.patch, LUCENE-3973.patch, LUCENE-3973.patch,
LUCENE-3973.patch, LUCENE-3973.patch, core.html, solr-core.html
This has been touched on a few times over the years. Having static analysis
as part of our build seems like a big win. For example, we could use PMD to
look at {{System.out.println}} statements like discussed in LUCENE-3877 and
we could possibly incorporate the nocommit / @author checks as well.
There are a few things to work out as part of this:
- Should we use both PMD and FindBugs or just one of them? They look at code
from different perspectives (bytecode vs source code) and target different
issues. At the moment I'm in favour of trying both but that might be too
heavy handed for our needs.
- What checks should we use? There's no point having the analysis if it's
going to raise too many false-positives or problems we don't deem
problematic.
- How should the analysis be integrated in our build? Need to work out when
the analysis should run, how it should be incorporated in Ant and/or Maven,
what impact errors should have.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Updated] (SOLR-3424) PhoneticFilterFactory threadsafety bug
[ https://issues.apache.org/jira/browse/SOLR-3424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Smiley updated SOLR-3424: --- Attachment: SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch My new patch creates initializes the encoder per call to create(). The testSpeed() tests seem about the same. I also added a test for the maxCodeLen arg, and i documented it too. PhoneticFilterFactory threadsafety bug -- Key: SOLR-3424 URL: https://issues.apache.org/jira/browse/SOLR-3424 Project: Solr Issue Type: Bug Components: Schema and Analysis Affects Versions: 3.6, 4.0 Reporter: David Smiley Assignee: David Smiley Priority: Minor Fix For: 4.0 Attachments: SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch, SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch, SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch, SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch, SOLR-3424_PhoneticFilterFactory_threadsafety_bug.patch PhoneticFilterFactory has a static HashMap registry mapping an encoder name to an implementation. There is a ReentrantLock used when the map is modified (when the encoder config specifies a class name). However, this map, which can be accessed by multiple indexing threads, isn't guarded on any of the reads, which isn't just the common path but also the error messages which dump the registry into the error message. I realize the likelihood of a problem is extremely slim, but a bug's a bug. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
