CommonGramsQueryFilter is not safe for a graph, such as the one here coming from ShingleFilter.
I am not sure how easy this would be to fix: the problem is essentially just like having two ShingleFilters in your analysis chain. I suspect this gives strange results as well. On Tue, Nov 13, 2012 at 10:54 PM, Policeman Jenkins Server <[email protected]> wrote: > Build: http://jenkins.sd-datasolutions.de/job/Lucene-Solr-trunk-Windows/1614/ > Java: 32bit/jdk1.6.0_37 -server -XX:+UseSerialGC > > 1 tests failed. > REGRESSION: org.apache.lucene.analysis.core.TestRandomChains.testRandomChains > > Error Message: > stage 3: inconsistent endOffset at pos=5: 15 vs 10; token=ޞ i ivqk_i > > Stack Trace: > java.lang.IllegalStateException: stage 3: inconsistent endOffset at pos=5: 15 > vs 10; token=ޞ i ivqk_i > at > __randomizedtesting.SeedInfo.seed([5D0182766DE523A4:60E0AB172AF73E64]:0) > at > org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:135) > at > org.apache.lucene.analysis.hi.HindiNormalizationFilter.incrementToken(HindiNormalizationFilter.java:51) > at > org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:78) > at > org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:632) > at > org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:542) > at > org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:443) > at > org.apache.lucene.analysis.core.TestRandomChains.testRandomChains(TestRandomChains.java:859) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > at java.lang.reflect.Method.invoke(Method.java:597) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787) > at > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) > at > org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442) > at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682) > at > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693) > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) > at > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) > at > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) > at java.lang.Thread.run(Thread.java:662) > > > > > Build Log: > [...truncated 3859 lines...] > [junit4:junit4] Suite: org.apache.lucene.analysis.core.TestRandomChains > [junit4:junit4] 2> TEST FAIL: useCharFilter=false text='\ue726 > \u06715\u01b8\ue6ac\u8e51\u079e i ivqk |\u0528 ' > [junit4:junit4] 2> Exception from random analyzer: > [junit4:junit4] 2> charfilters= > [junit4:junit4] 2> tokenizer= > [junit4:junit4] 2> > org.apache.lucene.analysis.standard.UAX29URLEmailTokenizer(LUCENE_50, > org.apache.lucene.util.AttributeSource$AttributeFactory$DefaultAttributeFactory@1cc6a6a, > > org.apache.lucene.analysis.core.TestRandomChains$CheckThatYouDidntReadAnythingReaderWrapper@1a2ac94) > [junit4:junit4] 2> filters= > [junit4:junit4] 2> > org.apache.lucene.analysis.shingle.ShingleFilter(org.apache.lucene.analysis.ValidatingTokenFilter@6c11df8, > 58) > [junit4:junit4] 2> > org.apache.lucene.analysis.ar.ArabicNormalizationFilter(org.apache.lucene.analysis.ValidatingTokenFilter@6c11df8) > [junit4:junit4] 2> > org.apache.lucene.analysis.commongrams.CommonGramsQueryFilter(org.apache.lucene.analysis.commongrams.CommonGramsFilter@6c11df8) > [junit4:junit4] 2> > org.apache.lucene.analysis.hi.HindiNormalizationFilter(org.apache.lucene.analysis.ValidatingTokenFilter@d162a12d) > [junit4:junit4] 2> offsetsAreCorrect=true > [junit4:junit4] 2> NOTE: reproduce with: ant test > -Dtestcase=TestRandomChains -Dtests.method=testRandomChains > -Dtests.seed=5D0182766DE523A4 -Dtests.slow=true -Dtests.locale=be_BY > -Dtests.timezone=Australia/Lord_Howe -Dtests.file.encoding=UTF-8 > [junit4:junit4] ERROR 3.15s | TestRandomChains.testRandomChains <<< > [junit4:junit4] > Throwable #1: java.lang.IllegalStateException: stage 3: > inconsistent endOffset at pos=5: 15 vs 10; token=? i ivqk_i > [junit4:junit4] > at > __randomizedtesting.SeedInfo.seed([5D0182766DE523A4:60E0AB172AF73E64]:0) > [junit4:junit4] > at > org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:135) > [junit4:junit4] > at > org.apache.lucene.analysis.hi.HindiNormalizationFilter.incrementToken(HindiNormalizationFilter.java:51) > [junit4:junit4] > at > org.apache.lucene.analysis.ValidatingTokenFilter.incrementToken(ValidatingTokenFilter.java:78) > [junit4:junit4] > at > org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:632) > [junit4:junit4] > at > org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:542) > [junit4:junit4] > at > org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:443) > [junit4:junit4] > at > org.apache.lucene.analysis.core.TestRandomChains.testRandomChains(TestRandomChains.java:859) > [junit4:junit4] > at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > [junit4:junit4] > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > [junit4:junit4] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > [junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:597) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) > [junit4:junit4] > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693) > [junit4:junit4] > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) > [junit4:junit4] > at > org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > [junit4:junit4] > at > com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) > [junit4:junit4] > at java.lang.Thread.run(Thread.java:662) > [junit4:junit4] 2> NOTE: test params are: codec=Lucene41: > {dummy=MockVariableIntBlock(baseBlockSize=31)}, sim=DefaultSimilarity, > locale=be_BY, timezone=Australia/Lord_Howe > [junit4:junit4] 2> NOTE: Windows 7 6.1 x86/Sun Microsystems Inc. 1.6.0_37 > (32-bit)/cpus=2,threads=1,free=19381432,total=47677440 > [junit4:junit4] 2> NOTE: All tests run in this JVM: > [TestShingleFilterFactory, TestPortugueseStemFilter, HunspellStemFilterTest, > TestStopAnalyzer, TestAllAnalyzersHaveFactories, > TestItalianLightStemFilterFactory, QueryAutoStopWordAnalyzerTest, > TestUAX29URLEmailAnalyzer, TestDelimitedPayloadTokenFilterFactory, > TestBugInSomething, TestFrenchLightStemFilterFactory, TestDuelingAnalyzers, > CommonGramsFilterTest, TestItalianAnalyzer, > TestDictionaryCompoundWordTokenFilterFactory, TestBasqueAnalyzer, > TestGreekStemmer, TestCzechStemmer, TestCharArrayMap, TestKeepWordFilter, > TestIrishAnalyzer, TestIndicNormalizer, NGramTokenizerTest, > TestRemoveDuplicatesTokenFilter, TestLatvianAnalyzer, > TestLengthFilterFactory, TestHindiNormalizer, TestPortugueseAnalyzer, > TestSolrSynonymParser, TestGreekStemFilterFactory, TestElisionFilterFactory, > ShingleFilterTest, TestNorwegianMinimalStemFilterFactory, GreekAnalyzerTest, > TestNorwegianMinimalStemFilter, TokenTypeSinkTokenizerTest, > TestEnglishMinimalStemFilterFactory, TestUAX29URLEmailTokenizerFactory, > TestWikipediaTokenizerFactory, TestKeepFilterFactory, > TestIrishLowerCaseFilter, TestGermanLightStemFilter, > TestPortugueseMinimalStemFilter, TestTrimFilterFactory, TestLengthFilter, > TestCJKBigramFilterFactory, TestGalicianMinimalStemFilter, TestCharArraySet, > TestUAX29URLEmailTokenizer, TestCJKWidthFilterFactory, > TestThaiWordFilterFactory, TestBulgarianStemFilterFactory, > TestGermanNormalizationFilter, TestSpanishAnalyzer, > TestFrenchLightStemFilter, TestPatternReplaceCharFilterFactory, > TestPatternReplaceFilter, TestTeeSinkTokenFilter, > TestLatvianStemFilterFactory, TestCommonGramsFilterFactory, > TestCharTokenizers, TestPorterStemFilter, TestIndonesianStemmer, > TestSnowballPorterFilterFactory, TestMappingCharFilter, > HTMLStripCharFilterTest, TestPatternReplaceCharFilter, > TestPorterStemFilterFactory, TestFactories, TestArabicAnalyzer, > PositionFilterTest, TestPortugueseStemFilterFactory, > TestCommonGramsQueryFilterFactory, TestFrenchAnalyzer, TestElision, > TestGalicianStemFilterFactory, TestSwedishLightStemFilterFactory, > TestFinnishLightStemFilter, TestDutchStemmer, TestSnowball, > TestNorwegianAnalyzer, TestSwedishLightStemFilter, TestFinnishAnalyzer, > TestStandardFactories, TestPatternTokenizerFactory, HunspellStemmerTest, > TestClassicAnalyzer, TestReversePathHierarchyTokenizer, TestCJKBigramFilter, > NumericPayloadTokenFilterTest, TestWordnetSynonymParser, TestCJKAnalyzer, > TestAnalysisSPILoader, TestBulgarianAnalyzer, TestKStemFilterFactory, > TestFrenchMinimalStemFilter, TestIndonesianAnalyzer, > TestCzechStemFilterFactory, EdgeNGramTokenizerTest, > ShingleAnalyzerWrapperTest, TestTurkishLowerCaseFilter, > TestGermanStemFilterFactory, TestRussianLightStemFilter, > TestCollationKeyAnalyzer, TestHindiAnalyzer, TestPortugueseLightStemFilter, > TestHTMLStripCharFilterFactory, TestSynonymMapFilter, TestAnalyzers, > TestStandardAnalyzer, TestRandomChains] > [junit4:junit4] Completed in 3.83s, 1 test, 1 error <<< FAILURES! > > [...truncated 315 lines...] > BUILD FAILED > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:337: The > following error occurred while executing this line: > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:39: The > following error occurred while executing this line: > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build.xml:520: > The following error occurred while executing this line: > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1692: > The following error occurred while executing this line: > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\analysis\build.xml:105: > The following error occurred while executing this line: > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\analysis\build.xml:38: > The following error occurred while executing this line: > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\module-build.xml:61: > The following error occurred while executing this line: > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1163: > The following error occurred while executing this line: > C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:827: > There were test failures: 213 suites, 1078 tests, 1 error, 1 ignored > > Total time: 10 minutes 37 seconds > Build step 'Invoke Ant' marked build as failure > Archiving artifacts > Recording test results > Description set: Java: 32bit/jdk1.6.0_37 -server -XX:+UseSerialGC > Email was triggered for: Failure > Sending email for trigger: Failure > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
