FieldCache goes insane in JoinUtils

2012-07-16 Thread Simon Willnauer 
Just wanna forward this to the list in the case this it serious:

simon

Changes:

[sarowe] LUCENE-4199: IntelliJ configuration: add lucene tools library
to allow compilation where the asm jar is a dependency

--
[...truncated 16387 lines...]
[junit4:junit4] Suite: org.apache.lucene.search.highlight.HighlighterTest
[junit4:junit4] Completed on J0 in 9.35s, 45 tests
[junit4:junit4]
[junit4:junit4] Suite:
org.apache.lucene.search.vectorhighlight.IndexTimeSynonymTest
[junit4:junit4] Completed on J1 in 4.23s, 17 tests
[junit4:junit4]
[junit4:junit4] Suite:
org.apache.lucene.search.vectorhighlight.SingleFragListBuilderTest
[junit4:junit4] Completed on J1 in 0.53s, 3 tests
[junit4:junit4]
[junit4:junit4] Suite:
org.apache.lucene.search.vectorhighlight.WeightedFragListBuilderTest
[junit4:junit4] Completed on J1 in 0.43s, 1 test
[junit4:junit4]
[junit4:junit4] Suite:
org.apache.lucene.search.vectorhighlight.ScoreOrderFragmentsBuilderTest
[junit4:junit4] Completed on J1 in 0.25s, 1 test
[junit4:junit4]
[junit4:junit4] Suite:
org.apache.lucene.search.vectorhighlight.SimpleBoundaryScannerTest
[junit4:junit4] Completed on J1 in 0.28s, 2 tests
[junit4:junit4]
[junit4:junit4] Suite: org.apache.lucene.search.vectorhighlight.FieldQueryTest
[junit4:junit4] Completed on J0 in 3.99s, 27 tests
[junit4:junit4]
[junit4:junit4] Suite:
org.apache.lucene.search.highlight.custom.HighlightCustomQueryTest
[junit4:junit4] Completed on J1 in 0.24s, 1 test
[junit4:junit4]
[junit4:junit4] JVM J0: 0.68 ..21.82 =21.14s
[junit4:junit4] JVM J1: 0.68 ..21.83 =21.15s
[junit4:junit4] Execution time total: 22 seconds
[junit4:junit4] Tests summary: 16 suites, 157 tests
 [echo] 5 slowest tests:
[junit4:tophints]   9.35s | org.apache.lucene.search.highlight.HighlighterTest
[junit4:tophints]   5.66s |
org.apache.lucene.search.vectorhighlight.FieldPhraseListTest
[junit4:tophints]   5.01s |
org.apache.lucene.search.vectorhighlight.SimpleFragListBuilderTest
[junit4:tophints]   4.23s |
org.apache.lucene.search.vectorhighlight.IndexTimeSynonymTest
[junit4:tophints]   3.99s |
org.apache.lucene.search.vectorhighlight.FieldQueryTest
 [echo] Building join...

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file =
builds.flonkings.com/job/Lucene-trunk-Linux-Java6-64/ws/checkout/lucene/ivy-settings.xml

resolve:

common.init:

compile-lucene-core:

jflex-uptodate-check:

jflex-notice:

javacc-uptodate-check:

javacc-notice:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file =
builds.flonkings.com/job/Lucene-trunk-Linux-Java6-64/ws/checkout/lucene/ivy-settings.xml

resolve:

init:

-clover.disable:

-clover.setup:

clover:

common.compile-core:
[javac] Compiling 1 source file to
builds.flonkings.com/job/Lucene-trunk-Linux-Java6-64/ws/checkout/lucene/build/core/classes/java

compile-core:

module-build.init:

check-grouping-uptodate:

jar-grouping:

init:

test:
 [echo] Building join...

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file =
builds.flonkings.com/job/Lucene-trunk-Linux-Java6-64/ws/checkout/lucene/ivy-settings.xml

resolve:

common.init:

compile-lucene-core:

module-build.init:

check-grouping-uptodate:

jar-grouping:

init:

compile-test:
 [echo] Building join...

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file =
builds.flonkings.com/job/Lucene-trunk-Linux-Java6-64/ws/checkout/lucene/ivy-settings.xml

resolve:

common.init:

compile-lucene-core:

module-build.init:

check-grouping-uptodate:

jar-grouping:

init:

-clover.disable:

-clover.setup:

clover:

compile-core:

compile-test-framework:

ivy-availability-check:

ivy-fail:

ivy-configure:
[ivy:configure] :: loading settings :: file =
builds.flonkings.com/job/Lucene-trunk-Linux-Java6-64/ws/checkout/lucene/ivy-settings.xml

resolve:

init:

compile-lucene-core:

compile-core:

common.compile-test:

install-junit4-taskdef:

-clover.disable:

-clover.setup:

clover:

validate:

common.test:
[mkdir] Created dir:
builds.flonkings.com/job/Lucene-trunk-Linux-Java6-64/ws/checkout/lucene/build/join/test
[junit4:junit4] JUnit4 says g'day! Master seed: 4613CDE1DF5391F
[junit4:junit4] Executing 2 suites with 2 JVMs.
[junit4:junit4] Suite: org.apache.lucene.search.join.TestBlockJoin
[junit4:junit4] Completed on J1 in 19.86s, 7 tests
[junit4:junit4]
[junit4:junit4] HEARTBEAT J0: 2012-07-15T15:30:03, no events in:
69.7s, approx. at: TestJoinUtil.testSingleValueRandomJoin
[junit4:junit4] Suite: org.apache.lucene.search.join.TestJoinUtil
[junit4:junit4] FAILURE 93.1s J0 | TestJoinUtil.testSingleValueRandomJoin
[junit4:junit4] Throwable #1: java.lang.AssertionError:
testSingleValueRandomJoin(org.apache.lucene.search.join.TestJoinUtil):
Insane FieldCache usage(s) found expected:0 but was:1
[junit4:junit4]at

Re: FieldCache goes insane in JoinUtils

2012-07-16 Thread Martijn v Groningen 
I tried to reproduce the failure on my local machine, but the test didn't fail:
ant test
-Dtestcase=TestJoinUtil -Dtests.method=testSingleValueRandomJoin
-Dtests.seed=4613CDE1DF5391F -Dtests.multiplier=3 -Dtests.nightly=true
-Dtests.slow=true
-Dtests.linedocsfile=/var/lib/jenkins/lucene-data/enwiki.random.lines.txt
-Dtests.locale=cs -Dtests.timezone=NET -Dtests.file.encoding=UTF-8

I'd expect that the test would fail on local machine as well.
Apparently there is a toplevel reader entry and a segment reader entry
in the FC for a field with the name 'from'.
This is weird since the FC is only invoked with a segment reader like this:
FieldCache.DEFAULT.getTerms(context.reader(), fromField)

I guess that I'm overlooking something...

Martijn

On 16 July 2012 11:35, Simon Willnauer sim...@apache.org wrote:
 Just wanna forward this to the list in the case this it serious:

 simon

 Changes:

 [sarowe] LUCENE-4199: IntelliJ configuration: add lucene tools library
 to allow compilation where the asm jar is a dependency

 --
 [...truncated 16387 lines...]
 [junit4:junit4] Suite: org.apache.lucene.search.highlight.HighlighterTest
 [junit4:junit4] Completed on J0 in 9.35s, 45 tests
 [junit4:junit4]
 [junit4:junit4] Suite:
 org.apache.lucene.search.vectorhighlight.IndexTimeSynonymTest
 [junit4:junit4] Completed on J1 in 4.23s, 17 tests
 [junit4:junit4]
 [junit4:junit4] Suite:
 org.apache.lucene.search.vectorhighlight.SingleFragListBuilderTest
 [junit4:junit4] Completed on J1 in 0.53s, 3 tests
 [junit4:junit4]
 [junit4:junit4] Suite:
 org.apache.lucene.search.vectorhighlight.WeightedFragListBuilderTest
 [junit4:junit4] Completed on J1 in 0.43s, 1 test
 [junit4:junit4]
 [junit4:junit4] Suite:
 org.apache.lucene.search.vectorhighlight.ScoreOrderFragmentsBuilderTest
 [junit4:junit4] Completed on J1 in 0.25s, 1 test
 [junit4:junit4]
 [junit4:junit4] Suite:
 org.apache.lucene.search.vectorhighlight.SimpleBoundaryScannerTest
 [junit4:junit4] Completed on J1 in 0.28s, 2 tests
 [junit4:junit4]
 [junit4:junit4] Suite: org.apache.lucene.search.vectorhighlight.FieldQueryTest
 [junit4:junit4] Completed on J0 in 3.99s, 27 tests
 [junit4:junit4]
 [junit4:junit4] Suite:
 org.apache.lucene.search.highlight.custom.HighlightCustomQueryTest
 [junit4:junit4] Completed on J1 in 0.24s, 1 test
 [junit4:junit4]
 [junit4:junit4] JVM J0: 0.68 ..21.82 =21.14s
 [junit4:junit4] JVM J1: 0.68 ..21.83 =21.15s
 [junit4:junit4] Execution time total: 22 seconds
 [junit4:junit4] Tests summary: 16 suites, 157 tests
  [echo] 5 slowest tests:
 [junit4:tophints]   9.35s | org.apache.lucene.search.highlight.HighlighterTest
 [junit4:tophints]   5.66s |
 org.apache.lucene.search.vectorhighlight.FieldPhraseListTest
 [junit4:tophints]   5.01s |
 org.apache.lucene.search.vectorhighlight.SimpleFragListBuilderTest
 [junit4:tophints]   4.23s |
 org.apache.lucene.search.vectorhighlight.IndexTimeSynonymTest
 [junit4:tophints]   3.99s |
 org.apache.lucene.search.vectorhighlight.FieldQueryTest
  [echo] Building join...

 ivy-availability-check:

 ivy-fail:

 ivy-configure:
 [ivy:configure] :: loading settings :: file =
 builds.flonkings.com/job/Lucene-trunk-Linux-Java6-64/ws/checkout/lucene/ivy-settings.xml

 resolve:

 common.init:

 compile-lucene-core:

 jflex-uptodate-check:

 jflex-notice:

 javacc-uptodate-check:

 javacc-notice:

 ivy-availability-check:

 ivy-fail:

 ivy-configure:
 [ivy:configure] :: loading settings :: file =
 builds.flonkings.com/job/Lucene-trunk-Linux-Java6-64/ws/checkout/lucene/ivy-settings.xml

 resolve:

 init:

 -clover.disable:

 -clover.setup:

 clover:

 common.compile-core:
 [javac] Compiling 1 source file to
 builds.flonkings.com/job/Lucene-trunk-Linux-Java6-64/ws/checkout/lucene/build/core/classes/java

 compile-core:

 module-build.init:

 check-grouping-uptodate:

 jar-grouping:

 init:

 test:
  [echo] Building join...

 ivy-availability-check:

 ivy-fail:

 ivy-configure:
 [ivy:configure] :: loading settings :: file =
 builds.flonkings.com/job/Lucene-trunk-Linux-Java6-64/ws/checkout/lucene/ivy-settings.xml

 resolve:

 common.init:

 compile-lucene-core:

 module-build.init:

 check-grouping-uptodate:

 jar-grouping:

 init:

 compile-test:
  [echo] Building join...

 ivy-availability-check:

 ivy-fail:

 ivy-configure:
 [ivy:configure] :: loading settings :: file =
 builds.flonkings.com/job/Lucene-trunk-Linux-Java6-64/ws/checkout/lucene/ivy-settings.xml

 resolve:

 common.init:

 compile-lucene-core:

 module-build.init:

 check-grouping-uptodate:

 jar-grouping:

 init:

 -clover.disable:

 -clover.setup:

 clover:

 compile-core:

 compile-test-framework:

 ivy-availability-check:

 ivy-fail:

 ivy-configure:
 [ivy:configure] :: loading settings :: file =
 builds.flonkings.com/job/Lucene-trunk-Linux-Java6-64/ws/checkout/lucene/ivy-settings.xml

 resolve:

 init:

 compile-lucene-core:

 compile-core:

 common.compile-test: