[
https://issues.apache.org/jira/browse/LUCENE-4482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13475658#comment-13475658
]
Uwe Schindler edited comment on LUCENE-4482 at 10/13/12 4:57 PM:
-----------------------------------------------------------------
Thanks Gil for the explanation,
Maybe it would be a good idea to provide both C4 -> Memory Management layers,
so also for plain kernels (as configuration option to the JVM like huge pages
in Oracle's). Or is your VM then only as fast as Oracle's?
bq. As we added Zing support for Ubunutu, primarily due to it's popularity with
developers, we found that kernel api signatures there change with practically
every patch, even with no semantic change. This creates some serious friction
with our current loadable module packaging and distribution choice for Ubuntu.
We are working to resolve this, either by using DKMS or some other alternative,
such that modules can continue to work or be properly updated as kernels rev up
in Ubunutu-style distros.
VirtualBOX has similar requirements and uses DKMS. They ship with a deb package
that contains the source code of their kernel module. It is rebuild on every
kernel installation automatically. DKMS itsself depends on compiler and
suggests kernel headers, so you only need to depend on "dkms" package and
"linux-headers" and almost nothing more. The Jenkins Server als runs Windows in
a virtual machine, and therefore uses their kernel module, too. As VirtualBOX's
module has similar use-cases like yours for virtualization, I hope yours does
not conflict with that one.
By the way, Ubuntu LTS is also very popular on servers!
was (Author: thetaphi):
Thanks Gil for the explanation,
Maybe it would be a good idea to provide both C4 -> Memory Management layers,
so also for plain kernels (as configuration option to the JVM like huge pages
in Oracle's). Or is your VM then only as fast as Oracle's?
bq. As we added Zing support for Ubunutu, primarily due to it's popularity with
developers, we found that kernel api signatures there change with practically
every patch, even with no semantic change. This creates some serious friction
with our current loadable module packaging and distribution choice for Ubuntu.
We are working to resolve this, either by using DKMS or some other alternative,
such that modules can continue to work or be properly updated as kernels rev up
in Ubunutu-style distros.
VirtualBOX has similar requirements and uses DKMS. They ship with a deb package
that contains the source code of their kernel module. It is rebuild on every
kernel installation automatically. DKMS itsself depends on compiler and
suggests kernel headers, so you only need to depend on "dkms" module and
"linux-headers" and almost nothing more. The Jenkins Server als runs Windows in
a virtual machine, and therefore uses their kernel module, too. As VirtualBOX's
module has similar use-cases like yours for virtualization, I hope yours does
not conflict with that one.
By the way, Ubuntu LTS is also very popular on servers!
> Likely Zing JVM bug causes failures in TestPayloadNearQuery
> -----------------------------------------------------------
>
> Key: LUCENE-4482
> URL: https://issues.apache.org/jira/browse/LUCENE-4482
> Project: Lucene - Core
> Issue Type: Bug
> Environment: Lucene trunk, rev 1397735
> Zing:
> {noformat}
> java version "1.6.0_31"
> Java(TM) SE Runtime Environment (build 1.6.0_31-6)
> Java HotSpot(TM) 64-Bit Tiered VM (build
> 1.6.0_31-ZVM_5.2.3.0-b6-product-azlinuxM-X86_64, mixed mode)
> {noformat}
> Ubuntu 12.04 LTS 3.2.0-23-generic kernel
> Reporter: Michael McCandless
> Attachments: LUCENE-4482.patch
>
>
> I dug into one of the Lucene test failures when running with Zing JVM
> (available free for open source devs...). At least one other test
> sometimes fails but I haven't dug into that yet.
> I managed to get the failure easily reproduced: with the attached
> patch, on rev 1397735 checkout, if you cd to lucene/core and run:
> {noformat}
> ant test -Dtests.jvms=1 -Dtests.seed=C3802435F5FB39D0
> -Dtests.showSuccess=true
> {noformat}
> Then you'll hit several failures in TestPayloadNearQuery, eg:
> {noformat}
> Suite: org.apache.lucene.search.payloads.TestPayloadNearQuery
> 1> FAILED
> 2> NOTE: reproduce with: ant test -Dtestcase=TestPayloadNearQuery
> -Dtests.method=test -Dtests.seed=C3802435F5FB39D0 -Dtests.slow=true
> -Dtests.locale=ga -Dtests.timezone=America/Adak -Dtests.file.encoding=US-ASCII
> ERROR 0.01s | TestPayloadNearQuery.test <<<
> > Throwable #1: java.lang.RuntimeException: overridden idfExplain method
> in TestPayloadNearQuery.BoostingSimilarity was not called
> > at
> __randomizedtesting.SeedInfo.seed([C3802435F5FB39D0:4BD41BEF5B075428]:0)
> > at
> org.apache.lucene.search.similarities.TFIDFSimilarity.computeWeight(TFIDFSimilarity.java:740)
> > at org.apache.lucene.search.spans.SpanWeight.<init>(SpanWeight.java:62)
> > at
> org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanWeight.<init>(PayloadNearQuery.java:147)
> > at
> org.apache.lucene.search.payloads.PayloadNearQuery.createWeight(PayloadNearQuery.java:75)
> > at
> org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:648)
> > at
> org.apache.lucene.search.AssertingIndexSearcher.createNormalizedWeight(AssertingIndexSearcher.java:60)
> > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:265)
> > at
> org.apache.lucene.search.payloads.TestPayloadNearQuery.test(TestPayloadNearQuery.java:146)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:597)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:737)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:773)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787)
> > at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
> > at
> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51)
> > at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> > at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> > at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> > at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> > at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:682)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693)
> > at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> > at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
> > at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
> > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:43)
> > at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
> > at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70)
> > at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358)
> > at java.lang.Thread.run(Thread.java:661)
> {noformat}
> The patch at least isolates the JVM bug even if it's not exactly a
> minimal test :) Somehow the idfExplain method, which
> is overridden in this test's BoostingSimilarity, fails to be called
> (the super.idfExplain is called instead), which leads to the test
> failures.
> The failure does not happen if you run this test in isolation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
