[
https://issues.apache.org/jira/browse/LUCENE-8714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Woodward updated LUCENE-8714:
----------------------------------
Description:
Jenkins found a test failure that stems from a combination of LimitHandleFS and
NoMergePolicy. There's no particular reason for BaseNormsFormatTestCase to
prevent merges, so we should just remove this IndexWriteConfig setting in the
test.
[~hossman] writes on LUCENE-8585:
This is from branch_8_0...
{code}
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestLucene70NormsFormat -Dtests.method=testFewValues
-Dtests.seed=C3613DC62817C401 -Dtests.multiplier=2 -Dtests.nightly=true
-Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es-CL
-Dtests.timezone=Asia/Anadyr -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 7.04s | TestLucene70NormsFormat.testFewValues <<<
[junit4] > Throwable #1: java.nio.file.FileSystemException:
/home/hossman/lucene/dev/lucene/build/backward-codecs/test/J0/temp/lucene.codecs.lucene70.TestLucene70NormsFormat_C3613DC62817C401-001/index-NIOFSDirectory-001/_1k.fdx:
Too many open files
[junit4] > at
__randomizedtesting.SeedInfo.seed([C3613DC62817C401:E1CC12DA3AA6A4FA]:0)
[junit4] > at
org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
[junit4] > at
org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
[junit4] > at
org.apache.lucene.mockfile.HandleTrackingFS.newFileChannel(HandleTrackingFS.java:197)
[junit4] > at
org.apache.lucene.mockfile.FilterFileSystemProvider.newFileChannel(FilterFileSystemProvider.java:202)
[junit4] > at java.nio.channels.FileChannel.open(FileChannel.java:287)
[junit4] > at java.nio.channels.FileChannel.open(FileChannel.java:335)
[junit4] > at
org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:81)
[junit4] > at
org.apache.lucene.util.LuceneTestCase.slowFileExists(LuceneTestCase.java:2801)
[junit4] > at
org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:747)
[junit4] > at
org.apache.lucene.store.Directory.openChecksumInput(Directory.java:157)
[junit4] > at
org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1069)
[junit4] > at
org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.<init>(CompressingStoredFieldsReader.java:128)
[junit4] > at
org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
[junit4] > at
org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173)
[junit4] > at
org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:126)
[junit4] > at
org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:83)
[junit4] > at
org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:66)
[junit4] > at
org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:58)
[junit4] > at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:680)
[junit4] > at
org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:81)
[junit4] > at
org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
[junit4] > at
org.apache.lucene.index.BaseNormsFormatTestCase.doTestNormsVersusDocValues(BaseNormsFormatTestCase.java:494)
[junit4] > at
org.apache.lucene.index.BaseNormsFormatTestCase.testFewValues(BaseNormsFormatTestCase.java:181)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at:
/home/hossman/lucene/dev/lucene/build/backward-codecs/test/J0/temp/lucene.codecs.lucene70.TestLucene70NormsFormat_C3613DC62817C401-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene80): {},
docValues:{}, maxPointsInLeafNode=322, maxMBSortInHeap=7.12719968491226,
sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@4d8a3cb),
locale=es-CL, timezone=Asia/Anadyr
[junit4] 2> NOTE: Linux 3.19.0-84-generic amd64/Oracle Corporation
1.8.0_144 (64-bit)/cpus=4,threads=1,free=292853816,total=321912832
[junit4] 2> NOTE: All tests run in this JVM: [TestLucene70NormsFormat]
[junit4] Completed [1/1 (1!)] in 7.48s, 1 test, 1 error <<< FAILURES!
{code}
(when running all methods of that test w/that seed, many of them fail _after_
this method, with identical exceptions, but those same methods pass in
isolation -- suggesting perhaps leaked open files? or maybe only leaked on
failure?)
Also: this test is called \{{TestLucene70NormsFormat}} but it has \{{new
Lucene80Codec()}} hardcoded in it ... which seems like a pretty big WTF?
was:Jenkins found a test failure that stems from a combination of
LimitHandleFS and NoMergePolicy. There's no particular reason for
BaseNormsFormatTestCase to prevent merges, so we should just remove this
IndexWriteConfig setting in the test.
> BaseNormsFormatTestCase should not use NoMergePolicy
> ----------------------------------------------------
>
> Key: LUCENE-8714
> URL: https://issues.apache.org/jira/browse/LUCENE-8714
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Alan Woodward
> Assignee: Alan Woodward
> Priority: Major
> Attachments: LUCENE-8714.patch
>
>
> Jenkins found a test failure that stems from a combination of LimitHandleFS
> and NoMergePolicy. There's no particular reason for BaseNormsFormatTestCase
> to prevent merges, so we should just remove this IndexWriteConfig setting in
> the test.
> [~hossman] writes on LUCENE-8585:
> This is from branch_8_0...
> {code}
> [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestLucene70NormsFormat -Dtests.method=testFewValues
> -Dtests.seed=C3613DC62817C401 -Dtests.multiplier=2 -Dtests.nightly=true
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=es-CL
> -Dtests.timezone=Asia/Anadyr -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
> [junit4] ERROR 7.04s | TestLucene70NormsFormat.testFewValues <<<
> [junit4] > Throwable #1: java.nio.file.FileSystemException:
> /home/hossman/lucene/dev/lucene/build/backward-codecs/test/J0/temp/lucene.codecs.lucene70.TestLucene70NormsFormat_C3613DC62817C401-001/index-NIOFSDirectory-001/_1k.fdx:
> Too many open files
> [junit4] > at
> __randomizedtesting.SeedInfo.seed([C3613DC62817C401:E1CC12DA3AA6A4FA]:0)
> [junit4] > at
> org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
> [junit4] > at
> org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
> [junit4] > at
> org.apache.lucene.mockfile.HandleTrackingFS.newFileChannel(HandleTrackingFS.java:197)
> [junit4] > at
> org.apache.lucene.mockfile.FilterFileSystemProvider.newFileChannel(FilterFileSystemProvider.java:202)
> [junit4] > at java.nio.channels.FileChannel.open(FileChannel.java:287)
> [junit4] > at java.nio.channels.FileChannel.open(FileChannel.java:335)
> [junit4] > at
> org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:81)
> [junit4] > at
> org.apache.lucene.util.LuceneTestCase.slowFileExists(LuceneTestCase.java:2801)
> [junit4] > at
> org.apache.lucene.store.MockDirectoryWrapper.openInput(MockDirectoryWrapper.java:747)
> [junit4] > at
> org.apache.lucene.store.Directory.openChecksumInput(Directory.java:157)
> [junit4] > at
> org.apache.lucene.store.MockDirectoryWrapper.openChecksumInput(MockDirectoryWrapper.java:1069)
> [junit4] > at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.<init>(CompressingStoredFieldsReader.java:128)
> [junit4] > at
> org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsReader(CompressingStoredFieldsFormat.java:121)
> [junit4] > at
> org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat.fieldsReader(Lucene50StoredFieldsFormat.java:173)
> [junit4] > at
> org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:126)
> [junit4] > at
> org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:83)
> [junit4] > at
> org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:66)
> [junit4] > at
> org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:58)
> [junit4] > at
> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:680)
> [junit4] > at
> org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:81)
> [junit4] > at
> org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
> [junit4] > at
> org.apache.lucene.index.BaseNormsFormatTestCase.doTestNormsVersusDocValues(BaseNormsFormatTestCase.java:494)
> [junit4] > at
> org.apache.lucene.index.BaseNormsFormatTestCase.testFewValues(BaseNormsFormatTestCase.java:181)
> [junit4] > at java.lang.Thread.run(Thread.java:748)
> [junit4] 2> NOTE: leaving temporary files on disk at:
> /home/hossman/lucene/dev/lucene/build/backward-codecs/test/J0/temp/lucene.codecs.lucene70.TestLucene70NormsFormat_C3613DC62817C401-001
> [junit4] 2> NOTE: test params are: codec=Asserting(Lucene80): {},
> docValues:{}, maxPointsInLeafNode=322, maxMBSortInHeap=7.12719968491226,
> sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@4d8a3cb),
> locale=es-CL, timezone=Asia/Anadyr
> [junit4] 2> NOTE: Linux 3.19.0-84-generic amd64/Oracle Corporation
> 1.8.0_144 (64-bit)/cpus=4,threads=1,free=292853816,total=321912832
> [junit4] 2> NOTE: All tests run in this JVM: [TestLucene70NormsFormat]
> [junit4] Completed [1/1 (1!)] in 7.48s, 1 test, 1 error <<< FAILURES!
> {code}
> (when running all methods of that test w/that seed, many of them fail _after_
> this method, with identical exceptions, but those same methods pass in
> isolation -- suggesting perhaps leaked open files? or maybe only leaked on
> failure?)
> Also: this test is called \{{TestLucene70NormsFormat}} but it has \{{new
> Lucene80Codec()}} hardcoded in it ... which seems like a pretty big WTF?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
