Hi Anshum, This looks like a good start to me. You would also need to remove the 6.x version constants so that TestBackwardCompatibility does not think they are worth testing, as well as all codecs, postings formats and doc values formats that are defined in the lucene/backward-codecs module since they are only about 6.x codecs.
Le mer. 28 juin 2017 à 09:57, Anshum Gupta <ansh...@apple.com> a écrit : > Thanks for confirming that Alan, I had similar thoughts but wasn’t sure. > > I don’t want to change anything that I’m not confident about so I’m just > going to create remove those and commit it to my fork. If someone who’s > confident agrees with what I’m doing, I’ll go ahead and make those changes > to the upstream :). > > -Anshum > > > > On Jun 28, 2017, at 12:54 AM, Alan Woodward <a...@flax.co.uk> wrote: > > We don’t need to support lucene5x codecs in 7, so you should be able to > just remove those tests (and the the relevant packages from > backwards-codecs too), I think? > > > On 28 Jun 2017, at 08:38, Anshum Gupta <ansh...@apple.com> wrote: > > I tried to move forward to see this work before automatically computing > the versions but I have about 30 odd failing test. I’ve made those changes > and pushed to my local GitHub account in case you have the time to look: > https://github.com/anshumg/lucene-solr > > Here’s the build summary if that helps: > > [junit4] Tests with failures [seed: 31C3B60E557C7E14] (first 10 out of > 31): > [junit4] - > org.apache.lucene.codecs.lucene53.TestLucene53NormsFormat.testOutliers2 > [junit4] - > org.apache.lucene.codecs.lucene53.TestLucene53NormsFormat.testShortRange > [junit4] - > org.apache.lucene.codecs.lucene53.TestLucene53NormsFormat.testFewValues > [junit4] - > org.apache.lucene.codecs.lucene53.TestLucene53NormsFormat.testFullLongRange > [junit4] - > org.apache.lucene.codecs.lucene53.TestLucene53NormsFormat.testRamBytesUsed > [junit4] - > org.apache.lucene.codecs.lucene53.TestLucene53NormsFormat.testFewLargeValues > [junit4] - > org.apache.lucene.codecs.lucene53.TestLucene53NormsFormat.testByteRange > [junit4] - > org.apache.lucene.codecs.lucene53.TestLucene53NormsFormat.testLongRange > [junit4] - > org.apache.lucene.codecs.lucene50.TestLucene50SegmentInfoFormat.testRandomExceptions > [junit4] - > org.apache.lucene.codecs.lucene62.TestLucene62SegmentInfoFormat.testRandomExceptions > [junit4] > [junit4] > [junit4] JVM J0: 0.56 .. 9.47 = 8.91s > [junit4] JVM J1: 0.56 .. 4.13 = 3.57s > [junit4] JVM J2: 0.56 .. 47.28 = 46.73s > [junit4] JVM J3: 0.56 .. 3.89 = 3.33s > [junit4] Execution time total: 47 seconds > [junit4] Tests summary: 8 suites, 215 tests, 30 errors, 1 failure, 24 > ignored (24 assumptions) > > > -Anshum > > > > On Jun 27, 2017, at 4:15 AM, Adrien Grand <jpou...@gmail.com> wrote: > > The test***BackwardCompatibility cases can be removed since they make sure > that Lucene 7 can read Lucene 6 norms, while Lucene 8 doesn't have to be > able to read Lucene 6 norms. > > TestSegmentInfos needs to be adapted to the new versions, we need to > replace 5 with 6 and 8 with 9. Maybe we should compute those numbers > automatically based on Version.LATEST.major so that it does not require > manual changes when moving to a new major version. That would give 5 -> > Version.LATEST.major-2 and 8 -> Version.LATEST.major+1. > > I can do those changes on Thursday if you don't feel comfortable doing > them. > > > > Le mar. 27 juin 2017 à 08:12, Anshum Gupta <ansh...@apple.com> a écrit : > >> Without making any changes at all and just bumping up the version, I hit >> these errors when running the tests: >> >> [junit4] 2> NOTE: reproduce with: ant test >> -Dtestcase=TestSegmentInfos -Dtests.method=testIllegalCreatedVersion >> -Dtests.seed=C818A61FA6C293A1 -Dtests.slow=true -Dtests.locale=es-PR >> -Dtests.timezone=Etc/GMT+4 -Dtests.asserts=true >> -Dtests.file.encoding=US-ASCII >> [junit4] FAILURE 0.01s J0 | TestSegmentInfos.testIllegalCreatedVersion >> <<< >> [junit4] > Throwable #1: junit.framework.AssertionFailedError: >> Expected exception IllegalArgumentException but no exception was thrown >> [junit4] > at >> __randomizedtesting.SeedInfo.seed([C818A61FA6C293A1:CE340683BE44C211]:0) >> [junit4] > at >> org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2672) >> [junit4] > at >> org.apache.lucene.index.TestSegmentInfos.testIllegalCreatedVersion(TestSegmentInfos.java:35) >> [junit4] > at java.lang.Thread.run(Thread.java:748) >> [junit4] 2> NOTE: reproduce with: ant test >> -Dtestcase=TestSegmentInfos -Dtests.method=testVersionsOneSegment >> -Dtests.seed=C818A61FA6C293A1 -Dtests.slow=true -Dtests.locale=es-PR >> -Dtests.timezone=Etc/GMT+4 -Dtests.asserts=true >> -Dtests.file.encoding=US-ASCII >> [junit4] ERROR 0.00s J0 | TestSegmentInfos.testVersionsOneSegment <<< >> [junit4] > Throwable #1: >> org.apache.lucene.index.CorruptIndexException: segments file recorded >> indexCreatedVersionMajor=8 but segment=_0(7.0.0):C1 has older version=7.0.0 >> (resource=BufferedChecksumIndexInput(MockIndexInputWrapper(RAMInputStream(name=segments_1)))) >> [junit4] > at >> __randomizedtesting.SeedInfo.seed([C818A61FA6C293A1:A7477EE8875F2E36]:0) >> [junit4] > at >> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:392) >> [junit4] > at >> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:293) >> [junit4] > at >> org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:443) >> [junit4] > at >> org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:440) >> [junit4] > at >> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692) >> [junit4] > at >> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644) >> [junit4] > at >> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:445) >> [junit4] > at >> org.apache.lucene.index.TestSegmentInfos.testVersionsOneSegment(TestSegmentInfos.java:67) >> [junit4] > at java.lang.Thread.run(Thread.java:748) >> [junit4] 2> NOTE: reproduce with: ant test >> -Dtestcase=TestSegmentInfos -Dtests.method=testVersionsTwoSegments >> -Dtests.seed=C818A61FA6C293A1 -Dtests.slow=true -Dtests.locale=es-PR >> -Dtests.timezone=Etc/GMT+4 -Dtests.asserts=true >> -Dtests.file.encoding=US-ASCII >> [junit4] ERROR 0.00s J0 | TestSegmentInfos.testVersionsTwoSegments >> <<< >> [junit4] > Throwable #1: >> org.apache.lucene.index.CorruptIndexException: segments file recorded >> indexCreatedVersionMajor=8 but segment=_0(7.0.0):C1 has older version=7.0.0 >> (resource=BufferedChecksumIndexInput(MockIndexInputWrapper(RAMInputStream(name=segments_1)))) >> [junit4] > at >> __randomizedtesting.SeedInfo.seed([C818A61FA6C293A1:4EE9CC4194FBB648]:0) >> [junit4] > at >> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:392) >> [junit4] > at >> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:293) >> [junit4] > at >> org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:443) >> [junit4] > at >> org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:440) >> [junit4] > at >> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:692) >> [junit4] > at >> org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:644) >> [junit4] > at >> org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:445) >> [junit4] > at >> org.apache.lucene.index.TestSegmentInfos.testVersionsTwoSegments(TestSegmentInfos.java:96) >> [junit4] > at java.lang.Thread.run(Thread.java:748) >> >> On applying the patch here (https://pastebin.com/tM4Fpy1Q), I end up >> with the following errors: >> >> [junit4] Tests with failures [seed: 5B388AB1E2BEFF87]: >> [junit4] - >> org.apache.lucene.search.similarities.TestSimilarityBase.testLengthEncodingBackwardCompatibility >> [junit4] - >> org.apache.lucene.search.similarities.TestClassicSimilarity.testNormEncodingBackwardCompatibility >> [junit4] - >> org.apache.lucene.index.TestSegmentInfos.testIllegalCreatedVersion >> [junit4] - >> org.apache.lucene.search.similarities.TestBM25Similarity.testLengthEncodingBackwardCompatibility >> >> Any ideas on what I’m missing here? >> >> >> -Anshum >> >> >> >> On Jun 26, 2017, at 2:55 PM, Ryan Ernst <r...@iernst.net> wrote: >> >> After changing that constant check, do you get the same test failures? >> What are the actual failure messages? >> >> On Mon, Jun 26, 2017 at 10:38 AM Anshum Gupta <ans...@anshumgupta.net> >> wrote: >> >>> Thanks Adrien, sadly, the tests aren't passing. That's the reason why I >>> didn't push the changes. I'll see if someone else can help while you're >>> away. >>> >>> -Anshum >>> >>> On Mon, Jun 26, 2017 at 9:55 AM Adrien Grand <jpou...@gmail.com> wrote: >>> >>>> Your proposed change looks correct to me. As far as whether other >>>> changes are required, I'm currently traveling but can look when I'm back on >>>> Thursday. Feel free to push the branches if tests are passing, we can fix >>>> things later? >>>> >>>> Le lun. 26 juin 2017 à 07:13, Anshum Gupta <ans...@anshumgupta.net> a >>>> écrit : >>>> >>>>> The only throw case that I think needs changing is in >>>>> SegmentInfos.java L315 >>>>> >>>>> Version luceneVersion = Version.fromBits(input.readVInt(), >>>>> input.readVInt(), input.readVInt()); >>>>> if (luceneVersion.onOrAfter(Version.LUCENE_7_0_0) == false) { >>>>> // TODO: should we check indexCreatedVersion instead? >>>>> throw new IndexFormatTooOldException(input, "this index is too old >>>>> (version: " + luceneVersion + ")"); >>>>> } >>>>> >>>>> >>>>> The Lucene version here should be LUCENE_7_0_0, instead of the >>>>> original LUCENE_6_0_0. >>>>> >>>>> Is there anything else that's needed (and is this the correct change?). >>>>> >>>>> -Anshum >>>>> >>>>> >>>>> On Sun, Jun 25, 2017 at 9:45 PM Anshum Gupta <ans...@anshumgupta.net> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I am working on creating the 7x, and 7.0 branches but I have the >>>>>> following failing tests: >>>>>> [junit4] Tests with failures [seed: 4FBDDCD3F96316D3]: >>>>>> [junit4] - >>>>>> org.apache.lucene.index.TestSegmentInfos.testVersionsTwoSegments >>>>>> [junit4] - >>>>>> org.apache.lucene.index.TestSegmentInfos.testIllegalCreatedVersion >>>>>> [junit4] - >>>>>> org.apache.lucene.index.TestSegmentInfos.testVersionsOneSegment >>>>>> >>>>>> I assume this is a result of me not doing anything about the >>>>>> following TODO that got printed when I ran addVersion.py. >>>>>> *TODO* >>>>>> * - Update IndexFormatTooOldException throw cases* >>>>>> >>>>>> Can someone shed more light on what needs to be done here? >>>>>> >>>>>> -Anshum >>>>>> >>>>> >> > > >
