Awesome - always refreshing to hear everyone leaning the same direction! And thanks Uwe for the additional context!
I'll push up a PR to take this out of the ref-guide shortly. On Tue, Jun 4, 2024 at 7:07 AM Uwe Schindler <u...@thetaphi.de> wrote: > > Hi, > > I think we should remove the page from the refguide. > > I won't remove that tool from Lucene's JARs as it is a good example vor the > > Actually there are many othe rthings some customers of myself wanted to > upgrade. I you have one, lead them to me. I helped a customer which was > unable to reindex and stuck on 3.x to go from Lucene 3 to Lucene 6 with > also converting all NumericFields to PointFields and adding docValues in > the process too. But this is not as easy as a simple IndexUpgrader tool. > If it would not be so hard to setup (you need several versions installed > in parallel), I would open it up on github. > > The tool does the following: > > * adds DocValues for all fields using the Uninverter (still available > in Lucene 6.x). It basically does the same like Solr, it just uses > addIndexes(CodecReader) with a Custom CodecReader and patch of > FieldInfos. > * converts NumericField to PointField. Same here, it just uses > addIndexes(CodecReader), but the CodecReader patches Fieldinfos and > lucily it is able to use the termsenum of the legacy numeric field > to extract the numeric values and then during merging it feeds the > numeric values (which are luckily also sorted correctly) to the > BKDWriter. The BKDWriter then does the rest and expands the tree > automatically :-) > > So basically some people want to convert old indexes, but the plain > IndexUpgrader is of little use. The only thing it does is to upgrade all > segments with older versions to actual codec version. This would happen > during normal use, too. The main reason why it existed was for the > migration 3 -> 4 where the terms enum has to change the order during > iteration (UTF-16 of Lucene 3 has a different order than UTF-8 starting > with Lucene 4). This slowed down searches, so people wanted to merge all > their segments to newest version. > > Uwe > > Am 04.06.2024 um 01:13 schrieb Jan Høydahl: > > I used it back in the days when you could migrate from v3->4->5->6. It > > solved > > the issue that Solr 6 could only read a Lucene 6 or Lucene 5 index, but > > after the > > sequence of upgrades you'd get there. I even wrote a wrapper to automate it > > all at > > https://github.com/cominvent/solr-tools/tree/master/upgradeindex > > > > However, now that Lucene 9 refuses to work with any index created before > > Lucene 8, > > I agree with Jason that I cannot see any utility for the upgrade tool > > anymore. Since, > > if you have a Solr 6 index, it won't work in Solr 9 even if you used the > > upgrader. > > And if you already have a Lucene 8 index, no need to upgrade by hand, just > > start > > Solr 9 on top of the Lucene 8 index and it will be upgraded. > > > > +1 to remove mention from ref-guide. > > > > PS: If the upgrader tool was able to patch the original-index-version from > > the binary > > index (and user understands the risks), then there could be some utility. > > But I'm not > > aware of such an option. > > > > Jan > > > >> 3. juni 2024 kl. 21:20 skrev Gus Heck<gus.h...@gmail.com>: > >> > >> I've fielded many questions on this from clients. Folks who have managed > >> databases expect to be able to upgrade the data serially across versions > >> and such, so these questions come up alot with organizations early in their > >> journey with search. Essentially, I tell them that it's a stop gap tool for > >> use if there's a serious security issue and you really need to move up one > >> version to get away from the security issue, but otherwise the correct > >> procedure is to re-index. (And this is sometimes when I find out if they've > >> really planned for reindexing or not). A down-side of the existence of this > >> tool is that its presence in the ref guide allows people to prolong the > >> period where they confuse managing a search index with managing a database. > >> That said, I think it may have a place to help folks who got going without > >> knowing enough about search engines to extend the period they have to dig > >> themselves out of bad situations (by allowing an upgrade while they figure > >> out how they are going to handle re-indexing more data than they previously > >> planned on indexing all at once). So far every one of my clients has taken > >> my advice and simply re-indexed (though not always without grumbling!), so > >> I have to admit I've not actually seen it used, but in theory it makes some > >> sense. > >> > >> -Gus > >> > >> On Mon, Jun 3, 2024 at 2:20 PM David Smiley<dsmi...@apache.org> wrote: > >> > >>> FWIW, in my experience I've never run this tool (nor colleagues) at > >>> any stage in my career that I can remember. For one reason, all the > >>> systems could re-index if they needed to. > >>> It may be best to remove this information, as it could introduce more > >>> confusion than it helps. > >>> > >>> On Mon, Jun 3, 2024 at 1:34 PM Jason Gerlowski<gerlowsk...@gmail.com> > >>> wrote: > >>>> Hey all, > >>>> > >>>> I was poking around the ref-guide a bit recently and noticed our page > >>>> on the "IndexUpgraderTool" that Lucene produces. [1] > >>>> > >>>> AFAICT, the page doesn't hint at when/why a user might want to use the > >>>> IndexUpgraderTool. Maybe at one point the tool might've been > >>>> preferred to loading the index in an upgraded Solr version, but I > >>>> haven't heard of anyone doing that in the last few years. > >>>> > >>>> Is this something we expect users to still do? If so, for what > >>>> usecase? And if not - should we drop it from the ref-guide - it seems > >>>> like it might confuse folks since it's not actually needed to upgrade > >>>> Solr versions... > >>>> > >>>> Best, > >>>> > >>>> Jason > >>>> > >>>> [1] > >>> https://solr.apache.org/guide/solr/latest/deployment-guide/indexupgrader-tool.html > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail:dev-unsubscr...@solr.apache.org > >>>> For additional commands, e-mail:dev-h...@solr.apache.org > >>>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail:dev-unsubscr...@solr.apache.org > >>> For additional commands, e-mail:dev-h...@solr.apache.org > >>> > >>> > >> -- > >> http://www.needhamsoftware.com (work) > >> https://a.co/d/b2sZLD9 (my fantasy fiction book) > > > -- > Uwe Schindler > Achterdiek 19, D-28357 Bremen > https://www.thetaphi.de > eMail:u...@thetaphi.de --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@solr.apache.org For additional commands, e-mail: dev-h...@solr.apache.org
