Re: IndexFormatTooNewException - MapReduceIndexerTool for PDF files

[Shawn Heisey]

On 5/1/2017 10:48 PM, ecos wrote:
> The cause of the error is:
> org.apache.lucene.index.IndexFormatTooNewException: Format version is not
> supported (resource: BufferedChecksumIndexInput (segments_1)): 4 (needs to
> be between 0 and 3).
>
> Reading out there I found the exception is thrown when Lucene detects an
> index that is newer that the Lucene version.

> In order to index I´m following the tutorial:  
> 
>  

Normally my guess would be that the Lucene/Solr version in the mapreduce
tool is at least 5.0, which would produce an index that Solr 4.10.3
cannot read.

I have located information saying that the Cloudera 5.8 version includes
Solr 4.10.3, which makes me wonder if maybe the Solr version that is
trying to read the index is perhaps older than 4.10.3.

You might need to go to Cloudera for support on this problem, because
you are running their software.

Thanks,
Shawn

Re: IndexFormatTooNewException - MapReduceIndexerTool for PDF files

[ravi432]

Hi ecos,

Is it giving solr documents when running mapreduce indexer tool with debug
mode.
if not can you run it with debug mode and send out any error.
 



--
View this message in context: 
http://lucene.472066.n3.nabble.com/IndexFormatTooNewException-MapReduceIndexerTool-for-PDF-files-tp4332881p4332935.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: IndexFormatTooNewException

[Shawn Heisey]

On 1/28/2015 2:51 PM, Joshi, Shital wrote:
> Thank you for replying. 
>
> We added new shard to same cluster where some shards are showing Solr version 
> 4.10.0 and this new shard is showing Solr version 4.8.0. All shards source 
> Solr software from same location and use same start up script. I am surprised 
> how older shards are still running Solr 4.10.0.
>
> How we do real downgrade index to 4.8? You mean replay all data? 

It is often not enough to simply replace the solr war.  You may also
need to wipe out the extracted war before restarting, or jars from the
previous version may still exist and some of them might be loaded
instead of the new version.

If you're using the jetty included in the example, the war is in the
webapps directory and the extracted files are under solr-webapp.  If
you're using another container, then I have no idea where the war gets
extracted.

If any index segments were written by the 4.10 version, they will not be
readable after downgrading to the 4.8 version.  Wiping out the index and
rebuilding it from scratch is usually the only way to fix that situation.

Thanks,
Shawn

RE: IndexFormatTooNewException

[Joshi, Shital]

Thank you for replying. 

We added new shard to same cluster where some shards are showing Solr version 
4.10.0 and this new shard is showing Solr version 4.8.0. All shards source Solr 
software from same location and use same start up script. I am surprised how 
older shards are still running Solr 4.10.0.

How we do real downgrade index to 4.8? You mean replay all data? 

-Original Message-
From: Chris Hostetter [mailto:hossman_luc...@fucit.org] 
Sent: Wednesday, January 28, 2015 4:10 PM
To: solr-user@lucene.apache.org
Subject: Re: IndexFormatTooNewException


: We upgraded our cluster to Solr 4.10.0 for couple days and again 
: reverted back to 4.8.0. However the dashboard still shows Solr 4.10.0. 
: Do you know why?

because you didn't fully revert - you are still running Solr 4.10.0 - the 
details of what steps you took to try and switch back make a huge 
differnet in understanding why you are still running .0 even though you 
don't want to.


: We recently added new shards to our cluster and dashboard shows correct 
: Solr version (4.8.0) for these new shards. We copied index from one of 
: old shards (where it is showing 4.10.0 on dashboard) to this new shard 
: and we see this error upon start up. How do we get rid of this error?

IndexFormatTooNewException means exactly what it sounds like -- you are 
asking Solr/Lucene to open an index that it can tell was created by a 
newer version of the software and it is incapable of doing so.

You either need to upgrade all of the nodes to 4.10, or you need to scrap 
this index, do a *real* downgrade to 4.8, and then rebuild your index (or 
restore a backup index from before you attempted to upgrade.

: Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version 
is not supported (resource: 
BufferedChecksumIndexInput(MMapIndexInput(path="/local/data/solr13/index.20140919180209018/segments_1tzz"))):
 3 (needs to be between 0 and 2)
: at 
org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:156)
: at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:335)
: at 
org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:416)
: at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:864)
: at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:710)
: at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:412)
: at org.apache.lucene.index.IndexWriter.(IndexWriter.java:749)
: 
: 
: 

-Hoss
http://www.lucidworks.com/

Re: IndexFormatTooNewException

[Chris Hostetter]

: We upgraded our cluster to Solr 4.10.0 for couple days and again 
: reverted back to 4.8.0. However the dashboard still shows Solr 4.10.0. 
: Do you know why?

because you didn't fully revert - you are still running Solr 4.10.0 - the 
details of what steps you took to try and switch back make a huge 
differnet in understanding why you are still running .0 even though you 
don't want to.


: We recently added new shards to our cluster and dashboard shows correct 
: Solr version (4.8.0) for these new shards. We copied index from one of 
: old shards (where it is showing 4.10.0 on dashboard) to this new shard 
: and we see this error upon start up. How do we get rid of this error?

IndexFormatTooNewException means exactly what it sounds like -- you are 
asking Solr/Lucene to open an index that it can tell was created by a 
newer version of the software and it is incapable of doing so.

You either need to upgrade all of the nodes to 4.10, or you need to scrap 
this index, do a *real* downgrade to 4.8, and then rebuild your index (or 
restore a backup index from before you attempted to upgrade.

: Caused by: org.apache.lucene.index.IndexFormatTooNewException: Format version 
is not supported (resource: 
BufferedChecksumIndexInput(MMapIndexInput(path="/local/data/solr13/index.20140919180209018/segments_1tzz"))):
 3 (needs to be between 0 and 2)
: at 
org.apache.lucene.codecs.CodecUtil.checkHeaderNoMagic(CodecUtil.java:156)
: at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:335)
: at 
org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:416)
: at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:864)
: at 
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:710)
: at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:412)
: at org.apache.lucene.index.IndexWriter.(IndexWriter.java:749)
: 
: 
: 

-Hoss
http://www.lucidworks.com/