Re: [Lucene.Net] Repro if LUCENENET-54 (ArgumentOutOfRangeException in SnowballProgram)

[Simon Svensson]

Hi,

I've tried this using both the latest from Nuget (2.9.4.1) and using the 
very latest from the subversion trunk at 
https://svn.apache.org/repos/asf/incubator/lucene.net/trunk. I've been 
using a patched snowball-assembly for some time as a workaround, but ran 
into this issue again when updating to the latest contrib stuff using Nuget.

The fix is easy; replacing current.ToString(bra, ket) with 
current.ToString(bra, len) in SnowballProgram.cs, line 467 makes the 
repro happy.

// Simon

On 2012-01-13 09:49, Prescott Nasser wrote:
Hey Simon Are you using the latest released version? ~P
  >  Date: Sat, 7 Jan 2012 22:58:12 +0100
From: si...@devhost.se
To: lucene-net-dev@lucene.apache.org
Subject: [Lucene.Net] Repro if LUCENENET-54 (ArgumentOutOfRangeException in 
SnowballProgram)

Hi,

I've written a simple reproduction of LUCENENET-54
(ArgumentOutOfRangeException in SnowballProgram). I'm not sure about the
correct workflow to reopen this issue (it was closed as invalid in 2007
due to missing information), so I'm throwing what I got into the
developer mailing list and hope that someone else knows the correct
approach. Problem originates in SnowballProgram.slice_to, where the
second argument to StringBuilder.ToString(start, length) where the last
parameter is passed an index instead of the length.

Reproduction:
using System.IO;
using Lucene.Net.Analysis.Snowball;
using Lucene.Net.Analysis.Tokenattributes;
using NUnit.Framework;

namespace ConsoleApplication {
      [TestFixture]
      public class LuceneRepo {
          [Test(Description = "LUCENENET-54")]
          public void Repro() {
              var analyzer = new SnowballAnalyzer("Finnish");

              var input = new StringReader("terve");
              var tokenStream = analyzer.TokenStream("fieldName", input);
              var termAttr =
(TermAttribute)tokenStream.AddAttribute(typeof (TermAttribute));

              Assert.That(tokenStream.IncrementToken(), Is.True);
              Assert.That(termAttr.Term(), Is.EqualTo("terv"));
          }
      }
}

Unexpected exception:
System.ArgumentOutOfRangeException: Index and length must refer to a
location within the string.
Parameter name: length
     at System.Text.StringBuilder.ToString(Int32 startIndex, Int32 length)
     at SF.Snowball.SnowballProgram.slice_to(StringBuilder s) in
C:\Dev\Third
Party\Lucene.NET\src\contrib\Snowball\SF\Snowball\SnowballProgram.cs:line 467
     at SF.Snowball.Ext.FinnishStemmer.r_tidy() in C:\Dev\Third
Party\Lucene.NET\src\contrib\Snowball\SF\Snowball\Ext\FinnishStemmer.cs:line
974
     at SF.Snowball.Ext.FinnishStemmer.Stem() in C:\Dev\Third
Party\Lucene.NET\src\contrib\Snowball\SF\Snowball\Ext\FinnishStemmer.cs:line
1137

Expected result:
The unit test should pass.

// Simon