RE: [Lucene.Net] [jira] [Created] (LUCENENET-469) Convert Java Iterator classes to implement IEnumerableT

2012-06-08 Thread Digy
Hi Andy,

I have used similar extension methods for a long time. What I like
especially in extension methods is that they don't require changes in
Lucene.Net kernel and make further ports of Lucene.java independent from
.Net structures.

+1 for a Lucene.Net extensions in contrib. Even a +1 for a
Lucene.Net.Extensions for the core.

DIGY

-Original Message-
From: Andy Pook [mailto:andy.p...@gmail.com] 
Sent: Friday, June 08, 2012 10:26 PM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] [jira] [Created] (LUCENENET-469) Convert Java
Iterator classes to implement IEnumerableT

If we don't want to add IEnumerable (though it seems that IEnumerable could
be added in parallel with the existing pattern) could we add a bunch of
extension methods?
Something like the following...

{noformat}
public static class LuceneExtensions
{
public static IEnumerableTerm GetEnumerable(this TermEnum termEnum) {
yield return termEnum.Term(); while (termEnum.Next()) yield return
termEnum.Term(); } } {noformat}

Then you can...
{noformat}
foreach(var e in myTernEnum.GetEnumerable()) {
// do stuff with e
}
{noformat}

Not as elegant as a direct implementation but gives easy enough access to
foreach sematics.


The second option is to realize that you don't need to explicitly implement
IEnumerable. You just need a GetEnumerator method.
So just add...

{noformat}
public IEnumerableTerm GetEnumerator() { yield return Term(); while
(Next()) yield return Term(); } {noformat}

Now you get nice foreach sematics without even mentioning IEnumerable.
Compiler magic is your friend :-)

BTW: Dispose() is only called automatically when exiting a using block.
Exiting a foreach will not.

Cheers,
  Andy

On 24 January 2012 06:37, Christopher Currens (Created) (JIRA) 
j...@apache.org wrote:

 Convert Java Iterator classes to implement IEnumerableT
 -

 Key: LUCENENET-469
 URL: https://issues.apache.org/jira/browse/LUCENENET-469
 Project: Lucene.Net
  Issue Type: Sub-task
  Components: Lucene.Net Contrib, Lucene.Net Core
Affects Versions: Lucene.Net 2.9.4, Lucene.Net 3.0.3, Lucene.Net 2.9.4g
 Environment: all
Reporter: Christopher Currens
 Fix For: Lucene.Net 3.0.3


 The Iterator pattern in Java is equivalent to IEnumerable in .NET.
  Classes that were directly ported in Java using the Iterator pattern, 
 cannot be used with Linq or foreach blocks in .NET.

 {{Next()}} would be equivalent to .NET's {{MoveNext()}}, and in the 
 below case, {{Term()}} would be as .NET's {{Current}} property.  In 
 cases as below, it will require {{TermEnum}} to become an abstract 
 class with {{Term}} and {{DocFreq}} properties, which would be 
 returned from another class or method that implemented
{{IEnumerableTermEnum}}.

 {noformat}
public abstract class TermEnum : IDisposable
{
public abstract bool Next();
public abstract Term Term();
public abstract int DocFreq();
public abstract void  Close();
public abstract void Dispose();
}
 {noformat}

 would instead look something like:

 {noformat}
public class TermFreq
{
public abstract Term { get; }
public abstract int { get; }
}

public abstract class TermEnum : IEnumerableTermFreq, IDisposable
{
// ...
}
 {noformat}

 Keep in mind that it is important that if the class being converted 
 implements {{IDisposable}}, the class that is enumerating the terms 
 (in this case {{TermEnum}}) should inherit from both 
 {{IEnumerableT}} *and* {{IDisposable}}.  This won't be any change to 
 the user, as the compiler automatically calls {{IDisposable}} when used in
a {{foreach}} loop.

 --
 This message is automatically generated by JIRA.
 If you think it was sent incorrectly, please contact your JIRA
 administrators:
 https://issues.apache.org/jira/secure/ContactAdministrators!default.js
 pa For more information on JIRA, see: 
 http://www.atlassian.com/software/jira




-

Checked by AVG - www.avg.com
Version: 2012.0.2177 / Virus Database: 2433/5056 - Release Date: 06/08/12



RE: Welcome Itamar Syn-Hershk​o as a new committer

2012-05-22 Thread Digy
Welcome Itamar!

DIGY

-Original Message-
From: Prescott Nasser [mailto:geobmx...@hotmail.com] 
Sent: Wednesday, May 23, 2012 12:06 AM
To: lucene-net-dev@lucene.apache.org; lucene-net-u...@lucene.apache.org
Subject: Welcome Itamar Syn-Hershk​o as a new committer


Hey all,
I'd like to officially welcome Itamar as a new committer. I know the community 
appreciates the work you've been doing with the Spatial contrib project and the 
past help you've provided on the mailing lists.
Please join me in welcoming Itamar,
~Prescott 
-

Checked by AVG - www.avg.com
Version: 2012.0.1913 / Virus Database: 2425/5015 - Release Date: 05/22/12



[jira] [Closed] (LUCENENET-486) Wildcard queries are not analyzed

2012-04-19 Thread Digy (Closed) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy closed LUCENENET-486.
--

Resolution: Not A Problem

http://wiki.apache.org/lucene-java/LuceneFAQ#Are_Wildcard.2C_Prefix.2C_and_Fuzzy_queries_case_sensitive.3F



 Wildcard queries are not analyzed
 -

 Key: LUCENENET-486
 URL: https://issues.apache.org/jira/browse/LUCENENET-486
 Project: Lucene.Net
  Issue Type: Bug
  Components: Lucene.Net Contrib, Lucene.Net Core
Affects Versions: Lucene.Net 2.9.2, Lucene.Net 2.9.4
 Environment: Windows 7, Visual Studio 2010, .net 4.0
Reporter: Björn
 Attachments: LuceneTest.zip


 The lucene 'QueryParser' doesn't analyze wildcard querys. The function 
 'GetPrefixQuery'(QueryParser.cs) returns the string without any analyzation.
 I have performed some queries to show the problem. The analyzer is the 
 'Contrib.Analyzers.DE.GermanAnalyzer'
 -- indexed word: 'Häuser'; in the index stemmed as: 'hau' --
 query: Hau*; hit: yes
 query: Hause*; hit: no; This should be a hit.
 -- indexed word: 'Angebote'; in the index stemmed as: 'angebo' 
 --
 query: Angebo*; hit: yes
 query: Angebot*; hit: no; This should be a hit.
 query: Angebote*; hit: no; This should be a hit.
 -- indexed word: 'Björn'; in the index stemmed as: 'bjor' --
 query: Bjor*; hit: yes
 query: Björ*; hit: no; This should be a hit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Closed] (LUCENENET-486) Wildcard queries are not analyzed

2012-04-19 Thread Digy (Closed) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy closed LUCENENET-486.
--

Resolution: Won't Fix

 Wildcard queries are not analyzed
 -

 Key: LUCENENET-486
 URL: https://issues.apache.org/jira/browse/LUCENENET-486
 Project: Lucene.Net
  Issue Type: Bug
  Components: Lucene.Net Contrib, Lucene.Net Core
Affects Versions: Lucene.Net 2.9.2, Lucene.Net 2.9.4
 Environment: Windows 7, Visual Studio 2010, .net 4.0
Reporter: Björn
 Attachments: LuceneTest.zip


 The lucene 'QueryParser' doesn't analyze wildcard querys. The function 
 'GetPrefixQuery'(QueryParser.cs) returns the string without any analyzation.
 I have performed some queries to show the problem. The analyzer is the 
 'Contrib.Analyzers.DE.GermanAnalyzer'
 -- indexed word: 'Häuser'; in the index stemmed as: 'hau' --
 query: Hau*; hit: yes
 query: Hause*; hit: no; This should be a hit.
 -- indexed word: 'Angebote'; in the index stemmed as: 'angebo' 
 --
 query: Angebo*; hit: yes
 query: Angebot*; hit: no; This should be a hit.
 query: Angebote*; hit: no; This should be a hit.
 -- indexed word: 'Björn'; in the index stemmed as: 'bjor' --
 query: Bjor*; hit: yes
 query: Björ*; hit: no; This should be a hit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (LUCENENET-486) Wildcard queries are not analyzed

2012-04-19 Thread Digy (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13257792#comment-13257792
 ] 

Digy commented on LUCENENET-486:


bq. Of course I have solved the problem for myself.

Then no problem

DIGY

 Wildcard queries are not analyzed
 -

 Key: LUCENENET-486
 URL: https://issues.apache.org/jira/browse/LUCENENET-486
 Project: Lucene.Net
  Issue Type: Bug
  Components: Lucene.Net Contrib, Lucene.Net Core
Affects Versions: Lucene.Net 2.9.2, Lucene.Net 2.9.4
 Environment: Windows 7, Visual Studio 2010, .net 4.0
Reporter: Björn
 Attachments: LuceneTest.zip


 The lucene 'QueryParser' doesn't analyze wildcard querys. The function 
 'GetPrefixQuery'(QueryParser.cs) returns the string without any analyzation.
 I have performed some queries to show the problem. The analyzer is the 
 'Contrib.Analyzers.DE.GermanAnalyzer'
 -- indexed word: 'Häuser'; in the index stemmed as: 'hau' --
 query: Hau*; hit: yes
 query: Hause*; hit: no; This should be a hit.
 -- indexed word: 'Angebote'; in the index stemmed as: 'angebo' 
 --
 query: Angebo*; hit: yes
 query: Angebot*; hit: no; This should be a hit.
 query: Angebote*; hit: no; This should be a hit.
 -- indexed word: 'Björn'; in the index stemmed as: 'bjor' --
 query: Bjor*; hit: yes
 query: Björ*; hit: no; This should be a hit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




RE: Wildcard queries are not analyzed

2012-04-17 Thread Digy
GetPrefixQuery doesn't use analyzers, and it is a well known issue of
Lucene.

Suppose a hypothetical analyzer(with stemming) which stems 'went' as 'go'
and you want to search 'wentworth miller'.
A search like 'went*' would be converted to 'go*' which i guess wouldn't be
what you want.

DIGY


-Original Message-
From: Björn Kremer [mailto:b...@patorg.de] 
Sent: Tuesday, April 17, 2012 12:59 PM
To: lucene-net-dev@lucene.apache.org
Subject: Wildcard queries are not analyzed

Hello,


maybe I have found a little lucene problem: Wildcard queries are not
analyzed correctly. I'm using the german analyzer with the
'GermanDIN2Stemmer'.

In the lucene-index my name('Björn') is stored as 'bjorn'. If I performe a
wildcard query like 'björ*' the function 'GetPrefixQuery' does not analyze
the search term. So the query result is 'björ*' instead of 'bjor*'. (björ* =
no match, bjor* = match)


Thank You
Björn

-

Checked by AVG - www.avg.com
Version: 2012.0.1913 / Virus Database: 2411/4940 - Release Date: 04/16/12



RE: [Lucene.Net] [VOTE] Apache-Lucene-2.9.4g-incubating-RC1 Release (take 2)

2012-01-25 Thread Digy
+1 
DIGY

-Original Message-
From: Prescott Nasser [mailto:geobmx...@hotmail.com] 
Sent: Thursday, January 26, 2012 1:56 AM
To: lucene-net-dev@lucene.apache.org
Cc: lucene-net-...@incubator.apache.org
Subject: RE: [Lucene.Net] [VOTE] Apache-Lucene-2.9.4g-incubating-RC1 Release 
(take 2)

Thanks for the +1, we need one more vote here, then Stefan will be comfortable 
giving us a plus one, which will give us two plus ones in general, and ill only 
have to beg for one more :)

Sent from my Windows Phone

From: Michael Herndon
Sent: 1/25/2012 11:15 AM
To: lucene-net-dev@lucene.apache.org
Cc: lucene-net-...@incubator.apache.org
Subject: Re: [Lucene.Net] [VOTE] Apache-Lucene-2.9.4g-incubating-RC1 Release 
(take 2)

verified tests pass and checksums match.

so +1


@P, I remember that thread.  Those guys stay busy though and devopt
mentality is different than a devs.

Our needs probably exceed what the svn CMS is meant for due to
documentation.  I am curious if infra allows for or would allow us to throw
up a static mono/asp.net mvc in the future just so that we could dog food
 the site with search using Lucene.Net and then have it index certain pages
or sites (wiki, tutorials, static site, docs).   We'll probably need to dig
out our CMS options again and weight against short term and long term
goals.






On Wed, Jan 25, 2012 at 12:31 PM, Prescott Nasser geobmx...@hotmail.comwrote:

 You know even making a small change to the website like updating the news
 takes like 30 minutes to run now because of all the files. Its absolutely
 ridiculous.

 I got chided by the CMS group, yet when asked how do we put documentation
 online with the new system there were crickets.



 Sent from my Windows Phone
 
 From: Michael Herndon
 Sent: 1/25/2012 8:26 AM
 To: lucene-net-dev@lucene.apache.org
 Cc: lucene-net-...@incubator.apache.org
 Subject: Re: [Lucene.Net] [VOTE] Apache-Lucene-2.9.4g-incubating-RC1
 Release (take 2)

 I was not able to download the binaries till this morning.  The wiki was
 also having issues.

 I ran rat on the the released source, that seems fine. did a compare on src
 zip and the tag. it matches.

 The only things I saw are nit picks.
 in the ReadMe the link should point to its respective tag instead of RC3
 for just 2_9_4

 https://svn.apache.org/repos/asf/incubator/lucene.net/tags/Lucene.Net_2_9_4_RC3/lib/should
 be

 when releasing the source in the future, we should either include a script
 that pulls the lib for the developers who want to compile from source
 inside a tag when the project is built using the solution. Or we should
 invest into using something like nuget for dependencies so that the
 dependencies are automatically fetched somehow and we can remove those from
 svn/scm altogether.

 the source currently violates the don't make me think about it principle.



 I know we all dislike chms, but until we figure out a better way of posting
 the generated msdn documentation online, we should include that in releases
 as well.  The static website version generates a high number of static html
 files and our current CMS requires that those files are pushed into SVN
 which just is not feasible. Committing that all at once will choke infra's
 setup (and if they hired ninjas to pay us a visit, I probably wouldn't
 blame them) and doing partial commits is just borderline insanity.


 Just waiting on the all the tests to finish running.
 http://xkcd.com/303/




 On Wed, Jan 25, 2012 at 6:29 AM, Stefan Bodewig bode...@apache.org
 wrote:

  On 2012-01-25, Michael Herndon wrote:
 
   Stefan what did you use to check the eof of files for svn?
 
  Pretty much a long and boring manual process.  I did something like
 
  find . -name \*.cs -print0 | xargs -0 -e svn ps svn:eol-style native
 
  i.e. tried to set the eol-style property on all C# source files.  This
  won't do anything if the property is set and tell you it has changed
  something in svn status if it the property hasn't been set before.
 
  svn will also fail if the file in question contains inconsistent line
  ends, this is the case for the NUnit doc files and even some of
  Lucene.NET sources.
 
  Repeat for all other file extension that should map to text files.
 
   I'm setting up RAT on my local.  Are there any other tools that you or
  ASF
   recommends in general to validate releases?
 
  I think Sebb has a bunch of scripts he uses, but never bothered to look
  them up.  If so, they'd be inside the comitters svn repo.
 
  For this release you don't even need to check line-feeds, the properties
  have not been set on all files.  The patch I provided a while ago only
  applied to trunk.  To me this is no reason to stop the release, in
  particular since most files have Windows line-ends and Prescott built
  the release on Windows so the files would be the same with and without
  svn:eol-style anyway.
 
  I intend to provide a new patch for the 3.0.3 branch once

RE: [Lucene.Net] Extracting IsStored fields from a search result

2012-01-11 Thread Digy
Hi Chris,

There are many methods of IndexReader you can make use of.
For ex, to get the field names in the index GetFieldNames or to get the
terms Terms.
I posted here (http://pastebin.com/aiCy26mj) a simple example to get all
terms in the field myfield.

DIGY


-Original Message-
From: Chris Lomax [mailto:chri...@helloevery1.com] 
Sent: Wednesday, January 11, 2012 1:27 PM
To: lucene-net-dev@lucene.apache.org
Subject: [Lucene.Net] Extracting IsStored fields from a search result

I am trying to extract anything that is a field IsStored for use in 
filtering lists

e.g. When you access an ecommerce site and you get the filter lists down 
the left hand side, I wanted to store anything that I wanted filtering 
and when I performed a search I wanted to create a lookup list from 
these IsStored values.

I started something yesterday where I iterated over the fields returned 
from the search results and picked out anything that was IsStored but I 
feel there must be a more efficient way of doing this. I can not find 
any documentation relating to this type of extraction.

Could anyone offer any advice on this?

Also, the dll does not work on vb.net as there are fields named the same 
in the classes. For example, ScoreDocs in the 
Lucene.Net.Search.TopDocs has a variable and a property declaration but 
they are both set to Public so .Net cannot see the property. I had to 
modify the class so the variable was private.

Thanks

Chris
-

Checked by AVG - www.avg.com
Version: 2012.0.1901 / Virus Database: 2109/4736 - Release Date: 01/11/12



[Lucene.Net] [jira] [Commented] (LUCENENET-463) Would like to be able to use a SimpleSpanFragmenter for extrcting whole sentances

2012-01-05 Thread Digy (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13180965#comment-13180965
 ] 

Digy commented on LUCENENET-463:


My guess:
Either it is not widely used as you think, or no one is willing to port it :)

DIGY

 Would like to be able to use a SimpleSpanFragmenter for extrcting whole 
 sentances 
 --

 Key: LUCENENET-463
 URL: https://issues.apache.org/jira/browse/LUCENENET-463
 Project: Lucene.Net
  Issue Type: New Feature
  Components: Lucene.Net Contrib
Reporter: Steven
Priority: Minor

 This is described in the Java version, but it does not seem to be in the 
 dot.net port, is there a reason for this as I would have imagined lots of 
 people doing document work would want it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




RE: [Lucene.Net] Lucene.Net 3 onwards and 2.9.4g

2011-12-29 Thread Digy
When I started that g branch, I had no intention to change the API, but at
the end it resulted in a few changes 
like StopAnalyzer(Liststring stopWords),
Query.ExtractTerms(ICollectionstring) etc.
But I think, a drop-in replacement will work for most of the Lucene.Net
users (Of course some contribs have been also modified accordingly)

Changing arraylists/collections with generic counterparts, GetEnumerator's
with foreach, AnonymousClass's with
Func or Action's and Fixing LUCENENET-172 are things most people would
not notice.

This g version includes also some other patches that were fixed for
.GE.(=) Lucene3.1 (Which? I have to rework on my commits)

So, there isn't much change in API, more changes for developers and more
stable code(At least I think so, since I use this g version in production
env. for months without any problem. For short, 2.9.4g is a superset of
2.9.4 in bugfix level)


As a result, creating a new branch for .Net friendly Lucene.Net or
continuing on this branch is just a matter of taste.

DIGY






-Original Message-
From: Scott Lombard [mailto:lombardena...@gmail.com] 
Sent: Thursday, December 29, 2011 5:05 PM
To: lucene-net-...@lucene.apache.org
Subject: RE: [Lucene.Net] Lucene.Net 3 onwards and 2.9.4g


I don't see the g branch differing all that much from the line-by-line port.
All the g branch does is change some data types as generics, but line by
line the code the same once the generics are declared.  

I don't see 2.9.4g being any closer to a .NET style version than 2.9.4.
While it does generics use for list style variable types the underlying
classes are still the same and all of the problems with 2.9.4 not being .NET
enough would be true in 2.9.4g.   

I would have to refer to Digy on if it changes how an end user interacts
with Lucene.NET.  If it does not affect how the end user interacts with
Lucene.NET then I think we should merge it into the Trunk and go from there
on 3.0.3.  


Scott 
 

 -Original Message-
 From: Prescott Nasser [mailto:geobmx...@hotmail.com] 
 Sent: Wednesday, December 28, 2011 8:28 PM
 To: lucene-net-...@lucene.apache.org
 Subject: RE: [Lucene.Net] Lucene.Net 3 onwards and 2.9.4g
 
 
 Any reason we can't continue this g branch and make it more 
 and more .net like? I was thinking about what we've expressed 
 at goals - we want a line by line port - it's easy to 
 maintain parity with java and easy to compare. We also want a 
 more .NET version - the g branch gets this started - although 
 it's not as .Net as people want (I think). 
 
  
 
 What if we used the g branch as our .Net version and 
 continued to make it more .Net like? and kept the trunk as 
 the line by line? The G branch seems like a good start to the 
 more .Net version anyway - we might as well build off of that?
 
  
 
  
 
  
 
  
 
   From: 
 digyd...@gmail.com  To: lucene-net-...@lucene.apache.org  
 Date: Thu, 29 Dec 2011 02:45:23 +0200  Subject: RE: 
 [Lucene.Net] Lucene.Net 3 onwards and 2.9.4gbut I 
 guess the future of 2.9.4g depends on the extent that it is 
 becoming  more .NET like   My intention while I was 
 creating that branch was just to make 2.9.4 a  little bit 
 more .Net like(+ maybe some performance).  I used many codes 
 from 3.0.3 Java. So it is somewhere between 2.9.4  3.0.3  
 But I didn't think it as a separate branch to evolve on its 
 own path. It  is(or I think it is) the final version of 2.9 
   DIGY   -Original Message-  From: Christopher 
 Currens [mailto:currens.ch...@gmail.com]  Sent: Wednesday, 
 December 28, 2011 9:20 PM  To: 
 lucene-net-...@lucene.apache.org  Cc: 
 lucene-net-u...@lucene.apache.org  Subject: Re: [Lucene.Net] 
 Lucene.Net 3 onwards and 2.9.4g   One of the benefits of 
 moving forward with the conversion of the Java  Lucene, is 
 that they're using more recent versions of Java that support 
  things like generics and enums, so the direct port is 
 getting more and more  like .NET, though not in all respects 
 of course. I'm of the mind, though,  that one of the larger 
 annoyances, Iterables, should be converted to  Enumerables 
 in the direct port. It makes it a pain to use it in .NET  
 without it inheriting from IEnumerable, since it can't be 
 used in a foreach  loop or with linq. Also, since the direct 
 port isn't perfect anyway, it  seems a port of the IDEA of 
 iterating would be more in the spirit of what  we're trying 
 to accomplish, since the code would pretty much be the same, 
  just with different method names.   I sort of got off 
 topic there for a second, but I guess the future of  2.9.4g 
 depends on the extent that it is becoming more .NET like.  
 Obviously, while java is starting to use similar constructs 
 that we have  in .NET, it will never be perfect. Admittedly, 
 I haven't looked at 2.9.4g  in a little while, so I'm not 
 sure how much it now differs from 3.x, since  there's a 
 relatively large change there already.   Thanks

RE: [Lucene.Net] Lucene.Net 3 onwards and 2.9.4g

2011-12-29 Thread Digy
I forgot to mention, 2.9.4g implements IDisposable for many of the classes
that has a Close() method 
which can be thought as .Net friendly API.

DIGY





-Original Message-
From: Scott Lombard [mailto:lombardena...@gmail.com] 
Sent: Thursday, December 29, 2011 5:05 PM
To: lucene-net-...@lucene.apache.org
Subject: RE: [Lucene.Net] Lucene.Net 3 onwards and 2.9.4g


I don't see the g branch differing all that much from the line-by-line port.
All the g branch does is change some data types as generics, but line by
line the code the same once the generics are declared.  

I don't see 2.9.4g being any closer to a .NET style version than 2.9.4.
While it does generics use for list style variable types the underlying
classes are still the same and all of the problems with 2.9.4 not being .NET
enough would be true in 2.9.4g.   

I would have to refer to Digy on if it changes how an end user interacts
with Lucene.NET.  If it does not affect how the end user interacts with
Lucene.NET then I think we should merge it into the Trunk and go from there
on 3.0.3.  


Scott 
 

 -Original Message-
 From: Prescott Nasser [mailto:geobmx...@hotmail.com] 
 Sent: Wednesday, December 28, 2011 8:28 PM
 To: lucene-net-...@lucene.apache.org
 Subject: RE: [Lucene.Net] Lucene.Net 3 onwards and 2.9.4g
 
 
 Any reason we can't continue this g branch and make it more 
 and more .net like? I was thinking about what we've expressed 
 at goals - we want a line by line port - it's easy to 
 maintain parity with java and easy to compare. We also want a 
 more .NET version - the g branch gets this started - although 
 it's not as .Net as people want (I think). 
 
  
 
 What if we used the g branch as our .Net version and 
 continued to make it more .Net like? and kept the trunk as 
 the line by line? The G branch seems like a good start to the 
 more .Net version anyway - we might as well build off of that?
 
  
 
  
 
  
 
  
 
   From: 
 digyd...@gmail.com  To: lucene-net-...@lucene.apache.org  
 Date: Thu, 29 Dec 2011 02:45:23 +0200  Subject: RE: 
 [Lucene.Net] Lucene.Net 3 onwards and 2.9.4gbut I 
 guess the future of 2.9.4g depends on the extent that it is 
 becoming  more .NET like   My intention while I was 
 creating that branch was just to make 2.9.4 a  little bit 
 more .Net like(+ maybe some performance).  I used many codes 
 from 3.0.3 Java. So it is somewhere between 2.9.4  3.0.3  
 But I didn't think it as a separate branch to evolve on its 
 own path. It  is(or I think it is) the final version of 2.9 
   DIGY   -Original Message-  From: Christopher 
 Currens [mailto:currens.ch...@gmail.com]  Sent: Wednesday, 
 December 28, 2011 9:20 PM  To: 
 lucene-net-...@lucene.apache.org  Cc: 
 lucene-net-u...@lucene.apache.org  Subject: Re: [Lucene.Net] 
 Lucene.Net 3 onwards and 2.9.4g   One of the benefits of 
 moving forward with the conversion of the Java  Lucene, is 
 that they're using more recent versions of Java that support 
  things like generics and enums, so the direct port is 
 getting more and more  like .NET, though not in all respects 
 of course. I'm of the mind, though,  that one of the larger 
 annoyances, Iterables, should be converted to  Enumerables 
 in the direct port. It makes it a pain to use it in .NET  
 without it inheriting from IEnumerable, since it can't be 
 used in a foreach  loop or with linq. Also, since the direct 
 port isn't perfect anyway, it  seems a port of the IDEA of 
 iterating would be more in the spirit of what  we're trying 
 to accomplish, since the code would pretty much be the same, 
  just with different method names.   I sort of got off 
 topic there for a second, but I guess the future of  2.9.4g 
 depends on the extent that it is becoming more .NET like.  
 Obviously, while java is starting to use similar constructs 
 that we have  in .NET, it will never be perfect. Admittedly, 
 I haven't looked at 2.9.4g  in a little while, so I'm not 
 sure how much it now differs from 3.x, since  there's a 
 relatively large change there already.   Thanks,  
 Christopher   On Thu, Dec 22, 2011 at 9:13 PM, Prescott 
 Nasser  wrote:  That's a great question - I know a 
 lot of people like the generics, and I   don't really want 
 it to disappear. I'd like to keep it in parity with the   
 trunk. But I know we also have a goal of making Lucene.Net 
 more .Net like   (further than 2.9.4g), and I don't know 
 how that fits in. We are a pretty   small community and I 
 know everyone has some pretty busy schedules so it   takes 
 us considerable time to make big progress. Trying to keep 
 three   different code bases probably isn't the right way 
 to go.  Date: Fri, 23 Dec 2011 13:02:03 
 +1100From: mitiag...@gmail.comTo: 
 lucene-net-u...@lucene.apache.orgSubject: [Lucene.Net] 
 Lucene.Net 3 onwards and 2.9.4g   I was browsing 
 Roadmap emails from November in Lucene developer list.   
 Itremains unclear

RE: [Lucene.Net] Lucene.Net 3 onwards and 2.9.4g

2011-12-28 Thread Digy
 but I guess the future of 2.9.4g depends on the extent that it is becoming
more .NET like

My intention while I was creating that branch was just to make 2.9.4 a
little bit more .Net like(+ maybe some performance).
I used many codes from 3.0.3 Java. So it is somewhere between 2.9.4  3.0.3
But I didn't think it as a separate branch to evolve on its own path. It
is(or I think it is) the final version of 2.9

DIGY

-Original Message-
From: Christopher Currens [mailto:currens.ch...@gmail.com] 
Sent: Wednesday, December 28, 2011 9:20 PM
To: lucene-net-dev@lucene.apache.org
Cc: lucene-net-u...@lucene.apache.org
Subject: Re: [Lucene.Net] Lucene.Net 3 onwards and 2.9.4g

One of the benefits of moving forward with the conversion of the Java
Lucene, is that they're using more recent versions of Java that support
things like generics and enums, so the direct port is getting more and more
like .NET, though not in all respects of course.  I'm of the mind, though,
that one of the larger annoyances, Iterables, should be converted to
Enumerables in the direct port.  It makes it a pain to use it in .NET
without it inheriting from IEnumerable, since it can't be used in a foreach
loop or with linq.  Also, since the direct port isn't perfect anyway, it
seems a port of the IDEA of iterating would be more in the spirit of what
we're trying to accomplish, since the code would pretty much be the same,
just with different method names.

I sort of got off topic there for a second, but I guess the future of
2.9.4g depends on the extent that it is becoming more .NET like.
 Obviously, while java is starting to use similar constructs that we have
in .NET, it will never be perfect.  Admittedly, I haven't looked at 2.9.4g
in a little while, so I'm not sure how much it now differs from 3.x, since
there's a relatively large change there already.

Thanks,
Christopher

On Thu, Dec 22, 2011 at 9:13 PM, Prescott Nasser
geobmx...@hotmail.comwrote:


 That's a great question - I know a lot of people like the generics, and I
 don't really want it to disappear. I'd like to keep it in parity with the
 trunk. But I know we also have a goal of making Lucene.Net more .Net like
 (further than 2.9.4g), and I don't know how that fits in. We are a pretty
 small community and I know everyone has some pretty busy schedules so it
 takes us considerable time to make big progress. Trying to keep three
 different code bases probably isn't the right way to go.



  Date: Fri, 23 Dec 2011 13:02:03 +1100
  From: mitiag...@gmail.com
  To: lucene-net-u...@lucene.apache.org
  Subject: [Lucene.Net] Lucene.Net 3 onwards and 2.9.4g
 
  I was browsing Roadmap emails from November in Lucene developer list.
 It
  remains unclear in what state Lucene 3 porting is , but my question more
  about 2.9.4g .
  Is it kind of experimental dead end variation of 2.9.4 with generics ?
Am
  I right in classifying it as more .Net like 2.9.4 which is unrelated to
  roadmap Lucene 3 porting effort.

-

Checked by AVG - www.avg.com
Version: 2012.0.1901 / Virus Database: 2109/4708 - Release Date: 12/28/11



[Lucene.Net] [jira] [Updated] (LUCENENET-459) Italian stemmer (from SnowballAnalyzer) does not work

2011-12-22 Thread Digy (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy updated LUCENENET-459:
---

Affects Version/s: Lucene.Net 2.9.4g
Fix Version/s: Lucene.Net 2.9.4g

 Italian stemmer (from SnowballAnalyzer) does not work
 -

 Key: LUCENENET-459
 URL: https://issues.apache.org/jira/browse/LUCENENET-459
 Project: Lucene.Net
  Issue Type: Bug
  Components: Lucene.Net Contrib
Affects Versions: Lucene.Net 2.9.2, Lucene.Net 2.9.4, Lucene.Net 2.9.4g
Reporter: Santiago M. Mola
 Fix For: Lucene.Net 2.9.4g


 Italian stemmer does not work.
 Consider this code:
   var englishAnalyzer = new SnowballAnalyzer(English);
   var tk = englishAnalyzer.TokenStream(text, new 
 StringReader(horses));
   var ta = 
 (TermAttribute)tk.GetAttribute(typeof(TermAttribute));
   tk.IncrementToken();
   Console.WriteLine(English stemmer: horses -  + 
 ta.Term());
   
   var italianAnalyzer = new SnowballAnalyzer(Italian);
   tk = italianAnalyzer.TokenStream(text, new 
 StringReader(abbandonata));
   ta = 
 (TermAttribute)tk.GetAttribute(typeof(TermAttribute));
   tk.IncrementToken();
   Console.WriteLine(Italian stemmer: abbandonata -  + 
 ta.Term());
 It outputs:
 English stemmer: horses - hors
 Italian stemmer: abbandonata - abbandonata
 While Java Lucene 2.9.4 outputs:
 English stemmer: horses - hors
 Italian stemmer: abbandonata - abbandon

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-459) Italian stemmer (from SnowballAnalyzer) does not work

2011-12-22 Thread Digy (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13175117#comment-13175117
 ] 

Digy commented on LUCENENET-459:


Fixed in 2.9.4g branch.
DIGY

 Italian stemmer (from SnowballAnalyzer) does not work
 -

 Key: LUCENENET-459
 URL: https://issues.apache.org/jira/browse/LUCENENET-459
 Project: Lucene.Net
  Issue Type: Bug
  Components: Lucene.Net Contrib
Affects Versions: Lucene.Net 2.9.2, Lucene.Net 2.9.4, Lucene.Net 2.9.4g
Reporter: Santiago M. Mola
 Fix For: Lucene.Net 2.9.4g


 Italian stemmer does not work.
 Consider this code:
   var englishAnalyzer = new SnowballAnalyzer(English);
   var tk = englishAnalyzer.TokenStream(text, new 
 StringReader(horses));
   var ta = 
 (TermAttribute)tk.GetAttribute(typeof(TermAttribute));
   tk.IncrementToken();
   Console.WriteLine(English stemmer: horses -  + 
 ta.Term());
   
   var italianAnalyzer = new SnowballAnalyzer(Italian);
   tk = italianAnalyzer.TokenStream(text, new 
 StringReader(abbandonata));
   ta = 
 (TermAttribute)tk.GetAttribute(typeof(TermAttribute));
   tk.IncrementToken();
   Console.WriteLine(Italian stemmer: abbandonata -  + 
 ta.Term());
 It outputs:
 English stemmer: horses - hors
 Italian stemmer: abbandonata - abbandonata
 While Java Lucene 2.9.4 outputs:
 English stemmer: horses - hors
 Italian stemmer: abbandonata - abbandon

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Resolved] (LUCENENET-412) Replacing ArrayLists, Hashtables etc. with appropriate Generics.

2011-12-22 Thread Digy (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy resolved LUCENENET-412.


Resolution: Fixed

2.9.4g is ready to go
DIGY

 Replacing ArrayLists, Hashtables etc. with appropriate Generics.
 

 Key: LUCENENET-412
 URL: https://issues.apache.org/jira/browse/LUCENENET-412
 Project: Lucene.Net
  Issue Type: Improvement
Affects Versions: Lucene.Net 2.9.4
Reporter: Digy
Priority: Minor
 Fix For: Lucene.Net 2.9.4g

 Attachments: IEquatable for QuerySubclasses.patch, 
 LUCENENET-412.patch, lucene_2.9.4g_exceptions_fix


 This will move Lucene.Net.2.9.4 closer to lucene.3.0.3 and allow some 
 performance gains.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2

2011-11-30 Thread Digy
FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
CloseableThreadLocal

(like uncommenting ThreadLocalT class and replacing extension-methods
calls 

with static calls to CloseableThreadLocalExtensions)

 

 

DIGY

 

 

-Original Message-
From: Christopher Currens [mailto:currens.ch...@gmail.com] 
Sent: Wednesday, November 30, 2011 7:26 PM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2

 

Trevor,

 

I'm not sure if you can use 2.9.4, though, it looks like you're using

VS2005 and .NET 2.0.  2.9.4 targets .NET 4.0, and I'm fairly certain we use

classes only available in 4.0 (or 3.5?).  However, if you can, I would

suggest updating, as 2.9.4 should be a fairly stable release.

 

The leak I'm talking about is addressed here:

https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code

isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4.  It may

or may not be what your issue is.  You say that it was at one time working

fine, I assume you mean no memory leak.  I would take some time to see what

else in your code has changed.  Make sure you're calling Close on whatever

needs to be closed (IndexWriter/IndexReader/Analyzers, etc).

 

Unfortunately for us, memory leaks are hard to debug over email, and it's

difficult for us to tell if it's any change to your code or an issue with

Lucene .NET.  As far as I can tell, this is the only memory leak I can find

that affects 2.9.2.

 

 

Thanks,

Christopher

 

On Wed, Nov 30, 2011 at 8:25 AM, Prescott Nasser
geobmx...@hotmail.comwrote:

 

 We just released 2.9.4 - the website didn't update last night, so ill have

 to try and update it later today. But if you follow the link to download

 2.9.2 dist you'll see folders for 2.9.4.

 

 I'll send an email to the user and dev lists once i get the website to

 update

 

 From: Trevor Watson

 Sent: 11/30/2011 8:14 AM

 To: lucene-net-dev@lucene.apache.org

 Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2

 

 You said pre 2.9.3  I checked the apache lucene.net page to try to see

 if

 I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2
and

 2.9.4(g).  Was this an un-released version?  Or am I looking in the wrong

 spot for updates to lucene.net?

 

 Thanks for all your help

 

 On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson 

 powersearchsoftw...@gmail.com wrote:

 

  I can send you the dll that I am using if you would like.  The documents

  are _mostly_ small documents.  Emails and office docs size of plain text

 

 

  On Tuesday, November 29, 2011, Christopher Currens 

  currens.ch...@gmail.com wrote:

   Do you know how big the documents are that you are trying to

  delete/update?

I'm trying to find a copy of 2.9.2 to see if I can reproduce it.

  

  

   Thanks,

   Christopher

  

   On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson 

   powersearchsoftw...@gmail.com wrote:

  

   Sorry for the duplicate post. I was on the road and posted both via
my

  web

   mail and office mail by mistake

  

   The increase is a very gradual,  the program starts at about 160,000k

   according to task manager (I know that's not entirely accurate, but
it

  was

   the best I had at the time) and would, after adding 25,000-40,000

  result in

   an out of memory exception (800,000k according to taskmanager). I

 tried

   building a copy of 2.9.4 to test, but could not find one that worked

 in

   visual studio 2005

  

   I did notice using Ants memory profiler that there were a number of

   byte[32789] arrays that I didn't know where they came from in memory.

  

   On Monday, November 28, 2011, Christopher Currens 

  currens.ch...@gmail.com

   

   wrote:

Hi Trevor,

   

What kind of memory increase are we talking about?  Also, how big

 are

  the

documents that you are indexing, the ones returned from

  getFileInfoDoc()?

 Is it putting an entire file into the index?  Pre 2.9.3 versions

 had

issues with holding onto allocated byte arrays far beyond when they

  were

used.  The memory could only be freed via closing the IndexWriter.

   

I'm a little unclear on exactly what's happening.  Are you noticing

   memory

spike and stay constant at that level or is it a gradual increase?

   Is it

causing your application to error, (ie OutOfMemory exception, etc)?

   

   

Thanks,

Christopher

   

On Mon, Nov 28, 2011 at 5:59 PM, Trevor Watson 

powersearchsoftw...@gmail.com wrote:

   

I'm attempting to use Lucene.Net v2.9.2.2 in a Visual Studio 2005

  (.NET

2.0) environment.  We had a piece of software that WAS working.

  I'm

  not

sure what has changed however, the following code results in a

 memory

   leak

in the Lucene.Net component (or a failure to clean up used
memory).

   

The code in issue is here:

   

 private void

RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2

2011-11-30 Thread Digy
OK, here is the code that can be compiled against .NET 2.0
http://pastebin.com/k2f7JfPd

DIGY


-Original Message-
From: Granroth, Neal V. [mailto:neal.granr...@thermofisher.com] 
Sent: Wednesday, November 30, 2011 9:26 PM
To: lucene-net-dev@lucene.apache.org
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2

DIGY,

Thanks for the tip, but could you be a little more specific?
Where and how are extension-methods calls replaced?

For example, how would I change the CloseableThreadLocalExtensions method
 
public static void SetT(this ThreadLocalT t, T val)
{
t.Value = val;
}


to eliminate the compile error

Error   2   Cannot define a new extension method because the compiler
required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot be
found. Are you missing a reference to System.Core.dll?

due to the SetT parameter this ThreadLocalt t ?

 
- Neal


-Original Message-
From: Digy [mailto:digyd...@gmail.com] 
Sent: Wednesday, November 30, 2011 12:27 PM
To: lucene-net-dev@lucene.apache.org
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2

FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
CloseableThreadLocal

(like uncommenting ThreadLocalT class and replacing extension-methods
calls 

with static calls to CloseableThreadLocalExtensions)

 

 

DIGY

 

 

-Original Message-
From: Christopher Currens [mailto:currens.ch...@gmail.com] 
Sent: Wednesday, November 30, 2011 7:26 PM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2

 

Trevor,

 

I'm not sure if you can use 2.9.4, though, it looks like you're using

VS2005 and .NET 2.0.  2.9.4 targets .NET 4.0, and I'm fairly certain we use

classes only available in 4.0 (or 3.5?).  However, if you can, I would

suggest updating, as 2.9.4 should be a fairly stable release.

 

The leak I'm talking about is addressed here:

https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code

isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4.  It may

or may not be what your issue is.  You say that it was at one time working

fine, I assume you mean no memory leak.  I would take some time to see what

else in your code has changed.  Make sure you're calling Close on whatever

needs to be closed (IndexWriter/IndexReader/Analyzers, etc).

 

Unfortunately for us, memory leaks are hard to debug over email, and it's

difficult for us to tell if it's any change to your code or an issue with

Lucene .NET.  As far as I can tell, this is the only memory leak I can find

that affects 2.9.2.

 

 

Thanks,

Christopher

 

On Wed, Nov 30, 2011 at 8:25 AM, Prescott Nasser
geobmx...@hotmail.comwrote:

 

 We just released 2.9.4 - the website didn't update last night, so ill have

 to try and update it later today. But if you follow the link to download

 2.9.2 dist you'll see folders for 2.9.4.

 

 I'll send an email to the user and dev lists once i get the website to

 update

 

 From: Trevor Watson

 Sent: 11/30/2011 8:14 AM

 To: lucene-net-dev@lucene.apache.org

 Subject: [Lucene.Net] Re: Memory Leak in 2.9.2.2

 

 You said pre 2.9.3  I checked the apache lucene.net page to try to see

 if

 I could get a copy of 2.9.3, but it was never on the site, just 2.9.2.2
and

 2.9.4(g).  Was this an un-released version?  Or am I looking in the wrong

 spot for updates to lucene.net?

 

 Thanks for all your help

 

 On Tue, Nov 29, 2011 at 2:59 PM, Trevor Watson 

 powersearchsoftw...@gmail.com wrote:

 

  I can send you the dll that I am using if you would like.  The documents

  are _mostly_ small documents.  Emails and office docs size of plain text

 

 

  On Tuesday, November 29, 2011, Christopher Currens 

  currens.ch...@gmail.com wrote:

   Do you know how big the documents are that you are trying to

  delete/update?

I'm trying to find a copy of 2.9.2 to see if I can reproduce it.

  

  

   Thanks,

   Christopher

  

   On Tue, Nov 29, 2011 at 9:11 AM, Trevor Watson 

   powersearchsoftw...@gmail.com wrote:

  

   Sorry for the duplicate post. I was on the road and posted both via
my

  web

   mail and office mail by mistake

  

   The increase is a very gradual,  the program starts at about 160,000k

   according to task manager (I know that's not entirely accurate, but
it

  was

   the best I had at the time) and would, after adding 25,000-40,000

  result in

   an out of memory exception (800,000k according to taskmanager). I

 tried

   building a copy of 2.9.4 to test, but could not find one that worked

 in

   visual studio 2005

  

   I did notice using Ants memory profiler that there were a number of

   byte[32789] arrays that I didn't know where they came from in memory.

  

   On Monday, November 28, 2011, Christopher Currens 

  currens.ch...@gmail.com

   

   wrote:

Hi Trevor,

   

What kind of memory increase are we talking about

RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2

2011-11-30 Thread Digy
If I recall it correctly, last memory leak problem for 2.9.2 was reported in
~August from RavenDB, and it was fixed in 2.9.4(g)

DIGY

-Original Message-
From: Christopher Currens [mailto:currens.ch...@gmail.com] 
Sent: Wednesday, November 30, 2011 11:33 PM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2

Trevor,

Unforunately I was unable to reproduce the memory leak you're experiencing
in 2.9.2.  Particularly with byte[], of the 18,277 that were created, only
13 were not garbage collected, and it's likely that they are not related to
Lucene (it's possible they are static, therefore would only be destroyed
with the AppDomain, outside of what the profiler can trace).  I tried to
emulate the code you showed us and there were no signs of any allocated
arrays that weren't cleaned up.  That doesn't mean there isn't one in your
code, but I just can't reproduce it with what you've shown us.  If it's
possible you can write a small program that has the same behavior, that
could help us track it down.

As a side note, what was a little disconcerting, though, was in 2.9.4 with
the same code, it created 28,565 byte[], and there was quite a few more
left uncollected (2,805 arrays).  The allocations are happening in
DocumentsWriter.ByteBlockAllocator, I'll have to look at it later though,
to see if its even a problem.


Thanks,
Christopher


On Wed, Nov 30, 2011 at 12:41 PM, Granroth, Neal V. 
neal.granr...@thermofisher.com wrote:

 Or maybe put the changes within a conditional compile code block?

 Thanks DIGY, works great.

 - Neal

 -Original Message-
 From: Prescott Nasser [mailto:geobmx...@hotmail.com]
 Sent: Wednesday, November 30, 2011 2:35 PM
 To: lucene-net-dev@lucene.apache.org
 Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2

 Probably makes for a good wiki entry

 Sent from my Windows Phone
 
 From: Digy
 Sent: 11/30/2011 12:04 PM
 To: lucene-net-dev@lucene.apache.org
 Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2

 OK, here is the code that can be compiled against .NET 2.0
 http://pastebin.com/k2f7JfPd

 DIGY


 -Original Message-
 From: Granroth, Neal V. [mailto:neal.granr...@thermofisher.com]
 Sent: Wednesday, November 30, 2011 9:26 PM
 To: lucene-net-dev@lucene.apache.org
 Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2

 DIGY,

 Thanks for the tip, but could you be a little more specific?
 Where and how are extension-methods calls replaced?

 For example, how would I change the CloseableThreadLocalExtensions method

public static void SetT(this ThreadLocalT t, T val)
{
t.Value = val;
}


 to eliminate the compile error

 Error   2   Cannot define a new extension method because the compiler
 required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot
 be
 found. Are you missing a reference to System.Core.dll?

 due to the SetT parameter this ThreadLocalt t ?


 - Neal


 -Original Message-
 From: Digy [mailto:digyd...@gmail.com]
 Sent: Wednesday, November 30, 2011 12:27 PM
 To: lucene-net-dev@lucene.apache.org
 Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2

 FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
 CloseableThreadLocal

 (like uncommenting ThreadLocalT class and replacing extension-methods
 calls

 with static calls to CloseableThreadLocalExtensions)





 DIGY





 -Original Message-
 From: Christopher Currens [mailto:currens.ch...@gmail.com]
 Sent: Wednesday, November 30, 2011 7:26 PM
 To: lucene-net-dev@lucene.apache.org
 Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2



 Trevor,



 I'm not sure if you can use 2.9.4, though, it looks like you're using

 VS2005 and .NET 2.0.  2.9.4 targets .NET 4.0, and I'm fairly certain we
use

 classes only available in 4.0 (or 3.5?).  However, if you can, I would

 suggest updating, as 2.9.4 should be a fairly stable release.



 The leak I'm talking about is addressed here:

 https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code

 isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4.  It
may

 or may not be what your issue is.  You say that it was at one time working

 fine, I assume you mean no memory leak.  I would take some time to see
what

 else in your code has changed.  Make sure you're calling Close on whatever

 needs to be closed (IndexWriter/IndexReader/Analyzers, etc).



 Unfortunately for us, memory leaks are hard to debug over email, and it's

 difficult for us to tell if it's any change to your code or an issue with

 Lucene .NET.  As far as I can tell, this is the only memory leak I can
find

 that affects 2.9.2.





 Thanks,

 Christopher



 On Wed, Nov 30, 2011 at 8:25 AM, Prescott Nasser
 geobmx...@hotmail.comwrote:



  We just released 2.9.4 - the website didn't update last night, so ill
 have

  to try and update it later today. But if you follow the link to download

  2.9.2

RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2

2011-11-30 Thread Digy
... and it was related with CloseableThreadLocal (fixed in 2.9.4(g)) which
now creates 
compilation problem against .Net20 :)

DIGY

-Original Message-
From: Digy [mailto:digyd...@gmail.com] 
Sent: Thursday, December 01, 2011 12:09 AM
To: 'lucene-net-dev@lucene.apache.org'
Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2

If I recall it correctly, last memory leak problem for 2.9.2 was reported in
~August from RavenDB, and it was fixed in 2.9.4(g)

DIGY

-Original Message-
From: Christopher Currens [mailto:currens.ch...@gmail.com] 
Sent: Wednesday, November 30, 2011 11:33 PM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2

Trevor,

Unforunately I was unable to reproduce the memory leak you're experiencing
in 2.9.2.  Particularly with byte[], of the 18,277 that were created, only
13 were not garbage collected, and it's likely that they are not related to
Lucene (it's possible they are static, therefore would only be destroyed
with the AppDomain, outside of what the profiler can trace).  I tried to
emulate the code you showed us and there were no signs of any allocated
arrays that weren't cleaned up.  That doesn't mean there isn't one in your
code, but I just can't reproduce it with what you've shown us.  If it's
possible you can write a small program that has the same behavior, that
could help us track it down.

As a side note, what was a little disconcerting, though, was in 2.9.4 with
the same code, it created 28,565 byte[], and there was quite a few more
left uncollected (2,805 arrays).  The allocations are happening in
DocumentsWriter.ByteBlockAllocator, I'll have to look at it later though,
to see if its even a problem.


Thanks,
Christopher


On Wed, Nov 30, 2011 at 12:41 PM, Granroth, Neal V. 
neal.granr...@thermofisher.com wrote:

 Or maybe put the changes within a conditional compile code block?

 Thanks DIGY, works great.

 - Neal

 -Original Message-
 From: Prescott Nasser [mailto:geobmx...@hotmail.com]
 Sent: Wednesday, November 30, 2011 2:35 PM
 To: lucene-net-dev@lucene.apache.org
 Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2

 Probably makes for a good wiki entry

 Sent from my Windows Phone
 
 From: Digy
 Sent: 11/30/2011 12:04 PM
 To: lucene-net-dev@lucene.apache.org
 Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2

 OK, here is the code that can be compiled against .NET 2.0
 http://pastebin.com/k2f7JfPd

 DIGY


 -Original Message-
 From: Granroth, Neal V. [mailto:neal.granr...@thermofisher.com]
 Sent: Wednesday, November 30, 2011 9:26 PM
 To: lucene-net-dev@lucene.apache.org
 Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2

 DIGY,

 Thanks for the tip, but could you be a little more specific?
 Where and how are extension-methods calls replaced?

 For example, how would I change the CloseableThreadLocalExtensions method

public static void SetT(this ThreadLocalT t, T val)
{
t.Value = val;
}


 to eliminate the compile error

 Error   2   Cannot define a new extension method because the compiler
 required type 'System.Runtime.CompilerServices.ExtensionAttribute' cannot
 be
 found. Are you missing a reference to System.Core.dll?

 due to the SetT parameter this ThreadLocalt t ?


 - Neal


 -Original Message-
 From: Digy [mailto:digyd...@gmail.com]
 Sent: Wednesday, November 30, 2011 12:27 PM
 To: lucene-net-dev@lucene.apache.org
 Subject: RE: [Lucene.Net] Re: Memory Leak in 2.9.2.2

 FYI, 2.9.4 can be compiled against .Net 2.0 with a few minor changes in
 CloseableThreadLocal

 (like uncommenting ThreadLocalT class and replacing extension-methods
 calls

 with static calls to CloseableThreadLocalExtensions)





 DIGY





 -Original Message-
 From: Christopher Currens [mailto:currens.ch...@gmail.com]
 Sent: Wednesday, November 30, 2011 7:26 PM
 To: lucene-net-dev@lucene.apache.org
 Subject: Re: [Lucene.Net] Re: Memory Leak in 2.9.2.2



 Trevor,



 I'm not sure if you can use 2.9.4, though, it looks like you're using

 VS2005 and .NET 2.0.  2.9.4 targets .NET 4.0, and I'm fairly certain we
use

 classes only available in 4.0 (or 3.5?).  However, if you can, I would

 suggest updating, as 2.9.4 should be a fairly stable release.



 The leak I'm talking about is addressed here:

 https://issues.apache.org/jira/browse/LUCENE-2467, and the ported code

 isn't available in 2.9.2, but I've confirmed the patch is in 2.9.4.  It
may

 or may not be what your issue is.  You say that it was at one time working

 fine, I assume you mean no memory leak.  I would take some time to see
what

 else in your code has changed.  Make sure you're calling Close on whatever

 needs to be closed (IndexWriter/IndexReader/Analyzers, etc).



 Unfortunately for us, memory leaks are hard to debug over email, and it's

 difficult for us to tell if it's any change to your code or an issue with

 Lucene .NET.  As far as I can tell

RE: [Lucene.Net] Roadmap

2011-11-21 Thread Digy

Chris,

Sorry, if you took my comments about pain of porting personally. That
wasn't my intension.

+1 for all your changes/divergences. I made/could have made them too.

DIGY

-Original Message-
From: Christopher Currens [mailto:currens.ch...@gmail.com] 
Sent: Monday, November 21, 2011 11:45 PM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] Roadmap

Digy,

I used 2.9.4 trunk as the base for the 3.0.3 branch, but I looked to the
code in 2.9.4g as a reference for many things, particularly the Support
classes.  We hit many of the same issues I'm sure, I moved some of the
anonymous classes into a base class where you could inject functions,
though not all could be replaced, nor did I replace all that could have
been.  Some of our code is different, I went for the option for
WeakDictionary to be completely generic, as in wrapping a generic
dictionary with WeakKeyT instead of wrapping the already existing
WeakHashTable in support.  In hindsight, it may have just been easier to
convert the WeakHashTable to generic, but alas, I'm only realizing that
now.  There is a problem with my WeakDictionary, specifically the function
that determines when to clean/compact the dictionary and remove the dead
keys.  I need a better heuristic of deciding when to run the clean.  That's
a performance issue though.

Regarding the pain of porting, I am a changed man.  It's nice, in a sad
way, to know that I'm not the only one who experienced those difficulties.
 I used to be in the camp that porting code that differed from java
wouldn't be difficult at all.  However, now I code corrected!  It threw me
a curve-ball, for sure.  I DO think a line-by-line port can definitely
include the things talked about below, ie the changes to Dispose and the
changes to IEnumerableT.  Those changes, I thing, can be made without a
heavy impact on the porting process.

There was one fairly large change I opted to use that differed quite a bit
from Java, however, and that was the use of the TPL in
ParallelMultiSearcher.  It was far easier to port this way, and I don't
think it affects the porting process too much.  Java uses a helper class
defined at the bottom of the source file that handles it, I'm simply using
a built-in one instead.  I just need to be careful about it, it would be
really easy to get carried away with it.


Thanks,
Christopher

On Mon, Nov 21, 2011 at 1:20 PM, Digy digyd...@gmail.com wrote:

 Hi Chris,

 First of all, thank you for your great work on 3.0.3 branch.
 I suppose you took 2.9.4 as a code base to make 3.0.3 port since some of
 your problems are the same with those I faced in 2.9.4g branch.
 (e.g,
Support/MemoryMappedDirectory.cs (but never used in core),
IDisposable,
introduction of some ActionTs, FuncTs ,
foreach instead of GetEnumerator/MoveNext,
IEquatableT,
WeakDictionaryT,
SetT
etc.
 )

 Since I also used 3.0.3 as a reference, maybe we can use some of 2.9.4g's
 code in 3.0.3 when necessary(I haven't had time to look into 3.0.3 deeply)

 Just to ensure the coordination, maybe you should create a new issue in
 JIRA, so that people send patches to that issue instead of directly
 commiting.


 @Prescott,
 2.9.4g is not behind of 2.9.4 in bug fixes  features level. So, It is (I
 think) ready for another release.(I use it in all my projects since long).


 PS: Hearing the pain of porting codes that greatly differ from Java made
 me just smile( sorry for that:( ). Be ready for responses that get beyond
 the criticism between With all due respect  Just my $0.02
paranthesis.

 DIGY

 -Original Message-
 From: Christopher Currens [mailto:currens.ch...@gmail.com]
 Sent: Monday, November 21, 2011 10:19 PM
 To: lucene-net-dev@lucene.apache.org; casper...@caspershouse.com
 Subject: Re: [Lucene.Net] Roadmap

 Some of the Lucene classes have Dispose methods, well, ones that call
Close
 (and that Close method may or may not call base.Close(), if needed or
not).
  Virtual dispose methods can be dangerous only in that they're easy to
 implement wrong.  However, it shouldn't be too bad, at least with a
 line-by-line port, as we would make the call to the base class whenever
 Lucene does, and that would (should) give us the same behavior,
implemented
 properly.  I'm not aware of differences in the JVM, regarding inheritance
 and base methods being called automatically, particularly Close methods.

 Slightly unrelated, another annoyance is the use of Java Iterators vs C#
 Enumerables.  A lot of our code is there simply because there are
 Iterators, but it could be converted to Enumerables. The whole HasNext,
 Next vs C#'s MoveNext(), Current is annoying, but it's used all over in
the
 base code, and would have to be changed there as well.  Either way, I
would
 like to push for that before 3.0.3 is relased.  IMO, small changes like
 this still keep the code similar to the line-by-line port, in that it
 doesn't add any difficulties

RE: [Lucene.Net] Roadmap

2011-11-21 Thread Digy
My english isn't enough to understand this answer. I hope it is not related
with
employee-employer relationship as in the past.

DIGY

-Original Message-
From: Christopher Currens [mailto:currens.ch...@gmail.com] 
Sent: Tuesday, November 22, 2011 1:08 AM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] Roadmap

To clarify, it wasn't as much *difficult* as it was more *painful*.  Above,
I was inferring that it was more difficult that the rest of the code, which
by comparison was easier.  It wasn't painless to try and map where code
changes were from the java classes into the .Net version.  I prefer that
style more for its readability and the niceties of working with a .Net
style of Lucene, however as I said before, it slowed down significantly the
porting process.  I hope it didn't come across that I thought that it was
bad code, because it's probably the most readable code we have in the
Contrib at the moment.

I want to make it clear that my intention right now is to get Lucene.Net up
to date with Java.  When I read the Java code, I understand its intent, and
I make sure the ported code represents it.  That takes enough time as it
is, moving to try and figure out where the code went in Lucene.Net, since
it wasn't a 1-1 map, was a MINOR annoyance, especially when you compare it
to the issues I had dealing with the differences between the two languages,
generics especialy.  That being said, I don't have a problem with code
being converted in a .Net idiomatic way, in fact, I welcome it, if it still
allows the changes to be ported with minimal effort.  I feel at this point
in the project, there are some limitations to how far I'd like it to
diverge.

Anyway, my opinion, which may not be in agreement with the group as a
whole, is that it would be better to bring the codebase up to date, or at
least more up to date with java's, and then maintaining a version with a
complete .net-concentric API.  I feel this would beeasier, as porting
Java's Lucene SVN commits by the week would be a relatively small workload.

On Mon, Nov 21, 2011 at 2:41 PM, Troy Howard thowar...@gmail.com wrote:

 So, if we're getting back to the line by line port discussion... I
 think either side of this discussion is too extreme. For the case in
 point Chris just mentioned (which I'm not really sure what part was so
 difficult, as I ported that library in about 30 minutes from
 scratch)... anything is a pain if it sticks out in the middle of doing
 something completely different.

 The only reason we are able to do this line by line is due to the
 general similarity between Java and C#'s language syntax. If we were
 porting Lucene to a completely different language, that had a totally
 different syntax, the process would go like this:

 - Look at the original code, understand it's intent
 - Create similar code in the new language that expresses the same intent

 When applying changes:

 - Look at the original code diffs, understanding the intent of the change
 - Look at the ported code, and apply the changed logic's meaning in
 that language

 So, is just a different thought process. In my opinion, it's a better
 process because it forces the developer to actually think about the
 code instead of blindly converting syntax (possibly slightly
 incorrectly and introducing regressions). While there is a large
 volume of unit tests in Lucene, they are unfortunately not really the
 right tests and make porting much more difficult, because it's hard to
 verify that your ported code behaves the same because you can't just
 rely on the unit tests to verify your port. Therefore, it's safer to
 follow a process that requires the developer to delve deeply into the
 meaning of the code. Following a line-by-line process is convenient,
 but doesn't focus on meaning, which I think is more important.

 Thanks,
 Troy

 On Mon, Nov 21, 2011 at 2:23 PM, Christopher Currens
 currens.ch...@gmail.com wrote:
  Digy,
 
  No worries.  I wasn't taking them personally.  You've been doing this
 for a
  lot longer than I have, but I didn't understand you pain until I had to
 go
  through it personally. :P
 
  Have you looked at Contrib in a while?  There's a lot of projects that
 are
  in Java's Contrib that are not in Lucene.Net?  Is this because there are
  some that can't easily (if at all) be ported over to .NET or just
because
  they've been neglected?  I'm trying to get a handle on what's important
 to
  port and what isn't.  Figured someone with experience could help me with
 a
  starting point over deciding where to start with everything that's
 missing.
 
 
  Thanks,
  Christopher
 
  On Mon, Nov 21, 2011 at 2:13 PM, Digy digyd...@gmail.com wrote:
 
 
  Chris,
 
  Sorry, if you took my comments about pain of porting personally. That
  wasn't my intension.
 
  +1 for all your changes/divergences. I made/could have made them too.
 
  DIGY
 
  -Original Message-
  From: Christopher Currens [mailto:currens.ch...@gmail.com]
  Sent

RE: [Lucene.Net] Roadmap

2011-11-21 Thread Digy
Hi Chris,

First of all, thank you for your great work on 3.0.3 branch. 
I suppose you took 2.9.4 as a code base to make 3.0.3 port since some of
your problems are the same with those I faced in 2.9.4g branch. 
(e.g, 
Support/MemoryMappedDirectory.cs (but never used in core), 
IDisposable, 
introduction of some ActionTs, FuncTs , 
foreach instead of GetEnumerator/MoveNext,
IEquatableT,
WeakDictionaryT,
SetT
etc.
)

Since I also used 3.0.3 as a reference, maybe we can use some of 2.9.4g's
code in 3.0.3 when necessary(I haven't had time to look into 3.0.3 deeply)

Just to ensure the coordination, maybe you should create a new issue in
JIRA, so that people send patches to that issue instead of directly
commiting.


@Prescott,
2.9.4g is not behind of 2.9.4 in bug fixes  features level. So, It is (I
think) ready for another release.(I use it in all my projects since long).


PS: Hearing the pain of porting codes that greatly differ from Java made
me just smile( sorry for that:( ). Be ready for responses that get beyond
the criticism between With all due respect  Just my $0.02 paranthesis.

DIGY

-Original Message-
From: Christopher Currens [mailto:currens.ch...@gmail.com] 
Sent: Monday, November 21, 2011 10:19 PM
To: lucene-net-...@lucene.apache.org; casper...@caspershouse.com
Subject: Re: [Lucene.Net] Roadmap

Some of the Lucene classes have Dispose methods, well, ones that call Close
(and that Close method may or may not call base.Close(), if needed or not).
 Virtual dispose methods can be dangerous only in that they're easy to
implement wrong.  However, it shouldn't be too bad, at least with a
line-by-line port, as we would make the call to the base class whenever
Lucene does, and that would (should) give us the same behavior, implemented
properly.  I'm not aware of differences in the JVM, regarding inheritance
and base methods being called automatically, particularly Close methods.

Slightly unrelated, another annoyance is the use of Java Iterators vs C#
Enumerables.  A lot of our code is there simply because there are
Iterators, but it could be converted to Enumerables. The whole HasNext,
Next vs C#'s MoveNext(), Current is annoying, but it's used all over in the
base code, and would have to be changed there as well.  Either way, I would
like to push for that before 3.0.3 is relased.  IMO, small changes like
this still keep the code similar to the line-by-line port, in that it
doesn't add any difficulties in the porting process, but provides great
benefits to the users of the code, to have a .NET centric API.  I don't
think it would violate our project desciption we have listed on our
Incubator page, either.


Thanks,
Christopher

On Mon, Nov 21, 2011 at 12:03 PM, casper...@caspershouse.com 
casper...@caspershouse.com wrote:

 +1 on the suggestion to move Close - IDisposable; not being able to use
 using is such a pain, and an eyesore on the code.


 Although it will have to be done properly, and not just have Dispose call
 Close (you should have proper protected virtual Dispose methods to take
 inheritance into account, etc).


 - Nick

 

 From: Christopher Currens currens.ch...@gmail.com

 Sent: Monday, November 21, 2011 2:56 PM

 To: lucene-net-...@lucene.apache.org

 Subject: Re: [Lucene.Net] Roadmap


 Regarding the 3.0.3 branch I started last week, I've put in a lot of late

 nights and gotten far more done in a week and a half than I expected.  The

 list of changes is very large, and fortunately, I've documented it in some

 files that are in the branches root of certain projects.  I'll list what

 changes have been made so far, and some of the concerns I have about them,

 as well as what still needs to be done.  You can read them all in detail
 in

 the files that are in the branch.


 All changes in 3.0.3 have been ported to the Lucene.Net and

 Lucene.Net.Test, except BooleanClause, LockStressTest, MMapDirectory,

 NIOFSDirectory, DummyConcurrentLock, NamedThreadFactory, and

 ThreadInterruptedException.


 MMapDirectory and NIOFSDirectory have never been ported in the first place

 for 2.9.4, so I'm not worried about those.  LockStressTest is a

 command-line tool, porting it should be easy, but not essential to a 3.0.3

 release, IMO.  DummyConcurrentLock also seems unnecessary (and

 non-portable) for .NET, since it's based around Java's Lock class and is

 only used to bypass locking, which can be done by passing new Object() to

 the method.

 NamedThreadFactory I'm unsure about.  It's used in ParallelMultiSearcher

 (in which I've opted to use the TPL), and seems to be only used for

 debugging, possibly testing.  Either way, I'm not sure it's necessary.

 Also, named threads would mean we probably would have to move the class

 from the TPL, which greatly simplified the code and parallelization of it

 all, as I can't see a way to Set names for a Task.  I suppose

RE: [Lucene.Net] Roadmap

2011-11-21 Thread Digy
Troy,
I am not againt it if you can continue to understand and port so easyly. 
No one here -I think- wants a java-tastes code.

DIGY

-Original Message-
From: Troy Howard [mailto:thowar...@gmail.com] 
Sent: Tuesday, November 22, 2011 12:42 AM
To: lucene-net-...@lucene.apache.org
Subject: Re: [Lucene.Net] Roadmap

So, if we're getting back to the line by line port discussion... I
think either side of this discussion is too extreme. For the case in
point Chris just mentioned (which I'm not really sure what part was so
difficult, as I ported that library in about 30 minutes from
scratch)... anything is a pain if it sticks out in the middle of doing
something completely different.

The only reason we are able to do this line by line is due to the
general similarity between Java and C#'s language syntax. If we were
porting Lucene to a completely different language, that had a totally
different syntax, the process would go like this:

- Look at the original code, understand it's intent
- Create similar code in the new language that expresses the same intent

When applying changes:

- Look at the original code diffs, understanding the intent of the change
- Look at the ported code, and apply the changed logic's meaning in
that language

So, is just a different thought process. In my opinion, it's a better
process because it forces the developer to actually think about the
code instead of blindly converting syntax (possibly slightly
incorrectly and introducing regressions). While there is a large
volume of unit tests in Lucene, they are unfortunately not really the
right tests and make porting much more difficult, because it's hard to
verify that your ported code behaves the same because you can't just
rely on the unit tests to verify your port. Therefore, it's safer to
follow a process that requires the developer to delve deeply into the
meaning of the code. Following a line-by-line process is convenient,
but doesn't focus on meaning, which I think is more important.

Thanks,
Troy

On Mon, Nov 21, 2011 at 2:23 PM, Christopher Currens
currens.ch...@gmail.com wrote:
 Digy,

 No worries.  I wasn't taking them personally.  You've been doing this for a
 lot longer than I have, but I didn't understand you pain until I had to go
 through it personally. :P

 Have you looked at Contrib in a while?  There's a lot of projects that are
 in Java's Contrib that are not in Lucene.Net?  Is this because there are
 some that can't easily (if at all) be ported over to .NET or just because
 they've been neglected?  I'm trying to get a handle on what's important to
 port and what isn't.  Figured someone with experience could help me with a
 starting point over deciding where to start with everything that's missing.


 Thanks,
 Christopher

 On Mon, Nov 21, 2011 at 2:13 PM, Digy digyd...@gmail.com wrote:


 Chris,

 Sorry, if you took my comments about pain of porting personally. That
 wasn't my intension.

 +1 for all your changes/divergences. I made/could have made them too.

 DIGY

 -Original Message-
 From: Christopher Currens [mailto:currens.ch...@gmail.com]
 Sent: Monday, November 21, 2011 11:45 PM
 To: lucene-net-...@lucene.apache.org
 Subject: Re: [Lucene.Net] Roadmap

 Digy,

 I used 2.9.4 trunk as the base for the 3.0.3 branch, but I looked to the
 code in 2.9.4g as a reference for many things, particularly the Support
 classes.  We hit many of the same issues I'm sure, I moved some of the
 anonymous classes into a base class where you could inject functions,
 though not all could be replaced, nor did I replace all that could have
 been.  Some of our code is different, I went for the option for
 WeakDictionary to be completely generic, as in wrapping a generic
 dictionary with WeakKeyT instead of wrapping the already existing
 WeakHashTable in support.  In hindsight, it may have just been easier to
 convert the WeakHashTable to generic, but alas, I'm only realizing that
 now.  There is a problem with my WeakDictionary, specifically the function
 that determines when to clean/compact the dictionary and remove the dead
 keys.  I need a better heuristic of deciding when to run the clean.  That's
 a performance issue though.

 Regarding the pain of porting, I am a changed man.  It's nice, in a sad
 way, to know that I'm not the only one who experienced those difficulties.
  I used to be in the camp that porting code that differed from java
 wouldn't be difficult at all.  However, now I code corrected!  It threw me
 a curve-ball, for sure.  I DO think a line-by-line port can definitely
 include the things talked about below, ie the changes to Dispose and the
 changes to IEnumerableT.  Those changes, I thing, can be made without a
 heavy impact on the porting process.

 There was one fairly large change I opted to use that differed quite a bit
 from Java, however, and that was the use of the TPL in
 ParallelMultiSearcher.  It was far easier to port this way, and I don't
 think it affects the porting process too

RE: [Lucene.Net] Final patches for rc3

2011-11-10 Thread Digy
What do folks think about including LUCENENET-427, LUCENENET-431,
LUCENENET-433  LUCENENET-443 fixes in that release and removing
Similarity.Net?

DIGY

-Original Message-
From: Prescott Nasser [mailto:geobmx...@hotmail.com] 
Sent: Thursday, November 10, 2011 10:19 PM
To: lucene-net-dev@lucene.apache.org
Subject: [Lucene.Net] Final patches for rc3

Im reviewing all the issues before I cut rc3. Looks like I have the
following:

.apply lucenenet-453 patch from Stefan to fix the line endings
.couple of license headers 
.update the readme
.add an rc3 tag.

Is there anything im missing? I will have some time tonight to fix these
items and then prepare new files for another vote.

-P

Sent from my Windows Phone
-

Checked by AVG - www.avg.com
Version: 2012.0.1869 / Virus Database: 2092/4608 - Release Date: 11/10/11



[Lucene.Net] [jira] [Updated] (LUCENENET-448) GeoHashFilteredDocIdSet does not work at all

2011-11-01 Thread Digy (Updated) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy updated LUCENENET-448:
---

Affects Version/s: (was: Lucene.Net 2.9.4)
   Lucene.Net 2.9.4g
Fix Version/s: (was: Lucene.Net 2.9.4)
   Lucene.Net 2.9.4g

 GeoHashFilteredDocIdSet does not work at all
 

 Key: LUCENENET-448
 URL: https://issues.apache.org/jira/browse/LUCENENET-448
 Project: Lucene.Net
  Issue Type: Bug
  Components: Lucene.Net Contrib
Affects Versions: Lucene.Net 2.9.4g
 Environment: Windows 7 x64
Reporter: Jeff Johnson
  Labels: contrib, spatial
 Fix For: Lucene.Net 2.9.4g

   Original Estimate: 1h
  Remaining Estimate: 1h

 The GeoHashFilteredDocIdSet is assuming the values are always in the cache 
 which is wrong. A proposed fix for the method is listed here for 
 GeoHashDistanceFilter.cs:
 public GeoHashFilteredDocIdSet(DocIdSet innerSet, string[] geoHashValues, 
 Dictionarystring, double distanceLookupCache, double lat, double lng, int 
 docBase, double distance, Dictionaryint, double distances) 
 : base(innerSet , (docid) = /* was: public override Match */
   {
   String geoHash = geoHashValues[docid];
   double[] coords = GeoHashUtils.Decode(geoHash);
   double x = coords[0];
   double y = coords[1];
   double cachedDistance;
 distanceLookupCache.TryGetValue(geoHash, out cachedDistance);
   double d;
   if (cachedDistance  0)
   {
   d = cachedDistance;
   }
   else
   {
   d = 
 DistanceUtils.GetInstance().GetDistanceMi(lat, lng, x, y);
   distanceLookupCache[geoHash] = d;
   }
   if (d  distance)
   {
   distances[docid + docBase] = d;
   return true;
   }
   
   return false;
   })
   {
   _geoHashValues = geoHashValues;
   _distances = distances;
   _distance = distance;
   _docBase = docBase;
   _lng = lng;
   _lat = lat;
   _distanceLookupCache = distanceLookupCache;
   }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Resolved] (LUCENENET-448) GeoHashFilteredDocIdSet does not work at all

2011-11-01 Thread Digy (Resolved) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy resolved LUCENENET-448.


Resolution: Fixed

Hi Jeff,
Your fix is for 2.9.4g branch and I committed it.
(Although I was hard to compare your code with the current code).
Thank you.

 GeoHashFilteredDocIdSet does not work at all
 

 Key: LUCENENET-448
 URL: https://issues.apache.org/jira/browse/LUCENENET-448
 Project: Lucene.Net
  Issue Type: Bug
  Components: Lucene.Net Contrib
Affects Versions: Lucene.Net 2.9.4g
 Environment: Windows 7 x64
Reporter: Jeff Johnson
  Labels: contrib, spatial
 Fix For: Lucene.Net 2.9.4g

   Original Estimate: 1h
  Remaining Estimate: 1h

 The GeoHashFilteredDocIdSet is assuming the values are always in the cache 
 which is wrong. A proposed fix for the method is listed here for 
 GeoHashDistanceFilter.cs:
 public GeoHashFilteredDocIdSet(DocIdSet innerSet, string[] geoHashValues, 
 Dictionarystring, double distanceLookupCache, double lat, double lng, int 
 docBase, double distance, Dictionaryint, double distances) 
 : base(innerSet , (docid) = /* was: public override Match */
   {
   String geoHash = geoHashValues[docid];
   double[] coords = GeoHashUtils.Decode(geoHash);
   double x = coords[0];
   double y = coords[1];
   double cachedDistance;
 distanceLookupCache.TryGetValue(geoHash, out cachedDistance);
   double d;
   if (cachedDistance  0)
   {
   d = cachedDistance;
   }
   else
   {
   d = 
 DistanceUtils.GetInstance().GetDistanceMi(lat, lng, x, y);
   distanceLookupCache[geoHash] = d;
   }
   if (d  distance)
   {
   distances[docid + docBase] = d;
   return true;
   }
   
   return false;
   })
   {
   _geoHashValues = geoHashValues;
   _distances = distances;
   _distance = distance;
   _docBase = docBase;
   _lng = lng;
   _lat = lat;
   _distanceLookupCache = distanceLookupCache;
   }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-450) Incorrect use of StreamReader in MoreLikeThis

2011-11-01 Thread Digy (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13141418#comment-13141418
 ] 

Digy commented on LUCENENET-450:


Similarity net is obsolete and all of its functionality has been moved to 
Contrib/Queries as in java(including unit tests).
https://svn.apache.org/repos/asf/incubator/lucene.net/trunk/src/contrib/README.txt



 Incorrect use of StreamReader in MoreLikeThis
 -

 Key: LUCENENET-450
 URL: https://issues.apache.org/jira/browse/LUCENENET-450
 Project: Lucene.Net
  Issue Type: Bug
  Components: Lucene.Net Contrib
Affects Versions: Lucene.Net 2.9.2, Lucene.Net 2.9.4, Lucene.Net 3.x
Reporter: Itamar Syn-Hershko
Assignee: Prescott Nasser
 Fix For: Lucene.Net 2.9.4

 Attachments: Lucenenet-450.patch


 In the MoreLike this implementation (looking at the trunk), line 772 
 incorrectly creates a new StreamReader instead of using StringReader. This 
 causes the following exception to be thrown, since the ctor expects a file 
 path and not an input string:
 System.ArgumentException: Illegal characters in path.
at 
 System.Security.Permissions.FileIOPermission.HasIllegalCharacters(String[] 
 str)
at 
 System.Security.Permissions.FileIOPermission.AddPathList(FileIOPermissionAccess
  access, AccessControlActions control, String[] pathListOrig, Boolean 
 checkForDuplicates, Boolean needFullPath, Boolean copyPathList)
at 
 System.Security.Permissions.FileIOPermission..ctor(FileIOPermissionAccess 
 access, AccessControlActions control, String[] pathList, Boolean 
 checkForDuplicates, Boolean needFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess 
 access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, 
 FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean 
 bFromProxy, Boolean useLongPath)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess 
 access, FileShare share, Int32 bufferSize, FileOptions options, String 
 msgPath, Boolean bFromProxy)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess 
 access, FileShare share, Int32 bufferSize, FileOptions options)
at System.IO.StreamReader..ctor(String path, Encoding encoding, Boolean 
 detectEncodingFromByteOrderMarks, Int32 bufferSize)
at System.IO.StreamReader..ctor(String path, Boolean 
 detectEncodingFromByteOrderMarks)
at Similarity.Net.MoreLikeThis.RetrieveTerms(Int32 docNum) in 
 MoreLikeThis.cs:line 773
at Similarity.Net.MoreLikeThis.Like(Int32 docNum) in MoreLikeThis.cs:line 
 507
 Simply replacing StreamReader with StringReader will do the job

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Closed] (LUCENENET-441) Encountered: EOF after : \\\\ during parsing a query

2011-11-01 Thread Digy (Closed) (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy closed LUCENENET-441.
--

Resolution: Invalid

Maverick904, Please use mailing lists to ask question.

 Encountered: EOF after :  during parsing a query
 --

 Key: LUCENENET-441
 URL: https://issues.apache.org/jira/browse/LUCENENET-441
 Project: Lucene.Net
  Issue Type: Bug
  Components: Lucene.Net Core
Affects Versions: Lucene.Net 2.9.2
 Environment: .Net Framework 4.0
Reporter: Maverick904

 Cannot parse '\': Lexical error at line 1, column 4.  Encountered: EOF 
 after :  |at Lucene.Net.QueryParsers.QueryParser.Parse(String 
 query)
at Lucene.Net.QueryParsers.MultiFieldQueryParser.Parse(Version 
 matchVersion, String query, String[] fields, Occur[] flags, Analyzer analyzer)
at Lucene.Net.QueryParsers.MultiFieldQueryParser.Parse(String query, 
 String[] fields, Occur[] flags, Analyzer analyzer)

  
 Lexical error at line 1, column 4.  Encountered: EOF after :  |
 at Lucene.Net.QueryParsers.QueryParserTokenManager.GetNextToken()
at Lucene.Net.QueryParsers.QueryParser.Jj_ntk()
at Lucene.Net.QueryParsers.QueryParser.Modifiers()
at Lucene.Net.QueryParsers.QueryParser.Query(String field)
at Lucene.Net.QueryParsers.QueryParser.Parse(String query) | 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira




RE: [Lucene.Net] 2.9.4

2011-09-21 Thread Digy
 Similarity was include, though are there any tests for this project ?
Similarity is obsolete (Queries.Net replaces it  has test cases). It has
already been removed in 2.9.4g

DIGY

-Original Message-
From: Michael Herndon [mailto:mhern...@wickedsoftware.net] 
Sent: Wednesday, September 21, 2011 10:40 PM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] 2.9.4

@all,

I updated the build scripts to increase it's granularity.
https://cwiki.apache.org/LUCENENET/build-system-scripts.html

Similarity was include, though are there any tests for this project ?

Some of the contrib tests are failing, I saw a few in Contrib.Highlighter
just glancing at the output .

I recieved some feedback Eric Woodruff. It looks like SHFB  Sandcastle
generate a plain file html, its been staring me in the face this whole time.
 I'll need to build in some targets that extract whats needed to push to
site branch. Then I'll start working on nuget.

@Prescott,
Can the volatile fields be wrapped in a lock statement and code that access
those fields with replaced with call to a property /method that wraps access
to that field?




On Wed, Sep 21, 2011 at 1:36 PM, Troy Howard thowar...@gmail.com wrote:

 I thought it was:

 2.9.2 and before are 2.0 compatible
 2.9.4 and before are 3.5 compatible
 After 2.9.4 are 4.0 compatible

 Thanks,
 Troy

 On Wed, Sep 21, 2011 at 10:15 AM, Michael Herndon
 mhern...@wickedsoftware.net wrote:
  if thats the case, then well need conditional statements for including
  ThreadLocalT
 
  On Wed, Sep 21, 2011 at 12:47 PM, Prescott Nasser geobmx...@hotmail.com
 wrote:
 
  I thought this was after 2.9.4
 
  Sent from my Windows Phone
 
  -Original Message-
  From: Michael Herndon
  Sent: Wednesday, September 21, 2011 8:30 AM
  To: lucene-net-dev@lucene.apache.org
  Cc: lucene-net-...@incubator.apache.org
  Subject: Re: [Lucene.Net] 2.9.4
 
  @Robert,
 
  I believe the overwhelming consensus on the mailing list vote was to
 move
  to
  .NET 4.0 and drop support for previous versions.
 
  I'll take care of build scripts issue while they being refactored into
  smaller chunks this week.
 
  @Troy, Agreed.
 
  On Wed, Sep 21, 2011 at 8:08 AM, Robert Jordan robe...@gmx.net wrote:
 
   On 20.09.2011 23:48, Prescott Nasser wrote:
  
   Hey all seems like we are set with 2.9.4? Feedback has been positive
 and
   its been quiet. Do we feel ready to vote for a new release?
  
  
   I don't know if the build infrastructure is part of the
   release. If yes, then there is an open issue:
  
   Contrib doesn't build right now because there
   are some assembly name mismatches between certain *.csproj
   files and  build/scripts/contrib.targets.
  
   The following patches should fix the issue:
  
   https://github.com/robert-j/**lucene.net/commit/**
   c5218bca56c19b3407648224781eec**7316994a39
 

https://github.com/robert-j/lucene.net/commit/c5218bca56c19b3407648224781eec
7316994a39
  
  
   https://github.com/robert-j/**lucene.net/commit/**
   50bad187655d59968d51d472b57c2a**40e201d663
 

https://github.com/robert-j/lucene.net/commit/50bad187655d59968d51d472b57c2a
40e201d663
  
  
  
   Also, the fix for [LUCENENET-358] is basically making
   Lucene.Net.dll a .NET 4.0-only assembly:
  
   https://github.com/apache/**lucene.net/commit/**
   23ea6f52362fc7dbce48fd012cea12**9a7350c73c
 

https://github.com/apache/lucene.net/commit/23ea6f52362fc7dbce48fd012cea129a
7350c73c
  
  
   Did we agree about abandoning .NET = 3.5?
  
   Robert
  
  
 
 


-

Checked by AVG - www.avg.com
Version: 2012.0.1809 / Virus Database: 2085/4510 - Release Date: 09/21/11



RE: [Lucene.Net] 2.9.4

2011-09-21 Thread Digy
@Robert 

 Also, the fix for [LUCENENET-358] is basically making Lucene.Net.dll a
.NET 4.0-only assembly:

There is a commented part at the end of the CloseableThreadLocal which may
seem familiar to you :)
No harm in uncommenting it and no conditional compilation is needed. 
It also pass all test cases.

DIGY



-Original Message-
From: Robert Jordan [mailto:robe...@gmx.net] 
Sent: Wednesday, September 21, 2011 3:09 PM
To: lucene-net-...@incubator.apache.org
Subject: Re: [Lucene.Net] 2.9.4

On 20.09.2011 23:48, Prescott Nasser wrote:
 Hey all seems like we are set with 2.9.4? Feedback has been positive and
its been quiet. Do we feel ready to vote for a new release?

I don't know if the build infrastructure is part of the
release. If yes, then there is an open issue:

Contrib doesn't build right now because there
are some assembly name mismatches between certain *.csproj
files and  build/scripts/contrib.targets.

The following patches should fix the issue:

https://github.com/robert-j/lucene.net/commit/c5218bca56c19b3407648224781eec
7316994a39

https://github.com/robert-j/lucene.net/commit/50bad187655d59968d51d472b57c2a
40e201d663


Also, the fix for [LUCENENET-358] is basically making
Lucene.Net.dll a .NET 4.0-only assembly:

https://github.com/apache/lucene.net/commit/23ea6f52362fc7dbce48fd012cea129a
7350c73c

Did we agree about abandoning .NET = 3.5?

Robert

-

Checked by AVG - www.avg.com
Version: 2012.0.1809 / Virus Database: 2085/4510 - Release Date: 09/21/11



RE: [Lucene.Net] 2.9.4

2011-09-21 Thread Digy
You are right in race condition  NullReferenceException. 
but 
static SupportClass.WeakHashTable slots = new
SupportClass.WeakHashTable();
wouldn't work since it is intented to be created in all threads not once.

Would you patch it or leave it to me?

Thanks,
DIGY

-Original Message-
From: Robert Jordan [mailto:robe...@gmx.net] 
Sent: Thursday, September 22, 2011 1:16 AM
To: lucene-net-...@incubator.apache.org
Subject: Re: [Lucene.Net] 2.9.4

Hi Digy,

On 21.09.2011 23:38, Digy wrote:
 @Robert

 Also, the fix for [LUCENENET-358] is basically making Lucene.Net.dll a
 .NET 4.0-only assembly:

 There is a commented part at the end of the CloseableThreadLocal which may
 seem familiar to you :)

Indeed :) I've missed this comment.

 No harm in uncommenting it and no conditional compilation is needed.
 It also pass all test cases.

BTW, there is an issue with this commented-out code. If Value
is not accessed at least once, Dispose() will fail with a
NullReferenceException. There is also a little chance for
a race condition.

I'd rather get rid of Init() for this code:

static SupportClass.WeakHashTable slots = new SupportClass.WeakHashTable();

Robert


 DIGY



 -Original Message-
 From: Robert Jordan [mailto:robe...@gmx.net]
 Sent: Wednesday, September 21, 2011 3:09 PM
 To: lucene-net-...@incubator.apache.org
 Subject: Re: [Lucene.Net] 2.9.4

 On 20.09.2011 23:48, Prescott Nasser wrote:
 Hey all seems like we are set with 2.9.4? Feedback has been positive and
 its been quiet. Do we feel ready to vote for a new release?

 I don't know if the build infrastructure is part of the
 release. If yes, then there is an open issue:

 Contrib doesn't build right now because there
 are some assembly name mismatches between certain *.csproj
 files and  build/scripts/contrib.targets.

 The following patches should fix the issue:


https://github.com/robert-j/lucene.net/commit/c5218bca56c19b3407648224781eec
 7316994a39


https://github.com/robert-j/lucene.net/commit/50bad187655d59968d51d472b57c2a
 40e201d663


 Also, the fix for [LUCENENET-358] is basically making
 Lucene.Net.dll a .NET 4.0-only assembly:


https://github.com/apache/lucene.net/commit/23ea6f52362fc7dbce48fd012cea129a
 7350c73c

 Did we agree about abandoning .NET= 3.5?

 Robert

 -

 Checked by AVG - www.avg.com
 Version: 2012.0.1809 / Virus Database: 2085/4510 - Release Date: 09/21/11




-

Checked by AVG - www.avg.com
Version: 2012.0.1809 / Virus Database: 2085/4510 - Release Date: 09/21/11



RE: [Lucene.Net] 2.9.4

2011-09-21 Thread Digy
I reconsidered it and there is no race condition.  A new slot will be
created for each thread.
But NullReferenceException bug is still there.

DIGY

-Original Message-
From: Robert Jordan [mailto:robe...@gmx.net] 
Sent: Thursday, September 22, 2011 1:16 AM
To: lucene-net-...@incubator.apache.org
Subject: Re: [Lucene.Net] 2.9.4

Hi Digy,

On 21.09.2011 23:38, Digy wrote:
 @Robert

 Also, the fix for [LUCENENET-358] is basically making Lucene.Net.dll a
 .NET 4.0-only assembly:

 There is a commented part at the end of the CloseableThreadLocal which may
 seem familiar to you :)

Indeed :) I've missed this comment.

 No harm in uncommenting it and no conditional compilation is needed.
 It also pass all test cases.

BTW, there is an issue with this commented-out code. If Value
is not accessed at least once, Dispose() will fail with a
NullReferenceException. There is also a little chance for
a race condition.

I'd rather get rid of Init() for this code:

static SupportClass.WeakHashTable slots = new SupportClass.WeakHashTable();

Robert


 DIGY



 -Original Message-
 From: Robert Jordan [mailto:robe...@gmx.net]
 Sent: Wednesday, September 21, 2011 3:09 PM
 To: lucene-net-...@incubator.apache.org
 Subject: Re: [Lucene.Net] 2.9.4

 On 20.09.2011 23:48, Prescott Nasser wrote:
 Hey all seems like we are set with 2.9.4? Feedback has been positive and
 its been quiet. Do we feel ready to vote for a new release?

 I don't know if the build infrastructure is part of the
 release. If yes, then there is an open issue:

 Contrib doesn't build right now because there
 are some assembly name mismatches between certain *.csproj
 files and  build/scripts/contrib.targets.

 The following patches should fix the issue:


https://github.com/robert-j/lucene.net/commit/c5218bca56c19b3407648224781eec
 7316994a39


https://github.com/robert-j/lucene.net/commit/50bad187655d59968d51d472b57c2a
 40e201d663


 Also, the fix for [LUCENENET-358] is basically making
 Lucene.Net.dll a .NET 4.0-only assembly:


https://github.com/apache/lucene.net/commit/23ea6f52362fc7dbce48fd012cea129a
 7350c73c

 Did we agree about abandoning .NET= 3.5?

 Robert

 -

 Checked by AVG - www.avg.com
 Version: 2012.0.1809 / Virus Database: 2085/4510 - Release Date: 09/21/11




-

Checked by AVG - www.avg.com
Version: 2012.0.1809 / Virus Database: 2085/4510 - Release Date: 09/21/11



RE: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

2011-09-21 Thread Digy


http://blogs.msdn.com/b/microsoft_press/archive/2010/02/03/jeffrey-richter-e
xcerpt-2-from-clr-via-c-third-edition.aspx

Yes, this is the trick some obfuscators use.(they use also some scrambling
fxns to hide the code in resource)

DIGY


-Original Message-
From: Michael Herndon [mailto:mhern...@wickedsoftware.net] 
Sent: Thursday, September 22, 2011 1:36 AM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

@Digy, that could be done post build with ILMerge or build an additional
uber assembly that stores other assemblies as a resource.
http://blogs.msdn.com/b/microsoft_press/archive/2010/02/03/jeffrey-richter-e
xcerpt-2-from-clr-via-c-third-edition.aspx

We can add the above to the build process if that would interest people.

To some nuget is just another disruption and  to others its a godsend.  Some
might say only hipsters would use nuget, others might say the cools kids
with iphones use nuget. (or android or wp7).

At the end of the day nuget or combining assemblies are just channels/ways
we can make it easier for various developers to consume  get their hands on
Lucene.Net. If anyone else has ideas along those lines and it can be
automated, post it in this thread.





On Wed, Sep 21, 2011 at 6:00 PM, Digy digyd...@gmail.com wrote:

 Even all contribs could be a single project/assembly. That way, users
could
 reference all contribs with a single assembly.
 I see no harm in putting a few KB pressure on RAM :)

 DIGY


 -Original Message-
 From: Troy Howard [mailto:thowar...@gmail.com]
 Sent: Wednesday, September 21, 2011 7:32 AM
 To: lucene-net-dev@lucene.apache.org
 Subject: Re: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

 While it may be a bit redundant, why couldn't there be an individual
 package for each piece of contrib and a Lucene.Net Contrib (All)
 package that drags them all down.

 That way users can grab just the bit they need, or if they just want
 to get the whole thing, grab the All package.

 Thanks,
 Troy


 On Tue, Sep 20, 2011 at 9:11 PM, Aaron Powell m...@aaron-powell.com wrote:
  I'm going to vote +1 for granular.
 
  With the RC you could look at myget and have a Lucene.Net repository on
 there so people can go for unstable on myget, stables on nuget.
 
  Also, I came across this article which explains how to setup a build
 server to automatically push to nuget/ myget which could be useful to the
 maintainers:
 http://brendanforster.com/doing-the-build-server-dance-with-nuget.html
 
  Aaron Powell
  MVP - Internet Explorer (Development) | FunnelWeb Team Member
 
  http://apowell.me | http://twitter.com/slace | Skype: aaron.l.powell |
 Github | BitBucket
 
  -Original Message-
  From: Prescott Nasser [mailto:geobmx...@hotmail.com]
  Sent: Wednesday, 21 September 2011 2:05 PM
  To: lucene-net-dev@lucene.apache.org
  Subject: RE: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts
 
 
  Right now there are two packages: Lucene  Lucene.Contrib. My question
  to the community is do you wish to finer grain packages, i.e. a
  package for each contrib project or continue to keep it simple.
 
 
 
  +1 Granular, we just need to be good about descriptions.
 
 
 
  Another topic to converse about is would you like to see an
  out-of-band project nuget feed for nightly builds, branches with new
  or experimental features, or stable code snapshots for a projected
 release?
 
 
  Having a package for the latest RC would probably be a good idea
 
 -

 Checked by AVG - www.avg.com
 Version: 2012.0.1808 / Virus Database: 2085/4508 - Release Date: 09/20/11



-

Checked by AVG - www.avg.com
Version: 2012.0.1809 / Virus Database: 2085/4510 - Release Date: 09/21/11



RE: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

2011-09-21 Thread Digy
I am not against it, but personally think it as a toy.
I am from the generation where people used vi to write codes.

DIGY

-Original Message-
From: Aaron Powell [mailto:m...@aaron-powell.com] 
Sent: Thursday, September 22, 2011 1:56 AM
To: lucene-net-dev@lucene.apache.org
Subject: RE: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

Any particular reason you guys are not interested in NuGet?

Aaron Powell
MVP - Internet Explorer (Development) | FunnelWeb Team Member

http://apowell.me | http://twitter.com/slace | Skype: aaron.l.powell |
Github | BitBucket 


-Original Message-
From: Digy [mailto:digyd...@gmail.com] 
Sent: Thursday, 22 September 2011 7:42 AM
To: lucene-net-dev@lucene.apache.org
Subject: RE: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

Sorry, but I feel the same as Neal.

DIGY

-Original Message-
From: Granroth, Neal V. [mailto:neal.granr...@thermofisher.com]
Sent: Wednesday, September 21, 2011 6:08 PM
To: lucene-net-dev@lucene.apache.org
Subject: RE: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

No interest in Nuget whatsoever.

- Neal

-Original Message-
From: Michael Herndon [mailto:mhern...@wickedsoftware.net]
Sent: Tuesday, September 20, 2011 10:57 PM
To: lucene-net-dev@lucene.apache.org; lucene-net-u...@lucene.apache.org
Subject: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

We're taking a quick poll over the next few days to see how people would
like use Lucene.Net through Nuget on the developers mailing list**

Currently version 2.9.2 is hosted on nuget.org, but that package was not
create by the project maintainers, thus nuget is not currently set up in
source.  Going forward, we would like to continue what someone else started
by creating nuget packages for Lucene.Net.

Right now there are two packages: Lucene  Lucene.Contrib.  My question to
the community is do you wish to finer grain packages, i.e. a package for
each contrib project or continue to keep it simple.

The granular approach will let you use only what you need. We can also
create additional higher level packages which have dependencies on the other
ones.   Possibly a Lucene.Net-Essentials and Lucene.Net-Full.

Or we can keep it simple and continue with only two packages.

My concerns are that the granular approach might overwhelm people with
choice. The simple choice might be considered bloat for importing and then
installing assemblies that you might never use.


Another topic to converse about is would you like to see an out-of-band
project nuget feed for  nightly builds, branches with new or experimental
features, or stable code snapshots for a projected release?


** when you post, please respond to lucene-net-dev@lucene.apache.org.  This
was posted to both lists to make sure everyone subscribed to both lists has
a chance to voice their use cases or concerns.
-

Checked by AVG - www.avg.com
Version: 2012.0.1809 / Virus Database: 2085/4510 - Release Date: 09/21/11

-

Checked by AVG - www.avg.com
Version: 2012.0.1809 / Virus Database: 2085/4510 - Release Date: 09/21/11



RE: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

2011-09-21 Thread Digy
Not that old :)
DIGY

-Original Message-
From: Prescott Nasser [mailto:geobmx...@hotmail.com] 
Sent: Thursday, September 22, 2011 2:14 AM
To: lucene-net-dev@lucene.apache.org
Subject: RE: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

Punch cards or bust!

Sent from my Windows Phone

-Original Message-
From: Digy
Sent: Wednesday, September 21, 2011 4:06 PM
To: lucene-net-dev@lucene.apache.org
Subject: RE: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

I am not against it, but personally think it as a toy.
I am from the generation where people used vi to write codes.

DIGY

-Original Message-
From: Aaron Powell [mailto:m...@aaron-powell.com]
Sent: Thursday, September 22, 2011 1:56 AM
To: lucene-net-dev@lucene.apache.org
Subject: RE: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

Any particular reason you guys are not interested in NuGet?

Aaron Powell
MVP - Internet Explorer (Development) |�FunnelWeb Team Member

http://apowell.me�|�http://twitter.com/slace�| Skype: aaron.l.powell |
Github | BitBucket


-Original Message-
From: Digy [mailto:digyd...@gmail.com]
Sent: Thursday, 22 September 2011 7:42 AM
To: lucene-net-dev@lucene.apache.org
Subject: RE: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

Sorry, but I feel the same as Neal.

DIGY

-Original Message-
From: Granroth, Neal V. [mailto:neal.granr...@thermofisher.com]
Sent: Wednesday, September 21, 2011 6:08 PM
To: lucene-net-dev@lucene.apache.org
Subject: RE: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

No interest in Nuget whatsoever.

- Neal

-Original Message-
From: Michael Herndon [mailto:mhern...@wickedsoftware.net]
Sent: Tuesday, September 20, 2011 10:57 PM
To: lucene-net-dev@lucene.apache.org; lucene-net-u...@lucene.apache.org
Subject: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

We're taking a quick poll over the next few days to see how people would
like use Lucene.Net through Nuget on the developers mailing list**

Currently version 2.9.2 is hosted on nuget.org, but that package was not
create by the project maintainers, thus nuget is not currently set up in
source.  Going forward, we would like to continue what someone else started
by creating nuget packages for Lucene.Net.

Right now there are two packages: Lucene  Lucene.Contrib.  My question to
the community is do you wish to finer grain packages, i.e. a package for
each contrib project or continue to keep it simple.

The granular approach will let you use only what you need. We can also
create additional higher level packages which have dependencies on the other
ones.   Possibly a Lucene.Net-Essentials and Lucene.Net-Full.

Or we can keep it simple and continue with only two packages.

My concerns are that the granular approach might overwhelm people with
choice. The simple choice might be considered bloat for importing and then
installing assemblies that you might never use.


Another topic to converse about is would you like to see an out-of-band
project nuget feed for  nightly builds, branches with new or experimental
features, or stable code snapshots for a projected release?


** when you post, please respond to lucene-net-dev@lucene.apache.org.  This
was posted to both lists to make sure everyone subscribed to both lists has
a chance to voice their use cases or concerns.
-

Checked by AVG - www.avg.com
Version: 2012.0.1809 / Virus Database: 2085/4510 - Release Date: 09/21/11

-

Checked by AVG - www.avg.com
Version: 2012.0.1809 / Virus Database: 2085/4510 - Release Date: 09/21/11

-

Checked by AVG - www.avg.com
Version: 2012.0.1809 / Virus Database: 2085/4510 - Release Date: 09/21/11



RE: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

2011-09-21 Thread Digy
Sorry, but I feel the same as Neal.

DIGY

-Original Message-
From: Granroth, Neal V. [mailto:neal.granr...@thermofisher.com] 
Sent: Wednesday, September 21, 2011 6:08 PM
To: lucene-net-...@lucene.apache.org
Subject: RE: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

No interest in Nuget whatsoever.

- Neal

-Original Message-
From: Michael Herndon [mailto:mhern...@wickedsoftware.net] 
Sent: Tuesday, September 20, 2011 10:57 PM
To: lucene-net-...@lucene.apache.org; lucene-net-u...@lucene.apache.org
Subject: [Lucene.Net] Nuget, Lucene.Net, and Your Thoughts

We're taking a quick poll over the next few days to see how people would
like use Lucene.Net through Nuget on the developers mailing list**

Currently version 2.9.2 is hosted on nuget.org, but that package was not
create by the project maintainers, thus nuget is not currently set up in
source.  Going forward, we would like to continue what someone else started
by creating nuget packages for Lucene.Net.

Right now there are two packages: Lucene  Lucene.Contrib.  My question to
the community is do you wish to finer grain packages, i.e. a package for
each contrib project or continue to keep it simple.

The granular approach will let you use only what you need. We can also
create additional higher level packages which have dependencies on the other
ones.   Possibly a Lucene.Net-Essentials and Lucene.Net-Full.

Or we can keep it simple and continue with only two packages.

My concerns are that the granular approach might overwhelm people with
choice. The simple choice might be considered bloat for importing and then
installing assemblies that you might never use.


Another topic to converse about is would you like to see an out-of-band
project nuget feed for  nightly builds, branches with new or experimental
features, or stable code snapshots for a projected release?


** when you post, please respond to lucene-net-...@lucene.apache.org.  This
was posted to both lists to make sure everyone subscribed to both lists has
a chance to voice their use cases or concerns.
-

Checked by AVG - www.avg.com
Version: 2012.0.1809 / Virus Database: 2085/4510 - Release Date: 09/21/11



RE: [Lucene.Net] 2.9.4

2011-09-21 Thread Digy
I think, there was a sync problem between our eMails :(

DIGY

-Original Message-
From: Robert Jordan [mailto:robe...@gmx.net] 
Sent: Thursday, September 22, 2011 1:22 AM
To: lucene-net-...@incubator.apache.org
Subject: Re: [Lucene.Net] 2.9.4

On 22.09.2011 00:16, Robert Jordan wrote:
 Hi Digy,

 On 21.09.2011 23:38, Digy wrote:
 @Robert

 Also, the fix for [LUCENENET-358] is basically making Lucene.Net.dll a
 .NET 4.0-only assembly:

 There is a commented part at the end of the CloseableThreadLocal which
 may
 seem familiar to you :)

 Indeed :) I've missed this comment.

 No harm in uncommenting it and no conditional compilation is needed.
 It also pass all test cases.

 BTW, there is an issue with this commented-out code. If Value
 is not accessed at least once, Dispose() will fail with a
 NullReferenceException. There is also a little chance for
 a race condition.

Scratch this. The whole point of ThreadLocalT is actually
Init(). Sorry for the noise :)

Robert

-

Checked by AVG - www.avg.com
Version: 2012.0.1809 / Virus Database: 2085/4510 - Release Date: 09/21/11



[Lucene.Net] [jira] [Updated] (LUCENENET-444) Snowball stemmers (Portuguese, Hungarian, Romanian, Turkish)

2011-09-15 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy updated LUCENENET-444:
---

Fix Version/s: (was: Lucene.Net 3.x)
   Lucene.Net 2.9.4g

 Snowball stemmers (Portuguese, Hungarian, Romanian, Turkish)
 

 Key: LUCENENET-444
 URL: https://issues.apache.org/jira/browse/LUCENENET-444
 Project: Lucene.Net
  Issue Type: New Feature
Reporter: Digy
Priority: Trivial
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g


 Some missing stemmers + a modified portuguese stemmer.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Updated] (LUCENENET-444) Snowball stemmers (Portuguese, Hungarian, Romanian, Turkish)

2011-09-15 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy updated LUCENENET-444:
---

Attachment: PortugueseStemmer.cs
TurkishStemmer.cs
RomanianStemmer.cs
HungarianStemmer.cs

 Snowball stemmers (Portuguese, Hungarian, Romanian, Turkish)
 

 Key: LUCENENET-444
 URL: https://issues.apache.org/jira/browse/LUCENENET-444
 Project: Lucene.Net
  Issue Type: New Feature
Reporter: Digy
Priority: Trivial
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g

 Attachments: HungarianStemmer.cs, PortugueseStemmer.cs, 
 RomanianStemmer.cs, TurkishStemmer.cs


 Some missing stemmers + a modified portuguese stemmer.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Resolved] (LUCENENET-444) Snowball stemmers (Portuguese, Hungarian, Romanian, Turkish)

2011-09-15 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy resolved LUCENENET-444.


Resolution: Fixed

 Snowball stemmers (Portuguese, Hungarian, Romanian, Turkish)
 

 Key: LUCENENET-444
 URL: https://issues.apache.org/jira/browse/LUCENENET-444
 Project: Lucene.Net
  Issue Type: New Feature
Reporter: Digy
Priority: Trivial
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g

 Attachments: HungarianStemmer.cs, PortugueseStemmer.cs, 
 RomanianStemmer.cs, TurkishStemmer.cs


 Some missing stemmers + a modified portuguese stemmer.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-444) Snowball stemmers (Portuguese, Hungarian, Romanian, Turkish)

2011-09-15 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105418#comment-13105418
 ] 

Digy commented on LUCENENET-444:


committed to trunk  2.9.4g branch.

 Snowball stemmers (Portuguese, Hungarian, Romanian, Turkish)
 

 Key: LUCENENET-444
 URL: https://issues.apache.org/jira/browse/LUCENENET-444
 Project: Lucene.Net
  Issue Type: New Feature
Reporter: Digy
Priority: Trivial
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g

 Attachments: HungarianStemmer.cs, PortugueseStemmer.cs, 
 RomanianStemmer.cs, TurkishStemmer.cs


 Some missing stemmers + a modified portuguese stemmer.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-443) SpellChecker finaliser calls close regardless of if closed already

2011-09-15 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13105478#comment-13105478
 ] 

Digy commented on LUCENENET-443:


+1 for IDisposable in 2.9.4g (since 
Analyzers,Searchers,Directories,IndexReader,IndexWriter already implement it).

 SpellChecker finaliser calls close regardless of if closed already
 --

 Key: LUCENENET-443
 URL: https://issues.apache.org/jira/browse/LUCENENET-443
 Project: Lucene.Net
  Issue Type: Improvement
  Components: Lucene.Net Contrib
Affects Versions: Lucene.Net 2.9.2
Reporter: Stuart Robinson
  Labels: lucene, spellcheck, spellchecker

 The SpellChecker Class currently has no publicly visible way of accessing the 
 closed field. It also calls close in the finaliser killing the process it is 
 in upon GC as this can throw an exceptin. I propose two changes:
 Change the already existing method IsClosed() to public:
 public bool IsClosed()
 {
 return closed;
 }
 and add a check on this in the finaliser:
 ~SpellChecker()
 {
 if (!IsClosed())
 this.Close();
 }
 Ideally this class should implement IDisposable but I think this would be a 
 bigger job than this two line change.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




RE: [Lucene.Net] Test case for: possible infinite loop bug in portuguese snowball stemmer?

2011-09-13 Thread Digy
I created a working portuguese stemmer (
http://people.apache.org/~digy/PortugueseStemmerNew.cs ) from 
  http://snowball.tartarus.org/archives/snowball-discuss/0943.html
 
http://snowball.tartarus.org/archives/snowball-discuss/att-0943/01-SnowballC
Sharp.zip

Since it has a BSD license (http://snowball.tartarus.org/license.php), I
don't think I can update the PortugueseStemmer.cs under contrib.

DIGY

-Original Message-
From: Robert Stewart [mailto:robert_stew...@epam.com] 
Sent: Tuesday, September 13, 2011 5:55 PM
To: lucene-net-...@lucene.apache.org
Subject: Re: [Lucene.Net] Test case for: possible infinite loop bug in
portuguese snowball stemmer?

Here is a test case:

string text = @Califórnia;

Lucene.Net.Analysis.KeywordTokenizer tokenizer = new KeywordTokenizer(new
StringReader(text));

Lucene.Net.Analysis.Snowball.SnowballFilter stemmer=
new Lucene.Net.Analysis.Snowball.SnowballFilter(tokenizer,
Portuguese);

Lucene.Net.Analysis.Token token;

while ((token = stemmer.Next()) != null)
{
System.Console.WriteLine(tokenText);

}

Seems to go into infinite loop.  Call to stemmer.Next() never returns.  Not
sure if this is the only stemmer I am having trouble with.  And it does
happen to us on a near daily basis.  

Thanks,
Bob


On Sep 13, 2011, at 9:37 AM, Robert Stewart wrote:

 Are there any known issues with snowball stemmers (portuguese in
particular) going into some infinite loop?  I have a problem that happens on
a recurring basis where IndexWriter locks up on AddDocument and never
returns (it has taken up to 3 days before we realize it), requiring manual
killing of the process.  It seems to happen only on portuguese documents
from what I can tell so far, and the stack trace when thread is aborted is
always as follows:
 
 System.Threading.ThreadAbortException: Thread was being aborted.
   at System.RuntimeMethodHandle._InvokeMethodFast(IRuntimeMethodInfo
method, Object target, Object[] arguments, SignatureStruct sig,
MethodAttributes methodAttributes, RuntimeType typeOwner)
   at System.RuntimeMethodHandle.InvokeMethodFast(IRuntimeMethodInfo
method, Object target, Object[] arguments, Signature sig, MethodAttributes
methodAttributes, RuntimeType typeOwner)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags
invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean
skipVisibilityChecks)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags
invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Lucene.Net.Analysis.Snowball.SnowballFilter.Next()
 System.SystemException: System.Threading.ThreadAbortException: Thread was
being aborted.
   at System.RuntimeMethodHandle._InvokeMethodFast(IRuntimeMethodInfo
method, Object target, Object[] arguments, SignatureStruct sig,
MethodAttributes methodAttributes, RuntimeType typeOwner)
   at System.RuntimeMethodHandle.InvokeMethodFast(IRuntimeMethodInfo
method, Object target, Object[] arguments, Signature sig, MethodAttributes
methodAttributes, RuntimeType typeOwner)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags
invokeAttr, Binder binder, Object[] parameters, CultureInfo culture, Boolean
skipVisibilityChecks)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags
invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Lucene.Net.Analysis.Snowball.SnowballFilter.Next()
   at Lucene.Net.Analysis.Snowball.SnowballFilter.Next()
   at Lucene.Net.Analysis.TokenStream.IncrementToken()
   at Lucene.Net.Index.DocInverterPerField.ProcessFields(Fieldable[]
fields, Int32 count)
   at Lucene.Net.Index.DocFieldProcessorPerThread.ProcessDocument()
   at Lucene.Net.Index.DocumentsWriter.UpdateDocument(Document doc,
Analyzer analyzer, Term delTerm)
   at Lucene.Net.Index.IndexWriter.AddDocument(Document doc, Analyzer
analyzer)
 
 
 Is there another list of contrib/snowball issues?  I have not been able to
reproduce a small test case yet however.  Have there been any such issues
with stemmers in the past?
 
 Thanks,
 Bob

-

Checked by AVG - www.avg.com
Version: 2012.0.1796 / Virus Database: 2082/4494 - Release Date: 09/13/11



RE: [Lucene.Net] 2.9.4

2011-09-10 Thread Digy
Good news. Thanks Itamar.

DIGY

-Original Message-
From: itamar.synhers...@gmail.com [mailto:itamar.synhers...@gmail.com] On
Behalf Of Itamar Syn-Hershko
Sent: Saturday, September 10, 2011 8:23 PM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] 2.9.4

We have been running some extensive tests 30hrs now against the 2.9.4
branch, and did not detect any leaks. We will have it running a few more
days, if you wish to wait for more conclusive findings.

On Wed, Sep 7, 2011 at 5:07 PM, Prescott Nasser
geobmx...@hotmail.comwrote:

 2.9.4 would make it in I assume because that will be our next official
 release.


 Sent from my Windows Phone

 -Original Message-
 From: Michael Herndon
 Sent: Wednesday, September 07, 2011 5:12 AM
 To: lucene-net-dev@lucene.apache.org
 Subject: Re: [Lucene.Net] 2.9.4

  What version is going to make it to nuget? 2.9.4 or 2.9.4g?
 ooo totally forgot about nuget. we definitely need to get that setup.


 On Wed, Sep 7, 2011 at 6:46 AM, digy digy digyd...@gmail.com wrote:

  Since it includes some level of divergence from java I committed it to
 only
  2.9.4g branch.
 
  https://issues.apache.org/jira/browse/LUCENE-1930
  https://issues.apache.org/jira/browse/LUCENENET-431
 
  DIGY
 
  On Wed, Sep 7, 2011 at 1:03 PM, Itamar Syn-Hershko ita...@code972.com
  wrote:
 
   Ok, core compiles, and all tests pass. We are now running long tests
to
   measure memory usage among other things.
  
   There is one show stopper tho. There was a patch sent by Matt Warren
 for
   Spatial.Net, that doesn't seem to be in. See
   http://groups.google.com/group/ravendb/msg/7517f095810c48f3
  
   Any chance you can get it in to 2.9.4?
  
   On Wed, Sep 7, 2011 at 1:01 AM, Itamar Syn-Hershko ita...@code972.com
   wrote:
  
Ok, great, we will run RavenDB on top of 2.9.4 in the next few days
 and
will let you know how it went.
   
   
On Tue, Sep 6, 2011 at 8:59 PM, Michael Herndon 
mhern...@wickedsoftware.net wrote:
   
I can't tell if the apache git mirror is updated via scheduler or
 from
commit hooks, but its generally stays close to being on par with
 svn.
 I'll
check next time I push something to svn.
   
But both of those items have made it to the mirror.
   
- michael
   
   
On Tue, Sep 6, 2011 at 1:44 PM, Digy digyd...@gmail.com wrote:
   
 I don't know how often github mirror is updated.
 These are the original locations
 2.9.4
 https://svn.apache.org/repos/asf/incubator/lucene.net/trunk/
 2.9.4g


   
  
 

https://svn.apache.org/repos/asf/incubator/lucene.net/branches/Lucene.Net_2_
 9_4g/

 Both versions include ThreadLocal fix + Signing.

 Thanks,
 DIGY



 -Original Message-
 From: itamar.synhers...@gmail.com [mailto:
  itamar.synhers...@gmail.com
   ]
On
 Behalf Of Itamar Syn-Hershko
 Sent: Tuesday, September 06, 2011 2:34 AM
 To: lucene-net-dev@lucene.apache.org
 Subject: Re: [Lucene.Net] 2.9.4

 Not a problem, we will test RavenDB on a separate branch, also
for
 potential
 memory leaks

 Digy, can you make sure the github mirror contains an updated
 2.9.4
   tag
I
 can pull from, which includes the latest ThreadLocal fix + the
   strongly
 signed patch applied to it?

 2011/9/6 Digy digyd...@gmail.com

  To avoid misunderstanding...
 
  Community==all Lucene.Net users
 
  DIGY
 
  -Original Message-
  From: Digy [mailto:digyd...@gmail.com]
  Sent: Monday, September 05, 2011 11:46 PM
  To: 'lucene-net-dev@lucene.apache.org'
  Subject: RE: [Lucene.Net] 2.9.4
 
  Not bad idea, but I would prefer community's feedback instead
of
testing
  against all projects using Lucene.Net
  DIGY
 
  -Original Message-
  From: Matt Warren [mailto:mattd...@gmail.com]
  Sent: Monday, September 05, 2011 11:09 PM
  To: lucene-net-dev@lucene.apache.org
  Subject: Re: [Lucene.Net] 2.9.4
 
  If you want to test it against a large project you could take a
  look
at
 how
  RavenDB uses it?
 
  At the moment it's using 2.9.2 (
 
 


   
  
 

https://github.com/ayende/ravendb/tree/master/SharedLibs/Sources/Lucene2.9.2
  )
  but if you were to recompile it against 2.9.4 and check that
all
   it's
  unit-tests still run that would give you quite a large test
 case.
 
  On 5 September 2011 19:22, Prescott Nasser 
 geobmx...@hotmail.com
  
 wrote:
 
  
   Hey All,
  
   How do people feel about the 2.9.4 code base? I've been using
 it
   for
   sometime, for my use cases it's be excellent. Do we feel we
 are
ready
 to
   package this up and make it an official release? Or do we
have
   some
 tasks
   left to take care of?
  
   ~Prescott

Re: [Lucene.Net] How to add document to more than one index (but only analyze once)?

2011-09-09 Thread digy digy
How about indexing the new document(s) in memory using RAMDirectory then
calling indexWriter.AddIndexesNoOptimize for NRT  master index?

DIGY

On Fri, Sep 9, 2011 at 5:33 PM, Robert Stewart robert_stew...@epam.comwrote:

 Is it possible to add a document to more than one index at the same time,
 such that document fields are only analyzed one time?  For instance, to add
 document to both a master index, and a smaller near real-time index.  I
 would like to avoid analyzing document fields more than once but I dont see
 if that is possible at all using Lucene API.

 Thanks,
 Bob


Re: [Lucene.Net] 2.9.4

2011-09-07 Thread digy digy
Since it includes some level of divergence from java I committed it to only
2.9.4g branch.

https://issues.apache.org/jira/browse/LUCENE-1930
https://issues.apache.org/jira/browse/LUCENENET-431

DIGY

On Wed, Sep 7, 2011 at 1:03 PM, Itamar Syn-Hershko ita...@code972.comwrote:

 Ok, core compiles, and all tests pass. We are now running long tests to
 measure memory usage among other things.

 There is one show stopper tho. There was a patch sent by Matt Warren for
 Spatial.Net, that doesn't seem to be in. See
 http://groups.google.com/group/ravendb/msg/7517f095810c48f3

 Any chance you can get it in to 2.9.4?

 On Wed, Sep 7, 2011 at 1:01 AM, Itamar Syn-Hershko ita...@code972.com
 wrote:

  Ok, great, we will run RavenDB on top of 2.9.4 in the next few days and
  will let you know how it went.
 
 
  On Tue, Sep 6, 2011 at 8:59 PM, Michael Herndon 
  mhern...@wickedsoftware.net wrote:
 
  I can't tell if the apache git mirror is updated via scheduler or from
  commit hooks, but its generally stays close to being on par with svn.
   I'll
  check next time I push something to svn.
 
  But both of those items have made it to the mirror.
 
  - michael
 
 
  On Tue, Sep 6, 2011 at 1:44 PM, Digy digyd...@gmail.com wrote:
 
   I don't know how often github mirror is updated.
   These are the original locations
   2.9.4  https://svn.apache.org/repos/asf/incubator/lucene.net/trunk/
   2.9.4g
  
  
 
 https://svn.apache.org/repos/asf/incubator/lucene.net/branches/Lucene.Net_2_
   9_4g/
  
   Both versions include ThreadLocal fix + Signing.
  
   Thanks,
   DIGY
  
  
  
   -Original Message-
   From: itamar.synhers...@gmail.com [mailto:itamar.synhers...@gmail.com
 ]
  On
   Behalf Of Itamar Syn-Hershko
   Sent: Tuesday, September 06, 2011 2:34 AM
   To: lucene-net-dev@lucene.apache.org
   Subject: Re: [Lucene.Net] 2.9.4
  
   Not a problem, we will test RavenDB on a separate branch, also for
   potential
   memory leaks
  
   Digy, can you make sure the github mirror contains an updated 2.9.4
 tag
  I
   can pull from, which includes the latest ThreadLocal fix + the
 strongly
   signed patch applied to it?
  
   2011/9/6 Digy digyd...@gmail.com
  
To avoid misunderstanding...
   
Community==all Lucene.Net users
   
DIGY
   
-Original Message-
From: Digy [mailto:digyd...@gmail.com]
Sent: Monday, September 05, 2011 11:46 PM
To: 'lucene-net-dev@lucene.apache.org'
Subject: RE: [Lucene.Net] 2.9.4
   
Not bad idea, but I would prefer community's feedback instead of
  testing
against all projects using Lucene.Net
DIGY
   
-Original Message-
From: Matt Warren [mailto:mattd...@gmail.com]
Sent: Monday, September 05, 2011 11:09 PM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] 2.9.4
   
If you want to test it against a large project you could take a look
  at
   how
RavenDB uses it?
   
At the moment it's using 2.9.2 (
   
   
  
  
 
 https://github.com/ayende/ravendb/tree/master/SharedLibs/Sources/Lucene2.9.2
)
but if you were to recompile it against 2.9.4 and check that all
 it's
unit-tests still run that would give you quite a large test case.
   
On 5 September 2011 19:22, Prescott Nasser geobmx...@hotmail.com
   wrote:
   

 Hey All,

 How do people feel about the 2.9.4 code base? I've been using it
 for
 sometime, for my use cases it's be excellent. Do we feel we are
  ready
   to
 package this up and make it an official release? Or do we have
 some
   tasks
 left to take care of?

 ~Prescott
   
  
  
  
 
 
 



RE: [Lucene.Net] 2.9.4

2011-09-06 Thread Digy
I don't know how often github mirror is updated.
These are the original locations 
2.9.4  https://svn.apache.org/repos/asf/incubator/lucene.net/trunk/ 
2.9.4g
https://svn.apache.org/repos/asf/incubator/lucene.net/branches/Lucene.Net_2_
9_4g/

Both versions include ThreadLocal fix + Signing.

Thanks,
DIGY



-Original Message-
From: itamar.synhers...@gmail.com [mailto:itamar.synhers...@gmail.com] On
Behalf Of Itamar Syn-Hershko
Sent: Tuesday, September 06, 2011 2:34 AM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] 2.9.4

Not a problem, we will test RavenDB on a separate branch, also for potential
memory leaks

Digy, can you make sure the github mirror contains an updated 2.9.4 tag I
can pull from, which includes the latest ThreadLocal fix + the strongly
signed patch applied to it?

2011/9/6 Digy digyd...@gmail.com

 To avoid misunderstanding...

 Community==all Lucene.Net users

 DIGY

 -Original Message-
 From: Digy [mailto:digyd...@gmail.com]
 Sent: Monday, September 05, 2011 11:46 PM
 To: 'lucene-net-dev@lucene.apache.org'
 Subject: RE: [Lucene.Net] 2.9.4

 Not bad idea, but I would prefer community's feedback instead of testing
 against all projects using Lucene.Net
 DIGY

 -Original Message-
 From: Matt Warren [mailto:mattd...@gmail.com]
 Sent: Monday, September 05, 2011 11:09 PM
 To: lucene-net-dev@lucene.apache.org
 Subject: Re: [Lucene.Net] 2.9.4

 If you want to test it against a large project you could take a look at
how
 RavenDB uses it?

 At the moment it's using 2.9.2 (


https://github.com/ayende/ravendb/tree/master/SharedLibs/Sources/Lucene2.9.2
 )
 but if you were to recompile it against 2.9.4 and check that all it's
 unit-tests still run that would give you quite a large test case.

 On 5 September 2011 19:22, Prescott Nasser geobmx...@hotmail.com wrote:

 
  Hey All,
 
  How do people feel about the 2.9.4 code base? I've been using it for
  sometime, for my use cases it's be excellent. Do we feel we are ready to
  package this up and make it an official release? Or do we have some
tasks
  left to take care of?
 
  ~Prescott





[Lucene.Net] [jira] [Resolved] (LUCENENET-414) The definition of CharArraySet is dangerously confusing and leads to bugs when used.

2011-09-06 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy resolved LUCENENET-414.


Resolution: Fixed

Fixed.

DIGY

 The definition of CharArraySet is dangerously confusing and leads to bugs 
 when used.
 

 Key: LUCENENET-414
 URL: https://issues.apache.org/jira/browse/LUCENENET-414
 Project: Lucene.Net
  Issue Type: Bug
  Components: Lucene.Net Core
Affects Versions: Lucene.Net 2.9.2
 Environment: Irrelevant
Reporter: Vincent Van Den Berghe
Priority: Minor
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g


 Right now, CharArraySet derives from System.Collections.Hashtable, but 
 doesn't actually use this base type for storing elements.
 However, the StandardAnalyzer.STOP_WORDS_SET is exposed as a 
 System.Collections.Hashtable. The trivial code to build your own stopword set 
 using the StandardAnalyzer.STOP_WORDS_SET and adding your own set of 
 stopwords like this:
 CharArraySet myStopWords = new CharArraySet(StandardAnalyzer.STOP_WORDS_SET, 
 ignoreCase: false);
 foreach (string domainSpecificStopWord in DomainSpecificStopWords)
 stopWords.Add(domainSpecificStopWord);
 ... will fail because the CharArraySet accepts an ICollection, which will be 
 passed the Hashtable instance of STOP_WORDS_SET: the resulting myStopWords 
 will only contain the DomainSpecificStopWords, and not those from 
 STOP_WORDS_SET.
 One workaround would be to replace the first line with this:
 CharArraySet stopWords = new 
 CharArraySet(StandardAnalyzer.STOP_WORDS_SET.Count + 
 DomainSpecificStopWords.Length, ignoreCase: false);
 foreach (string domainSpecificStopWord in 
 (CharArraySet)StandardAnalyzer.STOP_WORDS_SET)
 stopWords.Add(domainSpecificStopWord);
 ... but this makes use of the implementation detail (the STOP_WORDS_SET is 
 really an UnmodifiableCharArraySet which is itself a CharArraySet). It works 
 because it forces the foreach() to use the correct 
 CharArraySet.GetEnumerator(), which is defined as a new method (this has a 
 bad code smell to it)
 At least 2 possibilities exist to solve this problem:
 - Make CharArraySet use the Hashtable instance and a custom comparator, 
 instead of its own implementation.
 - Make CharArraySet use HashSetchar[], defined in .NET 4.0.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




RE: [Lucene.Net] 2.9.4

2011-09-06 Thread Digy
+1 for an official release.
DIGY

-Original Message-
From: Prescott Nasser [mailto:geobmx...@hotmail.com] 
Sent: Monday, September 05, 2011 9:22 PM
To: lucene-net-...@lucene.apache.org; lucene-net-u...@lucene.apache.org
Subject: [Lucene.Net] 2.9.4


Hey All,

 

How do people feel about the 2.9.4 code base? I've been using it for
sometime, for my use cases it's be excellent. Do we feel we are ready to
package this up and make it an official release? Or do we have some tasks
left to take care of?

 

~Prescott =
-
Bu iletide virüs bulunamadı.
AVG tarafından kontrol edildi - www.avg.com
Sürüm: 2012.0.1796 / Virüs Veritabanı: 2082/4478 - Sürüm Tarihi: 05.09.2011



[Lucene.Net] [jira] [Updated] (LUCENENET-442) ParallelMultiSearcher threads don't handle all exceptions

2011-09-05 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy updated LUCENENET-442:
---

Attachment: LUCENENET-442.patch

Thanks Andy.
Nice catch.
I prepared a patch for 2.9.4g and will commit to 2.9.4g branch  trunk soon.

DIGY

 ParallelMultiSearcher threads don't handle all exceptions
 -

 Key: LUCENENET-442
 URL: https://issues.apache.org/jira/browse/LUCENENET-442
 Project: Lucene.Net
  Issue Type: Bug
  Components: Lucene.Net Core
Affects Versions: Lucene.Net 2.9.2
Reporter: Andy Twidle
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g

 Attachments: LUCENENET-442.patch


 The ParallelMultiSearcher doesn't allow non-IOException exceptions to be 
 managed by the calling application.  
 LUCENENET-388 worked around one specific example of this, but any genuine 
 Lucene exception (eg: BooleanQuery.TooManyClauses) will also fall foul of 
 this pattern.
 In our specific instance we could treat the symptoms and up the max clause 
 count, but I'm sure there will be more.  Could the System.IOException be 
 generalised to System.Exception?  Or would that be too much deviation from 
 the Java code base?
 --
 Example stack trace of an exception thrown by a Searcher executed:
 Framework Version: v4.0.30319
 Description: The process was terminated due to an unhandled exception.
 Exception Info: Lucene.Net.Search.BooleanQuery+TooManyClauses
 Stack:
at Lucene.Net.Search.BooleanQuery.Add(Lucene.Net.Search.BooleanClause)
at Lucene.Net.Search.BooleanQuery.Add(Lucene.Net.Search.Query, Occur)
at Lucene.Net.Search.PrefixQuery.Rewrite(Lucene.Net.Index.IndexReader)
at Lucene.Net.Search.BooleanQuery.Rewrite(Lucene.Net.Index.IndexReader)
at Lucene.Net.Search.IndexSearcher.Rewrite(Lucene.Net.Search.Query)
at Lucene.Net.Search.Query.Weight(Lucene.Net.Search.Searcher)
at Lucene.Net.Search.Searcher.CreateWeight(Lucene.Net.Search.Query)
at Lucene.Net.Search.Searcher.Search(Lucene.Net.Search.Query, 
 Lucene.Net.Search.Filter, Lucene.Net.Search.HitCollector)
at Lucene.Net.Search.Searcher.Search(Lucene.Net.Search.Query, 
 Lucene.Net.Search.HitCollector)
at Lucene.Net.Search.QueryWrapperFilter.Bits(Lucene.Net.Index.IndexReader)
at 
 Lucene.Net.Search.CachingWrapperFilter.Bits(Lucene.Net.Index.IndexReader)
at Lucene.Net.Search.IndexSearcher.Search(Lucene.Net.Search.Weight, 
 Lucene.Net.Search.Filter, Lucene.Net.Search.HitCollector)
at Lucene.Net.Search.IndexSearcher.Search(Lucene.Net.Search.Weight, 
 Lucene.Net.Search.Filter, Int32)
at Lucene.Net.Search.MultiSearcherThread.Run()
at 
 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, 
 System.Threading.ContextCallback, System.Object, Boolean)
at 
 System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, 
 System.Threading.ContextCallback, System.Object)
at System.Threading.ThreadHelper.ThreadStart()

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




RE: [Lucene.Net] 2.9.4

2011-09-05 Thread Digy
To avoid misunderstanding...

Community==all Lucene.Net users

DIGY

-Original Message-
From: Digy [mailto:digyd...@gmail.com] 
Sent: Monday, September 05, 2011 11:46 PM
To: 'lucene-net-dev@lucene.apache.org'
Subject: RE: [Lucene.Net] 2.9.4

Not bad idea, but I would prefer community's feedback instead of testing
against all projects using Lucene.Net
DIGY

-Original Message-
From: Matt Warren [mailto:mattd...@gmail.com] 
Sent: Monday, September 05, 2011 11:09 PM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] 2.9.4

If you want to test it against a large project you could take a look at how
RavenDB uses it?

At the moment it's using 2.9.2 (
https://github.com/ayende/ravendb/tree/master/SharedLibs/Sources/Lucene2.9.2
)
but if you were to recompile it against 2.9.4 and check that all it's
unit-tests still run that would give you quite a large test case.

On 5 September 2011 19:22, Prescott Nasser geobmx...@hotmail.com wrote:


 Hey All,

 How do people feel about the 2.9.4 code base? I've been using it for
 sometime, for my use cases it's be excellent. Do we feel we are ready to
 package this up and make it an official release? Or do we have some tasks
 left to take care of?

 ~Prescott

-
Bu iletide virüs bulunamadı.
AVG tarafından kontrol edildi - www.avg.com
Sürüm: 2012.0.1796 / Virüs Veritabanı: 2082/4478 - Sürüm Tarihi: 05.09.2011



RE: [Lucene.Net] 2.9.4

2011-09-05 Thread Digy
Not bad idea, but I would prefer community's feedback instead of testing
against all projects using Lucene.Net
DIGY

-Original Message-
From: Matt Warren [mailto:mattd...@gmail.com] 
Sent: Monday, September 05, 2011 11:09 PM
To: lucene-net-...@lucene.apache.org
Subject: Re: [Lucene.Net] 2.9.4

If you want to test it against a large project you could take a look at how
RavenDB uses it?

At the moment it's using 2.9.2 (
https://github.com/ayende/ravendb/tree/master/SharedLibs/Sources/Lucene2.9.2
)
but if you were to recompile it against 2.9.4 and check that all it's
unit-tests still run that would give you quite a large test case.

On 5 September 2011 19:22, Prescott Nasser geobmx...@hotmail.com wrote:


 Hey All,

 How do people feel about the 2.9.4 code base? I've been using it for
 sometime, for my use cases it's be excellent. Do we feel we are ready to
 package this up and make it an official release? Or do we have some tasks
 left to take care of?

 ~Prescott

-
Bu iletide virüs bulunamadı.
AVG tarafından kontrol edildi - www.avg.com
Sürüm: 2012.0.1796 / Virüs Veritabanı: 2082/4478 - Sürüm Tarihi: 05.09.2011



[Lucene.Net] [jira] [Commented] (LUCENENET-358) CloseableThreadLocal memory leak in LocalDataStoreSlot (with workaround)

2011-08-28 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13092541#comment-13092541
 ] 

Digy commented on LUCENENET-358:


New CloseableThreadLocal implementation and its test case committed to trunk.

DIGY

 CloseableThreadLocal memory leak  in LocalDataStoreSlot (with workaround)
 -

 Key: LUCENENET-358
 URL: https://issues.apache.org/jira/browse/LUCENENET-358
 Project: Lucene.Net
  Issue Type: Bug
Affects Versions: Lucene.Net 2.9.2, Lucene.Net 2.9.4, Lucene.Net 2.9.4g
 Environment: Microsoft WIndows Server 2008 Enterprise x64. SP2.
 .NET Framework 4.0
Reporter: Rezgar Cadro
Assignee: Digy
Priority: Critical
  Labels: memory, CloseableThreadLocal, LocalDataStoreSlot, leak
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g

 Attachments: CloseableThreadLocal MemoryLeak.patch, 
 CloseableThreadLocal.diff, CloseableThreadLocal.diff, 
 CloseableThreadLocal.patch, TestMemLeakage.zip


 Recently we have been suffering from a severe memory leak when executing 
 intense open/close operations on IndexSearcher and IndexModifier. 
 Memory profiling showed that memory is being held by LocalDataStore[] objects.
 After some digging, the root of the problem has been found in 
 CloseableThreadLocal class:
 private System.LocalDataStoreSlot t = 
 System.Threading.Thread.AllocateDataSlot();
 What we see is that every instantiated object of CloseableThreadLocal causes 
 new data slot allocation performed for every thread. 
 Thread.AllocateDataSlot() does not simply allocate a new slot, replacing an 
 old one, but enlarging an existing buffer in-thread, appending data to the 
 end of internal LocalDataStore[] collection, which  causes a severe memory 
 leak .
 As long as t variable is instantiated on every object creation, and (in 
 current class implementation) every object is used by a single thread, 
 replacing private System.LocalDataStoreSlot t = 
 System.Threading.Thread.AllocateDataSlot(); with simple private object 
 dataSlot; and removing hardRefs Dictionary solves the problem and prevents 
 memory leak. 
 We have tried to implement the expected behavior by using [ThreadStatic] 
 attribute instead of LocalDataStoreSlot, but the attempt failed because of 
 unexpected exceptions being thrown.
 Patch can be found at Lucene.Net repository under 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Updated] (LUCENENET-358) CloseableThreadLocal memory leak in LocalDataStoreSlot (with workaround)

2011-08-27 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy updated LUCENENET-358:
---

Attachment: TestMemLeakage.zip

TestMemLeakage.zip shows that bug.

 CloseableThreadLocal memory leak  in LocalDataStoreSlot (with workaround)
 -

 Key: LUCENENET-358
 URL: https://issues.apache.org/jira/browse/LUCENENET-358
 Project: Lucene.Net
  Issue Type: Bug
 Environment: Microsoft WIndows Server 2008 Enterprise x64. SP2.
 .NET Framework 4.0
Reporter: Rezgar Cadro
Assignee: Digy
Priority: Critical
  Labels: memory, CloseableThreadLocal, LocalDataStoreSlot, leak
 Attachments: CloseableThreadLocal MemoryLeak.patch, 
 CloseableThreadLocal.diff, CloseableThreadLocal.diff, 
 CloseableThreadLocal.patch, TestMemLeakage.zip


 Recently we have been suffering from a severe memory leak when executing 
 intense open/close operations on IndexSearcher and IndexModifier. 
 Memory profiling showed that memory is being held by LocalDataStore[] objects.
 After some digging, the root of the problem has been found in 
 CloseableThreadLocal class:
 private System.LocalDataStoreSlot t = 
 System.Threading.Thread.AllocateDataSlot();
 What we see is that every instantiated object of CloseableThreadLocal causes 
 new data slot allocation performed for every thread. 
 Thread.AllocateDataSlot() does not simply allocate a new slot, replacing an 
 old one, but enlarging an existing buffer in-thread, appending data to the 
 end of internal LocalDataStore[] collection, which  causes a severe memory 
 leak .
 As long as t variable is instantiated on every object creation, and (in 
 current class implementation) every object is used by a single thread, 
 replacing private System.LocalDataStoreSlot t = 
 System.Threading.Thread.AllocateDataSlot(); with simple private object 
 dataSlot; and removing hardRefs Dictionary solves the problem and prevents 
 memory leak. 
 We have tried to implement the expected behavior by using [ThreadStatic] 
 attribute instead of LocalDataStoreSlot, but the attempt failed because of 
 unexpected exceptions being thrown.
 Patch can be found at Lucene.Net repository under 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-441) Encountered: EOF after : \\\\ during parsing a query

2011-08-25 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13091175#comment-13091175
 ] 

Digy commented on LUCENENET-441:


What does your query look like? 
What is your question?

 Encountered: EOF after :  during parsing a query
 --

 Key: LUCENENET-441
 URL: https://issues.apache.org/jira/browse/LUCENENET-441
 Project: Lucene.Net
  Issue Type: Bug
  Components: Lucene.Net Core
Affects Versions: Lucene.Net 2.9.2
 Environment: .Net Framework 4.0
Reporter: Maverick904

 Cannot parse '\': Lexical error at line 1, column 4.  Encountered: EOF 
 after :  |at Lucene.Net.QueryParsers.QueryParser.Parse(String 
 query)
at Lucene.Net.QueryParsers.MultiFieldQueryParser.Parse(Version 
 matchVersion, String query, String[] fields, Occur[] flags, Analyzer analyzer)
at Lucene.Net.QueryParsers.MultiFieldQueryParser.Parse(String query, 
 String[] fields, Occur[] flags, Analyzer analyzer)

  
 Lexical error at line 1, column 4.  Encountered: EOF after :  |
 at Lucene.Net.QueryParsers.QueryParserTokenManager.GetNextToken()
at Lucene.Net.QueryParsers.QueryParser.Jj_ntk()
at Lucene.Net.QueryParsers.QueryParser.Modifiers()
at Lucene.Net.QueryParsers.QueryParser.Query(String field)
at Lucene.Net.QueryParsers.QueryParser.Parse(String query) | 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Issue Comment Edited] (LUCENENET-437) Port Contrib.Shingle from Java

2011-07-18 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13067389#comment-13067389
 ] 

Digy edited comment on LUCENENET-437 at 7/18/11 11:28 PM:
--

bq. It ensures equality, but does not ensure inequality.
Sorry but I must object again.
It ensures inequality, but doesn't ensure equality.(if hashcodes are not equal 
objects are not also, but having the same hashcode doesn't say anything about 
equality)



  was (Author: digydigy):
bq. It ensures equality, but does not ensure inequality.
Sorry but I must object again.
It ensures inequality, but doesn't ensure equality.


  
 Port Contrib.Shingle from Java
 --

 Key: LUCENENET-437
 URL: https://issues.apache.org/jira/browse/LUCENENET-437
 Project: Lucene.Net
  Issue Type: Task
  Components: Lucene.Net Contrib, Lucene.Net Test
Affects Versions: Lucene.Net 2.9.4, Lucene.Net 2.9.4g
Reporter: Troy Howard
Assignee: Troy Howard
Priority: Minor
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g


 Port Contrib.Shingle from Java

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-437) Port Contrib.Shingle from Java

2011-07-18 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13067409#comment-13067409
 ] 

Digy commented on LUCENENET-437:


Already fixed in 2.9.4g

 Port Contrib.Shingle from Java
 --

 Key: LUCENENET-437
 URL: https://issues.apache.org/jira/browse/LUCENENET-437
 Project: Lucene.Net
  Issue Type: Task
  Components: Lucene.Net Contrib, Lucene.Net Test
Affects Versions: Lucene.Net 2.9.4, Lucene.Net 2.9.4g
Reporter: Troy Howard
Assignee: Troy Howard
Priority: Minor
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g


 Port Contrib.Shingle from Java

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Updated] (LUCENENET-437) Port Contrib.Shingle from Java

2011-07-18 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy updated LUCENENET-437:
---

Fix Version/s: Lucene.Net 2.9.4g

 Port Contrib.Shingle from Java
 --

 Key: LUCENENET-437
 URL: https://issues.apache.org/jira/browse/LUCENENET-437
 Project: Lucene.Net
  Issue Type: Task
  Components: Lucene.Net Contrib, Lucene.Net Test
Affects Versions: Lucene.Net 2.9.4, Lucene.Net 2.9.4g
Reporter: Troy Howard
Assignee: Troy Howard
Priority: Minor
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g


 Port Contrib.Shingle from Java

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-437) Port Contrib.Shingle from Java

2011-07-18 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13067361#comment-13067361
 ] 

Digy commented on LUCENENET-437:



Java Docs says:
public boolean equals(Object o)
Compares the specified object with this list for equality. Returns true if 
and only if the specified object is also a list, both lists have the same size, 
and *all corresponding pairs of elements in the two lists are equal* [No 
reference for Hashcode - DIGY]. (Two elements e1 and e2 are equal if (e1==null 
? e2==null : e1.equals(e2)).) In other words, two lists are defined to be equal 
if they contain the same elements in the same order. This definition ensures 
that the equals method works properly across different implementations of the 
List interface. 


Yes, the sample was from Eric Lippert's blog to show why GetHashCode should not 
be used for comparing objects.
bq. The issue you're describing is more of a problem with the .NET 
implementation of GetHashcode() rather than the correctness of using hashcode 
for comparison. 
No, the problem is not in the implementation of GetHashCode. In any 
implementation, you may have some unexpected collisions(since it is a 4-byte 
number). GetHashCode isn't meant for uniqueness or object identification. It's  
meant to provide a random distribution.
Therefore the problem really lies in using it for equality comparison.

DIGY




 Port Contrib.Shingle from Java
 --

 Key: LUCENENET-437
 URL: https://issues.apache.org/jira/browse/LUCENENET-437
 Project: Lucene.Net
  Issue Type: Task
  Components: Lucene.Net Contrib, Lucene.Net Test
Affects Versions: Lucene.Net 2.9.4, Lucene.Net 2.9.4g
Reporter: Troy Howard
Assignee: Troy Howard
Priority: Minor
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g


 Port Contrib.Shingle from Java

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-437) Port Contrib.Shingle from Java

2011-07-18 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13067366#comment-13067366
 ] 

Digy commented on LUCENENET-437:


See even with worst implementation, Equals method should work.

{code}
///
private void Form1_Load(object sender, EventArgs e)
{
Hashtable h = new Hashtable();

MyClass m1 = new MyClass() { I = 1 };
MyClass m2 = new MyClass() { I = 2 };

h.Add(m1,1);
h.Add(m2,2);

System.Diagnostics.Debug.Assert(h[m2].Equals(2));
}

public class MyClass
{
public int I;
public override int GetHashCode()
{
return 1;
}
}
{code}

 Port Contrib.Shingle from Java
 --

 Key: LUCENENET-437
 URL: https://issues.apache.org/jira/browse/LUCENENET-437
 Project: Lucene.Net
  Issue Type: Task
  Components: Lucene.Net Contrib, Lucene.Net Test
Affects Versions: Lucene.Net 2.9.4, Lucene.Net 2.9.4g
Reporter: Troy Howard
Assignee: Troy Howard
Priority: Minor
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g


 Port Contrib.Shingle from Java

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-437) Port Contrib.Shingle from Java

2011-07-18 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13067375#comment-13067375
 ] 

Digy commented on LUCENENET-437:


bq. This ensures that list1.equals(list2) implies that 
list1.hashCode()==list2.hashCode() for any two lists, list1 and list2, as 
required by the general contract of Object.hashCode.
but it doesn't ensure that 
if list1.hashCode()==list2.hashCode() then list1.equals(list2) should be true, 
as I showed using Eric Lippert's sample.




 Port Contrib.Shingle from Java
 --

 Key: LUCENENET-437
 URL: https://issues.apache.org/jira/browse/LUCENENET-437
 Project: Lucene.Net
  Issue Type: Task
  Components: Lucene.Net Contrib, Lucene.Net Test
Affects Versions: Lucene.Net 2.9.4, Lucene.Net 2.9.4g
Reporter: Troy Howard
Assignee: Troy Howard
Priority: Minor
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g


 Port Contrib.Shingle from Java

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-437) Port Contrib.Shingle from Java

2011-07-18 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13067389#comment-13067389
 ] 

Digy commented on LUCENENET-437:


bq. It ensures equality, but does not ensure inequality.
Sorry but I must object again.
It ensures inequality, but doesn't ensure equality.



 Port Contrib.Shingle from Java
 --

 Key: LUCENENET-437
 URL: https://issues.apache.org/jira/browse/LUCENENET-437
 Project: Lucene.Net
  Issue Type: Task
  Components: Lucene.Net Contrib, Lucene.Net Test
Affects Versions: Lucene.Net 2.9.4, Lucene.Net 2.9.4g
Reporter: Troy Howard
Assignee: Troy Howard
Priority: Minor
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g


 Port Contrib.Shingle from Java

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Updated] (LUCENENET-437) Port Contrib.Shingle from Java

2011-07-17 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy updated LUCENENET-437:
---

Affects Version/s: Lucene.Net 2.9.4g
Fix Version/s: Lucene.Net 2.9.4g

 Port Contrib.Shingle from Java
 --

 Key: LUCENENET-437
 URL: https://issues.apache.org/jira/browse/LUCENENET-437
 Project: Lucene.Net
  Issue Type: Task
  Components: Lucene.Net Contrib, Lucene.Net Test
Affects Versions: Lucene.Net 2.9.4, Lucene.Net 2.9.4g
Reporter: Troy Howard
Assignee: Troy Howard
Priority: Minor
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g


 Port Contrib.Shingle from Java

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-437) Port Contrib.Shingle from Java

2011-07-17 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13066717#comment-13066717
 ] 

Digy commented on LUCENENET-437:


Using HashCode for equality comparison is not a good idea.

{code}
///
ListComparerstring comp = new ListComparerstring();
b = comp.Equals(new Liststring() { \uA0A2\uA0A2 }, new 
Liststring() {  });
System.Diagnostics.Debug.Assert(b == false);
b = comp.Equals(new Liststring() { \uA0A2\uA0A2 }, new 
Liststring() { \uA0A2\uA0A2\uA0A2\uA0A2 });
System.Diagnostics.Debug.Assert(b == false);

b = new Lucene.Net.Support.EquatableListstring(){\uA0A2\uA0A2 
}.Equals(new Lucene.Net.Support.EquatableListstring() {});
System.Diagnostics.Debug.Assert(b == false);
new Lucene.Net.Support.EquatableListstring() { \uA0A2\uA0A2 
}.Equals(new Lucene.Net.Support.EquatableListstring() { 
\uA0A2\uA0A2\uA0A2\uA0A2});
System.Diagnostics.Debug.Assert(b == false);

///

{code}


DIGY

 Port Contrib.Shingle from Java
 --

 Key: LUCENENET-437
 URL: https://issues.apache.org/jira/browse/LUCENENET-437
 Project: Lucene.Net
  Issue Type: Task
  Components: Lucene.Net Contrib, Lucene.Net Test
Affects Versions: Lucene.Net 2.9.4, Lucene.Net 2.9.4g
Reporter: Troy Howard
Assignee: Troy Howard
Priority: Minor
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g


 Port Contrib.Shingle from Java

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Reopened] (LUCENENET-437) Port Contrib.Shingle from Java

2011-07-17 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy reopened LUCENENET-437:



 Port Contrib.Shingle from Java
 --

 Key: LUCENENET-437
 URL: https://issues.apache.org/jira/browse/LUCENENET-437
 Project: Lucene.Net
  Issue Type: Task
  Components: Lucene.Net Contrib, Lucene.Net Test
Affects Versions: Lucene.Net 2.9.4, Lucene.Net 2.9.4g
Reporter: Troy Howard
Assignee: Troy Howard
Priority: Minor
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g


 Port Contrib.Shingle from Java

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




Re: [Lucene.Net] Incubator Status Page

2011-07-11 Thread digy digy
On Sun, Jul 10, 2011 at 6:24 PM, Stefan Bodewig bode...@apache.org wrote:

 Hi all,

 http://incubator.apache.org/projects/lucene.net.html contains quite a
 few blanks that I think we could easily fill.  I intend to either add
 some N/A or real dates where I can during the coming week.

 On the IP issues part (copyright and distribution rights) I trust the
 Lucene PMC has been taking care of this before Lucene.NET headed back to
 the Incubator and after that all contributions have come either directly
 by people with a CLA on file or as patches via JIRA where the ASF may
 use this checkbox has been checked - is this correct?


absolutely.



 For the project specific tasks I'd ask all of you to fill in whatever
 you feel like adding.  All Lucene.NET committers should be able to
 modify the status page.

 Stefan


DIGY


[Lucene.Net] [jira] [Commented] (LUCENENET-434) Remove AnonymousXXXX classes to increase readablity

2011-07-09 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13062351#comment-13062351
 ] 

Digy commented on LUCENENET-434:


very nice.

 Remove Anonymous classes to increase readablity
 ---

 Key: LUCENENET-434
 URL: https://issues.apache.org/jira/browse/LUCENENET-434
 Project: Lucene.Net
  Issue Type: Improvement
  Components: Lucene.Net Core
Affects Versions: Lucene.Net 2.9.4g
Reporter: Scott Lombard
Assignee: Scott Lombard
 Fix For: Lucene.Net 2.9.4g

 Attachments: TeeSinkTokenFilter.patch

   Original Estimate: 168h
  Time Spent: 5h
  Remaining Estimate: 163h

 Replace Anonymous classes inhereted from JLCA which make the code 
 impossible to read.
 Follow Digy's template to replace the single abstract method with Func or 
 Action
  
 like in FilterCacheT from:
 protected abstract object MergeDeletes(IndexReader reader, object value);
 to:
 FuncIndexReader, object, object MergeDeletes;
  
 Determine a solution to the classes with more than 1 abstract method without 
 diverging much from Java.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Resolved] (LUCENENET-432) Concurrency issues in SegmentInfo.Files() (LUCENE-2584)

2011-07-07 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy resolved LUCENENET-432.


   Resolution: Fixed
Fix Version/s: Lucene.Net 2.9.4
   Lucene.Net 2.9.2
 Assignee: Digy

Patch committed to trunk  2.9.4g branch

 Concurrency issues in SegmentInfo.Files() (LUCENE-2584)
 ---

 Key: LUCENENET-432
 URL: https://issues.apache.org/jira/browse/LUCENENET-432
 Project: Lucene.Net
  Issue Type: Bug
Affects Versions: Lucene.Net 2.9.4, Lucene.Net 2.9.4g
Reporter: Digy
Assignee: Digy
 Fix For: Lucene.Net 2.9.2, Lucene.Net 2.9.4

 Attachments: SegmentInfo.patch


 The multi-threaded call of the files() in SegmentInfo could lead to the 
 ConcurrentModificationException if one thread is not finished additions to 
 the ArrayList (files) yet while the other thread already obtained it as 
 cached.
 https://issues.apache.org/jira/browse/LUCENE-2584

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Resolved] (LUCENENET-430) Contrib.ChainedFilter

2011-07-07 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy resolved LUCENENET-430.


Resolution: Fixed

Instead of creating a small project, I put it into Contrib.Analyzers.

 Contrib.ChainedFilter
 -

 Key: LUCENENET-430
 URL: https://issues.apache.org/jira/browse/LUCENENET-430
 Project: Lucene.Net
  Issue Type: New Feature
Affects Versions: Lucene.Net 2.9.4g
Reporter: Digy
Priority: Minor
 Fix For: Lucene.Net 2.9.4g

 Attachments: ChainedFilter.cs, ChainedFilterTest.cs


 Port of lucene.Java 3.0.3's ChainedFilter  its test cases.
 See the StackOverflow question: How to combine multiple filters within one 
 search?
 http://stackoverflow.com/questions/6570477/multiple-filters-in-lucene-net

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-418) LuceneTestCase should not have a static method could throw exceptions.

2011-07-07 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13061125#comment-13061125
 ] 

Digy commented on LUCENENET-418:


It works! Thanks.
DIGY

 LuceneTestCase should not have a static method could throw exceptions.  
 

 Key: LUCENENET-418
 URL: https://issues.apache.org/jira/browse/LUCENENET-418
 Project: Lucene.Net
  Issue Type: Bug
  Components: Lucene.Net Test
Affects Versions: Lucene.Net 3.x
 Environment: Linux, OSX, etc 
Reporter: michael herndon
Assignee: michael herndon
  Labels: test
 Fix For: Lucene.Net 2.9.4g

   Original Estimate: 2m
  Remaining Estimate: 2m

 Throwing an exception in a base classes for 90% tests in a static method 
 makes it hard to debug the issue in nunit.
 The test results came back saying that TestFixtureSetup was causing an issue 
 even though it was the Static Constructor causing problems and this then 
 propagates to all the tests that stem from LuceneTestCase. 
 The TEMP_DIR needs to be moved to a static util class as a property or even a 
 mixin method.  This caused me hours to debug and figure out the real issue as 
 the underlying exception method never bubbled up.  

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Created] (LUCENENET-433) AttributeSource can have an invalid computed state (LUCENE-3042)

2011-07-07 Thread Digy (JIRA)
AttributeSource can have an invalid computed state (LUCENE-3042)


 Key: LUCENENET-433
 URL: https://issues.apache.org/jira/browse/LUCENENET-433
 Project: Lucene.Net
  Issue Type: Bug
Reporter: Digy
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g


If you work a tokenstream, consume it, then reuse it and add an attribute to 
it, the computed state is wrong.
thus for example, clearAttributes() will not actually clear the attribute added.
So in some situations, addAttribute is not actually clearing the computed state 
when it should.

https://issues.apache.org/jira/browse/LUCENE-3042


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-433) AttributeSource can have an invalid computed state (LUCENE-3042)

2011-07-07 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13061214#comment-13061214
 ] 

Digy commented on LUCENENET-433:


Here is the test case
{code}
[Test]
public void Test_LUCENE_3042_LUCENENET_433()
{
String testString = t;

Analyzer analyzer = new 
Lucene.Net.Analysis.Standard.StandardAnalyzer();
TokenStream stream = analyzer.ReusableTokenStream(dummy, new 
System.IO.StringReader(testString));
stream.Reset();
while (stream.IncrementToken())
{
// consume
}
stream.End();
stream.Close();

AssertAnalyzesToReuse(analyzer, testString, new String[] { t });
}
{code}

 AttributeSource can have an invalid computed state (LUCENE-3042)
 

 Key: LUCENENET-433
 URL: https://issues.apache.org/jira/browse/LUCENENET-433
 Project: Lucene.Net
  Issue Type: Bug
Reporter: Digy
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g


 If you work a tokenstream, consume it, then reuse it and add an attribute to 
 it, the computed state is wrong.
 thus for example, clearAttributes() will not actually clear the attribute 
 added.
 So in some situations, addAttribute is not actually clearing the computed 
 state when it should.
 https://issues.apache.org/jira/browse/LUCENE-3042

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Resolved] (LUCENENET-172) This patch fixes the unexceptional exceptions ecountered in FastCharStream and SupportClass

2011-07-07 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy resolved LUCENENET-172.


Resolution: Fixed
  Assignee: Digy  (was: Scott Lombard)

Fixed in 2.9.4g. No fix for 2.9.4


 This patch fixes the unexceptional exceptions ecountered in FastCharStream 
 and SupportClass
 ---

 Key: LUCENENET-172
 URL: https://issues.apache.org/jira/browse/LUCENENET-172
 Project: Lucene.Net
  Issue Type: Improvement
  Components: Lucene.Net Core
Affects Versions: Lucene.Net 2.3.1, Lucene.Net 2.3.2
Reporter: Ben Martz
Assignee: Digy
Priority: Minor
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g

 Attachments: lucene_2.3.1_exceptions_fix.patch, 
 lucene_2.9.4g_exceptions_fix


 The java version of Lucene handles end-of-file in FastCharStream by throwing 
 an exception. This behavior has been ported to .NET but the behavior carries 
 an unacceptable cost in the .NET environment. This patch is based on the 
 prior work in LUCENENET-8 and LUCENENET-11, which I gratefully acknowledge 
 for the solution. While I understand that this patch is outside of the 
 current project specification in that it deviates from the pure nature of 
 the port, I believe that it is very important to make the patch available to 
 any developer looking to leverage Lucene.Net in their project. Thanks for 
 your consideration.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-433) AttributeSource can have an invalid computed state (LUCENE-3042)

2011-07-07 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13061304#comment-13061304
 ] 

Digy commented on LUCENENET-433:


Committed to 2.9.4g branch

 AttributeSource can have an invalid computed state (LUCENE-3042)
 

 Key: LUCENENET-433
 URL: https://issues.apache.org/jira/browse/LUCENENET-433
 Project: Lucene.Net
  Issue Type: Bug
Reporter: Digy
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g

 Attachments: LUCENENET-433.patch


 If you work a tokenstream, consume it, then reuse it and add an attribute to 
 it, the computed state is wrong.
 thus for example, clearAttributes() will not actually clear the attribute 
 added.
 So in some situations, addAttribute is not actually clearing the computed 
 state when it should.
 https://issues.apache.org/jira/browse/LUCENE-3042

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-172) This patch fixes the unexceptional exceptions ecountered in FastCharStream and SupportClass

2011-07-07 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13061595#comment-13061595
 ] 

Digy commented on LUCENENET-172:


Already fixed for 2.9.4g

 This patch fixes the unexceptional exceptions ecountered in FastCharStream 
 and SupportClass
 ---

 Key: LUCENENET-172
 URL: https://issues.apache.org/jira/browse/LUCENENET-172
 Project: Lucene.Net
  Issue Type: Improvement
  Components: Lucene.Net Core
Affects Versions: Lucene.Net 2.3.1, Lucene.Net 2.3.2
Reporter: Ben Martz
Assignee: Scott Lombard
Priority: Minor
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g

 Attachments: lucene_2.3.1_exceptions_fix.patch, 
 lucene_2.9.4g_exceptions_fix


 The java version of Lucene handles end-of-file in FastCharStream by throwing 
 an exception. This behavior has been ported to .NET but the behavior carries 
 an unacceptable cost in the .NET environment. This patch is based on the 
 prior work in LUCENENET-8 and LUCENENET-11, which I gratefully acknowledge 
 for the solution. While I understand that this patch is outside of the 
 current project specification in that it deviates from the pure nature of 
 the port, I believe that it is very important to make the patch available to 
 any developer looking to leverage Lucene.Net in their project. Thanks for 
 your consideration.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Updated] (LUCENENET-172) This patch fixes the unexceptional exceptions ecountered in FastCharStream and SupportClass

2011-07-07 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy updated LUCENENET-172:
---

Fix Version/s: Lucene.Net 2.9.4g

 This patch fixes the unexceptional exceptions ecountered in FastCharStream 
 and SupportClass
 ---

 Key: LUCENENET-172
 URL: https://issues.apache.org/jira/browse/LUCENENET-172
 Project: Lucene.Net
  Issue Type: Improvement
  Components: Lucene.Net Core
Affects Versions: Lucene.Net 2.3.1, Lucene.Net 2.3.2
Reporter: Ben Martz
Assignee: Scott Lombard
Priority: Minor
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g

 Attachments: lucene_2.3.1_exceptions_fix.patch, 
 lucene_2.9.4g_exceptions_fix


 The java version of Lucene handles end-of-file in FastCharStream by throwing 
 an exception. This behavior has been ported to .NET but the behavior carries 
 an unacceptable cost in the .NET environment. This patch is based on the 
 prior work in LUCENENET-8 and LUCENENET-11, which I gratefully acknowledge 
 for the solution. While I understand that this patch is outside of the 
 current project specification in that it deviates from the pure nature of 
 the port, I believe that it is very important to make the patch available to 
 any developer looking to leverage Lucene.Net in their project. Thanks for 
 your consideration.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-431) Spatial.Net Cartesian won't find docs in radius in certain cases

2011-07-06 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13060709#comment-13060709
 ] 

Digy commented on LUCENENET-431:


Thanks Olle and Matt,

I committed the LUCENE-1930 patch to the 2.9.4g branch (+ added Olle's test 
case).

(Another divergence from lucene.java; since this patch is still waiting to be 
applied).

DIGY

 Spatial.Net Cartesian won't find docs in radius in certain cases
 

 Key: LUCENENET-431
 URL: https://issues.apache.org/jira/browse/LUCENENET-431
 Project: Lucene.Net
  Issue Type: Bug
  Components: Lucene.Net Contrib
Affects Versions: Lucene.Net 2.9.4
 Environment: Windows 7 x64
Reporter: Olle Jacobsen
  Labels: spatialsearch

 To replicate change Lucene.Net.Contrib.Spatial.Test.TestCartesian to the 
 following witch should return 3 results.
 Line
 42: private double _lat = 55.6880508001;
 43: private double _lng = 13.5871808352; // This passes: 13.6271808352
 73: AddPoint(writer, Within radius, 55.6880508001, 13.5717346673);
 74: AddPoint(writer, Within radius, 55.6821978456, 13.6076183965);
 75: AddPoint(writer, Within radius, 55.673251569, 13.5946697607);
 76: AddPoint(writer, Close but not in radius, 55.8634157297, 13.5497731987);
 77: AddPoint(writer, Faar away, 40.7137578228, -74.0126901936);
 130: const double miles = 5.0;
 156: Console.WriteLine(Distances should be 3  + distances.Count);
 157: Console.WriteLine(Results should be 3  + results);
 159: Assert.AreEqual(3, distances.Count); // fixed a store of only needed 
 distances
 160: Assert.AreEqual(3, results);

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Resolved] (LUCENENET-431) Spatial.Net Cartesian won't find docs in radius in certain cases

2011-07-06 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy resolved LUCENENET-431.


   Resolution: Fixed
Fix Version/s: Lucene.Net 2.9.4g
 Assignee: Digy

 Spatial.Net Cartesian won't find docs in radius in certain cases
 

 Key: LUCENENET-431
 URL: https://issues.apache.org/jira/browse/LUCENENET-431
 Project: Lucene.Net
  Issue Type: Bug
  Components: Lucene.Net Contrib
Affects Versions: Lucene.Net 2.9.4
 Environment: Windows 7 x64
Reporter: Olle Jacobsen
Assignee: Digy
  Labels: spatialsearch
 Fix For: Lucene.Net 2.9.4g


 To replicate change Lucene.Net.Contrib.Spatial.Test.TestCartesian to the 
 following witch should return 3 results.
 Line
 42: private double _lat = 55.6880508001;
 43: private double _lng = 13.5871808352; // This passes: 13.6271808352
 73: AddPoint(writer, Within radius, 55.6880508001, 13.5717346673);
 74: AddPoint(writer, Within radius, 55.6821978456, 13.6076183965);
 75: AddPoint(writer, Within radius, 55.673251569, 13.5946697607);
 76: AddPoint(writer, Close but not in radius, 55.8634157297, 13.5497731987);
 77: AddPoint(writer, Faar away, 40.7137578228, -74.0126901936);
 130: const double miles = 5.0;
 156: Console.WriteLine(Distances should be 3  + distances.Count);
 157: Console.WriteLine(Results should be 3  + results);
 159: Assert.AreEqual(3, distances.Count); // fixed a store of only needed 
 distances
 160: Assert.AreEqual(3, results);

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Commented] (LUCENENET-418) LuceneTestCase should not have a static method could throw exceptions.

2011-07-05 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13059955#comment-13059955
 ] 

Digy commented on LUCENENET-418:


It fails in both builds.

 LuceneTestCase should not have a static method could throw exceptions.  
 

 Key: LUCENENET-418
 URL: https://issues.apache.org/jira/browse/LUCENENET-418
 Project: Lucene.Net
  Issue Type: Bug
  Components: Lucene.Net Test
Affects Versions: Lucene.Net 3.x
 Environment: Linux, OSX, etc 
Reporter: michael herndon
Assignee: michael herndon
  Labels: test
 Fix For: Lucene.Net 2.9.4g

   Original Estimate: 2m
  Remaining Estimate: 2m

 Throwing an exception in a base classes for 90% tests in a static method 
 makes it hard to debug the issue in nunit.
 The test results came back saying that TestFixtureSetup was causing an issue 
 even though it was the Static Constructor causing problems and this then 
 propagates to all the tests that stem from LuceneTestCase. 
 The TEMP_DIR needs to be moved to a static util class as a property or even a 
 mixin method.  This caused me hours to debug and figure out the real issue as 
 the underlying exception method never bubbled up.  

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Updated] (LUCENENET-430) Contrib.ChainedFilter

2011-07-04 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy updated LUCENENET-430:
---

Attachment: ChainedFilterTest.cs
ChainedFilter.cs

 Contrib.ChainedFilter
 -

 Key: LUCENENET-430
 URL: https://issues.apache.org/jira/browse/LUCENENET-430
 Project: Lucene.Net
  Issue Type: New Feature
Affects Versions: Lucene.Net 2.9.4g
Reporter: Digy
Priority: Minor
 Fix For: Lucene.Net 2.9.4g

 Attachments: ChainedFilter.cs, ChainedFilterTest.cs


 Port of lucene.Java 3.0.3's ChainedFilter  its test cases.
 See the StackOverflow question: How to combine multiple filters within one 
 search?
 http://stackoverflow.com/questions/6570477/multiple-filters-in-lucene-net

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Created] (LUCENENET-430) Contrib.ChainedFilter

2011-07-04 Thread Digy (JIRA)
Contrib.ChainedFilter
-

 Key: LUCENENET-430
 URL: https://issues.apache.org/jira/browse/LUCENENET-430
 Project: Lucene.Net
  Issue Type: New Feature
Affects Versions: Lucene.Net 2.9.4g
Reporter: Digy
Priority: Minor
 Fix For: Lucene.Net 2.9.4g
 Attachments: ChainedFilter.cs, ChainedFilterTest.cs

Port of lucene.Java 3.0.3's ChainedFilter  its test cases.

See the StackOverflow question: How to combine multiple filters within one 
search?
http://stackoverflow.com/questions/6570477/multiple-filters-in-lucene-net

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

2011-07-02 Thread Digy
 I've add a Paths Class under the Lucene.Net.Tests Util folder
Since it is a Lucene.Net specific code, may Support be a better place?
DIGY

-Original Message-
From: Michael Herndon [mailto:mhern...@wickedsoftware.net] 
Sent: Friday, July 01, 2011 11:53 PM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

@Rory,

Jira in the past had the ability to create sub tasks or convert a current
task to a sub task. I'm guessing that apache's jira should be able to do
that.

@All,

I've add a Paths Class under the Lucene.Net.Tests Util folder (feel free to
rename, refactor as long as the tests still work) to help with working with
paths in general.  This should help with forming paths relative to the temp
directory or the project root.

NUnit's Shadow Copy tends to throw people off when trying to get the path of
the current assembly being tested to build up a relative path, the Path
class should help with that.

- Michael

On Fri, Jul 1, 2011 at 4:09 PM, Rory Plaire codekai...@gmail.com wrote:

 My thinking is just a separate ticket for each one. This makes the work
 easier to manage and gives a better sense about how much work is left as
 well as makes it easier to prioritize independent issues. We could link
all
 the sub-issues to a single task / feature / whatever (that is, if JIRA has
 that capability).

 -r
 On Fri, Jul 1, 2011 at 12:48 PM, Michael Herndon 
 mhern...@wickedsoftware.net wrote:

  I think whatever makes sense to do.
 
  possibly create one jira for now with a running list that can be
modified
  and possibly as people pull from that list, cross things off or create a
  separate ticket that links back to to the main one.
 
 
 
 
  On Fri, Jul 1, 2011 at 3:35 PM, Rory Plaire codekai...@gmail.com
 wrote:
 
   @Michael -
  
   Should that list be in JIRA? It would be easier to manage, I think...
  
   If yes, I'll happily do it.
  
   -r
  
   On Fri, Jul 1, 2011 at 8:04 AM, Michael Herndon 
   mhern...@wickedsoftware.net
wrote:
  
* need to document what the build script does.  whut grammerz?
   
On Fri, Jul 1, 2011 at 10:52 AM, Michael Herndon 
mhern...@wickedsoftware.net wrote:
   
 @Rory, @All,

 The only tickets I currently have for those is LUCENE-419,
 LUCENE-418

 418, I should be able to push into the 2.9.4g branch tonight.
  419
  is
   a
 long term goal and not as important as getting the tests fixed, of
  have
the
 tests broken down into what is actually a unit test, functional
 test,
perf
 or long running test. I can get into more why it needs to be done.

 I'll also need to make document the what build script currently
 does
  on
the
 wiki  and make a few notes about testing, like using the
  RAMDirectory,
 etc.

 Things that need to get done or even be discussed.
  * There needs to be a running list of things to do/not to do with
testing.
 I don't know if this goes in a jira or do we keep a running list
on
  the
wiki
 or site for people to pick up and  help with.
  * Tests need to run on mono and not Fail (there is a good deal of
failing
 tests on mono, mostly due to the temp directory have the C:\ in
the
path).
  * Assert.ThrowExceptionType() needs to be used instead of
  Try/Catch
 Assert.Fail.  **
  * File  Path combines to the temp directory need helper methods,
  * e,g, having this in a hundred places is bad   new

   
  
 

System.IO.FileInfo(System.IO.Path.Combine(Support.AppSettings.Get(tempDir,
 ), testIndex));
  * We should still be testing deprecated methods, but we need to
 use
#pragma
 warning disable/enable 0618  for testing those. otherwise compiler
warnings
 are too numerous to be anywhere near helpful.
  * We should only be using deprecated methods in places where they
  are
 being explicitly tested, other tests that need that functionality
 in
order
 to validate those tests should be re factored to use methods that
 are
   not
 deprecated.
  * Identify code that could be abstracted into test utility
 classes.
  * Infrastructure Validation tests need to be made, anything that
  seems
 like infrastructure.  e.g. does the temp directory exist, does the
folders
 that the tests use inside the temp directory exist, can we
 read/write
   to
 those folders. (if a ton of tests fail due to the file system, we
   should
be
 able to point out that it was due to permissions or missing
 folders,
files,
 etc).
  * Identify what classes need an interface, abstract class or
  inherited
in
 order to create testing mocks. (once those classes are created,
 they
should
 be documented in the wiki).



 ** Asset.Throws needs to replace stuff like the following. We
 should
   also
 be checking the messages for exceptions and make sure they make
 sense

RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

2011-07-02 Thread Digy
Troy,

What do you mean by merging? 2.9.4g is a superset of 2.9.4 and has
* bux fixes like LUCENENET-414 
* new features  like LUCENENET-429, MemoryMappedDirectory(although not used 
yet) ,  PartiallyTrustedAppDomain tests
* improvements like LUCENENET-427, LUCENENET-418,  Refactoring of SupportClass
* API changes like 
  - StopAnalyzer(Liststring stopWords)
  - Query.ExtractTerms(ICollectionstring)
  - TopDocs.TotalHits, TopDocs.ScoreDocs
* readibily-changes like replacing some abstract methods with Func, 
while(XXX.MoveNext())s with foreachs
etc.

Is it something like creating a 2.9.4 tag and replacing trunk with 2.9.4g?

DIGY

-Original Message-
From: Troy Howard [mailto:thowar...@gmail.com] 
Sent: Friday, July 01, 2011 12:36 AM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

DIGY - Re: Why do I wait.. That's mostly because I intend to make some deep
changes, which would make merging the 2.9.4g branch back to trunk difficult.
So, it's easier to merge those changes first. Also, I won't have enough time
to make my changes until a little way in the future, but probably do have
the time to put together another release, so I'll do that first because it
fits with my work/life schedule.

Thanks,
Troy


On Thu, Jun 30, 2011 at 1:19 PM, Digy digyd...@gmail.com wrote:

 Michael,
 You interpret the report as whoever commits code wins? But when I look at
 it, I see a lof of talk, no work. .Net community is not interested in
 contributing.
 I really don't understand what hinders people to work on Lucene.Net. As I
 did for 2.9.4g, grab the code, do whatever you want on it and submit back.
 If it doesn't fit to the project's direction it can still find a place in
 contrib or in branch. All of the approaches can live side by side happily in
 the Lucene.Net repository.

 Troy,
 I also don't understand why do you wait for 2.9.4g? It is a *branch* and
 has nothing to do with the trunk. It need not be an offical release and can
 live in branch as a PoC.


 As a result, I got bored to listen to this should be done that way. What
 I want to see is I did it that way, should we continue with this.

 DIGY




 -Original Message-
 From: Troy Howard [mailto:thowar...@gmail.com]
 Sent: Thursday, June 30, 2011 10:47 PM
 To: lucene-net-dev@lucene.apache.org
 Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

 Michael,

 I agree with everything you said. My point in saying whoever commits code
 wins was to illustrate the reality of how and why the project has the
 current form.

 Building consensus is difficult. It is an essential first step before we
 can
 do something like make a list of bit-sized pieces of work that others can
 work on.

 This is why my real message of Let's find a way to accommodate both is so
 important. It allows us to build consensus, so that we can settle on a
 direction and structure our work.

 Until we accomplish that, it really is whoever commits code wins, and
 that
 is an unhealthy and unmaintainable way to operate.

 From a technical perspective, your statements about the unit tests are
 completely accurate. They really need a LOT of reworking. That's the very
 first step before making any significant changes. Part of the problem is
 that the tests themselves are not well written. The other part is that the
 Lucene object model was not designed for testability, and it makes writing
 good tests more difficult, and certain tests might not be possible. It will
 be difficult to write good unit tests without re-structuring. The biggest
 issue is the use of abstract classes with base behaviour vs interfaces or
 fully abstracted classes. Makes mocking tough. This is the direction I was
 going when I started the Lucere project. I'd like to start in on that work
 after the 2.9.4g release.

 Thanks,
 Troy


 On Thu, Jun 30, 2011 at 12:04 PM, Michael Herndon 
 mhern...@wickedsoftware.net wrote:

  I'd say that is all the more reasons that we need to work smarter and not
  harder. I'd also say thats a good reason to make sure we build consensus
  rather than just saying whoever commits code wins.
 
  And its a damn good reason to focus on the effort of growing the number
 of
  contributors and lowing the barrier to submitting patches, breaking
 things
  down into pieces that people would feel confident to work on without
  being overwhelmed by the complexity of Lucene.Net.
 
  There is a pretty big gap between Lucene 2.9.x to Lucene 4.0 and the
  internals and index formats are significantly different including nixing
  the
  current vint file format and using byte[] array slices for Terms instead
 of
  char[].
 
  So while porting 1 to 1 while require less knowledge or thought, its most
  likely going to require more hours of work. And Its definitely not going
 to
  guarantee the stability of the code or that its great code.
 
  I'd have to say that its not as stable as most would believe at the
 moment

RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

2011-07-02 Thread Digy
OK. Maybe I asked wrong question, Suppose I committed IsolatedStorageDirectory 
only to trunk. How would you merge this branch  trunk?

DIGY.

-Original Message-
From: Troy Howard [mailto:thowar...@gmail.com] 
Sent: Sunday, July 03, 2011 12:28 AM
To: lucene-net-dev@lucene.apache.org
Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

Yes. But if there are commits to trunk before that happens it's a merge.

-T
On Jul 2, 2011 1:53 PM, Digy digyd...@gmail.com wrote:
 Troy,

 What do you mean by merging? 2.9.4g is a superset of 2.9.4 and has
 * bux fixes like LUCENENET-414
 * new features like LUCENENET-429, MemoryMappedDirectory(although not used
yet) , PartiallyTrustedAppDomain tests
 * improvements like LUCENENET-427, LUCENENET-418, Refactoring of
SupportClass
 * API changes like
 - StopAnalyzer(Liststring stopWords)
 - Query.ExtractTerms(ICollectionstring)
 - TopDocs.TotalHits, TopDocs.ScoreDocs
 * readibily-changes like replacing some abstract methods with Func,
while(XXX.MoveNext())s with foreachs
 etc.

 Is it something like creating a 2.9.4 tag and replacing trunk with 2.9.4g?

 DIGY

 -Original Message-
 From: Troy Howard [mailto:thowar...@gmail.com]
 Sent: Friday, July 01, 2011 12:36 AM
 To: lucene-net-dev@lucene.apache.org
 Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

 DIGY - Re: Why do I wait.. That's mostly because I intend to make some
deep
 changes, which would make merging the 2.9.4g branch back to trunk
difficult.
 So, it's easier to merge those changes first. Also, I won't have enough
time
 to make my changes until a little way in the future, but probably do have
 the time to put together another release, so I'll do that first because it
 fits with my work/life schedule.

 Thanks,
 Troy


 On Thu, Jun 30, 2011 at 1:19 PM, Digy digyd...@gmail.com wrote:

 Michael,
 You interpret the report as whoever commits code wins? But when I look
at
 it, I see a lof of talk, no work. .Net community is not interested in
 contributing.
 I really don't understand what hinders people to work on Lucene.Net. As I
 did for 2.9.4g, grab the code, do whatever you want on it and submit
back.
 If it doesn't fit to the project's direction it can still find a place in
 contrib or in branch. All of the approaches can live side by side happily
in
 the Lucene.Net repository.

 Troy,
 I also don't understand why do you wait for 2.9.4g? It is a *branch* and
 has nothing to do with the trunk. It need not be an offical release and
can
 live in branch as a PoC.


 As a result, I got bored to listen to this should be done that way.
What
 I want to see is I did it that way, should we continue with this.

 DIGY




 -Original Message-
 From: Troy Howard [mailto:thowar...@gmail.com]
 Sent: Thursday, June 30, 2011 10:47 PM
 To: lucene-net-dev@lucene.apache.org
 Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

 Michael,

 I agree with everything you said. My point in saying whoever commits
code
 wins was to illustrate the reality of how and why the project has the
 current form.

 Building consensus is difficult. It is an essential first step before we
 can
 do something like make a list of bit-sized pieces of work that others can
 work on.

 This is why my real message of Let's find a way to accommodate both is
so
 important. It allows us to build consensus, so that we can settle on a
 direction and structure our work.

 Until we accomplish that, it really is whoever commits code wins, and
 that
 is an unhealthy and unmaintainable way to operate.

 From a technical perspective, your statements about the unit tests are
 completely accurate. They really need a LOT of reworking. That's the very
 first step before making any significant changes. Part of the problem is
 that the tests themselves are not well written. The other part is that
the
 Lucene object model was not designed for testability, and it makes
writing
 good tests more difficult, and certain tests might not be possible. It
will
 be difficult to write good unit tests without re-structuring. The biggest
 issue is the use of abstract classes with base behaviour vs interfaces or
 fully abstracted classes. Makes mocking tough. This is the direction I
was
 going when I started the Lucere project. I'd like to start in on that
work
 after the 2.9.4g release.

 Thanks,
 Troy


 On Thu, Jun 30, 2011 at 12:04 PM, Michael Herndon 
 mhern...@wickedsoftware.net wrote:

  I'd say that is all the more reasons that we need to work smarter and
not
  harder. I'd also say thats a good reason to make sure we build
consensus
  rather than just saying whoever commits code wins.
 
  And its a damn good reason to focus on the effort of growing the number
 of
  contributors and lowing the barrier to submitting patches, breaking
 things
  down into pieces that people would feel confident to work on without
  being overwhelmed by the complexity of Lucene.Net.
 
  There is a pretty big gap between

RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

2011-06-30 Thread Digy
Michael,
You interpret the report as whoever commits code wins? But when I look at it, 
I see a lof of talk, no work. .Net community is not interested in 
contributing.
I really don't understand what hinders people to work on Lucene.Net. As I did 
for 2.9.4g, grab the code, do whatever you want on it and submit back. If it 
doesn't fit to the project's direction it can still find a place in contrib or 
in branch. All of the approaches can live side by side happily in the 
Lucene.Net repository. 

Troy,
I also don't understand why do you wait for 2.9.4g? It is a *branch* and has 
nothing to do with the trunk. It need not be an offical release and can live in 
branch as a PoC. 


As a result, I got bored to listen to this should be done that way. What I 
want to see is I did it that way, should we continue with this.

DIGY




-Original Message-
From: Troy Howard [mailto:thowar...@gmail.com] 
Sent: Thursday, June 30, 2011 10:47 PM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

Michael,

I agree with everything you said. My point in saying whoever commits code
wins was to illustrate the reality of how and why the project has the
current form.

Building consensus is difficult. It is an essential first step before we can
do something like make a list of bit-sized pieces of work that others can
work on.

This is why my real message of Let's find a way to accommodate both is so
important. It allows us to build consensus, so that we can settle on a
direction and structure our work.

Until we accomplish that, it really is whoever commits code wins, and that
is an unhealthy and unmaintainable way to operate.

From a technical perspective, your statements about the unit tests are
completely accurate. They really need a LOT of reworking. That's the very
first step before making any significant changes. Part of the problem is
that the tests themselves are not well written. The other part is that the
Lucene object model was not designed for testability, and it makes writing
good tests more difficult, and certain tests might not be possible. It will
be difficult to write good unit tests without re-structuring. The biggest
issue is the use of abstract classes with base behaviour vs interfaces or
fully abstracted classes. Makes mocking tough. This is the direction I was
going when I started the Lucere project. I'd like to start in on that work
after the 2.9.4g release.

Thanks,
Troy


On Thu, Jun 30, 2011 at 12:04 PM, Michael Herndon 
mhern...@wickedsoftware.net wrote:

 I'd say that is all the more reasons that we need to work smarter and not
 harder. I'd also say thats a good reason to make sure we build consensus
 rather than just saying whoever commits code wins.

 And its a damn good reason to focus on the effort of growing the number of
 contributors and lowing the barrier to submitting patches, breaking things
 down into pieces that people would feel confident to work on without
 being overwhelmed by the complexity of Lucene.Net.

 There is a pretty big gap between Lucene 2.9.x to Lucene 4.0 and the
 internals and index formats are significantly different including nixing
 the
 current vint file format and using byte[] array slices for Terms instead of
 char[].

 So while porting 1 to 1 while require less knowledge or thought, its most
 likely going to require more hours of work. And Its definitely not going to
 guarantee the stability of the code or that its great code.

 I'd have to say that its not as stable as most would believe at the moment.

 Most of the tests avoid anything that remotely looks like it knows about
 the
 DRY principle and there is a static constructor in the core test case that
 throws an exception if it doesn't find an environment variable TEMP which
 will fail 90% of the tests and nunit will be unable to give you a clear
 reason why.  Just to name a few issues I came across working towards
 getting
 Lucene.Net into CI.  I haven't even started really digging in under the
 covers of the code yet.

 So my suggestion is to chew on this a bit more and build consensus, avoid
 fracturing people into sides.  Be open to reservations and concerns that
 others have and continue to address them.

 - Michael


 On Thu, Jun 30, 2011 at 2:10 PM, Digy digyd...@gmail.com wrote:

  Although there are a lot of people using Lucene.Net, this is our
  contribution report for the past 5 years.
 
 
 
 https://issues.apache.org/jira/secure/ConfigureReport.jspa?atl_token=A5KQ-2Q
 
 
 AV-T4JA-FDED|3204f7e696067a386144705075c074e991db2a2b|linversionId=-1issue
 
 
 Status=allselectedProjectId=12310290reportKey=com.sourcelabs.jira.plugin.r
  eport.contributions%3AcontributionreportNext=Next
 
 
  DIGY
 
  -Original Message-
  From: Ayende Rahien [mailto:aye...@ayende.com]
  Sent: Thursday, June 30, 2011 8:16 PM
  To: Rory Plaire; lucene-net-dev@lucene.apache.org
  Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed

RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

2011-06-30 Thread Digy
 A) I don't to want to commit anything  thats going to piss alot of people
off, 
 B) I don't want to spend time/waste time on modifications that are going
to be rejected.  

What I've learnt from Apache Way is creating a JIRA issue if you are
hesitant.
If no one answers in a reasonable time(mostly), then commit.

DIGY

-Original Message-
From: Michael Herndon [mailto:mhern...@wickedsoftware.net] 
Sent: Thursday, June 30, 2011 11:58 PM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

@Troy,

I've already started working towards fixing unit testing issues, and
prototyping some things that sure DRY up the testing just so that I can get
the tests running on mono.

Those changes are currently in a private git repo, however since we don't
have a CI, I'm need to make some time to manually test those on at least 3
different Os (windowx, osx, and ubuntu) before putting those back into the
2.9.4g branch.

The reason being I need those in working order so that I can do a write up
on pulling those from source and at least running the build script to
compile everything and run the tests for you.  I don't know about everyone
else, but thats a starting point I look for when I go to work on something
or commit something back.

They should make their way back sometime this month.  I think the next thing
I'll do is put my money where my mouth is, spend time break down the rest of
the CI tasks, then seeing how much stuff I can get documented into the wiki.
 The simple faceted search is a decent starting template.

@Digy I agree with the talk, no work.

Though coming from the outside in, I still cringe when I make any commits at
the moment. (even that little .gitnore file)  A) I don't to want to commit
anything  thats going to piss alot of people off, B) I don't want to spend
time/waste time on modifications that are going to be rejected.  C) it took
a good deal of going through things before I felt comfortable to even making
a commit.   D) yes I know I just need to get over it and so does everyone
else (hence the obsession with the unit tests at the moment).

and I think a key to relaying people to get over it, including myself, is to
make the point you had more clear across the board:

*If it doesn't fit to the project's direction it can still find a place in
contrib or in branch. All of the approaches can live side by side happily in
the Lucene.Net repository. * +1  because that makes feel there is more
leadway to experiment and any decent effort will at least go somewhere to
live and not be wasted.

On Thu, Jun 30, 2011 at 4:19 PM, Digy digyd...@gmail.com wrote:

 Michael,
 You interpret the report as whoever commits code wins? But when I look
at
 it, I see a lof of talk, no work. .Net community is not interested in
 contributing.
 I really don't understand what hinders people to work on Lucene.Net. As I
 did for 2.9.4g, grab the code, do whatever you want on it and submit back.
 If it doesn't fit to the project's direction it can still find a place in
 contrib or in branch. All of the approaches can live side by side happily
in
 the Lucene.Net repository.

 Troy,
 I also don't understand why do you wait for 2.9.4g? It is a *branch* and
 has nothing to do with the trunk. It need not be an offical release and
can
 live in branch as a PoC.


 As a result, I got bored to listen to this should be done that way. What
 I want to see is I did it that way, should we continue with this.

 DIGY




 -Original Message-
 From: Troy Howard [mailto:thowar...@gmail.com]
 Sent: Thursday, June 30, 2011 10:47 PM
 To: lucene-net-dev@lucene.apache.org
 Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

 Michael,

 I agree with everything you said. My point in saying whoever commits code
 wins was to illustrate the reality of how and why the project has the
 current form.

 Building consensus is difficult. It is an essential first step before we
 can
 do something like make a list of bit-sized pieces of work that others can
 work on.

 This is why my real message of Let's find a way to accommodate both is
so
 important. It allows us to build consensus, so that we can settle on a
 direction and structure our work.

 Until we accomplish that, it really is whoever commits code wins, and
 that
 is an unhealthy and unmaintainable way to operate.

 From a technical perspective, your statements about the unit tests are
 completely accurate. They really need a LOT of reworking. That's the very
 first step before making any significant changes. Part of the problem is
 that the tests themselves are not well written. The other part is that the
 Lucene object model was not designed for testability, and it makes writing
 good tests more difficult, and certain tests might not be possible. It
will
 be difficult to write good unit tests without re-structuring. The biggest
 issue is the use of abstract classes with base behaviour vs interfaces or
 fully

RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

2011-06-30 Thread Digy
I can not say I like this approach, but till we find an automated way(with good 
results), it seems to be the only way we can use.

DIGY

-Original Message-
From: Troy Howard [mailto:thowar...@gmail.com] 
Sent: Friday, July 01, 2011 12:43 AM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

Scott -

The idea of the automated port is still worth doing. Perhaps it makes sense
for someone more passionate about the line-by-line idea to do that work?

I would say, focus on what makes sense to you. Being productive, regardless
of the specific direction, is what will be most valuable. Once you start,
others will join and momentum will build. That is how these things work.

I like DIGY's approach too, but the problem with it is that it is a
never-ending manual task. The theory behind the automated port is that it
may reduce the manual work. It is complicated, but once it's built and
works, it will save a lot of future development hours. If it's built in a
sufficiently general manner, it could be useful for other project like
Lucene.Net that want to automate a port from Java to C#.

It might make sense for that to be a separate project from Lucene.Net
though.

-T


On Thu, Jun 30, 2011 at 2:13 PM, Scott Lombard lombardena...@gmail.comwrote:

 Ok I think I asked the wrong question.  I am trying to figure out where to
 put my time.  I was thinking about working on the automated porting system,
 but when I saw the response to the .NET 4.0 discussions I started to
 question if that is the right direction.  The community seemed to be more
 interested in the .NET features.

 The complexity of the automated tool is going to become very high and will
 probably end up with a line-for-line style port.  So I keep asking my self
 is the automated tool worth it.  I don't think it is.

 I like the method has been Digy is using for porting the code.  So I guess
 for me the real question is Digy where did you see 2.9.4g going next and
 what do you need help on?

 Scott




  -Original Message-
  From: Digy [mailto:digyd...@gmail.com]
  Sent: Thursday, June 30, 2011 4:20 PM
  To: lucene-net-dev@lucene.apache.org
  Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?
 
  Michael,
  You interpret the report as whoever commits code wins? But when I look
  at it, I see a lof of talk, no work. .Net community is not interested
 in
  contributing.
  I really don't understand what hinders people to work on Lucene.Net. As I
  did for 2.9.4g, grab the code, do whatever you want on it and submit
 back.
  If it doesn't fit to the project's direction it can still find a place in
  contrib or in branch. All of the approaches can live side by side happily
  in the Lucene.Net repository.
 
  Troy,
  I also don't understand why do you wait for 2.9.4g? It is a *branch* and
  has nothing to do with the trunk. It need not be an offical release and
  can live in branch as a PoC.
 
 
  As a result, I got bored to listen to this should be done that way.
 What
  I want to see is I did it that way, should we continue with this.
 
  DIGY
 
 
 
 
  -Original Message-
  From: Troy Howard [mailto:thowar...@gmail.com]
  Sent: Thursday, June 30, 2011 10:47 PM
  To: lucene-net-dev@lucene.apache.org
  Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?
 
  Michael,
 
  I agree with everything you said. My point in saying whoever commits
 code
  wins was to illustrate the reality of how and why the project has the
  current form.
 
  Building consensus is difficult. It is an essential first step before we
  can
  do something like make a list of bit-sized pieces of work that others can
  work on.
 
  This is why my real message of Let's find a way to accommodate both is
  so
  important. It allows us to build consensus, so that we can settle on a
  direction and structure our work.
 
  Until we accomplish that, it really is whoever commits code wins, and
  that
  is an unhealthy and unmaintainable way to operate.
 
  From a technical perspective, your statements about the unit tests are
  completely accurate. They really need a LOT of reworking. That's the very
  first step before making any significant changes. Part of the problem is
  that the tests themselves are not well written. The other part is that
 the
  Lucene object model was not designed for testability, and it makes
 writing
  good tests more difficult, and certain tests might not be possible. It
  will
  be difficult to write good unit tests without re-structuring. The biggest
  issue is the use of abstract classes with base behaviour vs interfaces or
  fully abstracted classes. Makes mocking tough. This is the direction I
 was
  going when I started the Lucere project. I'd like to start in on that
 work
  after the 2.9.4g release.
 
  Thanks,
  Troy
 
 
  On Thu, Jun 30, 2011 at 12:04 PM, Michael Herndon 
  mhern...@wickedsoftware.net wrote:
 
   I'd say that is all the more

RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

2011-06-30 Thread Digy
Although there are a lot of people using Lucene.Net, this is our
contribution report for the past 5 years.

https://issues.apache.org/jira/secure/ConfigureReport.jspa?atl_token=A5KQ-2Q
AV-T4JA-FDED|3204f7e696067a386144705075c074e991db2a2b|linversionId=-1issue
Status=allselectedProjectId=12310290reportKey=com.sourcelabs.jira.plugin.r
eport.contributions%3AcontributionreportNext=Next


DIGY

-Original Message-
From: Ayende Rahien [mailto:aye...@ayende.com] 
Sent: Thursday, June 30, 2011 8:16 PM
To: Rory Plaire; lucene-net-...@lucene.apache.org
Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

As someone from the nhibernate project
We stopped following hibernate a while ago, and haven't regretted it
We have mire features, less bugs and better code base

Sent from my Windows Phone From: Rory Plaire
Sent: Thursday, June 30, 2011 19:58
To: lucene-net-...@lucene.apache.org
Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?
I don't want to drag this out much longer, but I am curious with people who
hold the line-by-line sentiment - are you NHibernate users?

-r

On Thu, Jun 30, 2011 at 2:39 AM, Noel Lysaght lysag...@hotmail.com wrote:

 Can I just plug in my bit and say I agree 100% with what Moray has
outlined
 below.

 If we move away from the line by line port then over time we'll loose out
 on the momentum that is Lucene and the improvements that they make.
 It is only if the Lucene.NET community has expertise in search,  a  deep
 knowledge of the project and the community can guarantee that the
knowledge
 will survive members coming and going should such a consideration be give.

 When Lucene.NET has stood on it's feet for a number of years after it has
 moved out of Apache incubation should consideration be given to abandoning
a
 line by line port.
 By all means extend and wrap the libraries in .NET equivalents and .NET
 goodness like LINQ (we do this internally in our company at the moment);
but
 leave the core of the project on a line by line port.

 Just my tu-pence worth.

 Kind Regards
 Noel


 -Original Message- From: Moray McConnachie
 Sent: Thursday, June 30, 2011 10:25 AM

 To: lucene-net-user@lucene.apache.**orglucene-net-u...@lucene.apache.org
 Cc:
lucene-net-dev@incubator.**apache.orglucene-net-...@incubator.apache.org
 Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

 I don't think I'm as hard core on this as Neal, but remember: the
 history of the Lucene.NET project is that all the intellectual work, all
 the understanding of search, all the new features come from the Lucene
 Java folks. Theirs is an immensely respected project, and I trust them
 to add new features that will be well-tested and well-researched, and to
 have a decent roadmap which I can trust they will execute on.

 Now I know there's been an influx of capable developers to Lucene.NET
 who are ready, willing and (I'm going to assume) able to add a lot more
 value in a generic .NET implementation as they change it. But it'll take
 a while before I trust a .NET dedicated framework which is significantly
 diverged from Java in the way I do the line-by-line version. And at what
 stage is it not just not a line-by-line port, but not a port at all?

 At the same time, I recognise that if this project is going to continue,
 and attract good developers, it has to change in this direction.

 So that said, I can see why a line-by-line port might not be
 sustainable. And most people don't need it. But most of us using Lucene
 in production systems do need a system that we can trust and rely on. So
 let me chime in with someone else's plea, to keep the general structure
 close to Lucene, to keep the same general objects and inheritance
 set-up, and to keep the same method names, even if you add other methods
 and classes to provide additional functionality. ABSOLUTELY the same
 file formats. End users benefit a lot from a high degree of similarity,
 with good documentation and help being available from the Java
 community.

 Yours,
 Moray
 --**---
 Moray McConnachie
 Director of IT+44 1865 261 600
 Oxford Analytica  http://www.oxan.com

 -Original Message-
 From: Granroth, Neal V.
[mailto:neal.granroth@**thermofisher.comneal.granr...@thermofisher.com
 ]
 Sent: 29 June 2011 20:47
 To: lucene-net-user@lucene.apache.**orglucene-net-u...@lucene.apache.org
 Cc:
lucene-net-dev@incubator.**apache.orglucene-net-...@incubator.apache.org
 Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

 This is has been discussed many times.
 Lucene.NET is not valid, the code cannot be trusted, if it is not a
 line-by-line port.  It ceases to be Lucene.

 - Neal

 -Original Message-
 From: Scott Lombard
[mailto:lombardenator@gmail.**comlombardena...@gmail.com
 ]
 Sent: Wednesday, June 29, 2011 1:58 PM
 To: lucene-net-dev@lucene.apache.**org lucene-net-...@lucene.apache.org;
 lucene-net-user@lucene.apache

RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

2011-06-29 Thread Digy
Hi Rory,
I agree with you in theory. But collecting people to work on a project is
not easy as giving advise.
Till now, line-by-line port have seemed to be the best with a limited human
source. 

Would you be willing to work on your approach and maintain newer Lucene.Net
releases?

DIGY



-Original Message-
From: Rory Plaire [mailto:codekai...@gmail.com] 
Sent: Thursday, June 30, 2011 1:06 AM
To: lucene-net-dev@lucene.apache.org
Subject: Re: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

For what it's worth, I've participated in a number of projects which have
been ported from Java to .Net with varying levels of translation into
the native style and functionalty of the .Net framework. The largest are
NTS, a JTS port and NHibernate, a Java Hibernate port. My experience is that
a line-by-line port isn't as valuable as people would imagine.

Even if we discount the reality that a line-by-line port is really
unachievable due to various differences between the frameworks, keeping even
identical code in sync will always take some work: full automation on this
large of a project is infeasible. During manual effort, therefore, making
readable changes to the code is really not that much more work.

For update maintenance, porting over code from recent versions of both
projects to the .Net versions, and .Nettifying that code is little
trouble. Since both projects use source control, it's easy to see the
changes and translate them.

When it comes to debugging issues, in NTS or NHibernate, I go to the Java
sources, and even if the classes were largely rewritten to take advantage of
IEnumerable or generics or structures, running unit tests, tracing the code,
and seeing the output of each has always been straightforward.

Since I'm using .Net, I'd want the Lucene.Net project to be more .Net than a
line-by-line port of Java, in order to take advantage of the Framework as
well as provide a better code base for .Net developers to maintain. If large
.Net projects ported from Java do this, and have found considerable success,
it is, in my view, a well-proven practice and shouldn't be avoided due to
uncertainty of how the resulting code should work. Ultimately, that is what
unit tests are for, anyway.



RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

2011-06-29 Thread Digy
As a Lucene.Net user I wouldn't care whether it is line-by-line port or not.

But as a contributer, I would prefer a parallel code that makes the life
easier for manual ports of new releases(until this process is automated)

PS: I presume no one thinks of functional or index-level incompatibility.

DIGY

-Original Message-
From: Granroth, Neal V. [mailto:neal.granr...@thermofisher.com] 
Sent: Wednesday, June 29, 2011 10:47 PM
To: lucene-net-u...@lucene.apache.org
Cc: lucene-net-...@incubator.apache.org
Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

This is has been discussed many times.
Lucene.NET is not valid, the code cannot be trusted, if it is not a
line-by-line port.  It ceases to be Lucene.

- Neal

-Original Message-
From: Scott Lombard [mailto:lombardena...@gmail.com] 
Sent: Wednesday, June 29, 2011 1:58 PM
To: lucene-net-...@lucene.apache.org; lucene-net-u...@lucene.apache.org
Subject: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

 

After the large community response about moving the code base from .Net 2.0
to Net 4.0 I am trying to figure out what is the need for a line-by-line
port.  Starting with Digy's excellent work on the conversion to generics a
priority of the 2.9.4g release is the 2 packages would not be
interchangeable.  So faster turnaround from a java release won't matter to
non line-by-line users they will have to wait until the updates are made to
the non line-by-line code base.  

 

My question is there really a user base for the line-by-line port?  Anyone
have a comment?

 

Scott

 

  

 



[Lucene.Net] [jira] [Closed] (LUCENENET-428) How to do that the results are displayed in the first original tokens and them with synonyms?

2011-06-29 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy closed LUCENENET-428.
--

Resolution: Invalid

Please post questions to the mailing list, not in JIRA

 How to do that the results are displayed in the first original tokens and 
 them with synonyms?
 -

 Key: LUCENENET-428
 URL: https://issues.apache.org/jira/browse/LUCENENET-428
 Project: Lucene.Net
  Issue Type: Wish
  Components: Lucene.Net Core
Affects Versions: Lucene.Net 2.9.4
 Environment: .net 4.0
Reporter: Vladimir

 How to do that the results are displayed in the first original tokens and 
 them with synonyms?
 My Analyzer(part) :
 public override TokenStream TokenStream(string fieldName, TextReader reader)
 {
 TokenStream result = new StandardTokenizer(reader);
 result = new LowerCaseFilter(result);
   result = new StopFilter(result, stoptable);
 result = new SynonymFilter(result, synonymEngine); 
 result = new ExtendedRussianStemFilter(result, charset);
 return result;
 }
 My SynonymFilter :
 internal class SynonymFilter : TokenFilter
 {
 private readonly ISynonymEngine engine;
 private readonly QueueToken synonymTokenQueue
 = new QueueToken();
 public SynonymFilter(TokenStream tokenStream, ISynonymEngine engine) 
 : base(tokenStream)
 {
 this.engine = engine;
 }
 public override Token Next()
 {
 if (synonymTokenQueue.Count  0)
 {
 return synonymTokenQueue.Dequeue();
 }
 
 Token t = input.Next();
 
 if (t == null)
 return null;
 if (t.Type() == SYNONYM)
 return t;
 
 IEnumerablestring synonyms = engine.GetSynonyms(t.TermText());
 
 if (synonyms == null)
 {
 return t;
 }
 
 foreach (string syn in synonyms)
 {
 if (!t.TermText().Equals(syn))
 {
 var synToken = new Token(syn, t.StartOffset(),
  t.EndOffset(), SYNONYM);
 
 synToken.SetPositionIncrement(0);
 synonymTokenQueue.Enqueue(synToken);
 }
 }
 return t;
 }
 }
 Thanks!

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

2011-06-29 Thread Digy
Hi Scott,
Please avoid crossposting(as I do now). Since when I reply to your eMail, it
goes to one of the lists and thread is splitted into two.
It may be good for announcements but not for discussions.

DIGY

-Original Message-
From: Scott Lombard [mailto:lombardena...@gmail.com] 
Sent: Wednesday, June 29, 2011 9:58 PM
To: lucene-net-...@lucene.apache.org; lucene-net-u...@lucene.apache.org
Subject: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

 

After the large community response about moving the code base from .Net 2.0
to Net 4.0 I am trying to figure out what is the need for a line-by-line
port.  Starting with Digy's excellent work on the conversion to generics a
priority of the 2.9.4g release is the 2 packages would not be
interchangeable.  So faster turnaround from a java release won't matter to
non line-by-line users they will have to wait until the updates are made to
the non line-by-line code base.  

 

My question is there really a user base for the line-by-line port?  Anyone
have a comment?

 

Scott

 

  

 




RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

2011-06-29 Thread Digy
 I do not know if too much emphasis should be placed on user vs.
contributor.  
I am sorry for this misunderstanding.
What I tried to say with contributor(not committer) was the people that
works on Lucene.Net source code, not the ones who just consume it.

DIGY

-Original Message-
From: Granroth, Neal V. [mailto:neal.granr...@thermofisher.com] 
Sent: Wednesday, June 29, 2011 11:23 PM
To: lucene-net-...@lucene.apache.org
Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

I do not know if too much emphasis should be placed on user vs.
contributor.  The project needs to also consider those of us who use
Lucene.NET source releases only.
It is much easier to locally patch/fix the source when I can compare it
directly to Lucene core.

- Neal
 

-Original Message-
From: Digy [mailto:digyd...@gmail.com] 
Sent: Wednesday, June 29, 2011 2:58 PM
To: lucene-net-...@lucene.apache.org
Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

As a Lucene.Net user I wouldn't care whether it is line-by-line port or not.

But as a contributer, I would prefer a parallel code that makes the life
easier for manual ports of new releases(until this process is automated)

PS: I presume no one thinks of functional or index-level incompatibility.

DIGY

-Original Message-
From: Granroth, Neal V. [mailto:neal.granr...@thermofisher.com] 
Sent: Wednesday, June 29, 2011 10:47 PM
To: lucene-net-u...@lucene.apache.org
Cc: lucene-net-...@incubator.apache.org
Subject: RE: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

This is has been discussed many times.
Lucene.NET is not valid, the code cannot be trusted, if it is not a
line-by-line port.  It ceases to be Lucene.

- Neal

-Original Message-
From: Scott Lombard [mailto:lombardena...@gmail.com] 
Sent: Wednesday, June 29, 2011 1:58 PM
To: lucene-net-...@lucene.apache.org; lucene-net-u...@lucene.apache.org
Subject: [Lucene.Net] Is a Lucene.Net Line-by-Line Jave port needed?

 

After the large community response about moving the code base from .Net 2.0
to Net 4.0 I am trying to figure out what is the need for a line-by-line
port.  Starting with Digy's excellent work on the conversion to generics a
priority of the 2.9.4g release is the 2 packages would not be
interchangeable.  So faster turnaround from a java release won't matter to
non line-by-line users they will have to wait until the updates are made to
the non line-by-line code base.  

 

My question is there really a user base for the line-by-line port?  Anyone
have a comment?

 

Scott

 

  

 



[Lucene.Net] [jira] [Created] (LUCENENET-429) Corrupted segment file not detected and wipes index contents (LUCENE-3255)

2011-06-28 Thread Digy (JIRA)
Corrupted segment file not detected and wipes index contents (LUCENE-3255)
--

 Key: LUCENENET-429
 URL: https://issues.apache.org/jira/browse/LUCENENET-429
 Project: Lucene.Net
  Issue Type: New Feature
Reporter: Digy
Priority: Minor
 Fix For: Lucene.Net 2.9.4g


https://issues.apache.org/jira/browse/LUCENE-3255

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Updated] (LUCENENET-429) Corrupted segment file not detected and wipes index contents (LUCENE-3255)

2011-06-28 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy updated LUCENENET-429:
---

Attachment: LUCENENET-429.patch

 Corrupted segment file not detected and wipes index contents (LUCENE-3255)
 --

 Key: LUCENENET-429
 URL: https://issues.apache.org/jira/browse/LUCENENET-429
 Project: Lucene.Net
  Issue Type: New Feature
Reporter: Digy
Priority: Minor
 Fix For: Lucene.Net 2.9.4g

 Attachments: LUCENENET-429.patch


 https://issues.apache.org/jira/browse/LUCENE-3255

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Updated] (LUCENENET-427) Provide limit on phrase analysis in FastVectorHighlighter (LUCENE-3234)

2011-06-27 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy updated LUCENENET-427:
---

Attachment: FastVectorHighlighter.patch

 Provide limit on phrase analysis in FastVectorHighlighter (LUCENE-3234)
 ---

 Key: LUCENENET-427
 URL: https://issues.apache.org/jira/browse/LUCENENET-427
 Project: Lucene.Net
  Issue Type: Improvement
Affects Versions: Lucene.Net 2.9.2, Lucene.Net 2.9.4, Lucene.Net 2.9.4g
Reporter: Digy
Priority: Minor
 Fix For: Lucene.Net 2.9.4g

 Attachments: FastVectorHighlighter.patch


 https://issues.apache.org/jira/browse/LUCENE-3234

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Resolved] (LUCENENET-427) Provide limit on phrase analysis in FastVectorHighlighter (LUCENE-3234)

2011-06-27 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy resolved LUCENENET-427.


Resolution: Fixed

Committed 

 Provide limit on phrase analysis in FastVectorHighlighter (LUCENE-3234)
 ---

 Key: LUCENENET-427
 URL: https://issues.apache.org/jira/browse/LUCENENET-427
 Project: Lucene.Net
  Issue Type: Improvement
Affects Versions: Lucene.Net 2.9.2, Lucene.Net 2.9.4, Lucene.Net 2.9.4g
Reporter: Digy
Priority: Minor
 Fix For: Lucene.Net 2.9.4g

 Attachments: FastVectorHighlighter.patch


 https://issues.apache.org/jira/browse/LUCENE-3234

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[jira] [Commented] (LUCENE-3234) Provide limit on phrase analysis in FastVectorHighlighter

2011-06-25 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENE-3234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13054935#comment-13054935
 ] 

Digy commented on LUCENE-3234:
--

I am not sure how much it is related to this issue but there was
a similar issue in Lucene.Net.
https://issues.apache.org/jira/browse/LUCENENET-350



 Provide limit on phrase analysis in FastVectorHighlighter
 -

 Key: LUCENE-3234
 URL: https://issues.apache.org/jira/browse/LUCENE-3234
 Project: Lucene - Java
  Issue Type: Improvement
Affects Versions: 2.9.4, 3.0.3, 3.1, 3.2, 3.3
Reporter: Mike Sokolov
Assignee: Koji Sekiguchi
 Fix For: 3.4, 4.0

 Attachments: LUCENE-3234.patch, LUCENE-3234.patch, LUCENE-3234.patch


 With larger documents, FVH can spend a lot of time trying to find the 
 best-scoring snippet as it examines every possible phrase formed from 
 matching terms in the document.  If one is willing to accept
 less-than-perfect scoring by limiting the number of phrases that are 
 examined, substantial speedups are possible.  This is analogous to the 
 Highlighter limit on the number of characters to analyze.
 The patch includes an artifical test case that shows  1000x speedup.  In a 
 more normal test environment, with English documents and random queries, I am 
 seeing speedups of around 3-10x when setting phraseLimit=1, which has the 
 effect of selecting the first possible snippet in the document.  Most of our 
 sites operate in this way (just show the first snippet), so this would be a 
 big win for us.
 With phraseLimit = -1, you get the existing FVH behavior. At larger values of 
 phraseLimit, you may not get substantial speedup in the normal case, but you 
 do get the benefit of protection against blow-up in pathological cases.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira



-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org



[Lucene.Net] [jira] [Commented] (LUCENENET-426) Mark BaseFragmentsBuilder methods as virtual

2011-06-23 Thread Digy (JIRA)

[ 
https://issues.apache.org/jira/browse/LUCENENET-426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13053722#comment-13053722
 ] 

Digy commented on LUCENENET-426:


10 min. work done.
DIGY

 Mark BaseFragmentsBuilder methods as virtual
 

 Key: LUCENENET-426
 URL: https://issues.apache.org/jira/browse/LUCENENET-426
 Project: Lucene.Net
  Issue Type: Improvement
  Components: Lucene.Net Contrib
Affects Versions: Lucene.Net 2.9.2, Lucene.Net 2.9.4, Lucene.Net 3.x, 
 Lucene.Net 2.9.4g
Reporter: Itamar Syn-Hershko
Priority: Minor
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g

 Attachments: fvh.patch


 Without marking methods in BaseFragmentsBuilder as virtual, it is meaningless 
 to have FragmentsBuilder deriving from a class named Base, since most of 
 its functionality cannot be overridden. Attached is a patch for marking the 
 important methods virtual.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




[Lucene.Net] [jira] [Resolved] (LUCENENET-426) Mark BaseFragmentsBuilder methods as virtual

2011-06-16 Thread Digy (JIRA)

 [ 
https://issues.apache.org/jira/browse/LUCENENET-426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Digy resolved LUCENENET-426.


   Resolution: Fixed
Fix Version/s: Lucene.Net 2.9.4g
   Lucene.Net 2.9.4

Thanks Itamar.
Fixed in trunk  2.9.4g branch.

DIGY

 Mark BaseFragmentsBuilder methods as virtual
 

 Key: LUCENENET-426
 URL: https://issues.apache.org/jira/browse/LUCENENET-426
 Project: Lucene.Net
  Issue Type: Improvement
  Components: Lucene.Net Contrib
Affects Versions: Lucene.Net 2.9.2, Lucene.Net 2.9.4, Lucene.Net 3.x, 
 Lucene.Net 2.9.4g
Reporter: Itamar Syn-Hershko
Priority: Minor
 Fix For: Lucene.Net 2.9.4, Lucene.Net 2.9.4g

 Attachments: fvh.patch


 Without marking methods in BaseFragmentsBuilder as virtual, it is meaningless 
 to have FragmentsBuilder deriving from a class named Base, since most of 
 its functionality cannot be overridden. Attached is a patch for marking the 
 important methods virtual.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira




  1   2   3   4   >