Re: patch: adding IndexReader.isCurrent()

[Radomir Mladenovic]

Method isCurrrent() is a useful addition and I'm glad fix with
timestamp use for initial version is finally commited. However,
although version is not meaningful, I believe it should stay in API...

The software I'm working on is using pair "version, document_id" as a
fast reference to lucene document - if version is not current we have
to make an additional query first in order to get document id. Without
version information method isCurrent() is useless in this context as
we don't know which version of search results client has.

Here's one simple example... Let's say we have web application that's
paging search results (20 or so per page) and we re-run query for each
"next results page" request. Without version information, how can we
eventually warn user that index changed between its two "next results
page" requests?

So I think getCurentVersion() should stay and (as I suggested earler)
would like to see getVersion() added to IndexReader.

In addition, shouldn't SigmentInfos.readCurrentVersion() rely on
commit lock when reading segments file? Could someone with deeper
understanding of lucene internals check this?

Radomir


On 6/4/05, Otis Gospodnetic <[EMAIL PROTECTED]> wrote:
> Well, Radomir, the person who asked about that method should be on
> java-dev@, so I'll assume he'll speak up if he needs that method.  I
> think isCurrent() is good to go.
> 
> Otis
> 
> 
> --- Daniel Naber <[EMAIL PROTECTED]> wrote:
> 
> > On Friday 03 June 2005 23:51, Otis Gospodnetic wrote:
> >
> > > Isn't version a time stamp now?
> >
> > It starts as a time stamp, but then +1 is added for each change. So
> > that
> > number has no useful meaning I think. It's only useful for
> > comparison. Or
> > am I missing something?
> >
> > Regards
> >  Daniel
> >
> > --
> > http://www.danielnaber.de
> >
> > -
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
> >
> >
> 
> 
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
> 
>

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Error Initializing an FSDirctory - disableLuceneLocks is null

[Craig Stephen]

 

We've been getting an error when trying to initialize an FSDirectory.
It's on line 48 of the FSDirectory source file.  The system property
disableLuceneLocks is null.  Now we could set this manually, but it was
working up until about a week ago when we moved our Lucene Wrapper class
to a new package.  Has anyone had this problem before? 

 

According to the documentation, if you don't set disableLuceneLocks, it
should default to false which is what we want.  Any suggestions would be
appreciated.

RE: Error Initializing an FSDirctory - disableLuceneLocks is null (FollowUp)

[Craig Stephen]

We've been getting an error when trying to initialize an FSDirectory.
It's on line 48 of the FSDirectory source file.  The system property
disableLuceneLocks is null.  Now we could set this manually, but it was
working up until about a week ago when we moved our Lucene Wrapper class
to a new package.  How is the system property for disableLuceneLocks
normally set? Has anyone had this problem before? 

 

According to the documentation, if you don't set disableLuceneLocks, it
should default to false which is what we want.  Any suggestions would be
appreciated.

PS.
We are currently using Lucene 1.4.3.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: Error Initializing an FSDirctory - disableLuceneLocks is null (FollowUp)

[Bernhard Messer]

Craig,

what kind of error did you get when initializing FSDirectory ? The fact 
that Boolean.getBoolean("disableLuceneLocks") returns false or that the 
system property "disableLuceneLocks" is null is not an error. It  will 
be null, as long as you do not specify the system parameter with  
-DdisableLuceneLocks=true or -DdisableLuceneLocks=false during startup. 
The other case where disableLuceneLocks is set to true is running the 
software on a JVM version 1.1.x. So the documentation is ok when it says 
that per default disableLuceneLocks is false.


Bernhard

Craig Stephen wrote:


We've been getting an error when trying to initialize an FSDirectory.
It's on line 48 of the FSDirectory source file.  The system property
disableLuceneLocks is null.  Now we could set this manually, but it was
working up until about a week ago when we moved our Lucene Wrapper class
to a new package.  How is the system property for disableLuceneLocks
normally set? Has anyone had this problem before? 




According to the documentation, if you don't set disableLuceneLocks, it
should default to false which is what we want.  Any suggestions would be
appreciated.

PS.
We are currently using Lucene 1.4.3.


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


 




-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

DO NOT REPLY [Bug 35241] New: - MultiSearcher fails on WildCardQuery

[bugzilla]

DO NOT REPLY TO THIS EMAIL, BUT PLEASE POST YOUR BUG·
RELATED COMMENTS THROUGH THE WEB INTERFACE AVAILABLE AT
.
ANY REPLY MADE TO THIS MESSAGE WILL NOT BE COLLECTED AND·
INSERTED IN THE BUG DATABASE.

http://issues.apache.org/bugzilla/show_bug.cgi?id=35241

   Summary: MultiSearcher fails on WildCardQuery
   Product: Lucene
   Version: unspecified
  Platform: All
OS/Version: All
Status: NEW
  Severity: minor
  Priority: P2
 Component: Search
AssignedTo: lucene-dev@jakarta.apache.org
ReportedBy: [EMAIL PROTECTED]


If one of the index contains only one term for the WildCardQuery. The rewriting
of the query becomes a term query and if in the other indexes there is a
different term or more than one term, the combine fails.

The test on the contrib/highlighter show the bug.

Tested on the trunk from the svn revision 180311.

-- 
Configure bugmail: http://issues.apache.org/bugzilla/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the assignee for the bug, or are watching the assignee.

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: svn commit: r180010 - in /lucene/java/trunk/src: java/org/apache/lucene/index/IndexReader.java java/org/apache/lucene/index/SegmentInfos.java test/org/apache/lucene/index/TestIndexReader.java

[Bernhard Messer]

Daniel,

i would suggest to remove the deprecated flags from methods 
"getCurrentVersion()" and "lastModified()". Even if "lastModified()" 
returns a value which is operating system dependend, we should leave it 
up to the developer to use it or not. If we do not provide such a 
functionality, people has to develop their own solutions by reading the 
last modified date from the "segments" file itself. As long as the 
filename "segments" is internal used only, we will break all 
implementations when we try to change the name of the segments file.


Bernhard

[EMAIL PROTECTED] wrote:


Author: dnaber
Date: Sat Jun  4 11:16:00 2005
New Revision: 180010

URL: http://svn.apache.org/viewcvs?rev=180010&view=rev
Log:
add isCurrent() and initialise the segments version number with 
System.currentTimeMillis()

Modified:
   lucene/java/trunk/src/java/org/apache/lucene/index/IndexReader.java
   lucene/java/trunk/src/java/org/apache/lucene/index/SegmentInfos.java
   lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexReader.java

Modified: lucene/java/trunk/src/java/org/apache/lucene/index/IndexReader.java
URL: 
http://svn.apache.org/viewcvs/lucene/java/trunk/src/java/org/apache/lucene/index/IndexReader.java?rev=180010&r1=180009&r2=180010&view=diff
==
--- lucene/java/trunk/src/java/org/apache/lucene/index/IndexReader.java 
(original)
+++ lucene/java/trunk/src/java/org/apache/lucene/index/IndexReader.java Sat Jun 
 4 11:16:00 2005
@@ -224,6 +224,7 @@
   * @param directory where the index resides.
   * @return version number.
   * @throws IOException if segments file cannot be read
+   * @deprecated use [EMAIL PROTECTED] #isCurrent()} instead
   */
  public static long getCurrentVersion(String directory) throws IOException {
return getCurrentVersion(new File(directory));
@@ -236,6 +237,7 @@
   * @param directory where the index resides.
   * @return version number.
   * @throws IOException if segments file cannot be read
+   * @deprecated use [EMAIL PROTECTED] #isCurrent()} instead
   */
  public static long getCurrentVersion(File directory) throws IOException {
Directory dir = FSDirectory.getDirectory(directory, false);
@@ -251,9 +253,24 @@
   * @param directory where the index resides.
   * @return version number.
   * @throws IOException if segments file cannot be read.
+   * @deprecated use [EMAIL PROTECTED] #isCurrent()} instead
   */
  public static long getCurrentVersion(Directory directory) throws IOException {
return SegmentInfos.readCurrentVersion(directory);
+  }
+
+  /**
+   * Check whether this IndexReader still works on a current version of the 
index.
+   * If this is not the case you will need to re-open the IndexReader to
+   * make sure you see the latest changes made to the index.
+   * 
+   * @throws IOException

+   */
+  public boolean isCurrent() throws IOException {
+if (SegmentInfos.readCurrentVersion(directory) != 
segmentInfos.getVersion()) {
+  return false;
+}
+return true;
  }

  /**

Modified: lucene/java/trunk/src/java/org/apache/lucene/index/SegmentInfos.java
URL: 
http://svn.apache.org/viewcvs/lucene/java/trunk/src/java/org/apache/lucene/index/SegmentInfos.java?rev=180010&r1=180009&r2=180010&view=diff
==
--- lucene/java/trunk/src/java/org/apache/lucene/index/SegmentInfos.java 
(original)
+++ lucene/java/trunk/src/java/org/apache/lucene/index/SegmentInfos.java Sat 
Jun  4 11:16:00 2005
@@ -29,7 +29,11 @@
  public static final int FORMAT = -1;
  
  public int counter = 0;// used to name new segments

-  private long version = 0; //counts how often the index has been changed by 
adding or deleting docs
+  /**
+   * counts how often the index has been changed by adding or deleting docs.
+   * starting with the current time in milliseconds forces to create unique 
version numbers.
+   */
+  private long version = System.currentTimeMillis();

  public final SegmentInfo info(int i) {
return (SegmentInfo) elementAt(i);
@@ -59,7 +63,7 @@
  
  if(format >= 0){// in old format the version number may be at the end of the file

if (input.getFilePointer() >= input.length())
-  version = 0; // old file format without version number
+  version = System.currentTimeMillis(); // old file format without 
version number
else
  version = input.readLong(); // read version
  }

Modified: 
lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexReader.java
URL: 
http://svn.apache.org/viewcvs/lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexReader.java?rev=180010&r1=180009&r2=180010&view=diff
==
--- lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexReader.java 
(original)
+++ lucene/java/trunk/src/test/org/apache/lucene/index/TestIndexReader.java Sat 
Jun  4 11:16:0

Re: svn-commit: 168449 FSDirectory

[Bernhard Messer]

Hi Daniel,

i just had a look at the new implementation that FSDirectory deletes 
lucene related files only. I like the patch, but i think we left some 
room for optimization. In the current implementation, it's necessary to 
run thru all known Lucene extensions (13 for the moment), for each call 
of "LuceneFileFilter.accept()". If creating an index in a directory 
which contains several hundred files, this definitly will be a 
bottleneck. So creating a new Index in a directory containing 100 files, 
we will endup with 1300 calls to
"if (name.endsWith("."+IndexReader.FILENAME_EXTENSIONS[i]))" which 
always needs to create a new StringBuffer to merge the two strings.


Therefore i would like to propose two changes:
1) we should store the extension in a hash and not in String[] to have a 
faster lookup

2) check for the file extension only without using the "."

any thoughts

Bernhard


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

IndexFileNames

[Doug Cutting]

[EMAIL PROTECTED] wrote:

--- lucene/java/trunk/src/java/org/apache/lucene/store/FSDirectory.java 
(original)
+++ lucene/java/trunk/src/java/org/apache/lucene/store/FSDirectory.java Mon Jun 
 6 10:52:12 2005
@@ -52,8 +52,8 @@
 if (name.endsWith("."+IndexReader.FILENAME_EXTENSIONS[i]))
   return true;
   }
-  if (name.equals("deletable")) return true;
-  else if (name.equals("segments")) return true;
+  if (name.equals(Constants.INDEX_DELETABLE_FILENAME)) return true;
+  else if (name.equals(Constants.INDEX_SEGMENTS_FILENAME)) return true;
   else if (name.matches(".+\\.f\\d+")) return true;
   return false;


This really belongs in the index package.  That way, when we change the 
set of files in an index, the changes will be localized.


So this, LuceneFileFilter, Constants.INDEX_* and 
IndexReader.FILENAME_EXTENSIONS, should all be moved to a common home in 
the index package, like org.apache.lucene.index.IndexFileNames.  Thoughts?


Doug

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: svn-commit: 168449 FSDirectory

[Doug Cutting]

Bernhard Messer wrote:

Therefore i would like to propose two changes:
1) we should store the extension in a hash and not in String[] to have a 
faster lookup


Do you mean to use something like:

String lastDot = name.lastIndexOf('.');
if (lastDot >= 0) {
  String nameExt = name.substring(lastDot+1, name.length());
  if (FILENAME_EXTENSIONS.get(nameExt)) {
...
  }
}

That would allocate a new String in each case, which would be 
substantially faster.  Is that what you meant?



2) check for the file extension only without using the "."


Do you mean something like:

  String ext = IndexReader.FILENAME_EXTENSIONS[i];
  if (name.endsWith(ext)
  && name.length >= ext.length()+1
  && '.' == name.charAt(name.length()-(ext.length()+1))) {
...
  }

I don't see how this works with (1) above.  My guess is that (1) alone 
would be fastest, even though it still allocates objects.


Also, as mentioned in another message, I think this class, along with 
other index file name logic, should all move to a single place.


Doug

-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

RE: Error Initializing an FSDirctory - disableLuceneLocks is null (FollowUp)

[Craig Stephen]

The error we get is NoClassDefFoundError at
org.apache.lucene.index.IndexReader.open(IndexReader.java:95);  It's
breaking because the method that gets the disableLuceneLocks is
returning null, but here's the weird thing.  The product has been
running 7x24 for the last 6 months without any problem.  This error
cropped up when we repackaged the class we were using to wrap the Lucene
classes.  Currently we are manually setting the parameter in the code
but I'm trying to figure out how we've been able to use the class
without setting this parameter in the first place.

-Original Message-
From: Bernhard Messer [mailto:[EMAIL PROTECTED] 
Sent: Monday, June 06, 2005 11:17 AM
To: java-dev@lucene.apache.org
Subject: Re: Error Initializing an FSDirctory - disableLuceneLocks is
null (FollowUp)

Craig,

what kind of error did you get when initializing FSDirectory ? The fact 
that Boolean.getBoolean("disableLuceneLocks") returns false or that the 
system property "disableLuceneLocks" is null is not an error. It  will 
be null, as long as you do not specify the system parameter with  
-DdisableLuceneLocks=true or -DdisableLuceneLocks=false during startup. 
The other case where disableLuceneLocks is set to true is running the 
software on a JVM version 1.1.x. So the documentation is ok when it says

that per default disableLuceneLocks is false.

Bernhard

Craig Stephen wrote:

>We've been getting an error when trying to initialize an FSDirectory.
>It's on line 48 of the FSDirectory source file.  The system property
>disableLuceneLocks is null.  Now we could set this manually, but it was
>working up until about a week ago when we moved our Lucene Wrapper
class
>to a new package.  How is the system property for disableLuceneLocks
>normally set? Has anyone had this problem before? 
>
> 
>
>According to the documentation, if you don't set disableLuceneLocks, it
>should default to false which is what we want.  Any suggestions would
be
>appreciated.
>
>PS.
>We are currently using Lucene 1.4.3.
>
>
>-
>To unsubscribe, e-mail: [EMAIL PROTECTED]
>For additional commands, e-mail: [EMAIL PROTECTED]
>
>
>  
>


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]




-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Re: IndexFileNames

[Bernhard Messer]

Doug Cutting wrote:


[EMAIL PROTECTED] wrote:

--- 
lucene/java/trunk/src/java/org/apache/lucene/store/FSDirectory.java 
(original)
+++ 
lucene/java/trunk/src/java/org/apache/lucene/store/FSDirectory.java 
Mon Jun  6 10:52:12 2005

@@ -52,8 +52,8 @@
 if (name.endsWith("."+IndexReader.FILENAME_EXTENSIONS[i]))
   return true;
   }
-  if (name.equals("deletable")) return true;
-  else if (name.equals("segments")) return true;
+  if (name.equals(Constants.INDEX_DELETABLE_FILENAME)) return true;
+  else if (name.equals(Constants.INDEX_SEGMENTS_FILENAME)) 
return true;

   else if (name.matches(".+\\.f\\d+")) return true;
   return false;



This really belongs in the index package.  That way, when we change 
the set of files in an index, the changes will be localized.


So this, LuceneFileFilter, Constants.INDEX_* and 
IndexReader.FILENAME_EXTENSIONS, should all be moved to a common home 
in the index package, like org.apache.lucene.index.IndexFileNames.  
Thoughts?


yes, this makes sense. I will try to pack all the spreaded filenames and 
extensions together in a new class 
org.apache.lucene.index.IndexFileNames. So we have everything in one 
place and it will be much easier to maintain or even to change.


Bernhard


-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]