Re: [Dspace-tech] Reindexing vs Tomcat running
Graham: I just wanted to let you know that your patch has been working now for a week successfully. We are not getting any more errors while the indexer runs. I hope this patch will become part of the next release, since I think the sort of errors I was seeing are not that rare - especially for large repositories. Thanks again! Jose -Original Message- From: Graham Triggs [mailto:[EMAIL PROTECTED] Sent: Wednesday, February 14, 2007 2:22 PM To: Jose Blanco Cc: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Reindexing vs Tomcat running Jose, That depends on what you want to do the test for - and what platform you are running under. When running offline, the createIndex method (actually, the new overloaded version that takes a second parameter - more on that later), directly calls moveIndex(). So, to stop an offline indexer from moving the directory, you need to comment that call out (the offline indexer can only move the index if the destination isn't locked - ie. you are not running under Windows, or you don't have an IndexSearcher open). moveAlternateIndex() is actually used by DSQuery, and is the method by which the web application moves the index from search.dir.create. This method actually has the additional check for the existence of a 'dspace-index-complete' file, which is used to stop an already running web application from attempting to move the new index before it has finished being built. So, to stop a running web application from moving the index, you need to comment out the call to moveIndex() here. To stop both offline and web applications from moving the index, you therefore have to comment out both calls to moveIndex(). Or perhaps more easily, just make moveIndex() return without doing anything. However, as I mentioned, createIndex has an overloaded version with an additional parameter - a boolean flag that states whether this index is to be used only for testing purposes. If you call createIndex with this flag set to true, then it will ignore search.dir.create (if it has been set), and attempt to create an index in the directory specified by the environment variable java.io.tmpdir. So, you could extend the main method of the application you are using to call createIndex in this fashion. Note that by default, when specifying this is a test index, it will also delete the index directly after building it. If you need the index to remain after the execution of the program, you would need to comment out the appropriate call to deleteDirs() from the createIndex() method. G - Original Message - From: Jose Blanco [EMAIL PROTECTED] To: 'Graham Triggs' [EMAIL PROTECTED] Cc: dspace-tech@lists.sourceforge.net Sent: Wednesday, February 14, 2007 5:41 PM Subject: RE: [Dspace-tech] Reindexing vs Tomcat running Graham: Your new patch completes successfully. Now I would like to make a change to it. I would like it NOT to copy from search.dir.create to search.dir. I want to do this just for testing purposes. So I think in order to do this, I need to change the DSIndexer.java file. I believe if I Comment out return DSIndexer.moveIndex(c, offlineIndexDir, toPath); in the routine moveAlternateIndex, and just return false; That should do it, right? I don't think this will break anything. I simply want to see what goes on (resource being used) when the indexer is running, but I don't want it to replace the active one being used by DSpace. Thanks! Jose -Original Message- From: Graham Triggs [mailto:[EMAIL PROTECTED] Sent: Wednesday, February 14, 2007 5:52 AM To: Jose Blanco Cc: dspace-tech@lists.sourceforge.net Subject: RE: [Dspace-tech] Reindexing vs Tomcat running Jose, I've just uploaded what should be my final revision of this patch. It incorporates a couple of additional small fixes: 1) If somebody is logged into a console and are 'in' the search directory, it no longer stops the index replacement from happening 2) Minor structural improvements to the code. I've now tested just about every scenario that I can think of occurring for which it is possible to fabricate a test case for, on both Linux and Windows platforms, including with multiple Tomcat instances running. As before, let me know how you get on. G On Mon, 2007-02-12 at 17:06 -0500, Jose Blanco wrote: Graham: I installed you patch in my dev area, and I see that search.dir.create has the new indexes but it did not seem to copy it over to the search.dir area. I don't see anything in the log file about failing to copy it over. I'm assuming that I have something open as you indicate in the caveats section of your patch, but would I get some sort of message on the log file? Thanks! Jose -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Thursday, February 08, 2007 5:29 PM To: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace
Re: [Dspace-tech] Reindexing vs Tomcat running
Jose, I've just uploaded what should be my final revision of this patch. It incorporates a couple of additional small fixes: 1) If somebody is logged into a console and are 'in' the search directory, it no longer stops the index replacement from happening 2) Minor structural improvements to the code. I've now tested just about every scenario that I can think of occurring for which it is possible to fabricate a test case for, on both Linux and Windows platforms, including with multiple Tomcat instances running. As before, let me know how you get on. G On Mon, 2007-02-12 at 17:06 -0500, Jose Blanco wrote: Graham: I installed you patch in my dev area, and I see that search.dir.create has the new indexes but it did not seem to copy it over to the search.dir area. I don't see anything in the log file about failing to copy it over. I'm assuming that I have something open as you indicate in the caveats section of your patch, but would I get some sort of message on the log file? Thanks! Jose -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Thursday, February 08, 2007 5:29 PM To: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Reindexing vs Tomcat running Stephane, I've now submitted a patch for this reindexing issue: #1655583 You can read my notes and download the patch file here: https://sourceforge.net/tracker/?func=detailatid=319984aid=1655583group_i d=19984 With the patch applied, you will also need to add a 'search.dir.create' entry to your dspace.cfg as per the comments. Let me know if you have any questions. G -- Graham Triggs Technical Architect Open Repository +44 (0)20 7631 9942 This email has been scanned by Postini. For more information please visit http://www.postini.com - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Reindexing vs Tomcat running
Thanks Graham! I'll give it a go and let you know. -Jose -Original Message- From: Graham Triggs [mailto:[EMAIL PROTECTED] Sent: Wednesday, February 14, 2007 5:52 AM To: Jose Blanco Cc: dspace-tech@lists.sourceforge.net Subject: RE: [Dspace-tech] Reindexing vs Tomcat running Jose, I've just uploaded what should be my final revision of this patch. It incorporates a couple of additional small fixes: 1) If somebody is logged into a console and are 'in' the search directory, it no longer stops the index replacement from happening 2) Minor structural improvements to the code. I've now tested just about every scenario that I can think of occurring for which it is possible to fabricate a test case for, on both Linux and Windows platforms, including with multiple Tomcat instances running. As before, let me know how you get on. G On Mon, 2007-02-12 at 17:06 -0500, Jose Blanco wrote: Graham: I installed you patch in my dev area, and I see that search.dir.create has the new indexes but it did not seem to copy it over to the search.dir area. I don't see anything in the log file about failing to copy it over. I'm assuming that I have something open as you indicate in the caveats section of your patch, but would I get some sort of message on the log file? Thanks! Jose -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Thursday, February 08, 2007 5:29 PM To: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Reindexing vs Tomcat running Stephane, I've now submitted a patch for this reindexing issue: #1655583 You can read my notes and download the patch file here: https://sourceforge.net/tracker/?func=detailatid=319984aid=1655583group_i d=19984 With the patch applied, you will also need to add a 'search.dir.create' entry to your dspace.cfg as per the comments. Let me know if you have any questions. G -- Graham Triggs Technical Architect Open Repository +44 (0)20 7631 9942 This email has been scanned by Postini. For more information please visit http://www.postini.com - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech - Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.phpp=sourceforgeCID=DEVDEV ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Reindexing vs Tomcat running
Jose, That depends on what you want to do the test for - and what platform you are running under. When running offline, the createIndex method (actually, the new overloaded version that takes a second parameter - more on that later), directly calls moveIndex(). So, to stop an offline indexer from moving the directory, you need to comment that call out (the offline indexer can only move the index if the destination isn't locked - ie. you are not running under Windows, or you don't have an IndexSearcher open). moveAlternateIndex() is actually used by DSQuery, and is the method by which the web application moves the index from search.dir.create. This method actually has the additional check for the existence of a 'dspace-index-complete' file, which is used to stop an already running web application from attempting to move the new index before it has finished being built. So, to stop a running web application from moving the index, you need to comment out the call to moveIndex() here. To stop both offline and web applications from moving the index, you therefore have to comment out both calls to moveIndex(). Or perhaps more easily, just make moveIndex() return without doing anything. However, as I mentioned, createIndex has an overloaded version with an additional parameter - a boolean flag that states whether this index is to be used only for testing purposes. If you call createIndex with this flag set to true, then it will ignore search.dir.create (if it has been set), and attempt to create an index in the directory specified by the environment variable java.io.tmpdir. So, you could extend the main method of the application you are using to call createIndex in this fashion. Note that by default, when specifying this is a test index, it will also delete the index directly after building it. If you need the index to remain after the execution of the program, you would need to comment out the appropriate call to deleteDirs() from the createIndex() method. G - Original Message - From: Jose Blanco [EMAIL PROTECTED] To: 'Graham Triggs' [EMAIL PROTECTED] Cc: dspace-tech@lists.sourceforge.net Sent: Wednesday, February 14, 2007 5:41 PM Subject: RE: [Dspace-tech] Reindexing vs Tomcat running Graham: Your new patch completes successfully. Now I would like to make a change to it. I would like it NOT to copy from search.dir.create to search.dir. I want to do this just for testing purposes. So I think in order to do this, I need to change the DSIndexer.java file. I believe if I Comment out return DSIndexer.moveIndex(c, offlineIndexDir, toPath); in the routine moveAlternateIndex, and just return false; That should do it, right? I don't think this will break anything. I simply want to see what goes on (resource being used) when the indexer is running, but I don't want it to replace the active one being used by DSpace. Thanks! Jose -Original Message- From: Graham Triggs [mailto:[EMAIL PROTECTED] Sent: Wednesday, February 14, 2007 5:52 AM To: Jose Blanco Cc: dspace-tech@lists.sourceforge.net Subject: RE: [Dspace-tech] Reindexing vs Tomcat running Jose, I've just uploaded what should be my final revision of this patch. It incorporates a couple of additional small fixes: 1) If somebody is logged into a console and are 'in' the search directory, it no longer stops the index replacement from happening 2) Minor structural improvements to the code. I've now tested just about every scenario that I can think of occurring for which it is possible to fabricate a test case for, on both Linux and Windows platforms, including with multiple Tomcat instances running. As before, let me know how you get on. G On Mon, 2007-02-12 at 17:06 -0500, Jose Blanco wrote: Graham: I installed you patch in my dev area, and I see that search.dir.create has the new indexes but it did not seem to copy it over to the search.dir area. I don't see anything in the log file about failing to copy it over. I'm assuming that I have something open as you indicate in the caveats section of your patch, but would I get some sort of message on the log file? Thanks! Jose -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Thursday, February 08, 2007 5:29 PM To: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Reindexing vs Tomcat running Stephane, I've now submitted a patch for this reindexing issue: #1655583 You can read my notes and download the patch file here: https://sourceforge.net/tracker/?func=detailatid=319984aid=1655583group_i d=19984 With the patch applied, you will also need to add a 'search.dir.create' entry to your dspace.cfg as per the comments. Let me know if you have any questions. G -- Graham Triggs Technical Architect Open Repository +44 (0)20 7631 9942 This email has been scanned by Postini. For more information
Re: [Dspace-tech] Reindexing vs Tomcat running
Graham: Let me explain what I want to do. I want to leave the web application code unchanged, and so it will be using dspace 1.4.1 without your patch. But I want to update the code so that I can run filter-media from [dspace]/bin using your code. So I could either: 1. have moveIndex return false, and then I can run filter-media and see what happens to our server. Presently when filter-media goes to do the indexing, it seems to make our web instance of DSpace not run. I need to investigate this to find out what is going on. OR 2.I can change the code so that createIndex passes a 'true' indicating to use the dir specified in java.io.tmpdir. Now I'm not sure how/where to set this env variable, so I'm leaning towards option 1. Seems like the safest. Does that make sense? I'm being very cautious about this because I don't want to mess up our installation. Thanks for all your help with this! Jose -Original Message- From: Graham Triggs [mailto:[EMAIL PROTECTED] Sent: Wednesday, February 14, 2007 2:22 PM To: Jose Blanco Cc: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Reindexing vs Tomcat running Jose, That depends on what you want to do the test for - and what platform you are running under. When running offline, the createIndex method (actually, the new overloaded version that takes a second parameter - more on that later), directly calls moveIndex(). So, to stop an offline indexer from moving the directory, you need to comment that call out (the offline indexer can only move the index if the destination isn't locked - ie. you are not running under Windows, or you don't have an IndexSearcher open). moveAlternateIndex() is actually used by DSQuery, and is the method by which the web application moves the index from search.dir.create. This method actually has the additional check for the existence of a 'dspace-index-complete' file, which is used to stop an already running web application from attempting to move the new index before it has finished being built. So, to stop a running web application from moving the index, you need to comment out the call to moveIndex() here. To stop both offline and web applications from moving the index, you therefore have to comment out both calls to moveIndex(). Or perhaps more easily, just make moveIndex() return without doing anything. However, as I mentioned, createIndex has an overloaded version with an additional parameter - a boolean flag that states whether this index is to be used only for testing purposes. If you call createIndex with this flag set to true, then it will ignore search.dir.create (if it has been set), and attempt to create an index in the directory specified by the environment variable java.io.tmpdir. So, you could extend the main method of the application you are using to call createIndex in this fashion. Note that by default, when specifying this is a test index, it will also delete the index directly after building it. If you need the index to remain after the execution of the program, you would need to comment out the appropriate call to deleteDirs() from the createIndex() method. G - Original Message - From: Jose Blanco [EMAIL PROTECTED] To: 'Graham Triggs' [EMAIL PROTECTED] Cc: dspace-tech@lists.sourceforge.net Sent: Wednesday, February 14, 2007 5:41 PM Subject: RE: [Dspace-tech] Reindexing vs Tomcat running Graham: Your new patch completes successfully. Now I would like to make a change to it. I would like it NOT to copy from search.dir.create to search.dir. I want to do this just for testing purposes. So I think in order to do this, I need to change the DSIndexer.java file. I believe if I Comment out return DSIndexer.moveIndex(c, offlineIndexDir, toPath); in the routine moveAlternateIndex, and just return false; That should do it, right? I don't think this will break anything. I simply want to see what goes on (resource being used) when the indexer is running, but I don't want it to replace the active one being used by DSpace. Thanks! Jose -Original Message- From: Graham Triggs [mailto:[EMAIL PROTECTED] Sent: Wednesday, February 14, 2007 5:52 AM To: Jose Blanco Cc: dspace-tech@lists.sourceforge.net Subject: RE: [Dspace-tech] Reindexing vs Tomcat running Jose, I've just uploaded what should be my final revision of this patch. It incorporates a couple of additional small fixes: 1) If somebody is logged into a console and are 'in' the search directory, it no longer stops the index replacement from happening 2) Minor structural improvements to the code. I've now tested just about every scenario that I can think of occurring for which it is possible to fabricate a test case for, on both Linux and Windows platforms, including with multiple Tomcat instances running. As before, let me know how you get on. G On Mon, 2007-02-12 at 17:06 -0500, Jose Blanco wrote: Graham: I installed you patch in my
Re: [Dspace-tech] Reindexing vs Tomcat running
java.io.tmpdir is normally /tmp under unix, and c:\temp under windows. Although it really reflects whatever is set as the OS 'temp' environment variable (if set). You can override java.io.tmpdir by passing -Djava.io.tmpdir= on the command line when starting Java. But yes, in your case, the quickest / easiest way to make the change is to make moveIndex return without doing anything (doesn't really matter if it returns true or false - although if you return false, the code will add a 'dspace-index-complete' file to your newly created index directory). G - Original Message - From: Jose Blanco [EMAIL PROTECTED] To: 'Graham Triggs' [EMAIL PROTECTED] Cc: dspace-tech@lists.sourceforge.net Sent: Wednesday, February 14, 2007 8:14 PM Subject: RE: [Dspace-tech] Reindexing vs Tomcat running Graham: Let me explain what I want to do. I want to leave the web application code unchanged, and so it will be using dspace 1.4.1 without your patch. But I want to update the code so that I can run filter-media from [dspace]/bin using your code. So I could either: 1. have moveIndex return false, and then I can run filter-media and see what happens to our server. Presently when filter-media goes to do the indexing, it seems to make our web instance of DSpace not run. I need to investigate this to find out what is going on. OR 2.I can change the code so that createIndex passes a 'true' indicating to use the dir specified in java.io.tmpdir. Now I'm not sure how/where to set this env variable, so I'm leaning towards option 1. Seems like the safest. Does that make sense? I'm being very cautious about this because I don't want to mess up our installation. Thanks for all your help with this! Jose -Original Message- From: Graham Triggs [mailto:[EMAIL PROTECTED] Sent: Wednesday, February 14, 2007 2:22 PM To: Jose Blanco Cc: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Reindexing vs Tomcat running Jose, That depends on what you want to do the test for - and what platform you are running under. When running offline, the createIndex method (actually, the new overloaded version that takes a second parameter - more on that later), directly calls moveIndex(). So, to stop an offline indexer from moving the directory, you need to comment that call out (the offline indexer can only move the index if the destination isn't locked - ie. you are not running under Windows, or you don't have an IndexSearcher open). moveAlternateIndex() is actually used by DSQuery, and is the method by which the web application moves the index from search.dir.create. This method actually has the additional check for the existence of a 'dspace-index-complete' file, which is used to stop an already running web application from attempting to move the new index before it has finished being built. So, to stop a running web application from moving the index, you need to comment out the call to moveIndex() here. To stop both offline and web applications from moving the index, you therefore have to comment out both calls to moveIndex(). Or perhaps more easily, just make moveIndex() return without doing anything. However, as I mentioned, createIndex has an overloaded version with an additional parameter - a boolean flag that states whether this index is to be used only for testing purposes. If you call createIndex with this flag set to true, then it will ignore search.dir.create (if it has been set), and attempt to create an index in the directory specified by the environment variable java.io.tmpdir. So, you could extend the main method of the application you are using to call createIndex in this fashion. Note that by default, when specifying this is a test index, it will also delete the index directly after building it. If you need the index to remain after the execution of the program, you would need to comment out the appropriate call to deleteDirs() from the createIndex() method. G - Original Message - From: Jose Blanco [EMAIL PROTECTED] To: 'Graham Triggs' [EMAIL PROTECTED] Cc: dspace-tech@lists.sourceforge.net Sent: Wednesday, February 14, 2007 5:41 PM Subject: RE: [Dspace-tech] Reindexing vs Tomcat running Graham: Your new patch completes successfully. Now I would like to make a change to it. I would like it NOT to copy from search.dir.create to search.dir. I want to do this just for testing purposes. So I think in order to do this, I need to change the DSIndexer.java file. I believe if I Comment out return DSIndexer.moveIndex(c, offlineIndexDir, toPath); in the routine moveAlternateIndex, and just return false; That should do it, right? I don't think this will break anything. I simply want to see what goes on (resource being used) when the indexer is running, but I don't want it to replace the active one being used by DSpace. Thanks! Jose -Original Message- From
Re: [Dspace-tech] Reindexing vs Tomcat running
Hi,
What revision of the patch did you apply (the first one, or revised)?
And what OS are you using?
There are a couple of possibilities - something may be locked, or you
simply may not be able to rename the 'search.dir.create' to
'search.dir' (for example, going across drives).
I did include quite a few log messages - errors and warnings - are you
logging those levels?
That said, I'm just testing a further revision of the patch, that
combines some of Mark's code from #1613251. Specifically, it can create
a temporary directory in ${java.io.tmpdir} for test purposes or when
'search.dir.create'. Also, it makes use of the 'copy' code for when
simple renaming doesn't work.
Finally, I've also taken note of Mark's reason for using a temporary
directory - to ensure that the existing index isn't disabled until the
new index has finished generating - and ensured that the web application
can't attempt to swap in an 'offline' index before it is fully generated
(the current patch can attempt to do this too early).
I should have this tested very soon, and hopefully that will resolve the
situation you are finding.
G
On Mon, 2007-02-12 at 17:06 -0500, Jose Blanco wrote:
Graham:
I installed you patch in my dev area, and I see that search.dir.create has
the new indexes but it did not seem to copy it over to the search.dir area.
I don't see anything in the log file about failing to copy it over. I'm
assuming that I have something open as you indicate in the caveats section
of your patch, but would I get some sort of message on the log file?
Thanks!
Jose
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of
[EMAIL PROTECTED]
Sent: Thursday, February 08, 2007 5:29 PM
To: dspace-tech@lists.sourceforge.net
Subject: Re: [Dspace-tech] Reindexing vs Tomcat running
Stephane,
I've now submitted a patch for this reindexing issue: #1655583
You can read my notes and download the patch file here:
https://sourceforge.net/tracker/?func=detailatid=319984aid=1655583group_i
d=19984
With the patch applied, you will also need to add a 'search.dir.create'
entry to your dspace.cfg as per the comments.
Let me know if you have any questions.
G
--
Graham Triggs
Technical Architect
Open Repository
+44 (0)20 7631 9942
This email has been scanned by Postini.
For more information please visit http://www.postini.com
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job
easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
-
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642
___
DSpace-tech mailing list
DSpace-tech@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspace-tech
Re: [Dspace-tech] Reindexing vs Tomcat running
Interesting question. Bear in mind, that I don't believe we ever run index-all - we do run filter-media every day, and if Tomcat is left running that will almost certainly trash the index (any open IndexSearcher when it tries to recreate the index from scratch will cause the index to be trashed - search and ingest operations will then fail, although the instance is still alive and will serve, for example, item requests). So, we've never really been in a position to see the issues that you are describing. However, if you take a look at the (unpatched) code for DSQuery, specifically in the method getSearcher(), you will notice that whenever an update index is detected, then a new IndexSearcher is opened on it - but any old IndexSearcher object that might be referenced by the field searcher is never cleanly shutdown (the close() method is not called). That means it is up to the garbage collector to come along and clean those objects up - the timing and indeed success of that isn't ever guaranteed. There may be other contributing factors to your problems (I haven't looked at much code beyond the patch I created), but it is a distinct possibility that this is playing a part. With the patch that I supplied, even without specifying the new 'search.dir.create' configuration property (only necessary if you suffer the problems that I describe at the top), the way getSearcher() has been rewritten means that it will always call close() on an old IndexSearcher before it discards the reference. So the chances are you will see better resource / memory behaviour. G -Original Message- From: Jose Blanco [mailto:[EMAIL PROTECTED] Sent: Fri 2/9/2007 9:25 PM To: Graham Triggs; dspace-tech@lists.sourceforge.net Subject: RE: [Dspace-tech] Reindexing vs Tomcat running Graham: What I've noticed over here is that when indexing is running, say via index-all, there is an escalating use of memory and our DSpace instance goes down. In other words, when the routine indexAllItems in DSIndexer gets called. And this is considering, I have a routine that is suppose to clear the cache for each item being index that I got from the list. This problem seems to be more prevalent in 1.4.1 than it was in 1.4 (atually I don't remember seeing it in 1.4). So my question to you is, do you think your patch will help me with this, and have you seen this? Thanks! Jose -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Thursday, February 08, 2007 5:29 PM To: dspace-tech@lists.sourceforge.net Subject: Re: [Dspace-tech] Reindexing vs Tomcat running Stephane, I've now submitted a patch for this reindexing issue: #1655583 You can read my notes and download the patch file here: https://sourceforge.net/tracker/?func=detailatid=319984aid=1655583group_i d=19984 With the patch applied, you will also need to add a 'search.dir.create' entry to your dspace.cfg as per the comments. Let me know if you have any questions. G -- Graham Triggs Technical Architect Open Repository +44 (0)20 7631 9942 This email has been scanned by Postini. For more information please visit http://www.postini.com - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech This email has been scanned by Postini. For more information please visit http://www.postini.com - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier. Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642___ DSpace-tech mailing list DSpace-tech@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspace-tech
