On Wed, 16 Jan 2008, anurag uniyal wrote:
In my application I need to create several analyzers on the fly, which
leads to outOfMemory problem. see code below
Indeed, you found a leak of local refs to jstring objects when converting a
Python array into a java array. This is fixed in svn rev 380.
Thanks !
Andi..
----
import lucene
lucene.initVM(lucene.CLASSPATH, maxheap='1m')
stopwords = ["and", "if", "at", "the"]
for i in xrange(100000):
if i%1000==0: print i
analyzer = lucene.StandardAnalyzer(stopwords)
#analyzer = lucene.StandardAnalyzer() # works without stopwords
----
I hope analyzer should be freed?
It works if I do not pass any stopwords to StandardAnalyzer
rgds
Anurag
----- Original Message ----
From: Andi Vajda <[EMAIL PROTECTED]>
To: [email protected]
Sent: Monday, 14 January, 2008 11:06:54 PM
Subject: Re: [pylucene-dev] detachCurrentThread crash
On Mon, 14 Jan 2008, anurag uniyal wrote:
Is it expected to explicity remove any references to local/class variables
in threads?
If you create threads faster than Python and Java can collect the objects
you leave lying around, then yes.
For instance, closing the store you're opening in every thread instead of
None'ing it out may have the same effect.
Andi..
Otherwise VM crashes. e.g.
-----------
import threading
import lucene
lucene.initVM(lucene.CLASSPATH)
def threadFunc():
lucene.getVMEnv().attachCurrentThread()
_store = lucene.FSDirectory.getDirectory("/tmp/index/", True)
#_store = None # set to None to avoid crash!
lucene.getVMEnv().detachCurrentThread()
def main():
t = threading.Thread(target=threadFunc)
t.start()
t.join()
main()
----
I need to call detachCurrentThread because I use threads heavily andI will soon
get out-of-memory error if didn't call detachCurrentThread.
rgds
Anurag
Save all your chat conversations. Find them online at
http://in.messenger.yahoo.com/webmessengerpromo.php
_______________________________________________
pylucene-dev mailing list
[email protected]
http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
Forgot the famous last words? Access your message archive online at
http://in.messenger.yahoo.com/webmessengerpromo.php
_______________________________________________
pylucene-dev mailing list
[email protected]
http://lists.osafoundation.org/mailman/listinfo/pylucene-dev
