Re: Memory question

2012-05-15 Thread Sean Bridges
Another solution is to set -Xmx == -Xms, and make a native call to mlockall on application startup, this will prevent the jvm from swapping. Using JNA, calling mlockall is not very hard, you can see how cassandra does it here, https://github.com/apache/cassandra/blob/cassandra-1.1.0/src/java/org/

RE: Memory question

2012-05-15 Thread Lutz Fechner
CMS is the concurrent mark sweep garbage collector. Instead of waiting for the memory to fill up and being collected for memory to be freed up again, it runs concurrently while the app threads are running. Usually the JVM would call a full stop the world collection. All threads would be on hold unt

Re: Memory question

2012-05-15 Thread Chris Bamford
Hi John, Very interesting, thanks for the detailed explanation. It certainly sounds like the same symptoms! Can I please clarify a couple of things ? - I have googled CMS / iCMS as I wasn't familiar with those acronyms (apart from 'code management system' and that didn't sound right!) an

Re: Memory question

2012-05-15 Thread Nader, John P
We've encountered this issue and came up with a fairly good approach to address it. We are on Lucene 3.0.2 with Java 1.6.0_29. Our indices are about 35GB in size. Our JVM runs at 20GB of heap, with about 12GB of steady usage. Our server has 32GB total. What would happen in our case is that the

Re: RE: Memory question

2012-05-15 Thread Chris Bamford
Thanks Uwe. What I'd like to understand is the implications of this on a server which opens a large number of indexes over a long period. Will this non-heap memory continue to grow? Will gc be effective at spotting it and releasing it via references in the heap? I had an instance yesterday w

RE: Memory question

2012-05-15 Thread Uwe Schindler
It mmaps the files into virtual memory if it runs on a 64 bit JVM. Because of that you see the mmapped CFS files. This is outside Java Heap and is all *virtual* no RAM is explicitely occupied except the O/S cache. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMai

Re: Memory question

2012-05-15 Thread Ian Lea
In versions from 3.3 onwards MMapDirectory is the default on 64-bit linux. Not sure exactly what that means wrt your questions, but may well be relevant. -- Ian. On Tue, May 15, 2012 at 3:51 PM, Lutz Fechner wrote: > Hi, > > > By design memory outside the JVM heap space should not be accessib

RE: Memory question

2012-05-15 Thread Lutz Fechner
Hi, By design memory outside the JVM heap space should not be accessible for java applications. Why you might see is the disc cache of the Linux storage subsystem. Best Regards Lutz -Original Message- From: Chris Bamford [mailto:chris.bamf...@talktalk.net] Sent: Dienstag, 15. Mai 20

Memory question

2012-05-15 Thread Chris Bamford
Hi Can anyone tell me what happens to the memory when Lucene opens an index? Is it loaded into the JVM's heap or is it mapped into virtual memory outside of it? I am running on Linux and if I use pmap on the PID of my JVM, I can see lots of entries for index cfs files. Does this mean that inde

Re: Wildcards in field name

2012-05-15 Thread Ian Lea
No and no. MultiFieldQueryParser is the only thing that comes to mind as being remotely close but you have to tell it the field names. I guess you could use IndexReader.getFieldNames(...) to find indexed fields and pass the output from that through a wildcard regexp and feed the output from that

Wildcards in field name

2012-05-15 Thread Joachim Andersson
Is it possible todo a query with wildcards in field name? is there any QueryParser that handles wildcards in fields names? //Joachim -- View this message in context: http://lucene.472066.n3.nabble.com/Wildcards-in-field-name-tp3983865.html Sent from the Lucene - Java Users mailing list archive

Re: new feature in lucene3.6

2012-05-15 Thread S Eslamian
I got it. Thank you. On Tue, May 15, 2012 at 2:31 PM, Uwe Schindler wrote: > > Hello > > I have a question about this new feature in lucene 3.6 : > > " The QueryParser now interprets * as an open end for range queries. > > Literal asterisks may be represented by quoting or escaping (i.e. \* or >

Lucene + Spring - Indexer service configuration

2012-05-15 Thread Antonio Fernández
Hi all , greetings from the very south . I just ant to know if you can tell me some good tutorial on how to configure a lucene indexer in a Spring Container for an indexing process a number of files at every boot. After a little google search I found several resources, I understand that usi

Re: new feature in lucene3.6

2012-05-15 Thread Uwe Schindler
> Hello > I have a question about this new feature in lucene 3.6 : > " The QueryParser now interprets * as an open end for range queries. > Literal asterisks may be represented by quoting or escaping (i.e. \* or > "*") Custom QueryParser subclasses overriding getRangeQuery() will be > passed null f

new feature in lucene3.6

2012-05-15 Thread S Eslamian
Hello I have a question about this new feature in lucene 3.6 : " The QueryParser now interprets * as an open end for range queries. Literal asterisks may be represented by quoting or escaping (i.e. \* or "*") Custom QueryParser subclasses overriding getRangeQuery() will be passed null for any open