how do lucene read large index files?

[Kumaran Ramasubramanian]

Hi All,

how do lucene read large index files?
for example, if one file (for eg: .dat file) is 4GB.
lucene read only part of file to RAM? or
is it different approach for different lucene file formats?


Related Link:
How do applications (and OS) handle very big files?
http://superuser.com/a/361201


--
Kumaran R

Re: how do lucene read large index files?

[Erick Erickson]

see Uwe's blog:
http://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html

Short form: files are read into the OS's memory as needed. the whole
file isn't read at once.

Best,
Erick

On Wed, Nov 23, 2016 at 12:04 AM, Kumaran Ramasubramanian
 wrote:
> Hi All,
>
> how do lucene read large index files?
> for example, if one file (for eg: .dat file) is 4GB.
> lucene read only part of file to RAM? or
> is it different approach for different lucene file formats?
>
>
> Related Link:
> How do applications (and OS) handle very big files?
> http://superuser.com/a/361201
>
>
> --
> Kumaran R

-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org

Re: Understanding Query Parser Behavior

[Peru Redmi]

Could someone elaborate this.

On Tue, Nov 22, 2016 at 11:41 AM, Peru Redmi  wrote:

> Hello,
> Can you help me out on your "No" .
>
> On Mon, Nov 21, 2016 at 11:16 PM, wmartin...@gmail.com <
> wmartin...@gmail.com> wrote:
>
>> No
>>
>> Sent from my LG G4, an AT&T 4G LTE smartphone
>>
>> -- Original message--
>> *From: *Peru Redmi
>> *Date: *Mon, Nov 21, 2016 10:44 AM
>> *To: *java-user@lucene.apache.org;
>> *Cc: *
>> *Subject:*Understanding Query Parser Behavior
>>
>> Hello All ,Could someone explain *QueryParser* behavior on these cases1. 
>> While Indexing ,Document doc = new Document();doc.add(new Field("*Field*", 
>> "*http://www.google.com*";, Field.Store.YES, Field.Index.ANALYZED));  
>> index has *two* terms - *http* & *www.google.com**2.* While searching 
>> ,Analyzer anal = new *ClassicAnalyzer*(Version.LUCENE_30, 
>> newStringReader(""));QueryParser parser=new 
>> *MultiFieldQueryParser*(Version.LUCENE_30, 
>> newString[]{"*Field*"},anal);Query query = 
>> parser.parse("*http://www.google.com *");Now , query has *three *terms  -  
>> (Field:http) *(Field://)* (Field:www.google.com)i) Why I have got 3 terms 
>> while parsing , and 2 terms on indexing (Usingsame ClassicAnalyzer in both 
>> cases ) ?ii) is this expected behavior of ClassicAnalyzer(Version.LUCENE_30) 
>> onParser ?iii) what should be done to avoid query part *(Field://) 
>> *?Thanks,Peru.
>>
>>
>