Re: What is the best file system for Lucene?

2004-11-30 Thread Sanyi
Thanx for the replies to you all. I was looking for someone with the same experiences as mine ones, but it seems that I'll have to test this myself. I'll try out my ideas and the most interesting ideas from you guys. Regards, Sanyi __ Do you Yah

Re: What is the best file system for Lucene?

2004-11-30 Thread Otis Gospodnetic
Hello, > Lucene indexing completes in 13-15 hours on the desktop system while > it completes in about 29-33 > hours on the notebook. > > Now, combine it with the DROP INDEX tests completing in the same > amount of time on both and find > out why is the search only slightly faster :) > > > Until

RE: What is the best file system for Lucene?

2004-11-30 Thread Armbrust, Daniel C.
As I understand hyperthreading, this is not true: >Also, unless you take your hyperthreading off, with just one index you are >searching with just one half of the CPU - so your desktop is actually using >a 1.5GHz CPU for the search. You still have the full speed of the processor available - the

RE: What is the best file system for Lucene?

2004-11-30 Thread Armbrust, Daniel C.
You may want to give the IBM JVM a try - I've found it faster in some cases... http://www-106.ibm.com/developerworks/java/jdk/linux140/ Dan - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PR

Re: What is the best file system for Lucene?

2004-11-30 Thread Sanyi
> simply load your index into a > RAMDirectory instead of using FSDirectory. I have 3GByte RAM and my index is 3GByte big currently. (it'll be soon about 4GByte) So, I have to find out this another way. > First off, 1.8GHz Pentium-M machines are supposed to run at about the > speed of a 2.4GHz

Re: What is the best file system for Lucene?

2004-11-30 Thread Sanyi
> How large is the index? If it's less than a couple of GByte then it > will be entirely in memory It is 3GBytes big and it will grow a lot. I have to search from the HDD which is very fast compared to the notebook's HDD. Average seek time: Notebook: 8-9ms Desktop: 3.9ms Data read: Notebook:

Re: What is the best file system for Lucene?

2004-11-30 Thread Justin Swanhart
As a generalisation, SuSE itself is not a lot slower than Windows XP. I also very much doubt that filesystem is a factor. If you want to test w/out filesystem involvement, simply load your index into a RAMDirectory instead of using FSDirectory. That precludes filesystem overhead in searches. Th

Re: What is the best file system for Lucene?

2004-11-30 Thread Sanyi
> Could you try XP on your desktop Sure, but I'll only do that I run out of ideas. > so your desktop is actually using > a 1.5GHz CPU for the search. No, this is not true. It uses a 3.0GHz P4 then. (HT means that you have two 3.0GHz P4s) So, it is still surprising to me. Regards, Sanyi

Re: What is the best file system for Lucene?

2004-11-30 Thread Justin Swanhart
On Tue, 30 Nov 2004 12:07:46 -, Pete Lewis <[EMAIL PROTECTED]> wrote: > Also, unless you take your hyperthreading off, with just one index you are > searching with just one half of the CPU - so your desktop is actually using > a 1.5GHz CPU for the search. So, taking account of this its not too

Re: What is the best file system for Lucene?

2004-11-30 Thread John Haxby
Sanyi wrote: I'm testing Lucene 1.4.2 on two very different configs, but with the same index. I'm very surprised by the results: Both systems are searching at about the same speed, but I'd expect (and I really need) to run Lucene a lot faster on my stronger config. Config #1 (a notebook): WinXP P

Re: What is the best file system for Lucene?

2004-11-30 Thread Pete Lewis
; <[EMAIL PROTECTED]> Sent: Tuesday, November 30, 2004 11:28 AM Subject: Re: What is the best file system for Lucene? > > Interesting, what are your merge settings > > Sorry, I didn't mention that I was talking about search performance. > I'm using the same, fully optimi

Re: What is the best file system for Lucene?

2004-11-30 Thread sg
> What file systems are you people using Lucene on? And what are your > experiences? http://www.apple.com/xsan/ Actually it is a beta version and have some small issues but it is very fast and easy to manage in case you get it installed. The installation it self is tricky since it is very depen

Re: What is the best file system for Lucene?

2004-11-30 Thread Sanyi
> Interesting, what are your merge settings Sorry, I didn't mention that I was talking about search performance. I'm using the same, fully optimized index on both systems. (I've generated both indexes with the same code from the same database on the actual OS) > which JDK are you using? I'm usi

Re: What is the best file system for Lucene?

2004-11-30 Thread John Moylan
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Interesting, what are your merge settings, which JDK are you using?(there are big differences between versions). Have you tried with hyperthreading turned off on #2? - if so did it fare any differently? Regards, John Sanyi wrote: | Hi! | | I'm testing L