Hadoop is not designed for this type of scenario.
Have a look at Solr (http://lucene.apache.org/solr), this is pretty
much one of it's main use cases. I think it will do what you need to
do and will more than likely work w/ a minimal of configuration on
your existing index (but don't hold me to that statement).
Also, have you done profiling on your application such that you are
sure moving Lucene off the machine is going to help that much?
Cheers,
Grant
ps, the mailing lists strips attachments.
On Jun 28, 2007, at 10:19 AM, Samuel LEMOINE wrote:
Chun Wei Ho a écrit :
Hi,
We are currently running a Tomcat web application serving searches
over our Lucene index (10GB) on a single server machine (Dual 3GHz
CPU, 4GB RAM). Due to performance issues and to scale up to handle
more traffic/search requests, we are getting another server machine.
We are looking at two ways of scaling:
(1) duplicating the web application and index on the second machine
and load-balancing incoming users between the two servers.
(2) modifying our web application so that one machine will host our
web application (and associated MySQL database), while the other one
will host the Lucene index. The first machine would be dedicated to
our web application and database, while the second becomes our
dedicated Lucene search server. When users perform a search on the
website, the web application will send the request to the Lucene
index
server, which will perform the search and return the results to the
web application.
We would like comments from users who have set up similar systems on
how you have accomplished (1) in your setups, and whether (2) is a
good choice for scaling.
Attached is a more complete RTF document outlining our architecture
and proposal. We appreciate your perusal and comments.
Regards,
CW
---------------------------------------------------------------------
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
I'm acutely interrested by this issue too, as I'm working on
distributed architecture of Lucene. I'm only at the very beginning
of my study so that I can't help you much, but Hadoop maybe could
fit to your requirements. It's a sub-project of Lucene aiming to
parallelise Lucene.
See http://lucene.apache.org/hadoop/about.html but I don't know
wether it scales well to very small clusters...
About your attached, I couldn't access it, it was only a "Partie
1.2" file containing this text:
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Cordially,
Samuel
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------
Grant Ingersoll
Center for Natural Language Processing
http://www.cnlp.org/tech/lucene.asp
Read the Lucene Java FAQ at http://wiki.apache.org/lucene-java/LuceneFAQ
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
