Hi Ludovic,
Sorry for the multiple mails :(. The reason I have CC a copy to you is,
somewhere in GSOC discussions , it is suggested to keep all the
discussions only in mailing list and better CC it to the mentor.
Any ways... coming to the project,
Yes, performance might be an issuse when the indexes are over multiple
servers. Having index on single machine is much better but only when index
size is small.
But the problem is, as the number of servers in the cluster
increase and also the data on each XWiki server increases, it would take
lot of time to copy the index from each children node to main server and
also maintaining it would become a problem.
Also if indexes are on different machines, remote indexes can be
searched in parallel(using an api of lucene, remote searching can be done
with RMI). Parallel search becomes extremly helpful as, the size index
keeps on growing.
Ofcourse each method has it own pros and cons, but there performance varies as
the size of index varies.
I am thinking of testing those two methods in terms of response time, indexing
some data like english wikipedia which
about 2.3 million documents. Is this data enough ???
Another doubt I have in mind is,
"""have a way to make the children server search pages in reality call the
main server search."""
What if main server is down ??
---If the index is only on one main server then searching is not possible
until the server is up.
---Even if the indexes are distributed, searching is not possible as
children servers are supposed to call main search server
Instead what I am thinking is, the main server maintains information about
children servers; and passes that information to each of its children, as
and when the information about the children servers gets updated in the
main server. Now each server has information about other servers and each
server can "act" as main server while searching. Therefore even if one
server is down, search can be performed on remaining indexes(a benefit of
having multiple indexes).
Any comments on this ??
--Sai Krishna.
On Tue, 25 Mar 2008, Ludovic Dubost wrote:
>
> Hi,
>
> Don't be surprised you don't get an immediate reply.. Mailing list are
> asynchronous and it's on purpose.
> Also it does not help to add my email in CC one or multiple times, since
> I'm subscribed to the dev and users mailing list, and since the email is
> send to both the list and directly to me it still ends up in my mailing
> list folder !
>
> Concerning the distributed search,
>
> Yes, one method I have in mind is to register children servers to the
> main server and have a way to make the children server search pages in
> reality call the main server search. With the registration, the main
> server would get authorization to come and either take the index or get
> updates so that it can build a global index.
>
> It could be possible to have an index on each server, but this would
> make merging results and counting results more complex. I'm not sure
> also of the performance.
>
> Both methods could be studied and/or implemented.
>
> Concerning the document type things, it's not important. We can explain
> at implementation time.
>
> Ludovic
>
> Sai Krishna wrote:
>> Hi Asiri,
>>
>> Atlast I got a reply :).
>> Yes I have tried IRC earlier, but I didn't get any reply, also very few
>> people very on #xwiki channel, so thought it would better to mail the
>> mailing list.
>>
>> -Sai Krishna.
>>
>> On Tue, 25 Mar 2008, Asiri Rathnayake wrote:
>>
>>
>>> HI saikrishna,
>>>
>>> May be you should try the IRC channel ?
>>>
>>> server : irc.freenode.org
>>>
>>> channel : #xwiki
>>>
>>> - Asiri
>>>
>>> On Tue, Mar 25, 2008 at 12:19 AM, <[EMAIL PROTECTED]> wrote:
>>>
>>>
>>>> Hi all,
>>>>
>>>> Please excuse me if you have received this mail before, there seems to be
>>>> some problem with my gmail account and I am not sure whether the mail I
>>>> sent earlier was sent to XWiki devs list or not, so sending it again.
>>>>
>>>> please help me.
>>>>
>>>> """Support multiple installations of XWiki being indexed on one or
>>>> multiple XWiki Servers and share the index.
>>>> It should be possible to share easily the index so that multiple XWiki
>>>> installations can be shared in one UI."""
>>>> ---> I am assuming that one of the several servers, is the main server
>>>> and has information about other servers like the ip address. -- am I
>>>> correct ?
>>>> ---> Can each server maintain it's own index(instead of distributed
>>>> indexing), but while searching, indexes on multiple servers are used.
>>>> ---> please provide me some more info regarding this.
>>>>
>>>> """Also the search UI should be possible to customize depending on
>>>> Document types."""
>>>> ---> Restricting search to specified document type.
>>>>
>>>>
>>>> If any one has other relevant information, please provide it.
>>>>
>>>> -- Sai Krishna.
>>>>
>>>>
>>>> _______________________________________________
>>>> devs mailing list
>>>> devs@xwiki.org
>>>> http://lists.xwiki.org/mailman/listinfo/devs
>>>>
>>>>
>>> _______________________________________________
>>> devs mailing list
>>> devs@xwiki.org
>>> http://lists.xwiki.org/mailman/listinfo/devs
>>>
>>>
>>>
>>
>>
>
>
>
--
*******************************************************************************
*******************************************************************************
Success makes life easy but not living easy
**
Remember Things can be difficult but not impossible
_______________________________________________
devs mailing list
devs@xwiki.org
http://lists.xwiki.org/mailman/listinfo/devs
