Re: Customizing Directory and IndexWriter behavior via custom ES plug-in

2014-10-28 Thread Ákos Kitta
Awesome. Thanks a lot for the help. I'll give a try.

On Monday, October 27, 2014 2:30:51 PM UTC+1, Jörg Prante wrote:
>
> I stand corrected - there is also the possibility of the setting 
> "index.store.type", by setting this to a Java class name, you can use this 
> as the current index store implementation from a plugin.
>
> So, no patching/forking required.
>
> Jörg
>
> On Mon, Oct 27, 2014 at 2:26 PM, joerg...@gmail.com  <
> joerg...@gmail.com > wrote:
>
>> Regarding the deletion policy, you can set the class name of your 
>> deletion policy implementation in the setting "index.deletionpolicy.type"
>>
>> For custom Directory, you have to 
>> patch org.elasticsearch.index.store.IndexStoreModule with your custom index 
>> store. The index store is something like an IndexWriter / Lucene Directory 
>> on steroids. At the moment, it is not possible to add custom index stores 
>> from a plugin (see the fixed enumeration of implementations 
>> in IndexStoreModule)
>>
>> Jörg
>>
>> On Mon, Oct 27, 2014 at 1:22 PM, Ákos Kitta > > wrote:
>>
>>> Hi there,
>>>
>>> in the last couple of years we managed to customize Apache Lucene 
>>> (through its public API) to support branching, tagging and compare in a 
>>> concurrent fashion for our server application. We managed to achieve this 
>>> by using a couple of custom Directory, exactly one IndexDeletionPolicy and 
>>> one MergePolicy implementations. Currently we are considering to 
>>> replace Lucene with Elasticsearch on the server-side. Before we jumped into 
>>> the details of collecting the differences between the two technologies in 
>>> respect of the search and indexing functionality and for instance how to 
>>> port our custom collectors and how to replace NDVs, we would like to make 
>>> sure if it is possible at all.
>>>
>>> I've just checked out the source and realized that the registration of 
>>> the services  are done via various module implementations and the actual 
>>> configured service implementations are injected into the constructors. For 
>>> the sake of simplicity is there a way for example to create an 
>>> Elasticsearch module which forces the underlying IndexWriter to use the 
>>> FooCustomDeletionPolicy 
>>> instead of the default KeepOnlyLastDeletionPolicy? I assume if this is 
>>> straightforward we could use or custom implementations for the directory 
>>> and the IndexWriter what we are currently using with pure Lucene. After 
>>> doing some research I found this [1] thread. Am I close to the 
>>> answer/solution?
>>>
>>> I have to notice we would like to achieve this without forking the 
>>> public repository.
>>>
>>> Thanks in advance for the feedbacks.
>>>
>>> Cheers,
>>> Akos
>>>
>>> [1]: https://groups.google.com/forum/#!topic/elasticsearch/rFaLnI5FRf4
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to elasticsearc...@googlegroups.com .
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/elasticsearch/0345efea-3134-488d-b13d-199a24642422%40googlegroups.com
>>>  
>>> <https://groups.google.com/d/msgid/elasticsearch/0345efea-3134-488d-b13d-199a24642422%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/5f6310e5-dc98-4a54-9c2c-ce39cecab2b6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Customizing Directory and IndexWriter behavior via custom ES plug-in

2014-10-27 Thread Ákos Kitta
Hi there,

in the last couple of years we managed to customize Apache Lucene (through 
its public API) to support branching, tagging and compare in a concurrent 
fashion for our server application. We managed to achieve this by using a 
couple of custom Directory, exactly one IndexDeletionPolicy and one MergePolicy 
implementations. Currently we are considering to replace Lucene with 
Elasticsearch on the server-side. Before we jumped into the details of 
collecting the differences between the two technologies in respect of the 
search and indexing functionality and for instance how to port our custom 
collectors and how to replace NDVs, we would like to make sure if it is 
possible at all.

I've just checked out the source and realized that the registration of the 
services  are done via various module implementations and the actual 
configured service implementations are injected into the constructors. For 
the sake of simplicity is there a way for example to create an 
Elasticsearch module which forces the underlying IndexWriter to use the 
FooCustomDeletionPolicy 
instead of the default KeepOnlyLastDeletionPolicy? I assume if this is 
straightforward we could use or custom implementations for the directory 
and the IndexWriter what we are currently using with pure Lucene. After 
doing some research I found this [1] thread. Am I close to the 
answer/solution?

I have to notice we would like to achieve this without forking the public 
repository.

Thanks in advance for the feedbacks.

Cheers,
Akos

[1]: https://groups.google.com/forum/#!topic/elasticsearch/rFaLnI5FRf4

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/0345efea-3134-488d-b13d-199a24642422%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.