Re: Elasticsearch and cassandra integration?
Hi Utkarsh I want to backup old data from ElasticSearch, options are Cassandra and Hadoop. I want to know which one is better in terms of integration, scalability and performance. In cassandra, only do we need to install plugin or there are other pieces of code that we may need to write. On Thursday, October 16, 2014 9:07:06 PM UTC+5:30, Utkarsh Sengar wrote: I am not actively working on the elsaticsearch cassandra river now, but always open to pull requests! :) https://github.com/ebay/cassandra-river I found another fork of the project: https://github.com/srecon/elasticsearch-cassandra-river -Utkarsh On Thu, Oct 16, 2014 at 7:10 AM, José Guilherme Vanz guilhe...@gmail.com javascript: wrote: Hi, Utkarsh Are you still working on the cassandra river? Thanks Vanz On Monday, March 25, 2013 10:46:50 PM UTC-3, Utkarsh Sengar wrote: Thanks for the answer! I was able to write a simple river for cassandra while pulls data periodically (similar to couchdb's river). Which leads to some questions: 1. I saw that EsExecutors exists but there is no implementation of ScheduledExecutorService. So, is there any reason why EsExecutor is implemented other than having a custom name and priority? Can I use ScheduledExecutorService inside a river without any performance issues? 2. What I am doing for now is, I have 1 thread which wakes up every x hours and moves all the data from cassandra to ES, everytime. Its not very performant if the data is alot (will add some kind of batching of records). So wanted to know, are there some standard practices while throwing data to ES? The implementation is just 1 day old, very raw. I will put it up on github soon! I loved the simple APIs and it was very east to get started with (except lack of documentation, but reference implementations helped)! Thanks, -Utkarsh On Sat, Mar 23, 2013 at 2:31 AM, Jörg Prante joerg...@gmail.com wrote: 1. I use IntelliJ (previously Netbeans) and mvn on command line but Eclipse TestNG use is documented here: http://testng.org/doc/eclipse. html 2. Debugging running plugins works like debugging a running ES node. Beside extensive logging I use tools like jvisualvm to analyze runtime behaviour. 3. I think it is best to start from an existing river as boilerplate code. It helps to examine the river sources documented at http://www.elasticsearch.org/guide/reference/modules/plugins.html Jörg Am 23.03.13 04:56, schrieb Utkarsh Sengar: I agree with you. I am also inclined towards implementing a plugin due to lack of elastic search and cassandra integration. I have been looking at the jdbc and rss river and it surely helps to understand the anatomy of an ES river. Although I have some questions about elastic search plugin development: 1. These plugins have some nicely written tests whose test suits are defined in xml files under test/resources. How can I debug these tests via eclipse? 2. Say I have a working prototype of the plugin and I manually install it in my local elastic search instance by placing the plugin project in the plugins folder. What is the best way to debug the plugin in ES, except logging the output of-course. 3. Documentation about plugin development lacks but the sample rss river code helps. Can I safely assume that I can use rss river as a boildeplate project for cassandra river right? Or is there a way to create a plugin project for ES? Any pointers from you about ES plugin development will help :) -- You received this message because you are subscribed to a topic in the Google Groups elasticsearch group. To unsubscribe from this topic, visit https://groups.google.com/d/to pic/elasticsearch/9TJFiWr1oUQ/unsubscribe?hl=en-US. To unsubscribe from this group and all its topics, send an email to elasticsearc...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- Thanks, -Utkarsh -- You received this message because you are subscribed to a topic in the Google Groups elasticsearch group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/9TJFiWr1oUQ/unsubscribe. To unsubscribe from this group and all its topics, send an email to elasticsearc...@googlegroups.com javascript:. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2a8718cd-22b1-4b42-a938-e771b877fe6c%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/2a8718cd-22b1-4b42-a938-e771b877fe6c%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- Thanks, -Utkarsh -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit
Re: Elasticsearch and cassandra integration?
Hi, Utkarsh Are you still working on the cassandra river? Thanks Vanz On Monday, March 25, 2013 10:46:50 PM UTC-3, Utkarsh Sengar wrote: Thanks for the answer! I was able to write a simple river for cassandra while pulls data periodically (similar to couchdb's river). Which leads to some questions: 1. I saw that EsExecutors exists but there is no implementation of ScheduledExecutorService. So, is there any reason why EsExecutor is implemented other than having a custom name and priority? Can I use ScheduledExecutorService inside a river without any performance issues? 2. What I am doing for now is, I have 1 thread which wakes up every x hours and moves all the data from cassandra to ES, everytime. Its not very performant if the data is alot (will add some kind of batching of records). So wanted to know, are there some standard practices while throwing data to ES? The implementation is just 1 day old, very raw. I will put it up on github soon! I loved the simple APIs and it was very east to get started with (except lack of documentation, but reference implementations helped)! Thanks, -Utkarsh On Sat, Mar 23, 2013 at 2:31 AM, Jörg Prante joerg...@gmail.com javascript: wrote: 1. I use IntelliJ (previously Netbeans) and mvn on command line but Eclipse TestNG use is documented here: http://testng.org/doc/eclipse.html 2. Debugging running plugins works like debugging a running ES node. Beside extensive logging I use tools like jvisualvm to analyze runtime behaviour. 3. I think it is best to start from an existing river as boilerplate code. It helps to examine the river sources documented at http://www.elasticsearch.org/guide/reference/modules/plugins.html Jörg Am 23.03.13 04:56, schrieb Utkarsh Sengar: I agree with you. I am also inclined towards implementing a plugin due to lack of elastic search and cassandra integration. I have been looking at the jdbc and rss river and it surely helps to understand the anatomy of an ES river. Although I have some questions about elastic search plugin development: 1. These plugins have some nicely written tests whose test suits are defined in xml files under test/resources. How can I debug these tests via eclipse? 2. Say I have a working prototype of the plugin and I manually install it in my local elastic search instance by placing the plugin project in the plugins folder. What is the best way to debug the plugin in ES, except logging the output of-course. 3. Documentation about plugin development lacks but the sample rss river code helps. Can I safely assume that I can use rss river as a boildeplate project for cassandra river right? Or is there a way to create a plugin project for ES? Any pointers from you about ES plugin development will help :) -- You received this message because you are subscribed to a topic in the Google Groups elasticsearch group. To unsubscribe from this topic, visit https://groups.google.com/d/ topic/elasticsearch/9TJFiWr1oUQ/unsubscribe?hl=en-US. To unsubscribe from this group and all its topics, send an email to elasticsearc...@googlegroups.com javascript:. For more options, visit https://groups.google.com/groups/opt_out. -- Thanks, -Utkarsh -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2a8718cd-22b1-4b42-a938-e771b877fe6c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: Elasticsearch and cassandra integration?
I am not actively working on the elsaticsearch cassandra river now, but always open to pull requests! :) https://github.com/ebay/cassandra-river I found another fork of the project: https://github.com/srecon/elasticsearch-cassandra-river -Utkarsh On Thu, Oct 16, 2014 at 7:10 AM, José Guilherme Vanz guilherme@gmail.com wrote: Hi, Utkarsh Are you still working on the cassandra river? Thanks Vanz On Monday, March 25, 2013 10:46:50 PM UTC-3, Utkarsh Sengar wrote: Thanks for the answer! I was able to write a simple river for cassandra while pulls data periodically (similar to couchdb's river). Which leads to some questions: 1. I saw that EsExecutors exists but there is no implementation of ScheduledExecutorService. So, is there any reason why EsExecutor is implemented other than having a custom name and priority? Can I use ScheduledExecutorService inside a river without any performance issues? 2. What I am doing for now is, I have 1 thread which wakes up every x hours and moves all the data from cassandra to ES, everytime. Its not very performant if the data is alot (will add some kind of batching of records). So wanted to know, are there some standard practices while throwing data to ES? The implementation is just 1 day old, very raw. I will put it up on github soon! I loved the simple APIs and it was very east to get started with (except lack of documentation, but reference implementations helped)! Thanks, -Utkarsh On Sat, Mar 23, 2013 at 2:31 AM, Jörg Prante joerg...@gmail.com wrote: 1. I use IntelliJ (previously Netbeans) and mvn on command line but Eclipse TestNG use is documented here: http://testng.org/doc/eclipse. html 2. Debugging running plugins works like debugging a running ES node. Beside extensive logging I use tools like jvisualvm to analyze runtime behaviour. 3. I think it is best to start from an existing river as boilerplate code. It helps to examine the river sources documented at http://www.elasticsearch.org/guide/reference/modules/plugins.html Jörg Am 23.03.13 04:56, schrieb Utkarsh Sengar: I agree with you. I am also inclined towards implementing a plugin due to lack of elastic search and cassandra integration. I have been looking at the jdbc and rss river and it surely helps to understand the anatomy of an ES river. Although I have some questions about elastic search plugin development: 1. These plugins have some nicely written tests whose test suits are defined in xml files under test/resources. How can I debug these tests via eclipse? 2. Say I have a working prototype of the plugin and I manually install it in my local elastic search instance by placing the plugin project in the plugins folder. What is the best way to debug the plugin in ES, except logging the output of-course. 3. Documentation about plugin development lacks but the sample rss river code helps. Can I safely assume that I can use rss river as a boildeplate project for cassandra river right? Or is there a way to create a plugin project for ES? Any pointers from you about ES plugin development will help :) -- You received this message because you are subscribed to a topic in the Google Groups elasticsearch group. To unsubscribe from this topic, visit https://groups.google.com/d/to pic/elasticsearch/9TJFiWr1oUQ/unsubscribe?hl=en-US. To unsubscribe from this group and all its topics, send an email to elasticsearc...@googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out. -- Thanks, -Utkarsh -- You received this message because you are subscribed to a topic in the Google Groups elasticsearch group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/9TJFiWr1oUQ/unsubscribe. To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2a8718cd-22b1-4b42-a938-e771b877fe6c%40googlegroups.com https://groups.google.com/d/msgid/elasticsearch/2a8718cd-22b1-4b42-a938-e771b877fe6c%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- Thanks, -Utkarsh -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CADjjot_O9543Gnz3r%2BftFP-4-xaCKZE9ZjAWezt5JgD0B3imqg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
