restore snapshot not working

2014-06-09 Thread Hermanto Phang
Hi All, I try to restore my backup snapshot using command curl -XPOST localhost:9200/_snapshot/my_backup/snapshot_1/_restore But it was come up error {error:SnapshotRestoreException[[my_backup:snapshot_1] cannot restore index [cobbler_api] because it's open],status:500} Any input to fix

Elasticsearch stemmer issue

2014-06-09 Thread Александр Шаманов
Hello everyone, I have follow index mapping: raw curl -XPUT 'http://localhost:9200/some_content/' -d ' { settings:{ query_string:{ default_con:content, default_operator:AND }, index:{ analysis:{ analyzer:{ en_analyser:{

Re: Queries, filters and match_all

2014-06-09 Thread Ivan Brusic
Does it matter? From what I can tell, some code was added in 0.90 https://github.com/elasticsearch/elasticsearch/commit/6687ecb038b55416d4bb37d29746e86f2624f06b https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/index/query/FilteredQueryParser.java#L54 --

Re: Queries, filters and match_all

2014-06-09 Thread Arkadiy Zabazhanov
Yeah, it does matter because previously I had to add match_all to empty queries if I was need to get all the results in case of empty query. It seems currently any empty query replaces with match_all (this follows from your links), so I don't need to use it explicitly anymore. Am I correct? I

Re: restore snapshot not working

2014-06-09 Thread David Pilato
Try to close the index first: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-open-close.html#indices-open-close -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 9 juin 2014 à 08:19, Hermanto Phang hermyph...@gmail.com a écrit : Hi All, I

Re: restore snapshot not working

2014-06-09 Thread Mark Walkom
Take a look at http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-open-close.html Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 9 June 2014 16:19, Hermanto Phang hermyph...@gmail.com

Re: Re-creating ES Index

2014-06-09 Thread David Pilato
Use aliases. So you have an alias which goes to oldindex. Reindex in newindex. When done, switch the alias. You can read old index using scan and scroll feature and reindex in the new index as soon as you kept the _source field. -- David ;-) Twitter : @dadoonet / @elasticsearchfr /

Re: Re-creating ES Index

2014-06-09 Thread karthik jayanthi
Awesome. Thanks for the prompt reply. Thanks, Karthik On Mon, Jun 9, 2014 at 1:25 PM, David Pilato da...@pilato.fr wrote: Use aliases. So you have an alias which goes to oldindex. Reindex in newindex. When done, switch the alias. You can read old index using scan and scroll feature

Re: Sort nested documents in search result

2014-06-09 Thread David Pilato
Have a look at nested sort in docs: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-sort.html#_nested_sorting_example -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 9 juin 2014 à 00:12, Zdenek Pizl zdenek.p...@gmail.com a écrit :

JDBC river: trouble getting analyzer in type mapping to be applied

2014-06-09 Thread André Morais
Hello! I'm creating an index with a JDBC river. In my type mapping I specify an analyzer defined in the YML and working properly. However, it is not applied to the field. When I check its mapping I see that the analyzer specification is not there. If I create a similar index directly on ES

Re: JDBC river: trouble getting analyzer in type mapping to be applied

2014-06-09 Thread joergpra...@gmail.com
There is a bug in the JDBC river introduced recently that prevents it from using type_mapping parameter if there is no index_settings parameter defined. It will be fixed asap A work around might be adding an empty settings parameter like index_settings : {} Jörg On Mon, Jun 9, 2014 at 1:00

Re: ElasticSearch sort by date in order

2014-06-09 Thread sirkubax
When I asked a question, i think I'm close to anwser curl -XPUT localhost:9200/*/_mapping/loglog -d '\ {\ dynamic_templates : [ {\ string_fields : {\ mapping : {\ type : multi_field,\ fields : {\ raw : {\ index :

Re: Elasticsearch and Hadoop Questions

2014-06-09 Thread ES USER
Thanks. So just one final question. From what you said above that means that you can not run ES queries on data in Hadoop over something like a 6 month time range without it having to pull in all that data and index it first. And I am assuming that the opposite is all correct that Hadoop can

restore api vs custom copy script

2014-06-09 Thread Chetana
I have a secondary index storage where I have both snapshot (taken from ES server) and valid lucene index (content is same as snapshot). If ES server index is cleared/deleted though the mapping is retained, could someone tell me which is the better option to restore in terms of time taken: a)

Re: Sort nested documents in search result

2014-06-09 Thread Zdenek Pizl
Hi David, the example is known to me and probably it is not what I am looking for or I am doing something dumb. I'll try to explain it in deeper details. Let's say we have 2 documents within index /sort-test/ with following mapping: *Creating an index and its mapping*: curl -XPUT

Re: ElasticSearch sort by date in order

2014-06-09 Thread sirkubax
I have tried to add refault mapping: curl -XPUT localhost:9200/*/_mapping/_default_ -d ' { _default_ : { dynamic_date_formats : [date_optional_time, -MM-dd HH:mm:ss.SS], dynamic_templates : [ { string_fields : { mapping : { type :

elasticsearch: simple cluster. Can't change nodes. Please, help.

2014-06-09 Thread Dmitriy Kargapolov
I have a simple cluster with 2 nodes: master/data and data. I need to move master data to another server. I tried to change master to data and data to master, also made 3 node master. But always I had one result. Status of cluster was green and I all shards were 0, but in home directory I had

Elasticsearch field mapping, dynamic_templates

2014-06-09 Thread sirkubax
As I did mention here https://groups.google.com/forum/#!topic/elasticsearch/7Bn5Pc6TSgs I have some sorting issue when date is stored in string format. *I did try to set field format as date, but it is overide when date is inserted into index.The mapping was:* curl -XPUT

Re: elasticsearch: simple cluster. Can't change nodes. Please, help.

2014-06-09 Thread sirkubax
What this commnad show curl -XGET 'http://localhost:9200/_nodes/_all/process?pretty=true' |less W dniu poniedziałek, 9 czerwca 2014 16:41:25 UTC+2 użytkownik Dmitriy Kargapolov napisał: I have a simple cluster with 2 nodes: master/data and data. I need to move master data to another server.

Re: Marvel 1.2.0 java.lang.IllegalStateException

2014-06-09 Thread Boaz Leskes
Hi Pawel, We just did a quick minor release to marvel with a fix for this. Would be great if you can give it a try and confirm how it goes. Cheers, Boaz On Friday, June 6, 2014 12:01:52 PM UTC+2, Boaz Leskes wrote: Thx Pawel, Note huge but larger then limit. Working on a fix. On Friday,

Cannot Increase Write TPS in Elasticsearch by adding more nodes

2014-06-09 Thread pranav amin
Hi all, While doing some prototyping in ES using SSD's we got some good Write TPS. But the Write TPS saturated after adding some more nodes! Here are the details i used for prototyping - Requirement: To read data as soon as possible since the read is followed by write. Version of ES:1.0.0

Re: Marvel 1.2.0 java.lang.IllegalStateException

2014-06-09 Thread Paweł Krzaczkowski
Im out of office for today so ill test it tomorrow morning and let You know if it works pawel (at) mobile On 9 cze 2014, at 17:40, Boaz Leskes b.les...@gmail.com wrote: Hi Pawel, We just did a quick minor release to marvel with a fix for this. Would be great if you can give it a try

Re: Queries, filters and match_all

2014-06-09 Thread Ivan Brusic
I personally would not count any any default behaviors and would choose to use match_all explicitly in my calls. Having some filtered queries with no query (default match all) and some with a more detailed query would be inconsistent, so my preference would be to always have a query. It comes

Re: Using Regex in Query via Kibana

2014-06-09 Thread Richard Raseley
Bill, Thank you for taking the time to reply to this old thread. I can confirm that removing the explicit end-anchor causes the query to work! Thanks so much! Regards, Richard On Fri, Jun 6, 2014 at 1:56 PM, Bill Paulson wpauls...@gmail.com wrote: I think the '$' on the end of the regex may

ElasticSearch 1.1.0 is slower compared to ES 0.90.3 ?

2014-06-09 Thread coder
Hi, I was using ES 0.90.3 in production earlier but after moving to ES 1.1.0, it is taking a longer time for a search query to fetch results ? I'm using same mapping and same search query. The only difference is that I'm using jdk 1.6. Should I migrate to 1.7 ? Also, how does sharding and

Re: 'Shard Allocation' dashboard in Marvel 1.2 showing SearchParseExceptions

2014-06-09 Thread philsttr
Hi Boaz, The only other exception I see in the logs is this: [2014-06-09 15:38:11,340][ERROR][marvel.agent ] [] exporter [es_exporter] has thrown an exception: java.lang.IllegalStateException: array not available at

Re: 'Shard Allocation' dashboard in Marvel 1.2 showing SearchParseExceptions

2014-06-09 Thread philsttr
Looks like this has already been found and resolved. I'll upgrade to Marvel 1.2.1. http://www.elasticsearch.org/blog/marvel-1-2-1-released/ On Monday, June 9, 2014 9:35:30 AM UTC-7, phil...@gmail.com wrote: Hi Boaz, The only other exception I see in the logs is this: [2014-06-09

Re: 'Shard Allocation' dashboard in Marvel 1.2 showing SearchParseExceptions

2014-06-09 Thread Boaz Leskes
Yeah, I suspected that. Let me know how it goes... On Mon, Jun 9, 2014 at 6:39 PM, null phils...@gmail.com wrote: Looks like this has already been found and resolved. I'll upgrade to Marvel 1.2.1. http://www.elasticsearch.org/blog/marvel-1-2-1-released/ On Monday, June 9, 2014 9:35:30 AM

Re: Cannot Increase Write TPS in Elasticsearch by adding more nodes

2014-06-09 Thread joergpra...@gmail.com
There are many reasons that may cause this, just to name a few - benchmarking tool setup ( do they show correct numbers?) - network bandwidth limits - cluster setup (e.g. complex mapping, high latency between nodes) - pattern of the data input - method of data input (bulk vs. index, HTTP vs. Java

Re: Scaling Writes/Indexing

2014-06-09 Thread pranav amin
Hi Chris, Did you got better TPS after doing some change. Can you share your experience? If so, which parameter did you tweak to get Linear scaling of ES. We are kind of in the same situation, we aren't able to see better Write TPS by adding more nodes. Thanks in advance for your help.

Re: Marvel 1.2.0 java.lang.IllegalStateException

2014-06-09 Thread Paweł Krzaczkowski
It works .. thx for a quick fix 2014-06-09 17:48 GMT+02:00 Paweł Krzaczkowski pa...@krzaczkowski.pl: Im out of office for today so ill test it tomorrow morning and let You know if it works pawel (at) mobile On 9 cze 2014, at 17:40, Boaz Leskes b.les...@gmail.com wrote: Hi Pawel, We

Re: Elasticsearch and Hadoop Questions

2014-06-09 Thread Georgi Ivanov
Please find my comments below : From what you said above that means that you can not run ES queries on data in Hadoop over something like a 6 month time range without it having to pull in all that data and index it first. - *CORRECT *.* Es queries can run only on ES* And I am assuming that the

Re: Cannot Increase Write TPS in Elasticsearch by adding more nodes

2014-06-09 Thread pranav amin
Thanks Jorg for your help. Do you recommend any tool that can help me to point out the bottlenecks in terms of I/O, Memory, Network, GC, etc? I'm using some tools that are free (like Marvel, Elastic HQ, etc), but am not able to figure out if i'm hitting some limits. Thanks Pranav. -- You

Net Sample Code for Bulk Load

2014-06-09 Thread Madhavan Ramachandran
Hi All, I am new to Elastic Search. Going through the site and looking for solution to implement in .Net. Found that NEST dlls provides capabilities to interact with ES. Need help on bulk load data from SQL Server to elastic search for indexing the documents. Looking for the .net solution to

Re: UnavailableShardsException after loading 1.5M documents

2014-06-09 Thread Chris Mildebrandt
Well, I hit the same error again after 6.7M documents and the increased heap size. After a restart of the cluster, it's taking a very long time to bring up the replica shards. Until I have about 50% of my shards (primary + replica) initialized, I'm unable to load any more documents. When I

Re: Cannot Increase Write TPS in Elasticsearch by adding more nodes

2014-06-09 Thread joergpra...@gmail.com
How do you try to figure out you're hitting limits? I have not enough information to help. Marvel, Elastic HQ, etc. are all very useful tools but should be combined with OS-related monitoring to get an overall picture. Jörg On Mon, Jun 9, 2014 at 9:31 PM, pranav amin parulpate...@gmail.com

Re: ElasticSearch sort by date in order

2014-06-09 Thread sirkubax
*I guess I nailed it with dynamic_date_formats and extra * dynamic_templates http://www.elasticsearch.org/guide/en/elasticsearch/reference/current//mapping-root-object-type.html#_dynamic_date_formats *like that* curl -XPUT localhost:9200/_mapping/_default_ -d @mappings2 cat mappings2 {

Re: ElasticSearch 1.1.0 is slower compared to ES 0.90.3 ?

2014-06-09 Thread Mark Walkom
You should *not*be using JDK 1.7. Upgrade to at least 1.7u25, preferably 1.7u55, and Oracle not OpenJDK. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 10 June 2014 02:34, coder mukulnit...@gmail.com wrote: Hi,

Re: Cannot Increase Write TPS in Elasticsearch by adding more nodes

2014-06-09 Thread Mark Walkom
One thing you never mentioned was what version of Java you are on, which can impact things as well. To give you some idea, we had a 12 node cluster of VMs with 30GB heap and were seeing 12000 TPS (incoming events), so what you are seeing is very low. Regards, Mark Walkom Infrastructure

shard allocation for large amount of data

2014-06-09 Thread Chen Wang
We have huge amount of data (5Billion records, 3TB in size) organized in parent / child type in one index to enable the joins. My first question is, how should I allocate shards for this big index in order to make the parent/child query more efficient? Right now doing queries will cause out of

Re: shard allocation for large amount of data

2014-06-09 Thread Mark Walkom
You need to add more nodes. Changing shard layout is unlikely to help if you're getting OOM. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 10 June 2014 09:12, Chen Wang chen.apache.s...@gmail.com wrote: We have

Re: Inter-document Queries

2014-06-09 Thread Zennet Wheatcroft
Thank you Itamar and Jörg for your replies. I followed your suggestion Itamar and it works. Queries that took 300+ seconds are now 400 ms again. However, this model increases stored space complexity by O(N^2) which is usually not acceptable. So I would not consider this a general method. It

How to set replication to async for a index

2014-06-09 Thread Jinyuan Zhou
Hi there, I am new to ES. I understand the concept but cannot find documentation about where to put the setting. Thanks, Jack -- You received this message because you are subscribed to the Google Groups elasticsearch group. To unsubscribe from this group and stop receiving emails from it,

Re: How to set replication to async for a index

2014-06-09 Thread Mark Walkom
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html#index-replication will do it Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 10 June 2014 11:01, Jinyuan Zhou

Re: How to set replication to async for a index

2014-06-09 Thread Jinyuan Zhou
Thanks Mark, the page explain what needs to effectively be done but not how to achieve it. I just need an example. is it something like settings { . mapping.allow_type_wrapper:true, ... replication: async ... } in the index's setting? Thanks, Jack On Mon, Jun 9, 2014 at 6:05 PM, Mark

Re: How to set replication to async for a index

2014-06-09 Thread Mark Walkom
Yep that looks fine. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: ma...@campaignmonitor.com web: www.campaignmonitor.com On 10 June 2014 11:17, Jinyuan Zhou zhou.jiny...@gmail.com wrote: Thanks Mark, the page explain what needs to effectively be done but not how to

How to do MoreLikeTHESE in Elastic Search?

2014-06-09 Thread Alain Désilets
I just LOVE the MoreLikThis (MLT) feature of ElasticSearch and am finding new and interesting ways of using it all the time. However these days, I find myself needing a MoreLikeTHESE feature. In other words, given a group of documents that fit together, I want to find other documents that

Re: Queries, filters and match_all

2014-06-09 Thread Arkadiy Zabazhanov
Reasonable approach, I think, I'll use it in my gem for now. But will wait for the answer as well just to bate my curiosity :) понедельник, 9 июня 2014 г., 22:59:04 UTC+7 пользователь Ivan Brusic написал: I personally would not count any any default behaviors and would choose to use

Re: elasticsearch: simple cluster. Can't change nodes. Please, help.

2014-06-09 Thread Dmitriy Kargapolov
This command show: C:\Program Files\Elasticsearch\bincurl -XGET http://localhost:9204/_nodes/_all/process?pretty=true; { ok : true, cluster_name : mycluster, nodes : { RTlR2GvATs-jqYSRQ5c6bg : { name : node4, transport_address : inet[/192.168.0.89:9304], hostname :

Re: ElasticSearch 1.1.0 is slower compared to ES 0.90.3 ?

2014-06-09 Thread coder
Are you sure about jdk 1.7 ? My guess is one should use jdk 1.7 with ES 1.2.0. Also, earlier with ES 0.90.3 it was working fine. How can jdk version affect the search query time ? On Tuesday, 10 June 2014 03:56:53 UTC+5:30, Mark Walkom wrote: You should *not*be using JDK 1.7. Upgrade to at

Re: ElasticSearch 1.1.0 is slower compared to ES 0.90.3 ?

2014-06-09 Thread Mark Walkom
The means less than, ie you should not be using less than 1.7. There are a lot of bug and performance fixes in later versions of java, that will impact the performance and stability of any application that uses it. Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: