Re: DateRange aggregation semantics - include_lower/include_upper?

[mooky]

Oops. I copy-pasted the aggregation I was hacking about with.
The actual aggregation looks like this (I had added an additional 
millisecond as a hack to include_upper)

"aggregations" : {
  "intentDate" : {
"date_range" : {
  "field" : "intentDate",
  "ranges" : [ {
"key" : "Overdue",
"to" : "2014-02-17"
  }, {
"key" : "March",
"from" : "2014-02-18",
"to" : "2014-03-17"
  }, {
"key" : "April",
"from" : "2014-03-18",
"to" : "2014-04-15"
  } ]
}
  }
}







On Monday, 24 February 2014 18:16:07 UTC, mooky wrote:
>
> What are the expected semantics of the from/to fields in a DateRange 
> aggregation?
>
> Are the from/to values included? Should there be an *include_lower*/
> *include_upper* option like with filters?
>
> I want the aggregation to include the lower and upper values - but I 
> discovered today that it doesnt (I was getting a mismatch between the 
> aggregation & the filter with gte/lte values).
>
> My aggregation was defined like this:
>
>
> "aggregations" : {
>   "intentDate" : {
> "date_range" : {
>   "field" : "intentDate",
>   "ranges" : [ {
> "key" : "Overdue",
> "to" : "2014-02-17"
>   }, {
> "key" : "March",
> "from" : "2014-02-18",
> "to" : "2014-03-17"
>   }, {
> "key" : "April",
> "from" : "2014-03-18",
> "to" : "2014-04-15T00:00:00.001Z"
>   } ]
> }
>   }
> }
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/cd6a4ad2-12ee-4137-aa35-d313fc6a2872%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Indexing on customized fields.

[san]

Snippet 
from 
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping.html

"Explicit mapping is defined on an index/type level. By default, there 
isn’t a need to define an explicit mapping, since one is automatically 
created and registered when a new type or new field is introduced (with no 
performance overhead) and have sensible defaults."

Following is the json format of the log Logstash is generating:

{
"message" => "<6> Jan  9 07:19:26 w2k8r233110 
0|TEST|TESTPRODUCT|8.0.1310|TestSignature|This is test 
message|Medium|src=10.31.252.102",
   "@version" => "1",
 "@timestamp" => "2014-02-25T06:52:52.930Z",
   "type" => "syslog",
   "host" => [
[0] "127.0.0.1:38989",
[1] "w2k8r233110"
],
 "syslog_pri" => "6",
  "timestamp" => "Jan  9 07:19:26",
"cef_version" => "0",
  "device_vendor" => "TEST",
 "device_product" => "TESTPRODUCT",
 "device_version" => "8.0.1310",
   "signature_id" => "TestSignature",
"message_content" => "This is test message",
   "severity" => "Medium",
 "src_ip" => "10.31.252.102"
}

But i can't see the fields in Elasticsearch Kibana interface. Could anyone 
please help me out?





-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/6438c2fa-aacb-45cf-a19b-196c6aea6c3a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

upgrade to elasticsearch 1.0 now ClassCastException: class ElasticSearch090PostingsFormat

[Kevin J. Smith]

Hi,

I am using elasticsearch embedded in a tomcat 7 webapp container 
(everything running under java 7.) All libs for elasticsearch are in 
WEB-INF/lib. In v0.90 everything is running swimmingly. We upgraded to v1.0 
(libs and all and paid attention to breaking API calls) but now on Ubuntu 
Linux when I make a call to create an index via the following call:

final CreateIndexResponse response = 
_client.admin().indices().prepareCreate(index).setSource(mapping).execute().actionGet();

I get the following exception:

org.elasticsearch.common.util.concurrent.UncategorizedExecutionException: 
Failed execution
at 
org.elasticsearch.action.support.AdapterActionFuture.rethrowExecutionException(AdapterActionFuture.java:90)
at 
org.elasticsearch.action.support.AdapterActionFuture.actionGet(AdapterActionFuture.java:50)
at com.bitstew.search.SearchNode.createIndex(SearchNode.java:1507)
at 
com.bitstew.search.SystemInit.loadIndexDefinition(SystemInit.java:206)
at com.bitstew.search.SystemInit.loadIndex(SystemInit.java:81)
at com.bitstew.search.SystemInit.loadIndices(SystemInit.java:52)
at 
com.bitstew.ws.servlet.SystemAction.loadIndices(SystemAction.java:1798)
at 
com.bitstew.ws.servlet.SystemAction.executeAction(SystemAction.java:383)
at 
com.bitstew.ws.servlet.WebServicesDeployer.service(WebServicesDeployer.java:1888)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at 
org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)
at 
org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
at 
org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
at 
org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:381)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at 
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at 
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
at 
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023)
at 
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at 
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.NoClassDefFoundError: 
org/apache/lucene/codecs/PostingsFormat
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at 
org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1701)
at 
org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1559)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at 
org.apache.lucene.util.SPIClassIterator.next(SPIClassIterator.java:137)
at org.apache.lucene.util.NamedSPILoader.reload(NamedSPILoader.java:65)
at org.apache.lucene.util.NamedSPILoader.(NamedSP

the document payload of the Delete api

[Ivan Ji]

Hi all,

I am wondering whether the delete API support to return the content of the 
deleted documents because I need to know the content of the deleted 
documents.

Of course, I can get the document first and then delete it with two REST 
API. But are there any api to achieve it?

any suggestions?

Best,

Ivan


-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/b290ad84-5870-4651-a30a-c8848cf2f062%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Highlighting doesn't work on numeric and date fields

[Preeti Jain]

Thanks for your response.
I have to use numeric and date fields in range queries so the string field 
of mutli-field will not work.
Any other thought?

On Friday, February 21, 2014 6:44:09 PM UTC+5:30, Binh Ly wrote:
>
> You can do a multi-field on numeric fields with a string/not_analyzed 
> field and then highlight on that.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/349bc1ca-12bf-4f1c-a95f-785e0b245217%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: can't order by _boost field, even when "index":"not_analyzed"

[JoeZ99]

Ok, .

Good to know, I wish I knew this a few days before ;-) I was really loosing 
my mind on this!!

yet another reason for dropping indexing time defined boost, I guess. I 
really wish there were any way of defining per-document boost at index 
time. 

txs!

On Monday, February 24, 2014 5:14:48 PM UTC-5, Binh Ly wrote:
>
> Yup, this is a known bug. Since _boost is being deprecated and replaced by 
> function_score, this will likely not be fixed. For now if you want to sort 
> on a "boost" value, either remove the _boost from your mapping, or 
> introduce another field that you don't refer to from _boost.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/b1bd446d-8fc9-4d58-84aa-f86f6d3a43e1%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Book] "Mastering ElasticSearch" Review

[Nick Wood]

I read Elasticsearch Server several months ago and found it helpful.  But 
I'm hesitant to get any more books that aren't focused on 1.x - hopefully 
we'll see some pop up soon (nudge nudge).

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/5f02e712-b071-4cd8-9bfd-1585a1c6e4ca%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Question : Field behavior in Nested object

[Jun Ohtani]

Thanks for replying.

I open a ticket.
https://github.com/elasticsearch/elasticsearch/issues/5245



2014-02-25 8:34 GMT+09:00 Binh Ly :

> It does look like a bug. Can you please open a ticket/issue? Thank you.
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/3ab3b9de-05ae-4b5b-be36-2993d85ba9fc%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>



-- 
---
Jun Ohtani
blog : http://blog.johtani.info

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAPW8A5yd-SqitmZt-O8%3DOE2QnbgE-TJXGZ0ZZe%2BGk%2BthSdqdPg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: boosting fields based on type in multi-index searches

[seallison]

That works perfectly, Thanks! I had no idea you could preface the field 
paths with the type for boosting like that.

On Monday, February 24, 2014 7:08:31 PM UTC-5, Binh Ly wrote:
>
> If I understand you correctly, let's say you have one or more indexes. 
> Then you have 2 types named "product" and "store". "product" and "store" 
> both have the field "name", but you want to boost the product "name" 
> independent from the store "name". You should be able to do something like 
> this:
>
> {
>   "query": {
> "query_string": {
>   "query": "blah",
>   "fields": [
> "product.name^1.3",
> "store.name^1.7"
>   ]
> }
>   }
> }
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/b5860adf-2f96-430a-9b08-1f7064de29bb%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Guide how to deploy ES in a high availability configuration

[Tehmasp Chaudhri]

thanks. nice list of ES HA reminders.

On Wednesday, December 12, 2012 3:22:24 AM UTC-7, Karel Minařík wrote:
>
> Hello,
>
> first, there's a great presentation from Shay on the topic available at <
> http://www.elasticsearch.org/videos/2011/08/09/road-to-a-distributed-searchengine-berlinbuzzwords.html
> >.
>
> Couple of quick notes:
>
> * Elasticsearch is "highly available" by design and by default. As you 
> write, shards are replicated across the cluster for better performance and 
> availability. If a primary shard is not available (node goes down, etc), 
> one replica will take over the role of the primary.
>
> * Elasticsearch clusters, again by default, "take care of themselves". 
> Either with multicast or properly configured unicast topologies, a new node 
> with a correct `cluster.name` settings will join the cluster, and will 
> start serving queries, storing shards, etc.
>
> * Elasticsearch clusters don't have a SPOF master node: any node which is 
> configured to do so (which is a default) can become a master. Thus, if a 
> master goes down, another node will take over the role.
>
> * Given you have enough nodes to place replicas on, your cluster is highly 
> available by default. When a node goes down, unless you're running at 
> capacity, you just launch another one, and it will take on the duties, 
> without human intervention — apart from monitoring.
>
> * Your HTTP requests are automatically re-directed across the nodes in the 
> cluster. If you use a single IP in your application code, though, that is a 
> point of failure if that specific machine would go down.  There are 
> multiple approaches how to solve it. Some libraries automatically perform 
> requests in a round-robin fashion and detect healthy nodes. Another 
> approach is to launch an elasticsearch "client node" which would serve as a 
> proxy, automatically discovering new nodes in the cluster, etc. You could 
> also use Nginx or HAproxy in a similar fashion, making sure you 
> periodically update the list of nodes in its configuration.
>
> * EC2 elastic IPs are not suited well for the purpose, since you are 
> billed the traffic and it's using the external network channel (unless I'm 
> mistaken or the rules changed). EC load balancer (ELB) has the same 
> characteristics, unfortunately, in addition to being overloaded/not 
> resilient enough at times. A dedicated Nginx/HAproxy/Pound/etc proxy 
> machine is a much better solution from the architectural point of view.
>
>
> Karel
>
> On Tuesday, December 11, 2012 12:39:39 PM UTC+1, Juho Mäkinen wrote:
>>
>> We're looking on deploying Elastic Search into EC2 to power the search of 
>> our new product. After crawling elasticsearch.org for tutorials, I found 
>> the EC2 tutorial, which was quite helpful, but I didn't find any guides how 
>> I should implement high availability for elastic search.
>>
>> All our current production systems are configured and deployed so that 
>> any machine can fail and all traffic, requests etc are directed to the 
>> working nodes and a possible failover procedures have also been fully 
>> automated. This results in high availability without any human intervention 
>> (the techops team does monitoring, but they don't actually need to do much, 
>> because the high amount of automation)
>>
>> Now I'm thinking how I can do all this to Elastic Search, but I don't 
>> have any good pointers. Now I know that elastic search is powered by lucene 
>> and the indices are distributed into shards, which each have one master and 
>> one or more replicas. What if a master fails? I found some info about 
>> master election and also about the EC2 discovery module at 
>> http://www.elasticsearch.org/guide/reference/modules/discovery/ but that 
>> doesn't really tell me what happens when the master dies. Is there 
>> additional documentation?
>>
>> What about the RESTful endpoint? What if the machine running this dies? 
>> We are running haproxy in our production environment, so I could very well 
>> use that in front, but I couldn't find any good guides on that topic 
>> either. Do I have to configure some scripts which changes DNS CNAMEs to the 
>> new master, or can/should I use EC2 elastic IP addresses? 
>>
>> All responses and links to relevant tutorials are greatly appreciated.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/5330d434-4bcd-488c-9aea-53029fe59644%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

dynamic template multi-field w/ _all combo analyzer issue?

[asanderson]

I'm seeing duplicate concatenated values when using the combo analyzer for 
_all using a multi-field defined in a dynamic template.

e.g. Instead of seeing "Foo Bar" when listing the _all terms aggregation, 
I'm seeing "Foo Bar Foo Bar" for the token because my mulit-field defines 2 
sub-fields. If the multi-field is defined with 4 sub-fields, then "Foo Bar" 
is concatenated 4 times.

My set up is below.

*Elasticsearch 1.0.0 on CentOs 6.4 with Java 1.7.0_51.*

*$ES_HOME/config/default-mapping.json:*
{
   "_default_": {
  "_all": {
 "enabled": true,
 "analyzer": "combo",
 "store": false
  },
  "dynamic_templates": {
 "string_multifield_template": {
"match": "*",
"match_mapping_type": "string",
"mapping": {
   "include_in_all": false,
   "fields": {
  "{name}": {
 "index": "not_analyzed",
 "store": true,
 "type": "string"
  },
  "lowercase": {
 "analyzer": "lowercase",
 "index": "analyzed",
 "store": false,
 "type": "string"
  }
   }
}
 }
  }
   }
}

*$ES_HOME/config/elasticsearch.yml:*
...
index.analysis.analyzer.lowercase.type:custom
index.analysis.analyzer.lowercase.tokenizer: keyword
index.analysis.analyzer.lowercase.filter [ lowercase ]

index.analysis.analyzer.combo.type:   custom
index.analysis.analyzer.combo.sub_analyzers:[ keyword, lowercase ]
index.analysis.analyzer.combo.deduplication:  true
index.analysis.analyzer.combo.tokenstream_reuse: false
...

The aggregation query I use is the following:
{
   "aggs": {
  "_all": {
 "terms": {
"field": "_all"
 }
  }
   }
}


Thoughts?

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/f9344d45-33e6-45eb-b193-fdf1d24ebc1b%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: boosting fields based on type in multi-index searches

[Binh Ly]

If I understand you correctly, let's say you have one or more indexes. Then 
you have 2 types named "product" and "store". "product" and "store" both 
have the field "name", but you want to boost the product "name" independent 
from the store "name". You should be able to do something like this:

{
  "query": {
"query_string": {
  "query": "blah",
  "fields": [
"product.name^1.3",
"store.name^1.7"
  ]
}
  }
}

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/0fd7db3f-c691-4ed2-8251-5897dd93b881%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Question : Field behavior in Nested object

[Binh Ly]

It does look like a bug. Can you please open a ticket/issue? Thank you.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/3ab3b9de-05ae-4b5b-be36-2993d85ba9fc%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

boosting fields based on type in multi-index searches

[seallison]

I have two indices each storing a specific type "Products" and "Stores." 
 Some of the attribute names of each type overlap.  For instance, both 
Products and Stores have a "name" attribute.  How can I search across both 
indices while giving different boost values to the same attribute?  I want 
to give Products.name a boost of 1.3 and Stores.name a boost of 1.7.  I 
don't see a way to do this with the query string query.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/914d2bd2-84ec-4acc-9064-a66b9f94647d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Configure elasticsearch to store only index data

[Binh Ly]

Some ideas:

1) You can turn dynamic mapping to false and then explicitly specify only a 
handful of fields that will be indexed/searchable. Or, if you don't want to 
do this, just send in a smaller JSON document with only the fields you want 
searched or indexed.

2) RAM is mostly dependent on the types of searches. If you do a lot of 
cached filters, sorting, facets/aggregation, script field access, 
parent-child, that's where most of the RAM will be used. Otherwise, if 
you're just doing standard full-text searches and maybe some occasional 
filtering, you'll probably require "lesser" RAM. The best way to determine 
this is to test your queries (different query types) on a single node and 
monitor RAM usage. You'll want to run the node stats checking for the 
filter, fielddata, and id caches for usage.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/85307551-1041-4fd4-9b45-d18211adf614%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Search time field extraction

[Binh Ly]

Technically, you can probably do this with a little scripting and the 
script_fields functionality:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-script-fields.html

However, Kibana does not expose this at the moment so for now, you'll need 
to produce the exact field values at index time. And yes it will not apply 
to already indexed data as you discovered - you will need to delete those 
old indexes and then re-run the logs again using your new LS config file if 
you want the values from the old logs.

Btw, incase you've not seen this, there is a nifty grok debugger here 
(makes grokking and testing a lot quicker):

http://grokdebug.herokuapp.com/

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/8f8ba448-6330-4b4f-9410-14454845b6b7%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Book] "Mastering ElasticSearch" Review

[Alberto Paro]

Hello everyone,
in this weekend I’ve had the pleasure to read “Mastering ElasticSearch” 
(http://www.packtpub.com/mastering-elasticsearch-querying-and-data-handling/book)
 of Rafał Kuć. Everyone, who is in Lucene/Solr/ElasticSearch ecosystem, already 
knows him also for his blog and the famous series "Solr vs ElasticSearch" 
(http://blog.sematext.com/author/kucrafal/).
I write here my opinion on this book.

Using massively ElasticSearch in production brings the users to "hit the wall" 
of a lot of problems such as performance issues, shard allocation issues, 
quality issues (choosing the best algorithm for your documents) and generally 
limits of ElasticSearch. Most of these topics are described in the book with 
depth explanations and solutions: they are sum up of years of experience on 
these topics (even before ElasticSearch exists).

The book is target to advanced users that:
- manage huge datasets in ElasticSearch: the importance of shard allocation on 
a big cluster;
- need to improve the relevance of their results;
- need to speed up some indices;
- need to extend ElasticSearch with plugins: river, analyzers, custom facets, 
and so on (this is the real nature of ElasticSearch: it can be expanded to 
cover all possible scenarios)

Otherwise ElasticSearch newcomers should not buy this book due to required 
skills needed to understand it. (I suggest them to read my "ElasticSearch 
Cookbook” ;-) or "ElasticSearch Server").

I hope that my review could help someone.

Best regards until the next book review.
Alberto Paro

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/A09A2705-AAE7-4FDD-91BC-91F0860E3A5D%40gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Kibana: showing a ratio

[Binh Ly]

Unfortunately not at the moment. But if you're up to it, you can probably 
easily write a custom panel that will do this for you.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/48ac720f-da68-44c3-906b-c458f9353571%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: can't order by _boost field, even when "index":"not_analyzed"

[Binh Ly]

Yup, this is a known bug. Since _boost is being deprecated and replaced by 
function_score, this will likely not be fixed. For now if you want to sort 
on a "boost" value, either remove the _boost from your mapping, or 
introduce another field that you don't refer to from _boost.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/026c733c-9077-434d-b92d-9d8dacbf9f50%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Visualizing range facet using Kibana

[Binh Ly]

Unfortunately not at the moment out of the box. But, if you have some time, 
you certainly can build a Kibana panel yourself and call the range 
facet/aggregation and draw it yourself.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/cb3caf93-befe-4137-81a5-5753a3c3c6a1%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: TransportClient accessing Cluster via single port

[Hariharan Vadivelu]

you define your custom ports for additional nodes by 
setting transport.tcp.port and http.port in elasticsearch.yml
and accordingly punch firewall rules only for those ports.

On Monday, February 24, 2014 8:56:31 AM UTC-6, FFA wrote:
>
> Thank you. But what if multiple nodes have to be started to balance the 
> load?
>
> Regards,
>
> FFA
>
> On Friday, February 21, 2014 8:50:19 PM UTC-6, David Pilato wrote:
>>
>> Just set transport port in elasticsearch.yml.
>> But you can stay with defaults. As long as you start only one node, only 
>> 9300 needs to be opened.
>>
>> --
>> David ;-)
>> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>>
>>
>> Le 21 févr. 2014 à 23:10, FFA  a écrit :
>>
>>  Multinode ES cluster is behind the firewall. I am using Java API 
>> TransportClient to query the ES.
>>
>> I have a simple question:
>>  Do I have to open all ports 9300-9400 on firewall to be able to access 
>> ES with TC? Is there ES settings that would allow me to open only let's say 
>> port 9300 on firewall and reliably get the query results back?
>>
>> Thanks,
>>
>> Fred.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/043d3612-e774-4bff-b6b9-1162f12b2b42%40googlegroups.com
>> .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/2084b089-9f49-490d-8326-a50d161e42bf%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Visualizing range facet using Kibana

[vineeth mohan]

Hi ,

I have a field whose value is between 0 and 1.
I need to draw a bar graph which tells how many feeds have value of this
field between (0,.1) , (.1,.2) , (.2,.3) , (.3,.4) and so on.

Can i do this using Kibana.

Thanks
  Vineeth

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGdPd5m8B5WR27Y-xurc38EqexwkTVdB0gTUH0%3Di5yeh8PeAVQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Elasticsearch 1.0.0 is now GA

[InquiringMind]

I am not sure what the complaints are all about.

Over the past 20 years, my best practices are to treat the installed 
configurations as a template that is subject to change upon reinstallation. 
Then, I always create my own configuration and point the server to it, and 
never point a server to the package's installed configuration.

And then, I maintain all of my customized configurations separately from 
the installed packages.

Pointing to the installed configuration that you've modified is really no 
different that running the installed jars that you have modified.Would you 
really expect a reinstallation of Elasticsearch to preserve the changes you 
have made to the originally installed elasticsearch-1.0.0.jar file?

The beauty of Elasticsearch's configurations are that they document 
everything but actually set nothing. That's even better than the 
configurations for the servers I write in which I set everything but to the 
default values in the code. Same end result; different means of getting 
there. In fact, the installed config is a big part of the package's 
documentation about what is available to be configured. So I would expected 
it to change on each installation.

And for the turn-key servers I developed in the past where the configs were 
not maintained by Puppet or Chef or some other automated tool, I would 
write a post installation step that would copy the installed config over a 
taret config, but only if that target config did not exist. That way, the 
customer could modify the target config and their changes would be 
preserved. But today, our elasticsearch.yml file and other server configs 
are maintained by Puppet and because we don't touch the installed config we 
never have any problems with overwriting on a reinstallation.

Brian

On Monday, February 17, 2014 5:14:46 PM UTC-5, Tony Su wrote:
>
> What?!
>  
> Removing and re-installing the ES package either removes the original or 
>  the existing elasticsearch.yml
>  
> The is contrary to conventional packaging from what I've generally seen.
> Typically, when a package is removed, the configuration fie is left alone 
> and must be removed manually if desired
>  
> No big deal in my case, I've been working on elasticsearch.yml heavily for 
> several days so can remember all the customizations I've made, but IMO this 
> is a disaster waiting to happen for clusters with new Admins or those who 
> attempt to fix a problem by removing and re-installing.
>  
> Leaving the config file alone and re-using is the  option.
>  
> IMO,
> Tony
>
>  

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/2e56fc3f-5de6-474e-9923-ecf6f231754e%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: cluster in yellow for a week right now, several disk without space

[damian noseda]

version of the elastic

{
  "ok" : true,
  "status" : 200,
  "name" : "e-43ea",
  "version" : {
"number" : "0.90.7",
"build_hash" : "36897d07dadcb70886db7f149e645ed3d44eb5f2",
"build_timestamp" : "2013-11-13T12:06:54Z",
"build_snapshot" : false,
"lucene_version" : "4.5.1"
  },
  "tagline" : "You Know, for Search"
}


On Monday, February 24, 2014 5:04:01 PM UTC-3, damian noseda wrote:
>
> I add more info:
>
>  I have the plugin head with refresh quickly and I see that some replicas 
> are constatly jumping between nodes. Does it make any sense? for instance, 
> in one of the nodes I see the replica of shard 1 in yellow, then replica of 
> 2, then no shard nor replica at all, then the replica in yellow of the 9 
> shard. And the disk space is is turning to %80. I checked and all the nodes 
> have the same master. I also tried the rerouting api without success given 
> that I can't move shards that are not unassigned. any pointer can help. 
> thanks in advance
>
> On Monday, February 24, 2014 11:29:59 AM UTC-3, damian noseda wrote:
>>
>> Hello:
>>
>> I have a cluster in yellow with 27 instances 10 shards and 2 replicas.
>>
>> size: 56.2G (129G)
>> docs: 158.935.862 (159.132.956)
>>
>> I have the plugin head with refresh quickly and the replicas are jumping 
>> from instance to instance. Always trying in instances with no diskspace.
>>
>> All the instances have 60g and have some with 360g
>>
>> How can I resolve this?
>>
>> Any pointer would help
>>
>> I'm thinking in shuting down the instances with full disk. Delete the 
>> content of path.data and restart the instances againg and hope maybe the 
>> shards to stop jumping all the time.
>>
>> What do you think?
>>
>>  
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/661fc948-f591-4ade-99bf-cae752f02722%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: cluster in yellow for a week right now, several disk without space

[damian noseda]

I add more info:

 I have the plugin head with refresh quickly and I see that some replicas 
are constatly jumping between nodes. Does it make any sense? for instance, 
in one of the nodes I see the replica of shard 1 in yellow, then replica of 
2, then no shard nor replica at all, then the replica in yellow of the 9 
shard. And the disk space is is turning to %80. I checked and all the nodes 
have the same master. I also tried the rerouting api without success given 
that I can't move shards that are not unassigned. any pointer can help. 
thanks in advance

On Monday, February 24, 2014 11:29:59 AM UTC-3, damian noseda wrote:
>
> Hello:
>
> I have a cluster in yellow with 27 instances 10 shards and 2 replicas.
>
> size: 56.2G (129G)
> docs: 158.935.862 (159.132.956)
>
> I have the plugin head with refresh quickly and the replicas are jumping 
> from instance to instance. Always trying in instances with no diskspace.
>
> All the instances have 60g and have some with 360g
>
> How can I resolve this?
>
> Any pointer would help
>
> I'm thinking in shuting down the instances with full disk. Delete the 
> content of path.data and restart the instances againg and hope maybe the 
> shards to stop jumping all the time.
>
> What do you think?
>
>  
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/9e5adc16-b084-4e70-9a3a-34617b0ab9c8%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Regex not working for strings containing special characters

[Jamil Bou Kheir]

Ahh ok. I'll have to give the keyword analyzer a try then!

Thanks,
Jamil

On Friday, February 21, 2014 2:23:06 PM UTC-8, Binh Ly wrote:
>
> Assuming you have no prior mappings, your first example will put @message 
> through a standard analyzer - i.e. it will chop it up into pieces using 
> this analyzer:
>
>
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis-standard-analyzer.html
>
> So a query like this will not match (since the standard analyzer will make 
> it into multiple terms like: ["useragent", "mozilla", "5.0"], etc.):
>
> "regexp": {
>   "@message": "Mozilla.5.*"
> }
>
> But something like this will (since it matches one of the terms: 
> "mozilla"):
>
> "regexp": {
>   "@message": "mozill."
> }
>
> If instead you use something like a keyword analyzer (or not_analyzed), 
> then the whole string is a single token (["\"userAgent\": \"Mozilla/5.0 
> (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)\""]). 
>
> In this case a query like this will still not match:
>
> "regexp": {
>   "@message": "Mozilla.5.*"
> }
>
> But something like this will:
>
> "regexp": {
>   "@message": ".*Mozilla.5.*"
> }
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/af0df36c-2e29-4edc-be55-ded0f86b2f5e%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

DateRange aggregation semantics - include_lower/include_upper?

[mooky]

What are the expected semantics of the from/to fields in a DateRange 
aggregation?

Are the from/to values included? Should there be an *include_lower*/
*include_upper* option like with filters?

I want the aggregation to include the lower and upper values - but I 
discovered today that it doesnt (I was getting a mismatch between the 
aggregation & the filter with gte/lte values).

My aggregation was defined like this:


"aggregations" : {
  "intentDate" : {
"date_range" : {
  "field" : "intentDate",
  "ranges" : [ {
"key" : "Overdue",
"to" : "2014-02-17"
  }, {
"key" : "March",
"from" : "2014-02-18",
"to" : "2014-03-17"
  }, {
"key" : "April",
"from" : "2014-03-18",
"to" : "2014-04-15T00:00:00.001Z"
  } ]
}
  }
}



-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/7f38e3ce-2ff9-43e9-a476-c94f82256451%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Put mapping documentation -- What options are available? Specifically, how to store a property but without indexing it?

[Ivan Brusic]

Daniel,

Hariharan's link does cover all those topics. The putting mapping api just
describes how to put a mapping. :) The documentation for the actual
mappings is in its own section.

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html

Cheers,

Ivan


On Mon, Feb 24, 2014 at 8:20 AM, Daniel Winterstein <
daniel.winterst...@gmail.com> wrote:

> Hello Alex,
>
> To take the example from that page:
>
> $ curl -XPUT 'http://localhost:9200/twitter/tweet/_mapping' -d '
> {
> "tweet" : {
> "properties" : {
> "message" : {"type" : "string",
> // What can go here??
> // I've seen "analyzer", "store", "enabled" used in passing in
> examples without explanation.
> // Somewhere there must be a list of these! What they mean, and what
> the supported values are.
> }
> }
> }
> }
> '
>
> > Specifically, I have some large data properties I want to store and
> retrieve with results, but not have them indexed for search. So help with
> that would be great.
>
> I think "enabled" might fit this.
>
> Best regards,
>  - Daniel
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAEmLStmw9DtvyWLbyHbmmrmqc8PoZArWt1aCH637NGyyUqqFuw%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQBTNfiWp1afXLSFpVDXUEPjWW24eEFCs8rd8MHRurLbkQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Put mapping documentation -- What options are available? Specifically, how to store a property but without indexing it?

[Daniel Winterstein]

Hello Alex,

To take the example from that page:

$ curl -XPUT 'http://localhost:9200/twitter/tweet/_mapping' -d '
{
"tweet" : {
"properties" : {
"message" : {"type" : "string",
// What can go here??
// I've seen "analyzer", "store", "enabled" used in passing in
examples without explanation.
// Somewhere there must be a list of these! What they mean, and what
the supported values are.
}
}
}
}
'

> Specifically, I have some large data properties I want to store and retrieve 
> with results, but not have them indexed for search. So help with that would 
> be great.

I think "enabled" might fit this.

Best regards,
 - Daniel

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEmLStmw9DtvyWLbyHbmmrmqc8PoZArWt1aCH637NGyyUqqFuw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Logstash elasticsearch output plugin >0.90.9

[Alexander Reelsen]

Hey,

the recently released logstash 1.4 beta includes support for elasticsearch
1.0. See http://www.elasticsearch.org/blog/logstash-1-4-0-beta1/

However when using the elasticsearch_http output you dont need the
node_name directive for example. Please try if you can do a seamless
switch, it should work only changing the configuration. Feel free to report
otherwise.


--Alex


On Mon, Feb 24, 2014 at 1:39 PM, Ben Langers  wrote:

> Hello
>
> Any idea when the elasticsearch output plugin for Logstash will be
> available?
> I'm using Elasticsearch 1.0 but the plugin notes indicate:
>
> *VERSION NOTE*: Your elasticsearch cluster must be running elasticsearch
> 0.90.9. If you use any other version of elasticsearch, you should consider
> using the 
> elasticsearch_httpoutput
>  instead.
>
> I need some directives from the elastichsearch plugin though (f.e.
> node_name)
>
> Regards,
> B.
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/5ce7cdee-ca1c-4532-9e99-88184b3e92dc%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGCwEM_uo5j90w8nv1Vz6qyDF7bco0jv9%2BnpOBvgwDqjvcn4%2BQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Put mapping documentation -- What options are available? Specifically, how to store a property but without indexing it?

[Alexander Reelsen]

Hey,

it is all in the provided link from the first reply? Maybe you can be a bit
more specific with your problem and what infos you are missing and we can
try to help...


--Alex


On Sun, Feb 23, 2014 at 9:48 PM, Daniel Winterstein <
daniel.winterst...@gmail.com> wrote:

> Thank you Hariharan, but no. I am looking for documentation for the other
> options, such as analyze and store.
>
> Best regards,
> Daniel
>
> Sent from my phone. Please excuse the brevity.
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAEmLSt%3D_vb4x7zYor6gH%3DWi%2BAir3Kgf7NoV0kgzUy4N2RfFpTQ%40mail.gmail.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-_1BL2n1U23agy_jmP%3Ds43djoMZdRG%2BruCrTC35rg3ug%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Indexing custom date format

[Alexander Reelsen]

Hey,

you have to change the mapping to accept your custom date time format. You
can set it in your mapping using the format parameter like this

{"sales_date": {"type" : "date", "format" : "-MM-dd ... AND THE REST
HERE"}}

it uses the java SimpleDateFormat class for the defining the date format..



--Alex


On Sun, Feb 23, 2014 at 11:37 PM, pulkitsinghal wrote:

> My current mapping looks like:
>
> 'sale_date': {
>   'format': 'dateOptionalTime',
>   'type': 'date'
> }
>
> On Sunday, February 23, 2014 4:35:36 PM UTC-6, pulkitsinghal wrote:
>>
>> All the out-of-the-box date formats are available here:
>> http://www.elasticsearch.org/guide/en/elasticsearch/
>> reference/current/mapping-date-format.html
>>
>> But other than preprocessing a custom date format before indexing the
>> data, is there anything I can do on the mapping side to allow ES to process
>> a date like:
>>
>> "2013-11-23 22:39:11" and not throw errors like it is malformed at "
>> 22:39:11" ?
>>
>> Thanks!
>>
>> - Pulkit
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/61bc3b33-061d-4c02-b5ab-9bfae65c5b05%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-vavw8WD_-6g8Fnf%2B6L1gUp3_FLy0wk3VsyrJjJ4kaxQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Elasticsearch stops working suddenly

[Alexander Reelsen]

Hey,

please be more verbose than that. Use  the nodes stats and nodes info APIs
to find out how many files the elasticsearch process is allowed to open,
and how many files are open. Show us the output here.
Any special linux distribution or any special security settings which might
prevent setting the max open file descriptors? Please check the max open
files settings in your configuration file as well:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-service.html#_linux

Happy to hear back!


--Alex



On Sun, Feb 23, 2014 at 12:03 AM, Mehdi  wrote:

> Already tried
>
> Le samedi 22 février 2014 22:00:51 UTC, Mark Walkom a écrit :
>>
>> Take a look at http://www.elasticsearch.org/tutorials/too-many-open-
>> files/
>>
>> Regards,
>> Mark Walkom
>>
>> Infrastructure Engineer
>> Campaign Monitor
>> email: ma...@campaignmonitor.com
>> web: www.campaignmonitor.com
>>
>>
>> On 23 February 2014 06:08, Mehdi  wrote:
>>
>>> I have deployed Elasticsearch on a development ubunto VPS. We use that
>>> VPS only to host Api for mobile apps.
>>>
>>> Everything worked good till today, elasticsearch stops working. I tried
>>> to restart it, it says Server ok but when i check the status it gives me :
>>> is not running.
>>>
>>> The Ruby on Rails app log contains : Errno::ECONNREFUSED (Connection
>>> refused - connect(2)) After that, i start getting this error : *Too
>>> many open files*
>>>
>>> Does anyone can help me ? Thanks
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearc...@googlegroups.com.
>>>
>>> To view this discussion on the web visit https://groups.google.com/d/
>>> msgid/elasticsearch/c8481e98-bc29-4846-bb87-18988dfe41fe%
>>> 40googlegroups.com.
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>
>>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/637920e5-4b6a-4de6-bd4b-4ab164f24c6e%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGCwEM-Pvn2aqa9Pw6zSUTL7f33J%2Bg7YOUN5518b8UxZFajXpg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: inserting query into percolator - long delay getting response

[Alexander Reelsen]

Hey,

can you show mapping/setting of that index? How do you insert the data (can
you show the code)? Can you set debug log level on the elasticsearch nodes?
Does this also happen on a one node cluster (as a test)?


--Alex


On Sat, Feb 22, 2014 at 10:34 PM, Matt Price  wrote:

> Sorry a few other details:
> Settings for index that holds percolator:
> {
>   "rules" : {
> "settings" : {
>   "index" : {
> "codec" : {
>   "bloom" : {
> "load" : "false"
>   }
> },
> "uuid" : "_Flxtpi9R8C3wiCe3O04kA",
> "number_of_replicas" : "9",
> "number_of_shards" : "1",
> "version" : {
>   "created" : "199"
> }
>   }
> }
>   }
> }
>
> where we have 10 nodes serving as our data nodes
>
> On Saturday, February 22, 2014 4:31:32 PM UTC-5, Matt Price wrote:
>>
>> Hi,
>>
>> Today we starting noticing that we have a long response delay from ES
>> when inserting a single query into a percolator.
>>
>> Details:
>> Using ES 1.0.0
>> Using Java client library (transport client)
>> About 5 queries in this percolator
>>
>> The delay is ~8 seconds long and the environment is relatively stress
>> free.  Has anyone encountered this issue before or have any suggestions on
>> improving the insert time for a query?
>>
>> Note that we are not seeing this behavior with bulk indexing, etc. so the
>> problem seems to be just confined to the percolator itself.
>>
>> Thanks,
>> Matt
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/88af95e5-7ddf-4818-8f9f-4bfe83ff4643%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGCwEM8GJBOi0Py7u%2BNys9KXGTix_hN5rJzHt%2B1zs%3Dj2v0qcjw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Is there a difference between indexing envelopes or polygons.

[Alexander Reelsen]

Hey,

if there is an error, can you please open a github issue? However the
envelope shape expects you to set an upper left and lower right boundary.
Your coordinates more look like lower left and upper right (meaning you
might create quite a huge envelope acutally) - which obviously does not
matter for a polygon


--Alex


On Sat, Feb 22, 2014 at 11:14 AM, Nicolas THOMASSON <
nico.thomas...@gmail.com> wrote:

> Hello,
>
> I'm new to ES. Please forgive me if I'm asking something stupid.
>
> Is there a fundamental difference between indexing an envelope or indexing
> a polygon ?
>
> For example if I define the area as a envelope
>
> {
>   "frame":{
> "type":"envelope",
> "coordinates": [[3,4],[1,2]]
>   }
> }
>
> or as a polygon
>
> {
>   "frame":{
> "type":"polygon",
> "coordinates": [[[3,4],[3,2],[1,2],[1,4],[3,4]]]
>   }
> }
>
> As in my comprehension they both define the same area, should I be able to
> perform the same queries whatever the way I defined the area ? (Currently I
> have a search query that returns wrong results on the envelope and seems to
> perform well on the polygon.)
>
> Thanks for your help,
>
> Nicolas
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/a2fa0fd8-f9a9-435b-9d34-e603c7242d2f%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGCwEM8usW%3Deam07T1xjFsTDOP1Yuay-DvVrkec2JzeQu0e3ng%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: TransportClient accessing Cluster via single port

[FFA]

Thank you. But what if multiple nodes have to be started to balance the 
load?

Regards,

FFA

On Friday, February 21, 2014 8:50:19 PM UTC-6, David Pilato wrote:
>
> Just set transport port in elasticsearch.yml.
> But you can stay with defaults. As long as you start only one node, only 
> 9300 needs to be opened.
>
> --
> David ;-)
> Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs
>
>
> Le 21 févr. 2014 à 23:10, FFA > a écrit :
>
>  Multinode ES cluster is behind the firewall. I am using Java API 
> TransportClient to query the ES.
>
> I have a simple question:
>  Do I have to open all ports 9300-9400 on firewall to be able to access ES 
> with TC? Is there ES settings that would allow me to open only let's say 
> port 9300 on firewall and reliably get the query results back?
>
> Thanks,
>
> Fred.
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/043d3612-e774-4bff-b6b9-1162f12b2b42%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/670ea0ce-237b-40d2-a6c3-68070a2171e5%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Unassigned nodes giving the cluster red

[Alexander Reelsen]

Hey,

do you used some forced allocation awareness for some indices or the
cluster, which might lead to this?


--Alex


On Thu, Feb 20, 2014 at 10:12 PM,  wrote:

> Hi ,
>
> we had some network glitch .then we had rebounced  all the four nodes
> .after rebouncing we are getting the cluster status red and seeing some
> shards are unassiged.
>
> Could you guys please help what we need to do with those unassigned
> shards.to get the data back
>
> i had tried  cluster rerouting
> {
> "commands":[{"allocate": {"index":"indexname",
> "shard":2,"node":"Dionysus", "allow_primary" : true}}]
> }
>
> but gettign this excepition
>
> {
>   "error": 
> "RemoteTransportException[[Autolycus][inet[/64.102.209.58:9300]][cluster/reroute]];
> nested: ElasticSearchIllegalArgumentException[[allocate] allocation of
> [cqindex][2] on node [Dionysus][T08BTnRcQ162tynpYjCpzQ][inet[ ]] is not
> allowed, reason: [YES()][NO()][YES()][YES()][YES()][YES()][YES()][YES()]];
> ",
>   "status": 400
> }
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/5784cef8-62ae-4876-95d6-cda50882c1c5%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGCwEM973dP3mJom3BFBYKbO9ZC3LDzL%3Dvw_5CvnvP%2BxKOCAtQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [fielddata.format] "disabling field data loading" in mapper attachments type

[Alexander Reelsen]

Hey,

the attachment type consists of several string based types inside of it...
I think you will have to disable fielddata for each of the fields defined
in the metadata, see
https://github.com/elasticsearch/elasticsearch-mapper-attachments


--Alex


On Thu, Feb 20, 2014 at 1:47 PM, Camilo Sierra wrote:

> i will like to disable some information in the fielddata.format. but i
> dont know if i can disable the type "attachment" used by the
> mapper-attachments without problem ?
>
> *
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/fielddata-formats.html#_disabling_field_data_loading
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/285d2277-c875-4aaa-b25e-f0f1a8786a70%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGCwEM_L%2BMRtWx8-wEM2VaEmu6ijxqrbhZEpcR0ELJQsM3dDkQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

cluster in yellow for a week right now, several disk without space

[damian noseda]

Hello:

I have a cluster in yellow with 27 instances 10 shards and 2 replicas.

size: 56.2G (129G)
docs: 158.935.862 (159.132.956)

I have the plugin head with refresh quickly and the replicas are jumping 
from instance to instance. Always trying in instances with no diskspace.

All the instances have 60g and have some with 360g

How can I resolve this?

Any pointer would help

I'm thinking in shuting down the instances with full disk. Delete the 
content of path.data and restart the instances againg and hope maybe the 
shards to stop jumping all the time.

What do you think?

 

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/bb8508e8-bcf8-4e6b-b899-028bf4e1f501%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Zero Downtime Reindexing

[JoeZ99]

How about, while the scan is being done, let updates go to the old index 
but with an extra field? Once the alias points to the new index, it's just 
a query to fetch the fields with that new field from the old index and then 
reindex then into the new one. If the alias changing/new index creation is 
unsuccessful , then update old index to remove that new field.

On Friday, February 21, 2014 3:11:52 AM UTC-5, Andrew Kane wrote:
>
> I tried to post a reply yesterday but it looks like it never made it.
>
> Thank you all for the quick replies.  Here's a slightly better explanation 
> of where I believe the race condition occurs.
>
> When the scan/scroll starts, the alias is still pointing to the old index, 
> so updates go to the old index.  Let's say you update Document 1.  If the 
> scroll/scan has already passed Document 1, the new index never sees the 
> update.  The three solutions you mentioned Nik are to either:
>
> 1. Keep track of updates manually [tedious]
> 2. Pause the jobs that perform the updates [out of sync]
> 3. Send updates to both indexes [also tedious]
>
> However, none of these seem ideal.
>
> - Andrew
>
> On Tuesday, February 18, 2014 8:41:18 PM UTC-8, Andrew Kane wrote:
>>
>> Hi,
>>
>> I've followed the documentation for zero-downtime mapping changes and it 
>> works great.  
>> http://www.elasticsearch.org/blog/changing-mapping-with-zero-downtime/
>>
>> However, there is a (pretty big) race condition with this approach - 
>> while reindexing, changes may not make it to the new index.  I've looked 
>> all over and haven't found a single solution to address this.  The best 
>> attempt I've seen is to buffer updates, but this is tedious and still 
>> leaves a race condition (with a smaller window).  My initial thoughts were 
>> to create a write alias that points to the old and new indices and use 
>> versioning.  However, there is no way to write to multiple indices 
>> atomically.
>>
>> It seems like this issue should affect most Elasticsearch users (whether 
>> they realize it or not).  Does anyone have a good solution to this?
>>
>> Thanks,
>> Andrew
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/5eff28f1-aec6-4fd1-b52d-168191e1de30%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Hadoop] Cannot discover Elasticsearch version exception

[Yann Barraud]

Hi,

I'm going my way using Hortonworks Sandbox & Elasticsearch Hadoop.

I can't have Hive request Elasticsearch. Seems it attemps connecting to ES 
through hostname while I provided IP adress in Hive query...

I found a workaround setting up /etc/hosts in snadbox, but can't figure out 
why it resolves ip to hostname

14/02/24 05:41:37 INFO Configuration.deprecation: mapred.input.dir.recursive is 
deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive
> 14/02/24 05:41:37 INFO ql.Driver: 
> 14/02/24 05:41:37 INFO ql.Driver: 
> 14/02/24 05:41:37 INFO ql.Driver: 
> 14/02/24 05:41:37 INFO ql.Driver: 
> 14/02/24 05:41:37 INFO parse.ParseDriver: Parsing command: use default
> 14/02/24 05:41:37 INFO parse.ParseDriver: Parse Completed
> 14/02/24 05:41:37 INFO ql.Driver:  end=1393249297365 duration=1>
> 14/02/24 05:41:37 INFO ql.Driver: 
> 14/02/24 05:41:37 INFO ql.Driver: Semantic Analysis Completed
> 14/02/24 05:41:37 INFO ql.Driver:  start=1393249297365 end=1393249297365 duration=0>
> 14/02/24 05:41:37 INFO ql.Driver: Returning Hive schema: 
> Schema(fieldSchemas:null, properties:null)
> 14/02/24 05:41:37 INFO ql.Driver:  start=1393249297364 end=1393249297365 duration=1>
> 14/02/24 05:41:37 INFO ql.Driver: 
> 14/02/24 05:41:37 INFO ql.Driver: Starting command: use default
> 14/02/24 05:41:37 INFO ql.Driver:  start=1393249297364 end=1393249297366 duration=2>
> 14/02/24 05:41:37 INFO ql.Driver: 
> 14/02/24 05:41:37 INFO ql.Driver: 
> 14/02/24 05:41:37 INFO ql.Driver:  start=1393249297366 end=1393249297374 duration=8>
> 14/02/24 05:41:37 INFO ql.Driver:  start=1393249297366 end=1393249297374 duration=8>
> 14/02/24 05:41:37 INFO ql.Driver:  start=1393249297365 end=1393249297374 duration=9>
> OK
> 14/02/24 05:41:37 INFO ql.Driver: OK
> 14/02/24 05:41:37 INFO ql.Driver: 
> 14/02/24 05:41:37 INFO ql.Driver:  start=1393249297374 end=1393249297375 duration=1>
> 14/02/24 05:41:37 INFO ql.Driver:  start=1393249297364 end=1393249297375 duration=11>
> converting to local 
> hdfs://sandbox.hortonworks.com:8020/user/elasticsearch-hadoop-1.3.0.M2-yarn.jar
> 14/02/24 05:41:37 INFO SessionState: converting to local 
> hdfs://sandbox.hortonworks.com:8020/user/elasticsearch-hadoop-1.3.0.M2-yarn.jar
> Added 
> /tmp/2d776dfb-d58c-452d-90f0-28d774c6dd2d_resources/elasticsearch-hadoop-1.3.0.M2-yarn.jar
>  to class path
> 14/02/24 05:41:37 INFO SessionState: Added 
> /tmp/2d776dfb-d58c-452d-90f0-28d774c6dd2d_resources/elasticsearch-hadoop-1.3.0.M2-yarn.jar
>  to class path
> Added resource: 
> /tmp/2d776dfb-d58c-452d-90f0-28d774c6dd2d_resources/elasticsearch-hadoop-1.3.0.M2-yarn.jar
> 14/02/24 05:41:37 INFO SessionState: Added resource: 
> /tmp/2d776dfb-d58c-452d-90f0-28d774c6dd2d_resources/elasticsearch-hadoop-1.3.0.M2-yarn.jar
> 14/02/24 05:41:37 INFO ql.Driver: 
> 14/02/24 05:41:37 INFO ql.Driver: 
> 14/02/24 05:41:37 INFO parse.ParseDriver: Parsing command: -- stream data 
> from Elasticsearch
> SELECT * FROM fca
> 14/02/24 05:41:37 INFO parse.ParseDriver: Parse Completed
> 14/02/24 05:41:37 INFO ql.Driver:  end=1393249297389 duration=0>
> 14/02/24 05:41:37 INFO ql.Driver: 
> 14/02/24 05:41:37 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
> 14/02/24 05:41:37 INFO parse.SemanticAnalyzer: Completed phase 1 of Semantic 
> Analysis
> 14/02/24 05:41:37 INFO parse.SemanticAnalyzer: Get metadata for source tables
> 14/02/24 05:41:37 INFO parse.SemanticAnalyzer: Get metadata for subqueries
> 14/02/24 05:41:37 INFO parse.SemanticAnalyzer: Get metadata for destination 
> tables
> 14/02/24 05:41:37 INFO ql.Context: New scratch dir is 
> hdfs://sandbox.hortonworks.com:8020/tmp/hive-beeswax-hue/hive_2014-02-24_05-41-37_389_1622127390930327029-1
> 14/02/24 05:41:37 INFO parse.SemanticAnalyzer: Completed getting MetaData in 
> Semantic Analysis
> 14/02/24 05:41:37 INFO ppd.OpProcFactory: Processing for FS(8)
> 14/02/24 05:41:37 INFO ppd.OpProcFactory: Processing for SEL(7)
> 14/02/24 05:41:37 INFO ppd.OpProcFactory: Processing for TS(6)
> FAILED: IllegalStateException *Cannot discover Elasticsearch version*
> 14/02/24 05:41:57 ERROR ql.Driver: FAILED: IllegalStateException Cannot 
> discover Elasticsearch version
> java.lang.IllegalStateException: Cannot discover Elasticsearch version
>   at 
> org.elasticsearch.hadoop.hive.EsStorageHandler.init(EsStorageHandler.java:101)
>   at 
> org.elasticsearch.hadoop.hive.EsStorageHandler.configureInputJobProperties(EsStorageHandler.java:78)
>   at 
> org.apache.hadoop.hive.ql.plan.PlanUtils.configureJobPropertiesForStorageHandler(PlanUtils.java:755)
>   at 
> org.apache.hadoop.hive.ql.plan.PlanUtils.configureInputJobPropertiesForStorageHandler(PlanUtils.java:725)
>   at 
> org.apache.hadoop.hive.ql.optimizer.SimpleFetchOptimizer$FetchData.convertToWork(SimpleFetchOptimizer.java:208)
>   at 
> org.apache.hadoop.hive.ql.optimizer.SimpleFetchOptimizer$FetchData.access$000(SimpleFetchOptimizer.java:180)
>   at 
> org.apache.hadoop.hive.ql.optimizer

Re: Marvel houskeeping

[David Pilato]

It will be possible in the future when this issue will be fixed: 
https://github.com/elasticsearch/elasticsearch/issues/2114


-- 
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr


Le 24 février 2014 à 13:47:07, Thomas Andres (thomas.and...@ergon.ch) a écrit:

I know deleting them isn't hard, and it's good to know there is a tool to 
automate that. However, I do think that Marvel should do that itself. Shouldn't 
be too hard to e.g.  extend the code that creates a new daily index to also 
cleanup old ones.

As it is now, you install a plugin and suddenly run out of space, which I don't 
consider a good default behaviour (I know you guys take care of setting smart 
default values, which is one reason elasticsearch is so good!). I think this 
would be a small extension, that probably  prevents many users from a rather 
bad surprise.

Cheers
Thomas


Am Freitag, 14. Februar 2014 20:40:26 UTC+1 schrieb Boaz Leskes:

Marvel itself doesn't have a setting for this, but you can have a look at this 
tool, built by the logstash team to help management indices with time based 
data: https://github.com/elasticsearch/curator


--
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/d77c267c-1f88-4d58-9029-f69ae52ac409%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/etPan.530b4c6b.515f007c.5e46%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/groups/opt_out.

Kibana: showing a ratio

[Andrew Vine]

Hi All,

At the moment in kibana I can use the stats or terms panel with term_stats 
mode to show stats by term..

So for example I could get a total for term A. What I would like to do 
however is show the value {total for term A} / {total for all terms}.

So if the total for term A is 5 and the total for all terms is 10 then I 
would like to show 0.5.

Is that possible?

Thanks!
Andrew  

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/b1559039-c1cf-417a-9a08-890ffa80d1f4%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: failed to setup multicast discovery on port [54328] ES can't connect to cluster.

[Alexander Reelsen]

Hey,

first, you should really upgrade elasticsearch,  this is quite an old beta
version.
Second, if you are using something like amazon ec2, multicast is disabled.
You might want to test with unicast in that case, see
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-discovery-zen.html#unicast


--Alex


On Wed, Feb 19, 2014 at 7:56 PM, Oğulcan Selcuk Akbulut wrote:

> [2014-02-19 20:40:58,668][INFO ][node ]
>> [elastic_inst2] {0.90.0.Beta1}[1368]: initializing ...
>> [2014-02-19 20:40:58,676][INFO ][plugins  ]
>> [elastic_inst2] loaded [], sites []
>> [2014-02-19 20:41:01,402][INFO ][node ]
>> [elastic_inst2] {0.90.0.Beta1}[1368]: initialized
>> [2014-02-19 20:41:01,403][INFO ][node ]
>> [elastic_inst2] {0.90.0.Beta1}[1368]: starting ...
>> [2014-02-19 20:41:01,501][INFO ][transport]
>> [elastic_inst2] bound_address {inet[/0:0:0:0:0:0:0:0:9301]},
>> publish_address {inet[/192.168.2.130:9301]}
>> [2014-02-19 20:41:01,521][WARN ][discovery.zen.ping.multicast]
>> [elastic_inst2] disabled, failed to setup multicast discovery on port
>> [54328], [/192.168.2.130]: Invalid argument
>> [2014-02-19 20:41:04,563][INFO ][cluster.service  ]
>> [elastic_inst2] new_master
>> [elastic_inst2][VVoWrIc4QRuiSo9-0lhd1g][inet[/192.168.2.130:9301]],
>> reason: zen-disco-join (elected_as_master)
>> [2014-02-19 20:41:04,595][INFO ][discovery]
>> [elastic_inst2] elasticsearch_dc1/VVoWrIc4QRuiSo9-0lhd1g
>> [2014-02-19 20:41:05,573][INFO ][http ]
>> [elastic_inst2] bound_address {inet[/0:0:0:0:0:0:0:0:9201]},
>> publish_address {inet[/192.168.2.130:9201]}
>> [2014-02-19 20:41:05,573][INFO ][node ]
>> [elastic_inst2] {0.90.0.Beta1}[1368]: started
>> [2014-02-19 20:41:05,576][INFO ][gateway  ]
>> [elastic_inst2] recovered [0] indices into cluster_state
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/507affac-fe3f-4501-ad4d-6a8c4a5672fb%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGCwEM8t%3DPTOV%3DGLGyPnvX-zbN910NmR_GW6FmVb%3DBy9ZOVMjA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Can we configure backup and restore functionality throught yml file in the new 1.0 release ? i

[Alexander Reelsen]

Hey

no you cannot. You might want to check this blog post how to set it up
correctly: http://www.elasticsearch.org/blog/introducing-snapshot-restore/


--Alex


On Wed, Feb 19, 2014 at 9:13 AM, Amulya Varshney wrote:

>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/b75680ab-26af-444f-b257-453dc68f9fd4%40googlegroups.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGCwEM_ofGMa389qZFvhh51VnR_S3BVKwWAO0XqY-3uOn6F1ow%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Corruption error after upgrade to 1.0

[Alexander Reelsen]

Hey,

does the JVM where the TransportClient is running on, use the same JVM and
elasticsearch version than the elasticsearch cluster?


--Alex


On Mon, Feb 17, 2014 at 10:25 PM, Mohit Anchlia wrote:

> After upgrading to 1.0 I am unable to index any documents. I get the
> following error. Could somebody help?
>
>
> [Aardwolf] Message not fully read (response) for [0] handler
> future(org.elasticsearch.client.transport.TransportClientNodesService$SimpleNodeSampler$1@5c6e3b4c),
> error [true], resetting
>
> [Aardwolf] failed to get node info for
> [#transport#-1][inet[/10.80.140.59:9300]], disconnecting...
>
> org.elasticsearch.transport.RemoteTransportException: Failed to
> deserialize exception response from stream
>
> Caused by: org.elasticsearch.transport.TransportSerializationException:
> Failed to deserialize exception response from stream
>
> at
> org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:168)
>
> at
> org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:122)
>
> at
> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
>
> at
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
>
> at
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
>
> at
> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)
>
> at
> org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
>
> at
> org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
>
> at
> org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
>
> at
> org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
>
> at
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
>
> at
> org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
>
> at
> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
>
> at
> org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
>
> at
> org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
>
> at
> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
>
> at
> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
>
> at
> org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
>
> at
> org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
>
> at
> org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
>
> at
> org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>
> at java.lang.Thread.run(Unknown Source)
>
> Caused by: java.io.StreamCorruptedException: unexpected end of block data
>
> at java.io.ObjectInputStream.readObject0(Unknown Source)
>
> at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
>
> at java.io.ObjectInputStream.defaultReadObject(Unknown Source)
>
> at java.lang.Throwable.readObject(Throwable.java:913)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>
> at java.lang.reflect.Method.invoke(Unknown Source)
>
> at java.io.ObjectStreamClass.invokeReadObject(Unknown Source)
>
> at java.io.ObjectInputStream.readSerialData(Unknown Source)
>
> at java.io.ObjectInputStream.readOrdinaryObject(Unknown Source)
>
> at java.io.ObjectInputStream.readObject0(Unknown Source)
>
> at java.io.ObjectInputStream.defaultReadFields(Unknown Source)
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWoObGmewZSs-ZtT_qWY7mRJ_Z220LkW2QbAD4G%2BGGPXqA%40mail.gmail.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+u

Re: Marvel houskeeping

[Thomas Andres]

I know deleting them isn't hard, and it's good to know there is a tool to 
automate that. However, I do think that Marvel should do that itself. 
Shouldn't be too hard to e.g.  extend the code that creates a new daily 
index to also cleanup old ones.

As it is now, you install a plugin and suddenly run out of space, which I 
don't consider a good default behaviour (I know you guys take care of 
setting smart default values, which is one reason elasticsearch is so 
good!). I think this would be a small extension, that probably  prevents 
many users from a rather bad surprise.

Cheers
Thomas


Am Freitag, 14. Februar 2014 20:40:26 UTC+1 schrieb Boaz Leskes:
>
>
> Marvel itself doesn't have a setting for this, but you can have a look at 
> this tool, built by the logstash team to help management indices with time 
> based data: https://github.com/elasticsearch/curator
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/d77c267c-1f88-4d58-9029-f69ae52ac409%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Logstash elasticsearch output plugin >0.90.9

[Ben Langers]

Hello

Any idea when the elasticsearch output plugin for Logstash will be 
available?
I'm using Elasticsearch 1.0 but the plugin notes indicate:

*VERSION NOTE*: Your elasticsearch cluster must be running elasticsearch 
0.90.9. If you use any other version of elasticsearch, you should consider 
using the 
elasticsearch_httpoutput
 instead.

I need some directives from the elastichsearch plugin though (f.e. 
node_name)

Regards,
B.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/5ce7cdee-ca1c-4532-9e99-88184b3e92dc%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Elasticsearch Reverse Suggester Problem

[Garry Welding]

Argh, knew I'd forget something, the blooming ES version number! I'm on the 
latest 1.0 version.

On Monday, February 24, 2014 11:39:00 AM UTC, Garry Welding wrote:
>
> Hi guys, I'm hoping somebody on here can help me, I feel like I'm just 
> missing something really basic but I can't for the life of me figure out 
> what... I have the following index set up (it's very cut down for clarity's 
> sake):
>
> {
> "index":"products",
> "body":{
> "settings":{
> "number_of_shards":5,
> "number_of_replicas":1,
> "analysis":{
> "analyzer":{
> "default":{
> "type":"snowball",
> "language":"English"
> },
> "reverse":{
> "type":"custom",
> "language":"English",
> "tokenizer":"standard",
> 
> "filter":["standard","lowercase","stop","snowball","reverse"]
> }
> }
> }
> },
> "mappings":{
> "product":{
> "properties":{
> "_all":{"enabled":true},
> 
> "id":{"type":"string","include_in_all":true,"index":"analyzed","analyzer":"snowball","store":"yes"},
> 
> "name":{"type":"string","include_in_all":true,"index":"analyzed","analyzer":"snowball","store":"yes"},
> 
> "name_reverse":{"type":"string","include_in_all":true,"index":"analyzed","analyzer":"reverse","store":"yes"}
> }
> }
> }
> }
> }
>
> I'm then running search queries against this and I'm now trying to do 
> suggesters. The following query works fine and returns no "suggests" as 
> expected:
>
> {
> "index":"products",
> "type":"product",
> "body":{
> "indices_boost":{"id":2,"name":1.5},
> "query":{
> "filtered":{
> "query":{
> "query_string":{
> "query":"pushchair",
> "fields":["id","name"]
> }
> }
> }
> },
> "suggest":{
> "text":"pushchair",
> "simple_phrase":{
> "phrase":{
> "field":"name",
> "size":4,
> "real_word_error_likelihood":0.95,
> "confidence":1,
> "gram_size":1,
> "direct_generator":[
> {
> "field":"name",
> "suggest_mode":"always",
> "min_word_len":1
> }
> }
> }
> }
> }
> }
>
> The next query also works fine and returns the expected suggestions:
>
> {
> "index":"products",
> "type":"product",
> "body":{
> "indices_boost":{"id":2,"name":1.5},
> "query":{
> "filtered":{
> "query":{
> "query_string":{
> "query":"pushchiar",
> "fields":["id","name"]
> }
> }
> }
> },
> "suggest":{
> "text":"pushchair",
> "simple_phrase":{
> "phrase":{
> "field":"name",
> "size":4,
> "real_word_error_likelihood":0.95,
> "confidence":1,
> "gram_size":1,
> "direct_generator":[
> {
> "field":"name",
> "suggest_mode":"always",
> "min_word_len":1
> }
> }
> }
> }
> }
> }
>
> As you can see, "pushchair" is spelt incorrectly and then response from 
> Elasticsearch provides the correct suggestion. The problem comes when I try 
> to add in reverse support as follows:
>
> {
> "index":"products",
> "type":"product",
> "body":{
> "indices_boost":{"id":2,"name":1.5},
> "query":{
> "filtered":{
> "query":{
> "query_string":{
> "query":"pushchair",
> "fields":["id","name"]
> }
> }
> }
> },
> "suggest":{
> "text":"pushchair",
> "simple_phrase":{
> "phrase":{
> "field":"name",
> "size":4,
> "real_word_error_likelihood":0.95,
> "confidence":1,
> "gram_size":1,
> "direct_generator":[
> {
> "field":"name",
>  

Re: [Hadoop] Any goos tut to start with ?

[Costin Leau]

Have you looked at the video? It does exactly that.

Is there something missing?

On 2/24/2014 12:41 PM, Yann Barraud wrote:

Hi Costin,

What I'd love to see is a step by step tut have ES and Haddop working together.

Is there somewhere I can have something like this ?

Regards,
Yann

Le jeudi 20 février 2014 16:25:28 UTC+1, John Pauley a écrit :

Any more tutorials, say append to list?

On Wednesday, February 19, 2014 12:54:15 PM UTC-5, Costin Leau wrote:

Hi,

We tried to make the docs friendly in this regard - each section (from 
Map/Reduce to Pig) has several examples.
There's
also a short video which guides you through the various features (with 
code) available here [1].

Hope this helps,

[1] 
http://www.elasticsearch.org/videos/search-and-analytics-with-hadoop-and-elasticsearch/



On 19/02/2014 5:11 PM, Yann Barraud wrote:
> Hi everyone,
>
> Do you have a good pointer to a tut to start playing with ES & Hadoop 
? Using Hortonworks VM for example ?
>
> Thanks.
>
> Cheers,
> Yann
>
> --
> You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, 
send an email to
>elasticsearc...@googlegroups.com.
> To view this discussion on the web visit

>https://groups.google.com/d/msgid/elasticsearch/a7b35ba0-2b42-4270-bb64-228dad7fc426%40googlegroups.com

.
> For more options, visithttps://groups.google.com/groups/opt_out 
.

--
Costin

--
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to
elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c55aa6e1-adde-4044-baee-c80516fe00e6%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
Costin

--
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/530B3CA2.6010001%40gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Count versus Search API

[Pulkit Singhal]

Hey @David, since Luca had zeroed-in-on the issue, I'll skip providing the
query this time.

@Luca - Thanks! That was exactly the problem! Another thing that is
inconsistent right now is that I can use date math against the _search
endpoint for filters but not against the _count endpoint in 0.90:
"lt" : "now",
"gte" : "now-1w"
versus:
"lt" : "2014-02-24",
"gte" : "2014-02-17"


- Pulkit
(646)-535-2357
Cofounder & CTO, ShoppinPal
https://linkedin.com/in/pulkitsinghal
https://coderbits.com/pulkitsinghal


On Mon, Feb 24, 2014 at 1:38 AM, Luca Cavanna  wrote:

> I guess you are still on 0.90, where the count request is slightly
> different compared to a search request, as it contains the query without it
> being wrapped into the query object:
>
> {
> "term" : {"field":"value"}
> }
>
> instead of
>
> {
> "query" : {
> "term" : {"field":"value"}
> }
> }
>
> Beware that we changed this in 1.0 to have consistency between count api
> and search api.
>
>
> On Monday, February 24, 2014 4:51:43 AM UTC+1, pulkitsinghal wrote:
>>
>> Hello,
>>
>> I have a bool query with a filter applied to it, which runs against the
>> _search endpoint successfully giving "hits.total" but unfortunately it also
>> gives a lot more data than that which I don't need.
>>
>> So I thought of running the same query against the _count endpoint but
>> there I received a failure: 
>> "BroadcastShardOperationFailedException[[index_test][0]
>> ]; nested: QueryParsingException[[index_test] No query registered for
>> [query]]; "
>>
>> Is there some other lightweight manner in which I can get just
>> "hits.total" without the added bloat of each matching doc?
>>
>> Thanks!
>> - Pulkit
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/6d7827db-af6e-4f5e-a394-bfb09652be7a%40googlegroups.com
> .
>
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CADA%3DEV807-Us2vVqcf8vbif-1Mxsx4rGtV8nn%3D0KyX_tWJYLaA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Connecting ES Cluster using Thrift

[Honza Král]

Hi,

no, there is no other configuration. The error you are getting is
weird because it should at least be wrapped in our exception class.
Can you provide more details? maybe increase logging and see if
anything pops out in the logs, also exact versions and code how you
connect to the nodes.

thanks

On Mon, Feb 24, 2014 at 11:03 AM,   wrote:
> Hi,
>
> I am a newbie about Elasticsearch. I am trying ES for some of my projects. I
> want to connect ES cluster using Thrift with python client. I installed
> plugin to all machines and restarted them. Then, I try to connect via Thrift
> and it gives me "socket.timeout: timed out" error. I can connect to my
> cluster without any problem using HTTP.
>
> Is there any additional configurations on server side or client side?
>
> Thanks,
>
> Umutcan
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/b0c110df-1580-46fb-9fa0-2cb656fb2036%40googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CABfdDirWA3cJ6c6scKT0u8KCkjUyDZ2zf%2Bk7cNwUobR4Jvp7VQ%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

collectd indices RRA like consolidation

[rachid zarouali]

hy all,

i was wondering if it's possible to do RRA like consolidation on collectd 
indices stored in my ES cluster?
i've seen this kind of "option" in EMR (hadoop), so i though it was possible to 
do nearly the same with ES

any clues apreciated :)

Regards,

Rachid

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/4db20496-2e72-4860-8304-e8cbd894bc2d%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

ES doesn't take into account field level boost in prefix query over catch-all field?

[Eugene I.]

Greetings everyone,

I've faced with a following issue, and I'm not sure whether it's ES 
peculiarity or something is wrong with my setup.

1. Created ES mapping with two string fields, one of them is with 
customized boost.
2. Performed term search over default catch-all field and observed expected 
result -- sample doc containing the search term in the field with 
customized boost has a higher score.
3. Performed prefix query with customized rewrite over default catch-all 
field and got response which confuses me. ES ignores the field level boost 
in this case.

So, the question is - is it possible to get for prefix query the response 
which honors field level boost factor, as term query does?

The setup: https://gist.github.com/ei82/776d42d6a59262529c54

Thanks,
Eugene,

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/8ac1f92a-857c-43e2-8f11-64bbedd6d357%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Elasticsearch Reverse Suggester Problem

[Garry Welding]

Hi guys, I'm hoping somebody on here can help me, I feel like I'm just 
missing something really basic but I can't for the life of me figure out 
what... I have the following index set up (it's very cut down for clarity's 
sake):

{
"index":"products",
"body":{
"settings":{
"number_of_shards":5,
"number_of_replicas":1,
"analysis":{
"analyzer":{
"default":{
"type":"snowball",
"language":"English"
},
"reverse":{
"type":"custom",
"language":"English",
"tokenizer":"standard",

"filter":["standard","lowercase","stop","snowball","reverse"]
}
}
}
},
"mappings":{
"product":{
"properties":{
"_all":{"enabled":true},

"id":{"type":"string","include_in_all":true,"index":"analyzed","analyzer":"snowball","store":"yes"},

"name":{"type":"string","include_in_all":true,"index":"analyzed","analyzer":"snowball","store":"yes"},

"name_reverse":{"type":"string","include_in_all":true,"index":"analyzed","analyzer":"reverse","store":"yes"}
}
}
}
}
}

I'm then running search queries against this and I'm now trying to do 
suggesters. The following query works fine and returns no "suggests" as 
expected:

{
"index":"products",
"type":"product",
"body":{
"indices_boost":{"id":2,"name":1.5},
"query":{
"filtered":{
"query":{
"query_string":{
"query":"pushchair",
"fields":["id","name"]
}
}
}
},
"suggest":{
"text":"pushchair",
"simple_phrase":{
"phrase":{
"field":"name",
"size":4,
"real_word_error_likelihood":0.95,
"confidence":1,
"gram_size":1,
"direct_generator":[
{
"field":"name",
"suggest_mode":"always",
"min_word_len":1
}
}
}
}
}
}

The next query also works fine and returns the expected suggestions:

{
"index":"products",
"type":"product",
"body":{
"indices_boost":{"id":2,"name":1.5},
"query":{
"filtered":{
"query":{
"query_string":{
"query":"pushchiar",
"fields":["id","name"]
}
}
}
},
"suggest":{
"text":"pushchair",
"simple_phrase":{
"phrase":{
"field":"name",
"size":4,
"real_word_error_likelihood":0.95,
"confidence":1,
"gram_size":1,
"direct_generator":[
{
"field":"name",
"suggest_mode":"always",
"min_word_len":1
}
}
}
}
}
}

As you can see, "pushchair" is spelt incorrectly and then response from 
Elasticsearch provides the correct suggestion. The problem comes when I try 
to add in reverse support as follows:

{
"index":"products",
"type":"product",
"body":{
"indices_boost":{"id":2,"name":1.5},
"query":{
"filtered":{
"query":{
"query_string":{
"query":"pushchair",
"fields":["id","name"]
}
}
}
},
"suggest":{
"text":"pushchair",
"simple_phrase":{
"phrase":{
"field":"name",
"size":4,
"real_word_error_likelihood":0.95,
"confidence":1,
"gram_size":1,
"direct_generator":[
{
"field":"name",
"suggest_mode":"always",
"min_word_len":1
},{
"field":"name_reverse",
"suggest_mode":"always",
"min_word_len":1,
"pre_filter":"reverse",
"post_filter":"reverse"
}
]
}
}
  

Re: Official .NET client

[Loïc Wenkin]

Hi,

I would like to know if a release date of a first .NET client is already 
planned, and if it's the case, when is it ?

Thanks for your replies.

Loïc

Le jeudi 24 octobre 2013 16:47:20 UTC+2, cdhall a écrit :
>
> Yes, there will be an official client for .NET. 
>
> Our decision to create our own clients was mostly about consistency for 
> our users. To do that we created a set of low-level clients that map very 
> closely to the REST api. We made sure (and are continuously doing so) that 
> these clients implement all the API endpoints as well as all parameters and 
> to the right thing.
>
> We tried our best to avoid any design decisions in those clients to make 
> sure everybody could use them, even if it means creating their own 
> abstraction on top of it. The overhead of the clients are so small and 
> their design flexible enough (we hope) that all the other clients can live 
> on top of these.
>
> "the intention is to provide a solid base which can be used by other 
> implementors if they want to add specializations. we take care of the hard 
> stuff like node failover, while sticking very close to the standard ES REST 
> API". things like LINQ would be out of scope for this client, but eg the 
> NEST client could be implemented on top of the official .net client to 
> provide linq support while taking advantage of the networking improvments
>
> I hope that answers your first question. For more information Honza Krai 
> will be doing a webinar about this - why we chose to do the clients, what 
> drove the design decisions we took. It should take place next week sometime.
>
> On Thursday, October 24, 2013 5:58:38 AM UTC-5, joe castle wrote:
>>
>> I still think that it would be nice to get an answer if there will be an 
>> official .NET client. It has nothing to do with NEST being good or not 
>> (it's not that all python clients were bad so they had to make an official 
>> one).
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/993b7185-133b-4029-b5cc-0b1a67ff21d6%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Hadoop] Any goos tut to start with ?

[Yann Barraud]

Hi Costin,

What I'd love to see is a step by step tut have ES and Haddop working 
together. 

Is there somewhere I can have something like this ?

Regards,
Yann

Le jeudi 20 février 2014 16:25:28 UTC+1, John Pauley a écrit :
>
> Any more tutorials, say append to list?
>
> On Wednesday, February 19, 2014 12:54:15 PM UTC-5, Costin Leau wrote:
>>
>> Hi, 
>>
>> We tried to make the docs friendly in this regard - each section (from 
>> Map/Reduce to Pig) has several examples. There's 
>> also a short video which guides you through the various features (with 
>> code) available here [1]. 
>>
>> Hope this helps, 
>>
>> [1] 
>> http://www.elasticsearch.org/videos/search-and-analytics-with-hadoop-and-elasticsearch/
>>  
>>
>> On 19/02/2014 5:11 PM, Yann Barraud wrote: 
>> > Hi everyone, 
>> > 
>> > Do you have a good pointer to a tut to start playing with ES & Hadoop ? 
>> Using Hortonworks VM for example ? 
>> > 
>> > Thanks. 
>> > 
>> > Cheers, 
>> > Yann 
>> > 
>> > -- 
>> > You received this message because you are subscribed to the Google 
>> Groups "elasticsearch" group. 
>> > To unsubscribe from this group and stop receiving emails from it, send 
>> an email to 
>> > elasticsearc...@googlegroups.com. 
>> > To view this discussion on the web visit 
>> > 
>> https://groups.google.com/d/msgid/elasticsearch/a7b35ba0-2b42-4270-bb64-228dad7fc426%40googlegroups.com.
>>  
>>
>> > For more options, visit https://groups.google.com/groups/opt_out. 
>>
>> -- 
>> Costin 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/c55aa6e1-adde-4044-baee-c80516fe00e6%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Connecting ES Cluster using Thrift

[umutcan]

Hi,

I am a newbie about Elasticsearch. I am trying ES for some of my projects. 
I want to connect ES cluster using Thrift with python client. I installed 
plugin to all machines and restarted them. Then, I try to connect via 
Thrift and it gives me "socket.timeout: timed out" error. I can connect to 
my cluster without any problem using HTTP.

Is there any additional configurations on server side or client side? 

Thanks,

Umutcan

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/b0c110df-1580-46fb-9fa0-2cb656fb2036%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

geo_point in nested has the 'path' attribute added to it

[Tim S]

I was writing some tests to check if my mappings were being deployed 
correctly, and came across this: if you have a geo_point field inside a 
nested object, it will inherit the 'path' attribute from the nested object.

I.e. if you create an index like this:

curl -XPOST 'localhost:9200/test' -d '{
"mappings" : {
"type1" : {
"properties" : {
"geoInOuter" : {
"type" : "geo_point"
},
"obj1" : {
"type" : "nested",
"path" : "just_name",
"properties" : {
"geoInNested" : {
"type" : "geo_point"
}
}
}
}
}
}
}'

and then ask for the mapping back, like this
curl -XGET 'localhost:9200/test/_mapping'

the response will be this:

{
"test" : {
"mappings" : {
"type1" : {
"properties" : {
"geoInOuter" : {
"type" : "geo_point"
},
"obj1" : {
"type" : "nested",
"path" : "just_name",
"properties" : {
"geoInNested" : {
"type" : "geo_point",
"path" : "just_name"
}
}
}
}
}
}
}
}

Notice the extra 'path' attribute on the 'geoInNested' field.

Is this deliberate? In which case what affect does the path attribute have 
on a geo point field? Or is this a bug?

Thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/d60a0c7c-afb0-42f7-bfc0-19577189e34a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Slow cluster startup with zen discovery and large number of nodes.

[Michel Conrad]

Also the kernel complains about too many connections being made at
once on the joining node.
(Seems to occur after 30 nodes joined the cluster)

TCP: TCP: Possible SYN flooding on port 9300. Sending cookies.  Check
SNMP counters.

On Fri, Feb 21, 2014 at 6:44 PM, Thibaut Britz  wrote:
> Hi,
>
> I'm working with Michel on that issue:
>
> The cluster is completely empty and has no indexes at all. So it certainly
> is not related to revocery.
> The old elasticsearch version doesn't have code to wait for replies which
> causes the very slow startup.
>
> Thanks,
> Thibaut
>
>
>
>
>
> On Fri, Feb 21, 2014 at 6:11 PM, Binh Ly  wrote:
>>
>> You may be interested in some settings that help a full cluster restart:
>>
>>
>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-gateway.html#recover-after
>>
>> There is also a webinar that talks about some of the above:
>>
>> http://www.elasticsearch.org/webinars/elasticsearch-pre-flight-checklist/
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/a405d32a-50a5-4d83-a3c7-8a0ea3449d28%40googlegroups.com.
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAH0sEYjyTwFZFP7ZPCsOquSJDrYmdTrrhkmpD5XxbKpv-YO85A%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Text Categorization in ES

[Dawid Weiss]

If you want classification then Carrot2/ Lingo3G won't be of much use
-- in short classification is assigning an unlabeled example to a pool
of (previously known or computed) labels, Lingo3G and Carrot2 are for
clustering (finding "labels" in an otherwise untagged set of documents
or search results).

http://en.wikipedia.org/wiki/Statistical_classification
http://en.wikipedia.org/wiki/Cluster_analysis

I would agree with Hannes that the simplest way to "classify"
documents with an inverted index would be to use a knn-like algorithm.

Dawid

On Mon, Feb 24, 2014 at 9:06 AM, joergpra...@gmail.com
 wrote:
> Install the carrot2 plugin and see if it fits your requirments:
> http://download.carrotsearch.com/lingo3g/manual/#section.es
>
> Jörg
>
>
> On Mon, Feb 24, 2014 at 7:00 AM, prashant.agrawal
>  wrote:
>>
>> Hi Hannes,
>>
>> Thanks for the info , also I came to know about lingo3G/Carrot Search.
>> So whether that could also be a solution for that?
>>
>>
>>
>> --
>> View this message in context:
>> http://elasticsearch-users.115913.n3.nabble.com/Text-Categorization-in-ES-tp4050194p4050349.html
>> Sent from the ElasticSearch Users mailing list archive at Nabble.com.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/1393221646984-4050349.post%40n3.nabble.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE8hAPZvuXgPMwbRRdM%3DxuMNpff7q5LrdpnSeMM_A--kw%40mail.gmail.com.
>
> For more options, visit https://groups.google.com/groups/opt_out.

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAM21Rt-2L7%3D0TZY_%3D4RaqJ2UOxd9%2B3TsdS38-GexOp5HWzu-7Q%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.

Question : Field behavior in Nested object

[Jun Ohtani]

Hi all,

I am using the nested object.
There is strange behavior of field in nested object.

All sample mapping and data : https://gist.github.com/johtani/9183848

1. _source store
2. book.title is not stored
3. book.contents is stored

If fields parameter specify “book.title” and “book.contents”, I get only 
“book.title”.
This data is returned from _source.

Why did not return “book.contents”?
Should I use _source instead of store=“true" ?

Regards,
Jun


Jun Ohtani
joht...@gmail.com
blog : http://blog.johtani.info
twitter : http://twitter.com/johtani



signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: Text Categorization in ES

[joergpra...@gmail.com]

Install the carrot2 plugin and see if it fits your requirments:
http://download.carrotsearch.com/lingo3g/manual/#section.es

Jörg


On Mon, Feb 24, 2014 at 7:00 AM, prashant.agrawal <
prashant.agra...@paladion.net> wrote:

> Hi Hannes,
>
> Thanks for the info , also I came to know about lingo3G/Carrot Search.
> So whether that could also be a solution for that?
>
>
>
> --
> View this message in context:
> http://elasticsearch-users.115913.n3.nabble.com/Text-Categorization-in-ES-tp4050194p4050349.html
> Sent from the ElasticSearch Users mailing list archive at Nabble.com.
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/1393221646984-4050349.post%40n3.nabble.com
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE8hAPZvuXgPMwbRRdM%3DxuMNpff7q5LrdpnSeMM_A--kw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.