Re: Completion Suggester : Support for numeric value completion suggestion

2014-08-12 Thread Alexander Reelsen
Hey, The default analyzer used in the completion suggester is the simple one, which strips out numbers. --Alex On Thu, Aug 7, 2014 at 1:43 PM, Hemant hemant19...@gmail.com wrote: Hello, I was trying the following use cases using completion suggester - 1. Suggest Song on search song by id

Re: How to disable DELETE API

2014-08-06 Thread Alexander Reelsen
why we have setting in .yml file for Access-Control-Allow-Methods: OPTIONS, POST, GET, PUT . This raises question on the configuration usage . On Mon, Aug 4, 2014 at 3:31 AM, Alexander Reelsen a...@spinscale.de wrote: Hi, Elasticsearch currently does not allow you to enable/disable

Re: not able to install elasticsearch using reporsitory

2014-08-06 Thread Alexander Reelsen
Hey, looks like you have a slow network connection or the .org was not reachable when you tried it. Can you try to download the RPM directly and see if it works? Use http://packages.elasticsearch.org/elasticsearch/1.3/centos/elasticsearch-1.3.1.noarch.rpm - which works for me good at the moment

Re: Context suggest display results for all the categories (mimic the completion suggester behavior)

2014-08-05 Thread Alexander Reelsen
Hey, maybe the new transform feature could help here to create a default value (just an idea, havent tested it). See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-transform.html#mapping-transform --Alex On Tue, Aug 5, 2014 at 10:13 AM, Tihomir Lichev

Re: Dropped HTTP Connections when Indexing

2014-08-04 Thread Alexander Reelsen
Hey, are you using HTTP keep alive connections? If not consider switching to them, as reopening a new TCP connection not only results in high latencies but also removes file handle resources from the elasticsearch process (the number of open files). if your client/language does not support this,

Re: IP geolocation without Logstash

2014-08-04 Thread Alexander Reelsen
Hey, this is exactly what logstash is for, so you may want to give it a try, as it is already there. :-) Also you can use the geoip filter to extract the ip address from the header as well, granted you log that one. --Alex On Sat, Jul 19, 2014 at 6:26 AM, Otis Gospodnetic

Re: scaling out percolator performance?

2014-08-04 Thread Alexander Reelsen
Hey, there are several possibilties to increase performance. First you can have your own index for your percolation queries, so it scales independently from your data (there are use-cases where people do not have increasing data, but ever increasing amount of percolators). Second you can filter

Re: How do I query phrases?

2014-08-04 Thread Alexander Reelsen
Hey, maybe, exactly like that. However you should provide some more information and sample queries, so other people can help you. --Alex On Tue, Jul 22, 2014 at 7:55 PM, IronMike sabdall...@gmail.com wrote: How can I combine phrases like flight attendant AND/OR/NOT Boeing airlines ? --

Re: Node Client with bulk request indefinitely blocked thread when ClusterBlockException is being thrown

2014-08-04 Thread Alexander Reelsen
Hey, Just a remote guess without knowing more: On your client side, the exception is wrapped, so you need to unwrap it first. --Alex On Wed, Jul 23, 2014 at 9:47 AM, Cosmin-Radu Vasii cosminradu.va...@gmail.com wrote: I am using the dataless NodeClient to connect to my cluster (version is

Re: How to solve failed to merge org.apache.lucene.store.AlreadyClosedException: this Directory is closed

2014-08-04 Thread Alexander Reelsen
Hey, can you add some more information here? What are doing when this happens? Heavy indexing? Did you check the logfiles before that? Are there exceptions? What elasticsearch version are you using? What JVM version are you using? A bit of context would be great! --Alex On Sun, Jul 27, 2014

Re: Get children count using has_child query

2014-08-04 Thread Alexander Reelsen
Hey min/max_children is part of the query to define how many children a matching parent document may have, but it is not exposed. A possible solution would be to execute a count query with a has_parent on the matched parent. --Alex On Tue, Jul 29, 2014 at 4:20 AM, Maxime Nay

Re: Data too large error

2014-08-04 Thread Alexander Reelsen
Hey, in order for kibana to work, elasticsearch needs to load all @timestamp values into memory. This exception tries to tell the user, that loading this into memory would result in an Out-of-Memory exception, so elasticsearch aborts this request. You could start elasticsearch with more memory

Re: Failure connecting to x.x.x.x: dial tcp x.x.x.x:x: connection refused

2014-08-04 Thread Alexander Reelsen
Hey this means, that the logstash forwarder could not connect to its configured endpoint, because that endpoint does not seem to run. Check if that service is up and running or maybe just misconfigured. --Alex On Thu, Jul 31, 2014 at 2:56 PM, Indirajith V indiraji...@gmail.com wrote: Hi

Re: timestamp format

2014-08-04 Thread Alexander Reelsen
Hey, from the top of my head, joda (the underlying library used in Elasticsearch for date handling) only does millisecond and so does Elasticsearch. Maybe you can break down the micro to milliseconds and try again? Also, you might need to create your own date format, see

Re: Completion Suggestion Not working as documented.

2014-08-04 Thread Alexander Reelsen
hey, you can configure the count of the retured elements using the size parameter, see http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-suggesters.html#search-suggesters or the examples

Re: Context suggest display results for all the categories (mimic the completion suggester behavior)

2014-08-04 Thread Alexander Reelsen
Hey, you could simply index an 'all' color field value for every document, and then use that for your suggestions? --Alex On Mon, Aug 4, 2014 at 10:50 AM, Tihomir Lichev shot...@gmail.com wrote: Hi, Is there a way to tell the context suggester to display results for all the categories ? I

Re: Node Client with bulk request indefinitely blocked thread when ClusterBlockException is being thrown

2014-08-04 Thread Alexander Reelsen
Hi, on a second thought, you may have ran into this one as well: https://github.com/elasticsearch/elasticsearch/issues/7086 On Mon, Aug 4, 2014 at 10:27 AM, Alexander Reelsen a...@spinscale.de wrote: Hey, Just a remote guess without knowing more: On your client side, the exception

Re: Need mapping (or query) tips

2014-07-28 Thread Alexander Reelsen
Hey, you might want to use the phrase match query and check out its slop parameter for your second requirement. See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-match-query.html#_phrase --Alex On Wed, Jul 16, 2014 at 10:49 AM, Loïc Wenkin

Re: Efficiency of search vs get

2014-07-28 Thread Alexander Reelsen
Hey, it is sufficient to set the routing via setRouting in the Java API... in case of doubts, you can always check the RestActions in the source and see how they do it... --Alex On Wed, Jul 16, 2014 at 7:28 PM, thale jacobs thalejac...@gmail.com wrote: From the example:

Re: Support altitude in geospatial data type?

2014-07-18 Thread Alexander Reelsen
Hey, there is no support in geo_point or geo_shape for altitude. But what prevents you from simply adding another field to your document and using this in your queries? --Alex On Tue, Jul 1, 2014 at 8:46 PM, akee gogogo akeegog...@gmail.com wrote: Does Elasticsearch support altitude in

Re: How to search the records with locations all in a polygon or multiPolygon ??

2014-07-18 Thread Alexander Reelsen
Hey, if you want to search shapes inside of shapes (where a point is a shape as well), you should use the geo_shape type, not the geo_point one. Also use the geo_shape query and filters. --Alex On Thu, Jul 3, 2014 at 8:36 PM, akun baru patihgajahmad...@gmail.com wrote: I add the mappings

Re: _all not pulling subfields

2014-07-18 Thread Alexander Reelsen
Hey, from my birds eye view this should work. Can you create a fully fledged example (including index creation, mapping creation, document creation), so one can replay this. And tell your ES version along, if you have any special configuration going on. --Alex On Thu, Jul 10, 2014 at 8:31 PM,

Re: Is it possible to pass 2 queries as params to a script

2014-07-18 Thread Alexander Reelsen
Hey, no, this should be done on the client side instead of a script in Elasticsearch. --Alex On Thu, Jul 10, 2014 at 6:08 PM, Adrian eamoc...@gmail.com wrote: I want to compare the results of 2 queries. Is it possible to do 2 queries in one go and pass the result from query 1 as a param

Re: best reindex strategy to change only the number of Shards

2014-07-18 Thread Alexander Reelsen
Hey, maybe there is no need to reindex? Cant you just index all your new data into another index and use an alias to search on both indices? You might want to watch https://www.youtube.com/watch?v=gBOhCNcjC7k --Alex On Sat, Jul 12, 2014 at 3:19 PM, Chris Berry chriswbe...@gmail.com wrote:

Re: How to connect Tribe Node with kibana

2014-06-30 Thread Alexander Reelsen
Hey, this looks like a valid response, if you did not have defined any aliases. Do you get an error when connecting with Kibana? --Alex On Tue, Jun 24, 2014 at 9:46 AM, Rukshan Kothwala blog8...@gmail.com wrote: Hi, I want to combine different ES cluster and get one kibana dashboard for

Re: search across multiple children by single query

2014-06-30 Thread Alexander Reelsen
Hey, could you maybe explain where the problem is or which queries you already tried so far? A possible solution might be to write a has has_child query, that contains boolean query, which contains to two must clauses, like this: has_child : { bool : { must : [ term : {}, query_string : {} ] } }

Re: Disabling _all-Field but keep Netflow-Events searchable

2014-06-30 Thread Alexander Reelsen
Hey, you can set the index.query.default_field in the mapping to circumvent this, see http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-all-field.html#mapping-all-field --Alex On Tue, Jun 24, 2014 at 12:39 PM, horst knete baduncl...@hotmail.de wrote: Hey guys,

Re: How would ElasticSearch do it ?

2014-06-30 Thread Alexander Reelsen
Hey, for one, the Update API might help you, see http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-update.html#docs-update However, Elasticsearch does not keep a revision of older versions. This is something you need to implement in your application. --Alex On Tue,

Re: Can you do date histogram with specified number of buckets?

2014-06-30 Thread Alexander Reelsen
Hey, right now the interval needs to be defined on query time and is never dynamically calculated. The main reason is, that for each new document, you probably have to expand the buckets or you have to have a first to predetermine the oldest and youngest date of the dataset, which requires an

Re: Snapshot Restore in a cluster of two nodes

2014-06-30 Thread Alexander Reelsen
:12:49 UTC+2 schrieb Alexander Reelsen: Hey, can you be more precise and create a fully fledged example (generating the repository, executing the snapshot on cluster one, executing restore on cluster 2, etc) and include the concrete error message in order to find out what 'the process breaks

Re: RecoveryFailedException while adding a node during Elasticsearch upgrade from 0.2.0 to 1.2.1

2014-06-30 Thread Alexander Reelsen
Hey, can you check your log files and have access to to the full stack trace of the NullPointerException in order to find out what is going on? Thanks! --Alex On Wed, Jun 25, 2014 at 1:14 AM, anurag naidu anuragna...@gmail.com wrote: We are upgrading our ES server from 0.2.0 to 1.2.1 using

Re: Upgrade Path from ES 0.20.2 to 0.90.13 and finally to 1.2.1

2014-06-30 Thread Alexander Reelsen
Hey, when you mentioned you upgraded to 1.2.1, the version shown on startup is still 0.90.13? Is that on purpose and I am missing context here? Did you make sure, that all elasticsearch instances run on the same version and the same JVM version as well? --Alex On Wed, Jun 25, 2014 at 1:50 AM,

Re: Error for elasticsearch php api

2014-06-30 Thread Alexander Reelsen
Hey, my PHP skills are next to none, but if you dont clear out that params array, dont you try to execute the search operation with exactly the same as the index operation (which will very likely fail, as id is not a valid parameter for a search, as mentioned in the error message)? --Alex On

Re: Exact duplicate results (same _id) for a search query. Is this a bug?

2014-06-30 Thread Alexander Reelsen
Hey, couple of possible reasons here. You might have used routing accidentally? You can use the explain flag in your query to find out the shard, the document lives in, see this example, which returns the shard id for both indexed document DELETE /foo PUT /foo/bar/1?routing=2 { foo : bar }

Re: Elasticsearch wont start on Ubuntu 14.04

2014-06-30 Thread Alexander Reelsen
Hey, if you put something like 'set -x' into /etc/init.d/elasticsearch (not in the first line, but somewhere below that), you can see, how elasticsearch is started (and which parameters from the default file are used). Can you paste that output here, so we can debug further? Cant say if it is a

Re: ES fails to load plugin - what am I missing?

2014-06-30 Thread Alexander Reelsen
Hey, glad you solved it. Just for future reference: you can use the nodes info API to see your installed plugins like this: http://localhost:9200/_nodes/plugins If you have further problems with restarted using the init script, which is packaged with ES, dont hesitate to create an issue!

Re: enabling scripting for installed scripts only

2014-06-30 Thread Alexander Reelsen
Hey, I just tested with 1.2.1, and even if dynamic scripting is disabled, you can still execute locally stored scripts. See the example in http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-scripting.html#modules-scripting --Alex On Fri, Jun 27, 2014 at 8:44 PM,

Re: Extremely slow indexing -- java throwing http excetion errors

2014-06-20 Thread Alexander Reelsen
Hey. judging from the exception this looks like an unstable network connection? Are you using persistent HTTP connections? Pinging the nodes by each other is not a problem I guess? --Alex On Thu, Jun 19, 2014 at 12:12 AM, alekjouhar...@gmail.com wrote: Hello all, So here's the issue, our

Re: Clarification on has_child filter memory requirements

2014-06-20 Thread Alexander Reelsen
Hey, not all parent documents (and not the data), just their ids. Still this can accumulate, which is the reason why you should monitor the size of that data structure (exposed in the nodes stats). Hope that helps. --Alex On Thu, Jun 19, 2014 at 6:03 AM, Drew Kutcharian d...@venarc.com

Re: Losing data after Elasticsearch restart

2014-06-20 Thread Alexander Reelsen
: InvalidAliasNameException[[b7a76aa06cfd4048987d1117f3e0433a] Invalid alias name [1a4077872e41c0634cee780c1e5fc263bdd5f14b15ac9239480547ab2d3601eb], Unknown alias name was passed to alias Filter]; ]] Let us know.. Thanks, Rohit On Mon, Jun 16, 2014 at 6:13 AM, Alexander Reelsen a...@spinscale.de wrote: Hey

Re: Count request does not support [filter]. Why?

2014-06-20 Thread Alexander Reelsen
Hey, not a hundred percent sure, what you mean here. The post_filter setting? There are two possibilities: Either use the search_type=count or use a filtered query in the count API. See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-count.html

Re: Very frequent ES OOM's potential segment merge problems

2014-06-20 Thread Alexander Reelsen
Hey, can you provide more information about the OOM exception? Also you should use the nodes stats API to monitor your system, so you can maybe easily spot, where this memory consumption stems from. Also, are you just indexing or doing searches/queries/gets as well? --Alex On Thu, Jun 19,

Re: ElasticSearch Node.Client Options

2014-06-20 Thread Alexander Reelsen
) at org.elasticsearch.transport.TransportService.sendRequest( TransportService.java:189) ... 8 more On Wednesday, 18 June 2014 14:51:56 UTC-7, VB wrote: Thanks Alex for responding to my post from your busy schedule. Please find attached screenshot of eclipse memory analyzer On Monday, 16 June 2014 04:58:19 UTC-7, Alexander Reelsen wrote

Re: Snapshot Restore in a cluster of two nodes

2014-06-20 Thread Alexander Reelsen
Hey, can you be more precise and create a fully fledged example (generating the repository, executing the snapshot on cluster one, executing restore on cluster 2, etc) and include the concrete error message in order to find out what 'the process breaks' means here? Also provide info about

Re: Get X word before and after search word

2014-06-19 Thread Alexander Reelsen
Hey, you potentially could use the termvectors API for this, see http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-termvectors.html Not sure, if this is excalty, what you are after... maybe explain your use-case a bit more --Alex On Tue, Jun 17, 2014 at 2:19 PM,

Re: Strange behavior: same id in different types

2014-06-19 Thread Alexander Reelsen
Hey, your request is already ambigous. You specify a single ID, but expect two results to return, as you did not specify a type. However to identify a document in a unique fashion, you need the tuple of index/type/id, of which a type is missing here. So, either specify all three, or maybe

Re: Regarding autosuggest completion type of field.

2014-06-19 Thread Alexander Reelsen
Hey, just a wild guess: Are you having more than one type in your mapping and not every type has this field configured as a completion field? If it is not this cause, can you create a full blown recreation like mentioned in http://elasticsearch.org/help Thanks a lot! --Alex On Wed, Jun 18,

Re: ElasticsearchIllegalStateException: Maximum number of bytes allocated exceeded

2014-06-17 Thread Alexander Reelsen
Hey, do you have a full stack trace in one of your log files? Also, are you creating lots of new connections all the time and then stop them abruptly (just a wild guess here)? --Alex On Mon, Jun 16, 2014 at 11:18 PM, Sriram Kannan kes...@gmail.com wrote: I am getting this error constantly

Re: Could not lock IndexWriter isLocked [false]

2014-06-16 Thread Alexander Reelsen
Hey, what elasticsearch version are you using? Judging from the directory I dont think you are using NFS, right? Are you running multiple instances locally? Have you shutdown elasticsearch properly so that no other instance is lingering around (you can use jps or or to check) --Alex On Mon,

Re: Detective wanted: Green cluster but query timeout unless changing preference to local.

2014-06-16 Thread Alexander Reelsen
Hey, you should check your logfiles on each of the nodes. Maybe they disconnected from each other (and reconnect all the time), so that you get problems when you execute your requests round robin. Using curl from one node to another does not show any problems? They are reachable all the time (at

Re: Term Suggester Usage

2014-06-16 Thread Alexander Reelsen
Hey, the term suggester takes a query like search is awesome and returns suggestions for each of the terms, so three different suggestions are returned here. You maybe want to take a look at the completion suggester, see http://www.elasticsearch.org/blog/you-complete-me/

Re: Configure connections using TransportClient

2014-06-16 Thread Alexander Reelsen
Hey, those configuration parameters still exist (there are more for ping and recovery). Wondering why exactly you want to change those? Do you have any specific problems going on? --Alex On Tue, Jun 3, 2014 at 7:47 AM, Srinath C srinat...@gmail.com wrote: Hi, I wasn't able to figure

Re: ElasticSearch Node.Client Options

2014-06-16 Thread Alexander Reelsen
Hey, client nodes do execute a reduce phase (merging all the results from a search in order to find out the top-n results), and if you do deep pagination this can take some memory. There is no data stored on a client node (as node.data property is always set to false). Do you have a stack trace

Re: Index template requires settings object even if its value is empty

2014-06-16 Thread Alexander Reelsen
Hey, which ES version are you using? Seems to work with the latest version. You can also use the index template API, so you do not have to fiddle with local files (and copy them when adding new nodes). PUT _template/automap { template: *, mappings: { _default_: { numeric_detection:

Re: Prefix search on integer field

2014-06-16 Thread Alexander Reelsen
Hey, not sure here, whats your expectation of the data being returned? If you search for 1, should 1, 10, 111 be returned (otherwise a range query would make more sense)? If so, it might make more sense to have an additional multi field as a not_analyzed string and use that one for these kind of

Re: elasticsearch filter query with time range using java api

2014-06-16 Thread Alexander Reelsen
Hey, did you try something along the lines FilterBuilders.rangeFilter(@timestamp).from(now).to(now - 5d) - did that pose problems? make sure you are reading this paragraph about date range filters: http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/filter-caching.html --Alex

Re: MapperParsingException on data that should not be parsed but caught by my plugin

2014-06-16 Thread Alexander Reelsen
Hey, first you should really upgrade ES, 0.90.3 is not the freshest one and there have been many improvements. Second, logging this as DEBUG makes a lot of sense, as otherwise you logfiles might explode, just because you are indexing a lot of bogus documents. You source of truth should not be the

Re: Geo Distance Facet - ElasticsearchParseException

2014-06-16 Thread Alexander Reelsen
Hey, your ranges specification is broken, you need to use an array of objects. Also, please try to use aggregations if possible, as they are the successor of facets. --Alex On Thu, Jun 5, 2014 at 10:37 AM, Munjal Dhamecha masterm...@gmail.com wrote: Hello All, I've been facing problem

Re: Find all the geoshapes that insersects with a given latitude/longitude

2014-06-16 Thread Alexander Reelsen
Hey, this looks like a problem, because you are using an _id object, I filed an issue here: https://github.com/elasticsearch/elasticsearch/issues/6514 You can just specify ids manually or make sure, that you _id field is empty for now. --Alex On Sun, Jun 8, 2014 at 9:42 PM, Vidal Chriqui

Re: Elasticsearch field mapping, dynamic_templates

2014-06-16 Thread Alexander Reelsen
Hey, using your curl call you only set the mapping for existing indices, but not for newly created ones. You may want to consider using index templates for this specific field. --Alex On Mon, Jun 9, 2014 at 4:45 PM, sirkubax jakubxmuszyn...@googlemail.com wrote: As I did mention here

Re: Accessing Search Templates via Rest

2014-06-16 Thread Alexander Reelsen
Hey, no, this is not yet possible, but this will be added sooner or later as the search template API should behave like any other API. --Alex On Fri, Jun 13, 2014 at 9:51 AM, Sebastian Gräser gree...@googlemail.com wrote: so i guess its not possible? Am Dienstag, 10. Juni 2014 16:58:31

Re: does minimum_master_nodes include ones self?

2014-06-16 Thread Alexander Reelsen
Hey, yes, it is the number of nodes in a cluster, which are able to become a master node. If node A and node B are configured that way, both are counted, possibly including themselves. Makes sense? --Alex On Mon, Jun 16, 2014 at 5:11 PM, bitsofinf...@gmail.com wrote: running 1.2.1 If a

Re: Scripts reload on demand

2014-06-01 Thread Alexander Reelsen
Hey, can you still reproduce this with the latest 1.1.2 (or even 1.2.0) release? If so, please file a bug report (and all the steps to reproduce). Thanks! --Alex On Thu, May 8, 2014 at 5:45 PM, Thomas thomas.bo...@gmail.com wrote: Hi, I was wondering whether there is a way to reload the

Re: Suggest Query with additional condition

2014-06-01 Thread Alexander Reelsen
Hey, please try the new context suggester in Elasticsearch 1.2.0. See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/suggester-context.html#suggester-context --Alex On Tue, May 13, 2014 at 12:06 PM, Raghavendar T S raghav280...@gmail.com wrote: Hi I am trying to

Re: Elastic search High threads , 100% utilization of non heap memory.

2014-06-01 Thread Alexander Reelsen
Hey, you should use the nodes stats and nodes info APIs to monitor elasticsearch (or marvel, if its commercially viable for you). Also, where are you seing the 100% utilization? Not memory but CPU wise - might make sense to add some info here. --Alex On Fri, May 23, 2014 at 7:18 AM, srikanth

Re: All shards failed for phase: [query] message

2014-06-01 Thread Alexander Reelsen
hey, can you show a test which reproduces this behaviour? This means, that all of your searches against all shards seem to have failed. Any messages before that/after that or on other nodes (usually you should be able to spot a suggestion somewhere, maybe your index is not yet ready to be

Re: Indexing nonstandard geo_point field.

2014-06-01 Thread Alexander Reelsen
Hey, you could index this as a geo shape (as this is valid GeoJSON). If you really need the functionality for a geo_point, you need to change the structure of the data. --Alex On Sat, May 31, 2014 at 3:36 PM, Brian Thomas mynamei...@gmail.com wrote: I am new to Elasticsearch and I am trying

Re: Mapping created using Template does not work

2014-05-09 Thread Alexander Reelsen
Hey, can you just take some sample data and index it into elasticsearch manually and see if that works? --Alex On Thu, May 1, 2014 at 1:53 AM, Deepak Jha dkjhan...@gmail.com wrote: Hi, I have setup ELK stack and I am going by default index name, which is logstash-.MM.DD . Since this

Re: External datasets in ES

2014-05-09 Thread Alexander Reelsen
the concept of tribes? *From:* elasticsearch@googlegroups.com [mailto: elasticsearch@googlegroups.com] *On Behalf Of *Alexander Reelsen *Sent:* 05 May 2014 12:17 *To:* elasticsearch@googlegroups.com *Subject:* Re: External datasets in ES Hey, this would be a bit more tricky, as it requires

Re: randomly no response for search query

2014-05-05 Thread Alexander Reelsen
Hey, very hard to tell what has been happening here without more information. There has been a fix for this added recently, see https://github.com/elasticsearch/elasticsearch/commit/3ea1d00869a33b1e36984bc8c38e03fa16829778 However this only happens when you get rejected requests in your thread

Re: Snapshot not listed after nodes are restarted

2014-05-05 Thread Alexander Reelsen
Hey, did you make sure, that every node has access to that directory specified in the repository settings. This has to be a network share, where every node in your running cluster is allowed to write to. --Alex On Mon, Apr 28, 2014 at 6:03 PM, Hasitha hasitha...@gmail.com wrote: Hi I've a

Re: error message in logfile : Parse Failure [No mapping found for [@timestamp] in order to sort on]

2014-05-05 Thread Alexander Reelsen
Hey, the problem is, that you are trying to search over all indices. One of those indices contains your kibana dashboards - which is just fine. However that index (named kibana-int) does not contain a timestamp field. The most simple solution might be, to change your search to not search in that

Re: Does not match words exactly

2014-05-05 Thread Alexander Reelsen
Hey, in order for others to follow it might make a lot of sense to provide a complete example, which can be used by others to rebuild your problem (instead of just supplying a query), see http://www.elasticsearch.org/help Thanks! --Alex On Tue, Apr 29, 2014 at 4:08 PM, Brian Lamb

Re: Shards become unallocated during indexing

2014-05-05 Thread Alexander Reelsen
Hey, this is very hard to tell without further debugging/information. Can you check your logfiles, if there are any exceptions popping up? Can you check the logfile of your master node, if node joins/leaves or even master elections happen? Also you should add monitoring to your infrastructure

Re: Shard copying performance

2014-05-05 Thread Alexander Reelsen
Hey, you could change your default loglevel to find out, if those settings are actually applied (either DEBUG or TRACE). Depending on the elasticsearch version you are using, you might want to try with a lower-cased setting of max_bytes_per_sec and set it to 250mb. Also, can you show the

Re: Relationship between No of nodes , shards and replica

2014-05-05 Thread Alexander Reelsen
Hey, there is no predefined formula. The reason for this is, that it is very use-case dependent. If your data is really important, you might want to put it onto every node of your cluster. If your data is too big to be put onto a single node, you need to split it among several nodes, but still

Re: Mapping created using Template does not work

2014-05-05 Thread Alexander Reelsen
Hey, can you include a full example including data you have been indexing? Or include error messages, as you wrote you cannot index any data anymore? Also, your JSON is not valid as the mapping for the class field contains a comma too much. --Alex On Thu, May 1, 2014 at 1:53 AM, Deepak Jha

Re: how to aggregate by metadata (types/field names)?

2014-05-05 Thread Alexander Reelsen
Hey, you could take a look at the terms aggregation using the _type field, which contains the type information. See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#search-aggregations-bucket-terms-aggregation --Alex On

Re: External datasets in ES

2014-05-05 Thread Alexander Reelsen
Hey, this would be a bit more tricky, as it requires you to merge two events (the external dataset and your live visitor stats) into a single event as a sort of preprocessing step. I think I would start with the geoip support from logstash and use your apache logs, which at least allows you to

Re: Using different analyzers for a single field (depends on condition)

2014-05-05 Thread Alexander Reelsen
Hey, using different analyzers in one document, which are extracted from the document is not possible. However if you have one document per language, you could use the _analyzer capability for this, see

Re: geo_polygon not handling polygons that cross the date line properly?

2014-04-28 Thread Alexander Reelsen
Hey there, please file a github issue for this one, to make sure it does not get lost (havent checked the details, but looks like a valid bug from my birds eye view). Thanks a lot! --Alex On Mon, Apr 28, 2014 at 7:41 PM, Pablo Castro pabl...@gmail.com wrote: Using Elasticsearch 1.1.1. I'm

Re: Max latency between nodes

2014-04-25 Thread Alexander Reelsen
Hey, is there any reason in the logfile of the master node, why it was deelected? (network outage as well)? Did you give your master nodes also a huge heap which could cause long outages during GC? --Alex On Mon, Apr 21, 2014 at 5:51 PM, skik2s...@gmail.com wrote: We currently are running

Re: Automatically build `input` for `completion` fields?

2014-04-25 Thread Alexander Reelsen
`message_suggest.input` into array, but it doesn't seem to do it. What are they for then? On Mon, Apr 7, 2014 at 11:20 AM, Alexander Reelsen a...@spinscale.de wrote: Hey, there is no automation for this. The main reason why your solution might work in your specific use-case is, that you do not have

Re: Is concatenated search possible, where whitespace may or may not exist?

2014-04-25 Thread Alexander Reelsen
Hey, the word compound filter might help in your case. See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/analysis-compound-word-tokenfilter.html#analysis-compound-word-tokenfilter --Alex On Fri, Apr 25, 2014 at 5:04 PM, Raja Akhtar raja.akh...@gmail.com wrote: Hi

Re: action.admin.indices.create failed to create

2014-04-22 Thread Alexander Reelsen
Hey, from the elasticsearch exception it looks as if you did not specify an index name to create. Not sure how the scala client works.. --Alex On Mon, Apr 21, 2014 at 7:37 PM, miki haiat miko5...@gmail.com wrote: HI , I using elastic4s for an api client , i cant index anything im

Re: AutoCompletion Suggester - Duplicate record in suggestion return

2014-04-22 Thread Alexander Reelsen
21, 2014 7:26:13 PM UTC+7, Alexander Reelsen wrote: Hey, the output is used to unify the search results, otherwise the input is used. The payload itself is just meta information. The main reason, why you see the suggestion twice is, that even though a document is deleted and cannot be found

Re: Java Node api: isStarted() isStopped() methods needed.

2014-04-21 Thread Alexander Reelsen
Hey maybe isClosed() can help you here? --Alex On Thu, Apr 10, 2014 at 5:40 PM, Doru Sular doru.su...@gmail.com wrote: Hi guys, I am using elasticsearch embedded in my webapplication. I really need the methods isStarted and isStopped on Node api. Currently I wait in my current thread

Re: sbuqueries and distinct counts.

2014-04-21 Thread Alexander Reelsen
Hey, you may want to provide a gist with some sample data, so people can follow.. Also you might need to use the value_count aggregation somewhere, to make sure you are only using the counts, but this really depends on your data model and the data being indexed, so full gist might help a lot.

Re: 13mil documents and Elasticsearch status=red after server restart

2014-04-21 Thread Alexander Reelsen
Hey, can you check the log files for error messages (possibly also before the restart)? Also the recovery API may help you do identify if you still recovering shards or recovery is finished and you had problems while recovering the data (again, checking the log files should help a lot) --Alex

Re: Question about Scroll time

2014-04-21 Thread Alexander Reelsen
Hey, if that specified time expires, file handles are closed and the state of time you used to dump out all your data is no longer preserved. While running scroll requests in the background, new lucene segments might be created or deleted, and a scroll requests keeps those deleted file handles

Re: How are missing values treated with new cardinality feature

2014-04-21 Thread Alexander Reelsen
Hey only real values are counted, missing is not counted as 1 IIRC --Alex On Fri, Apr 11, 2014 at 6:30 AM, H Sun sunh11...@gmail.com wrote: Hi, With the new cardinality feature introduced in 1.1.0. How are missing values treated? For example, in a data set with a field city, are all

Re: regexp not working in head plugin

2014-04-21 Thread Alexander Reelsen
Hey, keep in mind, that the regexp query is a term query with regex support, so having analyzed fields is a problem, try the following example: DELETE /map PUT /map PUT /map/item/_mapping { properties: { combined : { type: string, index: not_analyzed} } } GET /map/item/_mapping PUT

Re: Find parents with most number of children

2014-04-21 Thread Alexander Reelsen
Hey, you would execute the facet query first and then execute a query to get all the documents, as you need the output of the first query as input for the second. --Alex On Fri, Apr 11, 2014 at 11:34 AM, Ludwig Magnusson ludwig.magnus...@gmail.com wrote: I have an index where the type

Re: sending failed shard

2014-04-21 Thread Alexander Reelsen
Hey, has there a stacktrace been logged below this message? Also the elasticsearch version would be interesting.. --Alex On Fri, Apr 11, 2014 at 8:14 PM, David Chamard chamm...@gmail.com wrote: Hi, I am currently having an issue that causes data loss with failed shards. At first sight it

Re: How to dump last 15 minutes of logstash data from elasticsearch?

2014-04-21 Thread Alexander Reelsen
Hey, you could use a scroll request for this task with a specific query. See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-scroll.html#search-request-scroll As this simply returns JSON, you could use another script to import it into another elasticearch

Re: Question on search on _source

2014-04-21 Thread Alexander Reelsen
Hey, elasticsearch operates on a per-document base. So if only one of the documents in the searchitems array is ifpresent=0, the whole document is being returned. If you want different behaviour, you could take a look at parent/child or nested documents or rethink your indexing strategy,

Re: Completion Suggester on existing index and fields

2014-04-21 Thread Alexander Reelsen
Hey this is currently not possible. From my experience, just adding arbitrary suggestions with the same weight is not resulting in a good user experience. Check out amazon for example, if you enter iphone, the first match is definately ranked in the suggestions, as this is not an iphone case or

Re: java 8, elasticsearch, and MVEL

2014-04-21 Thread Alexander Reelsen
Hey, this commits upgrades mvel, that seems to have fixed the java8 issues (still requires more testing on our side though): https://github.com/elasticsearch/elasticsearch/commit/21a36678883c159e50a03b76309d3da2a8e5d7b4 IIRC this bug has also been fixed in the new MVEL version:

Re: Performance of Indexed-Shape Queries Vs Geoshape Queries

2014-04-21 Thread Alexander Reelsen
Hey, the main difference is basically the network overhead. What happens behind the curtains is that a GET request for the shape is being executed if you specify it in the request and then this shape is used instead of the provided one. Makes sense? --Alex On Tue, Apr 15, 2014 at 6:50 AM,

Re: Local testing not working

2014-04-21 Thread Alexander Reelsen
Hey, I highly recommend reusing the existing elasticsearch test infrastructure, as it cleans up behind some test runs and also creates everything upfront, see http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/randomized-testing.html#randomized-testing The integration tests

  1   2   3   >