Re: Elastic Search Best Client

I'm using Marvel / Sense every day. I like the completion feature. David. Le 6 oct. 2014 à 07:56, naveen gayar navind...@gmail.com a écrit : Hi I wish to practice the usage of the following: 1. Term 2. wildcard 3. Prefix 4. fuzzy 5. range 6. query_string 7.text I am using a

Re: Get analyzer from client

Hi, Thank you for all of those suggestions. I think this will work for me. -- Paweł Róg On Mon, Oct 6, 2014 at 4:54 AM, Ivan Brusic i...@brusic.com wrote: It is definitely possible. You can create the analysis service locally and query it for the appropriate analyzers. I use this method the

able to filter a field from document

I am wondering if it is possible to filter a field from a document using aliases. For example if I have an index with documents such as { user : test, secret_field: value, post_date : 2014-06-05T09:12:12, message : trying out Elasticsearch } Now if I want to create an alias for

Re: able to filter a field from document

I'm not sure I understood what you are looking for. May be exclude is what you want? See: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-source-field.html#include-exclude Le 6 oct. 2014 à 09:02, Gurvinder Singh gurvindersinghdah...@gmail.com a écrit : I am

Error: No config files found: /etc/logstash/conf.d

Hey, when i start logstash as a service /etc/init.d/logstash start, i get the Error Message in the topic in file logstash.stdout. I have a logstash config file in the directory /etc/logstash/conf.d/server.conf. So i don't know why logstash doesnt find the file. Can anyone help me? Greets

Exists filter on fields with index_name not getting all documents

Hi guys, How to reproduce: https://gist.github.com/rmat0n/7b6f4f0398dc5f0966b4 I created 2 mapping with a testtext field on both, the difference is that each mapping have a different index_name for the field: the first mapping use 1.testtext and the second mapping use 2.testtext. Then putting

Architecture to prevent slow queries

I'm using ES for searching for events based on date and geo distance, as well as textual content. I'm also using logstash for handling app logging and analytics. I've noticed after I have millions of records from logging/analytics, the events search starts slowing down. I'm currently using

Re: Architecture to prevent slow queries

Also, I would be grateful if someone could point me to some good general information about this kind of thing. On Monday, October 6, 2014 4:28:31 AM UTC-4, Michael Irwin wrote: I'm using ES for searching for events based on date and geo distance, as well as textual content. I'm also using

Re: Architecture to prevent slow queries

Just add nodes. That's all :) Jörg On Mon, Oct 6, 2014 at 10:30 AM, Michael Irwin m...@livej.am wrote: Also, I would be grateful if someone could point me to some good general information about this kind of thing. On Monday, October 6, 2014 4:28:31 AM UTC-4, Michael Irwin wrote: I'm using

Re: Recommendation on reading the heart of the code.

When it comes to understanding any large code base your approach will always depend on your own preference. Things that I have found useful​ to get started: - Instead of working on the core inner workings of the code start with checking out the project, building it and using it. Build small demo

Re: Recommendation on reading the heart of the code.

A great start is by studying and writing plugins. Elasticsearch is one of the rare masterpieces of software that allow to plug in code that you have authored to extend functionality, without forking the main code base. There are a lot of plugins out there with small code base and easy to study,

Mapping versioning by using doc_type

Hi, We've constantly facing with updates/changes in our document mapping during development, so we've come up with a idea of using `doc_type` for versioning. Basically we start from a `doc_type` like: `thing_0` as the start point. And each time we have some changes in the mapping, we created a

Re: Recommendation on reading the heart of the code.

Hi all, These are very good advises. I really appreciate it. I am planning to start all of these in parallel and find which one will suite me the best. Thank you: Joerg, Isabel, Kevin and Ivan. I have a question: How do we vote an answer (like Quora) here (is this available with Google

Re: Error: No config files found: /etc/logstash/conf.d

On Monday, October 06, 2014 at 10:11 CEST, StueckJu j.stuec...@gmx.com wrote: Hey, when i start logstash as a service /etc/init.d/logstash start, i get the Error Message in the topic in file logstash.stdout. I have a logstash config file in the directory /etc/logstash/conf.d/server.conf.

Re: Architecture to prevent slow queries

You could run less intense queries. Get more ram. Finally if io wait is a problem then you could switch to/add more solid state disks. Or you can add more nodes. We've done all of those for our Elasticsearch (no Logstash/Kibana in front though). Nik On Mon, Oct 6, 2014 at 4:43 AM,

Cannot restore a snapshot with IndexMissingException[[_snapshot] missing]

We manage to occasionally put ES cluster into a particular state, where it would fail to restore index from a valid snapshot with 404 and: IndexMissingException[[_snapshot] missing] This is an exact quote, note '_snapshot', which is not the name of the index. Nothing in the main logs. The very

Re: Need Help Regarding design multivalue column field in elasticsearch from river

The mail is hard to read, the layout is scrambled. For simple integer array, you do not need type nested. What is the SQL statement you tried for JDBC river? You have seen the examples for bracket notation? Jörg On Mon, Oct 6, 2014 at 2:04 PM, phani.nadimi...@goktree.com wrote: Hi All,

Re: Marvel - monitoring multiple ES clusters by one Monitoring ES cluster

On Wednesday, October 1, 2014 2:59:00 PM UTC+2, W Koot wrote: Hi Boaz, Looking forward to the update. I presume this will also address the Warning: Multiple masters. 2 nodes report master role of the nodes panel? :-) If you monitor two clusters, you will have multiple masters - which is

Trying to create geo_point index: Root type mapping not empty after parsing! Remaining field

Hi, I have an index with data that looks like this: { id:52534, name:foo, location:{ lat:39.18552295869955, lon:-104.34581946342342 } } I'm attempting to make a mapping for

[ANN] Released experimental highlighter version 0.0.12

I just finished releasing the experimental highlighter https://github.com/wikimedia/search-highlighter Elasticsearch plugin version 0.0.12. Its fixes one bug: * Regex highlighting fails on strings containing multi-byte characters https://github.com/wikimedia/search-highlighter/issues/6 If you

'out of float scope for function score deviation' error using function_score and a gauss decay

Using elasticsearch 1.3.4 I have an index with user events and I'm trying to use function_score to get a sensible order by date (without using sort). The query below works, but only for relatively small result sets. While trying to test whether this orders things correctly (following up on

[ANN] Released trigram accelerated regex queries for Elasticsearch version 0.0.1

I just finished releasing the wikimedia extra https://github.com/wikimedia/search-extra Elasticsearch plugin which contains support for trigram accelerated regular expressions similar to PostgreSQL's implementation

Optimal usage of big server (use VMs or not ...)

Regarding the recommendations for the maximal Java heap size, what is the best practice to utilize one server with lots of RAM (12 or 16 cores, 256 GB RAM): 1. Create 4 VMs, each 64 GB RAM and 32 GB Java heap size per VM - more shards per index 2. Use bare metal (256 GB RAM total), 128 GB

Re: New version of Kibana in the works?

Thanks! Looks like Kibana 4.0.0-BETA1 is out: https://github.com/elasticsearch/kibana On Thu, Oct 2, 2014 at 11:38 PM, David Pilato da...@pilato.fr wrote: May be following elasticsearch Twitter account ? Envoyé de mon iPad Le 3 oct. 2014 à 07:55, Thibaut Labarre asim...@gmail.com a écrit :

more accurate date based scoring

PUT /test/test/1 { date:2013-04-01T00:00:00Z } PUT /test/test/2 { date:2013-04-01T00:00:01Z } PUT /test/test/3 { date:2013-04-01T00:00:03Z } PUT /test/test/4 { date:2013-04-01T00:01:03Z } Given these documents, I'm trying to come up with a query that scores them such that they come

KeyError using python helpers.scan

Hi Everyone! I'm using ES for 6 months in production and today I was caught by an error being thrown by helpers.scan() method, what I could see is that scan always expect the response to contain the key _scroll_id, so I'm trying to understand it it's a problem in python wrappers or a problem

Re: Recommendation on reading the heart of the code.

There is no voting or other gamification, just a plain ol' mailing list. Many of us respond as just another way to contribute to open-source. -- Ivan On Mon, Oct 6, 2014 at 3:37 AM, ahmed jamal maaz jamalahmedm...@gmail.com wrote: Hi all, These are very good advises. I really appreciate it.

Re: Logstash into Elasticsearch Mapping Issues

I haven't ever let logstash set the default mappings. Instead, whenever a logstash-style index is created, I let Elasticsearch set the default mappings from its template. That way, it works even if I replace logstash with something else. For example, with my $ES_CONFIG/templates/automap.json

Gibberish Classifier with Elastic Search - Is it Possible?

I have a system that stores user suggestions. Since the answers come from a touchscreen, the kids love to insert comments like: *asduhaduh3189189'uih'dh* *3* *asdasdasdas* *gfdghgfhe* *n035gh9091v* I created a bayesian classifier and trained it with thousands of answers. Now it filters the

Re: Gibberish Classifier with Elastic Search - Is it Possible?

Lucene comes with some classifier implementations, but they are not integrated into Elasticsearch API yet. For more details, see http://soleami.com/blog/comparing-document-classification-functions-of-lucene-and-mahout.html Jörg On Mon, Oct 6, 2014 at 10:45 PM, Mauricio Moraes

Re: Logstash into Elasticsearch Mapping Issues

I also have the following Logstash output configuration: output { # For testing only stdout { codec = rubydebug } # Elasticsearch via HTTP REST elasticsearch { protocol = http codec = json manage_template = false # Or whatever target ES host is required: host =

Re: Help with designing our document for graphs. Indexing single nodes in graph with thousands of incoming edges

Hi Jorg, Thanks for the response. I don't actually need to model the relationship per se, more that a document is used in a relationship via a filter, then search on it's properties. See the example below for more clarity. Restaurant: = {name: duo} Now, lets say I have 3 users, George,

How to make Kibana Terms panel to count complete string frequencies, rather than separate words frequencies?

I have documents in ES with the field Message, which normally represents some multi word text string. Trying to query it with Kibana to see which strings are in this property most frequently. What I actually get back is the table which shows frequency of the specific *words*, but not the whole

[hadoop] Multiple indexes setting for 'es.resource'

Hi, I have a problem with 'es.resource' configuration for including multiple indexes. The hive table I created is like below CREATE EXTERNAL TABLE test ( date timestamp, clientip string, request string ) STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler' TBLPROPERTIES ( 'es.resource'

Re: How to make Kibana Terms panel to count complete string frequencies, rather than separate words frequencies?

I use multi fields to have several different analysis types supported as need and also to have the raw version available like in your example. On Monday, October 6, 2014 8:34:34 PM UTC-5, Konstantin Erman wrote: I have documents in ES with the field Message, which normally represents some

Re: How to make Kibana Terms panel to count complete string frequencies, rather than separate words frequencies?

Does not it cause substantial inflation in the amount of data to be processed and stored at indexing time? As with most logs aggregation systems indexing is many orders of magnitude more frequent operation than querying and I'm concerned that using multi_fields instead of all simple string

[hadoop] Issue while joining two hive tables stored on ES

Hi, I got an exception/error while joining two hive tables stored on ES. the tables I used are like below CREATE EXTERNAL TABLE member ( mem_idx bigint, id string, msaram_nm string, dept_nm string ) STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler' TBLPROPERTIES ( 'es.resource' =