Re: CPU LOAD GETTING HIGH (0.90.3)

Are the geo queries all kind of queries you execute? How much is the query load? It seems you use heavy filters or something CPU intensive. Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving

Re: How to query custom rest handler in elastic search using Java api

Hi Ivan, Below given is my custom class. How do I direct my search queries to this end-point using the ES-Java API. public class CustomRestAction extends BaseRestHandler { @Inject public CustomRestAction(Settings settings, Client client, RestController controller) { super(settings, client);

Re: Term filter not working for select strings?

After all my years of development, I would be surprised if I did NOT make a silly mistake! -- Ivan On Thu, Jan 2, 2014 at 10:08 PM, kakaner wrote: > Thanks Ivan!! That makes a ton of sense and can't believe I overlooked > that. Happened to be one of our unmapped fields :) > > > On Friday, Jan

Re: Returning substring in a GET request

Tested in 0.90.9 (December 23, 2013) as seems newer than 1.0.0.beta1 (November 6, 2013) and same issue: no highlighting results when key does not match, despite *setHighlighterNoMatchSize(250) *has been set*.* -- You received this message because you are subscribed to the Google Groups "elasti

Re: Returning substring in a GET request

Tested with 1.0.0.beta1 but does not seem to be working. For the query below, the highlights are returned successfully when the record matches *key*. However, despite *setHighlighterNoMatchSize(250) *has been set, unmatching records does not return 250 characters from the start of the record as

Re: Term filter not working for select strings?

Thanks Ivan!! That makes a ton of sense and can't believe I overlooked that. Happened to be one of our unmapped fields :) On Friday, January 3, 2014 12:42:52 AM UTC-5, Ivan Brusic wrote: > > The default analyzer uses a stop word filter and "in" is considered a stop > word. If you are using term

Re: Term filter not working for select strings?

The default analyzer uses a stop word filter and "in" is considered a stop word. If you are using term queries, then setting your field to non_analyzed will probably be your best fix. Cheers, Ivan On Thu, Jan 2, 2014 at 8:27 PM, kakaner wrote: > Hi! > > I recently discovered select term filte

I have a question (docs count, deleted value,)

My elasticsearch version 0.90.8 and, 2 nodes(clustring). I have a question. I used "http://localhost:9200/_nodes/stats/indices?pretty=true"; "cluster_name" : "janecluster", "nodes" : { "jWjT2xNtQwOdn17oV9KhVg" : { "timestamp" : 1388722961082, "name" : "node_es01", "transp

Re: Returning substring in a GET request

Functionality is available from 1.0.0.beta1 : - New highlighter based on lucene postings highlighter #3704 , #4042

Term filter not working for select strings?

Hi! I recently discovered select term filters weren't working on my dataset, and narrowed it down to an example involving two letter strings. This is reproducible on 0.90.2/5/7 both locally and on remote hosts with the following commands: curl -XPUT 'http://localhost:9200/index1/type1/1' -d '{

Re: Returning substring in a GET request

This seems a very recent addition to ES and the config param finally called no_match_size. It looks like that the way to define this is in a Highlighter Field as follows: HighlightBuilder.Field fiel

Re: CPU LOAD GETTING HIGH (0.90.3)

Hi Jason, Here is a gist with IO statistics https://gist.github.com/Gster1/6aa9a689c2325823f315 Thank you Greg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email t

Re: Returning substring in a GET request

Hi Nikolas, yes, thanks, this is exactly my use case, i.e. when highlighting does not match any term. There seems to be something about this from 4 months ago: https://github.com/elasticsearch/elasticsearch/pull/3613 It looks that config param was called no_matches. Any idea how to configure t

Re: Returning substring in a GET request

On Thu, Jan 2, 2014 at 9:36 PM, Adolfo Rodriguez wrote: Hi, I have some quite large documents (about 30k) and I want to get only an *excerpt > of each document*, for example, 250 characters. It really does not matter > if they are the first 250 characters or a chunk in the middle. I would > prefer

Re: Returning substring in a GET request

Hi, I have some quite large documents (about 30k) and I want to get only an *excerpt of each document*, for example, 250 characters. It really does not matter if they are the first 250 characters or a chunk in the middle. I would prefer not doing it a client side to avoid these large documents

Re: CPU LOAD GETTING HIGH (0.90.3)

Hi Greg, do you have statistics for IO ? Jason On Fri, Jan 3, 2014 at 8:00 AM, Gregory S wrote: > Hi all, > > I am trying to find out what could be causing system load to be over 6.5 > on a 6 cores server. This is not yet critically alarming but this does not > look great. Before throwing mor

Re: RabbitMQ river plugin

Which Elasticsearch version are you using? How did you create the river? Could you gist the full log file? -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 2 janv. 2014 à 23:45, David Koblas a écrit : Was trying to get the https://github.com/elasticsearch/elasticsearch-ri

CPU LOAD GETTING HIGH (0.90.3)

Hi all, I am trying to find out what could be causing system load to be over 6.5 on a 6 cores server. This is not yet critically alarming but this does not look great. Before throwing more CPU at the problem I would like to troubleshoot and figure out what is the best solution here. I have gist

Re: facets on nested objects, plus facet_filter

AFAIK, you cannot filter on parent fields when faceting on nested documents. Cheers, Ivan On Thu, Jan 2, 2014 at 2:46 PM, Nathan Moon wrote: > Hi, I am using nested objects for indexing “ratings” on an object, where a > rating contains two properties: the owner and the rating. I want to be >

facets on nested objects, plus facet_filter

Hi, I am using nested objects for indexing “ratings” on an object, where a rating contains two properties: the owner and the rating. I want to be able to filter and facet on “my ratings”. So to filter, for example, on objects I have rated a “10", I am using a filter like { “nested”

RabbitMQ river plugin

Was trying to get the https://github.com/elasticsearch/elasticsearch-river-rabbitmq plugin working with our system. However, I wonder if it does work since it's not been updated to the current version. The only log message I have is: [2014-01-02 22:21:13,243][INFO ][plugins ]

Re: Need help retrieving field from ES

Lucene (and therefore elasticsearch) is schemaless, so every document in the same index does not need to have the same structure. Just because the mapping contains the keepalive does not mean the document does. You can enforce that every field in the document must exist in the mapping [1], but not

[hadoop] Push _id to ES via PIG ESStorage

Hi, Is it possible to push the _id field via ESStorage / PIG towards the ES cluster? Thanks, Dumitru -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elastics

Re: Need help retrieving field from ES

Ivan, That command confirms that no docs have the keepalive. My next silly question, is how do i get the field to appear? The mapping is there, are the options correct? Will the field appear when ES is loaded with data? Excuse my stupid questions, but I thought the field would be created if an

Re: Problems with excesive GC

Filtered query sounds fine. You should definitely look into your filters and try more efficient ones. Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to ela

Re: Order by name doesn't work as expected

Thank you, it's working. On Thursday, January 2, 2014 4:20:17 PM UTC, Nikolay Chankov wrote: > > Hi guys, > > for some reason, the order by name, _score is not working as I would > expect. > I've prepared a simple example to explain what I mean. > There are 2 records: john doe and jane doe. if

Re: Need help retrieving field from ES

Judging by the one sample document, the keepalive field is not there. You can use the missing filter to see if any documents do have that field. For example: curl -XPOST localhost:9200/2014010119/_count/ -d ' { "filtered": { "query": { "match_all": {} }, "filter": {

Re: Need help retrieving field from ES

Thanks for the pointers David. I am using the elastic search under Perl, to retrieve data based on facets, however the keepalive field is not returned. I guess this is because it is not in the _source. However I can’t seem to get that field to appear. We create new indexes every hour, the last

Re: Order by name doesn't work as expected

The "name" field is an analyzed string field, so it can generate numerous tokens depending on the text. In your case, the email is being split into two tokens: http://localhost:9200/_analyze?text=john%20doe {"tokens": [{"token": "john","start_offset": 0,"end_offset": 4,"type": " ","position": 1},

Re: Deb repos are offline?

All fine now, thanks. On Thu, Jan 2, 2014 at 9:58 PM, Demetri Mouratis wrote: > apt doesn't seem to like the way the key is setup. > > root@syslog2:/etc/apt/sources.list.d# wget -O - > http://packages.elasticsearch.org/GPG-KEY-elasticsearch | apt-key add - > > --2014-01-02 19:50:21-- > http://p

Re: Deb repos are offline?

apt doesn't seem to like the way the key is setup. root@syslog2:/etc/apt/sources.list.d# wget -O - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | apt-key add - --2014-01-02 19:50:21-- http://packages.elasticsearch.org/GPG-KEY-elasticsearch Resolving packages.elasticsearch.org (pack

Re: Need help retrieving field from ES

I think you will get more help if you follow what is explained here: http://www.elasticsearch.org/help/ Hard to help without more information. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 2 janv. 2014 à 18:44, Nick Toseland a écrit : > Can anyone help me please? > >

Re: Getting specific Fields

Not yet supported: https://github.com/elasticsearch/elasticsearch/issues/3022 Cheers, Ivan On Thu, Jan 2, 2014 at 4:27 AM, paul wrote: > My DATA > --- > { >"rankingList":[ > { > "value":9, > "key":"Academic" > }, > { > "value":6, >

Re: Problems with excesive GC

Between lines. Thanks! 2014/1/2 joergpra...@gmail.com > What ES version is that? > 0.90.8 > > What type of queries do you perform? > We have a very big query, with lots of filters, and because of that we have to add them to each facet (we are thinking to move it to a filtered query instead, do

Re: Problems with excesive GC

What ES version is that? What type of queries do you perform? What filter cache did you configure? Jörg -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elast

Re: How to query custom rest handler in elastic search using Java api

You would need to supply information about your custom class for more details. Did you create a new Action hierarchy? The handleRequest method of your BaseRestHandler implementation should provide details on how the Java API is used. Cheers, Ivan On Thu, Jan 2, 2014 at 5:14 AM, Shishir Kumar wr

Re: elastic and language stem (polish)

Analyzers are associated with fields, so in your mapping you can specify which analyzer to use. When you query a field, elasticsearch will know which analyzer to use (although it can be overridden). For example: "title": {"type": "string", "analyzer": "polish"} If you are using the plugin, there

Re: 4-5 second query time. Only 50 documents. Need help

I agree with Jorg that the size of your document is perhaps the primary reason for the slowness. Larger documents is one of the drawbacks of denormalization. Just to highlight some other inconsistencies: 1) You stated that you have 1 replica, but also that you have only one node. Elasticsearch wi

Re: Need help retrieving field from ES

Can anyone help me please? Many thanks On Monday, 23 December 2013 20:40:17 UTC, Nick Toseland wrote: > > Hi All > > I am new to ElasticSearch, please forgive my stupidity. > > I cant seem to get the keepalive field out of ES. > > { > "_index" : "lj-2013122320", > "_type" : "varnish"

Order by name doesn't work as expected

Hi guys, for some reason, the order by name, _score is not working as I would expect. I've prepared a simple example to explain what I mean. There are 2 records: john doe and jane doe. if there is no email in the index their score is the same, and the order is correct, jane goes before john, but

Re: Shard missing from response

I set threadpool.search.queue_size: 1 and now it works! Le jeudi 2 janvier 2014 15:28:15 UTC+1, Pierre Durand a écrit : > > Up? :( > > Le lundi 23 décembre 2013 12:14:28 UTC+1, Pierre Durand a écrit : >> >> I use Elasticsearch 0.90.5 for log management (1 index by day). >> >> In my query, I ha

Re: ElasticSearch Index Wrong Date

Not sure what happened, but after restarting Logstash everything is working fine. I guess it just wasn't a fan of the change in year. On Thursday, January 2, 2014 10:03:05 AM UTC-5, Eric Luellen wrote: > > Hello, > > I recently setup my elasticsearch instance and everything has been working > f

ElasticSearch Index Wrong Date

Hello, I recently setup my elasticsearch instance and everything has been working fine. However, when I looked at Kibana today I saw that the logs stopped showing up as soon as 2014 hit. When looking at my data on the cluster, I see this: ls -altr data/my-cluster/nodes/0/indices/ total 44 drwx

Re: Non Alphanumeric character searching

Thanx you very much On Thursday, 2 January 2014 01:43:30 UTC+5:30, Alexander Reelsen wrote: > > Hey, > > most likely those special chars have been removed before your data has > been stored in the inverted index - and thus cannot be searched for. This > highly depends on the mapping for a field.

Re: Non Alphanumeric character searching

Hey Thanx for the reply. I am not able to install the inquisitor, can you please show us the path on this On Thursday, 2 January 2014 01:43:30 UTC+5:30, Alexander Reelsen wrote: > > Hey, > > most likely those special chars have been removed before your data has > been stored in the inverted ind

Query: Parents with at least x children of type y

Hello, after some research without any results I have a question about parent/child relations. The case: I have a parent of type "parent_type" which has children of different types e.g. "child_type_1", "child_type_2", "child_type_3". My Question is: Is there any possibility to get only the pa

Re: Shard missing from response

Up? :( Le lundi 23 décembre 2013 12:14:28 UTC+1, Pierre Durand a écrit : > > I use Elasticsearch 0.90.5 for log management (1 index by day). > > In my query, I have a term facet in order to aggregate my data. > But sometimes, there is a really weird problem, I get: > _shards: {total:700, successfu

Re: Load balancing and a node with no primary shard

1. What you observe is correct. Replica shards can be instantly promoted to primary shards. Primary shards stay as long as the node is running. The reason is, there is no functional difference between a primary and replica. You should not worry about primary shards at all, they do not matter for lo

How to query custom rest handler in elastic search using Java api

Hi, I have implemented a simple custom rest handler class for elastic search. If I need to call it using curl it works just fine. curl -XGET '10.114.24.132:9200/_mastering/nodes?pretty' However, I want to call this using the elastic search Java api (with an embedded client node). Could you pleas

Load balancing and a node with no primary shard

Hi, We have a system based on Elasticsearch and we're performing volume tests in order to come up with hardware requirements. Following the tests, I am confused by the role of the primary shard in a load balanced environment and am wondering regarding performance after a node failure. I also

Getting specific Fields

My DATA --- { "rankingList":[ { "value":9, "key":"Academic" }, { "value":6, "key":"Flexibility" } ] } { "rankingList":[ { "value":12, "key":"Academic" }, { "value":6,

Re: Deb repos are offline?

Hi David, Thanks for the quick reply. Any alternative location to install ES-built Logstash debs from? On Thursday, January 2, 2014 5:38:51 AM UTC+2, David Pilato wrote: > > See this: > https://groups.google.com/forum/?nomobile=true#!original/elasticsearch/5CFRD-DLaT0/mShAN8rJFOAJ > > -- > Davi

Re: Need Clarification on Shards Replication

Hi David , Thanks for your reply . Until existing(not newly created) segments are fully copied to the new machine , no indexing operation will happen on replica shard right ? rather it notes down those new indexing documents in transaction log alone ?((Correct me if i am wrong)) Once all se

Re: Facet autocomplete for field with multiple values

Hi Peter, I don't see any magic solution with your current structure but it would be easier if you structure your data in the opposite way: - Index one document per user with all users information (firstname, lastname, uid, ...) - On each user object add the list of your document ID that the use

Re: Bulk throughput issues

Hi tdjb, I am also working on similar task, my requirement is to store around 70 million documents on a single mode. I have a 8 GB 8 core machine. Please guide me how should take approach. Also request you to share some code samples of how I can use Bulk Processor. Regards Geet On Thursday,

Re: Need Clarification on Shards Replication

It's relocation. Segments are copied over the wire. New updates/insert/delete operations which happen in the meantime are replayed from the transaction log on the new shard. HTH -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 2 janv. 2014 à 10:39, Anantha Govindarajan a

Re: Corrupt index creation when elasticsearch is killed just after index is created

All ES API calls are by default asynchronous and eventually consistent (quorum). For document indexing, you can use the refresh API call to make them visible for search. For index creation operation, you can add the parameter "replication=sync" and "consistency=all" to your API call to ensure tha

Re: Need Clarification on Shards Replication

Hi Alex Thanks for replying. If i understand correctly normal indexing flow is, - Document is indexed in primary shard machine , then replica shard machine then return the index response to client - in case of ReplicationType.SYNC. - Document is indexed in primary shard machine ,

Kibana 3 index settings

Hello, in Kibana 3 is possible to set from the dashboard settings, the index settings. I'm using for this a day based timestamping, with an index pattern such as [dc1_logstash-].MM.DD Now, we added a new datacenter to the elasticsearch cluster, with different index names. Is it possible to u

Re: Need Clarification on Shards Replication

Hey, replication is done per document (as opposed to relocation). So the document is indexed on the primary first, and if it was successful there, the document is indexed on all replicas of a shard in parallel. If that index operation on the replica(s) has returned, the index requests is returned

Re: Corrupt index creation when elasticsearch is killed just after index is created

Thanks clinton. But shouldn't index creation be an atomic operation, and the call should only return after the index is "properly" created, otherwise show a indexmissing exception, or should automatically look for unassigned primary shards for an index after the restart ? On Thu, Dec 26, 2013 at