OutOfMemoryError: Direct buffer memory

2014-03-17 Thread Daniel Guo
I use elasticsearch as an index server. And I deploy a web project to create index and search result from my es server. I got the following error from the logs of the web project: [elasticsearch[Vengeance][transport_client_worker][T#12]{New I/O worker #6249}] WARN

Re: OutOfMemoryError: Direct buffer memory

2014-03-17 Thread Daniel Guo
Yes, my bulk size is 10,000. I'll try to make it smaller. Thanks a lot, Clinton! On Monday, March 17, 2014 8:33:07 PM UTC+8, Clinton Gormley wrote: Are you sending an enormous bulk indexing request? If so, try to send fewer docs at a time, eg 1,000 On 17 March 2014 10:39, Daniel Guo

Auto-created spam index

2014-03-12 Thread Daniel Guo
In my cluster, many spam indices are automatically created. Please look at the image below, only the index named video is created by myself. Anybody any ideas? thanks! https://lh5.googleusercontent.com/-W8FKBOoV98Y/UyEQJucNThI/AXE/7w7I_3_hQqo/s1600/es.png -- You received this

How to do sum aggregations in Java API?

2014-03-04 Thread Daniel Guo
I have a query similar to the following SQL: select sum(count+displayCount) as total from day_inc_count group by video_id My problem is how to implement the above using aggregation in REST or Java API. I don't understand aggregations in elasticsearch 1.0 very much. Hope somebody could help.

What's wrong with my match query in Java API?

2014-02-26 Thread Daniel Guo
I want to do a match query, and the query works fine in REST: curl -XGET 'localhost:9200/search/video_search/_search?pretty' -d @query. json query.json: { query: { match: { tvName: { query: 决战华岩寺, operator: or,

Re: How to do incremental indexing in ElasticSearch?

2014-02-21 Thread Daniel Guo
03:04, Daniel Guo danie...@gmail.com javascript: a écrit : Here is my problem: I query some data from MySql, and then index those data to ElasticSearch. While the data in MySql is updating (update and insert) all the time, so I have to update ElasticSearch index accordingly. I cannot

What's the timeunit of setTTL(long ttl) in IndexRequestBuilder?

2014-02-21 Thread Daniel Guo
I'm trying to set documents sent to elasticsearch with TTL in Java. I call the method setTTL of IndexRequestBuilder when indexing documents. But I don't know what the time unit and I cannot find in docs and api. Could anybody helps, Thanks. -- You received this message because you are

Re: How to do incremental indexing in ElasticSearch?

2014-02-21 Thread Daniel Guo
;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 21 févr. 2014 à 10:12, Daniel Guo danie...@gmail.com javascript: a écrit : David, thanks for your answer. You mean that I have to control what to send to elasticsearch by myself ?! I heard that solr can configure delta index, so I'm just

Re: What's the timeunit of setTTL(long ttl) in IndexRequestBuilder?

2014-02-21 Thread Daniel Guo
@Binh, thank you, you are right, I find the reference here: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html#index-ttl but the problem is that, the TTL doesn't work when indexing a document, while it works when defined in type mapping. The user encountered

Re: What's the timeunit of setTTL(long ttl) in IndexRequestBuilder?

2014-02-21 Thread Daniel Guo
@Binh, enable ttl first in mapping and then use it in REST or client, it make sense. Thank you again! On Saturday, February 22, 2014 1:02:27 AM UTC+8, Binh Ly wrote: That is correct, the _ttl field is disabled by default:

How to do incremental indexing in ElasticSearch?

2014-02-20 Thread Daniel Guo
Here is my problem: I query some data from MySql, and then index those data to ElasticSearch. While the data in MySql is updating (update and insert) all the time, so I have to update ElasticSearch index accordingly. I cannot afford to do full index (data is huge), and I should not do that

Compared to Solr (with Solr Cloud), what is the advantage(s) of Elasticsearch?

2013-12-24 Thread Daniel Guo
I never used Apache Solr before, and I'm trying ElasticSearch in my project. The document of ES is a little scarce, but I have to explain to my supervisor why I chose ES over Solr. As far as I know, Solr (with Solr Cloud) also supports distributed indexing, near real-time update and searching,