Re: How many elastic search shards should I have for a physical node?

2015-06-01 Thread David Pilato
Each shard is a Lucene instance. It uses memory, file descriptors, CPU, IO... You should keep its number per node small. If you are a RDBMS user, you probably limit the number of databases you manage on a single physical machine, right? HTH -- David ;-) Twitter : @dadoonet / @elasticsearchfr

Re: Searching for "foo" should also find occurrence of "foo.bar"

2015-05-29 Thread David Pilato
I would use 2 analyzers and multi field: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-core-types.html#_multi_fields_3 <https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-core-types.html#_multi_fields_3> -- David Pilato - Developer | Evan

Re: Searching for "foo" should also find occurrence of "foo.bar"

2015-05-29 Thread David Pilato
Yes. Because « Hello. How are you? » is a sentence that can be broken in « hello », « how », « are », « you ». But in « I paid it 2.50 euros », I would most likely keep « 2.50 » as a whole token. -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoo

Re: Searching for "foo" should also find occurrence of "foo.bar"

2015-05-29 Thread David Pilato
one is more for english text which means that a dot need to have a space after it in order to be considered as a break between two tokens. Make sense? -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/

Re: Where is Elasticsearch storing data

2015-05-28 Thread David Pilato
So can anyone please point me to where we might be able to find the data? >>> If it helps, we are loading indexes from both hive and from logstash >>> directly. >>> >>> Thanks, >>> Jim >>> >>>> On Wednesday, May 27, 2015 at 6:56:54 P

Re: Elasticsearch: 2-node cluster with failover

2015-05-28 Thread David Pilato
Add another node (master only : data false) with minimal HEAP and set minimum master nodes to 2 or set minimum master nodes to 2 but in that case your cluster will stop working in case of failure of any node or network failure -- David Pilato - Developer | Evangelist elastic.co @dadoonet

Re: Where is Elasticsearch storing data

2015-05-27 Thread David Pilato
by default in path.data dir. In zip or tar.gz distribution, it’s in "data" dir -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.com/

Re: Wild card query with case insensitive search

2015-05-26 Thread David Pilato
Don't use wildcard query unless you want it to be "slow". Prefer using ngrams based analyzers (will cost you more disk space). HTH David > Le 26 mai 2015 à 22:29, Sagar Shah a écrit : > > Hello everyone, > We are trying to replace application log storage from postgres db with > elastic search

Re: Querying indexed data in different computer Elasticsearch

2015-05-24 Thread David Pilato
archPhaseExecutionException: Failed to execute phase [query], all shards > failed > > I am building the maven project where I use elasticsearch maven repositories > so I dont know where elasticsearch places its log files on that and I cannot > use logstash currently. > > Tha

Re: Querying indexed data in different computer Elasticsearch

2015-05-24 Thread David Pilato
mply "mycluster", still not working. > > Thanks > regards, > Kartik > >> On Mon, May 25, 2015 at 2:20 AM, David Pilato wrote: >> I meant something else. At first you indexed kimchy and search for kimchi. >> >> Check your file access rights. Might

Re: Querying indexed data in different computer Elasticsearch

2015-05-24 Thread David Pilato
SearchRequestBuilder srb = > client.prepareSearch("twitter").setTypes("tweet"); > SearchResponse big = srb.setQuery(qb).execute().actionGet(); > SearchHit[] results = big.getHits().getHits(); > > /* print results */ > node.close()

Re: Querying indexed data in different computer Elasticsearch

2015-05-24 Thread David Pilato
he above code in different computer with data copied to the same > path with the exception I mentioned. > What in your opinion I should do? > > Thanks for your reply, really helping me a lot. Sorry, I know I am doing some > silly mistake here but I am not able to identify it as I am new to

Re: Querying indexed data in different computer Elasticsearch

2015-05-24 Thread David Pilato
No I have no idea. But for sure I would never use such a cluster name! May be your cluster is not really started and you are not waiting enough time for you index to have at least a yellow status? -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoo

Re: Querying indexed data in different computer Elasticsearch

2015-05-24 Thread David Pilato
will just download my code > and the indexed data and then with the help of my code they will query the > data. > >> On Sun, May 24, 2015 at 3:59 PM, David Pilato wrote: >> It means that your users are connected on the same network or that you want >> the

Re: Querying indexed data in different computer Elasticsearch

2015-05-24 Thread David Pilato
ly. Actually my problem is that I first have to index data > on my server, then I will be deploying this data so that other users can > query it on their different different computers using the code I provide to > them. How can I do that? > > Thanks > regards, > Kartik > &g

Re: Querying indexed data in different computer Elasticsearch

2015-05-24 Thread David Pilato
I think you misunderstand how Elasticsearch works. You don't need to share a filesystem across multiple nodes. Just start a first node with a cluster name like "mycluster". Index your data. Start a client node or a transport client, which connects to your running cluster and you are done. -

Re: @timestamo and Date in the logs are not matching

2015-05-08 Thread David Pilato
Definitely a logstash question. You need to use the date filter to reconciliate dates. Please use the logstash discussion group for this at https://discuss.elastic.co/ <https://discuss.elastic.co/> -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com

Re: Error handling for HTTP search API

2015-05-07 Thread David Pilato
{ "simple_query_string": { "query": "bar baz", "fields": [ "foo" ], "default_operator": "AND" } } } But that’s not exactly what you are looking for. -- David Pilato - Developer | Evangelist elas

Re: Error handling for HTTP search API

2015-05-06 Thread David Pilato
s "+". For the > user base, that's possibly a bigger issue than big error messages. > > > >> On Thursday, 7 May 2015 15:01:55 UTC+10, David Pilato wrote: >> Try simple_query_string. >> >> BTW we moved to https://discuss.elastic.co/ >>

Re: Error handling for HTTP search API

2015-05-06 Thread David Pilato
Try simple_query_string. BTW we moved to https://discuss.elastic.co/ David > Le 7 mai 2015 à 06:56, Snixtor a écrit : > > I have an application allowing users to specify the "query" value of a > "query_string" query. If the user inputs invalid search syntax, e.g. their > search is only "AND"

Re: Multiple nodes on same machine : replicas?

2015-05-06 Thread David Pilato
Why do you want to have 2 nodes per machine? -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 6 mai 2015 à 13:45, Jason Wee a écrit : > > afaik, you don't change the cluster.routing .* , es 0.90 should be > intelligent enough to redistribute the shards evenly across the

Re: Cpu fluctuating due to java.exe

2015-05-06 Thread David Pilato
Nothing noticeable. Do you have an empty cluster when starting ? Is this happening all the time although you don't have any activity on the cluster? -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 6 mai 2015 à 10:45, Chetan Dev a écrit : > > Hi, > > here is the result

Re: Cpu fluctuating due to java.exe

2015-05-06 Thread David Pilato
> org.elasticsearch.search.aggregations.AggregationParseElement.parse(AggregationParseElement.java:60) > at > org.elasticsearch.search.SearchService.parseSource(SearchService.java:671) > ... 9 more > > > > > [2015-05-06 12:27:16,506][DEBUG][action.search.type ]

Re: Cpu fluctuating due to java.exe

2015-05-06 Thread David Pilato
What is wrong ? I mean that if you are doing indexing for example, that's fine if your CPU is used. But may be you are running out of memory and GC is running all the time ? Hard to say without any details / logs... BTW me moved to https://discuss.elastic.co/ -- David ;-) Twitter : @dadoonet /

Re: script_fields giving error

2015-05-05 Thread David Pilato
Read this: http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-scripting.html#_enabling_dynamic_scripting Note that we moved to https://discuss.elastic.co/ David > Le 6 mai 2015 à 06:26, Mohit Garg a écrit : > > I am using elastic 1.5.0. When I run this block in the query:

Re: get number of pending document writes?

2015-05-05 Thread David Pilato
In tests, you need to refresh the index before searching. Elasticsearch is a near real time search engine. See http://www.elastic.co/guide/en/elasticsearch/reference/current/indices-refresh.html Note that we moved the mailing list to https://discuss.elastic.co/ David > Le 6 mai 2015 à 04:32, D

Re: add mapping for geo_point with java

2015-05-03 Thread David Pilato
How do you send your mapping? -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 4 mai 2015 à 08:32, Dharshana Ratnayake a écrit : > > Hi Guys > > Im trying to save documents in elasticsearch where one field i want to map a > geo_point with lat and lon > But i would like

Re: how to safely clean old documents (by date)

2015-05-03 Thread David Pilato
> current index to the new one > > > >> On Monday, May 4, 2015 at 12:15:22 AM UTC+3, David Pilato wrote: >> Definitely the second option. >> Use scan and scroll (search for reindex on the website). >> >> Instead of renaming, I would use aliases and swit

Re: how to safely clean old documents (by date)

2015-05-03 Thread David Pilato
Definitely the second option. Use scan and scroll (search for reindex on the website). Instead of renaming, I would use aliases and switch the alias from old to new index. Then close or remove the old index. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 3 mai 2015 à

Re: Documents not deleted when using DeleteRequest within BulkProcessor

2015-05-01 Thread David Pilato
» ? Or do you mean that the document has not been removed ? How do you test all that? -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.com/scru

Re: Max documents 10,500?

2015-05-01 Thread David Pilato
Could you compare disk size (/data dir) for your two elasticsearch instances? Also, could you GIST the result of a simple _search?pretty on both nodes? -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://tw

Re: Max documents 10,500?

2015-05-01 Thread David Pilato
Any chance you are using the same id multiple times? -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.com/scrutmydocs> > Le 1 ma

Re: Max documents 10,500?

2015-05-01 Thread David Pilato
se if (callback !== undefined) { > callback('elastic search not connected', undefined); > } > }; > > > >> On Friday, May 1, 2015 at 11:42:12 AM UTC-5, David Pilato wrote: >> If you have nothing in logs it could mean that you have an issue with

Re: Max documents 10,500?

2015-05-01 Thread David Pilato
If you have nothing in logs it could mean that you have an issue with your injector. May be you are using bulk but you don't check the bulk response? David > Le 1 mai 2015 à 18:36, Blake McBride a écrit : > > Greetings, > > I have two similar but unrelated machines. I am adding 50,000+ docum

Re: "too many open files" problems and suggestions on cluster configuration

2015-05-01 Thread David Pilato
Add more nodes or reduce the number of shards per node. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 1 mai 2015 à 17:05, Ann Yablunovskaya a écrit : > > I am looking for suggestions on cluster configuration. > > I have 2 nodes (master/data and data), 544 indices, abo

[ANN] Elasticsearch Smart Chinese Analysis plugin 2.4.4 released

2015-04-30 Thread David Pilato
Heya, We are pleased to announce the release of the Elasticsearch Smart Chinese Analysis plugin, version 2.4.4. Smart Chinese Analysis plugin integrates Lucene Smart Chinese analysis module into elasticsearch.. https://github.com/elasticsearch/elasticsearch-analysis-smartcn/

Re: [ANN] Elasticsearch Japanese (kuromoji) Analysis plugin 2.4.3 released

2015-04-30 Thread David Pilato
Sorry. Was a typo. You should have read Update to elasticsearch 1.4.5 Le jeudi 30 avril 2015 15:48:44 UTC+2, David Pilato a écrit : > > Heya, > > We are pleased to announce the release of the *Elasticsearch Japanese > (kuromoji) Analysis plugin*, *version 2.4.3* > > T

Re: LIKE Query

2015-04-30 Thread David Pilato
rch/guide/current/mapping-analysis.html> Answering to your question, using ngrams would probably help here. Hope this helps -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @s

Re: Troubleshooting Elasticsearch queries

2015-04-30 Thread David Pilato
<https://gist.github.com/dadoonet/0b161423d0acafecb012> -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.com/scrutmydocs> &

Re: [ANN] Elasticsearch Mapper Attachment plugin 2.3.0 released

2015-04-30 Thread David Pilato
right? -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.com/scrutmydocs> > Le 30 avr. 2015 à 12:47, Prashant Agrawal a > écrit

Re: [ANN] Elasticsearch Mapper Attachment plugin 2.3.0 released

2015-04-30 Thread David Pilato
So why not doing that in your application? If you look into _source.file.content and _source.file._content, older docs will have BASE64 content in content and null in _content and the opposite for newer docs. Do I miss anything? -- David Pilato - Developer | Evangelist elastic.co @dadoonet

Re: How to fix cluster health yellow with Elastic Search

2015-04-30 Thread David Pilato
don’t need to do that. If your plan is to do that in production, please don’t! Don’t run more than one node per machine unless you have more than 64gb RAM. -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.

Re: [ANN] Elasticsearch Mapper Attachment plugin 2.3.0 released

2015-04-30 Thread David Pilato
-request-script-fields.html <http://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-script-fields.html> Does it help? -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/ela

Re: How to fix cluster health yellow with Elastic Search

2015-04-30 Thread David Pilato
Not enough information to say more for now. -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.com/scrutmydocs> > Le 30 avr. 2015 à 10:50,

Re: bulkProcessor never calls afterBulk?

2015-04-29 Thread David Pilato
Try to remove setBulkSize. What is the other part of your code? David > Le 30 avr. 2015 à 00:30, Scott Chapman a écrit : > > Here's my code example, I get something printed beforeBulk. But never see > anything printed for afterBulk. And yes, a document is getting written. > > What am I missin

Re: possible networking problem?

2015-04-29 Thread David Pilato
in the loop to capture the > data interchange between browser and Kibana. You can see GET outgoing on the > left and the refusal on the right. > >> On Wednesday, April 29, 2015 at 1:31:14 PM UTC-7, David Pilato wrote: >> Try 0.0.0.0 not localhost >> >> David

Re: Documents not deleted when using DeleteRequest within BulkProcessor

2015-04-29 Thread David Pilato
Do you try to delete a doc using an alias? Any failure or error reported by the bulk processor? Hard to tell more without seeing the code / data. David > Le 30 avr. 2015 à 02:03, Diana Tuck a écrit : > > Trying to index/delete documents within one BulkProcessor object in the Java > API. Inde

Re: possible networking problem?

2015-04-29 Thread David Pilato
Try 0.0.0.0 not localhost David > Le 29 avr. 2015 à 22:29, Sitka a écrit : > > Here is the message output from Kibana when I start it up: > > "c:\Kibana\kibana-4.0.2-windows\bin>kibana.bat > {"@timestamp":"2015-04-29T17:11:32.713Z","level":"info","message":"No > existing k > ibana index fou

Re: duplicate documents in query,

2015-04-29 Thread David Pilato
outing key when indexing one of your docs? -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.com/scrutmydocs> > Le 29 avr. 2015

Re: duplicate documents in query,

2015-04-29 Thread David Pilato
What do you have with: curl -XGET 'http://localhost:9200/track_2011*/' -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.com/scru

Re: i need some help

2015-04-29 Thread David Pilato
It sounds incorrect. Have a look at bulk documentation. You are probably missing something. Note that it’s hard to answer without a complete recreation script. Look at https://www.elastic.co/help <https://www.elastic.co/help> -- David Pilato - Developer | Evangelist elastic.co @da

Re: duplicate documents in query,

2015-04-29 Thread David Pilato
Which elasticsearch version have you? -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.com/scrutmydocs> > Le 29 avr. 2015 à 16:44, G

Re: elasticsearch 0.90 backup/snapshot

2015-04-29 Thread David Pilato
Yes. When you cluster is stopped, backup all your /data dirs for each node and you have a backup. Then upgrade. -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmy

Re: Safe to use prefix query on analysed field?

2015-04-29 Thread David Pilato
Well. If you’d like to enhance the documentation, feel free to send a pull request! :) -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.

Re: Kibana 4 not recognizing the indice?

2015-04-29 Thread David Pilato
Check that you have unchecked some checkboxes and that your Kibana dashboard settings looks like this http://david.pilato.fr/blog/images/elk-stations/settings-set-index.png <http://david.pilato.fr/blog/images/elk-stations/settings-set-index.png> It should work fine. -- David

Re: How to process "Lat" & "Long" fields using default Logstash config and mapping to use in Kibana 4 tile map

2015-04-28 Thread David Pilato
For some reason I am doing something wrong and > its driving me nuts. I've tried your method but the tile map is showing me no > results whatsoever. How did you define your template in Elasticsearch for > this "location" field? > > Thanks, > > Rodger >

Re: Tranfer Kibana 4 dashboards

2015-04-28 Thread David Pilato
I'm using SNAPSHOT elasticsearch feature for that. I save .kibana index. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 28 avr. 2015 à 19:55, Kellan Strong a écrit : > > Hello All, > > In Kibana 3 you were able to save the dashboards you created to your desktop > and

Re: marvel agent issue with elasticsearch 1.4.5

2015-04-28 Thread David Pilato
I would open an issue in Elasticsearch repo. David > Le 28 avr. 2015 à 14:48, Gurvinder Singh a > écrit : > > Today we upgraded the cluster to recent Elasticsearch 1.4 branch, after > upgrade we are seeing this message in the master logs. > > [2015-04-28 14:40:47,942][ERROR][marvel.agent

Re: Couchbase XDCR error

2015-04-28 Thread David Pilato
You should better report that to couchbase mailing list or transport plugin repo. The logs you shown are from their plugin AFAIK. -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasti

Re: Safe to use prefix query on analysed field?

2015-04-28 Thread David Pilato
It means that the text you give to the prefix query is not analyzed. So if you have in the inverted index * abc * def And you search with prefix AB, it won’t work as abc does not start with AB Make sense? -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.

Re: Search local only

2015-04-27 Thread David Pilato
enters. -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.com/scrutmydocs> > Le 27 avr. 2015 à 17:14, Christophe Rosko a écrit : > >

Re: Deploy elasticsearch to local maven repo

2015-04-27 Thread David Pilato
It’s a maven question not related to elasticsearch. That said mvn install will do what you want. -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <http

Re: How to process "Lat" & "Long" fields using default Logstash config and mapping to use in Kibana 4 tile map

2015-04-26 Thread David Pilato
It's not an issue IMO but just a default configuration. FYI here is a sample config file I just used to parse some CSV data: input { stdin {} } filter { csv { separator => ";" columns => [ "id","name","slug","uic","uic8_sncf","longitude","latitude", "parent_station_id","is_cit

Re: How elasticsearch search works??

2015-04-26 Thread David Pilato
If you use a query string or simple query string, searching for priority:2 will search in priority field only. Using a query dsl with filters will make this more obvious and more efficient performance wise. HTH David > Le 26 avr. 2015 à 14:23, jigish thakar a écrit : > > Dear All, > I am us

Re: How to process "Lat" & "Long" fields using default Logstash config and mapping to use in Kibana 4 tile map

2015-04-25 Thread David Pilato
You need to use the mutate filter and move your fields into a "location" one. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 25 avr. 2015 à 22:13, Rodger Moore a écrit : > > Hi there! > > My question is fairly simple but I'm having trouble finding a solution. I > have

Re: script_lang not supported [groovy]

2015-04-25 Thread David Pilato
Not sure it’s your issue but elasticsearch 1.5 uses Groovy 2.4.0 org.codehaus.groovy groovy-all 2.4.0 -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://tw

Re: FreeBSD 10.1 install elasticsearch plugin fails

2015-04-24 Thread David Pilato
Yes. I told you. The plugin version is 2.4.3 not 2.43 /usr/local/bin/elasticsearch-plugin install elasticsearch/elasticsearch-mapper-attachments/2.4.3 -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://tw

Re: FreeBSD 10.1 install elasticsearch plugin fails

2015-04-24 Thread David Pilato
No. 2.43 does not exist. -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.com/scrutmydocs> > Le 24 avr. 2015 à 12:53, Pccom Frank a

Re: BulkProcessor pest practices

2015-04-24 Thread David Pilato
est, Throwable failure) { logger.warn("We have a problem", failure); bulk.add(request); } }) .setBulkActions(pageSize) .setFlushInterval(TimeValue.timeValueSeconds(5)) .build(); } -- David Pilato - Developer | Ev

Re: Bulk indexing creates a lot of disk read OPS

2015-04-24 Thread David Pilato
ly, is it an issue for you? If not, keeping all defaults values might be good. Best -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.com/scrutmyd

Re: Bulk indexing creates a lot of disk read OPS

2015-04-24 Thread David Pilato
Merging segments could be the cause here? David > Le 24 avr. 2015 à 09:54, Eran a écrit : > > Forgot some stats: > > I have 10 shards, no replicas, all on the same machine. > ATM, there are some 1.5 billion records in the index. > > >> On Friday, April 24, 2015 at 10:18:27 AM UTC+3, Eran wro

Re: FreeBSD 10.1 install elasticsearch plugin fails

2015-04-23 Thread David Pilato
49, Pccom Frank a écrit : > > I am using Freebsd, there is no such thing as bin/plugin, only > elasticseach-plugin functioning as bin/plugin, I guess. It won't follow the > official doc. The elasticseach version in FreeBSD is 1.43, match mapper > attachment 2.43. >> On

Re: bulk index request dataloss

2015-04-23 Thread David Pilato
filling the memory? You could also think of setting replicas to 0 before bulk and the reactivate to 1 after injection. Having SSD drives can also help but may be you have already that? My 2 cents -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoo

Re: Users data flow

2015-04-23 Thread David Pilato
ttp://www.elastic.co/guide/en/elasticsearch/guide/current/scale.html> http://www.elastic.co/guide/en/elasticsearch/guide/current/faking-it.html <http://www.elastic.co/guide/en/elasticsearch/guide/current/faking-it.html> -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitt

Re: Users data flow

2015-04-23 Thread David Pilato
go to whichever shard. You don’t really need to worry about it. Then another question might be « how many shard I will need » and the answer is it depends but I would say: try to keep it as minimal as possible. Make sense? -- David Pilato - Developer | Evangelist elastic.co @dadoonet

Re: Master node refuse to accept its role

2015-04-23 Thread David Pilato
nable to see each other. So the data node is now reporting > that it cannot see any master nodes and its failing to start. > >> On Wednesday, April 22, 2015 at 4:01:12 PM UTC+3, David Pilato wrote: >> I just ran a small test on a ec2 instance. >> >> I just set &g

Re: FreeBSD 10.1 install elasticsearch plugin fails

2015-04-22 Thread David Pilato
The command you write is totally wrong. You set a url, you define the wrong version... And it sounds you renamed plugin script but that's not an issue. Whatever. Doc says: bin/plugin install elasticsearch/elasticsearch-mapper-attachments/2.5.0 Try bin/plugin install elasticsearch/elasticsearch-

Re: Master node refuse to accept its role

2015-04-22 Thread David Pilato
cloud.aws.access_key: cloud.aws.secret_key: discovery.type: ec2 node.master: true node.data: false -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <http

Re: Analyzers in Elastic search

2015-04-22 Thread David Pilato
You could start here http://www.elastic.co/guide/en/elasticsearch/guide/current/mapping-analysis.html ? -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 22 avr. 2015 à 12:58, Sharath Chandra a écrit : > > Hi every one, > I am very new to Elastic Search.Now i am reading

Re: Master node refuse to accept its role

2015-04-22 Thread David Pilato
Which versions for: Elasticsearch Cloud-aws-plugin ? -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 22 avr. 2015 à 12:41, Zaid Amir a écrit : > > Hi, > > I am starting a new cluster and I want it to be set with two nodes. One is a > data only node and the other is

Re: What web server Elasticsearch use on windows?

2015-04-22 Thread David Pilato
Elasticsearch provides its own web server so you don’t need to provide anything else than a JVM. Netty is used BTW. -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmy

Re: org.elasticsearch.index.mapper.MapperParsingException: failed to parse -- NEED HELP

2015-04-22 Thread David Pilato
I think you mapping expects an object for the error field but you sent a string in it. -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.

Re: upgrade java for elasticsearch node

2015-04-22 Thread David Pilato
conservative here and don’t try to mix things. My 2 cents -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.com/scrutmydocs> > Le 22 avr. 2015

Re: Elasticsearch Version Upgrade

2015-04-21 Thread David Pilato
ary, or only > post 1.x? > >> On 22/04/2015 12:03 am, "David Pilato" wrote: >> This should work in both ways. >> >> The client knows what is the node version. >> The node knows what is the client version. >> >> So basically, if one kn

Re: No hits if fields are not stored

2015-04-21 Thread David Pilato
I’m not saying that you need to send all your data. But to send one document at least which is supposed to match. So then we can play with your script and try to fix it. -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr

Re: Elasticsearch Version Upgrade

2015-04-21 Thread David Pilato
client won’t provide X or Y value. So he won’t try to read it. That said, the best thing to do is to test it! :D -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmy

Re: Elasticsearch Version Upgrade

2015-04-21 Thread David Pilato
It should work fine. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 21 avr. 2015 à 14:08, Costya Regev a écrit : > > Hi , > > We have Elasticsearch Servers running with Es Version 1.4.2,our client > version is 1.4.1. > > We are about to upgrade our Es cluster Version

Re: No hits if fields are not stored

2015-04-21 Thread David Pilato
it_norms": true, > "type": "string" > } > } > } > }' > > #After indexing some data, do a search query > curl -XPOST /files/rawfiles/_search -d '{ > "from": 0, > "size": 50, > "sor

Re: No hits if fields are not stored

2015-04-21 Thread David Pilato
I don’t understand. Could you GIST a full recreation scripts which demonstrate what you are seeing? -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <http

Re: How to diagnose slow queries every 10 minutes exactly?

2015-04-21 Thread David Pilato
is. That was surprising > and something I was going to investigate more on my own, but perhaps these > two problems are related. > > I'm really hitting the limit of what I know how to troubleshoot with ES, > hence I am really hoping for help here :) > > >

Re: Index Size and Replica Impact

2015-04-20 Thread David Pilato
You don't have to set replicas to 3. It depends on the number of shards you have for your index. If you are using default (5), then you probably have today something like: Node 1 : 4 shards Node 2 : 3 shards Node 3 : 3 shards Each shard should be around 600mb size (If using all defaults). What

Re: Bulk Index from Remote Host

2015-04-20 Thread David Pilato
That's fine but you need to split your bulk into smaller bulk requests. Don't send a 10gb bulk in one call! :) David > Le 21 avr. 2015 à 00:40, TB a écrit : > > We are planning to bulk insert about 10 Gig data ,however we are being forced > to do this from a remote host. > Is this a good pract

Re: How to diagnose slow queries every 10 minutes exactly?

2015-04-20 Thread David Pilato
Could you run a hot_threads API call when this happens? Anything in logs about GC? BTW 200 indices is a lot for 2 nodes. And how many shards/replicas do you have? Why do you need so many indices for 2m docs? David > Le 21 avr. 2015 à 01:16, Dave Reed a écrit : > > I have a 2-node cluster runn

Re: Elasticseach issue with some indicies not populating data

2015-04-20 Thread David Pilato
that is. You can also add this output to make sure something is meant to be sent in elasticsearch: output { stdout { codec => rubydebug } elasticsearch { protocol => "http" host => "localhost" } } -- David Pilato - Developer | Evangel

Re: Elasticseach issue with some indicies not populating data

2015-04-20 Thread David Pilato
Having unassigned shards is perfectly fine on a one node cluster. The fact that your cluster were yellow does not mean your cluster was not behaving correctly. -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://tw

Re: Elasticseach issue with some indicies not populating data

2015-04-19 Thread David Pilato
n a truck we got it, if it’s fun we do it” – RealTruck.com > >> On Sun, Apr 19, 2015 at 2:38 PM, David Pilato wrote: >> From an Elasticsearch point of view, I don't see anything wrong. >> You have a way too much shards for sure so you might hit OOM exception or &g

Re: Elasticseach issue with some indicies not populating data

2015-04-19 Thread David Pilato
>From an Elasticsearch point of view, I don't see anything wrong. You have a way too much shards for sure so you might hit OOM exception or other troubles. So to answer to your question, check your Elasticsearch logs and if nothing looks wrong, check logstash. Just adding that Elasticsearch is

Re: How do element-based array query?

2015-04-18 Thread David Pilato
You need to define nested documents. -- David Pilato - Developer | Evangelist elastic.co @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr <https://twitter.com/elasticsearchfr> | @scrutmydocs <https://twitter.com/scrutmydocs> > Le 18 avr. 2015 à 16:33,

Re: how to change the store value of a field

2015-04-16 Thread David Pilato
No you need to reindex -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 16 avr. 2015 à 17:52, Antoine Brun a écrit : > > Hello, > > is there any simple way to update a mapping and change the store value of a > field? > I'm trying to enable _timestamp: > curl -X PUT htt

Re: timestamp

2015-04-16 Thread David Pilato
You need to reindex in a new index. -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > Le 16 avr. 2015 à 17:33, Antoine Brun a écrit : > > Hello, > > based on the comments I could create a new index with _timestamp activated > and it works great. > Now my probleme arrives wh

  1   2   3   4   5   6   7   8   9   10   >