Re: False positive results for spatial search

2015-01-20 Thread Colin Goodheart-Smithe
HI Luke, This looks like a bug in the geo_shape query. I have raised the following issue in the Elasticsearch repo: https://github.com/elasticsearch/elasticsearch/issues/9360 Thanks for raising this. Colin On Monday, 19 January 2015 20:02:08 UTC, Lukasz Smoron wrote: hi, I got simple

Advice on mapping/searching with lowercase

2015-01-20 Thread Maarten Roosendaal
Hi, I'm trying to figure out the correct mapping for searching for exact values but lowercased. I thought this would work: exact_lowercase: { filter: [ lowercase ], type: custom, tokenizer: keyword } .. and the

Re: When searching for 'Boss' with fuzziness, get higher score for 'Bose' than 'Boss'. ???? How Comes !?!?

2015-01-20 Thread Itamar Syn-Hershko
Famous last words :) -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Lucene.NET committer and PMC member On Tue, Jan 20, 2015 at 11:11 AM, Mark Harwood mark.harw...@elasticsearch.com wrote: it doesn't seem like this would

Re: When searching for 'Boss' with fuzziness, get higher score for 'Bose' than 'Boss'. ???? How Comes !?!?

2015-01-20 Thread Mark Harwood
it doesn't seem like this would address the IDF Trust me, I wrote it. On Tuesday, January 20, 2015 at 12:16:44 AM UTC, kasper...@yahoo.com wrote: Thanks Mark. Sounds like this issue affects a lot of people. I looked at your suggestion about FLT, and the ignore_tf parameter should help,

Re: asciifolding character filter

2015-01-20 Thread Mathijs Biesmans
Thanks Jörg. So I guess the character filter didn't existed. Will this be pushed to official releases? Currently I'm using a hosted ES-cluster, and I don't want to install custom plugins... Op maandag 19 januari 2015 20:39:50 UTC+1 schreef Jörg Prante: Hey, cool idea. That's fairly easy to

Re: When searching for 'Boss' with fuzziness, get higher score for 'Bose' than 'Boss'. ???? How Comes !?!?

2015-01-20 Thread Mark Harwood
:) It's been nearly 10 years but I took a quick look at the code and the IDF balancing stuff is still in there. Testing queries against a large index of cars Lucene's standard fuzzy query on ford~ still has top matches that aren't Fords. FLT works fine. On Tuesday, January 20, 2015 at

Re: Advice on mapping/searching with lowercase

2015-01-20 Thread David Pilato
At index time, you are using your exact_lowercase analyzer. Steel Wheels is indexed as steel wheels. When searching for steel wheels it obviously does not match. Why not using the same analyzer at search and index time? -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 20

Elasticsearch at Google Cloud Engine

2015-01-20 Thread Klausen Schaefersinho
Hi, I have just the click to deploy features to set up a small ElasticSearch cluster. That seems to have worked fine and I can connect to the cluster of rest. For instance curl http://ip:9200 will return { status : 200, name : elasticsearch-8tqw, cluster_name :

Registered Search Templates in multi search API (_msearch)

2015-01-20 Thread Dany Gielow
Hi, Is it possible to use registered search templates in a multi search api (_msearch) request? If so, how? I tried giving the following query in a body. {template:{id:template_1},params:{a:1, b:2}} But that leads to the error: Parse Failure [No parser for element [template]] I tried

Re: Elasticsearch at Google Cloud Engine

2015-01-20 Thread Itamar Syn-Hershko
This requires port 9300 to be open on the cloud for you (UDP), and for your client code to set the cluster name correctly -- Itamar Syn-Hershko http://code972.com | @synhershko https://twitter.com/synhershko Freelance Developer Consultant Lucene.NET committer and PMC member On Tue, Jan 20,

Re: Stuck with re-syncing mappings with cluster state for types ... after upgrade 0.90.9 to 1.4.2

2015-01-20 Thread Eike Dehling
Thanks for the response. We did not upgrade java, we're running java 1.7.0_45, and Elasticsearch 1.4.2. For anyone else running into this: We did manage to solve the issue by closing and re-opening all indices. After that the continuous re-synchronization stopped. Certain events start it again

Hadoop Training in Hyderabad

2015-01-20 Thread madhu k
Hadoop Training in Hyderabad, Online Hadoop Training in USA, Online Hadoop Training in UK, Online Hadoop Training in Canada, Online Hadoop Training in Austria, Online Hadoop Training in Singapore,Online Hadoop Training in ameerpet http://www.primarkinfotech.com/hadoop.php, Hyderabad,

How to filter by date range in quarter using elasticsearch

2015-01-20 Thread Wasae Shoaib
I have an index /testdate/daterange with the following mapping PUT /testdate/daterange/_mapping { daterange: { properties: { text:{ type: string }, date: { type: date, format: dateOptionalTime }

My Own Phrase suggestion as module.

2015-01-20 Thread Silvestre Losada
Hi All, I have a problem with phrase suggesttion that is giving me some wrong results for my needs. So I need that termfrequency has not be taken into account. I want to add my own phrase suggester and then be able to use it from _search endpoint. curl -XPOST 'localhost:9200/_search' -d {

Re: Migrating lucene drill sideways query to elasticsearch

2015-01-20 Thread Bo Finnerup Madsen
I think you are right :) I tried to implement it, and performance seems to be good. Thank you (and Michael) very much for the help. Den fredag den 16. januar 2015 kl. 22.18.17 UTC+1 skrev Jörg Prante: I do not think you have to worry, I use a dozen of aggregations with filters with success

Re: Elasticsearch at Google Cloud Engine

2015-01-20 Thread Klausen Schaefersinho
Hi, using the transport client solved my issue. It seems the server lookup does not work. Cheers, Klaus On Tuesday, January 20, 2015 at 12:54:11 PM UTC+1, Itamar Syn-Hershko wrote: This requires port 9300 to be open on the cloud for you (UDP), and for your client code to set the cluster

Kibana query language - how do I filter fieldAfieldB

2015-01-20 Thread Gregory Touretsky
I want to filter my ES data via Kibana by comparing 2 numeric fields. Sample record: fieldA=25 fieldB=38 msg=whatever I need a way to show only records where fieldAfieldB The only suggestion I found so far is fieldA:[@fieldB TO *] However, this doesn't work - I get no results, although I

Re: Configure Kibana for HTTPS

2015-01-20 Thread Magnus Bäck
On Tuesday, January 20, 2015 at 14:54 CET, Karthik M karthik4...@gmail.com wrote: On Tuesday, January 20, 2015 at 2:17:49 AM UTC-5, Magnus Bäck wrote: I want the front end of ES (kibana) to run on SSL but keep the backend connection from Kibana to ES unencrypted since both are

Re: Configure Kibana for HTTPS

2015-01-20 Thread Karthik M
Kibana version 3 On Tuesday, January 20, 2015 at 2:17:49 AM UTC-5, Magnus Bäck wrote: On Monday, January 19, 2015 at 15:45 CET, Karthik M karth...@gmail.com javascript: wrote: I want the front end of ES (kibana) to run on SSL but keep the backend connection from Kibana to ES

Re: Configure Kibana for HTTPS

2015-01-20 Thread Magnus Bäck
On Tuesday, January 20, 2015 at 15:59 CET, Karthik Gmail karthik4...@gmail.com wrote: On Jan 20, 2015, at 9:01 AM, Magnus Bäck magnus.b...@sonymobile.com wrote: Kibana 3 doesn't have a backend. The connections to Elasticsearch originate from your browser so you'll want to encrypt

Re: Configure Kibana for HTTPS

2015-01-20 Thread Karthik Gmail
Thanks Magnus. Would be able to point me towards on how to set that up? On Jan 20, 2015, at 9:01 AM, Magnus Bäck magnus.b...@sonymobile.com wrote: On Tuesday, January 20, 2015 at 14:54 CET, Karthik M karthik4...@gmail.com wrote: On Tuesday, January 20, 2015 at 2:17:49 AM UTC-5,

Inconsistent query times using warmers

2015-01-20 Thread Darshat Shah
Hi, I've setup warmers for some aggregation queries. But the times for aggregation are still inconsistent. A single query takes upto 20s but if run in rapid succession, the same query times come down to 5s and then under 1s. After some time again if the query is run, the time taken goes back

Re: corruption when indexing large number of documents (4 billion+)

2015-01-20 Thread Darshat Shah
So an update on this. I set aside 7 dedicated master eligible nodes, and ensured the data upload connection pool only round robins over the data nodes. Updated batch size to 5k. With this, upload of about 6Billion documents (5TB of data) went through in about 12 hours. Additionally on windows

Basic authentication for each Kibana dashboards

2015-01-20 Thread Taylor Wood
I am looking to require passwords to access specific dashboards in kibana. I am using apache and currently have basic authentication working for the site as a whole but want to lock it down even more so only some users have access to specific dashboards. Kabana v3 Below is my httpd conf

Role of the analyzer in the completion suggester

2015-01-20 Thread Michele Palmia
The completion suggester is based on a field that has to be mapped with type *completion.* This has by default index- and search-analyzer set to *simple*, but a different analyzer can be freely set. After the indexing process, the input text is stored in a finite state transducer, and

Re: Configure Kibana for HTTPS

2015-01-20 Thread Iain Woolf
On Tuesday, January 20, 2015 at 10:03:47 AM UTC-5, Magnus Bäck wrote: On Tuesday, January 20, 2015 at 15:59 CET, Karthik Gmail karth...@gmail.com javascript: wrote: On Jan 20, 2015, at 9:01 AM, Magnus Bäck magnu...@sonymobile.com javascript: wrote: Kibana 3 doesn't have

behavior of scrolling search with from parameter in newer ES versions

2015-01-20 Thread Gordon Tillman
Greetings All, I ran into an interesting issue when upgrading from ES version 1.0.1 to newer versions. In particular, I tested the following with versions 1.2.4, 1.3.4 and 1.4.2. *summary* When doing a normal scroll search (not one with a *search_type=scan*), it appears that the from

Removing multiple snapshots using the API

2015-01-20 Thread abutalib zaidi
I am fairly new to ElasticSearch and was trying to do some backup and restore using the API. I wrote a shell script to remove snapshots older than certain days but it somehow removes the repo instead of the snapshots info stored in a variable. Any help would be appreciated: #Remove snapshots

Re: In jmv gc log message, what are two numbers after [gc][old] and before duration?

2015-01-20 Thread Jinyuan Zhou
I figured out that second number 169 is number of old gc events by run jstat -gcold pid. But still don't know the first number:15221. Jinyuan (Jack) Zhou On Mon, Jan 19, 2015 at 11:10 AM, Jinyuan Zhou zhou.jiny...@gmail.com wrote: Hi, I copied this log message from some postings. there two

Re: asciifolding character filter

2015-01-20 Thread joergpra...@gmail.com
I can't tell. Official elasticsearch ICU plugin is lagging behind Lucene 5.0, ICUCollationKeyAnalyzer / collation key field type, API deprecation updates etc. so I hope it soon will take up pace again to get ready for new features. Jörg On Tue, Jan 20, 2015 at 10:18 AM, Mathijs Biesmans

Re: JsonObject to SortBuilder object

2015-01-20 Thread Alex Thurston
Here's a simpler way to ask the question: I've got this: GeoDistanceSortBuilder sorter = new GeoDistanceSortBuilder(values.geo_location); sorter.point(0.0, 0.0); sorter.order(SortOrder.DESC); sorter.unit(DistanceUnit.KILOMETERS); sorter.geoDistance(GeoDistance.PLANE); sorter.toString() Which

Re: elk 1.4.2 + kibana-3.1.2 Connection Failed, add http.cors.enabled: true does not help.

2015-01-20 Thread Ashit Kumar
Remove the Kibana directory and reinstall it. That is the only thing that worked for me. Could not figure out what to clear. Ash On Monday, January 19, 2015 at 3:05:15 AM UTC-5, EShen wrote: 1. install elastic search 1.4.2 (up and running http://hostip:9200 works) 2. install kibana 3.1.2

Heavy indexing cause severe delay for searching

2015-01-20 Thread Hajime
Hi, I have a 10 indexes with 100 shards using 10 nodes which have 40 cores each. I have a problem when indexing 1000~2000/doc or 2M~4M per sec in new index, searching to the other old indexes happen to be very slow. How can I prevent from indexing to interfere the searching activity?

Re: Heavy indexing cause severe delay for searching

2015-01-20 Thread Hajime
I have 440 cores in my cluster and I find that having 100 shards are much more faster in searching compare to 20 shards. Does having many shards per index too costly, even I have so much cores? On Wed, Jan 21, 2015 at 12:12 PM, Mark Walkom markwal...@gmail.com wrote: Is that 100 shards per

Re: Slow Commands with 1.2.4 to 1.4.2 Upgrade

2015-01-20 Thread pskieu
Lowered to 16g and verified with top--same behavior. I've also tried 1.3 with the same results. 1.2 is the latest version that's working properly for me. On Saturday, January 17, 2015 at 5:02:03 AM UTC-8, Arie wrote: With 32GB of memory this should be around 16GB (50%), and check if this

Logging queries run via search templates

2015-01-20 Thread ppearcy
Hi, I am running some template search queries on 1.4.2 via Java APIs like so: esClient.prepareSearch(index) .setTemplateName(templateName) .setTemplateParams(templateParams) .setTemplateType(ScriptService.ScriptType.INDEXED) When I log the request, I get an empty json

Re: Heavy indexing cause severe delay for searching

2015-01-20 Thread Mark Walkom
Is that 100 shards per index? If to it's too many and you should drop that to 10-20 per index. Check your hot threads, it might show something. On 21 January 2015 at 15:09, Hajime placeofnomemor...@gmail.com wrote: Hi, I have a 10 indexes with 100 shards using 10 nodes which have 40 cores

Re: Heavy indexing cause severe delay for searching

2015-01-20 Thread David Pilato
But on different nodes? -- David ;-) Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 21 janv. 2015 à 07:28, Hajime placeofnomemor...@gmail.com a écrit : I run on parallel using REST API On Wed, Jan 21, 2015 at 2:42 PM, David Pilato da...@pilato.fr wrote: Did you try to send each

Re: Heavy indexing cause severe delay for searching

2015-01-20 Thread Hajime
I run on parallel using REST API On Wed, Jan 21, 2015 at 2:42 PM, David Pilato da...@pilato.fr wrote: Did you try to send each bulk request on different nodes in parallel? Or does only one node get the BULK request? David Le 21 janv. 2015 à 05:47, Hajime placeofnomemor...@gmail.com a écrit

Re: Slow Commands with 1.2.4 to 1.4.2 Upgrade

2015-01-20 Thread Mark Walkom
Do you have a firewall between you and the cluster? What sort of times does a ping response show you? Have you looked at your ES logs, hot threads? On 21 January 2015 at 14:51, psk...@gmail.com wrote: Lowered to 16g and verified with top--same behavior. I've also tried 1.3 with the same

Re: Heavy indexing cause severe delay for searching

2015-01-20 Thread David Pilato
Did you try to send each bulk request on different nodes in parallel? Or does only one node get the BULK request? David Le 21 janv. 2015 à 05:47, Hajime placeofnomemor...@gmail.com a écrit : I have 440 cores in my cluster and I find that having 100 shards are much more faster in searching

GroovyScriptCompilationException: MultipleCompilationErrorsException when attempting to ingest data into ES 1.4.1

2015-01-20 Thread Lewis John Mcgibbney
Hi Folks, I recently set up a new ES cluster and attempting to ingest data using Apache Nutch 1.10-SNAPSHOT e.g. Nutch trunk I paste two stack traces. The former from the Nutch job, the latter from the ES log. Can someone please help me to interpret the latter log? On this list I've seen