date:20150105

Re: Elasticsearch inserting date type documents as UTC timezone datetime while indexing

2015-01-05 Thread Masaru Hasegawa

HI,

XContentBuilder (I assume jsonBuilder() returns it) serialises date using UTC
timezone by default.
If you’d like to use different format, you’d need to build your own
DateTimeFormatter and pass it when you add date type field.

For example, to use ISO date time format with specified timezone:

DateTimeFormatter formatter =
ISODateTimeFormat.dateTime().withZone(DateTimeZone.forTimeZone(TimeZone.getTimeZone(ZONE_NAME)));
:
.field(@timestamp, date, formatter)
:
.field(creation_time, date, formatter)
:

Masaru

On January 5, 2015 at 13:45:19, Subhadip Bagui (i.ba...@gmail.com) wrote:
Hi,

Please help with suggestions.

For more options, visit https://groups.google.com/d/optout.

Re: What is ES / Marvel doing to my configurations?

2015-01-05 Thread Mark Walkom

index level settings will override cluster level ones.

On 6 January 2015 at 15:11, Chris Neal chris.n...@derbysoft.net wrote:

 Hi all.

 My elasticsearch.yml file has these settings with regards to merging:

 index:
   codec:
 bloom:
   load: false
   merge:
 policy:
   max_merge_at_once: 4
   max_merge_at_once_explicit: 4
   max_merged_segment: 1gb
   segments_per_tier: 4
   type: tiered
 scheduler:
   max_thread_count: 1
   type: concurrent
   number_of_replicas: 0
   number_of_shards: 1
   refresh_interval: 5s

 My elasticsearch.log file on server startup shows me this:

 [2015-01-06 04:04:53,320][INFO ][cluster.metadata ]
 [elasticsearch-ip-10-0-0-42] [.marvel-2015.01.06] update_mapping
 [cluster_state] (dynamic)
 [2015-01-06 04:04:56,704][INFO ][index.merge.policy   ]
 [elasticsearch-ip-10-0-0-42] [.marvel-2015.01.06][0] updating
 [segments_per_tier] from [4.0] to [10.0]
 [2015-01-06 04:04:56,704][INFO ][index.merge.policy   ]
 [elasticsearch-ip-10-0-0-42] [.marvel-2015.01.06][0] updating
 [max_merge_at_once] from [4] to [10]
 [2015-01-06 04:04:56,704][INFO ][index.merge.policy   ]
 [elasticsearch-ip-10-0-0-42] [.marvel-2015.01.06][0] updating
 [max_merge_at_once_explicit] from [4] to [30]
 [2015-01-06 04:04:56,704][INFO ][index.merge.policy   ]
 [elasticsearch-ip-10-0-0-42] [.marvel-2015.01.06][0] updating
 [max_merged_segment] from [1024.0mb] to [5gb]

 It sure looks like ES is overriding what is in the .yml file to me, but
 I'm not sure why.  Does anyone have an idea?

 Much appreciated!
 Chris

 --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAND3DpjXD1XqRUDxQt5zOoG-ho%2BdRrUgY5ejKw9%2BXMy_sbYkLA%40mail.gmail.com
 https://groups.google.com/d/msgid/elasticsearch/CAND3DpjXD1XqRUDxQt5zOoG-ho%2BdRrUgY5ejKw9%2BXMy_sbYkLA%40mail.gmail.com?utm_medium=emailutm_source=footer
 .
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_JdfH3SzG1OwYk4TH6z7q%2BZgFyA_qNzUjwOE8Xt959cw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: nested aggregation for parent/child relations?

2015-01-05 Thread Peter Pohar

Hey guys,

I'm interested in similar architecture. I have company - employee -
availability structure. Any update on grandchild aggregation?
While on 2nd level I get data, on 3rd level I'm always getting doc_count 0.
Or is this completely different issue?

Best,
Pete

On Tuesday, 20 May 2014 16:04:00 UTC+2, Jad wrote:

Thanks, Adrien!
On May 20, 2014 12:19 AM, Adrien Grand adrien...@elasticsearch.com
javascript: wrote:

Hi Jad,

This is a feature that we are considering but parent/child joins are
typically much more expensive to do than nested (block) joins so we still
need to figure out if we can expose such an aggregation with a reasonable
cost.

On Mon, May 19, 2014 at 7:03 PM, Jad jna...@gmail.com javascript:
wrote:

Hi,

Does anyone know if it is possible to do aggregations that are similar
to the nested aggregation but for parent/child relationships? I have the
following example:

parent doc:
{
_id: 123456,
name: I am the parent
}

child1:
{
_parent: 123456,
country: Germany
}

child2:
{
_parent: 123456,
language: German
}

I would like to execute a terms aggregation by language and a terms
sub-aggregation by country. Is that possible? I've done it with nested
relationships instead of parent/child, but we update this data often, and
nested relationships are harder to insert.

Many thanks!

Jad.

https://groups.google.com/d/msgid/elasticsearch/587022cc-cedd-48e9-b6ba-4cef8c996432%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to a topic in the
Google Groups elasticsearch group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/Ojd2ZyEMV3I/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearc...@googlegroups.com javascript:.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j7qcke7DbXwwDAE100pQ2AkA6796E5o_ooT-4VwgV06Tg%40mail.gmail.com

https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j7qcke7DbXwwDAE100pQ2AkA6796E5o_ooT-4VwgV06Tg%40mail.gmail.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/b4b71907-cf4b-4bc5-89a5-0f9cb8da03d0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Issue with multi_match type cross_fields

2015-01-05 Thread Anoop P R

Is multi_match cross_fields works in Elastic search version 1.0.1? 
If not is there is any alternative solution for implementing this?

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/05dcf92b-ac74-4091-a6d1-597aa8afcb3d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

elastic search architecutre help

2015-01-05 Thread phani . nadiminti

 Hi All,

 Can we maintain common data repository (data folder) for all the data 
nodes in a cluster?

can we maintain common data folder for dedicated data nodes ? will 
this be possible (common data repository for elastic search instead of 
maintain data folder on each node)

  

Thanks

phani

  

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/5320f96f-0c8c-4653-b7da-317573ed492b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elasticsearch logging

2015-01-05 Thread Magnus Bäck

On Friday, January 02, 2015 at 19:20 CET,
Jesse Redl jr...@vendasta.com wrote:

When reviewing the logs generated by elasticsearch (1.4.x), a single
log message is being split across multiple lines?

Yes, that's normal for Java logs.

Is this configurable within logging.yml? My undstanding is that
elasticsearch is powered by log4j however, I'm not familiar with
this product, nor can I find any decent documentation on the
logging.yml file.

I suspect the reason you're asking is that you're considering
ingesting the Elasticsearch logs with Logstash, in which case
you should look into the JSONEventLayoutV1 Log4j layout
(https://github.com/logstash/log4j-jsonevent-layout) to get the
logs in JSON format. You should be able to drop that jar file
along with its dependencies (net.minidev:json-smart:1.1.1 and
commons-lang:commons-lang:2.6) into the Elasticsearch lib directory
(typically /usr/share/elasticsearch/lib) and adjust logging.yml
to use that layout. See also
https://github.com/elasticsearch/elasticsearch/issues/8786.

--
Magnus Bäck| Software Engineer, Development Tools
magnus.b...@sonymobile.com | Sony Mobile Communications

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/20150105095131.GA12719%40seldlx20533.corpusers.net.
For more options, visit https://groups.google.com/d/optout.

Re: Modify the index setting after the index created ? what's the function of search_quote_analyzer ?

2015-01-05 Thread shallweqin

Here is what i get from 
http://search.cpan.org/~drtech/Elastic-Model-0.27/lib/Elastic/Manual/Attributes.pod#search_quote_analyzer

search_quote_analyzer sets the analyzer 
http://search.cpan.org/~drtech/Elastic-Model-0.27/lib/Elastic/Manual/Attributes.pod#analyzer
 
to use in a Query-String query 
http://www.elasticsearch.org/guide/reference/query-dsl/query-string-query.html
 
or Field query 
http://www.elasticsearch.org/guide/reference/query-dsl/field-query.html 
when the search phrase includes quotes (). If not set, then it falls back 
to thesearch_analyzer 
http://search.cpan.org/~drtech/Elastic-Model-0.27/lib/Elastic/Manual/Attributes.pod#search_analyzer
 
or the analyzer 
http://search.cpan.org/~drtech/Elastic-Model-0.27/lib/Elastic/Manual/Attributes.pod#analyzer
.

在 2014年6月13日星期五UTC+8上午11时32分14秒，Ivan Ji写道：

 Hi all,

 I want to modify one field's search analyzer from standard to keyword 
 after the index created. So I try to PUT mapping :

 $ curl -XPUT 'http://localhost:9200/qindex/main/_mapping' -d '
 {
 main : {
 properties : {
 name : { type: string, index: analyzed, 
 index_analyzer: filename_ngram, search_analyzer: keyword}
 }
 }
 }
 '


 The operation seems succeed. Because I expect it might conflict, what 
 would the situations that conflict might occur? This is my first question.

 Anyway then I try to get the mapping out: (partial)

   name: {
 type: string,
 index_analyzer: filename_ngram,
 search_analyzer: keyword,
 include_in_all: true,
 search_quote_analyzer: standard
 }


  So I am wondering whether my operation succeeded? and what is the 
 search_quote_analyzer function?  And it still remains standard, does it 
 matter?

 Could anyone answer me these questions?

 Cheers,

 Ivan


在 2014年6月13日星期五UTC+8上午11时32分14秒，Ivan Ji写道：

 Hi all,

 I want to modify one field's search analyzer from standard to keyword 
 after the index created. So I try to PUT mapping :

 $ curl -XPUT 'http://localhost:9200/qindex/main/_mapping' -d '
 {
 main : {
 properties : {
 name : { type: string, index: analyzed, 
 index_analyzer: filename_ngram, search_analyzer: keyword}
 }
 }
 }
 '


 The operation seems succeed. Because I expect it might conflict, what 
 would the situations that conflict might occur? This is my first question.

 Anyway then I try to get the mapping out: (partial)

   name: {
 type: string,
 index_analyzer: filename_ngram,
 search_analyzer: keyword,
 include_in_all: true,
 search_quote_analyzer: standard
 }


  So I am wondering whether my operation succeeded? and what is the 
 search_quote_analyzer function?  And it still remains standard, does it 
 matter?

 Could anyone answer me these questions?

 Cheers,

 Ivan


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/7bc327c0-6583-4a6b-877c-3fc933feb611%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Modify the index setting after the index created ? what's the function of search_quote_analyzer ?

2015-01-05 Thread shallweqin

Here is what i get from 
http://search.cpan.org/~drtech/Elastic-Model-0.27/lib/Elastic/Manual/Attributes.pod#search_quote_analyzer

search_quote_analyzer sets the analyzer 
http://search.cpan.org/~drtech/Elastic-Model-0.27/lib/Elastic/Manual/Attributes.pod#analyzer
 to 
use in a Query-String query 
http://www.elasticsearch.org/guide/reference/query-dsl/query-string-query.html
 or Field query 
http://www.elasticsearch.org/guide/reference/query-dsl/field-query.html when 
the search phrase includes quotes (). If not set, then it falls back to 
thesearch_analyzer 
http://search.cpan.org/~drtech/Elastic-Model-0.27/lib/Elastic/Manual/Attributes.pod#search_analyzer
 or 
the analyzer 
http://search.cpan.org/~drtech/Elastic-Model-0.27/lib/Elastic/Manual/Attributes.pod#analyzer
.
在 2014年6月13日星期五UTC+8上午11时32分14秒，Ivan Ji写道：

 Hi all,

 I want to modify one field's search analyzer from standard to keyword 
 after the index created. So I try to PUT mapping :

 $ curl -XPUT 'http://localhost:9200/qindex/main/_mapping' -d '
 {
 main : {
 properties : {
 name : { type: string, index: analyzed, 
 index_analyzer: filename_ngram, search_analyzer: keyword}
 }
 }
 }
 '


 The operation seems succeed. Because I expect it might conflict, what 
 would the situations that conflict might occur? This is my first question.

 Anyway then I try to get the mapping out: (partial)

   name: {
 type: string,
 index_analyzer: filename_ngram,
 search_analyzer: keyword,
 include_in_all: true,
 search_quote_analyzer: standard
 }


  So I am wondering whether my operation succeeded? and what is the 
 search_quote_analyzer function?  And it still remains standard, does it 
 matter?

 Could anyone answer me these questions?

 Cheers,

 Ivan


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/c8c53f74-e5b0-4207-8c0e-77085961c54e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: elastic search upgrade issues

2015-01-05 Thread Magnus Bäck

On Monday, January 05, 2015 at 10:13 CET, phani.nadimi...@goktree.com wrote:

   Thank you for the quick reply. i upgraded both nodes in elastic
 search.
   the following are the diskspaces in each node
   node 1 : 3Gb available out of 35 GB
   node 2 : 4GB available out of 35 GB
   is this be a problem?
 yes java and elastic search has same versions on each nodes.
  java version using : 1.7.0_55

Yes, this is probably your problem. By default Elasticsearch won't
allocate shards on nodes with 15% free disk space.

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-allocation.html#disk

-- 
Magnus Bäck| Software Engineer, Development Tools
magnus.b...@sonymobile.com | Sony Mobile Communications

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/20150105095715.GB12719%40seldlx20533.corpusers.net.
For more options, visit https://groups.google.com/d/optout.

Badly corrupted index

2015-01-05 Thread James Green

Our storage server threw it's toys of the pram. We got it back, and
everything bar ElasticSearch is singing sweetly again.

On start, we get an IndexFormatTooNewException. Not many get that one, but
CheckIndex appears to be worth a go, except that also barfs:

nicks@elasticsearch:/var/lib/elasticsearch/elasticsearch/nodes/0/indices/support/3/index$
sudo java -cp /usr/share/elasticsearch/lib/lucene-core-4.8.1.jar
-ea:org.apache.lucene... org.apache.lucene.index.CheckIndex . -fix
Opening index @ .
ERROR: could not read any segments file in directory
org.apache.lucene.index.IndexFormatTooNewException: Format version is not
supported (resource:
BufferedChecksumIndexInput(MMapIndexInput(path=/var/lib/elasticsearch/elasticsearch/nodes/0/indices/support/3/index/segments.gen))):
1071082519 (needs to be between -2 and -3)
at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:803)
at
org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:710)
at org.apache.lucene.index.SegmentInfos.read(SegmentInfos.java:412)
at
org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:393)
at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:1967)

This has rendered our ES service offline. Any suggestions to fix it would
be appreciated, we are uncertain how to proceed otherwise.

ES 1.2.1 from Debian packages. We are just (now) upgrading to 1.4 to see if
this results in some progress.

James

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAMH6%2BaxYVvHEnjLocbeHh9v5pjReZ0aDds0aOtkjy1sDGmfY1w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Marvel/Sense security?

2015-01-05 Thread John Bohne

Why would I want to do that? I'm using Apache by the way.

I saw something about marvel.agent.exporter.es.hosts and setting up
user:pwd@host:9200
but the documentation doesn't really specify where to do this. In
elasticsearch.yml? Under what heading? Would this even solve the problem?

On Sunday, January 4, 2015 12:15:23 PM UTC-6, David Pilato wrote:

If you have disabled access to PUT method (with Nginx for example), SENSE
won't be able to PUT any document.

My 2 cents.

--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 4 janv. 2015 à 17:16, John Bohne johnb...@gmail.com javascript: a
écrit :

I was looking in the documentation and online for Marvel but I am not sure
how to implement security for Marvel. For example, if one were to access
example.com:9200/_plugin/marvel/sense/index.html , one would be able to
put documents through Sense on my cluster(s). I saw something about
marvel.hosts but I am not sure how to implement that.

https://groups.google.com/d/msgid/elasticsearch/559eadaa-37cb-4129-8a65-38847c60126f%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/22d92adf-b8ab-4b7c-a955-64c05a9a96af%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: kibana empty dashboard

2015-01-05 Thread Eric



Here are the versions that I'm running:

# Kibana version
Kibana 3.1.2-07bbd7e
eeded13255f154eaeceb4cf83105e4b4  kibana-3.1.2.tar.gz

# Logstash version
[root@elk ~]# /opt/logstash/bin/logstash version
logstash 1.4.2-modified
1db9f0864ff4b89380b39c39bc419031  logstash-1.4.2-1_2c0f5a1.noarch.rpm

# Elasticsearch version
[root@elk ~]# /usr/share/elasticsearch/bin/elasticsearch -v
Version: 1.4.2, Build: 927caff/2014-12-16T14:11:12Z, JVM: 1.7.0_51
6e2061f0734f9dbab263c1616701c1fe  elasticsearch-1.4.2.noarch.rpm

# OS
CentOS (CentOS-7.0-1406-x86_64-Everything.iso)
Installed packages: Basic Web Server + Development tools

Logstash runs fine. Elasticsearch runs fine. Kibana runs, but only shows 
the screenshot shown below at, https://logstasht/#/dashboard


https://lh3.googleusercontent.com/-8mIiX5lKJ_U/VKpmMkRSftI/AAACYWM/v4LxHMzEAGI/s1600/kibana.png




On Wednesday, May 14, 2014 6:56:03 PM UTC-4, Mark Walkom wrote:

 I think you have extra quotes causing a problem, try - elasticsearch: 
 http://192.168.10.25:9200;,

 Regards,
 Mark Walkom

 Infrastructure Engineer
 Campaign Monitor
 email: ma...@campaignmonitor.com javascript:
 web: www.campaignmonitor.com


 On 15 May 2014 05:58, itbla...@gmail.com javascript: wrote:

 I have the following is showing up when I pull up my kibana dashboard: 

 http://192.168.10.25/#/dashboard

  {{dashboard.current.title}} 

 When I tail my logs I see the following 
 2014/05/14 13:31:45 [error] 17152#0: *7 open() 
 /var/www/kibana/app/diashboards/dashboard failed (2: No such file or 
 directory), client: 192.168.11.53, server: 192.168.10.25, request: GET 
 /app/diashboards/dashboard HTTP/1.1, host: 192.168.10.25 

 I have been pulling my hair out over this, all help would be appreciated 

 This is my config.js 

  /** @scratch /configuration/config.js/2 
* === Parameters 
*/ 
   return new Settings({ 

 /** @scratch /configuration/config.js/5 
  *  elasticsearch 
  * 
  * The URL to your elasticsearch server. You almost certainly don't 
  * want +a 
 href=http://localhost:9200+;http://localhost:9200+ here. Even if Kibana 
 and Elasticsearch are on 
  * the same host. By default this will attempt to reach ES at the 
 same host you have 
  * kibana installed on. You probably want to set it to the FQDN of 
 your 
  * elasticsearch host 
  */ 
 elasticsearch: http://192.168.10.25:9200;, 
 /*elasticsearch: http://+window.location.hostname+:9200;, 

 /** @scratch /configuration/config.js/5 
  *  default_route 
  * 
  * This is the default landing page when you don't specify a 
 dashboard to load. You can specify 
  * files, scripts or saved dashboards here. For example, if you had 
 saved a dashboard called 
  * `WebLogs' to elasticsearch you might use: 
  * 
  * +default_route: '/dashboard/elasticsearch/WebLogs',+ 
  */ 
 default_route : '/dashboard/file/default.json', 

 /** @scratch /configuration/config.js/5 
  *  kibana-int 
  * 
  * The default ES index to use for storing Kibana specific object 
  * such as stored dashboards 
  */ 
 kibana_index: kibana-int, 

 /** @scratch /configuration/config.js/5 
  *  panel_name 
  * 
  * An array of panel modules available. Panels will only be loaded 
 when they are defined in the 
  * dashboard, but this list is used in the add panel interface. 
  */ 
 panel_names: [ 
   'histogram', 
   'map', 
   'pie', 
   'table', 
   'filtering', 
   'timepicker', 
   'text', 
   'hits', 
   'column', 
   'trends', 
   'bettermap', 
   'query', 
   'terms', 
   'stats', 
   'sparklines' 
 ] 
   }); 
 }); 

 ngix (default)

 /** @scratch /configuration/config.js/1
  * == Configuration
  * config.js is where you will find the core Kibana configuration. This 
 file contains parameter that
  * must be set before kibana is run for the first time.
  */
 define(['settings'],
 function (Settings) {


   /** @scratch /configuration/config.js/2
* === Parameters
*/
   return new Settings({

 /** @scratch /configuration/config.js/5
  *  elasticsearch
  *
  * The URL to your elasticsearch server. You almost certainly don't
  * want +http://localhost:9200+ here. Even if Kibana and 
 Elasticsearch are on
  * the same host. By default this will attempt to reach ES at the 
 same host you have
  * kibana installed on. You probably want to set it to the FQDN of 
 your
  * elasticsearch host
  */
 elasticsearch: http://192.168.10.25:9200;,


  -- 
 You received this message because you are subscribed to the Google Groups 
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to elasticsearc...@googlegroups.com javascript:.
 To view this discussion on the web visit

Re: Data is not saved equally in each datanode

2015-01-05 Thread Xiaoliang Tian

Thanks,I got that

2014-12-27 4:54 GMT+08:00 Mark Walkom markwal...@gmail.com:

 You really need to upgrade, 0.90.X is no longer supported!

 On 26 December 2014 at 17:19, Xiaoliang Tian xiaoliang.t...@gmail.com
 wrote:

 Thanks,And M using 0.9.13.can it enable auto-balancing manually?

 2014-12-26 14:17 GMT+08:00 Michael deMan (ES) elasticsea...@deman.com:


 http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-reroute.html

 I can’t remember when auto-balancing got enabled by default, I think
 maybe 1.3.4.

 you can find out via api.

 http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-cluster.html



 On Dec 25, 2014, at 10:15 PM, Xiaoliang Tian xiaoliang.t...@gmail.com
 wrote:

 which version? and what is the API url exactly

 2014-12-26 12:44 GMT+08:00 Michael deMan (ES) elasticsea...@deman.com:

 Also, higher shards will help with the new indexes but not the existing
 ones.
 You can use the API to force ES to move shards off your ‘full’ disk
 over to the new one.
 Auto-balancing for data size should be on by default if you are running
 a newer version of ES.

 On Dec 25, 2014, at 8:13 PM, Michael deMan (ES) 
 elasticsea...@deman.com wrote:

 Try increasing the number of shards - maybe to 20 or 40.

 On Dec 25, 2014, at 8:10 PM, Xiaoliang Tian xiaoliang.t...@gmail.com
 wrote:

 index  number depend on how many days past,the index name is the day
 epoch. because we use elasticseach for log storage
 shard number is 8
 replica is 1

 2014-12-25 15:49 GMT+08:00 David Pilato da...@pilato.fr:

 How many index/shards/replicas do you have?

 --
 David ;-)
 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

 Le 25 déc. 2014 à 08:28, xiaoliang tian xiaoliang.t...@gmail.com a
 écrit :

 I also start a new data node(node 5),and there is new data come into
 cluster continuously,but There is no any data in data node 5.I don't know
 why,plz help

 在 2014年12月25日星期四UTC+8上午10时33分29秒，xiaoliang tian写道：

 Hi,I got 4 data nodes,1 master node and 1 search node


 For example,At first,the data was equally saved in 4 data nodes
 node1 1.6TB
 node2 1.6TB
 node3 1.6TB
 node4 1.6TB

 Since My disk is 2TB and it is almost full,I delete  some index to
 get more storage

 after deleting the data nodes are like below

 node1 1TB
 node2 1TB
 node3 1TB
 node4 1TB

 after a few days, I found the data is not equally saved in each node
 anymore

 node1 1.1TB
 node2 1.1TB
 node3 1.1TB
 node4 1.6TB

 node 4 is almost full
 I don't know why,and is there anyway to rebalance data in each data
 node


 --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/9766ba6d-f7b1-4071-8f4b-b2b5c6a14085%40googlegroups.com
 https://groups.google.com/d/msgid/elasticsearch/9766ba6d-f7b1-4071-8f4b-b2b5c6a14085%40googlegroups.com?utm_medium=emailutm_source=footer
 .
 For more options, visit https://groups.google.com/d/optout.


 --
 You received this message because you are subscribed to a topic in the
 Google Groups elasticsearch group.
 To unsubscribe from this topic, visit
 https://groups.google.com/d/topic/elasticsearch/ILjo-_VRQxA/unsubscribe
 .
 To unsubscribe from this group and all its topics, send an email to
 elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/129A7708-03D8-4F29-87BC-E6AC72369602%40pilato.fr
 https://groups.google.com/d/msgid/elasticsearch/129A7708-03D8-4F29-87BC-E6AC72369602%40pilato.fr?utm_medium=emailutm_source=footer
 .

 For more options, visit https://groups.google.com/d/optout.



 --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/CAJ%3DxLsWgSxiVHmFCGw_jKWAKEpfiGs2ROxYVVhBjSFsqSSKukg%40mail.gmail.com
 https://groups.google.com/d/msgid/elasticsearch/CAJ%3DxLsWgSxiVHmFCGw_jKWAKEpfiGs2ROxYVVhBjSFsqSSKukg%40mail.gmail.com?utm_medium=emailutm_source=footer
 .
 For more options, visit https://groups.google.com/d/optout.



 --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/D207CFFB-2B24-4FBA-89A4-CCCF2A100BEB%40deman.com
 https://groups.google.com/d/msgid/elasticsearch/D207CFFB-2B24-4FBA-89A4-CCCF2A100BEB%40deman.com?utm_medium=emailutm_source=footer
 .
 For more options, visit

Re: High CPU usage of elasticsearch

2015-01-05 Thread Danishka Navin

Hi Mark,

I have removed the indices.memory.index_buffer_size entry and also update
elasticsearch to 1.4.2
Moreover I have deleted lot of indexes.

I can't see any difference of CPU usage by elasticsearch.
PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
27719 elastics 20 0 152g 16g 4.5g S 395.1 55.1 29:06.51 /usr/bin/java
-Xms15g -Xmx15g -Xss256k -Djava.awt.headles
27634 logstash 39 19 3459m 670m 13m S 4.0 2.2 2:56.79 /usr/bin/java
-Djava.io.tmpdir=/var/lib/logstash -Xmx1g -X

here is the jstat information for both elasticsearch and logstash
http://ur1.ca/jbeja

Btw, Do I need to update Kibana to 3.1.2 ?

On Sat, Jan 3, 2015 at 6:34 AM, Mark Walkom markwal...@gmail.com wrote:

You're just hitting limits of your node. Drop some data, add more nodes or
more heap are pretty much the options you have. Upgrade to 1.4.2 while you
are at it.

Setting indices.memory.index_buffer_size so high probably isn't a good
idea unless you know what it does, if you have such a high index rate then
look at adding more nodes to spread the load.

On 3 January 2015 at 08:18, Danishka Navin danis...@gmail.com wrote:

Hi,

Here is stats of my single cluster
http://fpaste.org/165093/20232576/

Elasticsearch configuration
http://fpaste.org/165092/14202325/

I don't use Redis.
There were 20+ million events per 10minutes

(Attached screen-dump of events over time)

On Sat, Jan 3, 2015 at 2:26 AM, Christopher Rimondi
chris.rimo...@gmail.com wrote:

I agree with what Mark said. Nice will be just masking a deeper issue.
Have you tried looking at hot threads?
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/cluster-nodes-hot-threads.html
Also, if you are seeing CPUs sustained at 100% CPU that seems like old GCs
that just are never finishing. So check the GC logs.

Do you have any idea on the number of events per second you are trying
to index and size of the events? If you are using logstash and Redis is the
queue backing up because it can't index?

On Thu, Jan 1, 2015 at 4:49 PM, Mark Walkom markwal...@gmail.com
wrote:

How much data do you have on the node? How many indexes? Have you
checked the logs for GC issues?

You can use nice on the OS level to manage CPU use, but it's not a good
idea, and instead you should figure out why ES is using that CPU.

On 2 January 2015 at 05:09, Danishka Navin danis...@gmail.com wrote:

Hi,

I am new to elasticsearch and logstash.
using elasticsearch-1.1.1 and logstash-1.4.2-1 with Kibana.
Its a single node with 4 vCPU and 30GB of physical memory.

Currently logstash (single node) receive logs from 40 jboss servers.

Most of the time elasticsearch use almost all the CPU resource.
Is there any way I can limit the CPU consumption by tuning?

PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND
12216 elastics 20 0 514g 19g 3.6g S 389.8 63.5 3224:32
/usr/bin/java -Xms15g -Xmx15g -Xss256k -Djava.awt.headles
11722 logstash 39 19 3443m 1.2g 6496 S 8.6 3.9 2037:27
/usr/bin/java -Djava.io.tmpdir=/var/lib/logstash -Xmx1g -X

Any recommendations?

I already followed following article.
http://jablonskis.org/2013/elasticsearch-and-logstash-tuning/index.html

Appreciate your help.

Thanks,
Danishka

--
You received this message because you are subscribed to the Google
Groups elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/e623e1a9-2a59-4b2f-bce0-11c8d2006c54%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_9VizVn4m7C61Xo8GBakRfa00iQV5WiAKhdGHRWAic1g%40mail.gmail.com?utm_medium=emailutm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Chris Rimondi | http://twitter.com/crimondi | securitygrit.com

Re: Elasticsearch logging

2015-01-05 Thread Jesse Redl

For reference here is the configuration we are currently using:

  file:
 type: rollingFile
 file: ${path.logs}/${cluster.name}.log
 maxBackupIndex: 1
 maxFileSize: 100MB
 layout:
   type: pattern
   conversionPattern: [%d{ISO8601}][%-5p][%-25c] %m%n


On Friday, January 2, 2015 12:20:32 PM UTC-6, Jesse Redl wrote:

 Hey All,

 When reviewing the logs generated by elasticsearch (1.4.x), a single log 
 message is being  split across multiple lines? 

 Is this configurable within logging.yml? My undstanding is that 
 elasticsearch is powered by log4j however, I'm not familiar with this 
 product, nor can I find any decent documentation on the logging.yml file.  


 Thanks all.


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/e56ec7a0-ab0f-4152-8a67-52cd8c407bb9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Failed stopping 'elasticsearch-service-x64' service

2015-01-05 Thread Costin Leau

Can you turn on the logging of elasticsearch-service all the way to debug, start and stop the service and see whether
anything special appears in the logs?
You can do this by invoking service manager and in the logging tab, changing the level from Info to Debug (see the
screenshot below):

The service wrapper log is elasticsearch-service-x64, in the same service as Elasticsearch log. Try and do some dry runs
- simply starting and stopping the service (I also recommend removing the service and installing it fresh).
You could also try removing the plugin and simply starting and stopping the fresh install of Elasticsearch to see
whether it's a configuration problem or whether one of the plugins might affect the shutdown...

Cheers,

On 1/5/15 2:25 AM, Garrett Johnson wrote:

Log entries:
[2015-01-04 18:13:56,185][INFO ][node ] [Bucky III]
stopping ...
[2015-01-04 18:13:56,202][INFO ][river.jdbc.JDBCRiver ] river closed
[jdbc/users]
[2015-01-04 18:13:56,203][INFO ][river.jdbc.JDBCRiver ] river closed
[jdbc/product2]
[2015-01-04 18:13:56,342][INFO ][node ] [Bucky III] stopped
[2015-01-04 18:13:56,342][INFO ][node ] [Bucky III] closing
...
[2015-01-04 18:13:56,355][INFO ][node ] [Bucky III] closed

Windows Server 2008R2

ElasticSearch 1.4.2
Plugins ElasticSearch Head, jdbc river 1.4.0.6

Microsoft jdbc driver.

Thanks,

Garrett

On Saturday, January 3, 2015 10:10:42 AM UTC-6, Costin Leau wrote:

Do you see anything in the logs? Can you try removing and reinstalling the
service? What's your OS/configuration?

On 1/2/15 10:32 PM, Garrett Johnson wrote:
By own it's own I mean service stop or using services.msc and clicking
restart on the service. Both attempts
get the
same error.

On Friday, January 2, 2015 2:31:28 PM UTC-6, Garrett Johnson wrote:

I'm getting this error every time I try to start and stop the elastic
search windows service.

Takes a couple of minutes then fails. I can kill the task in task
manager and then restart but cannot get
it to
stop on its own.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to
elasticsearc...@googlegroups.com javascript:
mailto:elasticsearch+unsubscr...@googlegroups.com javascript:.
To view this discussion on the web visit

https://groups.google.com/d/msgid/elasticsearch/f4efa651-9c60-4abb-b04a-47992f1c3e82%40googlegroups.com

https://groups.google.com/d/msgid/elasticsearch/f4efa651-9c60-4abb-b04a-47992f1c3e82%40googlegroups.com?utm_medium=emailutm_source=footer

https://groups.google.com/d/msgid/elasticsearch/f4efa651-9c60-4abb-b04a-47992f1c3e82%40googlegroups.com?utm_medium=emailutm_source=footer.

For more options, visit https://groups.google.com/d/optout
https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscr...@googlegroups.com mailto:elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/fed85958-becc-4269-9300-044e22499624%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/fed85958-becc-4269-9300-044e22499624%40googlegroups.com?utm_medium=emailutm_source=footer.

For more options, visit https://groups.google.com/d/optout.

--
Costin

Re: elastic search upgrade issues

2015-01-05 Thread phani . nadiminti

Hi Mark,

Thank you for the quick reply. i upgraded both nodes in elastic search.

the following are the diskspaces in each node

node 1 : 3Gb available out of 35 GB
node 2 : 4GB available out of 35 GB

is this be a problem?

yes java and elastic search has same versions on each nodes.

java version using : 1.7.0_55

On Monday, January 5, 2015 11:30:06 AM UTC+5:30, Mark Walkom wrote:

Did you upgrade both nodes? Do the nodes have enough disk space?
Is java and Elasticsearch the same version on each node? Have you checked
your logs?

On 5 January 2015 at 16:27, phani.n...@goktree.com javascript: wrote:

Hi All,

Recently i upgraded elastic search version from 1.1.1 to 1.3.7 .
Before upgrade when i create new index it will be replicate to other nodes
automatically but after upgrade elastic search to 1.3.7 version shards are
not allocating to replica node it is residing on primary node only. since
cluster is showing the status is yellow.

please help where is the problem occurring.is it a problem with
elastic search upgrade?

Thanks
phani.

https://groups.google.com/d/msgid/elasticsearch/a30bdc7d-e8eb-4c34-be0c-4627e62944ca%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/0c7a9159-db3c-42da-bcbb-6eca1421cd06%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ES upgrade 0.20.6 to 1.4.2 - CorruptIndexException and FileNotFoundException

2015-01-05 Thread Georgeta

Any ideas?

On Wednesday, December 31, 2014 3:35:39 PM UTC+1, Georgeta wrote:

 Hi All,

 I have a 5 nodes cluster. I updated the cluster from 0.20.6 to 1.4.2.
 When I start the cluster with shard allocation disabled, it starts and 
 goes into a yellow state, all good. When I enable shard allocation WARN 
 messages are generated: 
  
 INFO || elasticsearch[node1][clusterService#updateTask][T#1] 
 org.elasticsearch.cluster.routing.allocation.decider  [node1] updating 
 [cluster.routing.allocation.disable_allocation] from [true] to [false]

 [2014-12-31 13:46:26.310 GMT] WARN || 
 elasticsearch[node1][[transport_server_worker.default]][T#4]{New I/O worker 
 #21} org.elasticsearch.cluster.action.shard  [node1] [index1][2] received 
 shard failed for [index1][2], node[x6PqV8RMS8eA9GmBMZwjNQ], [P], 
 s[STARTED], indexUUID [_na_], reason [engine failure, message [corrupt file 
 detected source: [recovery phase 
 1]][RecoverFilesRecoveryException[[index1][2] Failed to transfer [69] files 
 with total size of [6.5mb]]; nested: CorruptIndexException[checksum failed 
 (hardware problem?) : expected=17tw8li actual=1ig9y12 
 resource=(org.apache.lucene.store.FSDirectory$FSIndexOutput@61297ce5)]; ]]

 [2014-12-31 13:46:35.504 GMT] WARN || 
 elasticsearch[node1][[transport_server_worker.default]][T#14]{New I/O 
 worker #31} org.elasticsearch.cluster.action.shard  [node1] [index2][0] 
 received shard failed for [index2][0], node[GORnFBrmQLOAvK294MUHgA], [P], 
 s[STARTED], indexUUID [_na_], reason [engine failure, message [corrupt file 
 detected source: [recovery phase 
 1]][RecoverFilesRecoveryException[[index2][0] Failed to transfer [163] 
 files with total size of [238.1mb]]; nested: CorruptIndexException[checksum 
 failed (hardware problem?) : expected=ptu7cd actual=1jw7kx9 
 resource=(org.apache.lucene.store.FSDirectory$FSIndexOutput@38c14092)]; ]]

 [2014-12-31 13:46:36.777 GMT] WARN || 
 elasticsearch[node1][[transport_server_worker.default]][T#15]{New I/O 
 worker #32} org.elasticsearch.cluster.action.shard  [node1] [index2][0] 
 received shard failed for [index2][0], node[GORnFBrmQLOAvK294MUHgA], [P], 
 s[STARTED], indexUUID [_na_], reason [master 
 [node1][8zFPkXuvQQWJvErc458tFA][dw1949demum.int.demandware.com][inet[/127.0.0.1:48003]]{local=false,
  
 power_zone=default} marked shard as started, but shard has not been 
 created, mark shard as failed]

 [2014-12-31 13:46:36.792 GMT] WARN || 
 elasticsearch[node1][[transport_server_worker.default]][T#14]{New I/O 
 worker #31} org.elasticsearch.cluster.action.shard  [node1] [index1][2] 
 received shard failed for [index1][2], node[2mIDLcOcQJO4i73QHb7d6Q], [P], 
 s[INITIALIZING], indexUUID [_na_], reason [Failed to start shard, message 
 [IndexShardGatewayRecoveryException[[index1][2] failed recovery]; nested: 
 EngineCreationFailureException[[index1][2] failed to open reader on 
 writer]; nested: FileNotFoundException[No such file [_5aa.tis]]; ]]

 [2014-12-31 13:46:47.261 GMT] WARN || 
 elasticsearch[node1][[transport_server_worker.default]][T#6]{New I/O worker 
 #23} org.elasticsearch.cluster.action.shard  [node1] [index1][2] received 
 shard failed for [index1][2], node[x6PqV8RMS8eA9GmBMZwjNQ], [P], 
 s[INITIALIZING], indexUUID [_na_], reason [Failed to start shard, message 
 [IndexShardGatewayRecoveryException[[index1][2] failed to fetch index 
 version after copying it over]; nested: CorruptIndexException[[index1][2] 
 Preexisting corrupted index [corrupted_gExs5fftSwmCWWgUKN6Wbg] caused by: 
 CorruptIndexException[checksum failed (hardware problem?) : 
 expected=17tw8li actual=1ig9y12 
 resource=(org.apache.lucene.store.FSDirectory$FSIndexOutput@61297ce5)]
 org.apache.lucene.index.CorruptIndexException: checksum failed (hardware 
 problem?) : expected=17tw8li actual=1ig9y12 
 resource=(org.apache.lucene.store.FSDirectory$FSIndexOutput@61297ce5)
 at 
 org.elasticsearch.index.store.LegacyVerification$Adler32VerifyingIndexOutput.verify(LegacyVerification.java:73)
 at org.elasticsearch.index.store.Store.verify(Store.java:365)
 at 
 org.elasticsearch.indices.recovery.RecoveryTarget$FileChunkTransportRequestHandler.messageReceived(RecoveryTarget.java:599)
 at 
 org.elasticsearch.indices.recovery.RecoveryTarget$FileChunkTransportRequestHandler.messageReceived(RecoveryTarget.java:536)
 at 
 org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:745)
 Suppressed: org.elasticsearch.transport.RemoteTransportException: 
 [node5][inet[/127.0.0.1:48043]][internal:index/shard/recovery/file_chunk]
 Caused by: org.apache.lucene.index.CorruptIndexException: checksum failed 
 (hardware problem?) : expected=55hiu actual=16i3yt2

Re: Aggregations without doc-counts

2015-01-05 Thread Elliott Bradshaw

Adrian,

Thanks for that. I had a feeling that that might be the case.

Any tips on improving aggregation performance. I'm working with a 20 shard
index that is loaded on a 20 node cluster. Geohash grid aggregations on
the entire data set (with the size set to unlimited - a requirement) can
take as long as 8 seconds (and return ~ 1 million buckets). I am very
happy with that performance, but if there are any tricks to improve it I
would be glad to do so.

Thanks,

Elliott

On Tuesday, December 30, 2014 11:48:52 AM UTC-5, Adrien Grand wrote:

Hi Eliott,

The overhead of computing the doc counts is actually low, I don't think
you should worry about it.

On Tue, Dec 30, 2014 at 5:12 PM, Elliott Bradshaw ebrad...@gmail.com
javascript: wrote:

Hi,

I'm currently working on a project that visualizes geospatial data in
Elasticsearch. One of the things I am doing is generating heatmaps with
the geohash grid aggregation. I would like to take this to the extreme
case of gridding down to the individual pixel level to display raster
images of a data set, but I am not concerned with the total doc count of
each geohash. Is there a way (or could it be implemented) where an
optimized aggregation could be run that simply lists the existing terms
(geohashes) and does not bother with aggregating their counts? If this
significantly improved performance, such a feature would be very valuable.

Thanks!

- Elliott Bradshaw

https://groups.google.com/d/msgid/elasticsearch/834ebcb1-43b3-486d-bd1a-952005a6a66d%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/53882b08-db93-4116-8c70-b6c1158eb178%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Aggregations without doc-counts

2015-01-05 Thread Elliott Bradshaw

Just as a thought, would setting geohash = true or geohash_prefix = true at
index time improve performance?

On Monday, January 5, 2015 7:20:32 AM UTC-5, Elliott Bradshaw wrote:

Adrian,

Thanks for that. I had a feeling that that might be the case.

Any tips on improving aggregation performance. I'm working with a 20
shard index that is loaded on a 20 node cluster. Geohash grid aggregations
on the entire data set (with the size set to unlimited - a requirement) can
take as long as 8 seconds (and return ~ 1 million buckets). I am very
happy with that performance, but if there are any tricks to improve it I
would be glad to do so.

Thanks,

Elliott

On Tuesday, December 30, 2014 11:48:52 AM UTC-5, Adrien Grand wrote:

Hi Eliott,

The overhead of computing the doc counts is actually low, I don't think
you should worry about it.

On Tue, Dec 30, 2014 at 5:12 PM, Elliott Bradshaw ebrad...@gmail.com
wrote:

Hi,

Thanks!

- Elliott Bradshaw

https://groups.google.com/d/msgid/elasticsearch/834ebcb1-43b3-486d-bd1a-952005a6a66d%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d83c0bc5-bac5-4bae-9984-74ffbf6cd8b3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Geoquery Against Pre-Indexed Shape is Failing [Second Try]

2015-01-05 Thread Peter Johnson

Jörg is correct in saying that the 'geo_shape' filter only supports finding
shapes within shapes and not points within shapes.

It would be great if there was a filter which *did* support finding all
points within a pre-indexed shape!

You may want to open an issue regarding extending the 'geo-polygon' filter
to support 'indexed_shape' instead of just 'points'.
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-geo-polygon-filter.html

As it currently stands I don't think this is a bug, but the above feature
request sounds like it could be useful to others.

I also found a bunch of mistakes in your gist from spelling mistakes
(businesses!=business) to syntax errors and things like not actually
indexing a 'location' field for your business!?
https://gist.github.com/waldoj/004f77755179d862fe3a#file-elasticsearch-sh-L226

Feel free to use my minimal testcase as a template to make your debugging
easier:
https://gist.github.com/missinglink/4bf3e262fbd0a89d3f5e

-P

On Sunday, 4 January 2015 23:28:01 UTC, Jörg Prante wrote:

I think you already found the answer so I hesitate to comment. If you set
location to type geo_point and not to geo_shape, you can not execute a
geo shape filter on it. There are geo bounding box / polygon filters that
can be executed on geo points.

Best,

Jörg

On Sun, Jan 4, 2015 at 8:53 PM, Waldo Jaquith wa...@usodi.org
javascript: wrote:

Folks,

[I sent the following on December 7, and got no reply. I’m sending it
again and, if nobody tells me otherwise, I’ll assume that this is a bug,
and file a report accordingly.]

I’m trying to run a geo_shape filter query using a pre-indexed shape, and
it’s failing for reasons that I cannot identify. This is with Elasticsearch
v1.4.1-1. I’ve documented how to reproduce the problem at
https://gist.github.com/waldoj/004f77755179d862fe3a.

The pre-indexed shapes are GeoJSON converted from Census Bureau municipal
geodata. The indexed documents are state corporate registration records,
each of which have an address that I’ve geocoded into a latitude and
longitude (a field named “location”). The indexed shapes are in a
geometry”:{ “coordinates: { [1, 2] [3, 5] } } format. In the query I
specify path: “geometry”, which generates this error:

Failed to find geo_shape field [location]].”

Based on the error, it looks like Elasticsearch is looking for *shapes*
that fall within the pre-indexed shape, rather than *points*, and so while
the indexed documents have a geo_point field named “location,” there’s no
geo_shape field of that name. FWIW, I’m following the instructions in the
manual at
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-geo-shape-filter.html#_pre_indexed_shape
.

I’d be grateful to anybody who could offer some guidance. Thank you.

Best,
Waldo

---
Waldo Jaquith
Director
U.S. Open Data Institute
http://usodi.org/
202-719-5315

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com javascript:.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/450CBB58-AA14-4106-A6C2-5762DB4845F5%40usodi.org
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/3ea4e571-e192-4136-b860-65c19801193c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Connecting remotely to elastic search cluster

2015-01-05 Thread Vijay Tiwary

I am trying to connect remotely to elastic search cluster having 2 nodes
My question is I am able to query successfully only when we configure both 
the cluster name and host name of the individual nodes? Why it doesn't work 
if I only configure the cluster name in my client code

Settings settings = ImmutableSettings.settingsBuilder()
.put(cluster.name, myClusterName).build();

Client client = new TransportClient(settings ).addTransportAddress(new 
InetSocketTransportAddress(host1, 9300))

.addTransportAddress(new InetSocketTransportAddress(host2, 9300));

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/dd201d66-71b4-4d74-bb99-255144a083a2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Marvel Sense GET/POST

2015-01-05 Thread Paul Scott

Regarding the behaviour of Sense to automatically choose POST regardless of 
the user writing GET:

I was recently rewriting a search query I had written in the Sense UI from 
a search query to a request for a raw document by ID, using the GET API. In 
the process of changing from

GET /index/type/_search
{ ... }


to

GET /index/type/id


I submitted

GET /index/type/id
{... }


which Sense helpfully submitted as a POST request, over-writing the 
document with the search query. In production.

To say that this behaviour was unexpected and unhelpful would be an 
understatement. I would consider it irresponsible and unsafe, and I highly 
recommend the behaviour be disabled.

Please try to read past my obvious irritation and the part of this mistake 
which falls on my head, to the part in this mistake that the Sense UI 
played too.

All the best,

Paul

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/f2588751-a008-4b59-9041-b63c098580dd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Connecting remotely to elastic search cluster

2015-01-05 Thread David Pilato

If you don’t want to define transport addresses, you’d better use a NodeClient.
A NodeClient is actually a node running inside the cluster (with auto
discovery…) but without any data.

You just need to provide the cluster name with a NodeClient. See
http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/client.html#node-client

http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/client.html#node-client

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr
https://twitter.com/elasticsearchfr | @scrutmydocs
https://twitter.com/scrutmydocs

Le 5 janv. 2015 à 13:46, Vijay Tiwary vijaykr.tiw...@gmail.com a écrit :

I am trying to connect remotely to elastic search cluster having 2 nodes
My question is I am able to query successfully only when we configure both
the cluster name and host name of the individual nodes? Why it doesn't work
if I only configure the cluster name in my client code

Settings settings = ImmutableSettings.settingsBuilder()
.put(cluster.name, myClusterName).build();
Client client = new TransportClient(settings ).addTransportAddress(new
InetSocketTransportAddress(host1, 9300))
.addTransportAddress(new InetSocketTransportAddress(host2, 9300));

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com
mailto:elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/dd201d66-71b4-4d74-bb99-255144a083a2%40googlegroups.com

https://groups.google.com/d/msgid/elasticsearch/dd201d66-71b4-4d74-bb99-255144a083a2%40googlegroups.com?utm_medium=emailutm_source=footer.
For more options, visit https://groups.google.com/d/optout
https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/36D10C69-1F87-498C-A934-094D3C3B6516%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

Re: Aggregations without doc-counts

2015-01-05 Thread Adrien Grand

No it wouldn't. I don't have ideas about how to improve performance, are
you running only a geohash grid aggregation or do you also have sub
aggregations? Also 1 million buckets is a lot, if it would work for you to
decrease the value of the precision parameter, this could help with
performance.

On Mon, Jan 5, 2015 at 1:22 PM, Elliott Bradshaw ebradsh...@gmail.com
wrote:

Just as a thought, would setting geohash = true or geohash_prefix = true
at index time improve performance?

On Monday, January 5, 2015 7:20:32 AM UTC-5, Elliott Bradshaw wrote:

Adrian,

Thanks for that. I had a feeling that that might be the case.

Any tips on improving aggregation performance. I'm working with a 20
shard index that is loaded on a 20 node cluster. Geohash grid aggregations
on the entire data set (with the size set to unlimited - a requirement) can
take as long as 8 seconds (and return ~ 1 million buckets). I am very
happy with that performance, but if there are any tricks to improve it I
would be glad to do so.

Thanks,

Elliott

On Tuesday, December 30, 2014 11:48:52 AM UTC-5, Adrien Grand wrote:

Hi Eliott,

The overhead of computing the doc counts is actually low, I don't think
you should worry about it.

On Tue, Dec 30, 2014 at 5:12 PM, Elliott Bradshaw ebrad...@gmail.com
wrote:

Hi,

Thanks!

- Elliott Bradshaw

--
You received this message because you are subscribed to the Google
Groups elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/834ebcb1-43b3-486d-bd1a-952005a6a66d%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/834ebcb1-43b3-486d-bd1a-952005a6a66d%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/d83c0bc5-bac5-4bae-9984-74ffbf6cd8b3%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/d83c0bc5-bac5-4bae-9984-74ffbf6cd8b3%40googlegroups.com?utm_medium=emailutm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAL6Z4j7EHNTtNWqnbK-t1tECku-WDtxq2omRvOhQsw4ZLh_jsQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

ES Nodes storage capacity

2015-01-05 Thread Simon Thorley

Hi all,

I have designed and setup an ELK stack that can be expanded horizontally (i 
hope) using the following technologies:

haproxy x2 (failover) = Logstash x2 = nginx x2 (failover) = 
elasticsearch x2

The capacity of the elasticsearch nodes have come into question and this 
has raised a few questions regards spec of a new node.

Here is the current specs of the elasticsearch nodes ( i have 2 of these):

· HP DL360p Gen8 10-SFF CTO Server

· 64G PC3L-12800R-11

· 8x 900G 10K SAS

· 2x 300G 10K SAS

· 2G FBWC

· Dual 750W PSU

· 4P 1GBE 331FLR


So i am currently running the OS from the 2x 300Gb in a mirror raid, and 2 
data logical data drives using 4 of the 8 900GB drives (so 2 striped raids 
containing 4x900GB drives).


This is all working fine but the data capacity has become an issue (14TB 
total available) .  I think for the moment i have enough compute power but 
what would happen if i added a lower spec node (or multiple), marked them 
as a data node (non-master) but with different storage capacity available. 
 Say for example 20TB in each.


If the original 2 nodes filed there data stores and the only store 
available was the new node(s) then they would be processing the shards 
alone and there would be no protection from replica's if this node went 
down, no?


As an additional but not as critical at the moment:

If i eventually have the same issue with compute power, if these 2 someday 
become saturated and i make another node a master node but it was half the 
spec, would elasticsearch realize this and distribute the load or is this 
purely down to nginx load distribution?


Thanks for any help\advice in advance.

Simon

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/8450d29f-9a1b-40c0-87e5-9fa2b18f364d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Sorting with from/size parameter

2015-01-05 Thread Vipin Sudhakar



I have to use sorting along with from and size parameters in the elastic 
search query.

I am querying elastic search with 0 to 100 records and then 101 to 200 
records and then 201 to 300 records. I have to sort the entire set with 
salary field.

Will it sort the whole set and takes 100 records at a time or will it take 
100 records and then apply sorting and then proceed similarly.

Please confirm

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/30c31941-66ac-4183-b04c-61b179ae50ef%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Unicast Cluster / Host can't see its own publish_host

2015-01-05 Thread Matt Hughes

In my VM environment, a VM can't actually see its public IP address.  I 
have the following setup:

network.publish_host: 10.255.207.123
discovery.zen.ping.unicast.hosts: 10.255.207.123,10.255.207.124,10.255.
207.125



My VM can see 124 and 125 just fine, but due to issues completely unrelated 
to ES, it cannot see its own public IP.  As a result, the logs on each 
machine fill up with these exceptions:

org.elasticsearch.transport.ConnectTransportException: 
[elk2][inet[/10.255.207.123:9300]] 
connect_timeout[30s]
  at 
org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:807)
  at 
org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:741)
  at 
org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:714)
  at 
org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:150)
  at 
org.elasticsearch.cluster.service.InternalClusterService$ReconnectToNodes.run(InternalClusterService.java:521)
  at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.common.netty.channel.ConnectTimeoutException: 
connection timed out: /10.255.207.123:9300
  at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.
processConnectTimeout(NioClientBoss.java:139)
  at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process
(NioClientBoss.java:83)
  at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.
run(AbstractNioSelector.java:318)
  at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(
NioClientBoss.java:42)
  at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(
ThreadRenamingRunnable.java:108)
  at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(
DeadLockProofWorker.java:42)



I believe this is pretty innocuous as I've been running this way for 6 
months, but I've always been curious as to way my node is even *trying* to 
connect to itself like this?  As it goes through the list of 
'discovery.zen.ping.unicast.hosts', shouldn't it ignore itself?  Maybe ES 
doesn't use 'network.publish_host' to determine if the IP belongs to the 
current machine?

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/fa2e66cf-1f96-4c9c-8b69-bd146f868ec7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: High CPU and load since 1.4.1 - 1.4.2 update

2015-01-05 Thread Arie

Hi,,

The only things I see is ES_HEAP_SIZE with 50% this can bet set to 8gb.
I'm seeing some swap usage, you could disable swap completely.


hth,,
A.


On Monday, January 5, 2015 12:48:53 PM UTC+1, Darren Mansell wrote:

 Hi all,

 We have a 4 node VM dev cluster with 3 data nodes and 1 master node. The 3 
 data nodes are taking lots of CPU and the load average is high when the 
 servers are sitting idle with no accesses or indexing.

 This seemed to be fine before we updated to 1.4.2 before the new year, but 
 I can't confirm this is definitely the reason.

 The hot threads from each node seem to indicate something to do with the 
 filesystem most of the time e.g.

 87.7% (438.6ms out of 500ms) cpu usage by thread 
 'elasticsearch[potassium][management][T#4]'
  2/10 snapshots sharing following 21 elements
org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:223)
org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:242)

 org.apache.lucene.store.FileSwitchDirectory.listAll(FileSwitchDirectory.java:87)

 org.apache.lucene.store.FilterDirectory.listAll(FilterDirectory.java:48)

 org.elasticsearch.index.store.DistributorDirectory.listAll(DistributorDirectory.java:88)

 org.apache.lucene.store.FilterDirectory.listAll(FilterDirectory.java:48)

 org.elasticsearch.common.lucene.Directories.estimateSize(Directories.java:40)
org.elasticsearch.index.store.Store.stats(Store.java:216)

 org.elasticsearch.index.shard.service.InternalIndexShard.storeStats(InternalIndexShard.java:540)

 org.elasticsearch.action.admin.indices.stats.CommonStats.init(CommonStats.java:134)

 org.elasticsearch.action.admin.indices.stats.ShardStats.init(ShardStats.java:49)

 org.elasticsearch.indices.InternalIndicesService.stats(InternalIndicesService.java:212)
org.elasticsearch.node.service.NodeService.stats(NodeService.java:156)

 org.elasticsearch.action.admin.cluster.node.stats.TransportNodesStatsAction.nodeOperation(TransportNodesStatsAction.java:96)

 org.elasticsearch.action.admin.cluster.node.stats.TransportNodesStatsAction.nodeOperation(TransportNodesStatsAction.java:44)

 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:278)

 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:269)

 org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)

 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)


 Please also see the image attachment for the high CPU and load.


 The VMs are on ESXi 5.5 on 2 * 4 core HT Xeon E5530 so server hardware is 
 pretty heavy. The nodes are set up with 16GB RAM, 2 vCPUs, all ES defaults 
 apart from:


 [root@potassium ~]# grep -v ^# /etc/elasticsearch/elasticsearch.yml | grep -v 
 ^$cluster.name: es-devnode.name: potassium
 node.master: false
 node.data: true
 path.data: /data/elasticsearch
 marvel.agent.exporter.es.hosts: [hydrogen:9200]


 and


 [root@potassium ~]# grep -v ^# /etc/sysconfig/elasticsearch | grep -v ^$
 ES_HOME=/usr/share/elasticsearch
 ES_HEAP_SIZE=6g
 MAX_OPEN_FILES=65535
 MAX_MAP_COUNT=262144
 LOG_DIR=/var/log/elasticsearch
 DATA_DIR=/var/lib/elasticsearch
 WORK_DIR=/tmp/elasticsearch
 CONF_DIR=/etc/elasticsearch
 CONF_FILE=/etc/elasticsearch/elasticsearch.yml
 ES_USER=elasticsearch


 So far I've tried:



- Dropping all data and loading again using logstash
- Deleting XFS filesystem and changing to ext4
- Removing all plugins
- Leaving for about 2 weeks in case it was doing background optimisation
- and various other things

 Does anyone have any suggestions about where I should look next, or any 
 thoughts about what could be happening? Please let me know if I can pull any 
 other info off the nodes or cluster to help diagnose.

 Many thanks,
 Darren



-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/9447a9cc-f65b-4b46-8a0c-35002368c889%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: High CPU and load since 1.4.1 - 1.4.2 update

2015-01-05 Thread Darren Mansell

To update this, I've just found I can resolve the problem by stopping just 
the master node. Instantly the other data nodes settle down into single 
figure CPU usage and 1 load.

The layout is:

caesium: 
node.master: true
node.data: false

sodium, potassium, francium:
node.master: false
node.data: true

If I allow any of the data nodes to become master and then stop caesium, 
it's fine. Is there a problem with ES when operating in this way? This was 
the recommended way of working from the ES training course I attended, so I 
implemented it as soon as I got back.

Regards,
Darren


On Monday, 5 January 2015 11:48:53 UTC, Darren Mansell wrote:

 Hi all,

 We have a 4 node VM dev cluster with 3 data nodes and 1 master node. The 3 
 data nodes are taking lots of CPU and the load average is high when the 
 servers are sitting idle with no accesses or indexing.

 This seemed to be fine before we updated to 1.4.2 before the new year, but 
 I can't confirm this is definitely the reason.

 The hot threads from each node seem to indicate something to do with the 
 filesystem most of the time e.g.

 87.7% (438.6ms out of 500ms) cpu usage by thread 
 'elasticsearch[potassium][management][T#4]'
  2/10 snapshots sharing following 21 elements
org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:223)
org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:242)

 org.apache.lucene.store.FileSwitchDirectory.listAll(FileSwitchDirectory.java:87)

 org.apache.lucene.store.FilterDirectory.listAll(FilterDirectory.java:48)

 org.elasticsearch.index.store.DistributorDirectory.listAll(DistributorDirectory.java:88)

 org.apache.lucene.store.FilterDirectory.listAll(FilterDirectory.java:48)

 org.elasticsearch.common.lucene.Directories.estimateSize(Directories.java:40)
org.elasticsearch.index.store.Store.stats(Store.java:216)

 org.elasticsearch.index.shard.service.InternalIndexShard.storeStats(InternalIndexShard.java:540)

 org.elasticsearch.action.admin.indices.stats.CommonStats.init(CommonStats.java:134)

 org.elasticsearch.action.admin.indices.stats.ShardStats.init(ShardStats.java:49)

 org.elasticsearch.indices.InternalIndicesService.stats(InternalIndicesService.java:212)
org.elasticsearch.node.service.NodeService.stats(NodeService.java:156)

 org.elasticsearch.action.admin.cluster.node.stats.TransportNodesStatsAction.nodeOperation(TransportNodesStatsAction.java:96)

 org.elasticsearch.action.admin.cluster.node.stats.TransportNodesStatsAction.nodeOperation(TransportNodesStatsAction.java:44)

 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:278)

 org.elasticsearch.action.support.nodes.TransportNodesOperationAction$NodeTransportHandler.messageReceived(TransportNodesOperationAction.java:269)

 org.elasticsearch.transport.netty.MessageChannelHandler$RequestHandler.run(MessageChannelHandler.java:275)

 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)


 Please also see the image attachment for the high CPU and load.


 The VMs are on ESXi 5.5 on 2 * 4 core HT Xeon E5530 so server hardware is 
 pretty heavy. The nodes are set up with 16GB RAM, 2 vCPUs, all ES defaults 
 apart from:


 [root@potassium ~]# grep -v ^# /etc/elasticsearch/elasticsearch.yml | grep -v 
 ^$cluster.name: es-devnode.name: potassium
 node.master: false
 node.data: true
 path.data: /data/elasticsearch
 marvel.agent.exporter.es.hosts: [hydrogen:9200]


 and


 [root@potassium ~]# grep -v ^# /etc/sysconfig/elasticsearch | grep -v ^$
 ES_HOME=/usr/share/elasticsearch
 ES_HEAP_SIZE=6g
 MAX_OPEN_FILES=65535
 MAX_MAP_COUNT=262144
 LOG_DIR=/var/log/elasticsearch
 DATA_DIR=/var/lib/elasticsearch
 WORK_DIR=/tmp/elasticsearch
 CONF_DIR=/etc/elasticsearch
 CONF_FILE=/etc/elasticsearch/elasticsearch.yml
 ES_USER=elasticsearch


 So far I've tried:



- Dropping all data and loading again using logstash
- Deleting XFS filesystem and changing to ext4
- Removing all plugins
- Leaving for about 2 weeks in case it was doing background optimisation
- and various other things

 Does anyone have any suggestions about where I should look next, or any 
 thoughts about what could be happening? Please let me know if I can pull any 
 other info off the nodes or cluster to help diagnose.

 Many thanks,
 Darren



-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit

Which versions of elasticsearch will shield be compatible with?

2015-01-05 Thread Tim S

Hi,

Does anyone know which versions of elasticsearch shield can be used with? 
Will it just be the latest version at the time of release (1.4.2 currently) 
or will it work with older versions?

Thanks,

Tim.

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/4e9e2cda-fed0-435a-8765-a8288b6b6335%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Source field exclusion seems to be storing data anyway.

2015-01-05 Thread Randy McCluer

I have some sensitive data that I want excluded from source, but indexed. I 
am using _source: { excludes: [field1] }, and everything seems to be 
working just as expected with the source docs coming back without field1. 
If I update the mapping to not exclude field1, the docs still return 
without field1 as well. However, if I restart the service, they start 
coming back with field1 in the doc, indicating that the data was being 
stored all along. 

All of the documentation I've found indicates that the excluded fields are 
removed at write-time. My situation leads me to believe that this isn't the 
case. Can someone tell me if this is the expected behavior or a bug to be 
filed? I'd really hate to have to go down the Solr-style route of declaring 
all of my fields individually.

TIA

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/dd420599-3a17-43a5-b5f1-ab552e910a94%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Sorting on a boolean field

2015-01-05 Thread mbaker

I would like to sort the results of a query such that all of the records 
with a value of true in the expires field will be at the top. I am not 
using relevance for sorting. I tried adding this as a sort, but it doesn't 
seem to work with either sorting order: { expires: { order: desc } }

All of the records are in the result but they aren't sorted. Is there a 
different way I should be going about this?

For reference, here is my entire query:

{
  query: {
match_all: {
}
  },
  size: 50,
  from: 0,
  fields: [
id
  ],
  sort: [
{
  expires: {
order: desc
  }
}
  ]
}

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/0bfc5579-8855-44a8-97e3-58ede3bb10d4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Is there a preferred config for Index / Shard configuration? Lots of indexes with lots of shards or fewer indexes and bigger shards?

2015-01-05 Thread mike . giardinelli

Hi All,

We have started noticing in our environment that our query performance is 
starting to suffer for some of our datasets that span the roughly 1 year of 
data we keep online.  We are looking into optimizations we can make to our 
Index / Shard configuration and I was wondering if there is a preferable 
way to configure our indexes / shards? Right now we create a new index for 
each week and have 22 shards per index (We have 22 data nodes).  Would it 
be more optimal to reduce the number of indexes (index by month) and have 
larger shards? Our documents are kb in size so they are not all that big, 
we just have a lot of them. 

The feedback we typically get back from support is just test and see.  That 
is something we can do, but there is a fair amount of effort / time that we 
would need to put in to only find that it doesn't give us any benefit.  I 
was just hoping some of the more experienced folks could provide some input 
on possible solutions.  If all else fails, we can always try to test 
different configs. 

Thanks!


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/12e73093-236b-4656-b6d5-960b31df7747%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Unicast Cluster / Host can't see its own publish_host

2015-01-05 Thread Mark Walkom

It sounds like because that isn't a local interface that ES is bound to it
tries to access it. Are you using NAT on a higher layer?

On 6 January 2015 at 01:59, Matt Hughes hughes.m...@gmail.com wrote:

 In my VM environment, a VM can't actually see its public IP address.  I
 have the following setup:

 network.publish_host: 10.255.207.123
 discovery.zen.ping.unicast.hosts: 10.255.207.123,10.255.207.124,10.255.
 207.125



 My VM can see 124 and 125 just fine, but due to issues completely
 unrelated to ES, it cannot see its own public IP.  As a result, the logs on
 each machine fill up with these exceptions:

 org.elasticsearch.transport.ConnectTransportException: [elk2][inet[/
 10.255.207.123:9300]] connect_timeout[30s]
   at
 org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:807)
   at
 org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:741)
   at
 org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:714)
   at
 org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:150)
   at
 org.elasticsearch.cluster.service.InternalClusterService$ReconnectToNodes.run(InternalClusterService.java:521)
   at
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   at
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   at java.lang.Thread.run(Thread.java:744)
 Caused by: org.elasticsearch.common.netty.channel.ConnectTimeoutException:
 connection timed out: /10.255.207.123:9300
   at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.
 processConnectTimeout(NioClientBoss.java:139)
   at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.
 process(NioClientBoss.java:83)
   at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector
 .run(AbstractNioSelector.java:318)
   at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(
 NioClientBoss.java:42)
   at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(
 ThreadRenamingRunnable.java:108)
   at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.
 run(DeadLockProofWorker.java:42)



 I believe this is pretty innocuous as I've been running this way for 6
 months, but I've always been curious as to way my node is even *trying* to
 connect to itself like this?  As it goes through the list of
 'discovery.zen.ping.unicast.hosts', shouldn't it ignore itself?  Maybe ES
 doesn't use 'network.publish_host' to determine if the IP belongs to the
 current machine?

 --
 You received this message because you are subscribed to the Google Groups
 elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send an
 email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/elasticsearch/fa2e66cf-1f96-4c9c-8b69-bd146f868ec7%40googlegroups.com
 https://groups.google.com/d/msgid/elasticsearch/fa2e66cf-1f96-4c9c-8b69-bd146f868ec7%40googlegroups.com?utm_medium=emailutm_source=footer
 .
 For more options, visit https://groups.google.com/d/optout.


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-4Zk90wSrM8Z6oqHR1LGNdLLSB4vcYVqdPp%2BvLWoRMgA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Is there a preferred config for Index / Shard configuration? Lots of indexes with lots of shards or fewer indexes and bigger shards?

2015-01-05 Thread Mark Walkom

One shard per node is ideal as you spread the load.
Reducing the shard count can help but it depends on a few things.

How much data do you have in your cluster, how many indexes?

On 6 January 2015 at 08:51, mike.giardine...@gmail.com wrote:

Hi All,

We have started noticing in our environment that our query performance is
starting to suffer for some of our datasets that span the roughly 1 year of
data we keep online. We are looking into optimizations we can make to our
Index / Shard configuration and I was wondering if there is a preferable
way to configure our indexes / shards? Right now we create a new index for
each week and have 22 shards per index (We have 22 data nodes). Would it
be more optimal to reduce the number of indexes (index by month) and have
larger shards? Our documents are kb in size so they are not all that big,
we just have a lot of them.

The feedback we typically get back from support is just test and see.
That is something we can do, but there is a fair amount of effort / time
that we would need to put in to only find that it doesn't give us any
benefit. I was just hoping some of the more experienced folks could
provide some input on possible solutions. If all else fails, we can always
try to test different configs.

Thanks!

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/12e73093-236b-4656-b6d5-960b31df7747%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/12e73093-236b-4656-b6d5-960b31df7747%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-yUUxqLWaPtAtdPmzMNROXSzH%3D0qoVKK3XtJJDWSP0rg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Document not being found in alias

2015-01-05 Thread Stefanie

I have a document that is not being found when I run an alias but is being 
found if I run the filters on the index.  Is there anything that could be 
causing this?  

I have two mappings in this index, the records from one of the mappings has 
no problem being found but records from the other mapping are not being 
included.

My alias looks something like this:


   - index_name: 
   {
  - aliases: 
  {
 - alias_name: 
 {
- filter: 
{
   - terms: 
   {
  - term_id: 
  [
 - 127,
 - 102,
 - 128
 ]
  }
   }
}
 }
  }
   

Thank you,
Stefanie

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/d2ab941f-6fa9-41f3-a1df-e8f26e4e48d0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Marvel Sense GET/POST

2015-01-05 Thread Mark Walkom

I've since been informed this is a known issue and a bug has been raised
for it, so a fix is on the way.

On 6 January 2015 at 07:57, Mark Walkom markwal...@gmail.com wrote:

It'd be great if you could raise this as an issue on github for this
behaviour to be checked - https://github.com/elasticsearch/elasticsearch

On 6 January 2015 at 00:06, Paul Scott p...@duedil.com wrote:

Regarding the behaviour of Sense to automatically choose POST regardless
of the user writing GET:

I was recently rewriting a search query I had written in the Sense UI
from a search query to a request for a raw document by ID, using the GET
API. In the process of changing from

GET /index/type/_search
{ ... }

GET /index/type/id

I submitted

GET /index/type/id
{... }

which Sense helpfully submitted as a POST request, over-writing the
document with the search query. In production.

To say that this behaviour was unexpected and unhelpful would be an
understatement. I would consider it irresponsible and unsafe, and I highly
recommend the behaviour be disabled.

Please try to read past my obvious irritation and the part of this
mistake which falls on my head, to the part in this mistake that the Sense
UI played too.

All the best,

Paul

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f2588751-a008-4b59-9041-b63c098580dd%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/f2588751-a008-4b59-9041-b63c098580dd%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-Fxju8SodKDp_LuaK8WXfQuLjtgK2RGD%3DbOkQ2wTi9iQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

best approach to pull back only relevant fields

2015-01-05 Thread phil swenson

If I have a document I’m storing that looks like this:

doc = {


  userId: *‘*yoyomamma',
  physicians: [{
name: sanjay gupta,
id: 403494
  },
{
  name: daniel litvak,
  id: 3382
}],
  procedures: [{
name: appendectomy,
id: 3939,
date: 2/3/2012
  }

… etc a bunch of other stuff

}

When I search on “appendectomy” I want to only return the following:
“userId” and the matching  “procedures” fields…..


Any thoughts on the best approach to do this?


thanks

phil

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAGenvhVDwZdS8519af5RWisrEVVUbeOi55%2BUdGZNK55mwsLzNw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Max request size?

2015-01-05 Thread rogthefrog

Can this setting be updated dynamically? It doesn't look that way:

$ curl -XPUT localhost:9200/_cluster/settings -d '{
 persistent : {
 http.max_content_length: 1200mb
 }
 }'

$ curl -XGET localhost:9200/_cluster/settings?pretty=1
{
  persistent : { },
  transient : { }

  }
}
  }
}




--
View this message in context: 
http://elasticsearch-users.115913.n3.nabble.com/Max-request-size-tp3687861p4068519.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1420496175553-4068519.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.

Re: Max request size?

2015-01-05 Thread Mark Walkom

Try setting it in transient instead of persistent.

Persistent settings are usually read from the config file only.

On 6 January 2015 at 09:16, rogthefrog roger...@amino.com wrote:

Can this setting be updated dynamically? It doesn't look that way:

$ curl -XPUT localhost:9200/_cluster/settings -d '{
persistent : {
http.max_content_length: 1200mb
}
}'

$ curl -XGET localhost:9200/_cluster/settings?pretty=1
{
persistent : { },
transient : { }

}
}
}
}

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Max-request-size-tp3687861p4068519.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1420496175553-4068519.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-uvGDr2uHFGDw_zk9%2BopPn%2BDkDcwPvXkFfgWkGosRoHQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Max request size?

2015-01-05 Thread rogthefrog

@mark thanks for the suggestion, I hadn't thought of that. Same results, no
dice.



--
View this message in context: 
http://elasticsearch-users.115913.n3.nabble.com/Max-request-size-tp3687861p4068526.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1420501357336-4068526.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.

In the search when set the type slower than the speed of not set type very much

2015-01-05 Thread haochun91



hello all,
Recently,i encounter a very strange problem.like the title,In the 
search,when i set the index and type,it is very slow,but if i does not set 
the index and type is very fast.example:

[image: mk87p 4_h2ywse4_g_ e 9u] 
https://cloud.githubusercontent.com/assets/9320176/5613798/c3768e42-9524-11e4-987d-3560e1dcfd18.jpg

[image: r vup130 zzugiorgv9_fc0] 
https://cloud.githubusercontent.com/assets/9320176/5613808/d58450ce-9524-11e4-8772-c759df848687.jpg

This strange problem occurred in a few days ago my computer suddenly loses 
power, the cluster after the restart.but my cluster is health.i do not konw 
how to solve this strange problem.i do not know it was a configuration 
problem,or is a bug of elasticsearch.

i hope some one can help me!

mark:

 before my computer suddenly loses power,search speed is very fast.



-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/8f0d10d3-0ea1-45bf-85bb-f876ab79fcef%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: elasticsearch startup issue with mavel

2015-01-05 Thread Subhadip Bagui

Hi Jesse,

My marvel was installed in April. I didn't upgrade es or marvel after that.

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/22da0db0-3b34-4844-addd-2e2acb4ec44e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Geoquery Against Pre-Indexed Shape is Failing [Second Try]

2015-01-05 Thread Waldo Jaquith

This is fascinating—it hadn't crossed my mind that I might be doing
something that simply wasn't supported by Elasticsearch. As many times as I
read the documentation, I think I was reading for what I *wanted* it to do,
rather than as a blank slate. Perhaps I'm trying to do something that is
supported, but I'm doing it wrong?

This is for all businesses registered in a U.S. state (Virginia), to
identify which ones are registered in a given county or city (a sub-unit of
a state). The UK equivalent of this would be a list of all businesses
registered in the East Midlands, and wanting to identify all of the
businesses in Nottingham. I want somebody running a search for a business
name to be able to limit that search to a given county or city. Perhaps I'm
doing that all wrong, in terms of how Elasticsearch works. (At present, I'm
doing this by including the shapes in my HTML, rather than pre-indexing
them.) Is there a more Elasticsearch-y way to accomplish this, or is it
really best for me to open an issue proposing such a new feature?

My apologies for the errors in my gist! Thank you for forking it and fixing
them. This is my first time putting together a complete test case of a
problem for a mailing list, so when the test case failed, it only served to
(wrongly) reinforce that I'd correctly reproduced the problem!

Best,
Waldo

---
Waldo Jaquith
Director
U.S. Open Data
http://usopendata.org/
202-719-5315

On Monday, January 5, 2015 9:15:51 AM UTC-5, Peter Johnson wrote:

Jörg is correct in saying that the 'geo_shape' filter only supports
finding shapes within shapes and not points within shapes.

It would be great if there was a filter which *did* support finding all
points within a pre-indexed shape!

You may want to open an issue regarding extending the 'geo-polygon' filter
to support 'indexed_shape' instead of just 'points'.

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-geo-polygon-filter.html

As it currently stands I don't think this is a bug, but the above feature
request sounds like it could be useful to others.

I also found a bunch of mistakes in your gist from spelling mistakes
(businesses!=business) to syntax errors and things like not actually
indexing a 'location' field for your business!?

https://gist.github.com/waldoj/004f77755179d862fe3a#file-elasticsearch-sh-L226

Feel free to use my minimal testcase as a template to make your debugging
easier:
https://gist.github.com/missinglink/4bf3e262fbd0a89d3f5e

-P

On Sunday, 4 January 2015 23:28:01 UTC, Jörg Prante wrote:

Best,

Jörg

On Sun, Jan 4, 2015 at 8:53 PM, Waldo Jaquith wa...@usodi.org wrote:

Folks,

[I sent the following on December 7, and got no reply. I’m sending it
again and, if nobody tells me otherwise, I’ll assume that this is a bug,
and file a report accordingly.]

I’m trying to run a geo_shape filter query using a pre-indexed shape,
and it’s failing for reasons that I cannot identify. This is with
Elasticsearch v1.4.1-1. I’ve documented how to reproduce the problem at
https://gist.github.com/waldoj/004f77755179d862fe3a.

The pre-indexed shapes are GeoJSON converted from Census Bureau
municipal geodata. The indexed documents are state corporate registration
records, each of which have an address that I’ve geocoded into a latitude
and longitude (a field named “location”). The indexed shapes are in a
geometry”:{ “coordinates: { [1, 2] [3, 5] } } format. In the query I
specify path: “geometry”, which generates this error:

Failed to find geo_shape field [location]].”

I’d be grateful to anybody who could offer some guidance. Thank you.

Best,
Waldo

---
Waldo Jaquith
Director
U.S. Open Data Institute
http://usodi.org/
202-719-5315

--
You received this message because you are subscribed to the Google Groups

Re: shards allocation and limitations

2015-01-05 Thread phani . nadiminti

Thank you mark got your point.

On Monday, January 5, 2015 10:51:59 AM UTC+5:30, phani.n...@goktree.com 
wrote:

 Hi All,

I have an index with 51 millions records i have 2 nodes in my 
 cluster.

no of shards for the above index is  : 5
replicas are :1

 my question is how many records we can index to get better 
 search capabilities? Is there any limitation for the index that will hold 
 these many records. or please suggest me what are the settings in 
 elasticsearch.yml file to optimize the index which is hoding 51 million 
 records.


  can we increase number of shards other than five ? if we 
 increase that one is there any problems with searching records?

   please help me.

 Thanks

 phani


-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/83630a34-fcb1-4235-89f8-be72e3bbc8bd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: best approach to pull back only relevant fields

2015-01-05 Thread David Pilato

It somehow means to me that you are searching for procedures so you should
index procedures not doc.

If you need both, you could may be use Parent/child feature so you will have
different docs indexed.

My 2 cents

--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 5 janv. 2015 à 22:50, phil swenson phil.swen...@gmail.com a écrit :

If I have a document I’m storing that looks like this:

doc = {

userId: ‘yoyomamma',
physicians: [{
name: sanjay gupta,
id: 403494
},
{
name: daniel litvak,
id: 3382
}],
procedures: [{
name: appendectomy,
id: 3939,
date: 2/3/2012
}
… etc a bunch of other stuff
}
When I search on “appendectomy” I want to only return the following:
“userId” and the matching “procedures” fields…..

Any thoughts on the best approach to do this?

thanks
phil

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAGenvhVDwZdS8519af5RWisrEVVUbeOi55%2BUdGZNK55mwsLzNw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/00357F51-A710-4973-B428-FAAF6B17E5CB%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

Re: Ignore a field in the scoring

2015-01-05 Thread Roger de Cordova Farias

Now I ran the query with explain = true. The results are the following:


*Explain before the update:*


   details: [
 {
   value: 5.752348,
   description: fieldWeight in 424, product of:,
   details: [
 {
   value: 1,
   description: tf(freq=1.0), with freq of:,
   details: [
 {
   value: 1,
   description: termFreq=1.0
 }
   ]
 },
 {
   value: 9.203756,
   description: idf(docFreq=201, maxDocs=738240)
 },
 {
   value: 0.625,
   description: fieldNorm(doc=424)
 }
   ]
 }
   ]



*Update script (scriptLang = groovy, profileId = 1):*

if (ctx._source.bookmarked_by == null) {
 ctx._source.bookmarked_by = [profileId]
 } else if (ctx._source.bookmarked_by.contains(profileId)) {
 ctx.op = none
 } else {
 ctx._source.bookmarked_by += profileId
 }



*Explain after the update:*

  details: [
 {
   value: 5.749262,
   description: fieldWeight in 0, product of:,
   details: [
 {
   value: 1,
   description: tf(freq=1.0), with freq of:,
   details: [
 {
   value: 1,
   description: termFreq=1.0
 }
   ]
 },
 {
   value: 9.198819,
   description: idf(docFreq=202, maxDocs=738241)
 },
 {
   value: 0.625,
   description: fieldNorm(doc=0)
 }
   ]
 }
   ]



* Query used with the explain:*

{
   query: {
 query_string: {
   fields: [
 name
   ],
   query: roger
 }
   }
 }





The inverse document frequency (idf) is changed after adding a new field
that is not used in the query. Also, it changed the fieldWeight in 424
and fieldNorm(doc=424) to  fieldWeight in 0 and fieldNorm(doc=0) (idk
if it changes something)

Can someone help me on how to not change the score of the document after
running the update? Note that the update creates a new field if it was not
found (== null), but this field is not used in the query

2015-01-05 13:35 GMT-02:00 Roger de Cordova Farias 
roger.far...@fontec.inf.br:

 The added field is an array of Integers, but we are not using it in the
 query at all

 We are not querying the _all field, it is disabled in our type mapping

 Our query is something like this:

 {
   query: {
 query_string: {
   fields: [
 name
   ],
   query: roger
 }
   }
 }


 I ran this query. In the first result, I added a new field called
 bookmarked_by with a numeric value. Then I ran the same query again. The
 document in which I added the new field is no longer the first result

 2014-12-26 17:34 GMT-02:00 Doug Turnbull 
 dturnb...@opensourceconnections.com:

 Are you querying the _all field? How are you doing your searches?

 http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-all-field.html

 The _all field receives a copy of every  field you index, so adding data
 here could impact scores regardless of the source field.

 Otherwise, fields are scored independently before being put together by
 other queries like boolean queries or dismax. Are you using
 boolean/dismax/etc over multiple fields?

 -Doug

 On Fri, Dec 26, 2014 at 11:59 AM, Ivan Brusic i...@brusic.com wrote:

 Use the field in a filter and not part of the query. Is this field free
 text?

 Ivan
 On Dec 23, 2014 9:12 PM, Roger de Cordova Farias 
 roger.far...@fontec.inf.br wrote:

 Hello

 Our documents have metadata indexed with them, but we don't want the
 metadata to interfere in the scoring

 After a user searches for documents, they can bookmark them (what means
 we add more metadata to the document), then in the next search with the
 same query the bookmarked document  appears in a lower (worse) position

 Is there a way to completely ignore one or more specific fields in the
 scoring of every query? as in indexing time or something?

 Note that we are not using the metadata field in the query, but yet it
 lowers the score of every query

 We cannot set the index attribute of this field to no because we
 are gonna use it in other queries

 --
 You received this message because you are subscribed to the Google
 Groups elasticsearch group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to elasticsearch+unsubscr...@googlegroups.com.
 To view this discussion on the web visit

Re: Connecting remotely to elastic search cluster

2015-01-05 Thread Vijay Tiwary

Thanks David once again. Can you please help in deciding which one should I
go for NodeClient or TransportClient for production deployments.
From the elastic search documentation what I understand is that Node client
eliminates the double hop that is present in Transport Client. Shall I opt
for Node client then? Please suggest.

On Monday, January 5, 2015 8:17:40 PM UTC+5:30, David Pilato wrote:

If you don’t want to define transport addresses, you’d better use a
NodeClient.
A NodeClient is actually a node running inside the cluster (with auto
discovery…) but without any data.

You just need to provide the cluster name with a NodeClient. See
http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/client.html#node-client

--
*David Pilato* | *Technical Advocate* | *Elasticsearch.com
http://Elasticsearch.com*
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr
https://twitter.com/elasticsearchfr | @scrutmydocs
https://twitter.com/scrutmydocs

Le 5 janv. 2015 à 13:46, Vijay Tiwary vijaykr...@gmail.com javascript:
a écrit :

Settings settings = ImmutableSettings.settingsBuilder()
.put(cluster.name, myClusterName).build();

Client client = new TransportClient(settings ).addTransportAddress(new
InetSocketTransportAddress(host1, 9300))

.addTransportAddress(new InetSocketTransportAddress(host2, 9300));

https://groups.google.com/d/msgid/elasticsearch/dd201d66-71b4-4d74-bb99-255144a083a2%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/98810b45-e4b5-4776-ac45-ba6b41adeff9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: 1.3.2 snapshot file system question

2015-01-05 Thread Mark Walkom

It won't work, the snapshot is run against any node that has shards of the
index and doesn't funnel data back to the node you ran the command on.

On 6 January 2015 at 02:40, bitsofinf...@gmail.com wrote:

I have a cluster (1.3.2) of 10 data nodes and 5 master nodes.

I want to take a snapshot of one index.

I'd like to configure a new fs snapshot mybackupdir where the
location is ONLY accessible from the node (master node) I am issuing the
snapshot creation PUT against.

Next, if I issue a snapshot PUT for mybackupdir/backup1 against the
master node where that location is indeed accessible, will this work? Does
the node that gets the snapshot request pull all the shard data from the
data nodes over to itself and write them to the snapshot dir on disk? Or
does each data-node responsible for each shard attempt to write to that
same location? (thereby requiring that the snapshot location be
accessible by all 15 nodes...)

I ask this because I have a cluster that spans two data-centers and they
don't all have access to a globally available NFS share where I could have
a common mount path for the snapshots root

thanks

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1b2274d3-304d-4470-8cda-f9462c831aad%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/1b2274d3-304d-4470-8cda-f9462c831aad%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8h2LV0OSWmEwYZ_SAF7t41k4G00_p6_9XVw3SVLk5n_w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Intersect statement in elasticsearch?

2015-01-05 Thread john . nesh76

Hi,

Is it possible to have an output from an intersection of data that are in 
two different index or field?
Example:
We have a list of botIP in elasticsearch and in another index where we 
capture data from firewall.

We want the list that match a predetermined field (example DST.IP) taken 
from the firewall (real time log) and botIP list (mostly static).

It is like a real time intersect in SQL.

INSERT INTO table_a VALUES (1, 'A'), (2, 'B'), (3, 'B');INSERT INTO table_b 
VALUES (1, 'B');
SELECT value FROM table_aINTERSECTSELECT value FROM table_b

value-
B


John

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/f40d3068-c1c2-47c3-b3ae-d709a9fe841e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Slow Commands with 1.2.4 to 1.4.2 Upgrade

2015-01-05 Thread pskieu

It takes upwards an average of 10 to 30 seconds. This is a test instance, so 
there's no additional load other than what I'm doing.

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1286f276-e20c-4ff2-acc0-90a5a3d57b0c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Aggregations without doc-counts

2015-01-05 Thread Elliott Bradshaw

I am only running a geohash grid aggregation. I reduce the precision
parameter as much as I can in each case. Any guesses on where most of the
time is being spent? I could dig through the source...

On Monday, January 5, 2015 9:49:01 AM UTC-5, Adrien Grand wrote:

On Mon, Jan 5, 2015 at 1:22 PM, Elliott Bradshaw ebrad...@gmail.com
javascript: wrote:

Just as a thought, would setting geohash = true or geohash_prefix = true
at index time improve performance?

On Monday, January 5, 2015 7:20:32 AM UTC-5, Elliott Bradshaw wrote:

Adrian,

Thanks for that. I had a feeling that that might be the case.

Any tips on improving aggregation performance. I'm working with a 20
shard index that is loaded on a 20 node cluster. Geohash grid aggregations
on the entire data set (with the size set to unlimited - a requirement) can
take as long as 8 seconds (and return ~ 1 million buckets). I am very
happy with that performance, but if there are any tricks to improve it I
would be glad to do so.

Thanks,

Elliott

On Tuesday, December 30, 2014 11:48:52 AM UTC-5, Adrien Grand wrote:

Hi Eliott,

The overhead of computing the doc counts is actually low, I don't think
you should worry about it.

On Tue, Dec 30, 2014 at 5:12 PM, Elliott Bradshaw ebrad...@gmail.com
wrote:

Hi,

Thanks!

- Elliott Bradshaw

--
You received this message because you are subscribed to the Google
Groups elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/834ebcb1-43b3-486d-bd1a-952005a6a66d%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/834ebcb1-43b3-486d-bd1a-952005a6a66d%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

https://groups.google.com/d/msgid/elasticsearch/d83c0bc5-bac5-4bae-9984-74ffbf6cd8b3%40googlegroups.com?utm_medium=emailutm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Adrien Grand

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/2d55880c-e539-4614-a99e-77d9cede47f1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: kibana empty dashboard

2015-01-05 Thread Eric

I solved my problem. The documentation elasticsearch.org didn't work 
(http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-http.html),
 
but it wasn't entirely their fault.

The options to use in your /etc/elasticsearch/elasticsearch.yml file if 
you're using elasticsearch 1.4.x with kibana 3.x are:

http.cors.allow-origin: /.*/ 
http.cors.enabled: true

Source: http://stackoverflow.com/a/26884367/2015890

In the documentation on elasticsearch.org, it says to just use an asterisk, 
* , but that didn't work.

This didn't work:

http.cors.allow-origin: * 

But this did:

http.cors.allow-origin: * 


Figures.

Furthermore, I would like to get SSL to work, but I think this will have to do 
for now



On Monday, January 5, 2015 5:39:43 AM UTC-5, Eric wrote:

 Here are the versions that I'm running:

 # Kibana version
 Kibana 3.1.2-07bbd7e
 eeded13255f154eaeceb4cf83105e4b4  kibana-3.1.2.tar.gz

 # Logstash version
 [root@elk ~]# /opt/logstash/bin/logstash version
 logstash 1.4.2-modified
 1db9f0864ff4b89380b39c39bc419031  logstash-1.4.2-1_2c0f5a1.noarch.rpm

 # Elasticsearch version
 [root@elk ~]# /usr/share/elasticsearch/bin/elasticsearch -v
 Version: 1.4.2, Build: 927caff/2014-12-16T14:11:12Z, JVM: 1.7.0_51
 6e2061f0734f9dbab263c1616701c1fe  elasticsearch-1.4.2.noarch.rpm

 # OS
 CentOS (CentOS-7.0-1406-x86_64-Everything.iso)
 Installed packages: Basic Web Server + Development tools

 Logstash runs fine. Elasticsearch runs fine. Kibana runs, but only shows 
 the screenshot shown below at, https://logstasht/#/dashboard



 https://lh3.googleusercontent.com/-8mIiX5lKJ_U/VKpmMkRSftI/AAACYWM/v4LxHMzEAGI/s1600/kibana.png




 On Wednesday, May 14, 2014 6:56:03 PM UTC-4, Mark Walkom wrote:

 I think you have extra quotes causing a problem, try - elasticsearch: 
 http://192.168.10.25:9200;,

 Regards,
 Mark Walkom

 Infrastructure Engineer
 Campaign Monitor
 email: ma...@campaignmonitor.com
 web: www.campaignmonitor.com


 On 15 May 2014 05:58, itbla...@gmail.com wrote:

 I have the following is showing up when I pull up my kibana dashboard: 

 http://192.168.10.25/#/dashboard

  {{dashboard.current.title}} 

 When I tail my logs I see the following 
 2014/05/14 13:31:45 [error] 17152#0: *7 open() 
 /var/www/kibana/app/diashboards/dashboard failed (2: No such file or 
 directory), client: 192.168.11.53, server: 192.168.10.25, request: GET 
 /app/diashboards/dashboard HTTP/1.1, host: 192.168.10.25 

 I have been pulling my hair out over this, all help would be appreciated 

 This is my config.js 

  /** @scratch /configuration/config.js/2 
* === Parameters 
*/ 
   return new Settings({ 

 /** @scratch /configuration/config.js/5 
  *  elasticsearch 
  * 
  * The URL to your elasticsearch server. You almost certainly don't 
  * want +a 
 href=http://localhost:9200+;http://localhost:9200+ here. Even if Kibana 
 and Elasticsearch are on 
  * the same host. By default this will attempt to reach ES at the 
 same host you have 
  * kibana installed on. You probably want to set it to the FQDN of 
 your 
  * elasticsearch host 
  */ 
 elasticsearch: http://192.168.10.25:9200;, 
 /*elasticsearch: http://+window.location.hostname+:9200;, 

 /** @scratch /configuration/config.js/5 
  *  default_route 
  * 
  * This is the default landing page when you don't specify a 
 dashboard to load. You can specify 
  * files, scripts or saved dashboards here. For example, if you had 
 saved a dashboard called 
  * `WebLogs' to elasticsearch you might use: 
  * 
  * +default_route: '/dashboard/elasticsearch/WebLogs',+ 
  */ 
 default_route : '/dashboard/file/default.json', 

 /** @scratch /configuration/config.js/5 
  *  kibana-int 
  * 
  * The default ES index to use for storing Kibana specific object 
  * such as stored dashboards 
  */ 
 kibana_index: kibana-int, 

 /** @scratch /configuration/config.js/5 
  *  panel_name 
  * 
  * An array of panel modules available. Panels will only be loaded 
 when they are defined in the 
  * dashboard, but this list is used in the add panel interface. 
  */ 
 panel_names: [ 
   'histogram', 
   'map', 
   'pie', 
   'table', 
   'filtering', 
   'timepicker', 
   'text', 
   'hits', 
   'column', 
   'trends', 
   'bettermap', 
   'query', 
   'terms', 
   'stats', 
   'sparklines' 
 ] 
   }); 
 }); 

 ngix (default)

 /** @scratch /configuration/config.js/1
  * == Configuration
  * config.js is where you will find the core Kibana configuration. This 
 file contains parameter that
  * must be set before kibana is run for the first time.
  */
 define(['settings'],
 function (Settings) {


   /** @scratch /configuration/config.js/2
* === Parameters
*/
   return new Settings({

 /** @scratch /configuration/config.js/5
  *

Re: Connecting remotely to elastic search cluster

2015-01-05 Thread David Pilato

I myself prefer TransportClient as it has absolutely no impact on the cluster
state.
A new node joining the cluster generates a cluster state update which is
propagated to all nodes.

I prefer having a very stable cluster state than continuous updates. Think
about a Java batch which starts every minute and connect to the cluster to
perform whatever action.
That would be a lot of events per day.

That’s why I mostly prefer using TransportClient.

My 2 cents.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr
https://twitter.com/elasticsearchfr | @scrutmydocs
https://twitter.com/scrutmydocs

Le 5 janv. 2015 à 19:15, Vijay Tiwary vijaykr.tiw...@gmail.com a écrit :

On Monday, January 5, 2015 8:17:40 PM UTC+5:30, David Pilato wrote:
If you don’t want to define transport addresses, you’d better use a
NodeClient.
A NodeClient is actually a node running inside the cluster (with auto
discovery…) but without any data.

You just need to provide the cluster name with a NodeClient. See
http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/client.html#node-client

http://www.elasticsearch.org/guide/en/elasticsearch/client/java-api/current/client.html#node-client

--
David Pilato | Technical Advocate | Elasticsearch.com
http://elasticsearch.com/
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr
https://twitter.com/elasticsearchfr | @scrutmydocs
https://twitter.com/scrutmydocs

Le 5 janv. 2015 à 13:46, Vijay Tiwary vijaykr...@gmail.com javascript: a
écrit :

Settings settings = ImmutableSettings.settingsBuilder()
.put(cluster.name http://cluster.name/, myClusterName).build();
Client client = new TransportClient(settings ).addTransportAddress(new
InetSocketTransportAddress(host1, 9300))
.addTransportAddress(new InetSocketTransportAddress(host2, 9300));

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com
mailto:elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/98810b45-e4b5-4776-ac45-ba6b41adeff9%40googlegroups.com

https://groups.google.com/d/msgid/elasticsearch/98810b45-e4b5-4776-ac45-ba6b41adeff9%40googlegroups.com?utm_medium=emailutm_source=footer.
For more options, visit https://groups.google.com/d/optout
https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9740D8FF-F93A-4C5B-92E9-15602E1701FF%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

Kibana version 4 architecture

2015-01-05 Thread bitsofinfo . g

Hi 

Starting to experiment w/ Kibana 4. I see that now there is a server side 
component where it appears all client requests proxy through?

What is the recommended topology for deploying this for HA,* is there any 
client session state maintained in this server side process or is it pretty 
much a stateless proxy*?

What does the server side component actually do now, that was offloaded 
from the prior kibana architecture where everything was only in the UI?

thanks!

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/634084f9-c04d-488e-baa7-c1f692c06d8e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: How is data stored

2015-01-05 Thread Peter Johnson

have a look 
at https://github.com/polyfractal/elasticsearch-inquisitor#analyzer-testing

On Thursday, 1 January 2015 17:05:25 UTC, Bruno Kamiche wrote:

 Thanks for your replies, that gave me the clue for what I was looking for, 
 and now it is solved!

 On Wednesday, December 31, 2014 10:36:27 PM UTC-5, Bruno Kamiche wrote:

 Hello, I'm new in using elasticsearch, so maybe this is a basic 
 question...

 Is there any way to see how is the text stored or at least how it would 
 look once the filters defined for a field (in the analyzer) are applied?

 I know that the actual field is stored as is, and the filters are 
 considered for indexing purposes, but i want to see what is the result of 
 that operation.

 Why do I ask this, the reason is that I'm applying filters to a field 
 (for example removing urls), and performing aggregations on that field, but 
 the aggregations return http as a used word, and guess that is not 
 correct, as it shall be removed...




-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/59accfbd-46a4-437c-81b1-6978b752ac4c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Ignore a field in the scoring

2015-01-05 Thread Roger de Cordova Farias

The added field is an array of Integers, but we are not using it in the
query at all

We are not querying the _all field, it is disabled in our type mapping

Our query is something like this:

{
query: {
query_string: {
fields: [
name
],
query: roger
}
}
}

I ran this query. In the first result, I added a new field called
bookmarked_by with a numeric value. Then I ran the same query again. The
document in which I added the new field is no longer the first result

2014-12-26 17:34 GMT-02:00 Doug Turnbull
dturnb...@opensourceconnections.com:

Are you querying the _all field? How are you doing your searches?

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-all-field.html

The _all field receives a copy of every field you index, so adding data
here could impact scores regardless of the source field.

Otherwise, fields are scored independently before being put together by
other queries like boolean queries or dismax. Are you using
boolean/dismax/etc over multiple fields?

-Doug

On Fri, Dec 26, 2014 at 11:59 AM, Ivan Brusic i...@brusic.com wrote:

Use the field in a filter and not part of the query. Is this field free
text?

Ivan
On Dec 23, 2014 9:12 PM, Roger de Cordova Farias
roger.far...@fontec.inf.br wrote:

Hello

Our documents have metadata indexed with them, but we don't want the
metadata to interfere in the scoring

After a user searches for documents, they can bookmark them (what means
we add more metadata to the document), then in the next search with the
same query the bookmarked document appears in a lower (worse) position

Is there a way to completely ignore one or more specific fields in the
scoring of every query? as in indexing time or something?

Note that we are not using the metadata field in the query, but yet it
lowers the score of every query

We cannot set the index attribute of this field to no because we are
gonna use it in other queries

--
You received this message because you are subscribed to the Google
Groups elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAJp2533Rjjec4SwXe_p-0eHYkkyEegFyP9DUMGQfHhua8ZyMWQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAJp2533Rjjec4SwXe_p-0eHYkkyEegFyP9DUMGQfHhua8ZyMWQ%40mail.gmail.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAKsYquQJMbfztJ%2Ba2_jpi-fVG%3DvcnXYHS-7bKvaOX4hA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAKsYquQJMbfztJ%2Ba2_jpi-fVG%3DvcnXYHS-7bKvaOX4hA%40mail.gmail.com?utm_medium=emailutm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
Doug Turnbull
Search Big Data Architect
OpenSource Connections http://o19s.com

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALG6HL9ND_SWteSetZL9059WyGRZvJrO2k4PQ9FQ1zUFhjbsxw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CALG6HL9ND_SWteSetZL9059WyGRZvJrO2k4PQ9FQ1zUFhjbsxw%40mail.gmail.com?utm_medium=emailutm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAJp2533UjpAz2dvNitdD-%3DaoXL9rrkZdd%3DzC3LZz8xWYvBAoFQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

1.3.2 snapshot file system question

2015-01-05 Thread bitsofinfo . g

I have a cluster (1.3.2) of 10 data nodes and 5 master nodes.

I want to take a snapshot of one index. 

I'd like to configure a new fs snapshot mybackupdir where the 
location is ONLY accessible from the node (master node) I am issuing the 
snapshot creation PUT against. 

Next, if I issue a snapshot PUT for mybackupdir/backup1 against the 
master node where that location is indeed accessible, will this work? Does 
the node that gets the snapshot request pull all the shard data from the 
data nodes over to itself and write them to the snapshot dir on disk? Or 
does each data-node responsible for each shard attempt to write to that 
same location? (thereby requiring that the snapshot location be 
accessible by all 15 nodes...)

I ask this because I have a cluster that spans two data-centers and they 
don't all have access to a globally available NFS share where I could have 
a common mount path for the snapshots root

thanks

-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1b2274d3-304d-4470-8cda-f9462c831aad%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: ES Nodes storage capacity

2015-01-05 Thread Mark Walkom

Can you elaborate what you mean by becoming an issue?

When you add a node into the cluster it will automatically start to
reallocate shards to the new node, you can't have a node sitting there idle
and with lots of disk space free waiting for the other nodes to fill up
before being called upon.

As for nginx, it depends what you do with it. ES will spread the load
amongst the cluster automatically, but if you're using nginx as a front end
then it's up to you to factor the other nodes in.

On 6 January 2015 at 01:51, Simon Thorley si...@thenom.co.uk wrote:

Hi all,

I have designed and setup an ELK stack that can be expanded horizontally
(i hope) using the following technologies:

haproxy x2 (failover) = Logstash x2 = nginx x2 (failover) =
elasticsearch x2

The capacity of the elasticsearch nodes have come into question and this
has raised a few questions regards spec of a new node.

Here is the current specs of the elasticsearch nodes ( i have 2 of these):

· HP DL360p Gen8 10-SFF CTO Server

· 64G PC3L-12800R-11

· 8x 900G 10K SAS

· 2x 300G 10K SAS

· 2G FBWC

· Dual 750W PSU

· 4P 1GBE 331FLR

So i am currently running the OS from the 2x 300Gb in a mirror raid, and 2
data logical data drives using 4 of the 8 900GB drives (so 2 striped raids
containing 4x900GB drives).

This is all working fine but the data capacity has become an issue (14TB
total available) . I think for the moment i have enough compute power but
what would happen if i added a lower spec node (or multiple), marked them
as a data node (non-master) but with different storage capacity available.
Say for example 20TB in each.

If the original 2 nodes filed there data stores and the only store
available was the new node(s) then they would be processing the shards
alone and there would be no protection from replica's if this node went
down, no?

As an additional but not as critical at the moment:

If i eventually have the same issue with compute power, if these 2 someday
become saturated and i make another node a master node but it was half the
spec, would elasticsearch realize this and distribute the load or is this
purely down to nginx load distribution?

Thanks for any help\advice in advance.

Simon

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8450d29f-9a1b-40c0-87e5-9fa2b18f364d%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/8450d29f-9a1b-40c0-87e5-9fa2b18f364d%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8QvYr9hCBgYPC4LoPOv-eTqRLh-zFMxh1%3D95H4qYu4RQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Starting elasticsearch on Debian jessie with systemd

2015-01-05 Thread Jiri Jagos

After some panic and searching the way mentioned in the docs 
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/setup-repositories.html#_yum
 
(at the end of the section) worked for me.

For impatient:

sudo /bin/systemctl daemon-reload # this line was the one I missed before
sudo /bin/systemctl enable elasticsearch.service




On Friday, November 14, 2014 8:54:54 AM UTC+1, Alexandr Bravo wrote:

 Hi,

 I'm installing ElasticSearch from .deb package (from elasticsearch 
 repository) on the Linux server with Debian jessie (testing). This is a 
 second node I installed, the first one was done about 3 month ago also on 
 Debian jessie but before the distriv was switched to systemd. And I had no 
 any problems with that first node. 

 Now, on the fresh install, when I try to start the service by:

 /etc/init.d/elasticsearch start

 I get an error message:

 [] Starting elasticsearch (via systemctl): elasticsearch.serviceFailed 
 to start elasticsearch.service: Unit elasticsearch.service failed to load: 
 No such file or directory.
  failed!

 As I understand for starting service with systemd (I skip all words about 
 the idea to switch to it) I need first to register/enable it via something 
 like:

 /bin/systemctl enable elasticsearch.service

 But config file elasticsearch.service is included only in RPM package and 
 not available in .deb.

 Two questions:

 1. For developers - is it possible to include in elasticsearch.deb package 
 all config files that are needed for systemd?
 2. Can anybody to share the working examples of such configs to start 
 elasticsearch with systemd under Debian jessie?

 Many thanks,
   Alexandr 




-- 
You received this message because you are subscribed to the Google Groups 
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/fd214d2a-1cf2-4a04-8a96-fad00da16f7f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Marvel Sense GET/POST

2015-01-05 Thread Mark Walkom

It'd be great if you could raise this as an issue on github for this
behaviour to be checked - https://github.com/elasticsearch/elasticsearch

On 6 January 2015 at 00:06, Paul Scott p...@duedil.com wrote:

Regarding the behaviour of Sense to automatically choose POST regardless
of the user writing GET:

I was recently rewriting a search query I had written in the Sense UI from
a search query to a request for a raw document by ID, using the GET API. In
the process of changing from

GET /index/type/_search
{ ... }

GET /index/type/id

I submitted

GET /index/type/id
{... }

which Sense helpfully submitted as a POST request, over-writing the
document with the search query. In production.

To say that this behaviour was unexpected and unhelpful would be an
understatement. I would consider it irresponsible and unsafe, and I highly
recommend the behaviour be disabled.

Please try to read past my obvious irritation and the part of this mistake
which falls on my head, to the part in this mistake that the Sense UI
played too.

All the best,

Paul

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f2588751-a008-4b59-9041-b63c098580dd%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/f2588751-a008-4b59-9041-b63c098580dd%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_GammQv_kpXTGG6Cv9cGSjfyWOFaUf4Ln6bdLPewGM6w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Marvel/Sense security?

2015-01-05 Thread Mark Walkom

You set marvel.agent.exporter.es.hosts

in elasticsearch.yml.

It'd let you then put some kind of proxy layer between ES but still allow
marvel to operate.

On 5 January 2015 at 21:26, John Bohne johnboh...@gmail.com wrote:

Why would I want to do that? I'm using Apache by the way.

I saw something about marvel.agent.exporter.es.hosts and setting up
user:pwd@host:9200 but the documentation doesn't really specify where to
do this. In elasticsearch.yml? Under what heading? Would this even solve
the problem?

On Sunday, January 4, 2015 12:15:23 PM UTC-6, David Pilato wrote:

If you have disabled access to PUT method (with Nginx for example), SENSE
won't be able to PUT any document.

My 2 cents.

--
David ;-)
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 4 janv. 2015 à 17:16, John Bohne johnb...@gmail.com a écrit :

I was looking in the documentation and online for Marvel but I am not
sure how to implement security for Marvel. For example, if one were to
access example.com:9200/_plugin/marvel/sense/index.html , one would be
able to put documents through Sense on my cluster(s). I saw something about
marvel.hosts but I am not sure how to implement that.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/559eadaa-37cb-4129-8a65-38847c60126f%
40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/559eadaa-37cb-4129-8a65-38847c60126f%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/22d92adf-b8ab-4b7c-a955-64c05a9a96af%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/22d92adf-b8ab-4b7c-a955-64c05a9a96af%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9NUYMxEac3KYWWCwycQcztiCw1NDevR5CYTsfFiPfQqw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: shards allocation and limitations

2015-01-05 Thread Mark Walkom

Depends on your setup.

Increasing shard count is only going to be useful if you add more nodes.

On 5 January 2015 at 16:21, phani.nadimi...@goktree.com wrote:

Hi All,

I have an index with 51 millions records i have 2 nodes in my
cluster.

no of shards for the above index is : 5
replicas are :1

my question is how many records we can index to get better
search capabilities? Is there any limitation for the index that will hold
these many records. or please suggest me what are the settings in
elasticsearch.yml file to optimize the index which is hoding 51 million
records.

can we increase number of shards other than five ? if we
increase that one is there any problems with searching records?

please help me.

Thanks

phani

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ac901c96-05f8-49f3-ac6c-826ee6f081f0%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/ac901c96-05f8-49f3-ac6c-826ee6f081f0%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_P0%2BKCF6m6jeHZvyDf4Fb2ZU9dwk%3DA1UbjNxU%2BLAXsAQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Large Number of Shards are unassigned post cluster restart

2015-01-05 Thread Mark Walkom

There are settings you can change, see
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-indices.html#recovery

On 5 January 2015 at 12:50, Salman ahmed.sal...@gmail.com wrote:

On ES 1.4.2 cluster, post cluster restart, 90% of shards are unassigned. Is
there a way to stop the whole cluster again, and force quicker reassignment
of shards. I researched and few users have had similiar issues, so am
hoping
someone can guide me on optimal way to fix this.

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Large-Number-of-Shards-are-unassigned-post-cluster-restart-tp4068453.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/1420422652801-4068453.post%40n3.nabble.com
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9EZnvew-Bc%2BnKxZo%2B6fzeiYs%2BdsK0Q5epF1BJ5TeeFHA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Marvel Sense GET/POST

2015-01-05 Thread Boaz Leskes

Heya,

Yeah, I agree this is annoying and dangerous. For what it's worth , this
happens now is that browsers do not allow sending GETs with a body so Sense
allows you to use the GET syntax as documented but sends it as POST (which
is accepted by ES). The plan is to change and do this only for while listed
end points (ala _search). We already have an issue for it. I'll report here
what it's done.

Cheers,
Boaz

On Monday, January 5, 2015 9:57:41 PM UTC+1, Mark Walkom wrote:

It'd be great if you could raise this as an issue on github for this
behaviour to be checked - https://github.com/elasticsearch/elasticsearch

On 6 January 2015 at 00:06, Paul Scott p...@duedil.com wrote:

Regarding the behaviour of Sense to automatically choose POST regardless
of the user writing GET:

I was recently rewriting a search query I had written in the Sense UI
from a search query to a request for a raw document by ID, using the GET
API. In the process of changing from

GET /index/type/_search
{ ... }

GET /index/type/id

I submitted

GET /index/type/id
{... }

which Sense helpfully submitted as a POST request, over-writing the
document with the search query. In production.

To say that this behaviour was unexpected and unhelpful would be an
understatement. I would consider it irresponsible and unsafe, and I highly
recommend the behaviour be disabled.

Please try to read past my obvious irritation and the part of this
mistake which falls on my head, to the part in this mistake that the Sense
UI played too.

All the best,

Paul

https://groups.google.com/d/msgid/elasticsearch/f2588751-a008-4b59-9041-b63c098580dd%40googlegroups.com?utm_medium=emailutm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
elasticsearch group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/06de9586-41f1-4b5e-bce8-fe258dc93574%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

66 matches

Mail list logo