Re: Where is Elasticsearch storing data

[David Pilato]

To answer to the initial question, I would not really worry about having acces 
to the data dir.

There are APIs for that. Nodes Stats and Nodes Info should give you all metrics 
you need.
Snapshot and restore should help for backups.

HTH

David

> Le 29 mai 2015 à 03:04, Jim Malone  a écrit :
> 
> Thank you, that little query is very helpful.  Although there isn't any 
> actual "data" property anywhere in the response.  I'm guessing it's cause our 
> data is still just cached, and it appears to be in the "home" directory.  Now 
> we just have to figure out how to set the data dir using es-yarn so that it's 
> stored on the local nodes permanently.  Haven't had luck finding that yet.
> 
> And I will use the new forum going forward, just wanted to continue on this 
> thread instead of starting a new one.
> 
> Jim
> 
>> On Thursday, May 28, 2015 at 6:10:49 PM UTC-4, Mark Walkom wrote:
>> If you query "_nodes?pretty | grep data" you will see a value, that's where 
>> the data is.
>> 
>> 
>> PS - We're moving to https://discuss.elastic.co/, please join us there for 
>> any future discussions!
>> 
>>> On 29 May 2015 at 00:31, Jim Malone  wrote:
>>> I similarly can't find where the data is stored.  We have it running with 
>>> ES-yarn though.  This article says that the data is only stored in 
>>> container storage by default (we haven't changed any settings):
>>> 
>>> https://www.elastic.co/guide/en/elasticsearch/hadoop/master/ey-setup.html
>>> 
>>> However, the data is definitely persisting somehow between restarts so it 
>>> has to be stored somewhere.  We just can't find where.  There is no 
>>> "path.data" dir and also there's no "/var/lib/elasticsearch/data" dir like 
>>> I've seen on other posts.  And we can't find it in hdfs either.
>>> 
>>> So can anyone please point me to where we might be able to find the data?  
>>> If it helps, we are loading indexes from both hive and from logstash 
>>> directly.
>>> 
>>> Thanks,
>>> Jim
>>> 
 On Wednesday, May 27, 2015 at 6:56:54 PM UTC-4, Mark Walkom wrote:
 Check out 
 https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-dir-layout.html
 
 
 PS - We're moving to https://discuss.elastic.co/, please join us there for 
 any future discussions!
 
> On 28 May 2015 at 01:27, David Pilato  wrote:
> by default in path.data dir.
> In zip or tar.gz distribution, it’s in "data" dir
> 
> 
> -- 
> David Pilato - Developer | Evangelist 
> elastic.co
> @dadoonet | @elasticsearchfr | @scrutmydocs
> 
> 
> 
> 
> 
>> Le 27 mai 2015 à 17:25, John Adams  a écrit :
>> 
>> I cannot find any info on where Elasticsearch is storing it's data.
>> I would like to manage this potentially valuable information.
>> Where, how big, how to back it up, restore it, etc.
>> Then on to clusters, etc..
>> 
>> Thanks,
>>  John
>> 
>> -- 
>> Please update your bookmarks! We have moved to 
>> https://discuss.elastic.co/
>> --- 
>> You received this message because you are subscribed to the Google 
>> Groups "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send 
>> an email to elasticsearc...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/4893b923-dbf2-45c3-a0e3-63c6d0b37b1c%40googlegroups.com.
>> For more options, visit https://groups.google.com/d/optout.
> 
> -- 
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> --- 
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/4836BD52-F0A0-4507-BE1D-14E3CA4B0FD7%40pilato.fr.
> 
> For more options, visit https://groups.google.com/d/optout.
>>> 
>>> -- 
>>> Please update your bookmarks! We have moved to https://discuss.elastic.co/
>>> --- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to elasticsearc...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/elasticsearch/bc0ad06d-21b3-4ca7-aade-876f15b379a3%40googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
> 
> -- 
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> --- 
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion

Re: Aggregation profiling?

[James Macdonald]

I don't have an answer, but I really like this question. I too would love
to see more query and aggregation profiling tools for performance
optimization purposes.

Also, I assume you have already looked at this, but have you made sure you
are not evicting anything from your in memory field data?

James

On Mon, May 25, 2015 at 4:08 PM, Mike Sukmanowsky <
mike.sukmanow...@gmail.com> wrote:

> I don't believe there are any current endpoints in the API that support
> this, but are there plans to add better profiling information to ES
> aggregation queries? We'll see some agg queries return in 11s, then <5s
> then >11s again. Sometimes we can see associated filter cache expirations,
> but it's really hard to line these up to one specific query in our
> production environment since multiple users are executing queries
> simultaneously.
>
> It'd be really helpful to optionally see where aggregation queries are
> spending the bulk of their time to help us understand what to improve in
> the future.
>
> Anything we can do here right now?
>
> --
> Mike Sukmanowsky
> Aspiring Digital Carpenter
>
> *e*: mike.sukmanow...@gmail.com
>
> facebook  | twitter
>  | LinkedIn
>  | github
> 
>
>   --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAOH6cu5WSGqQ%2BZ0_qrofXEvwo8JuSH9xoSbZgSwiT90MJ_wxdA%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAABsnTZOmx-fk%2BG9dR6-XYB_1j7mGRNRwTqvQRwKx0YAcopFWA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Considering scalability , is it right to keep a large number of primary shards at beginning？

[xinmeike]

Thank you for your answer!  

I would like to learn Logstash and take reindex into consideration.

best regards,
shinyke

在 2015年5月28日星期四 UTC+8下午5:21:00，Mark Walkom写道：
>
> You don't need to stop everything to reindex, leverage aliases and you can 
> do it live.
>
> On 28 May 2015 at 18:57, > wrote:
>
>>
>> Thanks for your answer~
>>
>> May be when 10 servers extends to 100 servers it can work ,but I‘m afraid 
>> that if we do reindex to 100 servers  it may cost a long time and huge I/O 
>> resources.We need to stop the service for a long time and all the data need 
>> to transport from old index to new one.
>>
>> Is there any easier way to horizontal expansion？
>>
>> P.S. I cann’t visit  https://discuss.elastic.co/ 
>>  today .It is blank all the time. (･ˇ_ˇ･)
>>
>>
>> 在 2015年5月28日星期四 UTC+8下午3:05:13，Mark Walkom写道：
>>>
>>> You don't want 400 shards on 10 servers. You do want the ability to 
>>> reindex to allow you to reshard to deal with this issue.
>>> Logstash 1.5 can do this very easily, see this example 
>>> .
>>>
>>> However you probably don't want an index with 200 shards irrespective, 
>>> you may want to take a look at your data structure and split things out.
>>>
>>>
>>> PS - We're moving to https://discuss.elastic.co/, please join us there 
>>> for any future discussions!
>>>
>>>
>>>
>>> On 28 May 2015 at 12:46,  wrote:
>>>

 Hi，

 I feel very confuse when deciding the number of primary shards at 
 beginning.

 As we know the number of shards and replicas can be defined per index 
 at the time the index is created. After the index is created, we may 
 change 
 the number of replicas dynamically anytime but we cannot change the number 
 of primary shards after-the-fact. Our ES project may be run as trial 
 version at beginning an it is only 10 machines in cluster. However, if the 
 project run in production environment,  the machine magnitude must 
 increase and there will have 200 or more machines.

 How can we decide shards number at beginning? Is it encourage to run 
 400 or more shards in 10 machines, will it reduce the performance of 
 cluster? 


 Thank you for reading and look forward to your suggestions.


  -- 
 Please update your bookmarks! We have moved to 
 https://discuss.elastic.co/
 --- 
 You received this message because you are subscribed to the Google 
 Groups "elasticsearch" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to elasticsearc...@googlegroups.com.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/elasticsearch/a6eb715a-6543-4882-8635-81a0d22ca1d7%40googlegroups.com
  
 
 .
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>  -- 
>> Please update your bookmarks! We have moved to 
>> https://discuss.elastic.co/
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/65cd6e3e-97f1-416e-8f97-e34a8dd7671e%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/5591e786-fa10-4727-9bf0-ba8b10ad8786%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Where is Elasticsearch storing data

[Jim Malone]

Thank you, that little query is very helpful.  Although there isn't any 
actual "data" property anywhere in the response.  I'm guessing it's cause 
our data is still just cached, and it appears to be in the "home" 
directory.  Now we just have to figure out how to set the data dir using 
es-yarn so that it's stored on the local nodes permanently.  Haven't had 
luck finding that yet.

And I will use the new forum going forward, just wanted to continue on this 
thread instead of starting a new one.

Jim

On Thursday, May 28, 2015 at 6:10:49 PM UTC-4, Mark Walkom wrote:
>
> If you query "_nodes?pretty | grep data" you will see a value, that's 
> where the data is.
>
>
> PS - We're moving to https://discuss.elastic.co/, please join us there 
> for any future discussions!
>
> On 29 May 2015 at 00:31, Jim Malone > 
> wrote:
>
>> I similarly can't find where the data is stored.  We have it running with 
>> ES-yarn though.  This article says that the data is only stored in 
>> container storage by default (we haven't changed any settings):
>>
>> https://www.elastic.co/guide/en/elasticsearch/hadoop/master/ey-setup.html
>>
>> However, the data is definitely persisting somehow between restarts so 
>> it has to be stored somewhere.  We just can't find where.  There is no 
>> "path.data" dir and also there's no "/var/lib/elasticsearch/data" dir 
>> like I've seen on other posts. And we can't find it in hdfs either.
>>
>> So can anyone please point me to where we might be able to find the data? 
>> If it helps, we are loading indexes from both hive and from logstash 
>> directly.
>>
>> Thanks,
>> Jim
>>
>> On Wednesday, May 27, 2015 at 6:56:54 PM UTC-4, Mark Walkom wrote:
>>>
>>> Check out 
>>> https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-dir-layout.html
>>>
>>>
>>> PS - We're moving to https://discuss.elastic.co/, please join us there 
>>> for any future discussions!
>>>
>>> On 28 May 2015 at 01:27, David Pilato  wrote:
>>>
 by default in path.data dir.
 In zip or tar.gz distribution, it’s in "data" dir


 -- 
 *David Pilato* - Developer | Evangelist 
 *elastic.co *
 @dadoonet  | @elasticsearchfr 
  | @scrutmydocs 
 




  
 Le 27 mai 2015 à 17:25, John Adams  a écrit :

 I cannot find any info on where Elasticsearch is storing it's data.
 I would like to manage this potentially valuable information.
 Where, how big, how to back it up, restore it, etc.
 Then on to clusters, etc..

 Thanks,
  John

 -- 
 Please update your bookmarks! We have moved to 
 https://discuss.elastic.co/
 --- 
 You received this message because you are subscribed to the Google 
 Groups "elasticsearch" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to elasticsearc...@googlegroups.com.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/elasticsearch/4893b923-dbf2-45c3-a0e3-63c6d0b37b1c%40googlegroups.com
  
 
 .
 For more options, visit https://groups.google.com/d/optout.


  -- 
 Please update your bookmarks! We have moved to 
 https://discuss.elastic.co/
 --- 
 You received this message because you are subscribed to the Google 
 Groups "elasticsearch" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to elasticsearc...@googlegroups.com.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/elasticsearch/4836BD52-F0A0-4507-BE1D-14E3CA4B0FD7%40pilato.fr
  
 
 .

 For more options, visit https://groups.google.com/d/optout.

>>>
>>>  -- 
>> Please update your bookmarks! We have moved to 
>> https://discuss.elastic.co/
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/bc0ad06d-21b3-4ca7-aade-876f15b379a3%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"ela

Geo Context Suggester Index Size

[Bhuvan Bamba]

Hi,
I am fairly new to elasticsearch and I am trying to index a million 
residences with their geolocation for context suggester capabilities. The 
size of the index seems to be 1GB per million residences which seems 
exceptionally large to me. Without the geo context the size is roughly 
200MB per million residences. Can someone explain why adding the geo 
context increases the size of the index by such a large factor.

Thanks,
Bhuvan

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/ad41a3fa-3278-4c59-8af5-9cc425aee691%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Where is Elasticsearch storing data

[Mark Walkom]

If you query "_nodes?pretty | grep data" you will see a value, that's where
the data is.


PS - We're moving to https://discuss.elastic.co/, please join us there for
any future discussions!

On 29 May 2015 at 00:31, Jim Malone  wrote:

> I similarly can't find where the data is stored.  We have it running with
> ES-yarn though.  This article says that the data is only stored in
> container storage by default (we haven't changed any settings):
>
> https://www.elastic.co/guide/en/elasticsearch/hadoop/master/ey-setup.html
>
> However, the data is definitely persisting somehow between restarts so it
> has to be stored somewhere.  We just can't find where.  There is no
> "path.data" dir and also there's no "/var/lib/elasticsearch/data" dir
> like I've seen on other posts. And we can't find it in hdfs either.
>
> So can anyone please point me to where we might be able to find the data?
> If it helps, we are loading indexes from both hive and from logstash
> directly.
>
> Thanks,
> Jim
>
> On Wednesday, May 27, 2015 at 6:56:54 PM UTC-4, Mark Walkom wrote:
>>
>> Check out
>> https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-dir-layout.html
>>
>>
>> PS - We're moving to https://discuss.elastic.co/, please join us there
>> for any future discussions!
>>
>> On 28 May 2015 at 01:27, David Pilato  wrote:
>>
>>> by default in path.data dir.
>>> In zip or tar.gz distribution, it’s in "data" dir
>>>
>>>
>>> --
>>> *David Pilato* - Developer | Evangelist
>>> *elastic.co *
>>> @dadoonet  | @elasticsearchfr
>>>  | @scrutmydocs
>>> 
>>>
>>>
>>>
>>>
>>>
>>> Le 27 mai 2015 à 17:25, John Adams  a écrit :
>>>
>>> I cannot find any info on where Elasticsearch is storing it's data.
>>> I would like to manage this potentially valuable information.
>>> Where, how big, how to back it up, restore it, etc.
>>> Then on to clusters, etc..
>>>
>>> Thanks,
>>>  John
>>>
>>> --
>>> Please update your bookmarks! We have moved to
>>> https://discuss.elastic.co/
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearc...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/4893b923-dbf2-45c3-a0e3-63c6d0b37b1c%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>>  --
>>> Please update your bookmarks! We have moved to
>>> https://discuss.elastic.co/
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearc...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/4836BD52-F0A0-4507-BE1D-14E3CA4B0FD7%40pilato.fr
>>> 
>>> .
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/bc0ad06d-21b3-4ca7-aade-876f15b379a3%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9pYUtnr%2B4U35t1oNu%3Dwbom4UqZyJEs0iB96CikSOiD7g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elasticsearch Report Plugin to Generate Excel Reports

[Mark Walkom]

FYI rivers have been deprecated and will be removed from future versions of
ES.
While this plugin looks really handy, you may want to rearchitect it to
take this into account :)


PS - We're moving to https://discuss.elastic.co/, please join us there for
any future discussions!

On 29 May 2015 at 03:40, Raghavendar T S  wrote:

> Hi Friends
>
> I have developed a plugin for Elasticsearch to generate Excel reports and
> hosted the same in GitHub. The plugin exposes REST interface where simple
> input configuration can save the report to the system or E-Mail to the
> required recipients. Check it out and give your valuable comments.
>
> Plugin URL: https://github.com/raghavendar-ts/Elasticsearch-Report-Plugin
>
>
>
> Thank you
> Raghavendar T S
>
> --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/145a0f82-165f-4020-8083-50bebb73313e%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8JKJvFTgY4xv3_VA9sYQZhpyZtDJLNgcH_AA76GOxoLA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Snapshot files are created every time even no change in index.

[Mark Walkom]

Snapshots are taken on the shard level, so if a shard changes due to things
like merging, then it needs to record this.
You can delete snapshots, ES will just hold onto the shards it needs.

PS - We're moving to https://discuss.elastic.co/, please join us there for
any future discussions!

On 28 May 2015 at 21:13, shoebalig  wrote:

>  I have a cron-job runs daily. There are chances that there is no
> modification or changes found in original index. however cron-job create
> snapshot file again on its scheduled time. Due to which number of snapshot
> files is too large. which leads restoration of index, takes a lot of time.
> Is there anyway I should create snapshot file only when there is a
> difference in index data?
> What if I retain only last 5 or last 10 snapshot? Will this cause any data
> lose anyway? And on restore whole index will be restored?
>
> Please suggest.
>
>
>
> --
> View this message in context:
> http://elasticsearch-users.115913.n3.nabble.com/Snapshot-files-are-created-every-time-even-no-change-in-index-tp4074965.html
> Sent from the Elasticsearch Users mailing list archive at Nabble.com.
>
> --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/1432811614768-4074965.post%40n3.nabble.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9kFwpxBaDZt0C_HzUG_U5zHpTWgGBqfReEzrB0WK1umg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elasticsearch: 2-node cluster with failover

[Mark Walkom]

Master only nodes don't need to leverage FS caching as they hold no data.
So you can easily increase their heap to 75% of system.
However you really don't want to have less than 1GB of heap irrespective of
the role it plays.


PS - We're moving to https://discuss.elastic.co/, please join us there for
any future discussions!

On 28 May 2015 at 23:11, prakhar  wrote:

> Sorry to bother you @Mark, but this term *appropriate heap size* always
> haunts me. Suppose, I have 2 virtual machines (4 core, 8 gb RAM each).
> According to elasticsearch's documentation, half of RAM should be allocated
> to elasticsearch. So, this is what I came up with:
>
> Node1: Elasticsearch 1.5.2 *Master-Data* node with 4 gb HEAP (*server1*)
> Node2: Elasticsearch 1.5.2 *Master-Only*(no data) node with 1 gb HEAP
> (*server1*)
> Node3: Elasticsearch 1.5.2 *Master-Data* node with 4 gb HEAP (*server2*)
>
> Do you think this is a sane strategy? Moreover, if I make *Master-Only*
> node
> (Node2) invisible to load balancer i.e. I simply won't include it at all,
> will it still require 1 gb HEAP?
> Please forgive me if I am asking something too naive.
>
>
>
> --
> View this message in context:
> http://elasticsearch-users.115913.n3.nabble.com/Elasticsearch-2-node-cluster-with-failover-tp4074949p4074968.html
> Sent from the Elasticsearch Users mailing list archive at Nabble.com.
>
> --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/1432818693390-4074968.post%40n3.nabble.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_Oj_d3qG%3DQoTW0A_ChjdXigoj2Rn2iOQurnGpCcU21dQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: How do I index data from logs in a specific format in elastic search?

[Mark Walkom]

You can just send it as it and it will be indexed. Of course you can also
map things so they are correctly identified.
Depends on what you want to do.


PS - We're moving to https://discuss.elastic.co/, please join us there for
any future discussions!

On 28 May 2015 at 23:20,  wrote:

>  I receive data in this format: Name Date Details. How do I index it into
> indices of Elastic Search automatically ?
>
> --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/38bb9c44-b428-4111-8115-a48efcfe7790%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9En%2Bz1Bax85M%2B-pzMUKnW5aNrBWOHxPr92_MnzqdPVOg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Backup and Restore an index

[Mark Walkom]

Why not just use an alias, then you don't have to worry about conflicting
names.
Otherwise I have been having luck using LS 1.5 to reindex -
https://gist.github.com/markwalkom/8a7201e3f6ea4354ae06


PS - We're moving to https://discuss.elastic.co/, please join us there for
any future discussions!

On 29 May 2015 at 03:09, Sagar Shah  wrote:

> Hello everyone,
> I am using elastic search 1.4.2 and trying to work out a way on back up
> and restore operation of an index.
>
> Objective:
> An index is already present on a system with some mapping. During system
> upgrade, mapping needs to be updated, which is not possible easily in
> elastic search without reindexing.
>
> So here's one path that I have chosen.
> - Take backup of existing index on file system
> - Delete existing index
> - Create an index with same name but with new mapping
> - Restore the (backed up) index. - Issue: When restoring index here, it
> also overwrites mapping to previous state, the way it was defined on
> original index. Is there a way to restore only the data and not
> settings/mapping. Or is there a way to restore index explicitly with new
> mappings?
>
> Other option that I see is to create restore into a new index and then
> perform bulk indexing on original index name. But that seems too much to me
> just to perform data backup and restore.
>
> Index is expected to have about 80 millions of records
>
> Is there a way to ease this process?
>
> Appreciate your inputs & suggestions
>
>
> Regards,
> Sagar Shah
>
> --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/77d67425-4ed8-403a-b9ff-9410fc02e1f8%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-od439iz3tgNrSq0HP61%3DZEvDKTnoTkXhvTsFJ5BmZXg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Problem occured with same name of cluster in different machines

[Mark Walkom]

This is why -
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery-zen.html#multicast

Your best option is to use
https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-allocation.html#shard-allocation-filtering
and then move indices to their correct node, then shutdown one cluster and
change the cluster name.

PS - We're moving to https://discuss.elastic.co/, please join us there for
any future discussions!

On 29 May 2015 at 03:12, mehmet özer  wrote:

> Hello,
>
> I have a weird issue today and it gives me lots of problem. I had two
> cluster different but unfortunatelly I gave them the same names by mistake;
>
> lets say;
> test_cluster
>
>- both cluster have had totaly different machines ( it was working
>well its been one month)
>- today I added a machine in one of the cluster and I saw that all the
>machines were in the same cluster physically (it was not the case before
>adding the machine)
>- I have chacked the unicast list; there were no machine in common
>
> So someone can explain me how I had this problem and how I can seperate
> them ?
>
> Thank you
>
> --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/a3b29cb4-360f-4fea-9514-2dd90d25a682%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-q3V5xDDPg8HXC0dSZ3DoWxRTWjZF90Z-w4Ag3mC2vTA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Where is Elasticsearch storing data

[John Adams]

Thank you, I will check it out.



On Wednesday, May 27, 2015 at 8:27:23 AM UTC-7, David Pilato wrote:
>
> by default in path.data dir.
> In zip or tar.gz distribution, it’s in "data" dir
>
>
> -- 
> *David Pilato* - Developer | Evangelist 
> *elastic.co *
> @dadoonet  | @elasticsearchfr 
>  | @scrutmydocs 
> 
>
>
>
>
>  
> Le 27 mai 2015 à 17:25, John Adams > a 
> écrit :
>
> I cannot find any info on where Elasticsearch is storing it's data.
> I would like to manage this potentially valuable information.
> Where, how big, how to back it up, restore it, etc.
> Then on to clusters, etc..
>
> Thanks,
>  John
>
> -- 
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> --- 
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearc...@googlegroups.com .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/4893b923-dbf2-45c3-a0e3-63c6d0b37b1c%40googlegroups.com
>  
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/81307403-978f-45f2-972f-07595b9df337%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Filtered Nested Filter with Multiple condition Returning Bad Request

[Yusbel Garcia]

I'm having parse error when posting a filtered nested object during search, 
bellow my posting data and the indexed data.

Data posted:

{
  "from": 0,
  "size": 10,
  "query": {
"filtered": {
  "query": {
"multi_match": {
  "query": "ro",
  "fields": [
"LegalName^70",
"LegalName.raw^70",
"Alias^15",
"RelationshipName^12",
"Address^10",
"NaicsCode^1",
"Phone^10",
"LesseeNumber^1",
"FullName^40",
"FirstName^40",
"LastName^40",
"FullName.raw^50",
"Relationship^15",
"DobYear^5"
  ],
  "tie_breaker": 0.3,
  "operator": "or",
  "type": "best_fields"
}
  },
  *"filter": {
"nested": {
  "path": "SecurityScope",
  "filter": {
"bool": {
  "must": [
{
  "term": {
"SecurityScope.FunderId": "1"
  }
}
  ],
  "should": [
{
  "term": {
"ContentStatus": "1"
  }
}
  ]
}
  }
}
  }
}*
  },
  "highlight": {
"pre_tags": [
  ""
],
"post_tags": [
  "<\/em>"
],
"tags_schema": "styled",
"fragment_size": "150",
"number_of_fragments": "3",
"order": "score",
"fields": {
  "LegalName": {

  },
  "LegalName.raw": {

  },
  "Alias": {

  },
  "RelationshipName": {

  },
  "Address": {

  },
  "NaicsCode": {

  },
  "Phone": {

  },
  "LesseeNumber": {

  },
  "FullName": {

  },
  "FirstName": {

  },
  "LastName": {

  },
  "FullName.raw": {

  },
  "Relationship": {

  },
  "DobYear": {

  }
}
  }
}



Data Indexed:


{
"_index": "intellisearch",
"_type": "business",
"_id": "33009",
"_score": 1,
"_source": {
   "Identifier": "33009",
   "ContentStatus": 1,
   "Links": null,
   "ApplicantId": 33009,
   "ApplicantTypeCode": "COM",
   "LegalName": "INTEGRATED INDUSTRIAL SUPPLY INC",
   "Alias": "INTEGRATED INDUSTRIAL SUPPLY INC",
   "BusinessName": "INTEGRATED INDUSTRIAL SUPPLY INC",
   "OperationName": "",
   "RelationshipName": "",
   "Address": "1337 W BETTERAVIA RD SANTA MARIA CALIFORNIA USA",
   "AddressLine1": "1337 W BETTERAVIA RD",
   "AddressLine2": "",
   "AddressLine3": "",
   "City": "SANTA MARIA",
   "Province": "CALIFORNIA",
   "Country": "USA",
   "NaicsCode": "0",
   "NaicsDescription": "N/A",
   "Phone": "8059258468",
   "LesseeNumber": "",
   "Score": 0,
   "BusinessStartDateYear": "1900",
   "ContentDocType": 2,
   "Extended": null,
   "SecurityScope": {
  "FunderId": "1"
   }
}



Can you give me a hint?



Thank you,

Yusbel





-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/0b5abfb8-2807-44b7-9779-d00aa5c8df3b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elastic search synonym match involving numeric characters

[Siva Shanmuga Subramanian Murugan]

Woow this fixed the issue. Thanks a lot for your help

Regards
Siva

On Thursday, May 28, 2015 at 1:00:04 AM UTC-7, xinm...@163.com wrote:
>
> I think your filters' order may swap like this：
>
> "analysis": {
>   "analyzer": {
> "mysynonym": {
>   "tokenizer": "standard",
>   "filter": [
> "mysynonym","standard","lowercase", "stop"
>   ],
>   "ignore_case": true
> }
>   }
>
> That's because filters work like assembly line, if your first filter is 
> standard, your "2500 HD" will split to "2500" and "HD" before go through 
> other filters. So your mysynonym do not work.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *//before swap filters' order$ curl -POST 
> "http://myES/my_test/_analyze?pretty&analyzer=mysynonym 
> " -d "2500 HD"{  
> "tokens" : [ {"token" : "2500","start_offset" : 0,"end_offset" 
> : 4,"type" : "","position" : 1  }, {"token" : "hd",
> "start_offset" : 5,"end_offset" : 7,"type" : "",
> "position" : 2  } ]}//after$ curl -POST 
> "http://myES/my_test/_analyze?pretty&analyzer=mysynonym 
> " -d "2500 HD"{  
> "tokens" : [ {"token" : "2500hd","start_offset" : 0,
> "end_offset" : 7,"type" : "SYNONYM","position" : 1  } ]}*
>
>
> Sincerely hope this may helpful to you.
>
>
>
> 在 2015年5月28日星期四 UTC+8上午11:20:51，Siva Shanmuga Subramanian Murugan写道：
>>
>> I have documents indexed in elastic cluster with the below mapping. 
>> basically i have a field named model which holds car model names like 
>> "Silverado 2500HD", "Silverado 1500HD" "LX 350" etc etc.
>>
>> POST /location-test-no-boost {
>> "settings":{
>> "analysis":{
>> "analyzer":{
>> "mysynonym":{
>> "tokenizer":"standard",
>> "filter":[
>> "standard","lowercase","stop","mysynonym"
>> ],
>> "ignore_case":true
>> }
>> },
>> "filter":{
>> "mysynonym":{
>> "type":"synonym",
>> "synonyms": [
>> "2500 HD=>2500HD",
>> "chevy silverado=>Silverado"
>> ]
>> }
>> }
>> }
>> },
>> "mappings":{
>> "vehicles":{
>> "properties":{
>> "id":{
>> "type":"long",
>> "ignore_malformed":true
>> },
>> "model":{
>> "type":"String",
>> "index_analyzer": "standard",
>> "search_analyzer":"mysynonym"
>> }
>> }
>> }
>> }
>> }
>>
>> The sample document content is
>>
>> POST /location-test-no-boost/vehicles/10
>> {
>>   "model" : "Silverado 2500HD"
>> }
>>
>> When i tried to search with Query string "Chevy sivlerado", the synonym 
>> matches perfectly to Silverado and gives back the result, on the contrary 
>> when i tried to search via query string "2500 HD" it is returning 0 
>> results. I tried different combination on the Synonym involving number and 
>> found that elastic search synonym mapper does not support numbers is this 
>> correct?
>>
>> is there any way i can make some mapping when user searches for "2500 
>> HD", i can map the query to "2500HD"
>>
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/792b252c-7a46-49fd-9266-5b811d00b103%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

es search optimizing question

[Jay Danielian]

I know its dangerous to get general answers when talking about performance, 
as the answer usually is "it depends". But I am going to try anyway :) My 
question is as a general rule of thumb is it better to have a list of items 
in an array stored and the query only has to issue a single matching term? 
Or store a single value per document and create various terms in an array 
passing in those generated terms for the query?

My example use case is this. I am trying to find contacts by name and 
email. Emails usually fall into several common patterns (first.last@domain, 
first_last@domain, firstinitial_last@domain, etc), so I want to be able to 
search against all of those possible combinations in trying to find this 
contact in our index. The queries are all filter terms, no wildcard, etc. 
The fields are all not_analyzed, so its basically an exact term match that 
I am looking for. So, I can either store the extra possible combinations in 
the document, and have the query syntax only need to pass in one term (as 
the field stored is an array). Or I can pass in the multiple combinations 
in a term array in the query syntax, and search against the single email we 
have stored in the index.

I know its never a perfect answer, but even general rule of thumb response 
from someone with deep internal knowledge of lucene/ES would be 
appreciated. 

Thanks!

J

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/ef1b1d61-96b6-4dcd-a658-1385aa3f380f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Elasticsearch Report Plugin to Generate Excel Reports

[Raghavendar T S]

Hi Friends

I have developed a plugin for Elasticsearch to generate Excel reports and 
hosted the same in GitHub. The plugin exposes REST interface where simple 
input configuration can save the report to the system or E-Mail to the 
required recipients. Check it out and give your valuable comments.

Plugin URL: https://github.com/raghavendar-ts/Elasticsearch-Report-Plugin 



Thank you
Raghavendar T S

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/145a0f82-165f-4020-8083-50bebb73313e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Problem occured with same name of cluster in different machines

[mehmet özer]

Hello,

I have a weird issue today and it gives me lots of problem. I had two 
cluster different but unfortunatelly I gave them the same names by mistake;

lets say;
test_cluster

   - both cluster have had totaly different machines ( it was working well 
   its been one month)
   - today I added a machine in one of the cluster and I saw that all the 
   machines were in the same cluster physically (it was not the case before 
   adding the machine)
   - I have chacked the unicast list; there were no machine in common

So someone can explain me how I had this problem and how I can seperate 
them ?

Thank you

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/a3b29cb4-360f-4fea-9514-2dd90d25a682%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Backup and Restore an index

[Sagar Shah]

Hello everyone,
I am using elastic search 1.4.2 and trying to work out a way on back up and 
restore operation of an index.

Objective: 
An index is already present on a system with some mapping. During system 
upgrade, mapping needs to be updated, which is not possible easily in 
elastic search without reindexing.

So here's one path that I have chosen.
- Take backup of existing index on file system
- Delete existing index
- Create an index with same name but with new mapping 
- Restore the (backed up) index. - Issue: When restoring index here, it 
also overwrites mapping to previous state, the way it was defined on 
original index. Is there a way to restore only the data and not 
settings/mapping. Or is there a way to restore index explicitly with new 
mappings?

Other option that I see is to create restore into a new index and then 
perform bulk indexing on original index name. But that seems too much to me 
just to perform data backup and restore. 

Index is expected to have about 80 millions of records

Is there a way to ease this process?

Appreciate your inputs & suggestions


Regards,
Sagar Shah

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/77d67425-4ed8-403a-b9ff-9410fc02e1f8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Not able to search Upper case values

[vikas gopal]

Thank you Shoeb,

Actually the problem was I was using Not_analyzed along with doc_value : 
true, which causes the problem . Thanks to Mark , problem has been resolved 
now. Thank you again for your suggestion and support.

On Thursday, May 28, 2015 at 6:13:31 PM UTC+5:30, shoeb khan wrote:
>
> Use any analyzer to you input query string, then use analysed string to 
> process query. you will get the appropriate results. 
> I faced the same problem , i used standard analyser for my query string 
> and 
> problem got resolved. 
>
>
>
> -- 
> View this message in context: 
> http://elasticsearch-users.115913.n3.nabble.com/Not-able-to-search-Upper-case-values-tp4074950p4074966.html
>  
> Sent from the Elasticsearch Users mailing list archive at Nabble.com. 
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/a288c1ed-8ef3-459b-bed4-311b184e4e62%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Where is Elasticsearch storing data

[Jim Malone]

I similarly can't find where the data is stored.  We have it running with 
ES-yarn though.  This article says that the data is only stored in 
container storage by default (we haven't changed any settings):

https://www.elastic.co/guide/en/elasticsearch/hadoop/master/ey-setup.html

However, the data is definitely persisting somehow between restarts so it 
has to be stored somewhere.  We just can't find where.  There is no 
"path.data" dir and also there's no "/var/lib/elasticsearch/data" dir like 
I've seen on other posts. And we can't find it in hdfs either.

So can anyone please point me to where we might be able to find the data? 
If it helps, we are loading indexes from both hive and from logstash 
directly.

Thanks,
Jim

On Wednesday, May 27, 2015 at 6:56:54 PM UTC-4, Mark Walkom wrote:
>
> Check out 
> https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-dir-layout.html
>
>
> PS - We're moving to https://discuss.elastic.co/, please join us there 
> for any future discussions!
>
> On 28 May 2015 at 01:27, David Pilato > 
> wrote:
>
>> by default in path.data dir.
>> In zip or tar.gz distribution, it’s in "data" dir
>>
>>
>> -- 
>> *David Pilato* - Developer | Evangelist 
>> *elastic.co *
>> @dadoonet  | @elasticsearchfr 
>>  | @scrutmydocs 
>> 
>>
>>
>>
>>
>>  
>> Le 27 mai 2015 à 17:25, John Adams > a 
>> écrit :
>>
>> I cannot find any info on where Elasticsearch is storing it's data.
>> I would like to manage this potentially valuable information.
>> Where, how big, how to back it up, restore it, etc.
>> Then on to clusters, etc..
>>
>> Thanks,
>>  John
>>
>> -- 
>> Please update your bookmarks! We have moved to 
>> https://discuss.elastic.co/
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/4893b923-dbf2-45c3-a0e3-63c6d0b37b1c%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>>  -- 
>> Please update your bookmarks! We have moved to 
>> https://discuss.elastic.co/
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/4836BD52-F0A0-4507-BE1D-14E3CA4B0FD7%40pilato.fr
>>  
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/bc0ad06d-21b3-4ca7-aade-876f15b379a3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Not able to search Upper case values

[vikas gopal]

Wow that works thank you Mark

On Thursday, May 28, 2015 at 3:56:17 PM UTC+5:30, Mark Walkom wrote:
>
> You cannot set doc values on an analysed field.
>
>
> PS - We're moving to https://discuss.elastic.co/, please join us there 
> for any future discussions!
>
> On 28 May 2015 at 20:03, vikas gopal > 
> wrote:
>
>> Thanks Allan,
>>
>> I have changed it to ""Customer": {"type": "string","index" : 
>> "analyzed","doc_values" : true ,"analyzer":"analyzer_keyword"}," , but with 
>> this setting LS is not able to create an index.
>>
>>
>> On Thursday, May 28, 2015 at 2:40:14 PM UTC+5:30, Allan Mitchell wrote:
>>>
>>> Hi
>>>
>>>
>>> "Customer" is set to be "not_analyzed" so won't go through your analyser.
>>>
>>> not_analyzedIndex this field, so it is searchable, but index the value 
>>> exactly as specified. Do not analyze it.
>>>
>>> On 28 May 2015 at 09:45, vikas gopal  wrote:
>>>
 Hi Experts,

 In my log I have a field "Customer". It has all the values in capital 
 latter like "VIKAS". Now when I fire query in Kibana like Customer:"vikas" 
 or Customer:"VIK*" I got nothing . It works only when I do exact search 
 like Customer:"VIKAS".

 I have following question

 1) How I can search value with wildcard, something like name start with 
 vi OR VI ?

 My Template is like 

 {
  "template_vg":{
 "template" : "vg*",
 "settings" : {
 "number_of_shards" : 5,
 "index.cache.field.type" : "soft",
 "index.refresh_interval" : "5s",
 "index.store.compress.stored" : true,
 "index.query.default_field" : "message",
 "index.routing.allocation.total_shards_per_node" : 5,
 "index":{
 "analysis":{
 "analyzer":{
   "analyzer_keyword":{
  "type": "custom",
  "tokenizer":"keyword",
   "filter":"lowercase"
   }
}
 }
  }
 },
 "mappings" : {
 "_default_" : {
 "_all" : {"enabled" : false},
 "properties" : {
 "Order_ID": {"type": "string","index" :"not_analyzed","doc_values" : 
 true},
 "Customer": {"type": "string","index" : "not_analyzed","doc_values" : 
 true ,"analyzer":"analyzer_keyword"},
 "Ordered_On": {"type": "string","index" : "not_analyzed","doc_values" : 
 true },
 "Required_By": {"type": "string","index" : "not_analyzed","doc_values" 
 : true },
 "Freight": {"type": "string","index" : "not_analyzed","doc_values" : 
 true },
 "Shipped_On": {"type": "string","index" : "not_analyzed","doc_values" : 
 true },
 "Country": {"type": "string","index" : "not_analyzed","doc_values" : 
 true },
 "Post_Code": {"type": "string","index" : "not_analyzed","doc_values" : 
 true }
  }
 }
 }
 }
 }


 I read almost everything and got an idea that I need to use analyzer 
 with Lowercase filter which will convert my values to lowercase then I can 
 apply wildcard.But with the above analyzer for Customer I still got 
 nothing 
 . Can someone help me to understand what I am doing wrong . 

 -- 
 Please update your bookmarks! We have moved to 
 https://discuss.elastic.co/
 --- 
 You received this message because you are subscribed to the Google 
 Groups "elasticsearch" group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to elasticsearc...@googlegroups.com.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/elasticsearch/eb6bda79-428d-4fdc-b478-65f07905fe89%40googlegroups.com
  
 
 .
 For more options, visit https://groups.google.com/d/optout.

>>>
>>>  -- 
>> Please update your bookmarks! We have moved to 
>> https://discuss.elastic.co/
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/46c4f481-6876-463b-9c7e-bfb22e4acdee%40googlegroups.com
>>  
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receivi

How do I index data from logs in a specific format in elastic search?

[simarpreet . singh]

 I receive data in this format: Name Date Details. How do I index it into 
indices of Elastic Search automatically ?

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/38bb9c44-b428-4111-8115-a48efcfe7790%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elasticsearch: 2-node cluster with failover

[prakhar]

Sorry to bother you @Mark, but this term *appropriate heap size* always
haunts me. Suppose, I have 2 virtual machines (4 core, 8 gb RAM each).
According to elasticsearch's documentation, half of RAM should be allocated
to elasticsearch. So, this is what I came up with:

Node1: Elasticsearch 1.5.2 *Master-Data* node with 4 gb HEAP (*server1*)
Node2: Elasticsearch 1.5.2 *Master-Only*(no data) node with 1 gb HEAP
(*server1*)
Node3: Elasticsearch 1.5.2 *Master-Data* node with 4 gb HEAP (*server2*)

Do you think this is a sane strategy? Moreover, if I make *Master-Only* node
(Node2) invisible to load balancer i.e. I simply won't include it at all,
will it still require 1 gb HEAP?
Please forgive me if I am asking something too naive.



--
View this message in context: 
http://elasticsearch-users.115913.n3.nabble.com/Elasticsearch-2-node-cluster-with-failover-tp4074949p4074968.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1432818693390-4074968.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.

Snapshot files are created every time even no change in index.

[shoebalig]

 I have a cron-job runs daily. There are chances that there is no
modification or changes found in original index. however cron-job create
snapshot file again on its scheduled time. Due to which number of snapshot
files is too large. which leads restoration of index, takes a lot of time.
Is there anyway I should create snapshot file only when there is a
difference in index data?
What if I retain only last 5 or last 10 snapshot? Will this cause any data
lose anyway? And on restore whole index will be restored?

Please suggest.



--
View this message in context: 
http://elasticsearch-users.115913.n3.nabble.com/Snapshot-files-are-created-every-time-even-no-change-in-index-tp4074965.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1432811614768-4074965.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.

Re: Not able to search Upper case values

[shoebalig]

Use any analyzer to you input query string, then use analysed string to
process query. you will get the appropriate results.
I faced the same problem , i used standard analyser for my query string and
problem got resolved.



--
View this message in context: 
http://elasticsearch-users.115913.n3.nabble.com/Not-able-to-search-Upper-case-values-tp4074950p4074966.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1432811954546-4074966.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.

Re: Forums Are Now Live at http://discuss.elastic.co

[Leslie Hawthorn]

Apologies, I just saw this message. Are you still having access issues?

Also, it is much faster to report trouble in #elasticsearch on Freenode if
you are an IRC user. :)

On Thu, May 28, 2015 at 4:55 AM,  wrote:

>
> Can not visit https://discuss.elastic.co/ today.
>
> The website is loading all the time.
>
>
> 在 2015年5月5日星期二 UTC+8上午12:12:19，leslie.hawthorn写道：
>
>> Hello everyone,
>>
>> We took in feedback on moving to a Discourse based forum for about a
>> month, and it sounds like most of the folks who thought it might not be
>> optimal were people who preferred to interact with mailing lists instead of
>> forums.
>>
>> We're pretty confident the email functionality of Discourse will work
>> well for our community, so we've gone ahead and rolled out the forums. You
>> can visit them now and sign up for a user account at
>> http://discuss.elastic.co. Registration is one time only and you can do
>> so with any email address or authorizing via Facebook, GitHub, Google
>> Accounts or Twitter.
>>
>> Once you've created your account, you can set up your preferences to
>> receive email as often or as rarely as you would like.
>>
>> For those who'd prefer to interact solely via email, you can start doing
>> so as soon as you've set these preferences. You can find full documentation
>> on interacting with the forums solely via email here.[0]
>>
>> Some anticipated FAQs:
>>
>> 1) Should I ask for help using the forums on this mailing list?
>> You're welcome to ask for help on this list, on IRC (#elasticsearch,
>> #logstash or #kibana on Freenode) or within the forum in the Meta Elastic
>> category.
>>
>> 2) Are you going to stop answering questions here?
>> No, we're going to leave the mailing list active for at least 30 days so
>> our entire community can kick the tires. Quoting our internal company FAQ
>> for this very question:
>> "If our community members find Discourse to be suboptimal, we may make
>> the choice to use the forums only for certain tasks and preserve the
>> existing mailing lists."
>>
>> However, our goal is to provide a single source of truth for information
>> on our products, so we're hoping the Discourse based forums really work for
>> people. Our employees will be answering your questions with a pointer to a
>> forum thread to encourage people to actually interact with the new
>> resource, too.
>>
>> 3) So let's say that the forums work for people. What's the plan?
>> We'll set these mailing lists to read-only ~30 days from now (June 1,
>> 2015). We're still working with the creators of Discourse to do a full
>> import of our mailing list archives, so once that task is complete the
>> read-only archives will still be preserved, but you can search through all
>> that collective knowledge in one place at http://discuss.elastic.co
>>
>> 4) Is there an FAQ for using the forums?
>> We feel like the user interface for Discourse is pretty intuitive, but we
>> have a short Notes on Using This Forum document [1] available to help you
>> get started.
>>
>> 5) Where do I direct praise and pain points for the forums?
>> Feedback on the forums should be posted to the Meta Elastic category. [2]
>> You can update it via email using address discuss-meta [at] elastic [dot]
>> co once you've set up your user account.
>>
>> 6) I have another question that you have not answered. What should I do?
>> Please post a note in the Meta Elastic category, ping in IRC (I'm lh on
>> Freenode, though anyone with ChanOps/Voice in #elasticsearch, #logstash and
>> #kibana can help) or post a note in this thread.
>>
>>
>> [0] -
>> https://discuss.elastic.co/t/email-only-interaction-with-the-forums/106
>> [1] - https://discuss.elastic.co/t/notes-on-using-these-forums/118
>> [2] - https://discuss.elastic.co/c/meta
>>
>> Cheers,
>> LH
>>
>> Leslie Hawthorn
>> Director of Developer Relations
>> http://elastic.co
>>
>> Other Places to Find Me:
>> Freenode: lh
>> Twitter: @lhawthorn
>>
>  --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/68287801-937b-4deb-9f63-fc50f6f74774%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Leslie Hawthorn
Director of Developer Relations
http://elastic.co

Other Places to Find Me:
Freenode: lh
Twitter: @lhawthorn
Skype: mebelh
Voice: +31 20 794 7300

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this gr

Re: Elasticsearch: 2-node cluster with failover

[Magnus Bäck]

On Thursday, May 28, 2015 at 12:25 CEST,
 prakhar  wrote:

> If I add one *master only* node on one of the two existing servers (2
> nodes on the same physical server), with *HEAP*, lets say, /100mb/;
> will that work? Or I have to add another machine for that?

That should work, but it obviously means that if the machine with two
Elasticsearch nodes is taken out your cluster will be inoperable (but
it will survive if the one-node machine dies).

-- 
Magnus Bäck| Software Engineer, Development Tools
magnus.b...@sonymobile.com | Sony Mobile Communications

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/20150528110806.GA25289%40seldlx20533.corpusers.net.
For more options, visit https://groups.google.com/d/optout.

Re: Elasticsearch: 2-node cluster with failover

[Mark Walkom]

That's not enough heap, you want 1GB minimum.


PS - We're moving to https://discuss.elastic.co/, please join us there for
any future discussions!

On 28 May 2015 at 20:25, prakhar  wrote:

> Thanks for your reply.
>
> If I add one *master only* node on one of the two existing servers (2 nodes
> on the same physical server), with *HEAP*, lets say, /100mb/; will that
> work? Or I have to add another machine for that?
>
>
>
> --
> View this message in context:
> http://elasticsearch-users.115913.n3.nabble.com/Elasticsearch-2-node-cluster-with-failover-tp4074949p4074961.html
> Sent from the Elasticsearch Users mailing list archive at Nabble.com.
>
> --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/1432808731045-4074961.post%40n3.nabble.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8ZLP5A1k80t-D61CKLZNZ%3D8PdP4w8tsA7_O4eBWdYq4A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elasticsearch: 2-node cluster with failover

[prakhar]

Thanks for your reply.

If I add one *master only* node on one of the two existing servers (2 nodes
on the same physical server), with *HEAP*, lets say, /100mb/; will that
work? Or I have to add another machine for that?



--
View this message in context: 
http://elasticsearch-users.115913.n3.nabble.com/Elasticsearch-2-node-cluster-with-failover-tp4074949p4074961.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1432808731045-4074961.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.

Re: Not able to search Upper case values

[Mark Walkom]

You cannot set doc values on an analysed field.


PS - We're moving to https://discuss.elastic.co/, please join us there for
any future discussions!

On 28 May 2015 at 20:03, vikas gopal  wrote:

> Thanks Allan,
>
> I have changed it to ""Customer": {"type": "string","index" :
> "analyzed","doc_values" : true ,"analyzer":"analyzer_keyword"}," , but with
> this setting LS is not able to create an index.
>
>
> On Thursday, May 28, 2015 at 2:40:14 PM UTC+5:30, Allan Mitchell wrote:
>>
>> Hi
>>
>>
>> "Customer" is set to be "not_analyzed" so won't go through your analyser.
>>
>> not_analyzedIndex this field, so it is searchable, but index the value
>> exactly as specified. Do not analyze it.
>>
>> On 28 May 2015 at 09:45, vikas gopal  wrote:
>>
>>> Hi Experts,
>>>
>>> In my log I have a field "Customer". It has all the values in capital
>>> latter like "VIKAS". Now when I fire query in Kibana like Customer:"vikas"
>>> or Customer:"VIK*" I got nothing . It works only when I do exact search
>>> like Customer:"VIKAS".
>>>
>>> I have following question
>>>
>>> 1) How I can search value with wildcard, something like name start with
>>> vi OR VI ?
>>>
>>> My Template is like
>>>
>>> {
>>>  "template_vg":{
>>> "template" : "vg*",
>>> "settings" : {
>>> "number_of_shards" : 5,
>>> "index.cache.field.type" : "soft",
>>> "index.refresh_interval" : "5s",
>>> "index.store.compress.stored" : true,
>>> "index.query.default_field" : "message",
>>> "index.routing.allocation.total_shards_per_node" : 5,
>>> "index":{
>>> "analysis":{
>>> "analyzer":{
>>>   "analyzer_keyword":{
>>>  "type": "custom",
>>>  "tokenizer":"keyword",
>>>   "filter":"lowercase"
>>>   }
>>>}
>>> }
>>>  }
>>> },
>>> "mappings" : {
>>> "_default_" : {
>>> "_all" : {"enabled" : false},
>>> "properties" : {
>>> "Order_ID": {"type": "string","index" :"not_analyzed","doc_values" :
>>> true},
>>> "Customer": {"type": "string","index" : "not_analyzed","doc_values" :
>>> true ,"analyzer":"analyzer_keyword"},
>>> "Ordered_On": {"type": "string","index" : "not_analyzed","doc_values" :
>>> true },
>>> "Required_By": {"type": "string","index" : "not_analyzed","doc_values" :
>>> true },
>>> "Freight": {"type": "string","index" : "not_analyzed","doc_values" :
>>> true },
>>> "Shipped_On": {"type": "string","index" : "not_analyzed","doc_values" :
>>> true },
>>> "Country": {"type": "string","index" : "not_analyzed","doc_values" :
>>> true },
>>> "Post_Code": {"type": "string","index" : "not_analyzed","doc_values" :
>>> true }
>>>  }
>>> }
>>> }
>>> }
>>> }
>>>
>>>
>>> I read almost everything and got an idea that I need to use analyzer
>>> with Lowercase filter which will convert my values to lowercase then I can
>>> apply wildcard.But with the above analyzer for Customer I still got nothing
>>> . Can someone help me to understand what I am doing wrong .
>>>
>>> --
>>> Please update your bookmarks! We have moved to
>>> https://discuss.elastic.co/
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearc...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/eb6bda79-428d-4fdc-b478-65f07905fe89%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/46c4f481-6876-463b-9c7e-bfb22e4acdee%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_HfCJqEEf1QrNV48EFT6L48SL0Y0nJ-hSkxMbJYKmMow%40mail.gmail.com.
For more options, visit https://groups.goo

Re: Not able to search Upper case values

[Allan Mitchell]

Hi

That is correct.

Doc values can be enabled for numeric, date, Boolean, binary, and geo-point
fields, and for not_analyzed string fields.


They
do not currently work with analyzed string fields. Doc values are enabled
per field in the field mapping, which means that you can combine in-memory
fielddata with doc values:



Your error message will have been similar to

{
   "error": "MapperParsingException[mapping [mytype]]; nested:
MapperParsingException[Field [myfield] cannot be analyzed and have doc
values]; ",
   "status": 400
}





On 28 May 2015 at 11:03, vikas gopal  wrote:

> Thanks Allan,
>
> I have changed it to ""Customer": {"type": "string","index" :
> "analyzed","doc_values" : true ,"analyzer":"analyzer_keyword"}," , but with
> this setting LS is not able to create an index.
>
>
> On Thursday, May 28, 2015 at 2:40:14 PM UTC+5:30, Allan Mitchell wrote:
>>
>> Hi
>>
>>
>> "Customer" is set to be "not_analyzed" so won't go through your analyser.
>>
>> not_analyzedIndex this field, so it is searchable, but index the value
>> exactly as specified. Do not analyze it.
>>
>> On 28 May 2015 at 09:45, vikas gopal  wrote:
>>
>>> Hi Experts,
>>>
>>> In my log I have a field "Customer". It has all the values in capital
>>> latter like "VIKAS". Now when I fire query in Kibana like Customer:"vikas"
>>> or Customer:"VIK*" I got nothing . It works only when I do exact search
>>> like Customer:"VIKAS".
>>>
>>> I have following question
>>>
>>> 1) How I can search value with wildcard, something like name start with
>>> vi OR VI ?
>>>
>>> My Template is like
>>>
>>> {
>>>  "template_vg":{
>>> "template" : "vg*",
>>> "settings" : {
>>> "number_of_shards" : 5,
>>> "index.cache.field.type" : "soft",
>>> "index.refresh_interval" : "5s",
>>> "index.store.compress.stored" : true,
>>> "index.query.default_field" : "message",
>>> "index.routing.allocation.total_shards_per_node" : 5,
>>> "index":{
>>> "analysis":{
>>> "analyzer":{
>>>   "analyzer_keyword":{
>>>  "type": "custom",
>>>  "tokenizer":"keyword",
>>>   "filter":"lowercase"
>>>   }
>>>}
>>> }
>>>  }
>>> },
>>> "mappings" : {
>>> "_default_" : {
>>> "_all" : {"enabled" : false},
>>> "properties" : {
>>> "Order_ID": {"type": "string","index" :"not_analyzed","doc_values" :
>>> true},
>>> "Customer": {"type": "string","index" : "not_analyzed","doc_values" :
>>> true ,"analyzer":"analyzer_keyword"},
>>> "Ordered_On": {"type": "string","index" : "not_analyzed","doc_values" :
>>> true },
>>> "Required_By": {"type": "string","index" : "not_analyzed","doc_values" :
>>> true },
>>> "Freight": {"type": "string","index" : "not_analyzed","doc_values" :
>>> true },
>>> "Shipped_On": {"type": "string","index" : "not_analyzed","doc_values" :
>>> true },
>>> "Country": {"type": "string","index" : "not_analyzed","doc_values" :
>>> true },
>>> "Post_Code": {"type": "string","index" : "not_analyzed","doc_values" :
>>> true }
>>>  }
>>> }
>>> }
>>> }
>>> }
>>>
>>>
>>> I read almost everything and got an idea that I need to use analyzer
>>> with Lowercase filter which will convert my values to lowercase then I can
>>> apply wildcard.But with the above analyzer for Customer I still got nothing
>>> . Can someone help me to understand what I am doing wrong .
>>>
>>> --
>>> Please update your bookmarks! We have moved to
>>> https://discuss.elastic.co/
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearc...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/eb6bda79-428d-4fdc-b478-65f07905fe89%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/46c4f481-6876-463b-9c7e-bfb22e4acdee%40googlegroups.com
> 

Re: Not able to search Upper case values

[vikas gopal]

Thanks Allan,

I have changed it to ""Customer": {"type": "string","index" : 
"analyzed","doc_values" : true ,"analyzer":"analyzer_keyword"}," , but with 
this setting LS is not able to create an index.


On Thursday, May 28, 2015 at 2:40:14 PM UTC+5:30, Allan Mitchell wrote:
>
> Hi
>
>
> "Customer" is set to be "not_analyzed" so won't go through your analyser.
>
> not_analyzedIndex this field, so it is searchable, but index the value 
> exactly as specified. Do not analyze it.
>
> On 28 May 2015 at 09:45, vikas gopal > 
> wrote:
>
>> Hi Experts,
>>
>> In my log I have a field "Customer". It has all the values in capital 
>> latter like "VIKAS". Now when I fire query in Kibana like Customer:"vikas" 
>> or Customer:"VIK*" I got nothing . It works only when I do exact search 
>> like Customer:"VIKAS".
>>
>> I have following question
>>
>> 1) How I can search value with wildcard, something like name start with 
>> vi OR VI ?
>>
>> My Template is like 
>>
>> {
>>  "template_vg":{
>> "template" : "vg*",
>> "settings" : {
>> "number_of_shards" : 5,
>> "index.cache.field.type" : "soft",
>> "index.refresh_interval" : "5s",
>> "index.store.compress.stored" : true,
>> "index.query.default_field" : "message",
>> "index.routing.allocation.total_shards_per_node" : 5,
>> "index":{
>> "analysis":{
>> "analyzer":{
>>   "analyzer_keyword":{
>>  "type": "custom",
>>  "tokenizer":"keyword",
>>   "filter":"lowercase"
>>   }
>>}
>> }
>>  }
>> },
>> "mappings" : {
>> "_default_" : {
>> "_all" : {"enabled" : false},
>> "properties" : {
>> "Order_ID": {"type": "string","index" :"not_analyzed","doc_values" : 
>> true},
>> "Customer": {"type": "string","index" : "not_analyzed","doc_values" : 
>> true ,"analyzer":"analyzer_keyword"},
>> "Ordered_On": {"type": "string","index" : "not_analyzed","doc_values" : 
>> true },
>> "Required_By": {"type": "string","index" : "not_analyzed","doc_values" : 
>> true },
>> "Freight": {"type": "string","index" : "not_analyzed","doc_values" : true 
>> },
>> "Shipped_On": {"type": "string","index" : "not_analyzed","doc_values" : 
>> true },
>> "Country": {"type": "string","index" : "not_analyzed","doc_values" : true 
>> },
>> "Post_Code": {"type": "string","index" : "not_analyzed","doc_values" : 
>> true }
>>  }
>> }
>> }
>> }
>> }
>>
>>
>> I read almost everything and got an idea that I need to use analyzer with 
>> Lowercase filter which will convert my values to lowercase then I can apply 
>> wildcard.But with the above analyzer for Customer I still got nothing . Can 
>> someone help me to understand what I am doing wrong . 
>>
>> -- 
>> Please update your bookmarks! We have moved to 
>> https://discuss.elastic.co/
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/eb6bda79-428d-4fdc-b478-65f07905fe89%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/46c4f481-6876-463b-9c7e-bfb22e4acdee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Why is no org.joda.time.Months including in elasticsearch-jar ?

[Sven Jörns]

Hi,
because org.joda.time is included in elasticsearch, I use these classes to 
program against elasticsearch results etc.
Now I am trying to use 
 
import org.elasticsearch.common.joda.time.Months;

(http://www.joda.org/joda-time/apidocs/index.html)

but there isn't this Class in jar file of elasticsearch.
Why?

And when I will use the original lib of joda, witch version is used in 
elasticsearch?
So I can be sure, that e.g. all Formatters are used in elasticsearch too.

thx
Sven 

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/46fa1513-c9e8-4f98-af3e-cb86e6f9ab1e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Considering scalability , is it right to keep a large number of primary shards at beginning？

[Mark Walkom]

You don't need to stop everything to reindex, leverage aliases and you can
do it live.

On 28 May 2015 at 18:57,  wrote:

>
> Thanks for your answer~
>
> May be when 10 servers extends to 100 servers it can work ,but I‘m afraid
> that if we do reindex to 100 servers  it may cost a long time and huge I/O
> resources.We need to stop the service for a long time and all the data need
> to transport from old index to new one.
>
> Is there any easier way to horizontal expansion？
>
> P.S. I cann’t visit  https://discuss.elastic.co/
>  today .It is blank all the time. (･ˇ_ˇ･)
>
>
> 在 2015年5月28日星期四 UTC+8下午3:05:13，Mark Walkom写道：
>>
>> You don't want 400 shards on 10 servers. You do want the ability to
>> reindex to allow you to reshard to deal with this issue.
>> Logstash 1.5 can do this very easily, see this example
>> .
>>
>> However you probably don't want an index with 200 shards irrespective,
>> you may want to take a look at your data structure and split things out.
>>
>>
>> PS - We're moving to https://discuss.elastic.co/, please join us there
>> for any future discussions!
>>
>>
>>
>> On 28 May 2015 at 12:46,  wrote:
>>
>>>
>>> Hi，
>>>
>>> I feel very confuse when deciding the number of primary shards at
>>> beginning.
>>>
>>> As we know the number of shards and replicas can be defined per index at
>>> the time the index is created. After the index is created, we may change
>>> the number of replicas dynamically anytime but we cannot change the number
>>> of primary shards after-the-fact. Our ES project may be run as trial
>>> version at beginning an it is only 10 machines in cluster. However, if the
>>> project run in production environment,  the machine magnitude must
>>> increase and there will have 200 or more machines.
>>>
>>> How can we decide shards number at beginning? Is it encourage to run 400
>>> or more shards in 10 machines, will it reduce the performance of
>>> cluster?
>>>
>>>
>>> Thank you for reading and look forward to your suggestions.
>>>
>>>
>>>  --
>>> Please update your bookmarks! We have moved to
>>> https://discuss.elastic.co/
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to elasticsearc...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/a6eb715a-6543-4882-8635-81a0d22ca1d7%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>  --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/65cd6e3e-97f1-416e-8f97-e34a8dd7671e%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X89RqPb2atD%2B7VLWoEmbP5kyAcs%3D_Vs_TmOozGSEZCBWA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Help please

[mariam diawara]

Hi,
sorry my english is very bad
I have a problem,
I made a dashboard with Kibana 4 with data from two tables, but when I
click on a particular visualization from a table, Kibana not display the
exact number of data to another visualization done with data in another
table
cordially,

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAPRdzkN%2BMaWTfRnyoyh1meq1cOEQRVsMewVCPcCCKgY%3Dq7ZOGw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Versioning the Index Mapping

[mariam diawara]

Hi,
sorry my english is very bad
I have a problem,
I made a dashboard with Kibana 4 with data from two tables, but when I
click on a particular visualization from a table, Kibana not display the
exact number of data to another visualization done with data in another
table
cordially,

2015-05-19 5:07 GMT+02:00 Allan Mitchell :

> Hi
>
> Isn't this a job for source control?  In there you should have a releases
> branch and the latest release should contain your mapping file. I don't
> think using ES as a poor man's source control system is a good idea.  If ES
> dies and you need to do a complete rebuild then you will need to get the
> scripts from somewhere and source control is that place.
>
>
> Allan Mitchell
>
>
>
> On 18 May 2015, at 22:32, Sagar Shah  wrote:
>
> Hello everyone,
> Is there a way to create a mapping in elastic search (version 1.4.2) with
> a version number. So that next time, we can verify if mapping needs to be
> updated using PUT request in elastic search or not based on version number?
>
> For example,
> I create a mapping A within an index and version it A.x. During next
> release, I perform a check to determine the version of existing index
> mapping and accordingly perform an update using PUT API of index mapping.
>
> Please suggest.
>
> Regards,
> Sagar Shah
>
> --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/15a1d4ca-272d-45b4-b549-e983e56f4a8b%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
>  --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CDAC93A8-C41C-45B0-ADD0-638C39B88C8D%40gmail.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAPRdzkNqcZqLQZV9WtCr0JuyT2Fp3kqXvW80eXAormSO%2BJdLNw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Kibana 4 Process Crash

[VV]

Thanks for the answer, I have opened question on discuss.elastic.co, Kibana 
doesn't log, elastic log are ok.

Il giorno mercoledì 27 maggio 2015 02:54:54 UTC+2, Mark Walkom ha scritto:
>
> What do the logs show?
>
>
> PS - We're moving to https://discuss.elastic.co/, please join us there 
> for any future discussions!
>
> On 26 May 2015 at 18:03, VV > wrote:
>
>> Hello everyone
>> I have a problem with Kibana 4, every moorning I open my browser on 
>> kibana page and I received the message "Waiting for elasticsearch is 
>> loading messages", the page don't get results, If I refresh the page Kibana 
>> Process go down.
>>
>> I had the following configuration on 2 servers:
>>
>> - 1 with Logstash that send data to elasticsearch 1.5.0
>> - 1 with Elasticsearch 1.5.0 and Kibana 4.0.2 
>>
>> Running on RHEL 5.5 x64.
>>
>> Could anyone help me?
>> Thanks in advance.
>>
>> VV
>>
>> -- 
>> Please update your bookmarks! We have moved to 
>> https://discuss.elastic.co/
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/3efda0c2-dced-488f-a40f-e0c0785cfd04%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/205ea189-7acf-42f0-9eca-3d3b7a33abc2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Not able to search Upper case values

[Allan Mitchell]

Hi


"Customer" is set to be "not_analyzed" so won't go through your analyser.

not_analyzedIndex this field, so it is searchable, but index the value
exactly as specified. Do not analyze it.

On 28 May 2015 at 09:45, vikas gopal  wrote:

> Hi Experts,
>
> In my log I have a field "Customer". It has all the values in capital
> latter like "VIKAS". Now when I fire query in Kibana like Customer:"vikas"
> or Customer:"VIK*" I got nothing . It works only when I do exact search
> like Customer:"VIKAS".
>
> I have following question
>
> 1) How I can search value with wildcard, something like name start with vi
> OR VI ?
>
> My Template is like
>
> {
>  "template_vg":{
> "template" : "vg*",
> "settings" : {
> "number_of_shards" : 5,
> "index.cache.field.type" : "soft",
> "index.refresh_interval" : "5s",
> "index.store.compress.stored" : true,
> "index.query.default_field" : "message",
> "index.routing.allocation.total_shards_per_node" : 5,
> "index":{
> "analysis":{
> "analyzer":{
>   "analyzer_keyword":{
>  "type": "custom",
>  "tokenizer":"keyword",
>   "filter":"lowercase"
>   }
>}
> }
>  }
> },
> "mappings" : {
> "_default_" : {
> "_all" : {"enabled" : false},
> "properties" : {
> "Order_ID": {"type": "string","index" :"not_analyzed","doc_values" : true},
> "Customer": {"type": "string","index" : "not_analyzed","doc_values" : true
> ,"analyzer":"analyzer_keyword"},
> "Ordered_On": {"type": "string","index" : "not_analyzed","doc_values" :
> true },
> "Required_By": {"type": "string","index" : "not_analyzed","doc_values" :
> true },
> "Freight": {"type": "string","index" : "not_analyzed","doc_values" : true
> },
> "Shipped_On": {"type": "string","index" : "not_analyzed","doc_values" :
> true },
> "Country": {"type": "string","index" : "not_analyzed","doc_values" : true
> },
> "Post_Code": {"type": "string","index" : "not_analyzed","doc_values" :
> true }
>  }
> }
> }
> }
> }
>
>
> I read almost everything and got an idea that I need to use analyzer with
> Lowercase filter which will convert my values to lowercase then I can apply
> wildcard.But with the above analyzer for Customer I still got nothing . Can
> someone help me to understand what I am doing wrong .
>
> --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/eb6bda79-428d-4fdc-b478-65f07905fe89%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAECdJzBkQ%3DE%3DnFtrwbcUCazx8AZFnVv9SaC7X2Ju9fMSg7mD4Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elasticsearch: 2-node cluster with failover

[David Pilato]

Add another node (master only : data false) with minimal HEAP and set minimum 
master nodes to 2
or
set minimum master nodes to 2 but in that case your cluster will stop working 
in case of failure of any node or network failure


-- 
David Pilato - Developer | Evangelist 
elastic.co
@dadoonet  | @elasticsearchfr 
 | @scrutmydocs 






> Le 28 mai 2015 à 10:40, prakhar  a écrit :
> 
> I am using *Elasticsearch 1.5.2* and trying to setup a 2-node cluster. These
> 2 nodes are primarily for failover strategy (if any one node goes down, the
> other one is still there to handle requests), I don't need to divide primary
> shards or something like that, (total data is no more than 500mb on
> hard-disk).
> 
> Everything goes well, until *Split Brains* thing kicks in. Now, since I
> don't have much data, I don't feel any requirement of 3 nodes. And I want to
> have failover mechanism too. Which means,
> /discovery.zen.minimum_master_nodes/ cannot be more than /1/.
> 
> Now, I have two questions:
> 
> 1. Is there any configuration possible, which could overcome *2 master
> nodes* or *Split Brains *problem?
> 2. If not, what all other options do I have to make it work? Like, keeping
> both in different clusters (one online, other one offline) and updating
> offline with online, time to time, for the time when online cluster goes
> down. Or, do I have to go for 3-node cluster?
> 
> I am going on production environment. Please help.
> 
> Env Info: Two machines (VMWare driven), having 4 cores, 8gb RAM and CentOS.
> We are firing 7-8 queries (1 /_search/ and other /_count/) per search.
> Indexing op won't be very frequent (until we sync existing indices from DB
> through /_bulk/ API), but searching is going to be quite frequent.
> 
> 
> 
> --
> View this message in context: 
> http://elasticsearch-users.115913.n3.nabble.com/Elasticsearch-2-node-cluster-with-failover-tp4074949.html
> Sent from the Elasticsearch Users mailing list archive at Nabble.com.
> 
> -- 
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> --- 
> You received this message because you are subscribed to the Google Groups 
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/elasticsearch/1432802400465-4074949.post%40n3.nabble.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/42DC41A9-8F90-4BB9-BBBC-35D57105A809%40pilato.fr.
For more options, visit https://groups.google.com/d/optout.

Re: Considering scalability , is it right to keep a large number of primary shards at beginning？

[xinmeike]

Thanks for your answer~

May be when 10 servers extends to 100 servers it can work ,but I‘m afraid 
that if we do reindex to 100 servers  it may cost a long time and huge I/O 
resources.We need to stop the service for a long time and all the data need 
to transport from old index to new one.

Is there any easier way to horizontal expansion？

P.S. I cann’t visit  https://discuss.elastic.co/ 
 today .It is blank all the time. (･ˇ_ˇ･)


在 2015年5月28日星期四 UTC+8下午3:05:13，Mark Walkom写道：
>
> You don't want 400 shards on 10 servers. You do want the ability to 
> reindex to allow you to reshard to deal with this issue.
> Logstash 1.5 can do this very easily, see this example 
> .
>
> However you probably don't want an index with 200 shards irrespective, you 
> may want to take a look at your data structure and split things out.
>
>
> PS - We're moving to https://discuss.elastic.co/, please join us there 
> for any future discussions!
>
>
>
> On 28 May 2015 at 12:46, > wrote:
>
>>
>> Hi，
>>
>> I feel very confuse when deciding the number of primary shards at 
>> beginning.
>>
>> As we know the number of shards and replicas can be defined per index at 
>> the time the index is created. After the index is created, we may change 
>> the number of replicas dynamically anytime but we cannot change the number 
>> of primary shards after-the-fact. Our ES project may be run as trial 
>> version at beginning an it is only 10 machines in cluster. However, if the 
>> project run in production environment,  the machine magnitude must 
>> increase and there will have 200 or more machines.
>>
>> How can we decide shards number at beginning? Is it encourage to run 400 
>> or more shards in 10 machines, will it reduce the performance of 
>> cluster? 
>>
>>
>> Thank you for reading and look forward to your suggestions.
>>
>>
>>  -- 
>> Please update your bookmarks! We have moved to 
>> https://discuss.elastic.co/
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/a6eb715a-6543-4882-8635-81a0d22ca1d7%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/65cd6e3e-97f1-416e-8f97-e34a8dd7671e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Elasticsearch: 2-node cluster with failover

[prakhar]

I am using *Elasticsearch 1.5.2* and trying to setup a 2-node cluster. These
2 nodes are primarily for failover strategy (if any one node goes down, the
other one is still there to handle requests), I don't need to divide primary
shards or something like that, (total data is no more than 500mb on
hard-disk).

Everything goes well, until *Split Brains* thing kicks in. Now, since I
don't have much data, I don't feel any requirement of 3 nodes. And I want to
have failover mechanism too. Which means,
/discovery.zen.minimum_master_nodes/ cannot be more than /1/.

Now, I have two questions:

 1. Is there any configuration possible, which could overcome *2 master
nodes* or *Split Brains *problem?
 2. If not, what all other options do I have to make it work? Like, keeping
both in different clusters (one online, other one offline) and updating
offline with online, time to time, for the time when online cluster goes
down. Or, do I have to go for 3-node cluster?

I am going on production environment. Please help.

Env Info: Two machines (VMWare driven), having 4 cores, 8gb RAM and CentOS.
We are firing 7-8 queries (1 /_search/ and other /_count/) per search.
Indexing op won't be very frequent (until we sync existing indices from DB
through /_bulk/ API), but searching is going to be quite frequent.



--
View this message in context: 
http://elasticsearch-users.115913.n3.nabble.com/Elasticsearch-2-node-cluster-with-failover-tp4074949.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1432802400465-4074949.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.

Not able to search Upper case values

[vikas gopal]

Hi Experts,

In my log I have a field "Customer". It has all the values in capital 
latter like "VIKAS". Now when I fire query in Kibana like Customer:"vikas" 
or Customer:"VIK*" I got nothing . It works only when I do exact search 
like Customer:"VIKAS".

I have following question

1) How I can search value with wildcard, something like name start with vi 
OR VI ?

My Template is like 

{
 "template_vg":{
"template" : "vg*",
"settings" : {
"number_of_shards" : 5,
"index.cache.field.type" : "soft",
"index.refresh_interval" : "5s",
"index.store.compress.stored" : true,
"index.query.default_field" : "message",
"index.routing.allocation.total_shards_per_node" : 5,
"index":{
"analysis":{
"analyzer":{
  "analyzer_keyword":{
 "type": "custom",
 "tokenizer":"keyword",
  "filter":"lowercase"
  }
   }
}
 }
},
"mappings" : {
"_default_" : {
"_all" : {"enabled" : false},
"properties" : {
"Order_ID": {"type": "string","index" :"not_analyzed","doc_values" : true},
"Customer": {"type": "string","index" : "not_analyzed","doc_values" : true 
,"analyzer":"analyzer_keyword"},
"Ordered_On": {"type": "string","index" : "not_analyzed","doc_values" : 
true },
"Required_By": {"type": "string","index" : "not_analyzed","doc_values" : 
true },
"Freight": {"type": "string","index" : "not_analyzed","doc_values" : true },
"Shipped_On": {"type": "string","index" : "not_analyzed","doc_values" : 
true },
"Country": {"type": "string","index" : "not_analyzed","doc_values" : true },
"Post_Code": {"type": "string","index" : "not_analyzed","doc_values" : true 
}
 }
}
}
}
}


I read almost everything and got an idea that I need to use analyzer with 
Lowercase filter which will convert my values to lowercase then I can apply 
wildcard.But with the above analyzer for Customer I still got nothing . Can 
someone help me to understand what I am doing wrong . 

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/eb6bda79-428d-4fdc-b478-65f07905fe89%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elastic search synonym match involving numeric characters

[xinmeike]

I think your filters' order may swap like this：

"analysis": {
  "analyzer": {
"mysynonym": {
  "tokenizer": "standard",
  "filter": [
"mysynonym","standard","lowercase", "stop"
  ],
  "ignore_case": true
}
  }

That's because filters work like assembly line, if your first filter is 
standard, your "2500 HD" will split to "2500" and "HD" before go through 
other filters. So your mysynonym do not work.





























*//before swap filters' order$ curl -POST 
"http://myES/my_test/_analyze?pretty&analyzer=mysynonym"; -d "2500 HD"{  
"tokens" : [ {"token" : "2500","start_offset" : 0,"end_offset" 
: 4,"type" : "","position" : 1  }, {"token" : "hd",
"start_offset" : 5,"end_offset" : 7,"type" : "",
"position" : 2  } ]}//after$ curl -POST 
"http://myES/my_test/_analyze?pretty&analyzer=mysynonym"; -d "2500 HD"{  
"tokens" : [ {"token" : "2500hd","start_offset" : 0,
"end_offset" : 7,"type" : "SYNONYM","position" : 1  } ]}*


Sincerely hope this may helpful to you.



在 2015年5月28日星期四 UTC+8上午11:20:51，Siva Shanmuga Subramanian Murugan写道：
>
> I have documents indexed in elastic cluster with the below mapping. 
> basically i have a field named model which holds car model names like 
> "Silverado 2500HD", "Silverado 1500HD" "LX 350" etc etc.
>
> POST /location-test-no-boost {
> "settings":{
> "analysis":{
> "analyzer":{
> "mysynonym":{
> "tokenizer":"standard",
> "filter":[
> "standard","lowercase","stop","mysynonym"
> ],
> "ignore_case":true
> }
> },
> "filter":{
> "mysynonym":{
> "type":"synonym",
> "synonyms": [
> "2500 HD=>2500HD",
> "chevy silverado=>Silverado"
> ]
> }
> }
> }
> },
> "mappings":{
> "vehicles":{
> "properties":{
> "id":{
> "type":"long",
> "ignore_malformed":true
> },
> "model":{
> "type":"String",
> "index_analyzer": "standard",
> "search_analyzer":"mysynonym"
> }
> }
> }
> }
> }
>
> The sample document content is
>
> POST /location-test-no-boost/vehicles/10
> {
>   "model" : "Silverado 2500HD"
> }
>
> When i tried to search with Query string "Chevy sivlerado", the synonym 
> matches perfectly to Silverado and gives back the result, on the contrary 
> when i tried to search via query string "2500 HD" it is returning 0 
> results. I tried different combination on the Synonym involving number and 
> found that elastic search synonym mapper does not support numbers is this 
> correct?
>
> is there any way i can make some mapping when user searches for "2500 HD", 
> i can map the query to "2500HD"
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/803ed44d-2c8f-43fd-9cab-470e9a262b00%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Is the snapshot incremental?

[shoebalig]

This is helpful. It looks like that, this has been handled in 1.5, prior to
that like 1.4.4 in my case. it was not handled in this manner in older
version1.4.* ?
Any thought?

On Thu, May 28, 2015 at 11:20 AM, Allan Mitchell [via Elasticsearch Users] <
ml-node+s115913n4074935...@n3.nabble.com> wrote:

> Hi
> Does this help?
>
> From:
> https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html
>
> The index snapshot process is incremental. In the process of making the
> index snapshot Elasticsearch analyses the list of the index files that are
> already stored in the repository and copies only files that were created or
> changed since the last snapshot. That allows multiple snapshots to be
> preserved in the repository in a compact form.
>
> On 28 May 2015 at 06:40, shoeb khan <[hidden email]
> > wrote:
>
>> I too suffering from same problem. my cron-job runs daily. There are
>> chances that there is no modification or changes found ion original index.
>> however cron-job create snapshot again on its scheduled time. which leads
>> duplicity and restore takes a lot of time.
>>
>> What if I retain only last 5 or last 10 snapshot? Will this cause any
>> data lose anyway?
>>
>> And on restore whole index will be restored?
>>
>> --
>>
>>
>>
>> *Thanks & RegardsShoeb Khan8743990512*
>>
>> --
>> Please update your bookmarks! We have moved to
>> https://discuss.elastic.co/
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [hidden email]
>> .
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAPUopFR589RY-jUcNf7ZkUGz_YoRgtp0BgJCn86dFt9G-Hkxuw%40mail.gmail.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [hidden email]
> .
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAECdJzBuchY5FDMH8w3pMt_fU5nvsva55SqFckuabTfN%2BFjjog%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://elasticsearch-users.115913.n3.nabble.com/Is-the-snapshot-incremental-tp4074934p4074935.html
>  To unsubscribe from Is the snapshot incremental?, click here
> 
> .
> NAML
> 
>



-- 



*Thanks & RegardsShoeb Khan8743990512*




--
View this message in context: 
http://elasticsearch-users.115913.n3.nabble.com/Is-the-snapshot-incremental-tp4074934p4074939.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAPUopFQ2T8naVWOtxgHHCJV4ZHbjWF6nu_F7CdkgUO6c-fcq_g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Is the snapshot incremental?

[shoebalig]

This is helpful. It looks like that, this has been handled in 1.5, prior to
that like 1.4.4 in my case. it was not handled in this manner in older
version1.4.* ?



--
View this message in context: 
http://elasticsearch-users.115913.n3.nabble.com/Is-the-snapshot-incremental-tp4074934p4074938.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1432794827230-4074938.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.

Re: Is the snapshot incremental?

[shoebalig]

Thanks for feedback. It looks like this has been handled in 1.5, prior to
that like 1.4.4 in my case. it was not handled in this manner?



--
View this message in context: 
http://elasticsearch-users.115913.n3.nabble.com/Is-the-snapshot-incremental-tp4074934p4074936.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/1432792896579-4074936.post%40n3.nabble.com.
For more options, visit https://groups.google.com/d/optout.

Re: Is the snapshot incremental?

[shoebalig]

Thanks for feedback. This is helpful. It looks like this has been handled
in 1.5, prior to that like 1.4.4 in my case. it was not handled in this
manner?

On Thu, May 28, 2015 at 11:20 AM, Allan Mitchell [via Elasticsearch Users] <
ml-node+s115913n4074935...@n3.nabble.com> wrote:

> Hi
> Does this help?
>
> From:
> https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html
>
> The index snapshot process is incremental. In the process of making the
> index snapshot Elasticsearch analyses the list of the index files that are
> already stored in the repository and copies only files that were created or
> changed since the last snapshot. That allows multiple snapshots to be
> preserved in the repository in a compact form.
>
> On 28 May 2015 at 06:40, shoeb khan <[hidden email]
> > wrote:
>
>> I too suffering from same problem. my cron-job runs daily. There are
>> chances that there is no modification or changes found ion original index.
>> however cron-job create snapshot again on its scheduled time. which leads
>> duplicity and restore takes a lot of time.
>>
>> What if I retain only last 5 or last 10 snapshot? Will this cause any
>> data lose anyway?
>>
>> And on restore whole index will be restored?
>>
>> --
>>
>>
>>
>> *Thanks & RegardsShoeb Khan8743990512*
>>
>> --
>> Please update your bookmarks! We have moved to
>> https://discuss.elastic.co/
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [hidden email]
>> .
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/CAPUopFR589RY-jUcNf7ZkUGz_YoRgtp0BgJCn86dFt9G-Hkxuw%40mail.gmail.com
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>  --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [hidden email]
> .
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAECdJzBuchY5FDMH8w3pMt_fU5nvsva55SqFckuabTfN%2BFjjog%40mail.gmail.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
>  If you reply to this email, your message will be added to the discussion
> below:
>
> http://elasticsearch-users.115913.n3.nabble.com/Is-the-snapshot-incremental-tp4074934p4074935.html
>  To unsubscribe from Is the snapshot incremental?, click here
> 
> .
> NAML
> 
>



-- 



*Thanks & RegardsShoeb Khan8743990512*




--
View this message in context: 
http://elasticsearch-users.115913.n3.nabble.com/Is-the-snapshot-incremental-tp4074934p4074937.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAPUopFTu5L2AYOaBoTkU0vRCBudsXnW7tWLttstV7QmgyJt2jQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Is it possible to change the score mode of SHOULD clause?

[David Kemp]

Have you tried the disable_coord feature of bool queries?
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-bool-query.html

It might give you what you need.

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/61402bae-4ff5-42f9-bd4f-046cc7fc9972%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Elastic search synonym match involving numeric characters

[xinmeike]

may be you should debug like this:

POST  /location-test-no-boost/vehicles?pretty&analyzer=mysynonym 
"2500 HD"

and it will return the analyze result. You can compare to:

POST  /location-test-no-boost/vehicles?pretty&analyzer=standard
"Silverado 2500HD"

So you may know witch place has problem. 


在 2015年5月28日星期四 UTC+8上午11:20:51，Siva Shanmuga Subramanian Murugan写道：
>
> I have documents indexed in elastic cluster with the below mapping. 
> basically i have a field named model which holds car model names like 
> "Silverado 2500HD", "Silverado 1500HD" "LX 350" etc etc.
>
> POST /location-test-no-boost {
> "settings":{
> "analysis":{
> "analyzer":{
> "mysynonym":{
> "tokenizer":"standard",
> "filter":[
> "standard","lowercase","stop","mysynonym"
> ],
> "ignore_case":true
> }
> },
> "filter":{
> "mysynonym":{
> "type":"synonym",
> "synonyms": [
> "2500 HD=>2500HD",
> "chevy silverado=>Silverado"
> ]
> }
> }
> }
> },
> "mappings":{
> "vehicles":{
> "properties":{
> "id":{
> "type":"long",
> "ignore_malformed":true
> },
> "model":{
> "type":"String",
> "index_analyzer": "standard",
> "search_analyzer":"mysynonym"
> }
> }
> }
> }
> }
>
> The sample document content is
>
> POST /location-test-no-boost/vehicles/10
> {
>   "model" : "Silverado 2500HD"
> }
>
> When i tried to search with Query string "Chevy sivlerado", the synonym 
> matches perfectly to Silverado and gives back the result, on the contrary 
> when i tried to search via query string "2500 HD" it is returning 0 
> results. I tried different combination on the Synonym involving number and 
> found that elastic search synonym mapper does not support numbers is this 
> correct?
>
> is there any way i can make some mapping when user searches for "2500 HD", 
> i can map the query to "2500HD"
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/4981bbf5-86a7-4777-959c-c0b264deb900%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Considering scalability , is it right to keep a large number of primary shards at beginning？

[Mark Walkom]

You don't want 400 shards on 10 servers. You do want the ability to reindex
to allow you to reshard to deal with this issue.
Logstash 1.5 can do this very easily, see this example
.

However you probably don't want an index with 200 shards irrespective, you
may want to take a look at your data structure and split things out.


PS - We're moving to https://discuss.elastic.co/, please join us there for
any future discussions!



On 28 May 2015 at 12:46,  wrote:

>
> Hi，
>
> I feel very confuse when deciding the number of primary shards at
> beginning.
>
> As we know the number of shards and replicas can be defined per index at
> the time the index is created. After the index is created, we may change
> the number of replicas dynamically anytime but we cannot change the number
> of primary shards after-the-fact. Our ES project may be run as trial
> version at beginning an it is only 10 machines in cluster. However, if the
> project run in production environment,  the machine magnitude must
> increase and there will have 200 or more machines.
>
> How can we decide shards number at beginning? Is it encourage to run 400
> or more shards in 10 machines, will it reduce the performance of cluster?
>
>
> Thank you for reading and look forward to your suggestions.
>
>
>  --
> Please update your bookmarks! We have moved to https://discuss.elastic.co/
> ---
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/a6eb715a-6543-4882-8635-81a0d22ca1d7%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
Please update your bookmarks! We have moved to https://discuss.elastic.co/
--- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-PWRDC89CzDqSSmC%2BnjP4FD0o_F%2BDZjNOLzmDpXwDtaw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.