Re: Upgrade 0.26.6 -> 1.2.2 any catches?

2014-07-15 Thread Ivan Brusic 
Read more about it here:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-store.html#store-throttling

Previously it was unbounded, but now the default is 20mb, which I found to
be extremely low. Also, prior to 1.2, there was a Lucene bug which made
throttling even more limiting:

https://github.com/elasticsearch/elasticsearch/issues/6018

As far as settings goes, I can't think of any that have changed. Perhaps if
you posted your altered settings, others can give it a once over.

BTW, there is no need for the @ symbol when writing on a mailing list.
Normal English writing rules apply, not the Twitter ones.

Cheers,

Ivan


On Mon, Jul 14, 2014 at 8:41 PM, smonasco  wrote:

> @Ivan, what do you mean the stores are throttled with a very low level?
>  Do you mean index threads or merge factor or what?
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/63273182-325a-4253-bb79-f9a7b5e3b199%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAc9uaBvsH56y31KnaPHGJ67GMXsuTOsBwF7hn7N7PrCQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Upgrade 0.26.6 -> 1.2.2 any catches?

2014-07-15 Thread Scott Decker 
Very helpful everyone! (and yes, should have been .20.6!

so, bigger takeaways are
1 - see if we move from 20.6 to .90 and then to 1.2.2 (will add this to our 
conversations)
2 - optimize! that is a good one to note. had forgotten the internals of 
lucene while talking all of our steps. thanks for reminding me!

Can we run with the same elasticsearch.yml/json file? has anything changed 
in the config part that I should be aware of? I know sometimes the 
name.value pair naming changes, but, hard to find out what those are 
without code digging or trying it and seeing what fails.


On Monday, July 14, 2014 3:37:01 PM UTC-7, Ivan Brusic wrote:
>
> First of all, there is no version 0.26. I am assuming you meant 0.20.6. 
> Either way, any upgrade from prior of 1.0 to 1.x will require a full 
> cluster restart.
>
> 1. No clue
>
> 2. Many settings like omit_norms were deprecated, but are still support. I 
> think that omit_tf has been changed.
>
> 3. I would install via apt-get/yum or use the elasticearch wrapper
>
> 4. Lucene can read indicies up to 1 version behind, so since your index is 
> based on Lucene 3 and Elasticsearch 1.x is Lucene, you do not need to 
> upgrade your indices. You should run an optimize on your indices when the 
> cluster starts back up so that Lucene can upgrade the existing segments.
>
> 4a - You can skip the optimize, but you really shouldn't. :)
>
> 5. One gotcha that I can think about is that all stores are now throttled 
> with a very low level (IMHO). Increase it based on your hardware and 
> indexing requirements. The syntax for disabling allocations has change. I'm 
> sure there are other gotchas, but most settings are still the same.
>
> Cheers,
>
> Ivan
>
>
>
> On Mon, Jul 14, 2014 at 3:20 PM, Scott Decker  > wrote:
>
>> Hey all,
>>   We are going to start our battle plans for upgrading from 0.26 to 1.2.2 
>> and I have been reading through this doc:
>>
>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/breaking-changes.html
>>
>> to help see what things we may need to worry about.
>>
>> Some specific questions
>> 1 - Can the logging.yml file now be done as json like the elasticsearch 
>> one? Sorry, kind of hate the yml format, and json is much easier. here is 
>> hoping
>> 2 - I know our index mappings need to be updated/checked, but anything 
>> pop out as to really look for? Like, on standard analyzer something was set 
>> before, but now it is gone kind of things?
>> 3 - startup.  we have just been running ./bin/elasticsearch.  I see that 
>> this has now changed. any recommended ways to have similar function as that 
>> startup?
>> 4 - index formats? Pretty sure that when we upgrade, the indexes are 
>> redone in a new format, yes? Or, if we had static indexes, and we upgrade, 
>> are some in an older format and new ones in the new format?
>> 4a - if they are being re-written, is the limit more IO than anything 
>> else? So, if we had say 1 TB of indexes to do, we should expect the process 
>> to take while to update/write the new index files
>> 5 - any other gotchas people have experienced? Recommend to just go from 
>> .26 -> 1.2.2? Just making sure!
>>
>> We are looking forward to finally being able to get to this so we can 
>> also check out Marvel on real work being done. Should be fun and thanks for 
>> any insights everyone has!
>>
>>
>>  -- 
>> You received this message because you are subscribed to the Google Groups 
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to elasticsearc...@googlegroups.com .
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/26e360db-953a-48e6-acec-2ecd1eae2ede%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Upgrade 0.26.6 -> 1.2.2 any catches?

2014-07-14 Thread smonasco 
@Ivan, what do you mean the stores are throttled with a very low level?  Do you 
mean index threads or merge factor or what?

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/63273182-325a-4253-bb79-f9a7b5e3b199%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: Upgrade 0.26.6 -> 1.2.2 any catches?

2014-07-14 Thread Ivan Brusic 
First of all, there is no version 0.26. I am assuming you meant 0.20.6.
Either way, any upgrade from prior of 1.0 to 1.x will require a full
cluster restart.

1. No clue

2. Many settings like omit_norms were deprecated, but are still support. I
think that omit_tf has been changed.

3. I would install via apt-get/yum or use the elasticearch wrapper

4. Lucene can read indicies up to 1 version behind, so since your index is
based on Lucene 3 and Elasticsearch 1.x is Lucene, you do not need to
upgrade your indices. You should run an optimize on your indices when the
cluster starts back up so that Lucene can upgrade the existing segments.

4a - You can skip the optimize, but you really shouldn't. :)

5. One gotcha that I can think about is that all stores are now throttled
with a very low level (IMHO). Increase it based on your hardware and
indexing requirements. The syntax for disabling allocations has change. I'm
sure there are other gotchas, but most settings are still the same.

Cheers,

Ivan



On Mon, Jul 14, 2014 at 3:20 PM, Scott Decker  wrote:

> Hey all,
>   We are going to start our battle plans for upgrading from 0.26 to 1.2.2
> and I have been reading through this doc:
>
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/breaking-changes.html
>
> to help see what things we may need to worry about.
>
> Some specific questions
> 1 - Can the logging.yml file now be done as json like the elasticsearch
> one? Sorry, kind of hate the yml format, and json is much easier. here is
> hoping
> 2 - I know our index mappings need to be updated/checked, but anything pop
> out as to really look for? Like, on standard analyzer something was set
> before, but now it is gone kind of things?
> 3 - startup.  we have just been running ./bin/elasticsearch.  I see that
> this has now changed. any recommended ways to have similar function as that
> startup?
> 4 - index formats? Pretty sure that when we upgrade, the indexes are
> redone in a new format, yes? Or, if we had static indexes, and we upgrade,
> are some in an older format and new ones in the new format?
> 4a - if they are being re-written, is the limit more IO than anything
> else? So, if we had say 1 TB of indexes to do, we should expect the process
> to take while to update/write the new index files
> 5 - any other gotchas people have experienced? Recommend to just go from
> .26 -> 1.2.2? Just making sure!
>
> We are looking forward to finally being able to get to this so we can also
> check out Marvel on real work being done. Should be fun and thanks for any
> insights everyone has!
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQAoYBqBsx%3DDLe_cQy66YXY_UxxrVUNVsJD%2BXr9KMEu7zQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: Upgrade 0.26.6 -> 1.2.2 any catches?

2014-07-14 Thread Mark Walkom 
You probably want to go from 0.20.X to 0.90.12, then to 1.X to be safe.
Ensure you do a optimise (merge) on each index so that they are updated to
the newer lucene version number, then when they are all done move to the
next major version.

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: ma...@campaignmonitor.com
web: www.campaignmonitor.com


On 15 July 2014 08:20, Scott Decker  wrote:

> Hey all,
>   We are going to start our battle plans for upgrading from 0.26 to 1.2.2
> and I have been reading through this doc:
>
> http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/breaking-changes.html
>
> to help see what things we may need to worry about.
>
> Some specific questions
> 1 - Can the logging.yml file now be done as json like the elasticsearch
> one? Sorry, kind of hate the yml format, and json is much easier. here is
> hoping
> 2 - I know our index mappings need to be updated/checked, but anything pop
> out as to really look for? Like, on standard analyzer something was set
> before, but now it is gone kind of things?
> 3 - startup.  we have just been running ./bin/elasticsearch.  I see that
> this has now changed. any recommended ways to have similar function as that
> startup?
> 4 - index formats? Pretty sure that when we upgrade, the indexes are
> redone in a new format, yes? Or, if we had static indexes, and we upgrade,
> are some in an older format and new ones in the new format?
> 4a - if they are being re-written, is the limit more IO than anything
> else? So, if we had say 1 TB of indexes to do, we should expect the process
> to take while to update/write the new index files
> 5 - any other gotchas people have experienced? Recommend to just go from
> .26 -> 1.2.2? Just making sure!
>
> We are looking forward to finally being able to get to this so we can also
> check out Marvel on real work being done. Should be fun and thanks for any
> insights everyone has!
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/CAEM624YPvT%3DPu5haH2EhwOzWnEbW%2B6s8kxOEO-zfqMTwiS5UVw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Upgrade 0.26.6 -> 1.2.2 any catches?

2014-07-14 Thread Scott Decker 
Hey all,
  We are going to start our battle plans for upgrading from 0.26 to 1.2.2 
and I have been reading through this doc:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/master/breaking-changes.html

to help see what things we may need to worry about.

Some specific questions
1 - Can the logging.yml file now be done as json like the elasticsearch 
one? Sorry, kind of hate the yml format, and json is much easier. here is 
hoping
2 - I know our index mappings need to be updated/checked, but anything pop 
out as to really look for? Like, on standard analyzer something was set 
before, but now it is gone kind of things?
3 - startup.  we have just been running ./bin/elasticsearch.  I see that 
this has now changed. any recommended ways to have similar function as that 
startup?
4 - index formats? Pretty sure that when we upgrade, the indexes are redone 
in a new format, yes? Or, if we had static indexes, and we upgrade, are 
some in an older format and new ones in the new format?
4a - if they are being re-written, is the limit more IO than anything else? 
So, if we had say 1 TB of indexes to do, we should expect the process to 
take while to update/write the new index files
5 - any other gotchas people have experienced? Recommend to just go from 
.26 -> 1.2.2? Just making sure!

We are looking forward to finally being able to get to this so we can also 
check out Marvel on real work being done. Should be fun and thanks for any 
insights everyone has!


-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/66ff929d-1f36-406b-a53a-e82545abc98a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.