On 10/07/13 23:14, Jared Morrow wrote:
We try to post the packages to all the right places before we make the announcement so I'd highly recommend you don't just auto-update Riak packages when they hit apt/yum.
Ah, actually we don't automatically update them, but someone was performing a pass of unrelated package updates and accidentally pulled in the riak ones too.
I'm glad you eventually got all your nodes upgraded.
Thanks. It wasn't painful in the end; just a bit of a shock at the time when the cluster fell over. I'm still not sure why that was, but I wasn't the one performing the upgrades that seemed to cause it.
I suspect it was just too many nodes being down at once, possibly due to the delayed start-up during the upgrade.
T
On Wed, Jul 10, 2013 at 5:42 AM, Toby Corkindale <toby.corkind...@strategicdata.com.au <mailto:toby.corkind...@strategicdata.com.au>> wrote: Yeah, I didn't think 1.4.0 was into final release yet either -- yet it came through on the Debian and Ubuntu apt repositories automatically this evening. ----- Original Message ----- From: "Guido Medina" <guido.med...@temetra.com <mailto:guido.med...@temetra.com>> To: "riak-users" <riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>> Sent: Wednesday, 10 July, 2013 9:39:02 PM Subject: Re: Upgrade path 1.4.0 Hi Toby, I'm sure someone from Basho will answer soon, I just pointed you to the "release notes" direction. I have only overlooked the release notes until we decide to migrate to 1.4.0 when is final (Right now on rc1) HTH, Guido. On 10/07/13 12:29, Toby Corkindale wrote: > Thanks Guido. > > Looks like we've upgraded to 1.4.0 completely now and the cluster is back up. > > I'm not sure of the exact root cause, but what we were seeing was that too many nodes went down for the ring to be healthy, and then when nodes were restarted they waited for the ring to appear for a while, and then exited (or crashed) automatically. > > I *think* this is because after the package was upgraded, the Riak service didn't restart after the upgrade -- and our automated system to catch that and restart them wasn't kicking in quickly enough compared to the rate at which the servers were being upgraded. > > You mention in the release notes that the init scripts have been adjusted - maybe that has something to do with it? > > Cheers, > Toby > > ----- Original Message ----- > From: "Guido Medina" <guido.med...@temetra.com <mailto:guido.med...@temetra.com>> > To: "riak-users" <riak-users@lists.basho.com <mailto:riak-users@lists.basho.com>> > Sent: Wednesday, 10 July, 2013 9:05:16 PM > Subject: Re: Upgrade path 1.4.0 > > Release notes: https://github.com/basho/riak/blob/1.4/RELEASE-NOTES.md > > Maybe related to this? > Known Issues > leveldb 1.3 to 1.4 conversion > > > The first execution of 1.4.0 leveldb using a 1.3.x or 1.2.x dataset will initiate an automatic conversion that could pause the startup of each node by 3 to 7 minutes. The leveldb data in "level #1" is being adjusted such that "level #1" can operate as an overlapped data level instead of as a sorted data level. The conversion is simply the reduction of the number of files in "level #1" to being less than eight via normal compaction of data from "level #1" into "level #2". This is a one time conversion. > HTH, > > Guido. > > On 10/07/13 11:57, Toby Corkindale wrote: > > > > Hi, > some of our nodes upgraded to Riak 1.4.0, and are now refusing to start and join the cluster. > Is there documentation on the upgrade path from 1.3.2 to 1.4.0? It appears we have accidentally begun this journey, and I don't know if it's easier to go back or forwards now.. > > PS. It would have been pleasant to receive a warning about the new version on the mailing list! > > Thanks, > Toby
_______________________________________________ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
