Re: [appengine-java] Master/Slave -> High Replication migration experience

[Ikai Lan (Google)]

Kyle,

Oy, I wish you had posted earlier. We've shipped a new migration tool as
part of 1.6.1 that doesn't use datastore admin to do the copies.

--
Ikai Lan
Developer Programs Engineer, Google App Engine
plus.ikailan.com | twitter.com/ikai



On Sun, Dec 11, 2011 at 6:38 PM, Kyle Baley  wrote:

> I went through a Master/Slave -> High Replication migration over the
> weekend and it didn't go well. Hoping I can get some info on if we did
> anything wrong but mostly wanted to relay the experience for others.
>
> I did a couple of test migrations the week before, running the migration
> tool through to the end and reverting at the last "create alias" step. I
> also ran through the process in its entirety with our staging and build
> environments. It always ran smoothly
>
> (Note the use of the ellipsis above. It's what legitimate journalists
> refer to as "foreshadowing". Or possibly "foreboding".)
>
> First hurdle: When it came time to start the migration, the Datastore
> Admin tool didn't appear in the console for the destination. Normally, I'm
> used to it not showing up in Chrome but this time, it was blank in Firefox
> too. IE showed a generic HTTP error page. I needed it so I could clear out
> the data from the destination leftover from previous migration tests. I had
> already deployed the same version of the app that was currently in the
> Master/Slave version.
>
> Eventually, I looked at the source of the admin console and found the URL
> for the iframe it used and launched it directly in Firefox. It was confused
> which of the three active Google accounts I regular use should be logged in
> so I chose the correct one and carried on, deleting the existing data.
>
> Next, I launched the migration tool. It hung on the Copy phase (the second
> step). I let it go for about half an hour. (During my trial runs, it took
> less than ten minutes.)
>
> So I reverted and went back to the Datastore Admin tool (which now
> appeared properly in the Admin Console, at least in Firefox) and deleted
> all the data. I went to the Datastore Viewer to verify and could not access
> it. I received a 500 Server Error with a link to report the error on this
> group. (Incidentally, I am not a big 
> fanof
>  the new Google Groups, especially for AppEngine which has three to
> choose from)
>
> I relied on the Datastore Admin tool to tell me that the data had been
> cleared and tried the migration again. Once again, it hung on the copy
> phase, this time for almost an hour before I reverted. While it was doing
> its thing, I checked the Datastore Admin tool on the destination and almost
> from the start, it showed entity totals that were *almost* as high as the
> source. But they didn't move during the entire time I waited. I suspect it
> was reporting on old data.
>
> After reverting, I gave up on the migration tool. Before starting the
> whole process, I had done the following:
> - Switched the source app to a maintenance version that redirects all
> traffic to a maintenance page (and if a user is already running the app,
> the page is refreshed to show the maintenance page).
> - Backed up the data
>
> So I restored the backup to the HR app and all seems to be running
> smoothly on it. The source doesn't have an alias to the new app but I
> didn't intend to use one anyway.
>
> That's my migration experience. The app is up and running now and we just
> barely did it under our budget thanks to all the database reads and writes.
> An app that normally costs us about ten cents a day cost us almost $4.00
> last night.
>
> In summary, the major parts I'm not happy with:
> - Datastore Admin page not showing up in any browser
> - The migration tool apparently not working. (There's a chance it was
> actually doing something and I reverted prematurely but my earlier test
> runs suggest something went wrong.)
> - 500 Server Error on the destination's Datastore Viewer. Furthermore, it
> seems unprofessional to direct support to a Google Group for a paid
> application.
>
> The 40x cost increase for the migration I'm okay with. Four dollars still
> isn't a lot of money. That said, I'm not sure what I would have done had we
> reached our budget. I'm not the billing administrator for either app.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine-java/-/VsiiGEA26eAJ.
> To post to this group, send email to
> google-appengine-java@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this gr

Re: [appengine-java] Master/Slave -> High Replication migration experience

[Kyle Baley]

Alas, bad timing, I suppose. We pulled the trigger because we found we 
could make up tons of excuses to put it off if we set our minds to it. Wait 
to see if anyone else had issues, wait until new pricing takes effect, wait 
until the migration tool is available, etc, etc. So we sucked it up. 

I guess the advice I have is to make sure you have a backup plan in place 
just in case.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine-java/-/j8pW5j0fKSIJ.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Master/Slave -> High Replication migration experience

[Ikai Lan (Google)]

Okay. If there's anything I can do to make things better for you in this
case please let me know.

--
Ikai Lan
Developer Programs Engineer, Google App Engine
plus.ikailan.com | twitter.com/ikai



On Wed, Dec 14, 2011 at 7:08 PM, Kyle Baley  wrote:

> Alas, bad timing, I suppose. We pulled the trigger because we found we
> could make up tons of excuses to put it off if we set our minds to it. Wait
> to see if anyone else had issues, wait until new pricing takes effect, wait
> until the migration tool is available, etc, etc. So we sucked it up.
>
> I guess the advice I have is to make sure you have a backup plan in place
> just in case.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine-java/-/j8pW5j0fKSIJ.
>
> To post to this group, send email to
> google-appengine-java@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.