Re: [google-appengine] HRD migration tool hangs for hours - how to find a root cause?

2012-07-04 Thread Waleed Abdulla
On my initial copy I remember it staying at 99% for a long time, but it
eventually finished. I was copying a lot of data so the full copy took 3.5
days. In the sync phase, I remember it staying at 99% for almost half the
time of every sync and sometimes more. My first sync took 12 hours, and the
second 6 hours. On the last sync it stayed on 99% too long so I stopped it
and jumped to the last stage (read-only mode) and that made it much faster.

Waleed


On Tue, Jul 3, 2012 at 12:58 AM, Alexander Trakhimenok <
alexander.trakhime...@gmail.com> wrote:

> We are migrating our app to an HRD version and the migration tool process 
> hangs on 2nd step "Copy" at 99% for 10+ hours.
>
> We've restarted migration but it did not help - still hangs at 99% and the 
> "approximately remaining" time is steadily and slowly growing.
>
> I've been googling around and as I understand it can be because of indexes 
> issue (5K index entry limit per entity) or some other reason that prevent an 
> entity to be stored to the new HRD database. Actually removing some indexes 
> helped us to move from 98.50% to 99% and  not  any further.
>
> But we can not find any error/status logs that would point us on what exactly 
> entity kind/id causing the issue.
>
> On the Migration Tool screen it says:
> =
> If there is a problem, you can pause the migration. While paused, all 
> migration mapreduces will be paused and but no other actions will be taken 
> until an administrator either resumes the migration or reverts the process. 
> To pause the migration, please click the button below.
> =
>
> But how do we know if there is a problem and what exactly the problem is? It 
> would be cool to have a "last or list if error messages & entity keys for the 
> migration process.
>
> Any ideas how to identify and solve the issue? We have major release pending 
> due to limitations of M/S database and would greatly appreciate any help.
>
>
> I've raised a production issue for this as well 
> http://code.google.com/p/googleappengine/issues/detail?id=7787
>
>
> but may be someone from community can help us to resolve it before Google 
> Engineers notice the ticket.
>
>
> Alexander Trakhimenok
>
>
> Lead developer on http://www.myclasses.org/ project powered by GAE (AppEngine 
> + Python + Django).
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/AYfYbuyKGsYJ.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] HRD migration tool hangs for hours - how to find a root cause?

2012-07-03 Thread Alexander Trakhimenok


We are migrating our app to an HRD version and the migration tool process hangs 
on 2nd step "Copy" at 99% for 10+ hours.

We've restarted migration but it did not help - still hangs at 99% and the 
"approximately remaining" time is steadily and slowly growing.

I've been googling around and as I understand it can be because of indexes 
issue (5K index entry limit per entity) or some other reason that prevent an 
entity to be stored to the new HRD database. Actually removing some indexes 
helped us to move from 98.50% to 99% and  not  any further.

But we can not find any error/status logs that would point us on what exactly 
entity kind/id causing the issue.

On the Migration Tool screen it says:
=
If there is a problem, you can pause the migration. While paused, all migration 
mapreduces will be paused and but no other actions will be taken until an 
administrator either resumes the migration or reverts the process. To pause the 
migration, please click the button below.
=

But how do we know if there is a problem and what exactly the problem is? It 
would be cool to have a "last or list if error messages & entity keys for the 
migration process.

Any ideas how to identify and solve the issue? We have major release pending 
due to limitations of M/S database and would greatly appreciate any help.


I've raised a production issue for this as well 
http://code.google.com/p/googleappengine/issues/detail?id=7787


but may be someone from community can help us to resolve it before Google 
Engineers notice the ticket.


Alexander Trakhimenok


Lead developer on http://www.myclasses.org/ project powered by GAE (AppEngine + 
Python + Django).

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/AYfYbuyKGsYJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.