I wish i could +1 on postings like this ! Gonna try it out tomorrow ;]
Thnx.
Bart Thate
programming schizofrenic - "till freedom come!"
[!] http://protocol.by/botfather
[!] http://botfather.blogspot.com
[!] http://jsonbot.org
On Sat, Sep 24, 2011 at 8:23 PM, Adam Sah wrote:
> hey guys--
>
> If you're using Tasks and haven't tried public, dynamic Backends yet,
> they're wonderful:
> - for tiny apps, you can offload another $0.72/day for free (they're
> dynamic!)
> - for medium apps, it's a super easy way to get more RAM
> - easy migration from Tasks running on frontends to running them on
> public, dynamic backends (incl development, debugging, etc.)
> - nearly identical debugging/production methods as frontend Tasks
> - since they're public, you can launch Tasks directly on the backend,
> removing the frontend from the equation, e.g. you can test out how a
> Task will perform on a backend, before committing to the architecture
> change.
> - cool trick: keep the fast Tasks running on frontends, and use the
> (expensive) backends for RAM-intensive jobs-- this is a tiny code
> change, and also makes it easier to debug large jobs because the
> console isn't flooded with little jobs. For my app, I use Tasks for
> various indexing jobs in an online marketplace-- now, I send the
> "large" sellers with 1000s of SKUs, to a B4 backend.
>
> GAE team:
> - I've noticed that my backends can 500-error but not produce a stack
> trace in the logs? are other people seeing this?
>
> my choice of migration: (python)
>
> 1. create backends.xml, define *one* backend, e.g.
>backends:
> - name: backend2647324# since it's public, give it a hard-to-
> guess name
> class: B4# big+fast without blowing the free budget
> options: dynamic, public
>
> 2. add backend-enabled queued, by copying queues from frontends:
>
> queue:
>- name: image-processor
> ...other options here...
>
>- name: image-processor-backend # note how I append "-backend"
> to the name
> target: backend2647324
> ...other options here...
>
> 3. in your code, start firing Tasks at the backend
>
># see bottom for taskname() trick
>taskqueue.add(url=..., name=taskname("some-identifier-i-will-
> recognize"),
>queue_name="image-thumbnailer"+("-backend" if
> is_gigantic_image(...) else ""))
>
> 4. modify your deploy script to update both the front & backends.
> Mine (legacy) is in perl:
>
> # deploy bar ==> deploys to foo-bar and to foo-bar-backend
> # deploy bar-backend ==> deploys to foo-bar-backend only
> # deploy bar-frontend ==> deploys to foo-bar only
> $DEST = pop(@ARGV);
> my $push_frontend = 1;
> if ($DEST =~ s/-backend//) { $push_frontend = 0; }
> my $push_backend = 1;
> if ($DEST =~ s/-frontend//) { $push_backend = 0; }
> if ($push_frontend) {
> open(FH, "echo ".$ENV{"GAEPASSWD"}."|python2.5 ".$ENV{"GAEDIR"}."/
> appcfg.py --email=".$ENV{"GAEUSER"}." --passin --application=foo-$DEST
> update .|");
> while () { print; }
> close FH;
> }
> if ($push_backend) {
> open(FH, "echo ".$ENV{"GAEPASSWD"}."|python2.5 ".$ENV{"GAEDIR"}."/
> appcfg.py --email=".$ENV{"GAEUSER"}." --passin --application=foo-$DEST
> backends . update|");
> while () { print; }
> close FH;
> }
>
>
> that's it! now, some jobs will go to the frontends and others to the
> backends.
>
> hope this helps,
> adam
>
>
> def taskname(string):
> """safely create Google App Engine Task names."""
> # add timestamp and random for ultimate uniqueness, strip disallowed
> chars
> rnd_salt = str(random.randint(10, 99))
> return re.sub(r'[^a-zA-Z0-9-]', '-', string +
> str(datetime.datetime.now())) + "-" + rnd_salt
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
