[google-appengine] Re: public, dynamic backends are magical (in a good way!!)
private backends are wonderful too-- but making them public makes it easier to debug actions because you can connect directly to the backend. for example, you can use the App Engine hosted console ( http://con.appspot.com/) to read the memory of the backend directly. (I will post on gae console soon-- it's amazing...) adam -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/JQc8GyzxG58J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: public, dynamic backends are magical (in a good way!!)
I see. The backends looks great! I will try it soon. On Sep 30, 9:34 am, Adam Sah adam@gmail.com wrote: private backends are wonderful too-- but making them public makes it easier to debug actions because you can connect directly to the backend. for example, you can use the App Engine hosted console (http://con.appspot.com/) to read the memory of the backend directly. (I will post on gae console soon-- it's amazing...) adam -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: public, dynamic backends are magical (in a good way!!)
given the happy vibes, I'll see about posting a series of these. Let's call this post #1 of the series. cheers, adam -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/lkQ7E8S-tuoJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: public, dynamic backends are magical (in a good way!!)
why it must be public?
private backends doesn't work?
On Sep 25, 2:23 am, Adam Sah adam@gmail.com wrote:
hey guys--
If you're using Tasks and haven't tried public, dynamic Backends yet,
they're wonderful:
- for tiny apps, you can offload another $0.72/day for free (they're
dynamic!)
- for medium apps, it's a super easy way to get more RAM
- easy migration from Tasks running on frontends to running them on
public, dynamic backends (incl development, debugging, etc.)
- nearly identical debugging/production methods as frontend Tasks
- since they're public, you can launch Tasks directly on the backend,
removing the frontend from the equation, e.g. you can test out how a
Task will perform on a backend, before committing to the architecture
change.
- cool trick: keep the fast Tasks running on frontends, and use the
(expensive) backends for RAM-intensive jobs-- this is a tiny code
change, and also makes it easier to debug large jobs because the
console isn't flooded with little jobs. For my app, I use Tasks for
various indexing jobs in an online marketplace-- now, I send the
large sellers with 1000s of SKUs, to a B4 backend.
GAE team:
- I've noticed that my backends can 500-error but not produce a stack
trace in the logs? are other people seeing this?
my choice of migration: (python)
1. create backends.xml, define *one* backend, e.g.
backends:
- name: backend2647324 # since it's public, give it a hard-to-
guess name
class: B4 # big+fast without blowing the free budget
options: dynamic, public
2. add backend-enabled queued, by copying queues from frontends:
queue:
- name: image-processor
...other options here...
- name: image-processor-backend # note how I append -backend
to the name
target: backend2647324
...other options here...
3. in your code, start firing Tasks at the backend
# see bottom for taskname() trick
taskqueue.add(url=..., name=taskname(some-identifier-i-will-
recognize),
queue_name=image-thumbnailer+(-backend if
is_gigantic_image(...) else ))
4. modify your deploy script to update both the front backends.
Mine (legacy) is in perl:
# deploy bar == deploys to foo-bar and to foo-bar-backend
# deploy bar-backend == deploys to foo-bar-backend only
# deploy bar-frontend == deploys to foo-bar only
$DEST = pop(@ARGV);
my $push_frontend = 1;
if ($DEST =~ s/-backend//) { $push_frontend = 0; }
my $push_backend = 1;
if ($DEST =~ s/-frontend//) { $push_backend = 0; }
if ($push_frontend) {
open(FH, echo .$ENV{GAEPASSWD}.|python2.5 .$ENV{GAEDIR}./
appcfg.py --email=.$ENV{GAEUSER}. --passin --application=foo-$DEST
update .|);
while (FH) { print; }
close FH;}
if ($push_backend) {
open(FH, echo .$ENV{GAEPASSWD}.|python2.5 .$ENV{GAEDIR}./
appcfg.py --email=.$ENV{GAEUSER}. --passin --application=foo-$DEST
backends . update|);
while (FH) { print; }
close FH;
}
that's it! now, some jobs will go to the frontends and others to the
backends.
hope this helps,
adam
def taskname(string):
safely create Google App Engine Task names.
# add timestamp and random for ultimate uniqueness, strip disallowed
chars
rnd_salt = str(random.randint(10, 99))
return re.sub(r'[^a-zA-Z0-9-]', '-', string +
str(datetime.datetime.now())) + - + rnd_salt
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: public, dynamic backends are magical (in a good way!!)
i found a new problem - the cron job run twice (frontend and backend). how to specify frontend cron job only? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: public, dynamic backends are magical (in a good way!!)
This should never occur. A cron task should only run on one instance. If you have specified the 'target' parameter then it should execute on that version/backend otherwise it will execute on the default version of your application. What is your app id so I can look into this? On 25 September 2011 17:49, pman pollk...@gmail.com wrote: i found a new problem - the cron job run twice (frontend and backend). how to specify frontend cron job only? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: public, dynamic backends are magical (in a good way!!)
sorry - that's my mistake. confused by the log message in admin panel. On Sep 26, 7:45 pm, Greg Darke (Google) darke+goo...@google.com wrote: This should never occur. A cron task should only run on one instance. If you have specified the 'target' parameter then it should execute on that version/backend otherwise it will execute on the default version of your application. What is your app id so I can look into this? On 25 September 2011 17:49, pman pollk...@gmail.com wrote: i found a new problem - the cron job run twice (frontend and backend). how to specify frontend cron job only? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
