[google-appengine] Re: What if I dpn't want memcache to fail silently?

[Crizegg]

Looking into this solution, I've jumped farther down the rabbit hole
than anticipated. First off, memcache write failures are not very
common and neglecting to use compare and set was probably the main
source of my issues.
However memcache writes can still fail and regardless of whether they
throw errors, can cause stale data in race conditions. Also, since gae
transactions live solely within the datastore, you cannot solve this
problem elegantly with transactions. See here for more info:
http://groups.google.com/group/appengine-ndb-discuss/browse_thread/thread/cd05c3c8ac7b27cf#

On Dec 1, 2:36 pm, Crizegg  wrote:
> Setting the cache is where my pain point lies. I see zero log messages
> indicating a memcache failed write on the python side. I can't think
> of a case where you WOULDN'T want to know memcache failed to overwrite
> stale data. Since the next update is going to read the stale data from
> memcache and write it to the datastore!
>
> > In Javaland there is an error handler you can set on MemcacheService
>
> @Jeff, this is the exact type of solution I'm after for Python. I'm
> thinking proxy hooks with backed off retries.
>
> Will post soon.
>
> On Nov 30, 10:43 am, "Brandon Wirtz"  wrote:
>
>
>
>
>
>
>
> > Yes,
>
> > But that is "Drawer failed to open"
>
> > Memcache and the drawer should error when theyfailto be a place that may
> > or may not contain data.
>
> > Drawer is too busy an can't be bothered right now to check if it has your
> > socks, is different than there are no socks.  Drawer is suffering from
> > non-existence issues there is no drawer.  Or if you are on MS. Drawer only
> > allows you to read socks but you can't store new socks at this time so stop
> > asking.
>
> > From: google-appengine@googlegroups.com
> > [mailto:google-appengine@googlegroups.com] On Behalf Of Jeff Schnitzer
> > Sent: Wednesday, November 30, 2011 5:13 AM
> > To: google-appengine@googlegroups.com
> > Subject: Re: [google-appengine] Re: What if I dpn't want memcache tofail
> >silently?
>
> > FWIW, there are good reasons to ask when memcache fails - I mean a real
> > failure (say, timeout exception) rather than mere lack of data.
>
> > For example, if you are put()ing a new value in memcache and you want to
> > make sure the old value is replaced, you really want to be able to retry if
> > the put() fails.
>
> > In Javaland there is an error handler you can set on MemcacheService - the
> > default is to log and ignore errors but you can set a strict error handler
> > that propagates exceptions.  Dunno what the equivalent for Python is.
>
> > Jeff
>
> > On Wed, Nov 30, 2011 at 2:03 AM, Brandon Wirtz  wrote:
>
> > My socks are not in the drawer.
>
> > Return Error ("What has the Maid been doing all day?"
>
> > Just because the socks aren't in the drawer doesn't mean the drawer is in
> > error.
>
> > From: google-appengine@googlegroups.com
> > [mailto:google-appengine@googlegroups.com] On Behalf Of Tim Hoffman
> > Sent: Tuesday, November 29, 2011 9:39 PM
> > To: google-appengine@googlegroups.com
> > Subject: [google-appengine] Re: What if I dpn't want memcache tofail
> >silently?
>
> > Hi
>
> > Its not generally considered an error if something is not in the cache, (it
> > is after all a cache, not a guarunteed storage.)
>
> > Use a decorator to raise KeyError or whatever you feel is appropriate.
>
> > Rgds
>
> > Tim
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To view this discussion on the web 
> > visithttps://groups.google.com/d/msg/google-appengine/-/A3p__F-3o1kJ.
> > To post to this group, send email to google-appengine@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com.
> > For more options, visit this group 
> > athttp://groups.google.com/group/google-appengine?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appengine@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com
> >  .
> > For more options, visit this group 
> > athttp://groups.google.com/group/google-appengine?hl=en.
>
> > --
>
> > I am the 20%
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appengine@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com.
> > For more options, visit this group 
> > athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-app

[google-appengine] Re: a happy developer

[Thomas Wiradikusuma]

Couldn't agree more! Outstanding work guys!

PS: When can we see 1.6.1?


On Dec 2, 1:11 am, Vivek Puri  wrote:
> +1
>
> On Nov 30, 4:59 pm, alex  wrote:
> > I've seen many people bashing GAE dev team, especially lately.
>
> > What I want to say here is, I've been using it for years now and it's
> > pretty awesome what you guys have been doing and where you've gone
> > today. Of course there are bugs and stuff you need to find workarounds
> > for but it is nowhere near compared to the burden of managing a
> > cluster of EC2 on AWS or pay a double to Heroku or RackSpace even if
> > they're great platforms too.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

RE: [google-appengine] App is Down

[Brandon Wirtz]

Requests go better if you include the appid, and if you are on MS or HR, 

A Please probably helps. (unless you have premiere support then you can be
demanding).


-Original Message-
From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Vivek Puri
Sent: Thursday, December 01, 2011 6:02 PM
To: Google App Engine
Subject: [google-appengine] App is Down

My app is down. Can someone from AppEngine team email me. This is urgent.

--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: App is Down

[Vivek Puri]

App is back up after quota issues.

On Dec 1, 9:01 pm, Vivek Puri  wrote:
> My app is down. Can someone from AppEngine team email me. This is
> urgent.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] App is Down

[Vivek Puri]

My app is down. Can someone from AppEngine team email me. This is
urgent.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: What happened to the promised reliability improvement for email delivery?

[Vivek Puri]

'Cause we like startups!

On Dec 1, 8:19 pm, voscausa  wrote:
> Why Sendgrid and not AWS SES. I moved to AWS SES and it is looking veyy
> good.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: What happened to the promised reliability improvement for email delivery?

[voscausa]

Why Sendgrid and not AWS SES. I moved to AWS SES and it is looking veyy 
good.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/uboeYgkS_1UJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Doom day

[Andrius A]

We have a form for that, but so far looks like everyone is happy :) but if
you are not, please fill in
https://docs.google.com/spreadsheet/viewform?hl=en_US&formkey=dC1DMmpPSU1WZnk0d1FMa3JmNXIwaGc6MQ#gid=0

On 1 December 2011 15:27, Kaan Soral  wrote:

> * "The 50% discount for frontend instance hours will expire on December
> 1st. Please check your Billing History page and update your 
> budgetif
>  necessary."
>
> Since Python 2.7 is problematic, we have already been paying a lot with
> %50 discounted prices ... My daily costs went up to 90$ from 40$ before new
> pricing, now it will be ~160$s ... 4x ...
> *
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/xd3yViF2pggJ.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

RE: [google-appengine] Doom day

[Brandon Wirtz]

Don't think that I don't feel your pain, and I did rib the guys last night
about the time frame for the pricing. (they clearly are sensitive to the
matter)

 

But What did you expect?   I went in to the beta expecting release pricing
to be 4-5x the beta price.  Sounds like that was spot on for you.

 

I moved to 2.7 I'm very happy with it.  I don't know what is broken with
yours, I'll believe that it really is broken, but I will bet money, that if
you optimized your code you could knock 25% off your price. I'm also
guessing if you tuned the performance sliders you could knock another 25%
off.

 

-Brandon

 

 

From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Kaan Soral
Sent: Thursday, December 01, 2011 7:27 AM
To: google-appengine@googlegroups.com
Subject: [google-appengine] Doom day

 

"The 50% discount for frontend instance hours will expire on December 1st.
Please check your Billing History page and update your budget

if necessary."

Since Python 2.7 is problematic, we have already been paying a lot with %50
discounted prices ... My daily costs went up to 90$ from 40$ before new
pricing, now it will be ~160$s ... 4x ...

-- 
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/xd3yViF2pggJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Connection refused errors

[Rishi Arora]

I think if I let my costs rise by lowering the "max pending latency",
kicking off new instances, that might cause a more gradual cost increase,
than letting costs rise by increasing min idle instances.  Based on the
graph below, if a I set the min idle instances to 2, I'll have 2 instances
running 24 hours, costing $1.92 a day, per instance (not considering free
quota or pre-paid instance hours for the moment).  But decreasing max
pending latency might cost higher, but more proportional to heavy loads,
than just a flat $1.92 extra every day.  Comments?

[image: Requests/Second (24 hrs)]

On Thu, Dec 1, 2011 at 5:56 PM, Rishi Arora wrote:

> I have an Android app that sends HTTP GET and POST requests to my app on
> GAE.  Why would this client occasionally get errors like "Connection to
> https://shiprack-test1.appspot.com refused".  Looks like a TCP level
> connection refusal, and not an HTTP-level error.  My app has a max idle
> instances set to 1, and I have seen that the number of active instances is
> usually always 5 or 6 during the day, peaking at ~15 every now and then.
>  At night, the number of active instances is 1, and occasionally 2.  Is the
> connection refusal a sign of extremely high latencies due to excessive
> queueing?  My min-pending latency is 10ms, and max is 2 seconds.  Sounds
> like if I reduce the max pending latency to a smaller value, I'll cause
> more instances to get spawned in response to heavy load, but I might
> address my connection refused problem?  Am I thinking along the right track
> here?
>
> Thanks in advance.
> Rishi.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Connection refused errors

[Rishi Arora]

I have an Android app that sends HTTP GET and POST requests to my app on
GAE.  Why would this client occasionally get errors like "Connection to
https://shiprack-test1.appspot.com refused".  Looks like a TCP level
connection refusal, and not an HTTP-level error.  My app has a max idle
instances set to 1, and I have seen that the number of active instances is
usually always 5 or 6 during the day, peaking at ~15 every now and then.
 At night, the number of active instances is 1, and occasionally 2.  Is the
connection refusal a sign of extremely high latencies due to excessive
queueing?  My min-pending latency is 10ms, and max is 2 seconds.  Sounds
like if I reduce the max pending latency to a smaller value, I'll cause
more instances to get spawned in response to heavy load, but I might
address my connection refused problem?  Am I thinking along the right track
here?

Thanks in advance.
Rishi.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: What happened to the promised reliability improvement for email delivery?

[Vivek Puri]

FWIW, we are moving over all the critical emails to sendgrid.com.
Current blackbox nature of email delivery(no reporting) plus the
missed emails to users is impacting our end users. Will switch back
once things change for better.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

RE: [google-appengine] Re: What if I dpn't want memcache to fail silently?

[Brandon Wirtz]

Are you experiencing issues where Memcache holds data beyond expiration?  Or
just fails to over-write when you update with new before the old expires?

-Original Message-
From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Crizegg
Sent: Thursday, December 01, 2011 1:37 PM
To: Google App Engine
Subject: [google-appengine] Re: What if I dpn't want memcache to fail
silently?

Setting the cache is where my pain point lies. I see zero log messages
indicating a memcache failed write on the python side. I can't think of a
case where you WOULDN'T want to know memcache failed to overwrite stale
data. Since the next update is going to read the stale data from memcache
and write it to the datastore!

> In Javaland there is an error handler you can set on MemcacheService
@Jeff, this is the exact type of solution I'm after for Python. I'm thinking
proxy hooks with backed off retries.

Will post soon.

On Nov 30, 10:43 am, "Brandon Wirtz"  wrote:
> Yes,
>
> But that is "Drawer failed to open"
>
> Memcache and the drawer should error when they fail to be a place that 
> may or may not contain data.
>
> Drawer is too busy an can't be bothered right now to check if it has 
> your socks, is different than there are no socks.  Drawer is suffering 
> from non-existence issues there is no drawer.  Or if you are on MS. 
> Drawer only allows you to read socks but you can't store new socks at 
> this time so stop asking.
>
> From: google-appengine@googlegroups.com 
> [mailto:google-appengine@googlegroups.com] On Behalf Of Jeff Schnitzer
> Sent: Wednesday, November 30, 2011 5:13 AM
> To: google-appengine@googlegroups.com
> Subject: Re: [google-appengine] Re: What if I dpn't want memcache to 
> fail silently?
>
> FWIW, there are good reasons to ask when memcache fails - I mean a 
> real failure (say, timeout exception) rather than mere lack of data.
>
> For example, if you are put()ing a new value in memcache and you want 
> to make sure the old value is replaced, you really want to be able to 
> retry if the put() fails.
>
> In Javaland there is an error handler you can set on MemcacheService - 
> the default is to log and ignore errors but you can set a strict error 
> handler that propagates exceptions.  Dunno what the equivalent for Python
is.
>
> Jeff
>
> On Wed, Nov 30, 2011 at 2:03 AM, Brandon Wirtz 
wrote:
>
> My socks are not in the drawer.
>
> Return Error ("What has the Maid been doing all day?"
>
> Just because the socks aren't in the drawer doesn't mean the drawer is 
> in error.
>
> From: google-appengine@googlegroups.com 
> [mailto:google-appengine@googlegroups.com] On Behalf Of Tim Hoffman
> Sent: Tuesday, November 29, 2011 9:39 PM
> To: google-appengine@googlegroups.com
> Subject: [google-appengine] Re: What if I dpn't want memcache to fail 
> silently?
>
> Hi
>
> Its not generally considered an error if something is not in the 
> cache, (it is after all a cache, not a guarunteed storage.)
>
> Use a decorator to raise KeyError or whatever you feel is appropriate.
>
> Rgds
>
> Tim
>
> --
> You received this message because you are subscribed to the Google 
> Groups "Google App Engine" group.
> To view this discussion on the web
visithttps://groups.google.com/d/msg/google-appengine/-/A3p__F-3o1kJ.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group
athttp://groups.google.com/group/google-appengine?hl=en.
>
> --
> You received this message because you are subscribed to the Google 
> Groups "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
>  .
> For more options, visit this group
athttp://groups.google.com/group/google-appengine?hl=en.
>
> --
>
> I am the 20%
>
> --
> You received this message because you are subscribed to the Google 
> Groups "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group
athttp://groups.google.com/group/google-appengine?hl=en.

--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google

[google-appengine] Re: What if I dpn't want memcache to fail silently?

[Crizegg]

Setting the cache is where my pain point lies. I see zero log messages
indicating a memcache failed write on the python side. I can't think
of a case where you WOULDN'T want to know memcache failed to overwrite
stale data. Since the next update is going to read the stale data from
memcache and write it to the datastore!

> In Javaland there is an error handler you can set on MemcacheService
@Jeff, this is the exact type of solution I'm after for Python. I'm
thinking proxy hooks with backed off retries.

Will post soon.

On Nov 30, 10:43 am, "Brandon Wirtz"  wrote:
> Yes,
>
> But that is "Drawer failed to open"
>
> Memcache and the drawer should error when they fail to be a place that may
> or may not contain data.
>
> Drawer is too busy an can't be bothered right now to check if it has your
> socks, is different than there are no socks.  Drawer is suffering from
> non-existence issues there is no drawer.  Or if you are on MS. Drawer only
> allows you to read socks but you can't store new socks at this time so stop
> asking.
>
> From: google-appengine@googlegroups.com
> [mailto:google-appengine@googlegroups.com] On Behalf Of Jeff Schnitzer
> Sent: Wednesday, November 30, 2011 5:13 AM
> To: google-appengine@googlegroups.com
> Subject: Re: [google-appengine] Re: What if I dpn't want memcache to fail
> silently?
>
> FWIW, there are good reasons to ask when memcache fails - I mean a real
> failure (say, timeout exception) rather than mere lack of data.
>
> For example, if you are put()ing a new value in memcache and you want to
> make sure the old value is replaced, you really want to be able to retry if
> the put() fails.
>
> In Javaland there is an error handler you can set on MemcacheService - the
> default is to log and ignore errors but you can set a strict error handler
> that propagates exceptions.  Dunno what the equivalent for Python is.
>
> Jeff
>
> On Wed, Nov 30, 2011 at 2:03 AM, Brandon Wirtz  wrote:
>
> My socks are not in the drawer.
>
> Return Error ("What has the Maid been doing all day?"
>
> Just because the socks aren't in the drawer doesn't mean the drawer is in
> error.
>
> From: google-appengine@googlegroups.com
> [mailto:google-appengine@googlegroups.com] On Behalf Of Tim Hoffman
> Sent: Tuesday, November 29, 2011 9:39 PM
> To: google-appengine@googlegroups.com
> Subject: [google-appengine] Re: What if I dpn't want memcache to fail
> silently?
>
> Hi
>
> Its not generally considered an error if something is not in the cache, (it
> is after all a cache, not a guarunteed storage.)
>
> Use a decorator to raise KeyError or whatever you feel is appropriate.
>
> Rgds
>
> Tim
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web 
> visithttps://groups.google.com/d/msg/google-appengine/-/A3p__F-3o1kJ.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group 
> athttp://groups.google.com/group/google-appengine?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
>  .
> For more options, visit this group 
> athttp://groups.google.com/group/google-appengine?hl=en.
>
> --
>
> I am the 20%
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group 
> athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

RE: [google-appengine] Email Parser

[Brandon Wirtz]

Unless there is a reason for the form to send an email, you would be better
to put the forms "target" on appengine and convert the post data to a
database.  Java or python can do this relatively easily, if you don't speak
either now however I don't know as this is a super easy first task.

The DataStorage is easy, but making a spreadsheet out of the data (if you
want it to be editable) is not going to be simple.

Personally if I were doing the project you describe I would Check out PHP
Form Maker (Pro?)  or one of the  other form maker software suites depending
on what you or your Sys admin/web host determines would be easiest in your
web configuration.

Not that I want to dissuade you from GAE, just don't think that the learning
curve for the task is well balanced.

-Brandon

Brandon Wirtz 
BlackWaterOps: President / Lead Mercenary 

Work: 510-992-6548 
Toll Free: 866-400-4536 
IM: drak...@gmail.com (Google Talk) 
Skype: drakegreene 
BlackWater Ops 









-Original Message-
From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Cynthia
Sent: Thursday, December 01, 2011 9:03 AM
To: Google App Engine
Subject: [google-appengine] Email Parser

I'm brand new to Google App Engine and I'm not really sure I can get
something that will do what I want. Or maybe I need to go elsewhere. I am a
programmer by trade but I am still a beginner in many areas.

I am looking for a way to receive emails and parse out certain information,
then save the info to a DB or a Spreadsheet.  A Google Docs Spreadsheet
would be fine, if that's possible.

Would Google App Engine be able to do this? Which language would be best?
Maybe something like this already exists that I could use?

It's for a club membership list. People sign up with an HTML form, and the
form emails the information to an email address. I want to pluck the data
out of the email and have it save automatically.

Thanks,
Cynthia

--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Email Parser

[Xavier Sanchez]

Hi.

I think you don't need GAE. I think you have 2 options:

   1. Create a new form with Google Docs. Embed it in your web and get the 
   results directly at a Spreadsheet.
   2. If you receive the mails at a Gmail account you can use Google Apps 
   Script  to process mail 
   and get data into a Spreadsheet. I've not tried this myself but I know 
   you an process mail labels, so I suppose you can parse the content too.

Xavi.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/dDx3glpTmkkJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Email Parser

[Cynthia]

Thanks for your reply Rishi,
Our darn website is hosted on iPage. I'm about the only technical
person in our very small, and new, club, and they don't want to change
to a new host.  I get moans and groans and no one wants to change,
they want an easy Drag and Drop interface, though it drives me nuts
and I'd move hosts in a heartbeat if I could.

So.. I have thought of modifying the form itself so it can write
directly to a Database, but I'm not sure if my changes to the code
would remain if someone else updates the pages with all their
automatic junk.  So the fewer changes I can make to the HTML itself,
the better. The least intrusive way of doing this would be, I think,
to catch the emails that the form generates and then do what I want
with them.  But alas, I've never done such a thing.

Cynthia

On Dec 1, 1:10 pm, Rishi Arora  wrote:
> My app heavily relies on processing received emails, and even though I'm
> just as good (or bad) at python as I am at Java, I chose python because if
> its flexibility in string processing.  The emails my app receives are in
> hundreds of different formats and these formats change very often.  Python
> offers ease of initial development, and ease of quick and frequent changes.
>  Of course, these are personal preferences - experience Java developers can
> possibly do things much faster than I can.
>
> But this is only part 1 of your problem set.  Part is being able to save
> generated and organized data in a Google Spreadsheet.  I haven't developed
> any such thing myself, but in my related research, I found things that
> might interest you here (GData 
> Services):http://code.google.com/appengine/docs/python/howto/usinggdataservices
>
> Lastly, why would you have an email step in the middle at all.  Your
> current HTML form that people submit - where is it hosted?  If it isn't on
> Google App Engine, you can migrate it to GAE, and have the form directly
> write to Google Docs spreadsheet.  If migrating your HTML form to GAE isn't
> viable either, consider making your form send URL encoded data in HTTP Post
> requests to GAE, in a pre-formatted way so its easier to parse this data
> and store in a spreadsheet.
>
>
>
>
>
>
>
> On Thu, Dec 1, 2011 at 11:02 AM, Cynthia  wrote:
> > I'm brand new to Google App Engine and I'm not really sure I can get
> > something that will do what I want. Or maybe I need to go elsewhere. I
> > am a programmer by trade but I am still a beginner in many areas.
>
> > I am looking for a way to receive emails and parse out certain
> > information, then save the info to a DB or a Spreadsheet.  A Google
> > Docs Spreadsheet would be fine, if that's possible.
>
> > Would Google App Engine be able to do this? Which language would be
> > best?  Maybe something like this already exists that I could use?
>
> > It's for a club membership list. People sign up with an HTML form, and
> > the form emails the information to an email address. I want to pluck
> > the data out of the email and have it save automatically.
>
> > Thanks,
> > Cynthia
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appengine@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com.
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Email Parser

[Rishi Arora]

My app heavily relies on processing received emails, and even though I'm
just as good (or bad) at python as I am at Java, I chose python because if
its flexibility in string processing.  The emails my app receives are in
hundreds of different formats and these formats change very often.  Python
offers ease of initial development, and ease of quick and frequent changes.
 Of course, these are personal preferences - experience Java developers can
possibly do things much faster than I can.

But this is only part 1 of your problem set.  Part is being able to save
generated and organized data in a Google Spreadsheet.  I haven't developed
any such thing myself, but in my related research, I found things that
might interest you here (GData Services):
http://code.google.com/appengine/docs/python/howto/usinggdataservices.html.

Lastly, why would you have an email step in the middle at all.  Your
current HTML form that people submit - where is it hosted?  If it isn't on
Google App Engine, you can migrate it to GAE, and have the form directly
write to Google Docs spreadsheet.  If migrating your HTML form to GAE isn't
viable either, consider making your form send URL encoded data in HTTP Post
requests to GAE, in a pre-formatted way so its easier to parse this data
and store in a spreadsheet.


On Thu, Dec 1, 2011 at 11:02 AM, Cynthia  wrote:

> I'm brand new to Google App Engine and I'm not really sure I can get
> something that will do what I want. Or maybe I need to go elsewhere. I
> am a programmer by trade but I am still a beginner in many areas.
>
> I am looking for a way to receive emails and parse out certain
> information, then save the info to a DB or a Spreadsheet.  A Google
> Docs Spreadsheet would be fine, if that's possible.
>
> Would Google App Engine be able to do this? Which language would be
> best?  Maybe something like this already exists that I could use?
>
> It's for a club membership list. People sign up with an HTML form, and
> the form emails the information to an email address. I want to pluck
> the data out of the email and have it save automatically.
>
> Thanks,
> Cynthia
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Please help me how to move my data to the cloud

[Abhishek Mathur]

Hi ,

Please go to your application dashboard on GAE and on the left hand  side
you will see a link to your logs. You can have a look at the logs there and
if you find any error you can post it here.

--Thanks
Abhishek


On Thu, Dec 1, 2011 at 12:33 PM, Nandar Wai Yee wrote:

> hello
> Sorry for my late reply coz of my exam 
> My project hasn't any error message ...this is very strange for me
> But I wanna to show  u my link to test i uploaded my project to Google
> App Engine
>
> http://hz-uak-app.appspot.com/
>
> Please check my link ..pls
> in ewb.xml, i set session-enable=true ...
> but i didn't get answer ...:(  i don't know what i'm wrong ...
> Waiting ur reply
>
>
> On Wed, Oct 19, 2011 at 3:41 AM, Greg  wrote:
>
>> Please post the error message you get when you press the calculate
>> button. You can find this in the log, which is in the admin dashboard.
>>
>> Without this information, no-one can help you.
>>
>> Cheers
>> Greg.
>>
>> On Oct 15, 7:05 am, Nandar Wai Yee  wrote:
>> > Hello Masters,
>> >
>> > I want a solution for my problem
>> >
>> >  The problem is that ..how to move my data to the cloud.
>> >   I built an application.
>> >   It is ok in local ...no errorit can show the result
>> > without any difficulty.
>> >   But  after i deployed my application to the web,  my
>> > application can't work well as in local.
>> >   I mean, when i put an input to the  input box, then i
>> pressed
>> > calculate button, the result cannot be shown.
>> >   How can i change which part of the code and where?
>> >   I am waiting forward ur reply.
>> >
>> > with best regards,
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to google-appengine@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>
>
> --
> Yours,
>
>
>
>
>
>
>
>
>
>
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Email Parser

[Cynthia]

I'm brand new to Google App Engine and I'm not really sure I can get
something that will do what I want. Or maybe I need to go elsewhere. I
am a programmer by trade but I am still a beginner in many areas.

I am looking for a way to receive emails and parse out certain
information, then save the info to a DB or a Spreadsheet.  A Google
Docs Spreadsheet would be fine, if that's possible.

Would Google App Engine be able to do this? Which language would be
best?  Maybe something like this already exists that I could use?

It's for a club membership list. People sign up with an HTML form, and
the form emails the information to an email address. I want to pluck
the data out of the email and have it save automatically.

Thanks,
Cynthia

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: a happy developer

[Vivek Puri]

+1

On Nov 30, 4:59 pm, alex  wrote:
> I've seen many people bashing GAE dev team, especially lately.
>
> What I want to say here is, I've been using it for years now and it's
> pretty awesome what you guys have been doing and where you've gone
> today. Of course there are bugs and stuff you need to find workarounds
> for but it is nowhere near compared to the burden of managing a
> cluster of EC2 on AWS or pay a double to Heroku or RackSpace even if
> they're great platforms too.
>
> I'm using GAE whenever I can and wherever it fits but I often find
> myself stuck with companies that by all means want to keep their data
> on the servers behind their desks (almost literally) and that's where
> one truly understands what it means to design, maintain and keep up
> with any sorts of upgrades of a multitenant infrastructure. IT'S HARD.
>
> So, this is from a (very happy) developer on GAE, AWS, Heroku,
> RightScale and an admin of a custom designed in-house multitenant
> infrastructure to GAE team: you're doing an awesome job. Keep it up!
>
> alex.
>
> PS Brandon, please no self-promotional replies with 10k-a-day clients
> and porn sites. thanks.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

RE: [google-appengine] Google Bot is the Enemy but I'd like to make him my friend

[Brandon Wirtz]

If you aren't running a proxy you are doing it wrong.

 

Apache, IIS, GAE, well gae has edgecache so maybe it could do a little less.

 

You should always have Squid, Forefront TMG or CDNinabox doing Page level
caching.

 

Of course Google doesn't need mine.  But if your app serves the same page
more than once to a user you should be doing PLC. Are you? Do you have code
to do so?  Wouldn't you rather just have a class/api that says lives like
WebAPP2 and you say make all this output PLC'ed with expiration of Time X?

 

http://www.youtube.com/watch?v=PnKCZctn8TM

 

 

From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Barry Hunter
Sent: Thursday, December 01, 2011 6:47 AM
To: google-appengine@googlegroups.com
Subject: Re: [google-appengine] Google Bot is the Enemy but I'd like to make
him my friend

 

Surely very few people (other than the spammers!) actully run a proxy. Ie
they dont have 'full rendered pages' stored in datastore. 

 

A page will be dynamic, and/or built from components. eg, the page text
might be the datastore, but the template is rendered seperately. (no need to
store it redundantly) 

 

GoogleBot wants the full page, as a webuser would see it. Not just the
'juicy bit' content. 

 

They already run a proxy, they dont need yours. 

-- 
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] 100 vs 2000 email quota

[Leandro Rezende]

do u have your 100 working? be happy...

i have 200 to be used, but none of they work.

Quota consumed but never delivered... it was working last week, but it
stopped again.

2011/12/1 Joshua Smith 

> Who do I contact about getting my email quota fixed? It is still at 100,
> but we were told that if we migrated an M/S app with 2000 to an HR app,
> you'd grandfather in the 2000 quota.
>
> Old app ID was towngovernment, new is towngovernment-hr
>
> -Joshua
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] 100 vs 2000 email quota

[Joshua Smith]

Who do I contact about getting my email quota fixed? It is still at 100, but we 
were told that if we migrated an M/S app with 2000 to an HR app, you'd 
grandfather in the 2000 quota.

Old app ID was towngovernment, new is towngovernment-hr

-Joshua

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Doom day

[Kaan Soral]

 * "The 50% discount for frontend instance hours will expire on December 
1st. Please check your Billing History page and update your 
budgetif
 necessary."

Since Python 2.7 is problematic, we have already been paying a lot with %50 
discounted prices ... My daily costs went up to 90$ from 40$ before new 
pricing, now it will be ~160$s ... 4x ...
*

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/xd3yViF2pggJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Google Bot is the Enemy but I'd like to make him my friend

[Barry Hunter]

Surely very few people (other than the spammers!) actully run a proxy. Ie
they dont have 'full rendered pages' stored in datastore.

A page will be dynamic, and/or built from components. eg, the page text
might be the datastore, but the template is rendered seperately. (no need
to store it redundantly)

GoogleBot wants the full page, as a webuser would see it. Not just the
'juicy bit' content.

They already run a proxy, they dont need yours.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Typesafe JDOQL

[AAF]

Ok great news - but how did you do that?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/Mr6Vo5yxTMEJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: The responses of my GAE app is much faster than before.

[Tapir]

btw, the price of GAE is still too high.

On Dec 1, 8:37 pm, Tapir  wrote:
> Have you GAE team ever optimized the instance scheduler? hehe.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] The responses of my GAE app is much faster than before.

[Tapir]

Have you GAE team ever optimized the instance scheduler? hehe.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.