[google-appengine] Re: RequestTooLargeError

2010-11-10 Thread mattbeedle
Ok, I see, I thought that fetch was just a straight forward HTTP POST,
but now looking at the documentation I see that is part of the
appengine python API (I'm not a python programmer).  Thanks for the
email limits link.  So, that's the first part of my question answered,
I definitely now understand the problem.  It would be really great if
someone could suggest me a way around this issue now.  Maybe I can
just use httplib to post directly?

Thanks,

Matt

On Nov 8, 7:17 pm, Robert Kluin robert.kl...@gmail.com wrote:
 Sorry, but re-read the bullet point about the size limit increases.
 Specifically the last sentence:
    Note that API requests (e.g. memcache.set(), db.put()) are still
 limited to 1MB in size.

 http://code.google.com/appengine/docs/python/mail/overview.html#Quota...

 Robert







 On Mon, Nov 8, 2010 at 04:49, mattbeedle mattbee...@googlemail.com wrote:
  I'm having trouble with request size limiting, but I don't understand
  why.  Im this blog post (http://googleappengine.blogspot.com/2009/02/
  skys-almost-limit-high-cpu-is-no-more.html) and it seems to say that
  the limits on requests and responses were increased to 10mb.  My
  application receives emails and then forwards them on to a website I
  run using urlfetch.  Whenever I send an email with an attachment over
  1mb, I see theseRequestTooLargeErrorerrors.  Here is the actual
  piece of code that is failing, the error is occurring on the response
  =fetchline:

  import logging, email, yaml
  from django.utils import simplejson as json
  from google.appengine.ext import webapp
  from google.appengine.ext.webapp.mail_handlers import
  InboundMailHandler
  from google.appengine.api.urlfetch importfetch
  from google.appengine.api.urlfetch import Error as FetchError

  settings = yaml.load(open('settings.yaml'))

  def callback(raw):
   result = {'email': {'raw': raw}}

   response =fetch(settings['outbound_url'],
               payload=json.dumps(result),
               method=POST,
               headers={
                 'Authorization': settings['api_key'],
                 'Content-Type': 'application/json'
               },
               deadline=10
              )
   logging.info(response.status_code)
   if response.status_code != 200:
     raise FetchError()

  class InboundHandler(InboundMailHandler):
   def receive(self, message):
     logging.info(Received a message from:  + message.sender)
     callback(message.original.as_string(True))

  It would be great if someone could explain to me why this is not
  working and how I can get around it, thanks.

  --
  You received this message because you are subscribed to the Google Groups 
  Google App Engine group.
  To post to this group, send email to google-appeng...@googlegroups.com.
  To unsubscribe from this group, send email to 
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Re: RequestTooLargeError

2010-11-10 Thread Tim Hoffman
Hi

urllib/2 and everything else (httplib) etc..  all sit on top of
urlfetch.

You have to work within the confines of urlfetch.

Rgds

T

On Nov 10, 9:30 pm, mattbeedle mattbee...@googlemail.com wrote:
 Ok, I see, I thought that fetch was just a straight forward HTTP POST,
 but now looking at the documentation I see that is part of the
 appengine python API (I'm not a python programmer).  Thanks for the
 email limits link.  So, that's the first part of my question answered,
 I definitely now understand the problem.  It would be really great if
 someone could suggest me a way around this issue now.  Maybe I can
 just use httplib to post directly?

 Thanks,

 Matt

 On Nov 8, 7:17 pm, Robert Kluin robert.kl...@gmail.com wrote:







  Sorry, but re-read the bullet point about the size limit increases.
  Specifically the last sentence:
     Note that API requests (e.g. memcache.set(), db.put()) are still
  limited to 1MB in size.

 http://code.google.com/appengine/docs/python/mail/overview.html#Quota...

  Robert

  On Mon, Nov 8, 2010 at 04:49, mattbeedle mattbee...@googlemail.com wrote:
   I'm having trouble with request size limiting, but I don't understand
   why.  Im this blog post (http://googleappengine.blogspot.com/2009/02/
   skys-almost-limit-high-cpu-is-no-more.html) and it seems to say that
   the limits on requests and responses were increased to 10mb.  My
   application receives emails and then forwards them on to a website I
   run using urlfetch.  Whenever I send an email with an attachment over
   1mb, I see theseRequestTooLargeErrorerrors.  Here is the actual
   piece of code that is failing, the error is occurring on the response
   =fetchline:

   import logging, email, yaml
   from django.utils import simplejson as json
   from google.appengine.ext import webapp
   from google.appengine.ext.webapp.mail_handlers import
   InboundMailHandler
   from google.appengine.api.urlfetch importfetch
   from google.appengine.api.urlfetch import Error as FetchError

   settings = yaml.load(open('settings.yaml'))

   def callback(raw):
    result = {'email': {'raw': raw}}

    response =fetch(settings['outbound_url'],
                payload=json.dumps(result),
                method=POST,
                headers={
                  'Authorization': settings['api_key'],
                  'Content-Type': 'application/json'
                },
                deadline=10
               )
    logging.info(response.status_code)
    if response.status_code != 200:
      raise FetchError()

   class InboundHandler(InboundMailHandler):
    def receive(self, message):
      logging.info(Received a message from:  + message.sender)
      callback(message.original.as_string(True))

   It would be great if someone could explain to me why this is not
   working and how I can get around it, thanks.

   --
   You received this message because you are subscribed to the Google Groups 
   Google App Engine group.
   To post to this group, send email to google-appeng...@googlegroups.com.
   To unsubscribe from this group, send email to 
   google-appengine+unsubscr...@googlegroups.com.
   For more options, visit this group 
   athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Re: RequestTooLargeError

2010-11-10 Thread mattbeedle
ok, thanks for you response, but I do not know python and I really
could do with a little help here.  I think we have established that
fetch will not work for me, thank you. It would be really great then
if someone could suggest to me a different way of solving this issue.
I just want to get the raw email code into my application, I'm sure
there must be a simple way to do this!

Thanks,

Matt

On Nov 10, 4:09 pm, Tim Hoffman zutes...@gmail.com wrote:
 Hi

 urllib/2 and everything else (httplib) etc..  all sit on top of
 urlfetch.

 You have to work within the confines of urlfetch.

 Rgds

 T

 On Nov 10, 9:30 pm, mattbeedle mattbee...@googlemail.com wrote:







  Ok, I see, I thought thatfetchwas just a straight forward HTTP POST,
  but now looking at the documentation I see that is part of the
  appengine python API (I'm not a python programmer).  Thanks for the
  email limits link.  So, that's the first part of my question answered,
  I definitely now understand the problem.  It would be really great if
  someone could suggest me a way around this issue now.  Maybe I can
  just use httplib to post directly?

  Thanks,

  Matt

  On Nov 8, 7:17 pm, Robert Kluin robert.kl...@gmail.com wrote:

   Sorry, but re-read the bullet point about the size limit increases.
   Specifically the last sentence:
      Note that API requests (e.g. memcache.set(), db.put()) are still
   limited to 1MB in size.

  http://code.google.com/appengine/docs/python/mail/overview.html#Quota...

   Robert

   On Mon, Nov 8, 2010 at 04:49, mattbeedle mattbee...@googlemail.com 
   wrote:
I'm having trouble with request size limiting, but I don't understand
why.  Im this blog post (http://googleappengine.blogspot.com/2009/02/
skys-almost-limit-high-cpu-is-no-more.html) and it seems to say that
the limits on requests and responses were increased to 10mb.  My
application receives emails and then forwards them on to a website I
run using urlfetch.  Whenever I send an email with an attachment over
1mb, I see theseRequestTooLargeErrorerrors.  Here is the actual
piece of code that is failing, the error is occurring on the response
=fetchline:

import logging, email, yaml
from django.utils import simplejson as json
from google.appengine.ext import webapp
from google.appengine.ext.webapp.mail_handlers import
InboundMailHandler
from google.appengine.api.urlfetch importfetch
from google.appengine.api.urlfetch import Error as FetchError

settings = yaml.load(open('settings.yaml'))

def callback(raw):
 result = {'email': {'raw': raw}}

 response =fetch(settings['outbound_url'],
             payload=json.dumps(result),
             method=POST,
             headers={
               'Authorization': settings['api_key'],
               'Content-Type': 'application/json'
             },
             deadline=10
            )
 logging.info(response.status_code)
 if response.status_code != 200:
   raise FetchError()

class InboundHandler(InboundMailHandler):
 def receive(self, message):
   logging.info(Received a message from:  + message.sender)
   callback(message.original.as_string(True))

It would be great if someone could explain to me why this is not
working and how I can get around it, thanks.

--
You received this message because you are subscribed to the Google 
Groups Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group 
athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



Re: [google-appengine] Re: RequestTooLargeError

2010-11-10 Thread Robert Kluin
Hey Matt,
  My first thoughts: If you run the other site, maybe you can break
the email into parts then reassemble the parts at the other end.
Maybe it would be easier to break the email up and store the parts on
AE.  Then send a request to your other site to 'fetch' the email from
your AE app.




Robert






On Wed, Nov 10, 2010 at 11:22, mattbeedle mattbee...@googlemail.com wrote:
 ok, thanks for you response, but I do not know python and I really
 could do with a little help here.  I think we have established that
 fetch will not work for me, thank you. It would be really great then
 if someone could suggest to me a different way of solving this issue.
 I just want to get the raw email code into my application, I'm sure
 there must be a simple way to do this!

 Thanks,

 Matt

 On Nov 10, 4:09 pm, Tim Hoffman zutes...@gmail.com wrote:
 Hi

 urllib/2 and everything else (httplib) etc..  all sit on top of
 urlfetch.

 You have to work within the confines of urlfetch.

 Rgds

 T

 On Nov 10, 9:30 pm, mattbeedle mattbee...@googlemail.com wrote:







  Ok, I see, I thought thatfetchwas just a straight forward HTTP POST,
  but now looking at the documentation I see that is part of the
  appengine python API (I'm not a python programmer).  Thanks for the
  email limits link.  So, that's the first part of my question answered,
  I definitely now understand the problem.  It would be really great if
  someone could suggest me a way around this issue now.  Maybe I can
  just use httplib to post directly?

  Thanks,

  Matt

  On Nov 8, 7:17 pm, Robert Kluin robert.kl...@gmail.com wrote:

   Sorry, but re-read the bullet point about the size limit increases.
   Specifically the last sentence:
      Note that API requests (e.g. memcache.set(), db.put()) are still
   limited to 1MB in size.

  http://code.google.com/appengine/docs/python/mail/overview.html#Quota...

   Robert

   On Mon, Nov 8, 2010 at 04:49, mattbeedle mattbee...@googlemail.com 
   wrote:
I'm having trouble with request size limiting, but I don't understand
why.  Im this blog post (http://googleappengine.blogspot.com/2009/02/
skys-almost-limit-high-cpu-is-no-more.html) and it seems to say that
the limits on requests and responses were increased to 10mb.  My
application receives emails and then forwards them on to a website I
run using urlfetch.  Whenever I send an email with an attachment over
1mb, I see theseRequestTooLargeErrorerrors.  Here is the actual
piece of code that is failing, the error is occurring on the response
=fetchline:

import logging, email, yaml
from django.utils import simplejson as json
from google.appengine.ext import webapp
from google.appengine.ext.webapp.mail_handlers import
InboundMailHandler
from google.appengine.api.urlfetch importfetch
from google.appengine.api.urlfetch import Error as FetchError

settings = yaml.load(open('settings.yaml'))

def callback(raw):
 result = {'email': {'raw': raw}}

 response =fetch(settings['outbound_url'],
             payload=json.dumps(result),
             method=POST,
             headers={
               'Authorization': settings['api_key'],
               'Content-Type': 'application/json'
             },
             deadline=10
            )
 logging.info(response.status_code)
 if response.status_code != 200:
   raise FetchError()

class InboundHandler(InboundMailHandler):
 def receive(self, message):
   logging.info(Received a message from:  + message.sender)
   callback(message.original.as_string(True))

It would be great if someone could explain to me why this is not
working and how I can get around it, thanks.

--
You received this message because you are subscribed to the Google 
Groups Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group 
athttp://groups.google.com/group/google-appengine?hl=en.

 --
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appeng...@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.