[google-appengine] Re: Newbie GAE questions on pricing
I dont want to behave badly, but .. Any thoughts on this would be very helpful for my well being .. :) Thank you -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Xui3SCNvGCQJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Any suggestions for migrating an application from Google Apps Script to Google App Engine?
Hi Joshua, I am trying to make it easier to get a database app up and running on Google App Engine without having to write code. Check it out at : http://creator.ifreetools.com If iFreeTools Creator meets your requirements, you also have the option to go in for a single-tenant deployment (iFreeTools code will be deployed into your Google App Engine app instance). Apart from enabling you to use the free quota, you can further purchase additional resources directly from Google. It is true that storage is really very affordable and you will have to look out for CPU usage. iFreeTools can automatically cache views if they are going to take more CPU (enabled in our free plan, disabled by default in single-tenant deployment). The single-tenant deployment of iFreeTools offers free-trial which will enable you to get an idea of the resource usage and billing, before you can commit on taking this path. Thanks & Regards, R.Rajkumar On Wed, Aug 3, 2011 at 10:03 PM, Joshua Mulloy wrote: > Hello, > > I have been creating an application in Google Apps Script but have run into > a wall with Session.getUser() disabled for us Education Edition users. I > would still prefer not to serve the application from our own servers, but > that is a known system and I am running out of time to develop a solution. > I am looking into rewriting the app in GAE and was hoping for tips and > recommendations. > > I like Python better but I wonder if I might be able to salvage some of the > code from GAS if I use Java instead. Is there an easy way to migrate the > spreadsheet data to the datastore or should I just use csv? > > I am nervous about the quota system. Storage size is easier to predict but > bandwidth and CPU has me concerned. Are there any examples of applications > and the resources they used? > > I have setup a GAE account as myself, should I create it under the > role-based account I was writing the GAS instead? > > Thank you for any guidance you can provide! > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/dbBcc3ZF4HoJ. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- ~~ Build online database applications, over Google App Engine. iFreeTools Creator - http://creator.ifreetools.com -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: which custom HTTP header names allowed?
Hi, I think there is a standard Date header, right? Or perhaps you could use something like X-Request-Time. I agree, a list of headers would be good. I think the headers they filter are mostly things related to App Engine -- task headers, appid, etc Robert On Thu, Aug 4, 2011 at 01:46, Carter Maslan wrote: > thanks Robert; X-Forwarded-For is a great suggestion and thanks for > verifying that it's not filtered. > We also need to pass a timestamp representing the start of the client's > request. > So I'm still interested in any Google guidelines on which custom headers are > safe from their filtering generally. > thanks again, > Carter > > > On Wed, Aug 3, 2011 at 10:18 PM, Robert Kluin > wrote: >> >> Hi Carter, >> If it is a proxy how about using X-Forwarded-For? I tested using >> curl, it doesn't get filtered. >> >> >> Robert >> >> >> >> >> >> >> >> On Wed, Aug 3, 2011 at 12:04, Carter wrote: >> > Any info on GAE-acceptable names for custom HTTP headers? >> > We can use User-Agent as workaround, but we'd like to know our >> > options. >> > >> > >> > On Jul 27, 3:58 pm, Carter wrote: >> >> We have a proxy that needs to send the client's IP address to GAE. >> >> We tried setting acustomHTTPheader of "X-client-ip", but that seems >> >> to get filtered by GAE. >> >> We could include the info in use User-Agent, but that seems hack-ish. >> >> >> >> The doc (http://code.google.com/appengine/docs/java/runtime.html) >> >> says: >> >> RequestHeaders >> >> "An incomingHTTPrequest includes theHTTPheaderssent by the >> >> client. For security purposes, someheadersare sanitized or amended >> >> by intermediate proxies before they reach the application." >> >> >> >> So whatcustomheader name can we use? >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups "Google App Engine" group. >> > To post to this group, send email to google-appengine@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com. >> > For more options, visit this group at >> > http://groups.google.com/group/google-appengine?hl=en. >> > >> > >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: which custom HTTP header names allowed?
thanks Robert; X-Forwarded-For is a great suggestion and thanks for verifying that it's not filtered. We also need to pass a timestamp representing the start of the client's request. So I'm still interested in any Google guidelines on which custom headers are safe from their filtering generally. thanks again, Carter On Wed, Aug 3, 2011 at 10:18 PM, Robert Kluin wrote: > Hi Carter, > If it is a proxy how about using X-Forwarded-For? I tested using > curl, it doesn't get filtered. > > > Robert > > > > > > > > On Wed, Aug 3, 2011 at 12:04, Carter wrote: > > Any info on GAE-acceptable names for custom HTTP headers? > > We can use User-Agent as workaround, but we'd like to know our > > options. > > > > > > On Jul 27, 3:58 pm, Carter wrote: > >> We have a proxy that needs to send the client's IP address to GAE. > >> We tried setting acustomHTTPheader of "X-client-ip", but that seems > >> to get filtered by GAE. > >> We could include the info in use User-Agent, but that seems hack-ish. > >> > >> The doc (http://code.google.com/appengine/docs/java/runtime.html) > >> says: > >> RequestHeaders > >> "An incomingHTTPrequest includes theHTTPheaderssent by the > >> client. For security purposes, someheadersare sanitized or amended > >> by intermediate proxies before they reach the application." > >> > >> So whatcustomheader name can we use? > > > > -- > > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Automated way to switch from M/S to HR?
Preserving appid is what I am waiting on. Readonly is fine. I hope Ikai is correct saying we can preserve them. T -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/4etJODZ6kKAJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Any suggestions for migrating an application from Google Apps Script to Google App Engine?
Hi Joshua, Google Apps Script is just running JavaScript on the client isn't it? Unless you're going to try doing something clever with Rhino (or some other similar solution) I'm not sure Java will offer you any more than Python. Have you looked at the gdata-apis (http://code.google.com/apis/gdata/)? Once your app is running on GAE you can access spreadsheets stored in Google Docs using the gdata libs. Of course, depending on your app, it may be easier to directly use the datastore instead of a spreadsheet. It is very hard to say anything about what your app may cost to run on GAE. Also don't forget the pricing will soon be changing, no one yet knows how it will change things (other than becoming more expensive). Robert On Wed, Aug 3, 2011 at 12:33, Joshua Mulloy wrote: > Hello, > I have been creating an application in Google Apps Script but have run into > a wall with Session.getUser() disabled for us Education Edition users. I > would still prefer not to serve the application from our own servers, but > that is a known system and I am running out of time to develop a solution. > I am looking into rewriting the app in GAE and was hoping for tips and > recommendations. > I like Python better but I wonder if I might be able to salvage some of the > code from GAS if I use Java instead. Is there an easy way to migrate the > spreadsheet data to the datastore or should I just use csv? > I am nervous about the quota system. Storage size is easier to predict but > bandwidth and CPU has me concerned. Are there any examples of applications > and the resources they used? > I have setup a GAE account as myself, should I create it under the > role-based account I was writing the GAS instead? > Thank you for any guidance you can provide! > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/dbBcc3ZF4HoJ. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Verify ownership web master central problem with authentication google account redirect
You'll get more responses if you type your questions properly. Answer on SO. http://stackoverflow.com/questions/6929488/verify-ownership-web-master-central-problem-with-authentication-google-account-re/6930142#6930142 On Wed, Aug 3, 2011 at 11:54, Alexander Herrera wrote: > I'm trying to verify my site with a domain in appspot.com but i use > the authentication whit google account when the web master central try > to verify appears an error "We weren't able to verify your site: " > cuz . Your verification file redirects to a disallowed location , the > redirect goes to https://www.google.com/accounts/ServiceLogin?... so > how can i verify my site ?, can i disabled de athentication? or what > is the best way ... I hope some one can help me. Thx. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: which custom HTTP header names allowed?
Hi Carter, If it is a proxy how about using X-Forwarded-For? I tested using curl, it doesn't get filtered. Robert On Wed, Aug 3, 2011 at 12:04, Carter wrote: > Any info on GAE-acceptable names for custom HTTP headers? > We can use User-Agent as workaround, but we'd like to know our > options. > > > On Jul 27, 3:58 pm, Carter wrote: >> We have a proxy that needs to send the client's IP address to GAE. >> We tried setting acustomHTTPheader of "X-client-ip", but that seems >> to get filtered by GAE. >> We could include the info in use User-Agent, but that seems hack-ish. >> >> The doc (http://code.google.com/appengine/docs/java/runtime.html) >> says: >> RequestHeaders >> "An incomingHTTPrequest includes theHTTPheaderssent by the >> client. For security purposes, someheadersare sanitized or amended >> by intermediate proxies before they reach the application." >> >> So whatcustomheader name can we use? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Backends always fail on the dev server
Hello,
I've been working with the Backends API and tried to run the
counter_demo and am always unsuccessful. Each instance spits out an
error about the connection being refused to /_ah/start. Here's one
example for an instance of loadtest:
[Backend Instance] [loadtest.7] [dev_appserver.py:4248] INFO "GET /_ah/
start HTTP/1.1" 500 -
[Backend Instance] [counter.0] [dev_appserver.py:4201] ERROR Exception
encountered handling request
Traceback (most recent call last):
File "C:\Program Files\Google\google_appengine\google\appengine\tools
\dev_appserver.py", line 4134, in _HandleRequest
dev_appserver_index.SetupIndexes(config.application, root_path)
File "C:\Program Files\Google\google_appengine\google\appengine\tools
\dev_appserver_index.py", line 304, in SetupIndexes
existing_indexes = datastore_admin.GetIndices(app_id)
File "C:\Program Files\Google\google_appengine\google\appengine\api
\datastore_admin.py", line 53, in GetIndices
resp = _Call('GetIndices', req, resp)
File "C:\Program Files\Google\google_appengine\google\appengine\api
\datastore_admin.py", line 102, in _Call
result = apiproxy_stub_map.MakeSyncCall('datastore_v3', call, req,
resp)
File "C:\Program Files\Google\google_appengine\google\appengine\api
\apiproxy_stub_map.py", line 94, in MakeSyncCall
return stubmap.MakeSyncCall(service, call, request, response)
File "C:\Program Files\Google\google_appengine\google\appengine\api
\apiproxy_stub_map.py", line 308, in MakeSyncCall
rpc.CheckSuccess()
File "C:\Program Files\Google\google_appengine\google\appengine\api
\apiproxy_rpc.py", line 156, in _WaitImpl
self.request, self.response)
File "C:\Program Files\Google\google_appengine\google\appengine\ext
\remote_api\remote_api_stub.py", line 252, in MakeSyncCall
response)
File "C:\Program Files\Google\google_appengine\google\appengine\ext
\remote_api\remote_api_stub.py", line 178, in MakeSyncCall
self._MakeRealSyncCall(service, call, request, response)
File "C:\Program Files\Google\google_appengine\google\appengine\ext
\remote_api\remote_api_stub.py", line 190, in _MakeRealSyncCall
encoded_response = self._server.Send(self._path, encoded_request)
File "C:\Program Files\Google\google_appengine\google\appengine\tools
\appengine_rpc.py", line 365, in Send
f = self.opener.open(req)
File "C:\Program Files\Python\lib\urllib2.py", line 381, in open
response = self._open(req, data)
File "C:\Program Files\Python\lib\urllib2.py", line 399, in _open
'_open', req)
File "C:\Program Files\Python\lib\urllib2.py", line 360, in
_call_chain
result = func(*args)
File "C:\Program Files\Python\lib\urllib2.py", line 1107, in
http_open
return self.do_open(httplib.HTTPConnection, req)
File "C:\Program Files\Python\lib\urllib2.py", line 1082, in do_open
raise URLError(err)
URLError:
Does anyone have any idea how to fix this?
Thanks,
Richard
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] 407 Proxy Authentication Required for ISA
Are you sure this is something returned from your App Engine application and not something on your side? On Wed, Aug 3, 2011 at 03:42, vineeth ml wrote: > We have created a webservice using PROTORPC and we get json response > in different methods. > But when trying to access this webservice from other domains we get > the " 407 Proxy Authentication Required for ISA server" error. Can we > add domains that are allowed to access this service.Is there any > workaround for this. > How can i provide my webservice to the mass. > > Regards > Vineeth > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Getting Started
I just debug in GAE... Deploy/update rather than Run. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of samueldc Sent: Wednesday, August 03, 2011 1:04 PM To: Google App Engine Subject: [google-appengine] Getting Started Hi everybody, I trying to getting started with Google App Engine and Eclipse 3.7. After creating the project Guestbook, I'm getting the following error when trying to execute with Run > Debug As > Web Application: Usage: [options] Options: --help, -h Show this help message and exit. --server=SERVERThe server to use to determine the latest -s SERVER SDK version. --address=ADDRESS The address of the interface on the local machine -a ADDRESS to bind to (or 0.0.0.0 for all interfaces). --port=PORTThe port number to bind to on the local machine. -p PORT --sdk_root=rootOverrides where the SDK is located. --disable_update_check Disable the check for newer SDK versions. And I get a 404 error when trying http://localhost:8080/guestbook Anyone could please help me? SDC -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] How to find some to create an app. for me
I don't know anyone who works on a percentage, especially if you are just providing the idea. Even as a Product Manager in most companies you can't be app illiterate, you need to at minimum be able to read code, write product specs, and possibly create wire frames. GAE's Hello world examples are not that hard, start building. When you have something that mostly works, partner with a dev to finish it, and share in the profits and glory. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of michael manning Sent: Wednesday, August 03, 2011 12:51 PM To: Google App Engine Subject: [google-appengine] How to find some to create an app. for me I am completely apps. illiterate, though I feel I have great ideas for apps. How can I find people who could create the apps. and guide me thru the business for a percentage of the income? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] App Engine office hours for 2011/07/03 [PDT]
Chat log from today's App Engine office hours. All times are in AEST (UTC+10). 12:01 -!- Irssi: #appengine: Total of 115 nicks [0 ops, 0 halfops, 2 voices, 113 normal] 12:01 < jwbnyc> Thanks for joining, us Wesley! 12:01 < Wesley_Google> who? where? 12:01 < robertk> Wesley_Google: there -> 12:01 < robertk> ;) 12:02 <+gregdarke> Hey all 12:02 < johnlockwood> howdy 12:02 < mbw> ok, ive got one. Have you guys made any changes (improvements) in parallel db.get across entity groups on HRD? We had a developer testing async db.get and we were just wondering if that has gotten better yet. 12:02 < robertk> so what do you work on dave_google? don't remember seeing you around here 12:02 < dave_google> i'm on app engine 12:03 < dave_google> wes usually schedules these for when i can't make it. :) 12:03 < Wesley_Google> mbw> you mean batch get by key? 12:03 < jwbnyc> For our part, we're butting up against issues of code size. We're starting to encounter difficulty due to instance startup times. 12:03 < mbw> Wesley_Google: yep 12:03 < robertk> yeah which part of AE? gregdarke is a task queue guy, wesley is dev rel... just want to know which questions to direct at you :) 12:04 < dave_google> a bit of this, a bit of that. ask your question, and the right person will chime in. 12:05 < Wesley_Google> thanks dave!! (no, these are pre-scheduled all the time... it's either 7-8p PDT or 9-10a PDT) 12:05 < mbw> Wesley_Google: I know we have spoken to you guys about this before and you said things where being done to improve performance of that... since it was very slow comparead to master slave (when we first started using it... we havent done much benchmarking lately and didnt really notice a change) 12:05 < dave_google> 9am exists as an abstraction. 12:07 < jwbnyc> Has anyone been able to get a handle on where the initialization time goes? Our effort have shown wide differences between the Java dev server and GAE in terms of how our initial request performs. 12:07 < ronoaldojlp> I'm experiencing some instances startup latency, and posted an issue today. Does someone else is having the same issues too? 12:07 < robertk> mbw: when i was poking around in the sdk i did notice that they are putting in the code to auto parallelize the rpcs now 12:08 < ronoaldojlp> our Java instances usually take 3-6 sec to startup, but some of them throws (Hard)DeadlineExceededExceptions 12:09 < jwbnyc> ronoaldojlp: We've been trying to track down a similar set of issues. What's the URL for the issue you raised? 12:09 < mbw> robertk: right, but thats in the case where it would be too large to do it in one put right? 12:10 < robertk> mbw: nah, looked like something more general to me. judging by the comments at least 12:10 < mbw> robertk: which could be a bit scary actually since you could be writing them all to the same entitiy group, thus contention 12:10 < ronoaldojlp> jwbnyc: http://code.google.com/p/googleappengine/issues/detail?id=5477 12:10 < robertk> mbw: haven't dug as deeply as i wanted to yet though 12:10 < mbw> ya, who knows... frigen docs would be nice 12:10 < robertk> mbw: yeah there logic seemed to take entity groups into account 12:11 < dave_google> create an issue for where you want to see docs expanded. those do get listened to. 12:11 < robertk> mbw: seemed to break up the rpcs based on entity group -- there is now na entity_grpups_per_rpc (or something similar) config option 12:11 < jwbnyc> Thanks, Ron! I posted to the Appengine for Java group: https://groups.google.com/d/topic/google-appengine-java/rWfC6cypiwg/discussion 12:12 < robertk> mbw: here is one part of what i saw today: http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/datastore/datastore_rpc.py#488 12:12 < robertk> dave_google: mbw and i probably dig in a bit deeper than normal ;) 12:12 < robertk> dave_google: if you could get the internal comments left in when you push the code that would be great ;) 12:13 < ronoaldojlp> jwbnyc: we are trying to do some lazy initialization but not sure if there is any good "patter" on that area... we are using Google Guice to do a lot of work on that. 12:15 < JasonAtBobber> Anyone have thoughts on how best to profile GAE apps? Our initialization times are fairly short when run locally, but can be crazy-long when on AppEngine. Anyone use anything other than Appstats? AppWrench looked promising, except it appears to be dead. ;) 12:16 < ronoaldojlp> Googlers, is the "min idle instances" knob of the scheduler avaiable as preview? We're expecting a huge traffic tomorrow for a TV commercial and we thing that it may be a good Idea to leave some instances waiting ... Can I "emulate" it in a good way, by warming up some instances? 12:17 < robertk> what other scheduler config settings will be rolling out? i know min-idle-instances is planned. what about additional information to help us making decisions on scheduler settings, will there be more logging or analysis charts? 12:17 < robertk> and what i
Re: [google-appengine] Persisting Lists
Reference properties are a completely separate idea from entity groups (ie ancestor / parent - child relationships). Reference properties simply store a key; that key can be used to fetch the other entity. They can be changed to point at a different entity as often as you like. An entity's entity group (aka ancestor) is determined by its key -- this can not be changed. One advantage of entity groups is that they allow you to operate on entities within a given group transactionally. Robert On Wed, Aug 3, 2011 at 12:49, Bruno Sandivilli wrote: > So, if an user X adds and User Y, the user Y will become an ancestor of X ? > And if the user Y adds the user W too, so user X will have two > ancestors(???) ? Any code snippets will be greatful. > Thanks again, for the help guys. > > 2011/8/2 MiuMeet Support >> >> By the way, have a look >> at: http://devblog.miumeet.com/2011/08/much-more-efficient-implementation-of.html >> It's much more efficient than db.ListProperty(int) >> Cheers, >> -Andrin >> >> On Thu, Jul 28, 2011 at 6:17 PM, Pascal Voitot Dev >> wrote: >>> >>> good idea also :) >>> >>> On Wed, Jul 27, 2011 at 7:17 PM, Ernesto Oltra >>> wrote: And for followers, you could too shard the lists. You can have several entities, each with, about 100 results or so (or 1000, or 2000, I prefer 100 for easy of serializing/deserializing). All these would have the user as ancestor. When listing, take only one entity, deserializing its lists (only 100 results) and show some of them. When listing all, you can use cursors and some tricks to have the job done (job = paging =) ) -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/tRQCOATsWdUJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine" group. >>> To post to this group, send email to google-appengine@googlegroups.com. >>> To unsubscribe from this group, send email to >>> google-appengine+unsubscr...@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Automated way to switch from M/S to HR?
I felt your pain on MS; but after moving to HRD Tuesday, our app has not experienced a single timeout error in 24 hours. So don't give up. On Aug 3, 2011, at 2:25 PM, David wrote: > I've just about given up on the thought that AppEngine is a good > service. It's good in theory, but I get about 25% errors in bursts > and it's generally timeouts on the datastore. Those error bursts are > frequent. Lately is been almost daily. But pretty much every upgrade > they do has a couple weeks before it starts to feel stable again. The > bad part is we have no control over the servers so we just have to eat > the errors as they come since they are coming from the datastore. I > need a reliable/stable environment where I feel I can do something > about problems when they come up. So I'm probably making the switch, > just off of AppEngine. > > On Aug 3, 1:02 pm, Will Reiher wrote: >> Ikai, >> >> That's great news! I'm not so worried about a temporary read-only period >> it's the the idea of switching the application id that was scary. I've >> already done it once, and while it was fairly painless, it did affect our >> https requests point to the app-id appspot url. >> >> Thanks for the update - I'm sure many other people will be making the >> switch. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Automated way to switch from M/S to HR?
I've not heard of the ability to allow you preserve the app id as far as the datastore is concerned, only the ability to create an alias so it acts like the same app. I wonder if an alias is what Ikai was referring too or if he meant some way to really preserve the app id. Robert On Wed, Aug 3, 2011 at 20:50, johnP wrote: > Amen - temporary read-only is fine. The big problem is that my app > makes use of a ton of keys, and in many cases I need to store the keys > as strings (or store keys in pickled blobs). The real chore is to > write scripts to rebuild all the keys after the migration. > > I'm hoping that preserving the app ID will also preserve these > keys... > > please??? > > johnP > > > > > On Aug 3, 1:02 pm, Will Reiher wrote: >> Ikai, >> >> That's great news! I'm not so worried about a temporary read-only period >> it's the the idea of switching the application id that was scary. I've >> already done it once, and while it was fairly painless, it did affect our >> https requests point to the app-id appspot url. >> >> Thanks for the update - I'm sure many other people will be making the >> switch. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Stackoverflow
AFAIK , questions are filtered based on ontopic or offtopic and not if they are technological or not. so if there is a different category for appengine, I think most of the related question can be answered there, and they need not be a technological question. On Thu, Aug 4, 2011 at 1:26 AM, Vivek Puri wrote: > Most of the ideas and answers(objective and discussion categorization) > here are great. To give an example, yesterday i needed to know if > regex is supported in objective c. First search on google lead to this > - > > > http://stackoverflow.com/questions/422138/regular-expressions-in-an-objective-c-cocoa-application > > And in 2 mins, i knew what was needed to be known. Any similar answers > given here by Ikai, Nick or Robert are just lost. This is a big waste > on part of everyone who is contributing. > > Just to add. Few days back i wanted to get to a particular comment on > the post where Pricing changes were announced, and it took me quite a > bit of searches and keyword combination that resulted in nothing. > Eventually i had to email AppEngine team for information. > > > > On Aug 2, 6:04 pm, Stephen Johnson wrote: > > It is done. > > > > On Tue, Aug 2, 2011 at 2:29 PM, Ikai Lan (Google) > wrote: > > > > > > > > > > > > > > > > > So I saw this in my Google Alerts earlier, and I thought, what the > heck, > > > people ask development questions on Yahoo Answers? > > > > >http://answers.yahoo.com/question/index?qid=20110726102419AAFUzgy > > > > > Can someone with a Yahoo account point him here? I realized that I > haven't > > > logged into Yahoo in years and can't seem to remember my password. > > > > > FWIW, I've always though Yahoo Answers was for questions like this: > > > > >http://www.funnyyahooquestions.com/sons-birthday-fail/ > > > > > (I'm joking, of course. I've been in some pretty idiotic Google Groups. > > > It's the people; not the technology.) > > > > > -- > > > Ikai Lan > > > Developer Programs Engineer, Google App Engine > > > plus.ikailan.com | twitter.com/ikai > > > > > On Tue, Aug 2, 2011 at 5:42 AM, Gary Frederick wrote: > > > > >> I was forced to vote for Groups over Stackoverflow. My real vote is > both. > > > > >> Stackoverflow is ok for a focused question that does not get booted > > >> Groups are better if you have a question that does not fit into the > > >> criteria for Stackoverflow > > >> or you are more having a conversation. > > > > >> I do a lot with Google's App Inventor groups. We looked a bit at > > >> Stackoverflow a while ago and it did not fit. > > >> we get a lot more less technical types there... > > > > >> and I have failed to check my Stackoverflow account in a while. It was > not > > >> encouraging to me to see how much karma I would have to build in a > place > > >> with limited use to me to add the pages or whatever they call them to > get > > >> the App Inventor info there to where it should be to be useful. I > wanted to > > >> be able to point back to the Google forums and those messages got > crushed by > > >> the SO cops... > > > > >> and > > > > >> your milage may differ :-) > > > > >> Gary > > > > >> -- > > >> You received this message because you are subscribed to the Google > Groups > > >> "Google App Engine" group. > > >> To view this discussion on the web visit > > >>https://groups.google.com/d/msg/google-appengine/-/r33pUmrjRIkJ. > > > > >> To post to this group, send email to > google-appengine@googlegroups.com. > > >> To unsubscribe from this group, send email to > > >> google-appengine+unsubscr...@googlegroups.com. > > >> For more options, visit this group at > > >>http://groups.google.com/group/google-appengine?hl=en. > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Google App Engine" group. > > > To post to this group, send email to google-appengine@googlegroups.com > . > > > To unsubscribe from this group, send email to > > > google-appengine+unsubscr...@googlegroups.com. > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Can't download all data with bulkloader from non-default namespace
Yes, I fill it. http://code.google.com/p/googleappengine/issues/detail?id=5486 3 августа 2011 г. 15:46 пользователь Johan Euphrosine написал: > Hi Timofey, > > I managed to reproduce your issue: > proppy@proppy:~$ ~/google_appengine/appcfg.py download_data > --application=s~proppy-guestbook-namespace > --url=https://proppy-guestbook-namespace.appspot.com/_ah/remote_api > --namespace="foo" --filename=foo.data > Downloading data records. > [INFO] Logging to bulkloader-log-20110803.134504 > [INFO] Throttling transfers: > [INFO] Bandwidth: 25 bytes/second > [INFO] HTTP connections: 8/second > [INFO] Entities inserted/fetched/modified: 20/second > [INFO] Batch Size: 10 > [INFO] Opening database: bulkloader-progress-20110803.134504.sql3 > [INFO] Opening database: bulkloader-results-20110803.134504.sql3 > [INFO] Connecting to > proppy-guestbook-namespace.appspot.com/_ah/remote_api > [ERROR ] Unable to download kind stats for all-kinds download. > [ERROR ] Kind stats are generated periodically by the appserver > [ERROR ] Kind stats are not available on dev_appserver. > > Looks like a bug, can you fill it on > http://code.google.com/p/googleappengine/issues/entry and followup > with the url there ? > > Thanks in advance. > > On Tue, Aug 2, 2011 at 4:51 PM, Timofey Koolin > wrote: > > Yes, I can. My app ID: my-optimiser (my-optimiser.appspot.com). > > It isn't very need personal for me now - I have only one Entity type and > I > > can download it with --kind option, but I think it is bug in platform or > > SDK. > > Now I try use https://my-optimiser.appspot.com instead of > > https://3.my-optimiser.appspot.com - it have same error. > > > > 2011/8/2 Johan Euphrosine > >> > >> Can you share your application id, so we can investigate the issue ? > >> > >> Feel free to fill a production issue if this is directly affecting > >> your operation: > >> > >> > http://code.google.com/p/googleappengine/issues/entry?template=Production%20issue > >> > >> On Sat, Jul 30, 2011 at 10:19 AM, rekby > wrote: > >> > I use Python SDK 1.5.2. High-replication datastore. I see data > statistic > >> > for > >> > namespace 2 in dashboard. > >> > appcfg.py download_data --application=s~my-optimiser > >> > --url=https://3.my-optimiser.appspot.com/remote_api --namespace="2" > >> > --filename=optimiser.data > >> > Downloading data records. > >> > [INFO] Logging to bulkloader-log-20110730.121721 > >> > [INFO] Throttling transfers: > >> > [INFO] Bandwidth: 25 bytes/second > >> > [INFO] HTTP connections: 8/second > >> > [INFO] Entities inserted/fetched/modified: 20/second > >> > [INFO] Batch Size: 10 > >> > [INFO] Opening database: bulkloader-progress-20110730.121721.sql3 > >> > [INFO] Opening database: bulkloader-results-20110730.121721.sql3 > >> > [INFO] Connecting to 3.my-optimiser.appspot.com/remote_api > >> > [ERROR ] Unable to download kind stats for all-kinds download. > >> > [ERROR ] Kind stats are generated periodically by the appserver > >> > [ERROR ] Kind stats are not available on dev_appserver. > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "Google App Engine" group. > >> > To view this discussion on the web visit > >> > https://groups.google.com/d/msg/google-appengine/-/fN3ISosu8ZgJ. > >> > To post to this group, send email to > google-appengine@googlegroups.com. > >> > To unsubscribe from this group, send email to > >> > google-appengine+unsubscr...@googlegroups.com. > >> > For more options, visit this group at > >> > http://groups.google.com/group/google-appengine?hl=en. > >> > > >> > >> > >> > >> -- > >> Johan Euphrosine (proppy) > >> Developer Programs Engineer > >> Google Developer Relations > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Google App Engine" group. > >> To post to this group, send email to google-appengine@googlegroups.com. > >> To unsubscribe from this group, send email to > >> google-appengine+unsubscr...@googlegroups.com. > >> For more options, visit this group at > >> http://groups.google.com/group/google-appengine?hl=en. > >> > > > > > > > > -- > > С уважением, > > Кулин Тимофей. > > > > Телефон: +7 (4852) 974793 > > ICQ: 114902104 > > email: timo...@koolin.ru > > Blog: http://timofey.koolin.ru > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group at > > http://groups.google.com/group/google-appengine?hl=en. > > > > > > -- > Johan Euphrosine (proppy) > Developer Programs Engineer > Google Developer Relations > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To
Re: [google-appengine] Re: "The request to API call datastore_v3.Put() was too large."
How large is your put to datastore? On 4 August 2011 02:45, Will wrote: > It's on MS datastore. The program hasn't been changed. Seems started > happening several hours ago. > > On Wed, Aug 3, 2011 at 8:44 AM, Will wrote: >> Hi all, >> >> Some of my tasks suddenly started to report this error: >> >> error - > 'google.appengine.runtime.apiproxy_errors.RequestTooLargeError'>. The >> request to API call datastore_v3.Put() was too large. >> >> When I checked, the payloads of the tasks are usually under 200 bytes. >> >> Anything I should look into? >> >> Thanks, >> >> Will >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Greg Darke -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Automated way to switch from M/S to HR?
Amen - temporary read-only is fine. The big problem is that my app makes use of a ton of keys, and in many cases I need to store the keys as strings (or store keys in pickled blobs). The real chore is to write scripts to rebuild all the keys after the migration. I'm hoping that preserving the app ID will also preserve these keys... please??? johnP On Aug 3, 1:02 pm, Will Reiher wrote: > Ikai, > > That's great news! I'm not so worried about a temporary read-only period > it's the the idea of switching the application id that was scary. I've > already done it once, and while it was fairly painless, it did affect our > https requests point to the app-id appspot url. > > Thanks for the update - I'm sure many other people will be making the > switch. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Getting Started
Hi everybody, I trying to getting started with Google App Engine and Eclipse 3.7. After creating the project Guestbook, I'm getting the following error when trying to execute with Run > Debug As > Web Application: Usage: [options] Options: --help, -h Show this help message and exit. --server=SERVERThe server to use to determine the latest -s SERVER SDK version. --address=ADDRESS The address of the interface on the local machine -a ADDRESS to bind to (or 0.0.0.0 for all interfaces). --port=PORTThe port number to bind to on the local machine. -p PORT --sdk_root=rootOverrides where the SDK is located. --disable_update_check Disable the check for newer SDK versions. And I get a 404 error when trying http://localhost:8080/guestbook Anyone could please help me? SDC -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Great ideas
I have great ideas for apps. that could generate a nice income. Looking for a person who could create these apps. as on either a partner for a % of income, straight pay for building the app. for me. I am completely app. illiterate and need someone who not only is technically savy, but knows the ins and outs of the business end to guide me. I have no problem being very generous to the right person,, Please, no thiefs. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] How to find some to create an app. for me
I am completely apps. illiterate, though I feel I have great ideas for apps. How can I find people who could create the apps. and guide me thru the business for a percentage of the income? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Persisting Lists
Sorry, but I don't use Java. I will give you the Python version, it's pretty
straight forward. As I said before, any improvements in my design or code
will be welcome =)
class User(Model):
# for easy of use, i will consider user_id as the key name of the entity
name, email, etc
num_of_followers = IntegerProperty(default=0)
class FollowerInfo(Model):
master = StringProperty() I've used String property to store the key_name,
but you can use KeyProperty if you prefer. This will be the super-star
posting new things
followers = ListProperty() # or arrays, the other thing MiuMeet proposed
full = BooleanProperty() # whether we can store more followers here or not
class Post(Model):
content, author, etc...
class PostNotify(Model):
# key name/key_id of the entity will be key_name/key_id of the post
users = ListProperty()
*User X follows User Y:*
# Check if there is already a FollowerInfo model, and hasn't got more than
100 followers or so.
follower = Query(..).filter('full = ', False).filter('master',
UserHere).fetch(1)
if not follower: # create one if we need it
follower = FollowerInfo(parent=UserHere, followers=[,])
# Add follower, check if the info is full too
follower.followers.append(FollowerUserHere)
if len(follower.followers) >= 100:
follower.full = True
follower.put() # save
# maybe this must be in a transaction?
user.num_of_followers += 1
user.put()
*
*
*User X doesn't follow User Y anymore:*
*
*
follower = Query(...).filter('followers = ', FollowerUserHere).fetch(1)
follower.followers.delete(FollowerUserHere)
# check if the model was full, now it will have one empty slot more
if follower.full and len(follower.followers) < 100:
follower.full = False
follower.put() # save
# again, maybe this must be in a transaction?
user.num_of_followers -= 1
user.put()
*List followers:*
*
*
# I fetch one of the following info models. It may be null (no followers),
one model(1-100 followers). If the user has more following info models
(100-infinite), it will return only one of them.
follower = Query(..).filter('master =', UserHere).fetch(1)
if follower:
# List here the follower.followers
*Post:*
*
*
if UserHere.num_of_followers > 100:
taskqueue() # add new task to process it offline, too much followers!
# process it inline here otherwise (same code as in the task)
TASK:
- Get each one of the FollowerInfo models for the user
- Create a PostNotify with the key_name/key_id of the post for each 4500 or
so, followers.
- Save all the PostNotify entities
*List notifications:*
*
*
# I run a keys-only query, finding the first 50 (or whatever you want
notifications for this user).
# As you don't have to deserialize the list (keys-only), you'll never incur
in that perfomance cost. The key has all the info you need (the post
key_name/key_id, it's exactly the same)
notifications = Query(entity=PostNotify, keys_only=True).filter('users =',
CurrentUserHere).fetch(50)
ids = []
for notification in notifications
ids.append(notification.key_name)
# Now you have all the posts
posts = db.get(ids)
2011/8/3 Bruno Sandivilli
> So, if an user X adds and User Y, the user Y will become an ancestor of X ?
> And if the user Y adds the user W too, so user X will have two
> ancestors(???) ? Any code snippets will be greatful.
> Thanks again, for the help guys.
>
>
> 2011/8/2 MiuMeet Support
>
>> By the way, have a look at:
>> http://devblog.miumeet.com/2011/08/much-more-efficient-implementation-of.html
>>
>> It's much more efficient than db.ListProperty(int)
>>
>> Cheers,
>> -Andrin
>>
>>
>> On Thu, Jul 28, 2011 at 6:17 PM, Pascal Voitot Dev <
>> pascal.voitot@gmail.com> wrote:
>>
>>> good idea also :)
>>>
>>>
>>> On Wed, Jul 27, 2011 at 7:17 PM, Ernesto Oltra
>>> wrote:
>>>
And for followers, you could too shard the lists. You can have several
entities, each with, about 100 results or so (or 1000, or 2000, I prefer
100
for easy of serializing/deserializing). All these would have the user as
ancestor. When listing, take only one entity, deserializing its lists (only
100 results) and show some of them. When listing all, you can use cursors
and some tricks to have the job done (job = paging =) )
--
You received this message because you are subscribed to the Google
Groups "Google App Engine" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/tRQCOATsWdUJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "Google App Engine" group.
>>> To post to this group, send email to google-appengine@googlegroups.com.
>>> To unsubscribe from this group,
[google-appengine] Re: Datastore outage?
I'm still seeing this right now on my app. On Aug 3, 8:21 am, Johan Euphrosine wrote: > Hi Hugo, > > There was a storage issue in one of our datacenter that affected M/S > application. > > This should now be fixed. > > This was being tracked > here:http://code.google.com/p/googleappengine/issues/detail?id=5476 > > If you still have this problem, feel free to open a new production > issue:http://code.google.com/p/googleappengine/issues/entry?template=Produc... > > Thanks in advance. > > On Wed, Aug 3, 2011 at 4:36 PM, Hugo wrote: > > I assume somebody is on top of it, but just in case: > > > I have been getting datastore write timeouts for approximately 1h30 > > now. The status dashboard does not show any issue with the datastore. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > Johan Euphrosine (proppy) > Developer Programs Engineer > Google Developer Relations -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Automated way to switch from M/S to HR?
I've just about given up on the thought that AppEngine is a good service. It's good in theory, but I get about 25% errors in bursts and it's generally timeouts on the datastore. Those error bursts are frequent. Lately is been almost daily. But pretty much every upgrade they do has a couple weeks before it starts to feel stable again. The bad part is we have no control over the servers so we just have to eat the errors as they come since they are coming from the datastore. I need a reliable/stable environment where I feel I can do something about problems when they come up. So I'm probably making the switch, just off of AppEngine. On Aug 3, 1:02 pm, Will Reiher wrote: > Ikai, > > That's great news! I'm not so worried about a temporary read-only period > it's the the idea of switching the application id that was scary. I've > already done it once, and while it was fairly painless, it did affect our > https requests point to the app-id appspot url. > > Thanks for the update - I'm sure many other people will be making the > switch. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Automated way to switch from M/S to HR?
Ikai, That's great news! I'm not so worried about a temporary read-only period it's the the idea of switching the application id that was scary. I've already done it once, and while it was fairly painless, it did affect our https requests point to the app-id appspot url. Thanks for the update - I'm sure many other people will be making the switch. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/vQDY_-rtfH0J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Automated way to switch from M/S to HR?
We're working on a migration tool that'll be better than the one that exists now. You will be able to preserve your current application ID. Unfortunately, there will still be a copy step, so this will likely result in a read-only period for your app. -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Wed, Aug 3, 2011 at 12:53 PM, Will Reiher wrote: > Since I have been having a number of issues with Deadlines on the M/S I > would like to move to HR but creating a new app and migrating data is really > not an option. Is the app engine team working on a click of a button type > upgrade that doesn't require duplication and/or migrating to a new > application-id etc? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/ymI6tGec5fYJ. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Stackoverflow
Most of the ideas and answers(objective and discussion categorization) here are great. To give an example, yesterday i needed to know if regex is supported in objective c. First search on google lead to this - http://stackoverflow.com/questions/422138/regular-expressions-in-an-objective-c-cocoa-application And in 2 mins, i knew what was needed to be known. Any similar answers given here by Ikai, Nick or Robert are just lost. This is a big waste on part of everyone who is contributing. Just to add. Few days back i wanted to get to a particular comment on the post where Pricing changes were announced, and it took me quite a bit of searches and keyword combination that resulted in nothing. Eventually i had to email AppEngine team for information. On Aug 2, 6:04 pm, Stephen Johnson wrote: > It is done. > > On Tue, Aug 2, 2011 at 2:29 PM, Ikai Lan (Google) wrote: > > > > > > > > > So I saw this in my Google Alerts earlier, and I thought, what the heck, > > people ask development questions on Yahoo Answers? > > >http://answers.yahoo.com/question/index?qid=20110726102419AAFUzgy > > > Can someone with a Yahoo account point him here? I realized that I haven't > > logged into Yahoo in years and can't seem to remember my password. > > > FWIW, I've always though Yahoo Answers was for questions like this: > > >http://www.funnyyahooquestions.com/sons-birthday-fail/ > > > (I'm joking, of course. I've been in some pretty idiotic Google Groups. > > It's the people; not the technology.) > > > -- > > Ikai Lan > > Developer Programs Engineer, Google App Engine > > plus.ikailan.com | twitter.com/ikai > > > On Tue, Aug 2, 2011 at 5:42 AM, Gary Frederick wrote: > > >> I was forced to vote for Groups over Stackoverflow. My real vote is both. > > >> Stackoverflow is ok for a focused question that does not get booted > >> Groups are better if you have a question that does not fit into the > >> criteria for Stackoverflow > >> or you are more having a conversation. > > >> I do a lot with Google's App Inventor groups. We looked a bit at > >> Stackoverflow a while ago and it did not fit. > >> we get a lot more less technical types there... > > >> and I have failed to check my Stackoverflow account in a while. It was not > >> encouraging to me to see how much karma I would have to build in a place > >> with limited use to me to add the pages or whatever they call them to get > >> the App Inventor info there to where it should be to be useful. I wanted to > >> be able to point back to the Google forums and those messages got crushed > >> by > >> the SO cops... > > >> and > > >> your milage may differ :-) > > >> Gary > > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "Google App Engine" group. > >> To view this discussion on the web visit > >>https://groups.google.com/d/msg/google-appengine/-/r33pUmrjRIkJ. > > >> To post to this group, send email to google-appengine@googlegroups.com. > >> To unsubscribe from this group, send email to > >> google-appengine+unsubscr...@googlegroups.com. > >> For more options, visit this group at > >>http://groups.google.com/group/google-appengine?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Automated way to switch from M/S to HR?
Since I have been having a number of issues with Deadlines on the M/S I would like to move to HR but creating a new app and migrating data is really not an option. Is the app engine team working on a click of a button type upgrade that doesn't require duplication and/or migrating to a new application-id etc? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/ymI6tGec5fYJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Crazy number of Deadine Exceeded exceptions recently
To further my original comment... While it seems to be back to an almost normal state when it was really bad I was getting DeadlineExceeded errors on Warm ups and other otherwise trivial requests. Most of my issues were not related to any offline task work - in fact for the most part tasks seemed to be the most stable. As I said originally this included deployments that had been stable for 30+ days and were all of a sudden throwing way more Deadline exceptions. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/FYWO7w_MtbgJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] full text search and/or external search services
I'm using http://www.elasticsearch.org/ hosted on Amazon EC2. works great! -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Jdslewu0TL8J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Crazy number of Deadine Exceeded exceptions recently
I'm also experiencing this problem. :( -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/1Xjp0jsnprYJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Deadline exceeded during warm-up
Is happening on our app a lot too. Seeming more and more like the Master/Slave datastore is not a reliable place to be... -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/3CYqFYkW7_0J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Crazy number of Deadine Exceeded exceptions recently
We have exactly the same issues. Since a few weeks we have downtimes of almost 30 minutes. Or the request latency increases. Today there was again a spike in errors (duration of 2 hours), during that time our application was not usable. This happens to often the last weeks. We are using Java, M/S, single-threaded. This should be fixed soon. Thank you, Ralph -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] full text search and/or external search services
Sorry, we're working on getting this out as quickly as we can. We're not close to trusted tester/beta period yet. In the past, I've done App Engine talking HTTP over the public interwebs to an instance running Solr. The latency was acceptable for what I was doing: 20ms to either Slicehost or Linode (I forget which). -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Wed, Aug 3, 2011 at 11:03 AM, andreas schmid wrote: > i got that you weren't suggesting that :) > > On Aug 3, 2011, at 5:19 PM, Robert Kluin wrote: > > > On Wed, Aug 3, 2011 at 03:32, andreas schmid > wrote: > >> i tried whoosh-appengine and as i mentioned before it fails after > indexing a few 1000 because the way it works and it exceeds the soft process > limit size. > > > > Sorry, I was suggesting to *not* use whoosh-appengine -- rather try > > 'real' whoosh on a backend with more RAM. I've seen an example of > > doing this before, though I don't recall where. I think it was quite > > straightforward. > > > > > > > >> > >> never tried it on a backend. right now i customized bill katz's simple > full text search and im really surprised how simple it is implemented and > how easy it is to modify it to your needs. > >> i think ill go on with this solution until gaes full text search will be > released. > >> > >> thx > >> > >> On Aug 3, 2011, at 7:17 AM, Robert Kluin wrote: > >> > >>> Have you tried running whoosh (not whoosh-appengine) on a backend? > >>> Backends are expensive, but might help you out a bit. > >>> > >>> > >>> > >>> Robert > >>> > >>> > >>> > >>> > >>> > >>> On Thu, Jul 28, 2011 at 11:19, andreas schmid > wrote: > hi guys, > > i really need to have a full text search on my app and i wanted to > know what kind of external services you are using on your apps. or how you > implement such a service. > i tried whoosh-appengine but first its a really old version and second > it has huge issues with 1000+ indexed entities. > > we don't know when the full text search will be released for GAE > right? > > thx > > -- > You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > To post to this group, send email to > google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > >>> > >>> -- > >>> You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > >>> To post to this group, send email to google-appengine@googlegroups.com > . > >>> To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > >>> For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > >>> > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > >> To post to this group, send email to google-appengine@googlegroups.com. > >> To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > >> For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > >> > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Data Center (CDN) Locations
Patrick, did you follow the page speed advice? The majority of what makes web sites appear slow is the loading of assets and JavaScript execution. Steve Souders (the father of web page speed) has been quoted as saying this can account for up to 90% of web site slowness. Obviously, the first time visiting a site where nothing is in the cache or if you are serving static assets from North America, this will cause the site to appear slow, but take a look at these best practices the next time you are testing in Japan. -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Wed, Aug 3, 2011 at 4:59 AM, Patrick Poon wrote: > Ikai, > > It sure would be great if Google would consider adding regional data > centers on the roadmap. I live in Boston, but travel to Japan a few > times a year. Most recently, I stayed in Yokohama for the month of > June, actively developing and testing my app the entire month. There > was a noticeable degradation in performance compared to what I > experience in Boston. Not just after calling my app the first time > after a period of non-activity, but even after on-going active use. > > The latency didn't make my app unusable but it was noticeable enough > to cause me concern as to how potential customers in Japan might > perceive the speed of my app. > > Cheers, > > Patrick > > On Aug 2, 2:54 pm, "Ikai Lan (Google)" wrote: > > The primary data centers serving App Engine are in North America, > however, > > assets and items that are edge cached are served from Google edge caches > > which are located wherever there are Google data centers. > > > > When evaluating page speed, it's typically assets that cause page speeds > to > > appear slow: > > > > http://stevesouders.com/hpws/rules.php > > > > Only dynamic calls to your application logic would be routed to North > > America. If you follow these rules, websites should still load quickly - > > there is a fairly large App Engine community in Japan. > > > > -- > > Ikai Lan > > Developer Programs Engineer, Google App Engine > > plus.ikailan.com | twitter.com/ikai > > > > > > > > > > > > > > > > On Tue, Aug 2, 2011 at 3:30 AM, Simon Knott > wrote: > > > Hi, > > > > > As far as I'm aware, the most accurate information the Google staff > have > > > divulged is that all data centres are located in North America > somewhere. > > > > > Cheers, > > > Simon > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Google App Engine" group. > > > To view this discussion on the web visit > > >https://groups.google.com/d/msg/google-appengine/-/OvHwAZFC9aYJ. > > > > > To post to this group, send email to google-appengine@googlegroups.com > . > > > To unsubscribe from this group, send email to > > > google-appengine+unsubscr...@googlegroups.com. > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Much more efficient implementation of db.ListProperty(int) - created a lib
is there a way to use this ArrayProperty to store a list of strings? On Aug 3, 2011, at 10:15 AM, MiuMeet Support wrote: > You can fit up to 131'072 elements in there, afterwards you hit the 1mb > entity limit (entity overhead aside, but 100'000 are def possible). > > Put and Get performance are really fantastic, 10ms each, see the blog post. > > -Andrin > > On Wed, Aug 3, 2011 at 9:23 AM, andreas schmid wrote: > does this also mean that there is no 5000 objects limit on the list? > is the get performance good as the put? > > thx for sharing this. > > On Aug 2, 2011, at 9:04 PM, Ikai Lan (Google) wrote: > >> Thanks Andrin! >> >> If anyone is wondering why this is, it's because the List property is >> serialized and deserialized using protocol buffers. Protocol buffers are >> very useful for things like validation and cross language compatibility, but >> you can sometimes gain some performance benefits by serializing and >> deserializing using native methods. The protocol buffer implementation for >> Python is noticeably slow, even via the native extension (if you can improve >> this - Google wants to hire you), so I'm not surprised techniques like this >> work. >> >> -- >> Ikai Lan >> Developer Programs Engineer, Google App Engine >> plus.ikailan.com | twitter.com/ikai >> >> >> >> On Tue, Aug 2, 2011 at 9:18 AM, Andrin von Rechenberg >> wrote: >> So you want to store a "very long list of numbers" in your AppEngine Model >> which doesn't need to be indexed. >> >> Well, you could just do this: >> >> class Foo(db.Model): >> numbers = db.ListProperty(int, indexed=False) >> >> However, you will quickly notice that the performance and the memory >> consumption of this sucks if you add thousands of integers to this list. >> >> Instead you should use the native python "array.array" type as the list and >> store it in a BlobProperty. It's up to 30x faster! >> >> I created a library that does exactly this (free, apache 2.0 license): >> http://devblog.miumeet.com/2011/08/much-more-efficient-implementation-of.html >> >> Once you have the library, all you need to do is this to get a huge >> performance boost: >> >> class Foo(db.Model): >> numbers = db.ArrayProperty() >> >> The blogpost also has more details and an appstat comparison. >> >> Cheers & hope you like it >> -Andrin >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] full text search and/or external search services
i got that you weren't suggesting that :) On Aug 3, 2011, at 5:19 PM, Robert Kluin wrote: > On Wed, Aug 3, 2011 at 03:32, andreas schmid wrote: >> i tried whoosh-appengine and as i mentioned before it fails after indexing a >> few 1000 because the way it works and it exceeds the soft process limit size. > > Sorry, I was suggesting to *not* use whoosh-appengine -- rather try > 'real' whoosh on a backend with more RAM. I've seen an example of > doing this before, though I don't recall where. I think it was quite > straightforward. > > > >> >> never tried it on a backend. right now i customized bill katz's simple full >> text search and im really surprised how simple it is implemented and how >> easy it is to modify it to your needs. >> i think ill go on with this solution until gaes full text search will be >> released. >> >> thx >> >> On Aug 3, 2011, at 7:17 AM, Robert Kluin wrote: >> >>> Have you tried running whoosh (not whoosh-appengine) on a backend? >>> Backends are expensive, but might help you out a bit. >>> >>> >>> >>> Robert >>> >>> >>> >>> >>> >>> On Thu, Jul 28, 2011 at 11:19, andreas schmid wrote: hi guys, i really need to have a full text search on my app and i wanted to know what kind of external services you are using on your apps. or how you implement such a service. i tried whoosh-appengine but first its a really old version and second it has huge issues with 1000+ indexed entities. we don't know when the full text search will be released for GAE right? thx -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine" group. >>> To post to this group, send email to google-appengine@googlegroups.com. >>> To unsubscribe from this group, send email to >>> google-appengine+unsubscr...@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Federated Login redirecting to Google OpenID and not /_ah/login_required on Java GAE
Hi Robert, Yes I did follow that example. I didn't realize there's a separate Java group. I'll try asking there. Thanks! Vishal On Aug 2, 9:58 pm, Robert Kluin wrote: > Hi Vishal, > Did you go through Wesley's example doc: > http://code.google.com/appengine/articles/openid.html > > You might try asking on the app engine java group. > > Robert > > > > On Tue, Aug 2, 2011 at 00:49, Vishal wrote: > > Anyone? I still can't get Federated Login to work correctly. > > > Vishal > > > On Jul 29, 12:05 am, Vishal wrote: > >> Hello Wise AppEngineers, > > >> I need your help to figure out what I'm doing wrong. I've enabled > >> Federated Login for my Java app, but whenever I visit a security- > >> constraint protected url-pattern, I get redirected to a default Google > >> OpenID login screen. I want my /_ah/login_required servlet handler to > >> be called instead. The documentation says this is default behavior, > >> but that's not happening for me. > > >> I'm aware of the potential infinite redirect issue (that's not what > >> I'm facing). Here's how to reproduce the behavior I'm seeing: > >> 1) Visithttp://demo.hotels-z.appspot.com/_ah/login_required. Verify > >> that the servlet is installed and serves a page. > >> 2) Visithttp://demo.hotels-z.appspot.com/manage. Note that I'm not > >> redirected to my _ah/login_required handler but some default Google > >> OpenID handler. > > >> Here are parts of my web.xml: > >> > >> LoginRequiredServlet > >> /_ah/login_required > >> > > >> > >> > >> /manage/* > >> > >> > >> * > >> > >> > > >> What am I doing wrong? > > >> Thanks! > > >> Vishal > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Persisting Lists
So, if an user X adds and User Y, the user Y will become an ancestor of X ? And if the user Y adds the user W too, so user X will have two ancestors(???) ? Any code snippets will be greatful. Thanks again, for the help guys. 2011/8/2 MiuMeet Support > By the way, have a look at: > http://devblog.miumeet.com/2011/08/much-more-efficient-implementation-of.html > > It's much more efficient than db.ListProperty(int) > > Cheers, > -Andrin > > > On Thu, Jul 28, 2011 at 6:17 PM, Pascal Voitot Dev < > pascal.voitot@gmail.com> wrote: > >> good idea also :) >> >> >> On Wed, Jul 27, 2011 at 7:17 PM, Ernesto Oltra wrote: >> >>> And for followers, you could too shard the lists. You can have several >>> entities, each with, about 100 results or so (or 1000, or 2000, I prefer 100 >>> for easy of serializing/deserializing). All these would have the user as >>> ancestor. When listing, take only one entity, deserializing its lists (only >>> 100 results) and show some of them. When listing all, you can use cursors >>> and some tricks to have the job done (job = paging =) ) >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine" group. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msg/google-appengine/-/tRQCOATsWdUJ. >>> >>> To post to this group, send email to google-appengine@googlegroups.com. >>> To unsubscribe from this group, send email to >>> google-appengine+unsubscr...@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: "The request to API call datastore_v3.Put() was too large."
It's on MS datastore. The program hasn't been changed. Seems started happening several hours ago. On Wed, Aug 3, 2011 at 8:44 AM, Will wrote: > Hi all, > > Some of my tasks suddenly started to report this error: > > error - 'google.appengine.runtime.apiproxy_errors.RequestTooLargeError'>. The > request to API call datastore_v3.Put() was too large. > > When I checked, the payloads of the tasks are usually under 200 bytes. > > Anything I should look into? > > Thanks, > > Will > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Any suggestions for migrating an application from Google Apps Script to Google App Engine?
Hello, I have been creating an application in Google Apps Script but have run into a wall with Session.getUser() disabled for us Education Edition users. I would still prefer not to serve the application from our own servers, but that is a known system and I am running out of time to develop a solution. I am looking into rewriting the app in GAE and was hoping for tips and recommendations. I like Python better but I wonder if I might be able to salvage some of the code from GAS if I use Java instead. Is there an easy way to migrate the spreadsheet data to the datastore or should I just use csv? I am nervous about the quota system. Storage size is easier to predict but bandwidth and CPU has me concerned. Are there any examples of applications and the resources they used? I have setup a GAE account as myself, should I create it under the role-based account I was writing the GAS instead? Thank you for any guidance you can provide! -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/dbBcc3ZF4HoJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Crazy number of Deadine Exceeded exceptions recently
I can confirm this problem too, specially when a new instance is spinning up. But the erros are sporadically happenning. Some instances starts in 3-6 seconds, others throws DeadLine/HardDeadLine exceptions after lots of time ... I opened a production issue, sine no changes in the app code were made. Our app is using Java, single-threaded, on M/S. I know, this is the worse case possible :-/ We are enabling threads this week, and planing the migration to HR... Best Regards, -Ronoaldo -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/wEweTQMTca8J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Crazy number of Deadine Exceeded exceptions recently
just an update on this matter (I'm working with Alexis): - the application we are talking about is fp-dreamland. - it's not a minor issue for us as it impact directly our users. - we have been observing that for almost two weeks now. - it's not a small application either, we are talking about more than an hundred instances being killed at once. - we are paying customer, and growing fast, it would be really nice to have some feedback from google people. Thanks, Olivier On Aug 3, 4:45 am, Alexis wrote: > Some more details while searching the logs: > > The latest burst (we have about one every 1-2 hours) shows this: > We had about 90 instances up. > We got DeadlineExceeded error during two minutes. During the first > 30seconds, 50 successive error requests have no pending_ms. Then > during the next 1m30 they nearly all have pending_ms and one third are > loading requests. > > So my guess is that it's not linked to the scheduler: > something goes wrong and requests are killing the instances during the > first 30 seconds, then killing the remaining ones with an increased > latency until what goes wrong is resolved. > > On 3 août, 10:20, Alexis wrote: > > > > > > > > > We are using Python, and we are not using back-ends or taskqueue. > > But most of our requests fetch entities of the same kind, so a "kind- > > locking" can be relevant. > > I'll setup timeouts on datastore operations to see if this is what is > > going wrong (and looks good to setup these anyway). > > > Seems logical to have pending_ms values if the app have no instances > > available instead of the one hundred that used to serve the traffic... > > however requests should be able to perform in less than 20sec. Nearly > > all the requests failing with this error, and while the instances get > > killed off, have this pending_ms value, and don't have the > > loading_request=1. > > So I'm not sure whether these DeadlineExceeded errors come first or as > > a consequence of the instances being killed: they are not > > loading_request and the warning in the logs says it may kill the > > instance, but we have these pending_ms showing that we already lack of > > instances. > > > The traffic is very steady. > > > On 3 août, 05:49, Robert Kluin wrote: > > > > Interesting. I've been seen exactly the same strange behavior across > > > several apps as well. Suddenly instances will get killed and > > > restarted in large batches. This happens even with low request > > > latency, small memory usage (similar to yours < 50mb), low error > > > rates, and steady traffic. I pretty convinced this is tied to the > > > scheduler changes they've been making over the past few weeks. > > > > As a side note, the pending_ms value (9321) indicates that the request > > > sat there waiting to be serviced for quite a long time. That won't > > > leave as much time to respond to the requests. Do you always see > > > bursts of those when your instances get killed off? Are you getting > > > big spikes in traffic when this happens or is it steady? > > > > Robert > > > > On Tue, Aug 2, 2011 at 05:24, Alexis wrote: > > > > Hi, > > > > > I've got a similar issue: lots of DeadlineExceeded errors since a few > > > > weeks. I'm on the master-slave datastore too, but what I'm reporting > > > > happened again one hour ago. > > > > > These errors happen in bursts, and I recently realized that it was in > > > > fact shutting down ALL instances of the application. > > > > (In the logs, I also have this warning: A serious problem was > > > > encountered with the process that handled this request, causing it to > > > > exit. This is likely to cause a new process to be used for the next > > > > request to your application. If you see this message frequently, you > > > > may be throwing exceptions during the initialization of your > > > > application. (Error code 104)) > > > > This does not happen when an instance is spinning up but after several > > > > hours. > > > > > The trace I get along with the DeadlineExceeded errors show that it > > > > happens in the second phase: while the app is trying to fallback > > > > gracefully because of an other error (that does not appears in logs). > > > > Request reported processing time can be like this: ms=100878 > > > > cpu_ms=385 api_cpu_ms=58 cpm_usd=0.010945 pending_ms=9321 > > > > > Here is a screenshot of the admin page, showing that all instances > > > > have been shut down about 7 minutes ago, even resident ones: > > > >http://dl.dropbox.com/u/497622/spinDown.png > > > > > The app do work in batches (although not always small ones). But > > > > request processing time is usually good enough (see average latency on > > > > the screen shot). > > > > I'm trying things on my testing applications to see what can be wrong > > > > but it's still not clear for me and I'm running short of ideas... > > > > > Any suggestions? > > > > > On 2 août, 06:21, Robert Kluin wrote: > > > >> Hi Will, > > > >> I assume this is on the master-slave datasto
[google-appengine] Verify ownership web master central problem with authentication google account redirect
I'm trying to verify my site with a domain in appspot.com but i use the authentication whit google account when the web master central try to verify appears an error "We weren't able to verify your site: " cuz . Your verification file redirects to a disallowed location , the redirect goes to https://www.google.com/accounts/ServiceLogin?... so how can i verify my site ?, can i disabled de athentication? or what is the best way ... I hope some one can help me. Thx. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: which custom HTTP header names allowed?
Any info on GAE-acceptable names for custom HTTP headers? We can use User-Agent as workaround, but we'd like to know our options. On Jul 27, 3:58 pm, Carter wrote: > We have a proxy that needs to send the client's IP address to GAE. > We tried setting acustomHTTPheader of "X-client-ip", but that seems > to get filtered by GAE. > We could include the info in use User-Agent, but that seems hack-ish. > > The doc (http://code.google.com/appengine/docs/java/runtime.html) > says: > RequestHeaders > "An incomingHTTPrequest includes theHTTPheaderssent by the > client. For security purposes, someheadersare sanitized or amended > by intermediate proxies before they reach the application." > > So whatcustomheader name can we use? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] "The request to API call datastore_v3.Put() was too large."
Hi all, Some of my tasks suddenly started to report this error: error - . The request to API call datastore_v3.Put() was too large. When I checked, the payloads of the tasks are usually under 200 bytes. Anything I should look into? Thanks, Will -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Datastore outage?
Hi Hugo, There was a storage issue in one of our datacenter that affected M/S application. This should now be fixed. This was being tracked here: http://code.google.com/p/googleappengine/issues/detail?id=5476 If you still have this problem, feel free to open a new production issue: http://code.google.com/p/googleappengine/issues/entry?template=Production%20issue Thanks in advance. On Wed, Aug 3, 2011 at 4:36 PM, Hugo wrote: > I assume somebody is on top of it, but just in case: > > I have been getting datastore write timeouts for approximately 1h30 > now. The status dashboard does not show any issue with the datastore. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Johan Euphrosine (proppy) Developer Programs Engineer Google Developer Relations -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] full text search and/or external search services
On Wed, Aug 3, 2011 at 03:32, andreas schmid wrote: > i tried whoosh-appengine and as i mentioned before it fails after indexing a > few 1000 because the way it works and it exceeds the soft process limit size. Sorry, I was suggesting to *not* use whoosh-appengine -- rather try 'real' whoosh on a backend with more RAM. I've seen an example of doing this before, though I don't recall where. I think it was quite straightforward. > > never tried it on a backend. right now i customized bill katz's simple full > text search and im really surprised how simple it is implemented and how easy > it is to modify it to your needs. > i think ill go on with this solution until gaes full text search will be > released. > > thx > > On Aug 3, 2011, at 7:17 AM, Robert Kluin wrote: > >> Have you tried running whoosh (not whoosh-appengine) on a backend? >> Backends are expensive, but might help you out a bit. >> >> >> >> Robert >> >> >> >> >> >> On Thu, Jul 28, 2011 at 11:19, andreas schmid wrote: >>> hi guys, >>> >>> i really need to have a full text search on my app and i wanted to know >>> what kind of external services you are using on your apps. or how you >>> implement such a service. >>> i tried whoosh-appengine but first its a really old version and second it >>> has huge issues with 1000+ indexed entities. >>> >>> we don't know when the full text search will be released for GAE right? >>> >>> thx >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine" group. >>> To post to this group, send email to google-appengine@googlegroups.com. >>> To unsubscribe from this group, send email to >>> google-appengine+unsubscr...@googlegroups.com. >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine?hl=en. >>> >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] A server error occurred. Please contact the administrator.
Hi Federico, Can you fill a production issue ? http://code.google.com/p/googleappengine/issues/entry Thanks in advance. On Wed, Aug 3, 2011 at 1:53 PM, Federico Elles wrote: > Getting the message > A server error occurred. Please contact the administrator. > for all write operations. > > Status is green: http://code.google.com/status/appengine > > App: 3x3links > > Thanks. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Johan Euphrosine (proppy) Developer Programs Engineer Google Developer Relations -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Error 500s on deploy
Please comment on the issue with your applicaiton id if you are still unable to deploy: http://code.google.com/p/googleappengine/issues/detail?id=5474 Thanks in advance. On Wed, Aug 3, 2011 at 4:30 PM, Santiago López de Haro wrote: > +1 > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Johan Euphrosine (proppy) Developer Programs Engineer Google Developer Relations -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] App Engine Update Downtime
This issue is tracked there: http://code.google.com/p/googleappengine/issues/detail?id=5474 If you still have this issue, please comment with your application id. Thanks in advance. On Wed, Aug 3, 2011 at 4:00 PM, LSH wrote: > Starting update of app: *, version: no-underscore > Scanning files on local disk. > Scanned 500 files. > Could not guess mimetype for static/favicon.ico. Using application/ > octet-stream. > Cloning 207 static files. > Cloned 100 files. > Cloned 200 files. > Cloning 377 application files. > Cloned 100 files. > Cloned 200 files. > Rolling back the update. > Error 500: --- begin server output --- > > > > 500 Server Error > > > Error: Server Error > The server encountered an error and could not complete your > request.If the problem persists, please report your problem and > mention this error message and the query that caused it. > > > --- end server output --- > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Johan Euphrosine (proppy) Developer Programs Engineer Google Developer Relations -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Problems Deploying App - 500 Internal server Error
This issue is tracked there: http://code.google.com/p/googleappengine/issues/detail?id=5474 If you still have this issue, please comment with your application id. On Wed, Aug 3, 2011 at 4:40 PM, sebastián serrano wrote: > me too! > Sebastian > www.devsar.com > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/Fr9MotdBMf8J. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- Johan Euphrosine (proppy) Developer Programs Engineer Google Developer Relations -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Datastore outage?
I assume somebody is on top of it, but just in case: I have been getting datastore write timeouts for approximately 1h30 now. The status dashboard does not show any issue with the datastore. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Error 500s on deploy
+1 -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
R: Re: [google-appengine] Re: Can't deploy app. Stops on cloning
Thank you for reply and details, now my app is upgradable again. Martino -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/FFpV9qwvXjwJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Problems Deploying App - 500 Internal server Error
me too! Sebastian www.devsar.com -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Fr9MotdBMf8J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Error 500s on deploy
Sorry. No Replication Datastore but still my app si not updatable -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/DHQ96IKrCw0J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Cannot deploy app, cannot modify any settings
This issue is tracked there: http://code.google.com/p/googleappengine/issues/detail?id=5474 If it is still not working for you can you add a comment with your appid ? Thanks in advance. On Wed, Aug 3, 2011 at 1:34 PM, Pambo Paschalides wrote: > Today, whenever I try to deploy my app, I get the following: > ... > Closing update: new version is ready to start serving. > Rolling back the update. > Error 503: --- begin server output --- > Application Busy (503) > Multiple clients are trying to modify the same application. > --- end server output --- > Moreover, I cannot modify any settings in my app. For example disabling the > app, or modifying in anything, results in message "Application > busy. Multiple clients are trying to modify the same application.". > What is going on here? :S > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/Ha5DuRac-CYJ. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- Johan Euphrosine (proppy) Developer Programs Engineer Google Developer Relations -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Error 500s on deploy
This issue is tracked there: http://code.google.com/p/googleappengine/issues/detail?id=5474 - If your app is using M/S this should now be fixed. - If your app is using HDR please open a new production issue with your application id. Thanks in advance. On Wed, Aug 3, 2011 at 3:40 PM, Francois Masurel wrote: > Same here. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/SaqNaquYGFcJ. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- Johan Euphrosine (proppy) Developer Programs Engineer Google Developer Relations -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Can't deploy app. Stops on cloning
This issue is tracked there: http://code.google.com/p/googleappengine/issues/detail?id=5474 - If your app is using M/S this should now be fixed. - If your app is using HDR please open a new production issue with your application id. Thanks in advance. On Wed, Aug 3, 2011 at 3:56 PM, Harald Schilly wrote: > uhm no, deploying a new version doesn't work for me. it takes a long time to > process and i guess there is an internal timeout. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/XBtmd6txOHwJ. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- Johan Euphrosine (proppy) Developer Programs Engineer Google Developer Relations -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] App Engine Update Downtime
Starting update of app: *, version: no-underscore Scanning files on local disk. Scanned 500 files. Could not guess mimetype for static/favicon.ico. Using application/ octet-stream. Cloning 207 static files. Cloned 100 files. Cloned 200 files. Cloning 377 application files. Cloned 100 files. Cloned 200 files. Rolling back the update. Error 500: --- begin server output --- 500 Server Error Error: Server Error The server encountered an error and could not complete your request.If the problem persists, please report your problem and mention this error message and the query that caused it. --- end server output --- -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: JDO schema change nightmare, app going live
+1 to getting away from JDO. I know I'm not helping you specifically so sorry about that, but GAE development for me was one struggle after another until I tossed JDO. GAE and JDO are NOT a good fit. On Aug 3, 12:18 am, Pascal Voitot Dev wrote: > You can also give a try to Sienahttp://www.sienaproject.com > It provides the same level of features as Objectify just above Datastore. > But it's not specialized to GAE and aims at providing a bridge between NoSQL > DB (and also SQL DB when it's possible). > Currently we support GAE (sync/async) + MySQL/Postgres/H2. > We also provide high-level features around iteration, pagination (based on > cursors, forward & backward)... > We also recently added GAE entity groups and entity direct embedding > support. > > SimpleDB is being coded right now and is already quite advanced! > Redis is under preparation. > > regards > Pascal > > PS: i'm lead dev of Siena, don't hesitate to contact me if you have > questions! > > On Wed, Aug 3, 2011 at 12:04 AM, nischalshetty > wrote: > > > > > > > > > I would recommend you to scrap JDO completely and use objectify. We were in > > a similar situation as you, changing over to objectify was the best thing > > ever! And it's easier than you could ever imagine. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To view this discussion on the web visit > >https://groups.google.com/d/msg/google-appengine/-/8DgsKuEO1jwJ. > > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Can't deploy app. Stops on cloning
uhm no, deploying a new version doesn't work for me. it takes a long time to process and i guess there is an internal timeout. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/XBtmd6txOHwJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: full text search and/or external search services
We are using indextank.com for our service and it works perfectly. On Aug 3, 10:32 am, andreas schmid wrote: > i tried whoosh-appengine and as i mentioned before it fails after indexing a > few 1000 because the way it works and it exceeds the soft process limit size. > > never tried it on a backend. right now i customized bill katz's simple full > text search and im really surprised how simple it is implemented and how easy > it is to modify it to your needs. > i think ill go on with this solution until gaes full text search will be > released. > > thx > > On Aug 3, 2011, at 7:17 AM, Robert Kluin wrote: > > > > > > > > > Have you tried running whoosh (not whoosh-appengine) on a backend? > > Backends are expensive, but might help you out a bit. > > > Robert > > > On Thu, Jul 28, 2011 at 11:19, andreas schmid wrote: > >> hi guys, > > >> i really need to have a full text search on my app and i wanted to know > >> what kind of external services you are using on your apps. or how you > >> implement such a service. > >> i tried whoosh-appengine but first its a really old version and second it > >> has huge issues with 1000+ indexed entities. > > >> we don't know when the full text search will be released for GAE right? > > >> thx > > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "Google App Engine" group. > >> To post to this group, send email to google-appengine@googlegroups.com. > >> To unsubscribe from this group, send email to > >> google-appengine+unsubscr...@googlegroups.com. > >> For more options, visit this group > >> athttp://groups.google.com/group/google-appengine?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Can't deploy app. Stops on cloning
Deploying a new version seems to work. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/NHoSoeTp118J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Error 500s on deploy
Same here. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/SaqNaquYGFcJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Error 500s on deploy
I can't deploy for half an hour now. Wow, I hope this will be fixed soon! H -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Fa1_q9ntsDUJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Problems Deploying App - 500 Internal server Error
I've been getting this same problem consistently for the past 15 minutes or so. Creating staging directory Scanning for jsp files. Compiling jsp files. Scanning files on local disk. Scanned 250 files. Initiating update. Cloning 55 static files. Cloning 246 application files. Cloned 100 files. Rolling back the update. java.io.IOException: Error posting to URL: https://appengine.google.com/[...] 500 Internal Server Error I'll try again later I suppose. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/lQA13jUhPTgJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Cannot deploy app, cannot modify any settings
Today, whenever I try to deploy my app, I get the following: *...* *Closing update: new version is ready to start serving.* *Rolling back the update.* *Error 503: --- begin server output ---* * * *Application Busy (503)* *Multiple clients are trying to modify the same application.* *--- end server output ---* Moreover, I cannot modify any settings in my app. For example disabling the app, or modifying in anything, results in message *"Application busy. Multiple clients are trying to modify the same application."*. What is going on here? :S -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Ha5DuRac-CYJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Error 500s on deploy
For the past 2 days I've been entirely unable to deploy to any of my google app engine instances. It was bad like this at the beginning of the year but has been much better recently. Has anyone else been experiencing similar issues? I noticed the write latency peaking on the app engine status page yesterday and assumed the problems with deploy were due to that but today the status page shows no such peak. (I'm using python and django-nonrel) -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/a5gyiOlHzLIJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] 407 Proxy Authentication Required for ISA
We have created a webservice using PROTORPC and we get json response in different methods. But when trying to access this webservice from other domains we get the " 407 Proxy Authentication Required for ISA server" error. Can we add domains that are allowed to access this service.Is there any workaround for this. How can i provide my webservice to the mass. Regards Vineeth -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Data Center (CDN) Locations
That blows away my thought of app engine requests being served by people running the sdk. On Aug 2, 3:11 pm, "Brandon Wirtz" wrote: > After doing some research it turns out that the reason GAE is so fast is > that it is really a bot net running on every machine with Chrome installed. > The speed comes from the fact that in many cases when you hit your own app > it is very fast because it is running locally, but since very few servers > are running Chrome, speed test tools often return a slower result because > they have to hit some poor schmuck's computer, and if he is at Starbucks > that could be slow. > > :-) > > From: google-appengine@googlegroups.com > [mailto:google-appengine@googlegroups.com] On Behalf Of Ian Wilson > Sent: Tuesday, August 02, 2011 3:20 AM > To: google-appengine@googlegroups.com > Subject: [google-appengine] Data Center (CDN) Locations > > I have been searching around but have not seen information on where App > Engine data centers are located and how loads are spread? > > So I have two questions: > > - Where are the App Engine data centers? (i.e. West Coast, East Coast, > Europe...) > > - What criteria are used to serve an app from one data center over another? > > We have an application we are building for a customer in Japan and would > like to know if there is an App Engine data center in Asia so we will have > lower latency. > > Thanks. > > Ian > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web > visithttps://groups.google.com/d/msg/google-appengine/-/atIjL0tAAAYJ. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Data Center (CDN) Locations
That is interesting, I will also be testing and developing in Yokohama and Tokyo this month. I am currently testing from europe but will see how much of a difference there is from Japan. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/5tSIqzGtEoMJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Can't deploy app. Stops on cloning
I have the same problem, Python flavored appengine. It still worked at 13:00 UTC, and stopped working right after that time and persists until now. H -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/MIFcHKUV_0UJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] A server error occurred. Please contact the administrator.
Getting the message A server error occurred. Please contact the administrator. for all write operations. Status is green: http://code.google.com/status/appengine App: 3x3links Thanks. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Data Center (CDN) Locations
Ikai, It sure would be great if Google would consider adding regional data centers on the roadmap. I live in Boston, but travel to Japan a few times a year. Most recently, I stayed in Yokohama for the month of June, actively developing and testing my app the entire month. There was a noticeable degradation in performance compared to what I experience in Boston. Not just after calling my app the first time after a period of non-activity, but even after on-going active use. The latency didn't make my app unusable but it was noticeable enough to cause me concern as to how potential customers in Japan might perceive the speed of my app. Cheers, Patrick On Aug 2, 2:54 pm, "Ikai Lan (Google)" wrote: > The primary data centers serving App Engine are in North America, however, > assets and items that are edge cached are served from Google edge caches > which are located wherever there are Google data centers. > > When evaluating page speed, it's typically assets that cause page speeds to > appear slow: > > http://stevesouders.com/hpws/rules.php > > Only dynamic calls to your application logic would be routed to North > America. If you follow these rules, websites should still load quickly - > there is a fairly large App Engine community in Japan. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > plus.ikailan.com | twitter.com/ikai > > > > > > > > On Tue, Aug 2, 2011 at 3:30 AM, Simon Knott wrote: > > Hi, > > > As far as I'm aware, the most accurate information the Google staff have > > divulged is that all data centres are located in North America somewhere. > > > Cheers, > > Simon > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To view this discussion on the web visit > >https://groups.google.com/d/msg/google-appengine/-/OvHwAZFC9aYJ. > > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Can't download all data with bulkloader from non-default namespace
Hi Timofey, I managed to reproduce your issue: proppy@proppy:~$ ~/google_appengine/appcfg.py download_data --application=s~proppy-guestbook-namespace --url=https://proppy-guestbook-namespace.appspot.com/_ah/remote_api --namespace="foo" --filename=foo.data Downloading data records. [INFO] Logging to bulkloader-log-20110803.134504 [INFO] Throttling transfers: [INFO] Bandwidth: 25 bytes/second [INFO] HTTP connections: 8/second [INFO] Entities inserted/fetched/modified: 20/second [INFO] Batch Size: 10 [INFO] Opening database: bulkloader-progress-20110803.134504.sql3 [INFO] Opening database: bulkloader-results-20110803.134504.sql3 [INFO] Connecting to proppy-guestbook-namespace.appspot.com/_ah/remote_api [ERROR ] Unable to download kind stats for all-kinds download. [ERROR ] Kind stats are generated periodically by the appserver [ERROR ] Kind stats are not available on dev_appserver. Looks like a bug, can you fill it on http://code.google.com/p/googleappengine/issues/entry and followup with the url there ? Thanks in advance. On Tue, Aug 2, 2011 at 4:51 PM, Timofey Koolin wrote: > Yes, I can. My app ID: my-optimiser (my-optimiser.appspot.com). > It isn't very need personal for me now - I have only one Entity type and I > can download it with --kind option, but I think it is bug in platform or > SDK. > Now I try use https://my-optimiser.appspot.com instead of > https://3.my-optimiser.appspot.com - it have same error. > > 2011/8/2 Johan Euphrosine >> >> Can you share your application id, so we can investigate the issue ? >> >> Feel free to fill a production issue if this is directly affecting >> your operation: >> >> http://code.google.com/p/googleappengine/issues/entry?template=Production%20issue >> >> On Sat, Jul 30, 2011 at 10:19 AM, rekby wrote: >> > I use Python SDK 1.5.2. High-replication datastore. I see data statistic >> > for >> > namespace 2 in dashboard. >> > appcfg.py download_data --application=s~my-optimiser >> > --url=https://3.my-optimiser.appspot.com/remote_api --namespace="2" >> > --filename=optimiser.data >> > Downloading data records. >> > [INFO ] Logging to bulkloader-log-20110730.121721 >> > [INFO ] Throttling transfers: >> > [INFO ] Bandwidth: 25 bytes/second >> > [INFO ] HTTP connections: 8/second >> > [INFO ] Entities inserted/fetched/modified: 20/second >> > [INFO ] Batch Size: 10 >> > [INFO ] Opening database: bulkloader-progress-20110730.121721.sql3 >> > [INFO ] Opening database: bulkloader-results-20110730.121721.sql3 >> > [INFO ] Connecting to 3.my-optimiser.appspot.com/remote_api >> > [ERROR ] Unable to download kind stats for all-kinds download. >> > [ERROR ] Kind stats are generated periodically by the appserver >> > [ERROR ] Kind stats are not available on dev_appserver. >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "Google App Engine" group. >> > To view this discussion on the web visit >> > https://groups.google.com/d/msg/google-appengine/-/fN3ISosu8ZgJ. >> > To post to this group, send email to google-appengine@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com. >> > For more options, visit this group at >> > http://groups.google.com/group/google-appengine?hl=en. >> > >> >> >> >> -- >> Johan Euphrosine (proppy) >> Developer Programs Engineer >> Google Developer Relations >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > > > -- > С уважением, > Кулин Тимофей. > > Телефон: +7 (4852) 974793 > ICQ: 114902104 > email: timo...@koolin.ru > Blog: http://timofey.koolin.ru > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- Johan Euphrosine (proppy) Developer Programs Engineer Google Developer Relations -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Can't deploy app. Stops on cloning
Hi there, this morning after two deployments to my appengine app i can't deploy it anymore. When i try to deploy the app i get this error message: ... Cloning 2039 application files. Cloned 100 files. Cloned 200 files. Cloned 300 files. Cloned 400 files. Cloned 500 files. Cloned 600 files. Cloned 700 files. Cloned 800 files. Cloned 900 files. Rolling back the update. Error 500: --- begin server output --- 500 Server Error Error: Server Error The server encountered an error and could not complete your request.If the problem persists, please http://code.google.com/appengine/community.html";>report your problem and mention this error message and the query that caused it. --- end server output --- Anyone have suggestions on this? Martino. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/GvwJNqgs--sJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Crazy number of Deadine Exceeded exceptions recently
Some more details while searching the logs: The latest burst (we have about one every 1-2 hours) shows this: We had about 90 instances up. We got DeadlineExceeded error during two minutes. During the first 30seconds, 50 successive error requests have no pending_ms. Then during the next 1m30 they nearly all have pending_ms and one third are loading requests. So my guess is that it's not linked to the scheduler: something goes wrong and requests are killing the instances during the first 30 seconds, then killing the remaining ones with an increased latency until what goes wrong is resolved. On 3 août, 10:20, Alexis wrote: > We are using Python, and we are not using back-ends or taskqueue. > But most of our requests fetch entities of the same kind, so a "kind- > locking" can be relevant. > I'll setup timeouts on datastore operations to see if this is what is > going wrong (and looks good to setup these anyway). > > Seems logical to have pending_ms values if the app have no instances > available instead of the one hundred that used to serve the traffic... > however requests should be able to perform in less than 20sec. Nearly > all the requests failing with this error, and while the instances get > killed off, have this pending_ms value, and don't have the > loading_request=1. > So I'm not sure whether these DeadlineExceeded errors come first or as > a consequence of the instances being killed: they are not > loading_request and the warning in the logs says it may kill the > instance, but we have these pending_ms showing that we already lack of > instances. > > The traffic is very steady. > > On 3 août, 05:49, Robert Kluin wrote: > > > > > > > > > Interesting. I've been seen exactly the same strange behavior across > > several apps as well. Suddenly instances will get killed and > > restarted in large batches. This happens even with low request > > latency, small memory usage (similar to yours < 50mb), low error > > rates, and steady traffic. I pretty convinced this is tied to the > > scheduler changes they've been making over the past few weeks. > > > As a side note, the pending_ms value (9321) indicates that the request > > sat there waiting to be serviced for quite a long time. That won't > > leave as much time to respond to the requests. Do you always see > > bursts of those when your instances get killed off? Are you getting > > big spikes in traffic when this happens or is it steady? > > > Robert > > > On Tue, Aug 2, 2011 at 05:24, Alexis wrote: > > > Hi, > > > > I've got a similar issue: lots of DeadlineExceeded errors since a few > > > weeks. I'm on the master-slave datastore too, but what I'm reporting > > > happened again one hour ago. > > > > These errors happen in bursts, and I recently realized that it was in > > > fact shutting down ALL instances of the application. > > > (In the logs, I also have this warning: A serious problem was > > > encountered with the process that handled this request, causing it to > > > exit. This is likely to cause a new process to be used for the next > > > request to your application. If you see this message frequently, you > > > may be throwing exceptions during the initialization of your > > > application. (Error code 104)) > > > This does not happen when an instance is spinning up but after several > > > hours. > > > > The trace I get along with the DeadlineExceeded errors show that it > > > happens in the second phase: while the app is trying to fallback > > > gracefully because of an other error (that does not appears in logs). > > > Request reported processing time can be like this: ms=100878 > > > cpu_ms=385 api_cpu_ms=58 cpm_usd=0.010945 pending_ms=9321 > > > > Here is a screenshot of the admin page, showing that all instances > > > have been shut down about 7 minutes ago, even resident ones: > > >http://dl.dropbox.com/u/497622/spinDown.png > > > > The app do work in batches (although not always small ones). But > > > request processing time is usually good enough (see average latency on > > > the screen shot). > > > I'm trying things on my testing applications to see what can be wrong > > > but it's still not clear for me and I'm running short of ideas... > > > > Any suggestions? > > > > On 2 août, 06:21, Robert Kluin wrote: > > >> Hi Will, > > >> I assume this is on the master-slave datastore? I think there were > > >> a number of large latency spikes in both the datastore and serving > > >> last week. > > > >> Some things to try: > > >> - do work in smaller batches. > > >> - if you're doing work serially, do it in batches. > > >> - use async interfaces to do work in batches, but in parallel using > > >> async. > > > >> http://code.google.com/appengine/docs/python/datastore/async.html > > > >> Robert > > > >> On Fri, Jul 29, 2011 at 18:35, Will Reiher wrote: > > >> > I'm trying to debug this issue but I keep hitting a wall. > > >> > I keep trying new things on one of my deployments to see if I an get > > >> > the > >
Re: [google-appengine] Much more efficient implementation of db.ListProperty(int) - created a lib
cool! ill def use it! thx On Aug 3, 2011, at 10:15 AM, MiuMeet Support wrote: > You can fit up to 131'072 elements in there, afterwards you hit the 1mb > entity limit (entity overhead aside, but 100'000 are def possible). > > Put and Get performance are really fantastic, 10ms each, see the blog post. > > -Andrin > > On Wed, Aug 3, 2011 at 9:23 AM, andreas schmid wrote: > does this also mean that there is no 5000 objects limit on the list? > is the get performance good as the put? > > thx for sharing this. > > On Aug 2, 2011, at 9:04 PM, Ikai Lan (Google) wrote: > >> Thanks Andrin! >> >> If anyone is wondering why this is, it's because the List property is >> serialized and deserialized using protocol buffers. Protocol buffers are >> very useful for things like validation and cross language compatibility, but >> you can sometimes gain some performance benefits by serializing and >> deserializing using native methods. The protocol buffer implementation for >> Python is noticeably slow, even via the native extension (if you can improve >> this - Google wants to hire you), so I'm not surprised techniques like this >> work. >> >> -- >> Ikai Lan >> Developer Programs Engineer, Google App Engine >> plus.ikailan.com | twitter.com/ikai >> >> >> >> On Tue, Aug 2, 2011 at 9:18 AM, Andrin von Rechenberg >> wrote: >> So you want to store a "very long list of numbers" in your AppEngine Model >> which doesn't need to be indexed. >> >> Well, you could just do this: >> >> class Foo(db.Model): >> numbers = db.ListProperty(int, indexed=False) >> >> However, you will quickly notice that the performance and the memory >> consumption of this sucks if you add thousands of integers to this list. >> >> Instead you should use the native python "array.array" type as the list and >> store it in a BlobProperty. It's up to 30x faster! >> >> I created a library that does exactly this (free, apache 2.0 license): >> http://devblog.miumeet.com/2011/08/much-more-efficient-implementation-of.html >> >> Once you have the library, all you need to do is this to get a huge >> performance boost: >> >> class Foo(db.Model): >> numbers = db.ArrayProperty() >> >> The blogpost also has more details and an appstat comparison. >> >> Cheers & hope you like it >> -Andrin >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Connect from GAE to external PostgreSQL DB?
Thanks Pascal / Ikai for reaching out! Unfortunately, I do not have any access to this Postgres DB, except a logon and the ability to perform select queries (i.e. data retrieval only, no create/update/delete). Basically, I would like to get the data out of this DB by executing [java.sql.]Statement's and using the returned ResultSets in the GAE environment to extract data and then further use this in a GWTP framework (which I really like btw). But then, it all comes down to a working connection between GAE and the DB... Regards, Wouter On 2 aug, 20:58, "Ikai Lan (Google)" wrote: > Yeah, you'll need a REST proxy. I wonder if someone has built a version of > DBSlayer for Postgres? > > http://code.nytimes.com/projects/dbslayer > > DBSlayer's biggest fault: no authentication. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > plus.ikailan.com | twitter.com/ikai > > On Tue, Aug 2, 2011 at 7:56 AM, Pascal Voitot Dev < > > > > > > > > pascal.voitot@gmail.com> wrote: > > Not exactly the answer you expect certainly but wouldn't it better to > > implement a Web/REST service somewhere in the same network as the Postgres > > DB (if you can) which would just extract from Postgres and serve the data > > you need from your GAE app? > > Like that, you don't expose directly Postgres DB to your remote GAE app > > which is something more secure IMHO. > > > Pascal > > > On Tue, Aug 2, 2011 at 3:49 PM, wouterdeg wrote: > > >> Hi All, > > >> For an app I need to obtain data from an external PostgreSQL database, > >> do some calculations on it and then show the results to the clients in > >> an MVP set-up. > > >> However, from some posts I gather that GAE cannot connect with a > >> PostgreSQL DB over the internet out of its servlet context. Others > >> suggest setting up an 'http proxy' for this. > > >> I'm quite a newbie on this, however, I would love to use GAE / GWT for > >> the most part (particularly GTWP). Anyone that has a suggestion how I > >> can get to the data on the Postrgres DB out of my GAE servlet > >> context?? > > >> Thanks a lot! > > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "Google App Engine" group. > >> To post to this group, send email to google-appengine@googlegroups.com. > >> To unsubscribe from this group, send email to > >> google-appengine+unsubscr...@googlegroups.com. > >> For more options, visit this group at > >>http://groups.google.com/group/google-appengine?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: High CPU usage and how to improve performance
Sorry for the late followup, I just updated your issue. FYI, we believe that the problems experienced by most of the Apps on this thread were due to a temporary datastore latency, as you can see there: http://code.google.com/status/appengine/detail/datastore/2011/07/27#ae-trust-detail-datastore-get-latency This cascaded into a higher HTTP latencies for some Python applications running on Master/Slave Datastore Replication system: http://code.google.com/status/appengine/detail/serving/2011/07/27#ae-trust-detail-helloworld-get-latency We offered the affected app to assist them to migrate to HRD, where apps are not subject to those transient latency issues. For the record Issue #5415 might be a different beast, as appstats doesn't report any datastore operation but still high latency. I already escalated it to the engineering team and will followup on the bug report once I have any updates. Hope that helps. On Wed, Aug 3, 2011 at 4:18 AM, observer247 wrote: > Issue number is 5415 > > It has been more than a week and I am yet to get a useful response on > the ticket. > > On Jul 21, 4:20 am, Johan Euphrosine wrote: >> >> >> Feel free to open a Production Issue for problem like this affecting >> directly your application >> operation:http://code.google.com/p/googleappengine/issues/entry?template=Produc... >> >> If you do so, please reply to this thread with the issue number, so we >> can followup. >> >> Hope that helps. >> >> >> >> >> >> >> >> >> >> On Thu, Jul 21, 2011 at 10:24 AM,observer247 wrote: >> > Is this is because of the new pricing, I would rather move out of >> > Google app engine. >> >> > We have over 10 apps serving our iPhone apps. It will take us a lot of >> > effort to port this code to php/mysql but I guess if the pricing >> > changes like this, we cannot afford to pay such high prices. >> >> > I am not sure if there is a phone number or support chat. When I am >> > paying $1000 per month for a service, I would expect to talk to a real >> > person and share with him the issues I am facing. >> >> > On Jul 20, 1:14 am,observer247 wrote: >> >> Almost the same usage is now triggering 10times the daily cost ! Not >> >> sure what changed as far as billing. >> >> >> Checking appstats and instances. >> >> >> On Jul 19, 2:55 pm, Barry Hunter wrote: >> >> >> > Have you tried >> >> > AppStats?http://code.google.com/appengine/docs/python/tools/appstats.html >> >> >> > On Tue, Jul 19, 2011 at 8:31 PM,observer247 wrote: >> >> > > My python app engine app is somewhat popular, mainly used as via api >> >> > > from my iphone app. >> >> >> > > Recently I am seeing high over usage charges. >> >> >> > > The most used call already use 15 CPU hours , with stats: >> >> > > 3.7 calls/sec 26.15K requests in 12hrs 603 (0) [cpu] >> >> >> > > HTTP/1.1" 200 1839 - "QandA/5.4 CFNetwork/485.12.7 Darwin/10.4.0" >> >> > > "may.latest.nuvusanswers.appspot.com" ms=1490 cpu_ms=467 api_cpu_ms=0 >> >> > > cpm_usd=0.013200 pending_ms=1058 throttle_code=1 >> >> > > instance=00c61b117c8b0de1286e8d90c22f4d5b3897c8 >> >> >> > > This call just reads a memcache and returns the data. Thats it. Will >> >> > > really like to know are memcache reads so expensive ? Am I using a lot >> >> > > more memcache that I need ? >> >> >> > > I have removed all other versions and have just one version running. >> >> > > The number of instances now is >> >> >> > > 28 total 0.510 (Average QPS) 281.1 ms (Average Latency) >> >> > > 59.5 MBytes >> >> > > (Average Memory) >> >> >> > > As this is the most expensive call, I am first going after that. but I >> >> > > am seeing most of my calls are now using a LOT of CPU. Will be great >> >> > > if anyone can point me to things I can debug to find out the high >> >> > > usage across the board. >> >> >> > > Thank you for reading this ! >> >> >> > > -- >> >> > > You received this message because you are subscribed to the Google >> >> > > Groups "Google App Engine" group. >> >> > > To post to this group, send email to >> >> > > google-appengine@googlegroups.com. >> >> > > To unsubscribe from this group, send email to >> >> > > google-appengine+unsubscr...@googlegroups.com. >> >> > > For more options, visit this group >> >> > > athttp://groups.google.com/group/google-appengine?hl=en. >> >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Google App Engine" group. >> > To post to this group, send email to google-appengine@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com. >> > For more options, visit this group >> > athttp://groups.google.com/group/google-appengine?hl=en. >> >> -- >> Johan Euphrosine (proppy) >> Developer Programs Engineer >> Google Developer Relations > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To u
[google-appengine] Re: Crazy number of Deadine Exceeded exceptions recently
We are using Python, and we are not using back-ends or taskqueue. But most of our requests fetch entities of the same kind, so a "kind- locking" can be relevant. I'll setup timeouts on datastore operations to see if this is what is going wrong (and looks good to setup these anyway). Seems logical to have pending_ms values if the app have no instances available instead of the one hundred that used to serve the traffic... however requests should be able to perform in less than 20sec. Nearly all the requests failing with this error, and while the instances get killed off, have this pending_ms value, and don't have the loading_request=1. So I'm not sure whether these DeadlineExceeded errors come first or as a consequence of the instances being killed: they are not loading_request and the warning in the logs says it may kill the instance, but we have these pending_ms showing that we already lack of instances. The traffic is very steady. On 3 août, 05:49, Robert Kluin wrote: > Interesting. I've been seen exactly the same strange behavior across > several apps as well. Suddenly instances will get killed and > restarted in large batches. This happens even with low request > latency, small memory usage (similar to yours < 50mb), low error > rates, and steady traffic. I pretty convinced this is tied to the > scheduler changes they've been making over the past few weeks. > > As a side note, the pending_ms value (9321) indicates that the request > sat there waiting to be serviced for quite a long time. That won't > leave as much time to respond to the requests. Do you always see > bursts of those when your instances get killed off? Are you getting > big spikes in traffic when this happens or is it steady? > > Robert > > > > > > > > On Tue, Aug 2, 2011 at 05:24, Alexis wrote: > > Hi, > > > I've got a similar issue: lots of DeadlineExceeded errors since a few > > weeks. I'm on the master-slave datastore too, but what I'm reporting > > happened again one hour ago. > > > These errors happen in bursts, and I recently realized that it was in > > fact shutting down ALL instances of the application. > > (In the logs, I also have this warning: A serious problem was > > encountered with the process that handled this request, causing it to > > exit. This is likely to cause a new process to be used for the next > > request to your application. If you see this message frequently, you > > may be throwing exceptions during the initialization of your > > application. (Error code 104)) > > This does not happen when an instance is spinning up but after several > > hours. > > > The trace I get along with the DeadlineExceeded errors show that it > > happens in the second phase: while the app is trying to fallback > > gracefully because of an other error (that does not appears in logs). > > Request reported processing time can be like this: ms=100878 > > cpu_ms=385 api_cpu_ms=58 cpm_usd=0.010945 pending_ms=9321 > > > Here is a screenshot of the admin page, showing that all instances > > have been shut down about 7 minutes ago, even resident ones: > >http://dl.dropbox.com/u/497622/spinDown.png > > > The app do work in batches (although not always small ones). But > > request processing time is usually good enough (see average latency on > > the screen shot). > > I'm trying things on my testing applications to see what can be wrong > > but it's still not clear for me and I'm running short of ideas... > > > Any suggestions? > > > On 2 août, 06:21, Robert Kluin wrote: > >> Hi Will, > >> I assume this is on the master-slave datastore? I think there were > >> a number of large latency spikes in both the datastore and serving > >> last week. > > >> Some things to try: > >> - do work in smaller batches. > >> - if you're doing work serially, do it in batches. > >> - use async interfaces to do work in batches, but in parallel using > >> async. > > >> http://code.google.com/appengine/docs/python/datastore/async.html > > >> Robert > > >> On Fri, Jul 29, 2011 at 18:35, Will Reiher wrote: > >> > I'm trying to debug this issue but I keep hitting a wall. > >> > I keep trying new things on one of my deployments to see if I an get the > >> > number of errors down but nothing seems to help. It all started in the > >> > last > >> > week or so ago. I also have some existing deployments that I have not > >> > changed and are seeing these same errors while the code was never changed > >> > and had been stable. > >> > 1. This is is happening on existing code that has not changed recently > >> > 2. The DeadlineExceededErrors are coming up randomly and at different > >> > points > >> > in the code. > >> > 3. Latency is pretty high and app engine seems to be spawning a lot of > >> > new > >> > instances beyond my 3 included ones. > > >> > -- > >> > You received this message because you are subscribed to the Google Groups > >> > "Google App Engine" group. > >> > To view this discussion on the web visit > >> >https://groups.g
Re: [google-appengine] Much more efficient implementation of db.ListProperty(int) - created a lib
You can fit up to 131'072 elements in there, afterwards you hit the 1mb entity limit (entity overhead aside, but 100'000 are def possible). Put and Get performance are really fantastic, 10ms each, see the blog post. -Andrin On Wed, Aug 3, 2011 at 9:23 AM, andreas schmid wrote: > does this also mean that there is no 5000 objects limit on the list? > is the get performance good as the put? > > thx for sharing this. > > On Aug 2, 2011, at 9:04 PM, Ikai Lan (Google) wrote: > > Thanks Andrin! > > If anyone is wondering why this is, it's because the List property is > serialized and deserialized using protocol buffers. Protocol buffers are > very useful for things like validation and cross language compatibility, but > you can sometimes gain some performance benefits by serializing and > deserializing using native methods. The protocol buffer implementation for > Python is noticeably slow, even via the native extension (if you can improve > this - Google wants to hire you), so I'm not surprised techniques like this > work. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > plus.ikailan.com | twitter.com/ikai > > > > On Tue, Aug 2, 2011 at 9:18 AM, Andrin von Rechenberg > wrote: > >> So you want to store a "very long list of numbers" in your AppEngine Model >> which doesn't need to be indexed. >> >> Well, you could just do this: >> >> *class Foo(db.Model): >> numbers = db.ListProperty(int, indexed=False) >> * >> However, you will quickly notice that the performance and the memory >> consumption of this sucks if you add thousands of integers to this list. >> >> Instead you should use the native python "array.array" type as the list >> and store it in a BlobProperty. It's up to 30x faster! >> >> I created a library that does exactly this (free, apache 2.0 license): >> >> http://devblog.miumeet.com/2011/08/much-more-efficient-implementation-of.html >> >> Once you have the library, all you need to do is this to get a huge >> performance boost: >> >> *class Foo(db.Model): >> numbers = db.ArrayProperty()* >> >> The blogpost also has more details and an appstat comparison. >> >> Cheers & hope you like it >> -Andrin >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] full text search and/or external search services
i tried whoosh-appengine and as i mentioned before it fails after indexing a few 1000 because the way it works and it exceeds the soft process limit size. never tried it on a backend. right now i customized bill katz's simple full text search and im really surprised how simple it is implemented and how easy it is to modify it to your needs. i think ill go on with this solution until gaes full text search will be released. thx On Aug 3, 2011, at 7:17 AM, Robert Kluin wrote: > Have you tried running whoosh (not whoosh-appengine) on a backend? > Backends are expensive, but might help you out a bit. > > > > Robert > > > > > > On Thu, Jul 28, 2011 at 11:19, andreas schmid wrote: >> hi guys, >> >> i really need to have a full text search on my app and i wanted to know what >> kind of external services you are using on your apps. or how you implement >> such a service. >> i tried whoosh-appengine but first its a really old version and second it >> has huge issues with 1000+ indexed entities. >> >> we don't know when the full text search will be released for GAE right? >> >> thx >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Much more efficient implementation of db.ListProperty(int) - created a lib
does this also mean that there is no 5000 objects limit on the list? is the get performance good as the put? thx for sharing this. On Aug 2, 2011, at 9:04 PM, Ikai Lan (Google) wrote: > Thanks Andrin! > > If anyone is wondering why this is, it's because the List property is > serialized and deserialized using protocol buffers. Protocol buffers are very > useful for things like validation and cross language compatibility, but you > can sometimes gain some performance benefits by serializing and deserializing > using native methods. The protocol buffer implementation for Python is > noticeably slow, even via the native extension (if you can improve this - > Google wants to hire you), so I'm not surprised techniques like this work. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > plus.ikailan.com | twitter.com/ikai > > > > On Tue, Aug 2, 2011 at 9:18 AM, Andrin von Rechenberg > wrote: > So you want to store a "very long list of numbers" in your AppEngine Model > which doesn't need to be indexed. > > Well, you could just do this: > > class Foo(db.Model): > numbers = db.ListProperty(int, indexed=False) > > However, you will quickly notice that the performance and the memory > consumption of this sucks if you add thousands of integers to this list. > > Instead you should use the native python "array.array" type as the list and > store it in a BlobProperty. It's up to 30x faster! > > I created a library that does exactly this (free, apache 2.0 license): > http://devblog.miumeet.com/2011/08/much-more-efficient-implementation-of.html > > Once you have the library, all you need to do is this to get a huge > performance boost: > > class Foo(db.Model): > numbers = db.ArrayProperty() > > The blogpost also has more details and an appstat comparison. > > Cheers & hope you like it > -Andrin > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
