[google-appengine] Re: External subdomain is not routed to service with the same name but to default service

[Fred Erik]

Thank you for the extensive answer. Now I understand, what I got wrong. I 
referred to:

https://service-dot-app-id.appspot.com
http://service.my-custom-domain.com

and registered service.my-custom-domain.com to my AppEngine, but you only had 
to register my-custom-domain. With registering service.my-custom-domain.com the 
actual url becomes 
service.service.my-custom-domain.com. Maybe that could be explained more 
clearly in the How Requests are Routed section. Anyway, thank you!


Am Donnerstag, 8. Dezember 2016 17:04:43 UTC+1 schrieb Jordan (Cloud 
Platform Support):
>
> Whenever you associate a new custom domain to your App Engine project, it 
> will always route to your Default service and version. Even if you 
> associate a subdomain to the project as you did, it will still route to the 
> default as the CNAME and A records are all the same. 
>
> The way to reach a different service or version without using a dispatch 
> file would be to use the full Routing URL 
> 
>  
> 'http://version.service.subdomain.customDomain.com'. In your case this 
> would look like 'http://*console*.console-sandbox.my-domain.store' to 
> have your subdomain 'console-sandbox.my-domain.store' route to the default 
> version of your 'console' service. 
>
> Alternatively, you can instead use a wildcard subdomain 
> 
>  
> '*' to allow App Engine to perform the routing of the URL to your specific 
> services and versions. 
>
> Note: If you have an SSL cert for your custom domain and want to use 
> HTTPS, the URL will use '-dot-' instead of actual '.', like so '
> https://version-dot-service-dot-customDomain.com' 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/3e3d5969-6a89-433a-88e4-3f26bea962b7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: External subdomain is not routed to service with the same name but to default service

[Fred Erik]

Thank you for the answer, I got it working setting up a dispatch.yaml file. 
Still I don't get why the default routing does not apply. It doesn't matter 
anymore for me but other users might run into it as well.

Am Sonntag, 4. Dezember 2016 22:19:48 UTC+1 schrieb Fred Erik:
>
> Hi,
>
> I want to setup one subdomain console-sandbox.my-domain.store for my 
> sandbox project and one subdomain console.my-domain.store for my production 
> project. The web frontend service is called console-sandbox / console, thus 
> matching the subdomains. My API service is the default service in both 
> environments. However, both subdomains get routed to the default service, 
> which is not the behaviour the documentation 
> <https://cloud.google.com/appengine/docs/python/how-requests-are-routed> 
> describes.
>
> Any ideas?
> Thanks!
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/0593f678-e452-4fa6-b9a4-1f6ec8ff833d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] External subdomain is not routed to service with the same name but to default service

[Fred Erik]

Hi,

I want to setup one subdomain console-sandbox.my-domain.store for my 
sandbox project and one subdomain console.my-domain.store for my production 
project. The web frontend service is called console-sandbox / console, thus 
matching the subdomains. My API service is the default service in both 
environments. However, both subdomains get routed to the default service, 
which is not the behaviour the documentation 
 
describes.

Any ideas?
Thanks!

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/14de4dc1-b3b6-4eed-bddb-c032776a43f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Quota error without being over quota

[Fred Erik]

Hi,

I'm getting the following error:
com.google.apphosting.api.ApiProxy$OverQuotaException: The API call 
blobstore.CreateUploadURL() required more quota than is available. 
But my quotas are nowhere near to be depleted. Is there any other reason 
for this exception?

Best,
Frederik

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/a33a1088-c188-43a8-80b4-b29e46cab7e5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] my projects suspend

[Fred]

I got couple of mails from google cloud ask for enable billing to my 
projects, so I create billing account on one of my accounts, but google 
cloud suspend my projects. I flow the work to submit account verification 
form. but the payment support guy told me my payment account work fine.

have no idea about it.


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/32482cb3-b19f-4425-9d27-74b33b76369c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] JAVA Search API: index namespace in Java?

[Fred Janon]

Index namespaces are supported in Python but don't seem to be in Java. Is
it not supported or is the doc just missing the method?

Thanks

Fred

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Java Search API: text field search with accents?

[Fred Janon]

Do queries ignore the accents on text field search? For example, for a text
field
containing Place des Fêtes, and searching for place des fetes would the
document with the field stored with accents be found?

Is there a specific Google support group for the Java Search API or here or
is it through StackOverflow?

Thanks

Fred

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] New app engine

[Fred Mort]

Hey guys,

I'm trying to create my first app engine for an app I made in App Inventor.

I have downloaded Googleappengine 1.7.4  along with python 2.7.2.5
I fully do not understand what is said by a 'terminal'

However I open the python shell with a new window and type the commands :

dev_appserver.py mytinywebdb  and

appcfg.py update mytinywebdb

The first line entered shoots and invalid syntax for mytinywebdb and the 
second says invalid syntax for 'update'
I cannot figure anything else out

Thanks for the help.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/fIpEUXLJprwJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] cannot run app engine

[Fred Mort]

I am trying to create an app engine for an app I made in app inventor.

i have googleappengine 1.7.4 and python 2.7.2.5

I can not run the command  dev_appserver.py mytinywebdb
or appcfg.py update mytinywebdb

The tutorial says to use a terminal? Though they were talking about a 
python shell that it was I have been using.

https://sites.google.com/site/appinventor/custom-tinywebdb-web-service

Not sure hwere to go.

Thanks.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/AEbm00Sc-QkJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] cannot run app engine

[Fred Mort]

terminal is the same as CMD promt correct?

On Thursday, January 24, 2013 2:23:57 PM UTC-8, Takashi Matsuo (Google) 
wrote:

 Please use terminal :)


 On Thu, Jan 24, 2013 at 1:30 PM, Fred Mort fuzzymo...@gmail.comjavascript:
  wrote:

 I am trying to create an app engine for an app I made in app inventor.

 i have googleappengine 1.7.4 and python 2.7.2.5

 I can not run the command  dev_appserver.py mytinywebdb
 or appcfg.py update mytinywebdb

 The tutorial says to use a terminal? Though they were talking about a 
 python shell that it was I have been using.

 https://sites.google.com/site/appinventor/custom-tinywebdb-web-service

 Not sure hwere to go.

 Thanks.

  -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To view this discussion on the web visit 
 https://groups.google.com/d/msg/google-appengine/-/AEbm00Sc-QkJ.
 To post to this group, send email to 
 google-a...@googlegroups.comjavascript:
 .
 To unsubscribe from this group, send email to 
 google-appengi...@googlegroups.com javascript:.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.




 -- 
 Takashi Matsuo | Developers Advocate | tma...@google.com javascript:
  

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/TMfBrDeWyhAJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] cannot run app engine

[Fred Mort]

When I run the first command it is saying  rdbms API is not available 
becuase the MySQLdb library could not be loaded.  

when I run the second command it says :

Directory does not contain a  appenginename.yaml configuration on file.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/Fvj5JjvnI7gJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] cannot run app engine

[Fred Mort]

Okay my appengine is up and running and on the web.

Now when I save data to it and try to request it through my app it does not 
show. I am seeing that requests are being sent the site and being made, but 
nothing is returned to my phone,  Do I need to add/upload Indexes?  If so, 
how would I do this?

Thanks.

On Thursday, January 24, 2013 2:23:57 PM UTC-8, Takashi Matsuo (Google) 
wrote:

 Please use terminal :)


  

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] How to migrate an app to HRD and keep its application id?

[Fred Janon]

Thanks Dan. I wanted to save an app (against the total count of 10), but
reading some other posts, I realize that's probably a moo point.

Fred


On Fri, Nov 2, 2012 at 10:46 PM, Dan Holevoet danielholev...@google.comwrote:

 The final step of the HRD migration will alias your old app ID to the new
 one. A visitor to foo.appspot.com will see the behavior of
 foo-hrd.appspot.com (or whatever you called your HRD app). If your only
 concern is user-facing behavior, it's not an issue.

 Thanks,
 Dan


 On Fri, Nov 2, 2012 at 1:24 PM, Andrew Mackenzie 
 and...@mackenzie-serres.net wrote:

 After wanting to do the same I found no way to do so, and believe this is
 impossible.

 My best suggestion: use your own DNS domain, and map to
 myappid.appspot.com, and get your users to use that.

 After migration, change DNS to map to newappid.appspot.com

 Andrew

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/QIt987GzmWcJ.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




 --
 Dan Holevoet
 Google Developer Relations

  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] How to run 2 versions (go java) of the same app at the same time in dev?

[Fred Janon]

Is there a way to run locally two versions of the same app in two different
languages (Java  Go)? The only way I can think of is to run the two apps
on a different port. But I would like to call one app from the other one
without having to specify the port.

Any suggestion?

Thanks

Fred

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Can we have 2 versions (java go) of an app in app.yaml?

[Fred Janon]

Is there a way to package two different versions of an app in the same
project (app.yaml)?

I would like to call the java part from the go part in the same app.

Thanks

Fred

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] How to migrate an app to HRD and keep its application id?

[Fred Janon]

Hi,

I created an application that I never used, now I am being asked to migrate
it to HRD.

Datastore Replication Options:
Master/Slave Replication
Deprecated!https://developers.google.com/appengine/docs/adminconsole/migration

I saw that the migrated application must have a new application id. Is
there a way I can keep the original application id or reset the application
parameters so it keeps the app id and get the HRD storage from scratch?

Thanks

Fred

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] I am a new GAEer, it promts as below, how should I do ?

[Fred Liu]

Thanks Tkashi,

but the log said 

   1. 
  1.  2012-10-09 14:22:11.730 / 500 27ms 0kb Mozilla/5.0 (Windows NT 
  6.1; WOW64; rv:15.0) Gecko/20100101 Firefox/15.0.1 
  
  203.169.242.35 - - [08/Oct/2012:23:22:11 -0700] GET / HTTP/1.1 500 0 - 
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20100101 Firefox/15.0.1 
billing-require.appspot.com ms=28 cpu_ms=0 
instance=00c61b117ce57c62bb58c57fde5a91febad2 
https://appengine.google.com/instances?app_id=s%7Ebilling-requireversion_id=1.362304267631812841key=00c61b117ce57c62bb58c57fde5a91febad2#00c61b117ce57c62bb58c57fde5a91febad2
  
  2.  E 2012-10-09 14:22:11.728  
  
  Traceback (most recent call last):
File 
/base/python27_runtime/python27_lib/versions/1/google/appengine/runtime/wsgi.py,
 line 195, in Handle
  handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File 
/base/python27_runtime/python27_lib/versions/1/google/appengine/runtime/wsgi.py,
 line 239, in _LoadHandler
  handler = __import__(path[0])
  ImportError: No module named home
  
  
 BUT, actually, it works well in my local server. is it related to the 
python version ? my python version is Python 2.7.1+.


Thanks,
Fred
在 2012年10月9日星期二UTC+8上午9时28分01秒，Takashi Matsuo (Google)写道：


 Please go to:
 https://appengine.google.com/logs?app_id=YOUR_APP_ID


 On Mon, Oct 8, 2012 at 6:46 PM, Fred Liu laush...@gmail.com javascript:
  wrote:


 Error: Server Error The server encountered an error and could not 
 complete your request.

 If the problem persists, please 
 reporthttp://code.google.com/appengine/community.htmlyour problem and 
 mention this error message and the query that caused it.
  
 -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To view this discussion on the web visit 
 https://groups.google.com/d/msg/google-appengine/-/3C8D644P8qoJ.
 To post to this group, send email to 
 google-a...@googlegroups.comjavascript:
 .
 To unsubscribe from this group, send email to 
 google-appengi...@googlegroups.com javascript:.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.




 -- 
 Takashi Matsuo | Developers Advocate | tma...@google.com javascript:

 

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/zmFgueMtHY4J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] I am a new GAEer, it promts as below, how should I do ?

[Fred Liu]

Error: Server Error The server encountered an error and could not complete 
your request.

If the problem persists, please 
reporthttp://code.google.com/appengine/community.htmlyour problem and mention 
this error message and the query that caused it.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/3C8D644P8qoJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Is it possible to build a mixed app in Java Go?

[Fred Janon]

Thanks Barry, the version trick is a good idea. I'll try that with a small
prototype.

The services in Java would be used internally from the Go app, nothing from
the Java app would be exposed publicly, so I would not have to proxy
anything, if I understood what you meant.

Cheers,

Fred

On Thu, Aug 30, 2012 at 6:44 PM, Barry Hunter barrybhun...@gmail.comwrote:

 You can deploy different runtimes to different versions of your
 application.

 While the non-default versions can have public urls, they use a specific
 hostname. So in general you would have the default version (possibly in
 Go), proxy as required to a java application/version.

 You will of course need a seperate running instance for each runtime, so
 it will increase costs.



 On Thu, Aug 30, 2012 at 5:27 PM, Fred Janon fja...@gmail.com wrote:

 Hi,

 I would like to start a new app in Go but some of the services are only
 offered in Java for the moment. I am thinking about writing everything I
 can in Go and call the other services in Java when not available in Go.

 Is there a way to have a GAE app with some of the URLs handled by a Go
 app and some others in Java?

 Thanks

 Fred

  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Is it possible to build a mixed app in Java Go?

[Fred Janon]

Hi,

I would like to start a new app in Go but some of the services are only
offered in Java for the moment. I am thinking about writing everything I
can in Go and call the other services in Java when not available in Go.

Is there a way to have a GAE app with some of the URLs handled by a Go app
and some others in Java?

Thanks

Fred

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Should Google deprecate Master/Slave data store option?

[Fred Sauer]

For the wizard, using an entity group will give you strong consistency. Use
ancestor query or plain old fetch (get) by key to get strongly consistent
results.

In general, query public data using regular (eventually consistent) queries.
Merge in the user's own, just committed, data.

Most of the time you're probably only dealing with 100s of milliseconds or a
few seconds.

Fred on my Android.
On Sep 3, 2011 2:47 AM, NM nirmallya.mukher...@gmail.com wrote:
 In case of a wizard based interface with multiple steps where users
 provide data at every step we have to ensure that we work off session
 data. By and large we can conclude that the following design in the
 application will help alleviate the eventual read consistency scenario
 in HR which is not a problem in M/S.
 1. If the appointment (or any object) is created/updated in step 1 it
 also needs to go in the user session (I am a Java developer)
 2. Step 2 gets the appointments (or any object collection) from the
 data store and checks if the object created in step 1 is present. If
 not add it in the collection manually
 3. Pass it on to the JSP for display

 We have to live with the fact that all appointments will not show if
 the calendar is global where appointments created by multiple users
 need to be displayed.

 I was also wondering if Bigtable/Datanucleus can let the program know
 the newly generated PK like a traditional relational database. If we
 can get the PK then the process of checking the collection becomes
 simple in addition to other benefits of knowing the PK after insert.

 Finally how much latency are we really talking about between the last
 write and the consistent read?

 On Sep 2, 10:11 pm, Fred Sauer fre...@google.com wrote:
 On Fri, Sep 2, 2011 at 12:18 AM, NM nirmallya.mukher...@gmail.com
wrote:
  Here is a usecase that may support M/S over HRD.

  I have a form where we capture appointment information such as date,
time
  and some other details. Upon a successful save the resulting page is a
  calendar view that needs to read all appointments including this newly
  created appointment.

  Given this and eventual consistency mode of HRD the calendar at times
does
  not show the newly saved appointment. When I refresh the screen after a
few
  seconds it does; obviously by then the data would have replicated. This
is
  obviously not an issue with M/S.

  Thoughts?

 The calendar times you are querying are all for one user, you could put
the
 calendar entities in a single entity group and then use an ancestor
query,
 which will return consistent results.

 If you need to query across entity groups, but still want the calendar
entry
 you just saved to be included in the results, then the recommended
approach
 is:
 - Perform your cross entity group query in the normal way
 - In your application code, insert the just inserted entity into the
result
 set if it's not there
 - Show the user the merged result set

 You can store away or track the just inserted calender entity in a
cookie,
 in a url / form parameter, in memcache, or in an HTTP session object
(Java).

 Fred

   --
  You received this message because you are subscribed to the Google
Groups
  Google App Engine group.
  To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/lC6R_999_vQJ.

  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

 --
 Fred Sauer
 Developer Advocate
 Google Inc.
 1600 Amphitheatre Parkway
 Mountain View, CA 94043
 fre...@google.com

 --
 You received this message because you are subscribed to the Google Groups
Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Should Google deprecate Master/Slave data store option?

[Fred Sauer]

The entity id (or name) is known / available after a put().

Fred on my Android.
On Sep 3, 2011 2:47 AM, NM nirmallya.mukher...@gmail.com wrote:
 In case of a wizard based interface with multiple steps where users
 provide data at every step we have to ensure that we work off session
 data. By and large we can conclude that the following design in the
 application will help alleviate the eventual read consistency scenario
 in HR which is not a problem in M/S.
 1. If the appointment (or any object) is created/updated in step 1 it
 also needs to go in the user session (I am a Java developer)
 2. Step 2 gets the appointments (or any object collection) from the
 data store and checks if the object created in step 1 is present. If
 not add it in the collection manually
 3. Pass it on to the JSP for display

 We have to live with the fact that all appointments will not show if
 the calendar is global where appointments created by multiple users
 need to be displayed.

 I was also wondering if Bigtable/Datanucleus can let the program know
 the newly generated PK like a traditional relational database. If we
 can get the PK then the process of checking the collection becomes
 simple in addition to other benefits of knowing the PK after insert.

 Finally how much latency are we really talking about between the last
 write and the consistent read?

 On Sep 2, 10:11 pm, Fred Sauer fre...@google.com wrote:
 On Fri, Sep 2, 2011 at 12:18 AM, NM nirmallya.mukher...@gmail.com
wrote:
  Here is a usecase that may support M/S over HRD.

  I have a form where we capture appointment information such as date,
time
  and some other details. Upon a successful save the resulting page is a
  calendar view that needs to read all appointments including this newly
  created appointment.

  Given this and eventual consistency mode of HRD the calendar at times
does
  not show the newly saved appointment. When I refresh the screen after a
few
  seconds it does; obviously by then the data would have replicated. This
is
  obviously not an issue with M/S.

  Thoughts?

 The calendar times you are querying are all for one user, you could put
the
 calendar entities in a single entity group and then use an ancestor
query,
 which will return consistent results.

 If you need to query across entity groups, but still want the calendar
entry
 you just saved to be included in the results, then the recommended
approach
 is:
 - Perform your cross entity group query in the normal way
 - In your application code, insert the just inserted entity into the
result
 set if it's not there
 - Show the user the merged result set

 You can store away or track the just inserted calender entity in a
cookie,
 in a url / form parameter, in memcache, or in an HTTP session object
(Java).

 Fred

   --
  You received this message because you are subscribed to the Google
Groups
  Google App Engine group.
  To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/lC6R_999_vQJ.

  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

 --
 Fred Sauer
 Developer Advocate
 Google Inc.
 1600 Amphitheatre Parkway
 Mountain View, CA 94043
 fre...@google.com

 --
 You received this message because you are subscribed to the Google Groups
Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Should Google deprecate Master/Slave data store option?

[Fred Sauer]

On Fri, Sep 2, 2011 at 12:18 AM, NM nirmallya.mukher...@gmail.com wrote:

 Here is a usecase that may support M/S over HRD.

 I have a form where we capture appointment information such as date, time
 and some other details. Upon a successful save the resulting page is a
 calendar view that needs to read all appointments including this newly
 created appointment.

 Given this and eventual consistency mode of HRD the calendar at times does
 not show the newly saved appointment. When I refresh the screen after a few
 seconds it does; obviously by then the data would have replicated. This is
 obviously not an issue with M/S.

 Thoughts?


The calendar times you are querying are all for one user, you could put the
calendar entities in a single entity group and then use an ancestor query,
which will return consistent results.

If you need to query across entity groups, but still want the calendar entry
you just saved to be included in the results, then the recommended approach
is:
- Perform your cross entity group query in the normal way
- In your application code, insert the just inserted entity into the result
set if it's not there
- Show the user the merged result set

You can store away or track the just inserted calender entity in a cookie,
in a url / form parameter, in memcache, or in an HTTP session object (Java).

Fred



  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/lC6R_999_vQJ.

 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




-- 
Fred Sauer
Developer Advocate
Google Inc.
1600 Amphitheatre Parkway
Mountain View, CA 94043
fre...@google.com

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Should Google deprecate Master/Slave data store option?

[Fred Sauer]

On Sat, Aug 20, 2011 at 7:52 AM, Sekhar allur...@gmail.com wrote:

 We're about to launch a healthcare app, and we recently switched from M/S
 to HRD (both with reserved instances). I'm really, really happy with HRD so
 far, most importantly we don't see random hiccups in data access any more.
 Performance has been really consistent.


Great to hear. It's also exactly what I'm hearing from others I've talked to
who have switched from M/S to HRD.


We are occasionally getting a new error too much contention on these
 datastore entities. please try again. however. Clearly it's not due to
 traffic because it's like 5 of us testing the site at this point. I'm a bit
 worried our patients/doctors will see this after launch next month. What's
 this and is there a way to avoid it?


In both cases, you should design your application for at most 1 datastore
write/second (=1qps) per entity group. You can write simultaneously to as
many entity groups as you like.

The problem you mention usually comes up in one of two scenarios:

   1. You have a some shared / global data structure which is updated by
   multiple users at the same time, ultimately at a rate  1qps. See
   http://code.google.com/appengine/articles/sharding_counters.html for a
   common technique for dealing with this.
   2. You are testing with multiple users, but they are all logged in with
   the same account, or are otherwise artificially updating the same data. In
   this case you have a usage pattern which doesn't match what production
   traffic would look like.

If you're wanting to build high throughput pipelines, check out
http://www.google.com/events/io/2010/sessions/high-throughput-data-pipelines-appengine.html



  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/QygaPE8F_dwJ.

 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




-- 
Fred Sauer
Developer Advocate
Google Inc.
1600 Amphitheatre Parkway
Mountain View, CA 94043
fre...@google.com

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Should Google deprecate Master/Slave data store option?

[Fred Sauer]

On Sat, Aug 20, 2011 at 1:38 PM, thstart thst...@gmail.com wrote:

 +Jeff Schnitzer

  pry the details out of you.

 I posted a code if you care to read, the advice is EVENTUAL_CONSISTENCY
 which is a compromise and I don't need to do it in M/S.



When fetching a list of keys use max_entity_groups_per_rpc=1 for speed.


Use EVENTUAL_CONSISTENCY whenever you can as it has to the potential to
speed up your app. I say whenever you can because sometimes you won't be
able to.





 get better reception ...

 Yes I got a reception, Brett will update his presentations, Dan will update
 his book - I just purchased an access to his 2nd edition - a work in
 progress, and yes - I make every effort to do what is needed.

 The point is if you have a project which you really care you will do the
 same to make sure it works. If you have the time to thinker 24/7 with
 database experimenting, then good luck.

 Just checking - have you looked at the code I posted, made tests both in HR
 and M/S, performed benchmarks or you just want to talk not having  what else
 to do?
 Do this and then come back ton talk about real world situations.

 get a terapist
 If you are at the point to have a waiting list of paying customers, and you
 invested a year in developing, testing and almost to the final, and at this
 point you have to completely redesign you code without knowing in advance,
 if you an in areal world situation probably you will get a terapist, I don't
 need one.

  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/Q3CS2U8McpAJ.

 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




-- 
Fred Sauer
Developer Advocate
Google Inc.
1600 Amphitheatre Parkway
Mountain View, CA 94043
fre...@google.com

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Should Google deprecate Master/Slave data store option?

[Fred Sauer]

On Fri, Aug 19, 2011 at 1:38 AM, Stephen sdeasey+gro...@gmail.com wrote:



 On Thu, Aug 18, 2011 at 10:58 PM, thstart thst...@gmail.com wrote:


 I think Master/Slave is a lot faster in my case up to 10 times faster. I
 tested it with only read, no write operations,
 no transactions, nothing special for example I have a heavy use of this
 expression:

 e = MyList.get_profile(email)
 my_keys = e.my_keys
 my_members = db.get(my_keys)

 for member in my_members:
 member.property, etc. -- do something with the
 property


 Try this:

 e = MyList.get_profile(email)
  my_keys = e.my_keys
 my_members = db.get(my_keys, config=db.create_config(
 read_policy=db.EVENTUAL_CONSISTENCY))


If you don't want to give you eventual consistency, try this:

my_members = db.get(my_keys,
 config=db.create_config(max_entity_groups_per_rpc=1))


Similarly, in Java you can set max entity groups per RPC:
- Set  the appengine.datastore.defaultMaxEntityGroupsPerRpc system property
to 1
- Use
http://code.google.com/appengine/docs/java/javadoc/com/google/appengine/api/datastore/DatastoreServiceConfig.html#maxEntityGroupsPerRpc(int)





 for member in my_members:
 member.property, etc. -- do something with the
 property


 It's a relatively new option:


 http://code.google.com/intl/en/appengine/docs/python/datastore/functions.html#create_config

  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




-- 
Fred Sauer
Developer Advocate
Google Inc.
1600 Amphitheatre Parkway
Mountain View, CA 94043
fre...@google.com

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Should Google deprecate Master/Slave data store option?

[Fred Sauer]

On Fri, Aug 19, 2011 at 10:46 AM, Fred Sauer fre...@google.com wrote:



 On Fri, Aug 19, 2011 at 1:38 AM, Stephen sdeasey+gro...@gmail.com wrote:



 On Thu, Aug 18, 2011 at 10:58 PM, thstart thst...@gmail.com wrote:


 I think Master/Slave is a lot faster in my case up to 10 times faster. I
 tested it with only read, no write operations,
 no transactions, nothing special for example I have a heavy use of this
 expression:

 e = MyList.get_profile(email)
 my_keys = e.my_keys
 my_members = db.get(my_keys)

 for member in my_members:
 member.property, etc. -- do something with the
 property


 Try this:

 e = MyList.get_profile(email)
  my_keys = e.my_keys
 my_members = db.get(my_keys, config=db.create_config(
 read_policy=db.EVENTUAL_CONSISTENCY))


 If you don't want to give you eventual consistency, try this:

 my_members = db.get(my_keys,
  config=db.create_config(max_entity_groups_per_rpc=1))


Also, for reference, here's what this does:
  @ConfigOption
  def max_entity_groups_per_rpc(value):
The maximum number of entity groups that can be represented in one
rpc.

For a non-transactional operation that involves more entity groups than
the
maximum, the operation will be performed by executing multiple,
asynchronous
rpcs to the datastore, each of which has no more entity groups
represented
than the maximum.  So, if a put() operation has 8 entity groups and the
maximum is 3, we will send 3 rpcs, 2 with 3 entity groups and 1 with 2
entity groups.  This is a performance optimization - in many cases
multiple, small, concurrent rpcs will finish faster than a single large
rpc.  The optimal value for this property will be application-specific,
so
experimentation is encouraged.





 Similarly, in Java you can set max entity groups per RPC:
 - Set  the appengine.datastore.defaultMaxEntityGroupsPerRpc system property
 to 1
 - Use
 http://code.google.com/appengine/docs/java/javadoc/com/google/appengine/api/datastore/DatastoreServiceConfig.html#maxEntityGroupsPerRpc(int)





 for member in my_members:
 member.property, etc. -- do something with the
 property


 It's a relatively new option:


 http://code.google.com/intl/en/appengine/docs/python/datastore/functions.html#create_config

  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




 --
 Fred Sauer
 Developer Advocate
 Google Inc.
 1600 Amphitheatre Parkway
 Mountain View, CA 94043
 fre...@google.com





-- 
Fred Sauer
Developer Advocate
Google Inc.
1600 Amphitheatre Parkway
Mountain View, CA 94043
fre...@google.com

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Should Google deprecate Master/Slave data store option?

[Fred Sauer]

Fred on my Android.
On Aug 19, 2011 11:18 AM, thstart thst...@gmail.com wrote:

 +Ikai Lan
 +Fred Sauer
 +Stephen

 Thank you, I need to rethink again with this information: it is useful. I
am fine
 with new technologies, in my case the product is very sophisticated and
the balance
 between new Google technology and what my business requires was tilted to
business
 side - e.g. less new technology and techniques, more pro business
solution.

 Definitely an extension to the book, an update to the excellent Brett
Slatkin (he explains very well)
 materials will be of much help and am awaiting them and am wondering how i
will know if they are ready?

 There is some practical questions: these changes obviously have to be
tested,

You can also deploy another app version for testing.

I am fine to set up a second
 app just for that, but still it would be not the real thing in real
situations. Right now I have a group of trusted people
 who are already using it and looking at the logs I can see real life
situations which never pop up in test accounts.
 So my question is - I obviously will want to implement the changes one by
one and see the effect, being right now
 on M/S could this be done in M/S environment and then move my users to HR?

The impact of these changes are material on HRD, and may have limited
performance impact on M/S.

Test where you like, but test the scenario which you plan to use.

Above all else, migrate to HRD as soon as practical.


 Also I suppose when Google goes full HR and obviously wants we to move out
of M/S, do Google has benchmarks
 for this kind of usage as I described in order to have a good motivation
to move to HR?

While you should care that your app has acceptable performance, and you
should use the suggested tips to get optimal performance out of HRD, your
primary concern or reasons for migrating off of M/S are
1 Avoiding scheduled read-only periods
2 Avoiding unscheduled outages and maintenance periods
3 Avoiding datastore latency spikes

Every developer who I've communicated with has been happy with their app on
HRD and none want to go back. In particular the 3rd point I mentioned often
weighs the heaviest for developers.

To put it another way: I have yet to find a use case, when the whole app is
considered, where M/S is preferable to HRD.

Hope that helps.
Fred


 Thanks to all!



 --
 You received this message because you are subscribed to the Google Groups
Google App Engine group.
 To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/cmDqAoiZDiwJ.

 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Should Google deprecate Master/Slave data store option?

[Fred Sauer]

Feel free to set the default to 1 for your apps.

Fred on my Android.
On Aug 19, 2011 11:52 AM, Stephen sdeasey+gro...@gmail.com wrote:
 On Fri, Aug 19, 2011 at 6:49 PM, Fred Sauer fre...@google.com wrote:

 If you don't want to give you eventual consistency, try this:

 my_members = db.get(my_keys,
  config=db.create_config(max_entity_groups_per_rpc=1))

 Also, for reference, here's what this does:

   @ConfigOption
   def max_entity_groups_per_rpc(value):
 The maximum number of entity groups that can be represented in one
 rpc.
 For a non-transactional operation that involves more entity groups
than
 the
 maximum, the operation will be performed by executing multiple,
 asynchronous
 rpcs to the datastore, each of which has no more entity groups
 represented
 than the maximum.  So, if a put() operation has 8 entity groups and
the
 maximum is 3, we will send 3 rpcs, 2 with 3 entity groups and 1 with
2
 entity groups.  This is a performance optimization - in many cases
 multiple, small, concurrent rpcs will finish faster than a single
large
 rpc.  The optimal value for this property will be
application-specific,
 so
 experimentation is encouraged.
 

 It feels like you'd always want to set this to 1. What situations
 might make 1 perform better?

 --
 You received this message because you are subscribed to the Google Groups
Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Should Google deprecate Master/Slave data store option?

[Fred Sauer]

You should mirate to HRD if you care about consistent, predictable speed,
performance and reliability. Your app will be happier. You will be happier.
Your customers will be happier.

The advice to migrate to HRD is unqualified: you should migrate.

Fred on my Android.
On Aug 19, 2011 3:51 PM, thstart thst...@gmail.com wrote:
 A bit from business side reality - today my partners where on a meeting
 with a BIG customer representatives. Functionality and speed especially
 (M/S currently) impressed them so much that they wanted to add 500 users
 immediately
 to our app and paid for it to be sure they get an access.

 Our app is with Billing On so I don't see this High Replication vs.
Master/
 Slave
 thing as an issue for Google - we are paying customers.

 So now the question is open before me and my partners especially:
 1) Things are moving very fast from business side and I will mark the app
 from Alpha
 to Beta soon after looking how it works with more simultaneous users.
 2) The users are already feeling the speed and any speed decrease will be
 mentioned
 immediately. That is because the app is actually a mobile HTML5 app and
used
 in
 all kind of devices: Android, iPhone, Blackberry. For such kind of app the

 speed is of
 1st priority and any decision about functionality is rejected if the speed

 is slowed down.
 3) Now the question is what benefit we will get (if any) moving from M/S
to
 HR WITHOUT
 disrupting the current functionality and speed? Does the amount of time,
 effort and cost
 which need to be invested in migration to HR is justified? As a start-up
 this is critical for us.
 Every day we have a NEW list of priorities which are business related -
 users need more and more -
 we get this from the feedback. Nobody cares about underlying technology if

 it works good enough.

 Best,
 --Constantine



 --
 You received this message because you are subscribed to the Google Groups
Google App Engine group.
 To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/iDQzYaVOSioJ.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Should Google deprecate Master/Slave data store option?

[Fred Sauer]

On Wednesday, August 17, 2011 5:24:48 AM UTC-7, Joshua Smith wrote:

 I certainly hope they don't deprecate M/S, unless/until the following 
 issues are addressed:

 1) Ability to migrate without changing app ID.  This is fundamental, and 
 we've never seen any *reason* why this is impossible.  I suspect that this 
 is very hard, but not impossible.

We are able to create an alias for migrated apps, so that traffic sent to 
oldappid.appspot.com will be transparently routed to your new application. 
So while you won't be able to keep the same app id, you can keep using the 
same appspot.com domain, which is probably what you care about most.

 

  Changing the app ID causes problems because: 

- External references to entity keys (not IDs) are pervasive in many apps 
 (such as google search links to deep pages, user bookmarks, or RSS feed 
 URLs) because the documentation encourages the use of these keys in URLs

This remains an issues, but can be easily addressed in a couple of lines of 
application code in the appropriate request handlers. The string encoded 
keys that are referenced externally have encoded within them the old app id. 
If you take the decoded key and extract the parent, kind and id/name to 
create a new key (again, this is 1-2 lines of code), then your app can use 
that new key to locate the migrated entity within the new datastore.

As far as reference (list) properties: those reference keys are 
automatically updated as part of the the datastore migration and should not 
cause you any issues.


 

 - Many users have keys wrapped up in properties that the migration tool 
 cannot recognize [this doesn't impact my app, but it's a common issue on 
 this list]

This does affect some apps. As I mentioned above, a couple of lines of code 
in the right place should take care of this.

 

 As an alternative, the infrastructure could be fixed to recognize the case 
 of using the old app ID instead of the new app ID in a key, and quietly make 
 the substitution.

That's a neat idea.

 

 2) Migration of large data sets takes an extremely long time.  Given that 
 we regularly see M/S downtimes in the couple of hours range, a couple of 
 hours is probably a reasonable amount of time to migrate.  A couple of days 
 is much too much.  [Note that I have not personally gotten far enough into a 
 migration to see this duration; I'm basing this on many user complaints I've 
 read on this list.]

Applications with small amounts of data can easily self migrate in a short 
period of time using datastore_admin. Applications with more data will 
benefit from the new migration tool Ikai mentioned. We created the tool 
specifically to simplify and speed up the migration for larger apps, and to 
reduce the read-only period required to make the final switch.

 

 3) The problem of transactional consistency needs to be addressed.  It is 
 not acceptable for transactions to end in a maybe this worked, maybe it 
 didn't state.  They should either succeed or fail.  (Although the docs not 
 not say whether this problem is exclusive to HR, it seems to be, and 
 certainly it matters much more in HR since every put is implicitly 
 transactional.)

There are subtle differences between the consistency guarantees offered by 
M/S and HRD. In some cases, apps need to make a few tweaks in order to work 
correctly in the new environment. The benefit is much more 
predictable/consistency latency and increased reliability. A good article 
which reviews the consistency guarantees can be found here:
  http://code.google.com/appengine/articles/transaction_isolation.html


4) As Ikai said, there are some unresolved issues in HR that need to be 
 fixed, such as the current mystery in the 'Serious Problem: Rollback of data 
 on HRD' thread (which is eerily familiar: I think somebody else reported a 
 similar issue a while back).


I haven't looked at that thread yet, but I suspect that reviewing the above 
article 
(http://code.google.com/appengine/articles/transaction_isolation.html) would 
probably be beneficial in the context of that thread.


Even if all these are addressed, it would certainly be nice if M/S remained 
 an option.  It makes different trade-offs than HR, which may be better 
 trade-offs for some applications.  I will be very disappointed if they 
 decide to make HR a requirement for multi-threading Python, since the two 
 are completely orthogonal choices.  From my reading, M/S seems to have 
 better throughput than HR, at the expense of more frequent downtime, and 
 spurious datastore timeouts.  I'd like to have that option in a 
 multi-threaded app.


Thanks for the input. I hope the above response addresses some of your 
concerns.

 

 -Joshua



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/sLkBsAfFc84J.
To post to this group, send email to 

Re: [google-appengine] Should Google deprecate Master/Slave data store option?

[Fred Sauer]

That's accurate. Performing a batch get to get a N entities from different
entity groups is slower or HRD. As N increases the issue is more pronounced.
This is because the datastore needs to fetch a consistent copy for each
entity.

Placing these entities into a single entity group can help greatly. Fetching
fewer entities initially, so that the UI updates quickly, and then fetching
more as needed, is always a good idea.

A great tool to help understand your use of the datastore is Appstats, which
can be easily enabled and provides negligible overhead to most sites. (If
you have a high traffic site, or don't want to enable Appstats in production
for other reasons, you can easily deploy a non-default version of your app
with Appstats enabled). Appstats will provide you a graphical overview of
the datastore (and other RPC) calls you are making.

On Thu, Aug 18, 2011 at 12:19 PM, Robert Kluin robert.kl...@gmail.comwrote:

 When switching to the HR datastore, you might need to make some
 adjustments.  For instance, fetching a large list of keys from
 multiple entity groups may be slower due to the transaction overhead.
 So you might need to evaluate your use of entity groups.  It may also
 be that you've got a design that is making many datastore RPCs in
 serial, that will probably be more of an issue when on HR since there
 is a slightly higher average latency.


 Robert





 On Thu, Aug 18, 2011 at 15:11, thstart thst...@gmail.com wrote:
  Let me share my experience.
  I was developing an AppEngine app for a year.
  It was very fast (extremely fast), with latency  1 sec.
  When High Replication data-store was released I decided to give it
  a try and switched to it. Unfortunately the latency got 1 Sec
  sometimes  2 Sec. The app worked a wile that way and because
  still in beta I could experiment. So I decided to go to Billed option
  to see if the reliability and more importantly the latency could improve.
  Unfortunately this do not affected latency. Then I decided to go back to
  Master/Slave - our customers mentioned immediately our app is 2 times
  faster - again  1 Sec latency. Then I needed to add https which is
  very important for our app. This do not added significant latency so our
  customers are very happy with this solution. Soon we will go public.
  So the latency issue is a show stopper for high replication version
  and we would want to stay with master slave.
  Anybody with similar experience?
  Thank you,
 
 
 
  --
  You received this message because you are subscribed to the Google Groups
  Google App Engine group.
  To view this discussion on the web visit
  https://groups.google.com/d/msg/google-appengine/-/P1c_gMSRtl0J.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
  http://groups.google.com/group/google-appengine?hl=en.
 

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




-- 
Fred Sauer
Developer Advocate
Google Inc.
1600 Amphitheatre Parkway
Mountain View, CA 94043
fre...@google.com

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Should Google deprecate Master/Slave data store option?

[Fred Sauer]

I recommend looking at Appstats, and verifying whether you are performing
batch gets across entity groups. I have not met an app yet that is unable to
get acceptable HRD performance.

I should note that apps built on M/S have (usually unintentionally) been
optimized for the M/S performance characteristics. It makes sense that some
apps may need to or want to make adjustments before migrating to HRD.

I should also point out that I have yet to meet a developer whose not
happier on HRD and migrating their app. You may well be the first (although
I'm not trying to imply that you are the only one)... :)

On Thu, Aug 18, 2011 at 12:11 PM, thstart thst...@gmail.com wrote:

 Let me share my experience.

 I was developing an AppEngine app for a year.
 It was very fast (extremely fast), with latency  1 sec.
 When High Replication data-store was released I decided to give it
 a try and switched to it. Unfortunately the latency got 1 Sec
 sometimes  2 Sec. The app worked a wile that way and because
 still in beta I could experiment. So I decided to go to Billed option
 to see if the reliability and more importantly the latency could improve.
 Unfortunately this do not affected latency. Then I decided to go back to
 Master/Slave - our customers mentioned immediately our app is 2 times
 faster - again  1 Sec latency. Then I needed to add https which is
 very important for our app. This do not added significant latency so our
 customers are very happy with this solution. Soon we will go public.

 So the latency issue is a show stopper for high replication version
 and we would want to stay with master slave.

 Anybody with similar experience?

 Thank you,




  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/P1c_gMSRtl0J.

 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




-- 
Fred Sauer
Developer Advocate
Google Inc.
1600 Amphitheatre Parkway
Mountain View, CA 94043
fre...@google.com

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Should Google deprecate Master/Slave data store option?

[Fred Sauer]

On Thu, Aug 18, 2011 at 11:48 AM, Joshua Smith joshuaesm...@charter.netwrote:


 On Aug 18, 2011, at 2:10 PM, Fred Sauer wrote:

 On Wednesday, August 17, 2011 5:24:48 AM UTC-7, Joshua Smith wrote:

 I certainly hope they don't deprecate M/S, unless/until the following
 issues are addressed:

 1) Ability to migrate without changing app ID.  This is fundamental, and
 we've never seen any *reason* why this is impossible.  I suspect that this
 is very hard, but not impossible.

 We are able to create an alias for migrated apps, so that traffic sent to
 oldappid.appspot.com will be transparently routed to your new application.
 So while you won't be able to keep the same app id, you can keep using the
 same appspot.com domain, which is probably what you care about most.


 Perhaps that what some people care about most, but not me.  What I care
 about is what I spelled out in the next two bullet points :)


  Changing the app ID causes problems because:

 - External references to entity keys (not IDs) are pervasive in many apps
 (such as google search links to deep pages, user bookmarks, or RSS feed
 URLs) because the documentation encourages the use of these keys in URLs

 This remains an issues, but can be easily addressed in a couple of lines of
 application code in the appropriate request handlers. The string encoded
 keys that are referenced externally have encoded within them the old app id.
 If you take the decoded key and extract the parent, kind and id/name to
 create a new key (again, this is 1-2 lines of code), then your app can use
 that new key to locate the migrated entity within the new datastore.

 As far as reference (list) properties: those reference keys are
 automatically updated as part of the the datastore migration and should not
 cause you any issues.

 - Many users have keys wrapped up in properties that the migration tool
 cannot recognize [this doesn't impact my app, but it's a common issue on
 this list]

 This does affect some apps. As I mentioned above, a couple of lines of code
 in the right place should take care of this.


 1-2 lines of code, in dozens of handlers is the crux of the problem.
  Dealing with this at the application layer is problematic because there are
 so many code paths that need to be examined.



 As an alternative, the infrastructure could be fixed to recognize the case
 of using the old app ID instead of the new app ID in a key, and quietly make
 the substitution.

 That's a neat idea.


 Neat enough that you guys might implement it?

 I've suggested this before, but I'm glad that I've got your attention this
 time!


I'll certainly bring up with the team.




 This would be a pretty huge step toward making the migration practical for
 me.


 2) Migration of large data sets takes an extremely long time.  Given that
 we regularly see M/S downtimes in the couple of hours range, a couple of
 hours is probably a reasonable amount of time to migrate.  A couple of days
 is much too much.  [Note that I have not personally gotten far enough into a
 migration to see this duration; I'm basing this on many user complaints I've
 read on this list.]

 Applications with small amounts of data can easily self migrate in a short
 period of time using datastore_admin. Applications with more data will
 benefit from the new migration tool Ikai mentioned. We created the tool
 specifically to simplify and speed up the migration for larger apps, and to
 reduce the read-only period required to make the final switch.


 Yes, I saw that in his note, and if it works as advertised, I suspect it
 will resolve this issue.


 3) The problem of transactional consistency needs to be addressed.  It is
 not acceptable for transactions to end in a maybe this worked, maybe it
 didn't state.  They should either succeed or fail.  (Although the docs not
 not say whether this problem is exclusive to HR, it seems to be, and
 certainly it matters much more in HR since every put is implicitly
 transactional.)

 There are subtle differences between the consistency guarantees offered by
 M/S and HRD. In some cases, apps need to make a few tweaks in order to work
 correctly in the new environment. The benefit is much more
 predictable/consistency latency and increased reliability. A good article
 which reviews the consistency guarantees can be found here:
   http://code.google.com/appengine/articles/transaction_isolation.html


 I'm not talking about consistency guarantees.  I get that things change
 under HR, and it's pretty easy to understand what the implications of that
 are.

 What I'm talking about is the problem where you have to write idempotent
 transactions because you can never tell whether a transaction that threw
 certain exceptions succeeded or failed.  That's really messed up, and you
 really need to fix that.

 But, to be fair, I'm not sure this problem will cause me actual problems if
 I switch to HR.  It just makes me really, really uncomfortable.


 4) As Ikai said, there are some

[google-appengine] remote_api and java - put in batches

[Fred Ross-Perry]

I'm working on a tool using Java on the client side to backup and 
restore data.
The restore side involves reconstructing entities form XML data, and 
putting them

to the datastore.  Right now I am doing them one at a time, like this:

while (there are more entites)
{
Entity entity = convert from XML to Entity here
datastore.put(entity);
}

But I'd like to do it in batches.
I see that there is also

datastore.put(IterableEntity)

How do I build up an IterableEntity form a series of Entitys?

thanks

--
You received this message because you are subscribed to the Google Groups Google 
App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [appengine-java] Re: Gettin white page when trying to log in - cause of exception.... even after deleting all _ah_SESSION Entities???

[Fred Pallesen]

we added an admin-servlet to call clearAll() on the cache.
note you may have to clear all sessions too, in the appspot Datastore Viewer
... as this will be loaded on next version and may conflict with a next
serialVersionUID

how may this be done programmatically? anyone?

- Fred

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [google-appengine] Error logging into domain

[Fred Ross-Perry]

Ok thanks. 

On Feb 16, 2011, at 4:37 PM, Jon McAlister jon...@google.com wrote:

 Sorry about that. We were just testing out a change, but it does not
 work for a certain class of google accounts. Have reverted the change
 so it should be working again.
 
 On Wed, Feb 16, 2011 at 3:09 PM, fross-pe...@conceptuamath.com
 fross-pe...@conceptuamath.com wrote:
 When I go here:
 
  https://appengine.google.com/a/conceptuamath.com/
 
 and login with my credentials, I get
 
  Error: Server Error
 
  The server encountered an error and could not complete your request.
  If the problem persists, please report your problem and mention this
 error message and the query that caused it.
 
 This started happening today, 2/16/2011, on FF, Safari and Chrome on OS X.
 Thoughts?  I am stuck.  Thanks.
 
 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.
 

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[appengine-java] Re: Request was aborted after waiting too long to attempt to service your request.

[Fred]

I'm also experiencing this with two larger apps that use Guice and had
the suspicion it appears due to Guice start-up time.
On a smaller app that has only few bindings it does not seem to occur.
Anyone sharing this experience?

Cheers,
Fred

On Nov 4, 7:27 pm, James jamesk...@gmail.com wrote:
 Also, pretty much EVERYONE who doesn't dislike reliability should be
 starring this issue 
 -http://code.google.com/p/googleappengine/issues/detail?id=2396

 :p

 On Nov 4, 1:25 pm, James jamesk...@gmail.com wrote:

  Someone correct me if any of this is inaccurate, but:
  - This occurs exactly 10s into a request.
  - They are not catchable; your code doesn't even start to spin up;
  there's nothing you can do about it.
  - Status page -http://code.google.com/status/appengine-usually
  shows all is fine when this occurs.
  - These are a fact of life w/ GAE right now.

  This has been my experience, at least.  Just got one on my own app
  which prompted me to look around and see if there was any news on the
  issue.

  James

  On Oct 28, 6:11 pm, Shaun Clark shaun.cl...@lexmatter.com wrote:

   I'm sure I'm doing something wrong, but I feel like I am getting a lot
   of these:

   10-28 02:36PM 34.293 /googlesyncactions.do?action=syncemailforeveryone
   500 10005ms 0cpu_ms 0kb AppEngine-Google; (+http://code.google.com/
   appengine)
   W 10-28 02:36PM 44.299Request was aborted after waiting too long 
   toattempt to service your request. This may happen sporadically when the
   App Engine serving cluster is und
   10-28 02:34PM 31.118 /googlesyncactions.do 500 10302ms 0cpu_ms 0kb
   AppEngine-Google; (+http://code.google.com/appengine)
   W 10-28 02:34PM 41.420Request was aborted after waiting too long 
   toattempt to service your request. This may happen sporadically when the
   App Engine serving cluster is und
   10-28 02:34PM 31.089 /googlesyncactions.do 500 10008ms 0cpu_ms 0kb
   AppEngine-Google; (+http://code.google.com/appengine)
   W 10-28 02:34PM 41.098Request was aborted after waiting too long 
   toattempt to service your request. This may happen sporadically when the
   App Engine serving cluster is und
   10-28 02:34PM 31.066 /googlesyncactions.do 500 10020ms 0cpu_ms 0kb
   AppEngine-Google; (+http://code.google.com/appengine)
   W 10-28 02:34PM 41.086Request was aborted after waiting too long 
   toattempt to service your request. This may happen sporadically when the
   App Engine serving cluster is und
   10-28 02:34PM 31.035 /googlesyncactions.do 500 10050ms 0cpu_ms 0kb
   AppEngine-Google; (+http://code.google.com/appengine)
   W 10-28 02:34PM 41.085Request was aborted after waiting too long 
   toattempt to service your request. This may happen sporadically when the
   App Engine serving cluster is und
   10-28 02:34PM 19.565 /dashboard 500 10280ms 0cpu_ms 0kb AppEngine-
   Google; (+http://code.google.com/appengine)
   W 10-28 02:34PM 29.846Request was aborted after waiting too long 
   toattempt to service your request. This may happen sporadically when the
   App Engine serving cluster is und
   10-28 02:16PM 17.637 /settingsActions.do?
   action=updateSettingssettingsjson=%7B%22key
   %22:%22aglsZXhtYXR0ZXJyGgsSEk1haW5DbGllbnRTZXR0aW5ncxja6xgM%22,%22name
   %22:%22LexMatter%22,%22url%22:%22lexmatter%22,%22paypalusername
   %22:%22shaunc869%22,%22paypalpassword
   %22:%22shaunc869%22,%22defaultbillingrate
   %22:%220.0%22,%22targetbillingsperyear%22:%22871,000%22,%22firmyearend
   %22:%2212/23/2010%22,%22paypaldepositaccount
   %22:%22aglsZXhtYXR0ZXJyEwsSC0JhbmtBY2NvdW50GKn1Ggw
   %22,%22adwordsaccountnumber%22:%22null
   %22%7Dsettingskey=aglsZXhtYXR0ZXJyGgsSEk1haW5DbGllbnRTZXR0aW5ncxja6xgM
   500 117ms 133cpu_ms 16api_cpu_ms 0kb Mozilla/5.0 (Windows; U; Windows
   NT 6.1; en-US) AppleWebKit/534.7 (KHTML, like Gecko) Chrome/7.0.517.41
   Safari/534.7,gzip(gfe),gzip(gfe)
   W 10-28 02:16PM 17.743 /settingsActions.do
   com.google.gson.JsonParseException: The JsonDeserializer
   DoubleDeserializer failed to deserialized json object 871,000 given
   the
   C 10-28 02:16PM 17.746 Uncaught exception from servlet
   com.google.gson.JsonParseException: The JsonDeserializer
   DoubleDeserializer failed to deserialized json object 871,00
   10-28 02:09PM 54.105 /dashboard 500 10297ms 0cpu_ms 0kb AppEngine-
   Google; (+http://code.google.com/appengine)
   W 10-28 02:10PM 04.403Request was aborted after waiting too long 
   toattempt to service your request. This may happen sporadically when the
   App Engine serving cluster is und
   10-28 02:09PM 52.418 /googlesyncactions.do 500 10085ms 0cpu_ms 0kb
   AppEngine-Google; (+http://code.google.com/appengine)
   W 10-28 02:10PM 02.504Request was aborted after waiting too long 
   toattempt to service your request. This may happen sporadically when the
   App Engine serving cluster is und
   10-28 02:09PM 52.384 /googlesyncactions.do 500 10118ms 0cpu_ms 0kb
   AppEngine-Google; (+http://code.google.com/appengine)
   W 10-28 02:10PM 02.503Request

[appengine-java] appcfg update_indexes does not require a password?

[Fred]

Hi Group,

am I the only one who can update his indexes without requiring to
enter a password to do so? If not, am I the only one who considers
that being scary? If yes, why should I not worry?

Here is what I just got from appcfg:

29.10.2010 21:35:04 java.util.prefs.WindowsPreferences init
WARNUNG: Could not open/create prefs root node Software\JavaSoft\Prefs
at root 0x8002. Windows RegCreateKeyEx(...) returned error code 5.
Reading application configuration data...
29.10.2010 21:35:05
com.google.apphosting.utils.config.AppEngineWebXmlReader
readAppEngineWebXml
INFO: Successfully processed war\WEB-INF/appengine-web.xml
29.10.2010 21:35:05
com.google.apphosting.utils.config.AbstractConfigXmlReader
readConfigXml
INFO: Successfully processed war\WEB-INF/web.xml
29.10.2010 21:35:05
com.google.apphosting.utils.config.AbstractConfigXmlReader
readConfigXml
INFO: Successfully processed war\WEB-INF/cron.xml
29.10.2010 21:35:05
com.google.apphosting.utils.config.AbstractConfigXmlReader
readConfigXml
INFO: Successfully processed war\WEB-INF/queue.xml
29.10.2010 21:35:05
com.google.apphosting.utils.config.IndexesXmlReader readConfigXml
INFO: Successfully processed war\WEB-INF/datastore-indexes.xml
Beginning server interaction for #obscuredappid#...
0% Uploading index definitions.
Success.
Cleaning up temporary files...

Hope somebody can shed some light :-)

Cheers and thanks,
Fred

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: JDO unique Long ID keys across entity groups

[Fred]

Thanks guys! Think I'll just concatenate parent and child ID for
display purposes and keep the rest as it is.

Fred

On 1 Okt., 08:16, John Patterson jdpatter...@gmail.com wrote:
 You can use DatastoreService.allocateIds() to get a range of unique  
 keys that you can then assign to the children.  The kind name can be  
 any old String - not just an existing kind.  This method does not  
 guarantee the ids will be sequential but they will be unique.

 If you really need sequential ids you can use memcache's atomic  
 increment method.  If the value is not found in memcache you can do a  
 query to find the maximum sequence number in the datastore and use the  
 memcache increment method that allows you to specify an initial  
 missing value atomically.

 John

 On 30 Sep 2010, at 17:28, Fred wrote:

  Hi Group,

  I know key ids are only unique across entity groups so what GAE + JDO
  gives me is:
  /Parent[1]/Child[1]
  /Parent[1]/Child[2]
  /Parent[1]/Child[3]

  /Parent[2]/Child[1]
  /Parent[2]/Child[2]

  /Parent[3]/Child[1]

  etc.

  What I need to achieve (aka implement) is a way to assign keys in the
  following fashion:
  /Parent[1]/Child[1]
  /Parent[1]/Child[2]
  /Parent[1]/Child[3]

  /Parent[2]/Child[4]
  /Parent[2]/Child[5]

  /Parent[3]/Child[6]

  etc... so child ids are in sequence across entity groups (or at least
  unique). Does anyone have experience with this or any thoughts on how
  to go about it in a secure fashion?
  I can't seem to come up with a secure way of doing this, but don't
  want to give up on it because it would give my use case a lot of
  querying power.

  Hope someone can help.
  Thanks in advance,
  Fred

  --
  You received this message because you are subscribed to the Google  
  Groups Google App Engine for Java group.
  To post to this group, send email to google-appengine-java@googlegroups.com
  .
  To unsubscribe from this group, send email to 
  google-appengine-java+unsubscr...@googlegroups.com
  .
  For more options, visit this group 
  athttp://groups.google.com/group/google-appengine-java?hl=en
  .

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] JDO unique Long ID keys across entity groups

[Fred]

Hi Group,

I know key ids are only unique across entity groups so what GAE + JDO
gives me is:
/Parent[1]/Child[1]
/Parent[1]/Child[2]
/Parent[1]/Child[3]

/Parent[2]/Child[1]
/Parent[2]/Child[2]

/Parent[3]/Child[1]

etc.

What I need to achieve (aka implement) is a way to assign keys in the
following fashion:
/Parent[1]/Child[1]
/Parent[1]/Child[2]
/Parent[1]/Child[3]

/Parent[2]/Child[4]
/Parent[2]/Child[5]

/Parent[3]/Child[6]

etc... so child ids are in sequence across entity groups (or at least
unique). Does anyone have experience with this or any thoughts on how
to go about it in a secure fashion?
I can't seem to come up with a secure way of doing this, but don't
want to give up on it because it would give my use case a lot of
querying power.

Hope someone can help.
Thanks in advance,
Fred


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Typing mistake in Java Application Configuration documentation: Inbound Services?

[Fred Janon]

I think (I hope!) there is a slight mistake on this page:

http://code.google.com/appengine/docs/java/config/appconfig.html

in the paragraph:

*Inbound Services*

Before an application can receive email and XMPP messages, the application
must be configured to enable those services. You enable these services for a
*Python *app by including an inbound-services element in the
appengine-web.xml file.

I think it should say Java instead of Python. Can someone confirm?

Thanks

Fred

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: Any examples on how to use the data store low level Java API?

[Fred Janon]

Hi Tristan

Sorry I didn't answer earlier, I got sick and I am trying to catch up
with other things at the moment.

Thanks for your answer and your example. I will take up on your offer
some time soon.

Cheers,

Fred

On Tue, Apr 13, 2010 at 01:03, Tristan tristan.slomin...@gmail.com wrote:
 I should be able to help... what are you trying to do?

 to use a predefined type that is storable in the datastore (http://
 code.google.com/appengine/docs/java/datastore/
 dataclasses.html#Core_Value_Types)

 all you do is cast that property  so..

 Entity entity = new Entity(myKind);
 Email myEmail = new Email(g...@email.com);
 entity.setProperty(emailProperty, myEmail);
 Key key = datastore.put(entity);

 to get it back just cast it

 Entity storedEntity = datastore.get(key);
 Email storedEmail = (Email) storedEntity.getProperty(emailProperty);

 they also appear to implement Comparable interface, so you can compare
 these things...

 is there anything more specific you need?

 Tristan

 On Apr 9, 4:31 am, Fred Janon fja...@gmail.com wrote:
 Hi,

 Are there any examples on how to use the low level data store API,
 other than the short example on the Javadoc? Especially on how to use
 the predefined types like email, GeoPt, address... Is the source code
 available for these objects? I am interested in the compare method for
 these objects.

 Thanks

 Fred

 --
 You received this message because you are subscribed to the Google Groups 
 Google App Engine for Java group.
 To post to this group, send email to google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine-java+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine-java?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: Any examples on how to use the data store low level Java API?

[Fred Janon]

Thanks a lot for the pointer, Didier. I'll buy the book ASAP.

Sorry I didn't answer earlier, I got sick and I am trying to catch up
with other things at the moment.

Fred

On Tue, Apr 13, 2010 at 22:14, Didier Durand durand.did...@gmail.com wrote:
 Hi Fred,

 The most detailed samples that I found up to now are in this book:
 http://www.amazon.com/Programming-Google-App-Engine-Infrastructure/dp/059652272X/ref=sr_1_1?ie=UTF8s=booksqid=1271167954sr=8-1

 Many chapters devoted to Datastore in all its aspects. The definite
 reading for your needs!
 regards
 didier

 On Apr 9, 11:31 am, Fred Janon fja...@gmail.com wrote:
 Hi,

 Are there any examples on how to use the low level data store API,
 other than the short example on the Javadoc? Especially on how to use
 the predefined types like email, GeoPt, address... Is the source code
 available for these objects? I am interested in the compare method for
 these objects.

 Thanks

 Fred

 --
 You received this message because you are subscribed to the Google Groups 
 Google App Engine for Java group.
 To post to this group, send email to google-appengine-j...@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine-java+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine-java?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Any examples on how to use the data store low level Java API?

[Fred Janon]

Hi,

Are there any examples on how to use the low level data store API,
other than the short example on the Javadoc? Especially on how to use
the predefined types like email, GeoPt, address... Is the source code
available for these objects? I am interested in the compare method for
these objects.

Thanks

Fred

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[google-appengine] app engine down for me too

[Fred]

Getting 500 server errors this morning when I try to access any of
our apps on GAE (a couple of hundred apps).

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Printing a Google Code page shows the footer at the top of the page...

[Fred Janon]

It seems more a Google Code issue than GAE, but I couldn't find a way to
file the issue on any of the Google Code pages.

When I print a page, the footer with the license information prints at the
top of the page, over the header and title. This shows with FF3 and Opera as
well. I checked the HTML/CSS code and it seems that the problem comes
from *clear:
both* in the footer* **'div** id=gc-footer'. *It's on all the GAE web
pages, but I am pretty sure that's everywhere in Google Code.

If someone can fix that, that would be great.

Fred

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Extra '%' and blank lines on the Using JPA with App Engine web page

[Fred Janon]

Not sure where to report it, so here is is:

http://code.google.com/appengine/docs/java/datastore/usingjpa.html

There a superflous '%' character that should not be there and a bunch
of useless blank lines (no big deal).

Source:
=
%!DOCTYPE html PUBLIC -//W3C//DTD XHTML 1.0 Transitional//EN DTD/
xhtml1-transitional.dtd





























html
  head
=

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: New App Engine Java and Python forums

[Fred Janon]

Excellent! I was waiting for that to port my intranet app from Tomcat onto
GAE and move my customer to Google Apps.

Hopefully GAE can run a Groovy/Grails app?

Fred

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] random 404 responses

[Fred]

Hi,

I've just noticed that my app is returning random 404 responses for
some requests (such as javascript, images or ajax GET requests).
Hitting refresh seems to return different 404 responses each time -
sometimes the request loads OK, sometimes it doesn't...

I'm using Firebug in Firefox to view files as they are loaded. Safari
seems to have the same problem. The issue appears to have started
recently - any ideas?


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: random 404 responses

[Fred]

appears to be fixed at the moment...

On Jan 28, 12:54 pm, Fred fred.leef...@gmail.com wrote:
 Looks like it might be related to this (I'm based in the 
 UK):http://groups.google.com/group/google-appengine-downtime-notify/brows...

 On Jan 28, 12:50 pm, Fred fred.leef...@gmail.com wrote:

  Hi,

  I've just noticed that my app is returning random 404 responses for
  some requests (such as javascript, images or ajax GET requests).
  Hitting refresh seems to return different 404 responses each time -
  sometimes the request loads OK, sometimes it doesn't...

  I'm using Firebug in Firefox to view files as they are loaded. Safari
  seems to have the same problem. The issue appears to have started
  recently - any ideas?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Support for Decimal in Datastore?

[Fred Janon]

Float and Decimal are two different types. Float is used in scientific
calculations, Decimal for accounting/financial applications. Decimal exists
in Python but doesn't seem to be implemented in the Datastore.

Fred

On Fri, Dec 5, 2008 at 14:58, lock [EMAIL PROTECTED] wrote:


 There is the FloatProperty, pretty sure that's what your after.


 http://code.google.com/appengine/docs/datastore/typesandpropertyclasses.html#FloatProperty

 


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] 1.2 high CPU warning accessing Robots.txt?

[Fred]

Hi,

I was just checking my warning logs in the dashboard and noticed this:

65.55.231.106 - - [04/12/2008:03:23:08 -0800] GET /robots.txt HTTP/
1.1 404 84 - -

This request used a high amount of CPU, and was roughly 1.2 times over
the average request CPU limit. High CPU requests have a small quota,
and if you exceed this quota, your app will be temporarily disabled.

Is CPU time reporting accurate, or is it still being perfected?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Timeout - operation took too long

[Fred]

I seem to be getting unusual timeouts too... www.bigbikemap.com

On Nov 26, 8:28 am, ksjun [EMAIL PROTECTED] wrote:
 I am also. Just single put operation somtimes caused 'datastore
 timeout: operation took too long.'

 - my application id : lifography

 My application doesn't exceed to any quota. (1%)

 One of this error was occured at  Nov 26,2008 AM 12:06 (PST).

 
  File /base/python_lib/versions/1/google/appengine/ext/db/
 __init__.py, line 656, in put
return datastore.Put(self._entity)

  File /base/python_lib/versions/1/google/appengine/api/datastore.py,
 line 162, in Put
raise _ToDatastoreError(err)

  File /base/python_lib/versions/1/google/appengine/api/datastore.py,
 line 1636, in _ToDatastoreError
raise errors[err.application_error](err.error_detail)

 Timeout: datastore timeout: operation took too long.
 -

 On 10월11일, 오후8시19분, gg [EMAIL PROTECTED] wrote:

  I am also seeing a lot of time spent in WAIT? Sometimes as much as 3/4
  of total CPU for any given set of calls! Has thedatastorelatency
  issue mentioned above been resolved?

  Here is an example from a profile run:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
 110.6490.0590.9370.085
  {google3.apphosting.runtime._apphosting_runtime___python__apiproxy.Wait}

  On Oct 2, 2:33 pm, Sylvain [EMAIL PROTECTED] wrote:

   How can we help for this ?

   Each day, I've severalTimeout.
   Do you still need more info about it (date,time, app_id,...) ?

   Regards

   On 30 sep, 18:57, johnP [EMAIL PROTECTED] wrote:

For a while today, I started seeing errors in custom form validation.
Requesting self.instance in ModelForm form resulted in ValueErrors.
Now, it seems to have recovered a bit...

On Sep 30, 9:43 am, Adam Loving [EMAIL PROTECTED] wrote:

 I am seeing the datastoretimeouterror intermittently several times a
 day (application = toyvirtualgifts) on a fairly simple putoperation.
 Could this be caused by the rest of the request takingtoolong(like
 if thetimeoutfor the entire request fires during the put)? It
 doesn't seem like that would be the case here, but that's the only
 explanation I can think of.

   File /base/python_lib/versions/1/google/appengine/ext/db/
 __init__.py, line 618, in put
 returndatastore.Put(self._entity)
   File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 162, in Put
 raise _ToDatastoreError(err)
   File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 1627, in _ToDatastoreError
 raise errors[err.application_error](err.error_detail)
Timeout

 Thanks,
 Adam

 On Sep 28, 3:19 am, Ronald [EMAIL PROTECTED] wrote:

  Hi,
  I don't know how much work has been done on this front, but my app 
  is
  still cripple by this issue.
  Maybe the amount oftimeouterrors in the log has decrease a little,
  but so has the amount of happy users of my site =(

  /ronald

  On Sep 19, 9:45 pm, Marzia Niccolai [EMAIL PROTECTED] wrote:

   Hi,

   Our engineers are looking in to your reports.

   If all of the people experiencing quota issues like this could 
   email me
   directly with your application id, as well as the approximate 
   times these
   errors were occurring, that would be helpful!

   Thanks,
   Marzia

   On Fri, Sep 19, 2008 at 11:17 AM, Michael Bailey [EMAIL 
   PROTECTED]wrote:

I got the error below on a app that has almost zero traffic:
see screenshot:
   http://imagebin.ca/img/3rlwzkBy.png

Traceback (most recent call last):
 File /base/python_lib/versions/1/google/appengine/ext/webapp/
__init__.py, line 496, in __call__
   handler.get(*groups)
  File /base/data/home/apps/***/1.4/search.py, line 25, 
in get
   for kw in kws:
  File /base/python_lib/versions/1/google/appengine/ext/db/
__init__.py, line 1257, in __iter__
   return self.run()
  File /base/python_lib/versions/1/google/appengine/ext/db/
__init__.py, line 1589, in run
   query_run = self._proto_query.Run(*self._args, **self._kwds)
 File /base/python_lib/versions/1/google/appengine/ext/gql/
__init__.py, line 572, in Run
   it = bind_results.Run()
 File /base/python_lib/versions/1/google/appengine/ext/gql/
__init__.py, line 1211, in Run
   results.append(bound_query.Run())
  File /base/python_lib/versions/1/google/appengine/api/
   datastore.py, line 860, in Run
   return self._Run()
  File /base/python_lib/versions/1/google/appengine/api/
   datastore.py, line 879, in _Run
   apiproxy_stub_map.MakeSyncCall('datastore_v3', 'RunQuery', 
pb,
result)
  File /base/python_lib/versions/1/google/appengine/api/

[google-appengine] Re: Surprisingly consistent timeouts on a simple query

[Fred]

I'm getting timeouts at the moment when fetching a single item from
the datastore:

Traceback (most recent call last):
  File /base/python_lib/versions/1/google/appengine/ext/webapp/
__init__.py, line 499, in __call__
handler.get(*groups)
  File /base/data/home/apps/mycycleroute/1.329543171657420466/
main.py, line 1583, in get
'greeting' : getUserGreeting(self),
  File /base/data/home/apps/mycycleroute/1.329543171657420466/
main.py, line 1773, in getUserGreeting
userPreferencesQuery = db.GqlQuery(SELECT * FROM UserPreferences
WHERE user = :1, user).fetch(1)
  File /base/python_lib/versions/1/google/appengine/ext/db/
__init__.py, line 1377, in fetch
raw = self._get_query().Get(limit, offset)
  File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 938, in Get
return self._Run(limit, offset)._Next(limit)
  File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 879, in _Run
apiproxy_stub_map.MakeSyncCall('datastore_v3', 'RunQuery', pb,
result)
  File /base/python_lib/versions/1/google/appengine/api/
apiproxy_stub_map.py, line 46, in MakeSyncCall
stub.MakeSyncCall(service, call, request, response)
  File /base/python_lib/versions/1/google/appengine/runtime/
apiproxy.py, line 246, in MakeSyncCall
rpc.CheckSuccess()
  File /base/python_lib/versions/1/google/appengine/runtime/
apiproxy.py, line 189, in CheckSuccess
raise self.exception
DeadlineExceededError: The API call datastore_v3.RunQuery() took too
long to respond and was cancelled.

:(

This is on the front page of my site!


On Nov 26, 4:27 am, Rob [EMAIL PROTECTED] wrote:
 Hi Marzia,

   Thanks again for the help.  I've done some careful tracing through
 my code to document exactly the number of datastore requests on a
 typical failure.  I believe they are very minimal but I'd be very
 interested to hear your advice.

  Is that the only datastore query in the request, or does the request also
  contain other datastore queries?  To clarify, it may be timing out here as a
  result of the total number of results fetched in the entire request.

 Here is a list of the datastore gets and puts that are called, in
 order for a typical failed execution.  There are no queries which
 return more than a single Entity and I tried to grab the database
 Entities directly by id when possible:

     # Grabs a single player out of the database by id
     self.db = datamodel.PlayerDb2.get(db.Key(id))

     # Checks the state of a single room (by id)
     self.db = id
     data = self.db.state

     # For the current player, checks the list of drawers
     # to verify the player is not next to draw
     if self.db.wannaDrawList and len(self.db.wannaDrawList):
       return self.db.wannaDrawList[0]

     # Updates the player's last connection time and
     # puts it back into the database
     self.db.lastPing = time
     self.db.put()

 I just did a scan of my recent errors in my logs and it appears that
 the total execution time for all of the above commands (plus the logic
 surrounding it) is about 80 to 130ms.  This part never seems to time
 out--but the problem query below seems to hang for several seconds
 before regularly timing out.  In addition, I used to have many more
 database accesses in this part of the code before and those have all
 been removed in order to try to fix the consistent time outs for the
 pruneInactive function.  So far I've never been able to move the
 timeout around or see it fail in another place.

 Here is the current version of the pruneInactive function where the
 timeout happens:

 -- Code Snippet --

         # first, grab all the players who's ping is older than 10
 seconds in our room
         # 8 maximum players per room
         query = datamodel.PlayerDb2.all()
         query.filter(room =, self.db.room)
         query.filter(lastPing , time.time() - TIMEOUT_TIME)

         logging.debug(Query is built\n)

         # Run the query for up to 5 players
         results = query.fetch(5)  # this is the timeout line
         logging.debug(Query is executed (limit 5)\n)

             ...

 -- End Code Snippet --

 Thanks again for your help with this confusing issue.  (At least it's
 confusing to me!)

 Sincerely,

 Rob
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Surprisingly consistent timeouts on a simple query

[Fred]

Seems to fluctuate between an 'internal error' and the deadline
exceeded error.

On Nov 26, 7:09 pm, Fred [EMAIL PROTECTED] wrote:
 I'm getting timeouts at the moment when fetching a single item from
 the datastore:

 Traceback (most recent call last):
   File /base/python_lib/versions/1/google/appengine/ext/webapp/
 __init__.py, line 499, in __call__
     handler.get(*groups)
   File /base/data/home/apps/mycycleroute/1.329543171657420466/
 main.py, line 1583, in get
     'greeting' : getUserGreeting(self),
   File /base/data/home/apps/mycycleroute/1.329543171657420466/
 main.py, line 1773, in getUserGreeting
     userPreferencesQuery = db.GqlQuery(SELECT * FROM UserPreferences
 WHERE user = :1, user).fetch(1)
   File /base/python_lib/versions/1/google/appengine/ext/db/
 __init__.py, line 1377, in fetch
     raw = self._get_query().Get(limit, offset)
   File /base/python_lib/versions/1/google/appengine/api/
 datastore.py, line 938, in Get
     return self._Run(limit, offset)._Next(limit)
   File /base/python_lib/versions/1/google/appengine/api/
 datastore.py, line 879, in _Run
     apiproxy_stub_map.MakeSyncCall('datastore_v3', 'RunQuery', pb,
 result)
   File /base/python_lib/versions/1/google/appengine/api/
 apiproxy_stub_map.py, line 46, in MakeSyncCall
     stub.MakeSyncCall(service, call, request, response)
   File /base/python_lib/versions/1/google/appengine/runtime/
 apiproxy.py, line 246, in MakeSyncCall
     rpc.CheckSuccess()
   File /base/python_lib/versions/1/google/appengine/runtime/
 apiproxy.py, line 189, in CheckSuccess
     raise self.exception
 DeadlineExceededError: The API call datastore_v3.RunQuery() took too
 long to respond and was cancelled.

 :(

 This is on the front page of my site!

 On Nov 26, 4:27 am, Rob [EMAIL PROTECTED] wrote:

  Hi Marzia,

    Thanks again for the help.  I've done some careful tracing through
  my code to document exactly the number of datastore requests on a
  typical failure.  I believe they are very minimal but I'd be very
  interested to hear your advice.

   Is that the only datastore query in the request, or does the request also
   contain other datastore queries?  To clarify, it may be timing out here 
   as a
   result of the total number of results fetched in the entire request.

  Here is a list of the datastore gets and puts that are called, in
  order for a typical failed execution.  There are no queries which
  return more than a single Entity and I tried to grab the database
  Entities directly by id when possible:

      # Grabs a single player out of the database by id
      self.db = datamodel.PlayerDb2.get(db.Key(id))

      # Checks the state of a single room (by id)
      self.db = id
      data = self.db.state

      # For the current player, checks the list of drawers
      # to verify the player is not next to draw
      if self.db.wannaDrawList and len(self.db.wannaDrawList):
        return self.db.wannaDrawList[0]

      # Updates the player's last connection time and
      # puts it back into the database
      self.db.lastPing = time
      self.db.put()

  I just did a scan of my recent errors in my logs and it appears that
  the total execution time for all of the above commands (plus the logic
  surrounding it) is about 80 to 130ms.  This part never seems to time
  out--but the problem query below seems to hang for several seconds
  before regularly timing out.  In addition, I used to have many more
  database accesses in this part of the code before and those have all
  been removed in order to try to fix the consistent time outs for the
  pruneInactive function.  So far I've never been able to move the
  timeout around or see it fail in another place.

  Here is the current version of the pruneInactive function where the
  timeout happens:

  -- Code Snippet --

          # first, grab all the players who's ping is older than 10
  seconds in our room
          # 8 maximum players per room
          query = datamodel.PlayerDb2.all()
          query.filter(room =, self.db.room)
          query.filter(lastPing , time.time() - TIMEOUT_TIME)

          logging.debug(Query is built\n)

          # Run the query for up to 5 players
          results = query.fetch(5)  # this is the timeout line
          logging.debug(Query is executed (limit 5)\n)

              ...

  -- End Code Snippet --

  Thanks again for your help with this confusing issue.  (At least it's
  confusing to me!)

  Sincerely,

  Rob


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] 'automated requests' error message - locked out from my site

[Fred]

Hi,

I have a google map application hosted on GAE which loads multiple
polylines from the datastore, using ajax. To avoid high CPU warnings,
I make incremental ajax calls - fetching batches of encoded polylines,
10 at a time.

You can see the site at http://www.bigbikemap.com/

However, today I have been locked out of my website with the following
google generated message:

We're sorry.. but your query looks similar to automated requests from
a computer virus or spyware application. To protect our users, we
can't process your request right now.

The block affects my whole network at IP level. I suspect the block is
a result of my incremental ajax facility.

Can anyone suggest a way around this? To load all segments in one ajax
request might generate high CPU warnings (which could potentially shut
my site down).

Thanks,

Fred.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Customize login page

[Fred]

As a further note - to customize the application name that appears on
the log-in screen, go to your application dashboard, click
'Application Settings' and then change the application title. Finally,
re-deploy your application and the new name will appears in the log-in
screen!

On Sep 25, 6:15 am, Calvin Spealman [EMAIL PROTECTED] wrote:
 Which is a google-run app, and the undoubtedly have permissions and
 knowledge of things we haven't or wont be given access to. They would need
 to find a secure way to let you customize this page, without allowing any
 attacks. It is not an extremely simple thing to fix.



 On Thu, Sep 25, 2008 at 12:32 AM, Sudhir [EMAIL PROTECTED] wrote:

  how about the moderator.appspot.com page? they have a customized
  login

  On Sep 24, 1:40 am, Sam G [EMAIL PROTECTED] wrote:
   It is possible to add a logo if you have tethered the domain to your
   Google Apps account.
   It uses the logo you upload for Google Apps.
   Beyond that, the only things that are dynamic on the login page is
   your domain name and your application name.

   I'm sure that Google wants to maintain a constant user experience
   throughout all of their products, so a user knows what a login screen
   looks like.

   Sam

   On Sep 23, 10:16 am, neich [EMAIL PROTECTED] wrote:

Hi,

I've been searching in the docs for any info on how to customize the
Google Account login page. The users.create_login_url() method doesn't
have any parameter (other than destination url). So, Am I right ? Is
it impossible to customize the login page ? It would be great to be
able to add some logo, or some personalized text.

Nacho

 --
 Read my blog! I depend on your acceptance of my opinion! I am 
 interesting!http://techblog.ironfroggy.com/
 Follow me if you're into that sort of thing:http://www.twitter.com/ironfroggy
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Unable to initialize images API - ImportError: No module named PIL

[Fred]

To get this working on Leopard, I ran the installers as suggested in
the GAE docs, but I kept on getting the 'ImportError: No module named
PIL' error message.

I tried the suggestions listed on
http://www.p16blog.com/p16/2008/05/appengine-installing-pil-on-os-x-1053.html
but this made no difference. So, in the end I simply copied the
PIL.pth file and PIL directory from /Library/Frameworks/
Python.framework/Versions/2.5/lib/python2.5/site-packages to /Library/
Python/2.5/site-packages and this appears to be working OK.

On Aug 31, 10:36 am, JLD000 [EMAIL PROTECTED] wrote:
 http://jetfar.com/cygwin-install-python-imaging-library/

 This worked for me.  (using Imaging-1.1.6)

 On Jul 7, 1:24 pm, RyanW [EMAIL PROTECTED] wrote:

  I goofed, you're right.  I thought it was the PIL package that was
  causing it to error.  It was actually because I had another
  application listening on port 8080.  Changed the port and it's running
  the service with the PIL error now.

  Still, has anyone gotten PIL to work under cygwin?


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Custom filters for django templates

[Fred]

Hello,

Just had this same problem myself, and found the solution -- you need
to do the following:

- create a templatetags directory INSIDE one of what Django calls your
'installed apps.'  Apps are modules directly under your project root,
for example, if your project has an app named home, your directory
structure should look like:

project/
   home/
  templatetags/

- Create an empty __init__.py in the templatetags directory.
- In the templatetags folder, create another file for your filter,
e.g., my_awesome_filter.py

You have, now:
project/
   home/
  templatetags/
  __init__.py
  my_awesome_filter.py

Now, at this point, tutorials leave the expectation that {% load
my_awesome_filter %} should just work, but there's one little piece of
magic they leave out: you have to add your app to Django's list of
INSTALLED_APPS so that the template tag loader can find it.  Edit
settings.py, add an entry to the INSTALLED_APPS tuple as follows
(excerpted from my live app, yours will differ):
INSTALLED_APPS = (
 'appengine_django',# Use the App Engine helper
 'django.contrib.auth', # Load up the Django auth system
(optional, but I like it)
 'home'  # -- Added
)

Worked like a charm for me.

best,
- Fred.


On Sep 3, 6:28 am, joune [EMAIL PROTECTED] wrote:
 Hi there,

 just meant to say that i have the very same problem..
 i followed django documentation to create a custom filter in
 my_utils.py that i put under a templatetags directory under my app and
 along with an empty __init__.py file
 the {%loadmy_utils.py %} tag throws the same
 TemplateSyntaxError: 'my_utils' is not a valid tag library

 i came to wonder if there was a limitation on custom filters in google
 appEngine.. (but it seems to work for g-man)
 i based my code on this 
 sample:http://google-app-engine-samples.googlecode.com/files/django_example_...
 and didn't change anything to settings.py (maybe i should have?)

 any input no this would be very helpful!

 thanks..

 On Aug 1, 12:11 pm, Arun Shanker Prasad [EMAIL PROTECTED]
 wrote:

  Hi,
  @g-man ,

  I am also not using the webapp with google app engine,
  but when I try toloadthe filter in the template an exception is
  raised

  'customfilters' is not a valid tag library: Could notloadtemplate
  library from django.templatetags.customfilters, No module named
  customfilters

  My Code:
  import django
  from django import template

  register = template.Library()

  @register.filter('modulo')
  def modulo(value, arg):
      if value % arg == 0:
          return True
      else:
          return False

  In Template
  {%loadcustomfilters %}

  Am i missing something? any help will be appricicated

  On Jul 22, 8:19 am, g-man [EMAIL PROTECTED] wrote:

   I, for one, appreciate your keeping all of us up-to-date on your
   explorations and discoveries, Larry!

   As a companion piece, here's my Djangotemplate tagthat creates a
   table of a fixed number of columns, for display of my growing tag
   collection, as in:

   {% table_row_cells 4 forloop.counter %}

   ... which simply divides the current count by the desired column
   cutoff (4 in my case), and if the remainder (modulo) is zero, starts a
   new row; comes in handy for those long lists of things.

   ps - I don't know why you need to bring webapp into the mix; I never
   need to get near it!

   -
   from django import template
   register = template.Library()

   @register.tag
   def table_row_cells(parser, token):
     try:
       tag_name, cells, loop_count = token.split_contents()
     except ValueError:
       msg = '%r tag requires a single number of row cells desired...' %
   token.contents[0]
       raise template.TemplateSyntaxError(msg)
     return TableRowCellsNode(int(cells))

   class TableRowCellsNode(template.Node):
     def __init__(self, cells):
       self.cells = cells
     def render(self, context):
       if context['forloop']['counter']%self.cells is 0:
         return '/tr'
       return ''
   -

   My learning app is here:

  http://archi-checker.appspot.com

   (I think the above implementation is hidden behind my 'admin' curtain,
   but it works, trust me!)

   On Jul 21, 2:33 pm, hairylarry [EMAIL PROTECTED] wrote:

Hi,

Here's the article.

   http://daily.profeth.de/2008/04/using-custom-django-template-helpers

I found it in another post.

Pay attention to the comment. The line.

webapp.template.register_template_library('common.templatefilters')

usually comes right after the import statements.

Also you must create an empty text file called __init__.py

This is shown in the illustration but not mentioned in the article.

I followed his instructions exactly without modifying any code. I got
an error until I created the __init__.py file. Then it worked.

Then I easily created 2 filters, one to replace underscores with
spaces and one to replace spaces with underscores. I prefer
underscores in urls to %20

[google-appengine] Google maps custom tile hosting on GAE

[Fred]

Hi,

I'm developing a google maps app, hosted on GAE, for which I've
developed a custom tile overlay, consisting of thousands of png
images, at about 4 - 8k each in size. I've tried uploading about 15mb
worth of tiles (just over 3,500 tiles), and have hit the 1000 file
limit. (Detailed in issue 161 - 
http://code.google.com/p/googleappengine/issues/detail?id=161
).

The suggestion from google is: We encourage developers to find
creative ways to reduce the number of files in an app - but for a
custom overlay in maps I don't think there is any other solution that
to have multiple tiles.

So - my question is whether anyone can suggest a way around this. I
think this is a showstopper for me otherwise.

Thanks!

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Google maps custom tile hosting on GAE

[Fred]

but An app can read files, but only files uploaded with the
application code from 
http://code.google.com/appengine/docs/whatisgoogleappengine.html

So should the zip file would be imported into the datastore?

On Sep 2, 4:37 pm, Brian Clapper [EMAIL PROTECTED] wrote:
 On 9/2/08 11:29 AM, Fred wrote:



  Hi,

  I'm developing a google maps app, hosted on GAE, for which I've
  developed a custom tile overlay, consisting of thousands of png
  images, at about 4 - 8k each in size. I've tried uploading about 15mb
  worth of tiles (just over 3,500 tiles), and have hit the 1000 file
  limit. (Detailed in issue 161 
  -http://code.google.com/p/googleappengine/issues/detail?id=161
  ).

  The suggestion from google is: We encourage developers to find
  creative ways to reduce the number of files in an app - but for a
  custom overlay in maps I don't think there is any other solution that
  to have multiple tiles.

  So - my question is whether anyone can suggest a way around this. I
  think this is a showstopper for me otherwise.

 You could store them in a zip file, then extract what you need from the zip
 file on the fly.
 --
 -Brian

 Brian Clapper,http://www.clapper.org/bmc/
 It is a miracle that curiosity survives formal education.
         -- Albert Einstein
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Google maps custom tile hosting on GAE

[Fred]

OK, thanks Brian. Final question - any pointers for reading from zip
files on the fly?

On Sep 2, 5:02 pm, Brian Clapper [EMAIL PROTECTED] wrote:
 On 9/2/08 11:45 AM, Fred wrote:

  but An app can read files, but only files uploaded with the
  application code 
  fromhttp://code.google.com/appengine/docs/whatisgoogleappengine.html

  So should the zip file would be imported into the datastore?

 Your first paragraph should give you the answer. If it contains static data,
 you can put it in your local development tree and upload it with appcfg. Or
 you can put it in the data store as the equivalent of a BLOB.Brian

 Brian Clapper,http://www.clapper.org/bmc/
 Boy, n.:
         A noise with dirt on it.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Google maps custom tile hosting on GAE

[Fred]

Tim:

 why aren't you storing your tiles as entities in the store rather than
 as files?

I was concerned about CPU quotas - each tile request would hit the
datastore.

On Sep 2, 5:03 pm, Tim Hoffman [EMAIL PROTECTED] wrote:
 why aren't you storing your tiles as entities in the store rather than
 as files?

 T

 On Sep 2, 11:29 pm, Fred [EMAIL PROTECTED] wrote:

  Hi,

  I'm developing a google maps app, hosted on GAE, for which I've
  developed a custom tile overlay, consisting of thousands of png
  images, at about 4 - 8k each in size. I've tried uploading about 15mb
  worth of tiles (just over 3,500 tiles), and have hit the 1000 file
  limit. (Detailed in issue 161 
  -http://code.google.com/p/googleappengine/issues/detail?id=161
  ).

  The suggestion from google is: We encourage developers to find
  creative ways to reduce the number of files in an app - but for a
  custom overlay in maps I don't think there is any other solution that
  to have multiple tiles.

  So - my question is whether anyone can suggest a way around this. I
  think this is a showstopper for me otherwise.

  Thanks!


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Google maps custom tile hosting on GAE

[Fred]

Hi Brian,

 personally, i would not try the datastore method first. I have not
 seen an issue yet with storing tiles as blobs

do you mean you *would* try the datastore method first? How many tiles
are you typically storing in the datastore?

On Sep 2, 5:42 pm, bFlood [EMAIL PROTECTED] wrote:
 1) put them in the datastore and memcache the mostly commonly used
 (like the tiles at the startup level)
 2) set client side cache headers for the tiles if they don't change
 often
 3) use alternative storage like Amazon S3

 personally, i would not try the datastore method first. I have not
 seen an issue yet with storing tiles as blobs

 cheers
 brian

 On Sep 2, 12:09 pm, Fred [EMAIL PROTECTED] wrote:

  Tim:

   why aren't you storing your tiles as entities in the store rather than
   as files?

  I was concerned about CPU quotas - each tile request would hit the
  datastore.

  On Sep 2, 5:03 pm, Tim Hoffman [EMAIL PROTECTED] wrote:

   why aren't you storing your tiles as entities in the store rather than
   as files?

   T

   On Sep 2, 11:29 pm, Fred [EMAIL PROTECTED] wrote:

Hi,

I'm developing a google maps app, hosted on GAE, for which I've
developed a custom tile overlay, consisting of thousands of png
images, at about 4 - 8k each in size. I've tried uploading about 15mb
worth of tiles (just over 3,500 tiles), and have hit the 1000 file
limit. (Detailed in issue 161 
-http://code.google.com/p/googleappengine/issues/detail?id=161
).

The suggestion from google is: We encourage developers to find
creative ways to reduce the number of files in an app - but for a
custom overlay in maps I don't think there is any other solution that
to have multiple tiles.

So - my question is whether anyone can suggest a way around this. I
think this is a showstopper for me otherwise.

Thanks!- Hide quoted text -

  - Show quoted text -


--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---

[google-appengine] Re: Google maps custom tile hosting on GAE

[Fred]

Thanks all, the verdict then is to try the datastore first. The tiles
will be refreshed on a regular basis (probably every 24 hours) using
an application outside the GAE, so the reliability and speed of the
bulk uploader will be an important factor. The last time I tried the
bulk uploader I found it would fail occassionally - perhaps it's more
reliable now.

Otherwise, I'll give Amazon's S3 a try.

On Sep 2, 6:16 pm, Wooble [EMAIL PROTECTED] wrote:
 On Sep 2, 12:09 pm, Fred [EMAIL PROTECTED] wrote:

  Tim:

   why aren't you storing your tiles as entities in the store rather than
   as files?

  I was concerned about CPU quotas - each tile request would hit the
  datastore.

 If the information in Issue 161 is correct and the reason for the 1000
 file limit is that the files themselves are being stored in the
 datastore, I imagine you'll get a similar CPU hit whether you're
 accessing files or pulling them out of the datastore directly, and
 I'd also imagine that the CPU time involved in loading a zip file of
 thousands of tiles, unarchiving it, and then finding the tile(s) you
 want would be even worse.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---