[appengine-java] Application works fine on localhost but not when deployed
Hi, I keep having the same problem of not being able to get my application to work when deployed on GAE. It works fine locally on my laptop when i run it with the google app engine server. Online, I am able to view a 'login.jsp' page, but when I try to login as an authenticated user, it will return a 500 Server Error. However, if I try to login using a wrong user/password, it would return fine and say that there is an error. I suspect that the problem lies with my homepage.jsp, which an AuthenticateServlet of mine sendRedirects to. Really hope someone can help me solve it. Has been bothering me very much. Appreciate any help! The error logs are as such: W 10-11 11:53PM 30.903 /homepage.jsp java.lang.NullPointerException at com.google.appengine.runtime.Request.process-687049c18cdffe79(Request.java) at org.apache.jsp.homepage_jsp._jspService(homepage_jsp.java:66) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:94) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166) at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:97) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418) at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:238) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923) at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:135) at com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java:251) at com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:6784) at com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:6782) at com.google.net.rpc.impl.BlockingApplicationHandler.handleRequest(BlockingApplicationHandler.java:24) at com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java:398) at com.google.net.rpc.impl.Server$2.run(Server.java:852) at com.google.tracing.LocalTraceSpanRunnable.run(LocalTraceSpanRunnable.java:56) at com.google.tracing.LocalTraceSpanBuilder.internalContinueSpan(LocalTraceSpanBuilder.java:576) at com.google.net.rpc.impl.Server.startRpc(Server.java:807) at com.google.net.rpc.impl.Server.processRequest(Server.java:369) at com.google.net.rpc.impl.ServerConnection.messageReceived(ServerConnection.java:442) at com.google.net.rpc.impl.RpcConnection.parseMessages(RpcConnection.java:319) at com.google.net.rpc.impl.RpcConnection.dataReceived(RpcConnection.java:290) at com.google.net.async.Connection.handleReadEvent(Connection.java:474) at com.google.net.async.EventDispatcher.processNetworkEvents(EventDispatcher.java:831) at com.google.net.async.EventDispatcher.internalLoop(EventDispatcher.java:207) at com.google.net.async.EventDispatcher.loop(EventDispatcher.java:103) at com.google.net.rpc.RpcService.runUntilServerShutdown(RpcService.java:251) at com.google.apphosting.runtime.JavaRuntime$RpcRunnable.run(JavaRuntime.java:418) at java.lang.Thread.run(Thread.java:636) C 10-11 11:53PM 30.907 Uncaught exception from servlet java.lang.NullPointerException at com.google.appengine.runtime.Request.process-687049c18cdffe79(Request.java) at org.apache.jsp.homepage_jsp._jspService(homepage_jsp.java:66) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:94) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166) at
[appengine-java] Re: Application works fine on localhost but not when deployed
Just to add on, my AuthenticateServlet has this error: /AuthenticateServlet java.lang.RuntimeException: java.io.NotSerializableException: BiddingSystem.model.Student at com.google.apphosting.runtime.jetty.SessionManager.serialize(SessionManager.java:387) at com.google.apphosting.runtime.jetty.SessionManager.createEntityForSession(SessionManager.java:364) at com.google.apphosting.runtime.jetty.SessionManager$AppEngineSession.save(SessionManager.java:164) at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:41) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418) at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:238) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923) at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:135) at com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java:251) at com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:6784) at com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:6782) at com.google.net.rpc.impl.BlockingApplicationHandler.handleRequest(BlockingApplicationHandler.java:24) at com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java:398) at com.google.net.rpc.impl.Server$2.run(Server.java:852) at com.google.tracing.LocalTraceSpanRunnable.run(LocalTraceSpanRunnable.java:56) at com.google.tracing.LocalTraceSpanBuilder.internalContinueSpan(LocalTraceSpanBuilder.java:576) at com.google.net.rpc.impl.Server.startRpc(Server.java:807) at com.google.net.rpc.impl.Server.processRequest(Server.java:369) at com.google.net.rpc.impl.ServerConnection.messageReceived(ServerConnection.java:442) at com.google.net.rpc.impl.RpcConnection.parseMessages(RpcConnection.java:319) at com.google.net.rpc.impl.RpcConnection.dataReceived(RpcConnection.java:290) at com.google.net.async.Connection.handleReadEvent(Connection.java:474) at com.google.net.async.EventDispatcher.processNetworkEvents(EventDispatcher.java:831) at com.google.net.async.EventDispatcher.internalLoop(EventDispatcher.java:207) at com.google.net.async.EventDispatcher.loop(EventDispatcher.java:103) at com.google.net.rpc.RpcService.runUntilServerShutdown(RpcService.java:251) at com.google.apphosting.runtime.JavaRuntime$RpcRunnable.run(JavaRuntime.java:418) at java.lang.Thread.run(Thread.java:636) Caused by: java.io.NotSerializableException: BiddingSystem.model.Student at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1173) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:343) at java.util.HashMap.writeObject(HashMap.java:1018) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:962) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1478) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1409) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1167) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:343) at com.google.apphosting.runtime.jetty.SessionManager.serialize(SessionManager.java:384) ... 39 more -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Application works fine on localhost but not when deployed
Looks like your Student class is not serializable. On Tue, Oct 12, 2010 at 09:35, swin swin swin...@gmail.com wrote: Just to add on, my AuthenticateServlet has this error: /AuthenticateServlet java.lang.RuntimeException: java.io.NotSerializableException: BiddingSystem.model.Student at com.google.apphosting.runtime.jetty.SessionManager.serialize(SessionManager.java:387) at com.google.apphosting.runtime.jetty.SessionManager.createEntityForSession(SessionManager.java:364) at com.google.apphosting.runtime.jetty.SessionManager$AppEngineSession.save(SessionManager.java:164) at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:41) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418) at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:238) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923) at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:135) at com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java:251) at com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:6784) at com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingRequest(RuntimePb.java:6782) at com.google.net.rpc.impl.BlockingApplicationHandler.handleRequest(BlockingApplicationHandler.java:24) at com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java:398) at com.google.net.rpc.impl.Server$2.run(Server.java:852) at com.google.tracing.LocalTraceSpanRunnable.run(LocalTraceSpanRunnable.java:56) at com.google.tracing.LocalTraceSpanBuilder.internalContinueSpan(LocalTraceSpanBuilder.java:576) at com.google.net.rpc.impl.Server.startRpc(Server.java:807) at com.google.net.rpc.impl.Server.processRequest(Server.java:369) at com.google.net.rpc.impl.ServerConnection.messageReceived(ServerConnection.java:442) at com.google.net.rpc.impl.RpcConnection.parseMessages(RpcConnection.java:319) at com.google.net.rpc.impl.RpcConnection.dataReceived(RpcConnection.java:290) at com.google.net.async.Connection.handleReadEvent(Connection.java:474) at com.google.net.async.EventDispatcher.processNetworkEvents(EventDispatcher.java:831) at com.google.net.async.EventDispatcher.internalLoop(EventDispatcher.java:207) at com.google.net.async.EventDispatcher.loop(EventDispatcher.java:103) at com.google.net.rpc.RpcService.runUntilServerShutdown(RpcService.java:251) at com.google.apphosting.runtime.JavaRuntime$RpcRunnable.run(JavaRuntime.java:418) at java.lang.Thread.run(Thread.java:636) Caused by: java.io.NotSerializableException: BiddingSystem.model.Student at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1173) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:343) at java.util.HashMap.writeObject(HashMap.java:1018) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:962) at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1478) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1409) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1167) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:343) at
Re: [appengine-java] Is MemcacheService obtained from getMemcacheService(namespace) thread safe?
Great, thank you for clarifying this. Perhaps It would be helpful to mention this in the javadoc ? Or maybe introduce some kind of @ThreadSafe annotation with RetentionPolicy.SOURCE ? Thanks, Maxim. On Mon, Oct 11, 2010 at 9:10 PM, Ikai Lan (Google) ikai.l+gro...@google.comikai.l%2bgro...@google.com wrote: Yes. It's just a client. If you're really worried about this, you don't really save much by doing this, however, since the only real cost is object allocation. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Mon, Oct 11, 2010 at 5:01 AM, Maxim Veksler ma...@vekslers.org wrote: Hi, Well the title pretty much says it all. Is the memcache low level interface of AppEngine Java SDK thread safe? I would like to hold a reference to it in a static class level reference field. Thanks, Maxim. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: OpenID Development server
AFAIK you have two different behaviors on purpose. The idea is to allow anybody in your local environment, but require a proper OpenID account for your site. This means that you will have to test the OpenID behavior online or invoking directly _ah/login_required On Oct 10, 4:16 pm, Fabrizio Accatino fht...@gmail.com wrote: I have no problem opening an issue. But actually I don't understand if I have a problem/wrong_configuration or App Engine has a bug. I'm waiting for a reply from Google guys... :) fabrizio 2010/10/9 ArtemGr artem...@gmail.com OpenID support is still experimental, there are several open issues concerning it. If you feel you encountered a new problem, maybe you should file an issue! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: GAE and Log4j - A little solution to make things easier.
JUL has been reported as 60 times slower than other implementations because of the way they log: http://www.infoq.com/news/2007/08/logback Just configuring properly log4j/logback to log to System.err or System.out should be enough to see messages properly in the AppEngine console. In logback, it's something like this: appender name=STDOUT class=ch.qos.logback.core.ConsoleAppender encoder pattern%-5level %logger{0} - %msg%n /pattern /encoder filter class=ch.qos.logback.classic.filter.LevelFilter levelERROR/level onMatchDENY/onMatch onMismatchACCEPT/onMismatch /filter /appender appender name=STDERR class=ch.qos.logback.core.ConsoleAppender targetSystem.err/target encoder pattern%-5level %logger{0} - %msg%n /pattern /encoder filter class=ch.qos.logback.classic.filter.LevelFilter levelERROR/level onMatchACCEPT/onMatch onMismatchDENY/onMismatch /filter /appender root level=DEBUG appender-ref ref=STDOUT / appender-ref ref=STDERR / /root Then, you can use slf4j-jcl to redirect also your commons-logging messages. On Oct 3, 6:14 pm, Eurig Jones eurigjo...@gmail.com wrote: Hey, I've written a Log4j appender which passes on log4j severity levels to GAE's JUL framework so you can view your log4j events (or the events your chosen framework provides) if you need to. http://androidisland.blogspot.com/2010/10/gae-and-log4j-getting-them-... This is a quick implementation of this Appender for me and so far it works, but I'd love to hear some feedback about what it doesn't do (or doesn't do properly) as I've not used it in anger. Regards, Eurig Jones -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Configuring AName
Hi everyone, I want allow the user of my site to have user pages with urls like: http://username.mysite.com To do this on GoDaddy I need to configure the Aname to star (*), but I can do this only if I have an IP address. I tried to ping ghs.l.google.com and I retrieve the ip address: 74.125.43.121. I tried to map star(*) with this ip address, but I receive a page with the following error: The requested URL / was not found on this server. somebody can help me? best regards, Davide -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Configuring AName
imho you can associate domains with your app only through the appengin admin interface. from your email i guess you try to make a subdomain per user registered with your site. that would be great, but i don't think it will work. On Tue, Oct 12, 2010 at 1:12 PM, Davide Cerbo davidece...@gmail.com wrote: Hi everyone, I want allow the user of my site to have user pages with urls like: http://username.mysite.com To do this on GoDaddy I need to configure the Aname to star (*), but I can do this only if I have an IP address. I tried to ping ghs.l.google.com and I retrieve the ip address: 74.125.43.121. I tried to map star(*) with this ip address, but I receive a page with the following error: The requested URL / was not found on this server. somebody can help me? best regards, Davide -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- jgabios http://bash.editia.info -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: ClassCastException: String to Collection
I may have stumbled across the answer by commenting out fields one at a time. Vendor had an embedded class PremiumInfo that referred to a non-embedded class Review. When I either a) removed the Review field from PremiumInfo or b) made PremiumInfo not embedded, the problem went away. Is having an embedded class owning a collection of non-embedded classes a no-no? -- decitrig -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: OneToOne relationship with java.lang.Long ids
The ID for the owned child User instances has to be a Key or String type because the it contains both the reference to the parent(s) and the unique id of the object. A Long alone will not contain enough information to navigate the object model to it. That statement doesn't make sense to me. The relationship from Event - User is uni-directional so it isn't necessary to have a link from User back to Event. You can create/encode/decode Keys easily using the KeyFactory. I want to create a portable JPA domain model and that is why I coded my model as such. I would like the option to be able to move this model from GAE over to MySQL... so I don't really like the requirement to use a String key either. Perhaps this information need to be added to the unsupported features section? (Long ids and relationships don't play nicely.) Thanks, Rick On Oct 11, 10:33 pm, andy stevko andy.ste...@gmail.com wrote: The ID for the owned child User instances has to be a Key or String type because the it contains both the reference to the parent(s) and the unique id of the object. A Long alone will not contain enough information to navigate the object model to it. I usually stick with String ids for all my types. You can create/encode/decode Keys easily using the KeyFactory. On Mon, Oct 11, 2010 at 8:03 PM, Rick Curtis curti...@gmail.com wrote: Hello all! I am new to GAE and have a pretty simple domain model that I'm trying to persist. I am creating an Event(with a generated ID) and a User(with a pre-defined ID) which is hosting the event. This is a one to one relationship owned by the Event(@See below for code snippets). Upon commiting the transaction, I'm getting an exception which states Error in meta-data for User._id: Cannot have a java.lang.Long primary key and be a child object (owning field is Event._host). I did some searching and came across this[1] post which suggested that I need to change my key from a java.lang.Long to a com.google.appengine.api.datastore.Key. Since The App Engine Java SDK includes an implementation of JPA 1.0 for the App Engine datastore[2]... I don't really need to use these internal GAE classes to describe this mapping do I? Maybe I'm being a bit naive here, but I have a hard time believing that this requirement/limitation is due to the underlying database not being a relational database. Hopefully someone can shed some light on my findings. Thanks, Rick @Entity public class Event { @Id @GeneratedValue(strategy=GenerationType.IDENTITY) Long _id; @OneToOne User _host; ... } @Entity public class User { @Id Long _id; ... } [1]http://code.google.com/p/datanucleus-appengine/issues/detail?id=26 [2]http://code.google.com/appengine/docs/java/datastore/usingjpa.html -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Application works fine on localhost but not when deployed
Thanks Guillaume. I did a search for what you said and found out I had to implement serializable for anything that i wanted to use with session. It worked. :) -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Implementing inequalities with multiple properties.
I have a requirement to make searches with inequalities with multiple properties. This is not possible, as described in http://code.google.com/appengine/docs/java/datastore/queriesandindexes.html#Restrictions_on_Queries But I have to do it anyways. So, is there any hint on how to deal with this? I think many people has faced this problem. Thank you. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Implementing inequalities with multiple properties.
My workaround (I'm doing the testing so far) : - create a extra property that merges the content of the N properties you wish to filter on - query on that single extrat property with inequality filters That is what geohash does when merging latitude AND longitude into one single property : http://hitching.net/2009/11/10/scalable-fast-accurate-geo-apps-using-google -app-engine-geohash-faultline-correction/ On 12/10/10 17:51, roberto_sc roberto.cal...@gmail.com wrote: I have a requirement to make searches with inequalities with multiple properties. This is not possible, as described in http://code.google.com/appengine/docs/java/datastore/queriesandindexes.htm l#Restrictions_on_Queries But I have to do it anyways. So, is there any hint on how to deal with this? I think many people has faced this problem. Thank you. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Problem with large entities? Store in one or two separate entities
If it's a few small fields, I don't think the performance difference is worth the extra complexity. The size of the entities does have an impact on the overall performance of the fetch, but there difference is usually more material if each entity stores a few hundred kilobytes of data. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Mon, Oct 11, 2010 at 2:14 PM, terran terran.le...@gmail.com wrote: General question - If I have very large entities with information that varies in usage, should I split the information I use less into another entity? I'm creating fairly large entities (column wise) and was wondering what the proper strategy was to approach storing the properties. An example is, I'm creating profiles for people. This includes basic information like name, email, url, but also information like bios, interests, affiliations. 90%+ of the time I will not need the bios/interests/affiliations, just the name/email/url. Should I be splitting the larger blocks of information into another entity, and rather than just have a Person entity, instead have a Person entity AND a PersonDetails entity? I arrived at this concern when I was doing queries, and I fear that with loading larger entities I'm consuming too many resources getting each property. Is this something I should be concerned about? Or just query large entities and it won't matter. Thanks! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Cant compile javax.crypto
Doh, your quite right Didier, it was because I had my class in shared. Thanks, Stuart. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Is there a way to turn off Task Queue timeouts?
I really want to be able to debug step through a Task Queue initiated task and have it not time out. http method POST against URL http://0.0.0.0:8080/task/simple timed out. Also - have you noticed that the dev server does retry tasks that throw errors? This is contrary to the docs - the development server does not retry tasks. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Google App Engine MUD
What you want is An Easy Version MUD or Complete Version MUD? If you want to hardcode all the thing, you only have to know what is AJAX. It's easy. But if you want to implement a real MUD, Complete Version MUD, you have to know a lot of things. ex: how to write a compiler... Because MUD is including MUD OS and MUD Lib. You have to make a wizard to make Object on your platform, by coding !! For example a wizard can create Car to be Driven or Horse to be Mounted.. because your MUD OS support User, Entity and Action. Car is Entity. Drive is Action. And User can Action the Entity So User can Drive Car, and Mount the Horse. But what kind of the Car? Super User in the MUD (King or Admin) can describe what's the car, Toyota, BMW... But you can hardcode all of that. FYI: http://www.mudos.org/ On 10月5日, 下午12時20分, MLS mlsjunkm...@gmail.com wrote: Hi Fattom, Thanks for your response. I don't think most browsers support Websockets quite yet, and I don't want to tie people to a specific one. What is the most challenging part of writing a MUD for http aside from real time delivery, which I'm hoping Ajax push solves? Aside from that I believe I can write the methods required to produce the output that is expected from the various commands in the game I plan to implement, and can manage player attributes with player objects and back them with a nice database. I could certainly be missing something crucial though, so please let me know. To me it sounds much more difficult to write a client and server using telnet. Thanks, MLS On Oct 3, 10:55 pm, 番頭Fattom fattom.rid...@gmail.com wrote: If the client access server with HTTP, GAE is definitely OK. But in my experience, Telnet is a better protocol for Mud/BBS. And I think GAE do not support socket function. If GAE support WebSocket, it will be a good way to approach that... (but I am not sure about this feature.) The most important thing is that it's very difficult to write a MUD including MUD OS/MUD Lib/MUD compiler... It's not a easy job. On 10月2日, 上午11時02分, MLS mlsjunkm...@gmail.com wrote: My friend and I are looking to rewrite an old BBS MUD. The game is a few thousand rooms and purely text based with commands issued by individual players and the game responding and updating players stats based on the results of their commands. I do not have a programming background, and I'm wondering about the best way to approach writing it for GAE. Attempting to translate the game to objects, I have the following layout: World object holds all of the zone objects, zone objects hold rooms, room objects hold players, player objects hold inventory objects, spellbook objects, and attributes. Then write appropriate methods for manipulating the objects appropriately and returning the results. I have a few questions though: 1. Is this a technically correct way of approaching the problem? If not, what would be better? (keep in mind, I am not a professional programmer by any means. I can find my way if I have something already written to make changes, but I've never written anything from scratch before). Would I populate all these objects, then write world to the db and retrieve it for each player action? I have no idea how servlets work to store state, or really how they work in general really. Basically, how would I be storing and retrieving stored player statistics which are constantly changing many times a second as players play the game. 2. A key issue is the ability to push updates to players in real time, and track individuals who are connected. For example, I would like the ability to be able to call something like room[someroomnum].sendtoall(Some message here); to push a message to all players in that room, or player[someplayernum].send(some message here); to send replies to players directly. I understand this is a complex issue with the stateless http protocol, but I have read some about the upcoming channel API which sounds like it may offer this ability but I don't know enough to be certain. Overall I'm looking to avoid devoting countless hours to this project if it is going to be beyond my abilities to complete successfully. Thanks very much for your time and consideration, MLS -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Recommendations for a Greenfield project
Objectify is cool, but for real performance you should use GWT + slim3. I don't know about Roo, but the RequestFactory looks really good. http://code.google.com/intl/es/webtoolkit/doc/trunk/DevGuideRequestFactory.html On Tue, Oct 12, 2010 at 2:53 PM, Vikas Hazrati vhazr...@gmail.com wrote: We recently helped an organization move their pretty successful standalone enterprise timesheet product to the app engine and offer it as a SaaS application. The project used Spring, Wicket, Hibernate(which we converted to JPA). However we had to face many obstacles on the way. You can read about ALL our logs here http://thoughts.inphina.com/tag/GAE/ The problems that I would like to highlight with Spring was the slow startup time and the time killing our application on each cold start because the beans were brought back to life again. Several times we breached the 30 s barrier. With wicket we faced serialization issues and with JPA we have a lot of blogs on how we made our way around them. You can access the application here http://www.bookmyhours.com/demo.jsp OK, the main question - Now we are getting into a greenfield project, we do not have any baggage of frameworks used in the past. The project is a B2C portal, mostly reads, form submissions and results, payment gateway etc. What would you recommend as the application stack for such a development? We are contemplating between GWT + Spring Roo + Objectify GWT + slim3 + Objectify / Twig GWT + Guice + Objectify / Twig Non GWT + Any of above options Any other options? What would you propose for new development and what should be our deciding factors? Regards | Vikas -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Guit: Elegant, beautiful, modular and *production ready* gwt applications. http://code.google.com/p/guit/ -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: OneToOne relationship with java.lang.Long ids
Ah, if you want to do this with a relational model rather than a networked model, you would define your objects such that each is a root entity (and thereby ids can be type Long ). This is analogous to holding a foreign key rather than embedding the owned entity. @Entity public class Event { @Id @GeneratedValue(strategy=GenerationType.IDENTITY) Long _id; // store the id of the User rather than the User object itself. Long _host_id; ... } The hardest thing with proofing a model decision like this is scoping the transactions based on your use cases. Coordinating multiple separate transactions on separate entities makes for interesting roll-forward / rollback coding. re: design portability I believe DataNucleus supports MySQL. If you are going to use a ORM, you just have to make sure it is portable -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[google-appengine] Re: Site not working after update
Hi For this error BadValueError: Property starttime is required it looks like you have changed your model to make starttime required and your trying to retrieve and entity that doesn't have a value for starttime. You will need to fix your entities if you want that property required. The import error will be because you have a broken instance (failed on startup with DeadlineExceeded and can't run now due to the failed imports) or you haven't got a signals.py somewhere ;-) T On Oct 12, 9:21 am, Andrew andriu...@gmail.com wrote: Hello, could someone from google have a look at site: bidshapeauction Site stopped working after update, the same code works fine on other app ids. We are getting different errors: type 'exceptions.ImportError': cannot import name signals or BadValueError: Property starttime is required Site is really slow to load. On google app engine I see anomaly with Memcache, anyone having issues? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Checking if new version is ready to serve only after 8 seconds
Hi I deploy over 3g, all the time. (Its the only type of link I can get where I live). And I can test to the fact , the polling/backoff waiting for the deploy are not really relevant in the scheme of things. It takes much longer to actually send everything ;-). T On Oct 9, 1:54 am, Ikai Lan (Google) ikai.l+gro...@google.com wrote: That's the thing about exponential backoffs: it grows so quickly. It goes from 1-2-4-8-16-32 seconds. That's only 6 HTTP requests, and if you set it to 8 seconds, it'd be only 3 HTTP requests worth of savings. I certainly hope you're not deploying via 3G, as the overhead of 3-6 HTTP requests is tiny compared to deploying an application. That being said, you'll probably want to file a feature request here:http://code.google.com/p/googleappengine/issues/list. It seems like a reasonable enough request to make this a parameter on deploy. It's not a great answer: but if you want to do this with the Python SDK, there's always monkeypatching ... -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger:http://googleappengine.blogspot.com Reddit:http://www.reddit.com/r/appengine Twitter:http://twitter.com/app_engine On Fri, Oct 8, 2010 at 10:27 AM, Peter Ondruska peter.ondru...@gmail.comwrote: Whenever I deploy new version/update to my GAE applications (be it Java or Python--it does not matter) it never happened to me that new version was ready earlier than 8 seconds (mostly 32, 64 or even later). I do not believe it is just me. Can I somehow configure on my side when to check for readiness earliest? And I plan to use 3G connection in near future and those extra requests in first 1+2+4+8 seconds are just waste of bandwidth and data... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: How about JavaScript v8 as the next language for Google App Engine?
Demis, It's the opposite: Py's data access is slower than Java's, at least according to Darshan's experiments: http://groups.google.com/group/google-appengine/t/7a8b716ecc8d5bb5 see also the reply in stackoverflow: http://stackoverflow.com/questions/3886341/is-appengine-python-datastore-query-much-3x-slower-than-java (btw, the api ms are exactly the same, suggesting he's doing the same in both implementations) In general, it makes imho more sense to use server side tools that save resources (which also run faster, by coincidence), so I'd rather like to see a C++ sandbox on GAE than Java script. Furthermore, if you like to have a homogeneous environment on client and server, have a look at GWT. Ok, its Java instead of js, but for us it works very well. Maybe J's API is very extensive (I'd rather not say bloated), but mostly not in the sense of more overhead, but more choice on the menu. Nobody urges you to use all of it, but there are some nice gems around in the Java community. Cheers, R. On 11 Okt., 23:24, Demis Bellot demis.bel...@gmail.com wrote: Cool, though Ringojs is no Expressjs and ultimately if I'm not mistaken all calls will eventually goes through Java's Sync IO and standardized API's no? I'm not sure why Java data access is slower than Pythons (http://gaejava.appspot.com/) but I'm guessing this probably has to do needing to provide adapter implementations to Java's standard interfaces? I never liked adding layers of abstraction on top of already bloated API's and this would be a chance for a fresh start with no backward compatibility requirements and existing APIs to adhere to. I'm sure all the BigData access is async underneath no? And we know Memcached screams in Async/UDP mode! (http://www.facebook.com/note.php?note_id=39391378919) - Demis On Mon, Oct 11, 2010 at 10:00 PM, gabriel munteanu jajali...@gmail.comwrote: guys, thanks to java gae, we already have javascript on appspot checkout ringojs , a ~commonjs framework. or you can go to a much lower level through rhino itself. cheers On Mon, Oct 11, 2010 at 11:38 PM, nickmilon nickmi...@gmail.com wrote: I think a new language should be out of the question, well until current platform is stabilized. But then again who am I to tell big G what to do and what not. ;-) On Oct 11, 10:06 pm, Ikai Lan (Google) ikai.l+gro...@google.comikai.l%2bgro...@google.com wrote: Believe it or not, we've talked about this. There's a lot of interest in V8 and JavaScript. From a technical standpoint, there are still advances that need to be made in V8 (garbage collection comes to mind) - see for yourself by searching for articles about Node.js and GC. Ultimately, it comes down to resources, which is probably why we aren't working on this right away. It makes more sense for us to harden our Java and Python runtimes, allow more classes into the whitelist, and look into ways we can support versions of Python that are higher than 2.5. As much traction as Node.js has in the blogosphere and Hacker News, it's still hard to make a business case for a platform that has, at best, poor tooling and a small (albeit enthusiastic) core community. You have a gripe with Python being slow - I'm surprised this is an issue, as most of your time is spent blocking on RPC calls, not interpreter gotchas. It doesn't matter how fast your Ferrari is if there are stop signs on every block. I'd look into Heroku's experimental node.js support: http://blog.heroku.com/archives/2010/4/28/node_js_support_experimental/ Try building an application using node.js as your full application stack. I'd love to see more article content about the challenges/benefits of doing this. There isn't enough, as far as I am concerned. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger:http://googleappengine.blogspot.com Reddit:http://www.reddit.com/r/appengine Twitter:http://twitter.com/app_engine On Sun, Oct 10, 2010 at 8:46 PM, John McLaughlin johnmclaugh...@massanimation.com wrote: Thanks for the info Demis, I alway like listening to Douglas Crockford. I also checked out http://developer.yahoo.com/yui/theater/video.php?v=glass-node. Very interesting stuff, I can definitely see using it in the future. In particular using JavaScript as the templating engine seems really powerful. So you've changed my mind. However since I'm still worried about the plethora part of ...plethora of well-tested JavaScript frameworks and libraries available..., I'll say that if node.js, YUI3, and JSLint are defined as best practices -- I'm on board -- server-side JavaScript all the way. - John On Oct 10, 4:21 pm, Demis Bellot demis.bel...@gmail.com wrote: Unless I'm mistaken about recent advancements, Python belongs in the same performance category as Ruby, Perl and PHP i.e.
Re: [google-appengine] Re: Site not working after update
Thanks Tim, you are right, i made changes by wrapping model with timezones which is returning starttime as null. Its easy to fix, but deployment doesnt work anymore! Its stuck for waiting when new deployed site will become available.. On 12 Oct 2010 07:55, Tim Hoffman zutes...@gmail.com wrote: Hi For this error BadValueError: Property starttime is required it looks like you have changed your model to make starttime required and your trying to retrieve and entity that doesn't have a value for starttime. You will need to fix your entities if you want that property required. The import error will be because you have a broken instance (failed on startup with DeadlineExceeded and can't run now due to the failed imports) or you haven't got a signals.py somewhere ;-) T On Oct 12, 9:21 am, Andrew andriu...@gmail.com wrote: Hello, could someone from google have a look at site: bidshapeauction Site stopped working after update, the same code works fine on other app ids. We are getting different errors: type 'exceptions.ImportError': cannot import name signals or BadValueError: Property starttime is required Site is really slow to load. On google app engine I see anomaly with Memcache, anyone having issues? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Oauth in Appengine
Hi, I couldnt get my way out of this problem. gdocs = gdata.service.GDataService(source=SETTINGS['APP_NAME']) gdocs.SetOAuthInputParameters(SETTINGS['SIG_METHOD'], SETTINGS['CONSUMER_KEY'], consumer_secret = SETTINGS['CONSUMER_SECRET']) gdata.alt.appengine.run_on_appengine(gdocs) def get_oauth_token(request): sess = Session() oauth_token = gdata.auth.OAuthTokenFromUrl(request.build_absolute_uri()) if oauth_token: oauth_token.secret = sess['oauth_token_secret'] oauth_token.oauth_input_params = gdocs.GetOAuthInputParameters() gdocs.SetOAuthToken(oauth_token) oauth_verifier = request.GET.get('oauth_verifier', '') access_token = gdocs.UpgradeToOAuthAccessToken(oauth_verifier=oauth_verifier) if access_token and users.get_current_user(): gdocs.token_store.add_token(access_token) elif access_token: gdocs.current_token = access_token gdocs.SetOAuthToken(access_token) return HttpResponseRedirect('/listdocs') else: if users.get_current_user(): req_token = gdocs.FetchOAuthRequestToken( scopes=SETTINGS['SCOPES'],oauth_callback=request.build_absolute_uri() ) sess['oauth_token_secret'] = req_token.secret approval_page_url = gdocs.GenerateOAuthAuthorizationURL() return HttpResponseRedirect(approval_page_url); It always fail when try to get Authorization URL. And got this error : Exception Type: NonOAuthToken Exception Value: Exception Location: C:\Users\hudarsono\Sites\appengine\shinemob\gdata \service.py in GenerateOAuthAuthorizationURL, line 4 But it works if we i change the client from gdocs = gdata.service.GDataService(source=SETTINGS['APP_NAME']) to gdocs = gdata.docs.service.DocsService(source=SETTINGS['APP_NAME']) May I know what is the difference? Since i know that DocsService extend GDataService. I use GDataService because want to make it more general, since I use several scopes, not only docs. Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Checking if new version is ready to serve only after 8 seconds
Thanks guys.. On Oct 12, 9:00 am, Tim Hoffman zutes...@gmail.com wrote: Hi I deploy over 3g, all the time. (Its the only type of link I can get where I live). And I can test to the fact , the polling/backoff waiting for the deploy are not really relevant in the scheme of things. It takes much longer to actually send everything ;-). T On Oct 9, 1:54 am, Ikai Lan (Google) ikai.l+gro...@google.com wrote: That's the thing about exponential backoffs: it grows so quickly. It goes from 1-2-4-8-16-32 seconds. That's only 6 HTTP requests, and if you set it to 8 seconds, it'd be only 3 HTTP requests worth of savings. I certainly hope you're not deploying via 3G, as the overhead of 3-6 HTTP requests is tiny compared to deploying an application. That being said, you'll probably want to file a feature request here:http://code.google.com/p/googleappengine/issues/list. It seems like a reasonable enough request to make this a parameter on deploy. It's not a great answer: but if you want to do this with the Python SDK, there's always monkeypatching ... -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger:http://googleappengine.blogspot.com Reddit:http://www.reddit.com/r/appengine Twitter:http://twitter.com/app_engine On Fri, Oct 8, 2010 at 10:27 AM, Peter Ondruska peter.ondru...@gmail.comwrote: Whenever I deploy new version/update to my GAE applications (be it Java or Python--it does not matter) it never happened to me that new version was ready earlier than 8 seconds (mostly 32, 64 or even later). I do not believe it is just me. Can I somehow configure on my side when to check for readiness earliest? And I plan to use 3G connection in near future and those extra requests in first 1+2+4+8 seconds are just waste of bandwidth and data... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Site not working after update
Thanks for your help guys! Site deployment finally worked! Jamie, is it working for you as well? Would be nice to get response from google, since dont know if site deployment fixed automagically or someone from google did something. Code bugs should not block the site deployment process. On 12 October 2010 08:45, Andrius A andriu...@gmail.com wrote: Thanks Tim, you are right, i made changes by wrapping model with timezones which is returning starttime as null. Its easy to fix, but deployment doesnt work anymore! Its stuck for waiting when new deployed site will become available.. On 12 Oct 2010 07:55, Tim Hoffman zutes...@gmail.com wrote: Hi For this error BadValueError: Property starttime is required it looks like you have changed your model to make starttime required and your trying to retrieve and entity that doesn't have a value for starttime. You will need to fix your entities if you want that property required. The import error will be because you have a broken instance (failed on startup with DeadlineExceeded and can't run now due to the failed imports) or you haven't got a signals.py somewhere ;-) T On Oct 12, 9:21 am, Andrew andriu...@gmail.com wrote: Hello, could someone from google have a look at site: bidshapeauction Site stopped working after update, the same code works fine on other app ids. We are getting different errors: type 'exceptions.ImportError': cannot import name signals or BadValueError: Property starttime is required Site is really slow to load. On google app engine I see anomaly with Memcache, anyone having issues? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Why initial_value's type of memcache.incr() atomically change to str?
Hi, I found a problem when setting initial_value of memcache.incr() and memcache.decr(). See this code: from google.appengine.api import memcache memcache.delete('a') memcache.incr('a', 1, initial_value=0) print type(memcache.get('a')) print (memcache.get('a')) Result: type 'str' 1 I think it should be type 'int' instead of type 'str'. Is it a bug? -- keakon -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Large datastore queries much slower with Python than Java?
Hi I know this is a nit, but with the python version, is there any particular reason why you are doing objects = list(q.Get(fetch_size)) nfetched += len(objects) at line 108 in test_load_query.py the list() is redundant, the result from Get is for all purposes is a list or at least exceedingly list like that you wouldn't bother creating another list from it. objects = g.Get(fetch_size) nfetched += len(objects) It really won't make any difference to the performance. Rgds T On Oct 12, 6:26 am, Darshan Shaligram scinti...@gmail.com wrote: On Mon, Oct 11, 2010 at 5:18 PM, Eli Jones eli.jo...@gmail.com wrote: Well.. for one.. you are doing a datastore.query() instead of a db.query() Most all documentation on working with the datastore indicates to use db from google.appengine.ext instead of datastore from google.appengine.api. Maybe there is a difference in how they perform in this context? The performance of the high-level db api is pretty similar to the performance of the low-level datastore API. I used the low-level api so that I could keep the code reasonably similar for both Python and Java. I'm much less familiar with the Java datastore API and I didn't want to use Java layers that might muddy the waters performance-wise. Having used the low-level api for Java, I used the closest corresponding apis for Python. The reason I wrote these simple Python and Java projects was to investigate datastore query performance issues I had in real Python code (using google.appengine.ext.db), and comparing notes with a colleague who was familiar with Java datastore performance. Also, are you doing these tests on Appengine or in the Dev_appserver? These tests are on the appengine, not the dev server. I use the same application name for both Java and Python versions, and different versions for both (Java = v1, Python = v2) so that they share the same datastore. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] index [X, X, X, Y, Z] built failed and always be resulted in Error state
Hi Eric, This is a classic 'exploding' index. With 10 elements in the list, this index will try to generate 10*9*8*7*6 = 30,240 index entries - far above the limit. It's not possible to build this index. Consider restructuring your data, or using merge join queries. -Nick Johnson On Tue, Oct 12, 2010 at 3:37 PM, Eric Ka Ka Ng ngk...@gmail.com wrote: I would like to build an index for the following entity Class entityA (db.Model): X db.StringListProperty() Y db.IntegerProperty() Z db.IntegerProperty() in index.yaml - kind: entityA properties: - name: Y - name: X - name: X - name: X - name: X - name: X - name: Z direction: desc but the index cannot be built and is always resulted in Error State Thx now we can vaccum the 'error' index now without being stuck in the building state and have to send emails to G to ask to mark the index as 'Error' But I have tried several times to vaccum the index, then re-build, but always fail May i know why it keeps in failing? The StringList property X usually contains 5-10 strings. Best Regards, Eric -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: High Latency Issues
We (appid steprep) are starting to see bursts of 30-s and 10-s timeouts occur again. E.g., 10-12 07:59AM 45.772 /_ah/queue/deferred 500 10012ms 10-12 07:59AM 43.185 /_ah/queue/deferred 500 10202ms 10-12 07:59AM 16.844 /_ah/queue/deferred 500 11799ms 10-12 07:57AM 44.693 /_ah/queue/deferred 500 29328ms 10-12 07:57AM 40.662 /_ah/queue/deferred 500 29515ms 10-12 07:57AM 29.826 /_ah/queue/deferred 500 35161ms 10-12 07:57AM 34.542 /_ah/queue/deferred 500 29803ms 10-12 07:57AM 28.881 /_ah/queue/deferred 500 33018ms 10-12 07:57AM 29.604 /_ah/queue/deferred 500 29740ms 10-12 07:57AM 29.205 /_ah/queue/deferred 500 29261ms 10-12 07:57AM 26.146 /_ah/queue/deferred 500 30481ms 10-12 07:57AM 25.589 /_ah/queue/deferred 500 29358ms 10-12 07:57AM 23.584 /_ah/queue/deferred 500 29611ms 10-12 07:57AM 41.243 /mgmt/alerts/trigger/ 500 10005ms 10-12 07:57AM 04.023 /_ah/queue/deferred 500 35939ms 10-12 07:57AM 05.307 /_ah/queue/deferred 500 29630ms 10-12 07:56AM 54.670 /_ah/queue/deferred 500 29673ms 10-12 07:56AM 52.672 /_ah/queue/deferred 500 29831ms 10-12 07:56AM 52.380 /_ah/queue/deferred 500 29921ms 10-12 07:56AM 52.379 /_ah/queue/deferred 500 29452ms 10-12 07:56AM 30.482 /_ah/queue/deferred 500 29534ms 10-12 07:56AM 28.382 /_ah/queue/deferred 500 29502ms 10-12 07:56AM 19.713 /_ah/queue/deferred 500 37933ms 10-12 07:56AM 24.241 /_ah/queue/deferred 500 29304ms 10-12 07:56AM 22.052 /_ah/queue/deferred 500 29793ms 10-12 07:56AM 18.893 /_ah/queue/deferred 500 31390ms 10-12 07:56AM 19.341 /_ah/queue/deferred 500 30034ms 10-12 07:56AM 18.722 /_ah/queue/deferred 500 29845ms 10-12 07:56AM 18.695 /_ah/queue/deferred 500 29467ms 10-12 07:56AM 33.521 /exec-reports/queue/increment-all-time-data- counter 500 10027ms 10-12 07:54AM 12.524 /_ah/queue/deferred 500 29353ms 10-12 07:54AM 11.365 /_ah/queue/deferred 500 29925ms and a little bit earlier: 10-12 07:48AM 42.456 /_ah/queue/deferred 500 10008ms 10-12 07:48AM 41.223 /_ah/queue/deferred 500 10007ms 10-12 07:48AM 38.840 /_ah/queue/deferred 500 10058ms 10-12 07:48AM 37.868 /_ah/queue/deferred 500 10033ms 10-12 07:48AM 37.283 /_ah/queue/deferred 500 10006ms 10-12 07:48AM 36.481 /_ah/queue/deferred 500 10008ms 10-12 07:48AM 36.218 /_ah/queue/deferred 500 10001ms 10-12 07:48AM 36.184 /mgmt/jobs/put/ 500 10027ms 10-12 07:48AM 35.902 /_ah/queue/deferred 500 10001ms 10-12 07:48AM 35.888 /_ah/queue/deferred 500 10002ms 10-12 07:48AM 35.792 /exec-reports/queue/increment-all-time-data- counter 500 10007ms 10-12 07:48AM 34.955 /_ah/queue/deferred 500 10074ms 10-12 07:48AM 34.678 /_ah/queue/deferred 500 10007ms 10-12 07:48AM 33.743 /exec-reports/queue/increment-all-time-data- counter 500 10005ms 10-12 07:48AM 31.323 /mgmt/jobs/put/deferred/?jobId=f54cddcf-05db-4cdc- a2ec-3859f8cb970e 500 10045ms 10-12 07:48AM 30.146 /_ah/queue/deferred 500 10001ms 10-12 07:48AM 30.106 /_ah/queue/deferred 500 10039ms 10-12 07:48AM 29.551 /_ah/queue/deferred 500 10013ms 10-12 07:48AM 29.332 /mgmt/jobs/put/deferred/?jobId=1135b765-3199-4961- a232-1ee46c235058 500 10036ms 10-12 07:48AM 27.608 /_ah/queue/deferred 500 10015ms 10-12 07:48AM 27.566 /mgmt/jobs/put/deferred/? jobId=26addd00-9958-4747-9f48-b8f5acc61103 500 10012ms 10-12 07:48AM 27.172 /mgmt/jobs/put/deferred/? jobId=8b69f017-1dc9-4b0f-8491-c3154a2b5dbb 500 10117ms 10-12 07:48AM 27.154 /mgmt/jobs/put/deferred/?jobId=5f387ec5-142a-43fc- be04-5d911d34c528 500 10001ms 10-12 07:48AM 26.930 /exec-reports/queue/increment-daily-data-counter 500 10024ms 10-12 07:48AM 26.942 /exec-reports/queue/increment-all-time-data- counter 500 10009ms 10-12 07:48AM 26.869 /mgmt/jobs/put/deferred/? jobId=aa4d1923-0ce9-4ad5-88ef-7c835ba3db14 500 10031ms 10-12 07:48AM 26.377 /_ah/queue/deferred 500 10001ms 10-12 07:48AM 26.376 /exec-reports/queue/increment-all-time-data- counter 500 10001ms 10-12 07:48AM 26.357 /_ah/queue/deferred 500 10020ms 10-12 07:48AM 26.347 /mgmt/jobs/put/deferred/?jobId=27b61fd1- b980-4a8e-9b26-cb517a2fbe2a 500 10006ms 10-12 07:48AM 26.345 /_ah/queue/deferred 500 10006ms 10-12 07:48AM 26.342 /exec-reports/queue/increment-all-time-data- counter 500 10008ms There are all tasks that run quickly (i.e., much less than 30-s) under normal operation. I can find more example if desired. j On Oct 11, 11:04 am, Jeff Schwartz jefftschwa...@gmail.com wrote: I am only guessing but I imagine that 'cluster' is some form of load balancing segment that Google uses to service and route requests to the virtual servers and this error/warning is indicative of when that queue is very busy. And regardless of if it is an error or a mere warning it still resulted in a 500, extremely high resource usage and a negative experience for the end user. On Mon, Oct 11, 2010 at 12:56 PM, Jamie H ja...@mhztech.com wrote: What do you mean by cold instance startup that timed out ? Sometimes I will get the error: Request was aborted after waiting too long to attempt to service your request. This may happen sporadically when the App Engine serving cluster is under
Re: [google-appengine] index [X, X, X, Y, Z] built failed and always be resulted in Error state
Hi Nick. It doesn't appear from the orignal poster's example that even if he had 5000 entries (the max) in his list property that that would cause an exploding index. Can you elaborate. On Tue, Oct 12, 2010 at 10:59 AM, Nick Johnson (Google) nick.john...@google.com wrote: Hi Eric, This is a classic 'exploding' index. With 10 elements in the list, this index will try to generate 10*9*8*7*6 = 30,240 index entries - far above the limit. It's not possible to build this index. Consider restructuring your data, or using merge join queries. -Nick Johnson On Tue, Oct 12, 2010 at 3:37 PM, Eric Ka Ka Ng ngk...@gmail.com wrote: I would like to build an index for the following entity Class entityA (db.Model): X db.StringListProperty() Y db.IntegerProperty() Z db.IntegerProperty() in index.yaml - kind: entityA properties: - name: Y - name: X - name: X - name: X - name: X - name: X - name: Z direction: desc but the index cannot be built and is always resulted in Error State Thx now we can vaccum the 'error' index now without being stuck in the building state and have to send emails to G to ask to mark the index as 'Error' But I have tried several times to vaccum the index, then re-build, but always fail May i know why it keeps in failing? The StringList property X usually contains 5-10 strings. Best Regards, Eric -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] index [X, X, X, Y, Z] built failed and always be resulted in Error state
Hi Nick, From the docs at http://code.google.com/appengine/docs/java/datastore/queriesandindexes.html#Big_Entities_and_Exploding_Indexes : You could call an index that refers to multiple properties with multiple values an exploding index, because it can get very large with just a few values. I believe the key words in the above are multiple properties with multiple values however I cannot see anywhere where there is a compound index using multiple properties that include the list property which according to the above is what would/could create an exploding index. I should preface this by saying that I am not a python developer so if there is something in the yaml file that does create this situation I apologize but would you mind pointing out what in the yaml file is? Thanks in advance. On Tue, Oct 12, 2010 at 11:22 AM, Nick Johnson (Google) nick.john...@google.com wrote: Hi Jeff, X is a StringListProperty, and it's indexed on multiple times. The number of index entries generated for multiple instances of the same list property is n!/(n-m)!, where n is the number of elements in the list, and m is the number of repeats in the index. -Nick Johnson On Tue, Oct 12, 2010 at 4:18 PM, Jeff Schwartz jefftschwa...@gmail.comwrote: Hi Nick. It doesn't appear from the orignal poster's example that even if he had 5000 entries (the max) in his list property that that would cause an exploding index. Can you elaborate. On Tue, Oct 12, 2010 at 10:59 AM, Nick Johnson (Google) nick.john...@google.com wrote: Hi Eric, This is a classic 'exploding' index. With 10 elements in the list, this index will try to generate 10*9*8*7*6 = 30,240 index entries - far above the limit. It's not possible to build this index. Consider restructuring your data, or using merge join queries. -Nick Johnson On Tue, Oct 12, 2010 at 3:37 PM, Eric Ka Ka Ng ngk...@gmail.com wrote: I would like to build an index for the following entity Class entityA (db.Model): X db.StringListProperty() Y db.IntegerProperty() Z db.IntegerProperty() in index.yaml - kind: entityA properties: - name: Y - name: X - name: X - name: X - name: X - name: X - name: Z direction: desc but the index cannot be built and is always resulted in Error State Thx now we can vaccum the 'error' index now without being stuck in the building state and have to send emails to G to ask to mark the index as 'Error' But I have tried several times to vaccum the index, then re-build, but always fail May i know why it keeps in failing? The StringList property X usually contains 5-10 strings. Best Regards, Eric -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to
Re: [google-appengine] index [X, X, X, Y, Z] built failed and always be resulted in Error state
Hi Jeff, On Tue, Oct 12, 2010 at 4:32 PM, Jeff Schwartz jefftschwa...@gmail.comwrote: Hi Nick, From the docs at http://code.google.com/appengine/docs/java/datastore/queriesandindexes.html#Big_Entities_and_Exploding_Indexes : You could call an index that refers to multiple properties with multiple values an exploding index, because it can get very large with just a few values. I believe the key words in the above are multiple properties with multiple values however I cannot see anywhere where there is a compound index using multiple properties that include the list property which according to the above is what would/could create an exploding index. I should preface this by saying that I am not a python developer so if there is something in the yaml file that does create this situation I apologize but would you mind pointing out what in the yaml file is? The same (list) property multiple times has the same effect as using multiple different properties. When you include a list in a custom index, all the values of the list must be indexed; when you include multiple lists, or multiple instances of the same list, all unique combinations of values must be indexed. -Nick Johnson Thanks in advance. On Tue, Oct 12, 2010 at 11:22 AM, Nick Johnson (Google) nick.john...@google.com wrote: Hi Jeff, X is a StringListProperty, and it's indexed on multiple times. The number of index entries generated for multiple instances of the same list property is n!/(n-m)!, where n is the number of elements in the list, and m is the number of repeats in the index. -Nick Johnson On Tue, Oct 12, 2010 at 4:18 PM, Jeff Schwartz jefftschwa...@gmail.comwrote: Hi Nick. It doesn't appear from the orignal poster's example that even if he had 5000 entries (the max) in his list property that that would cause an exploding index. Can you elaborate. On Tue, Oct 12, 2010 at 10:59 AM, Nick Johnson (Google) nick.john...@google.com wrote: Hi Eric, This is a classic 'exploding' index. With 10 elements in the list, this index will try to generate 10*9*8*7*6 = 30,240 index entries - far above the limit. It's not possible to build this index. Consider restructuring your data, or using merge join queries. -Nick Johnson On Tue, Oct 12, 2010 at 3:37 PM, Eric Ka Ka Ng ngk...@gmail.comwrote: I would like to build an index for the following entity Class entityA (db.Model): X db.StringListProperty() Y db.IntegerProperty() Z db.IntegerProperty() in index.yaml - kind: entityA properties: - name: Y - name: X - name: X - name: X - name: X - name: X - name: Z direction: desc but the index cannot be built and is always resulted in Error State Thx now we can vaccum the 'error' index now without being stuck in the building state and have to send emails to G to ask to mark the index as 'Error' But I have tried several times to vaccum the index, then re-build, but always fail May i know why it keeps in failing? The StringList property X usually contains 5-10 strings. Best Regards, Eric -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email
Re: [google-appengine] Re: Multiple Domains for one app, even if you don't own the domain, or if other domain is already registered with Google Apps
Hi James, On Mon, Oct 11, 2010 at 8:42 AM, James M jmort...@gmail.com wrote: I edited my GoDaddy DNS A and CNAME records and was able to point the naked domain at my app engine app. It's possible to do even though different resources on the Internet claim that it's impossible. I suspect that Google means that it's impossible to do from Google Apps or the App Engine Dashboard, but that doesn't include advanced DNS configuration. * I configured an A record to point *.domain.com to my app engine app IP Address *.domain.com will map all subdomains to the same IP, not the 'naked' domain. Further, ghs.google.com resolves to a number of IPs; these IPs change over time and depending on the location of the requester. Simply choosing one from the list will not work reliably. Finally, App Engine will not be able to route your incoming requests correctly unless it's aware of your mapping, which it's not possible to do for a naked domain. -Nick Johnson * I configured a CNAME for www.domain.com to point to ghs.google.com I suspect that one of those two changes may have something to do with my success. Although honestly this isn't something that I do on a regular basis. Hope this helps! If it does, awesome! James On Sep 20, 6:27 pm, Geoffrey Spear geoffsp...@gmail.com wrote: You can't point the naked domain at an App Engine app at all. However, App Engine doesn't control your DNS settings. You set CNAMEs for the subdomains you want handled by App Engine or other Google Apps services to ghs.google.com, and setup CNAME or A records for the other addresses in your domain to point to wherever you want them to go. On Sep 20, 6:02 pm, nacho vela.igna...@gmail.com wrote: What about the inverse case? Let's say i have mydomain.com poiting to my appengine app and i want to use something.mydomain.com poiting to another server? How can i do it? On 20 sep, 05:11, adams ad...@contentedweb.com wrote: I've been playing around with this kind of scenario as well. As far as I can tell you don't have to do anything from your side, except give them your app engine id. Your client needs to have a Google Apps account for their domain name. They can then add your app engine application to their Google Apps account and point any URL for that domain to your app engine application. They would do this by: 1. Logging into their Google Apps account 2. Click on Add more services 3. Enter app id on the Google App Engine section under Other services I am not 100% sure the above is true, but it seems to have worked for one of my Google Apps domains. On Sep 17, 10:05 pm, NickFranceschina nickfrancesch...@gmail.com wrote: I promise I have been searching... but I haven't been unable to find the correct answer. Thus I turn to you fine folks. So lets say I have my super fancy new killer app-engine app: http://mykillerapp.com which is my custom domain pointed to: http://mykillerapp.appspot.com and then because I'm a genius salesman, I convince Coca-Cola that they need my app on their site. They write me a huge check with lots of zeros and say OK, lets set it up athttp://awesome.coca-cola.com; how can I make that happen? the only way i can see is to add coca- cola.com to my google apps for domains account (verifying it with CNAME or .HTML file)... but yeah, Coke isn't gonna let me do that cuz it's obviously their domain. or what if I sell it to JoeSchmoe.com who already has a google apps account... I can't add his domain to my google apps because google tells me this domain is already registered (or something like that) SOOO... is there a way for me to only add awesome.coca-cola.com to my google apps domain so I can link the app to it? or setup awesome.JoeSchmoe.com if JoeSchmoe.com is already a google apps customer? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at
Re: [google-appengine] index [X, X, X, Y, Z] built failed and always be resulted in Error state
So a list property with 5000 entries in it that is not part of any compound index then would not in itself create an exploding index situation, but would if it were part of a compound index, correct? Nor would querying on a list property with 5000 entries not part of any compound index using an equality filer, correct? The reason I am asking all these questions is because I've implemented a fanout solution based on Brett Slatkin's youtube video - http://www.youtube.com/watch?v=AgaL6NGpkB8 - and was concerned that the principle's he promoted in the video using entitie with list properties as indexes to their parents aren't now problematic or considered bad practice or that changes have been made to the datastore where employing his technique would not create exploding indexes. Thanks in advance and for bearing with my many questions. Jeff On Tue, Oct 12, 2010 at 11:41 AM, Nick Johnson (Google) nick.john...@google.com wrote: Hi Jeff, On Tue, Oct 12, 2010 at 4:32 PM, Jeff Schwartz jefftschwa...@gmail.comwrote: Hi Nick, From the docs at http://code.google.com/appengine/docs/java/datastore/queriesandindexes.html#Big_Entities_and_Exploding_Indexes : You could call an index that refers to multiple properties with multiple values an exploding index, because it can get very large with just a few values. I believe the key words in the above are multiple properties with multiple values however I cannot see anywhere where there is a compound index using multiple properties that include the list property which according to the above is what would/could create an exploding index. I should preface this by saying that I am not a python developer so if there is something in the yaml file that does create this situation I apologize but would you mind pointing out what in the yaml file is? The same (list) property multiple times has the same effect as using multiple different properties. When you include a list in a custom index, all the values of the list must be indexed; when you include multiple lists, or multiple instances of the same list, all unique combinations of values must be indexed. -Nick Johnson Thanks in advance. On Tue, Oct 12, 2010 at 11:22 AM, Nick Johnson (Google) nick.john...@google.com wrote: Hi Jeff, X is a StringListProperty, and it's indexed on multiple times. The number of index entries generated for multiple instances of the same list property is n!/(n-m)!, where n is the number of elements in the list, and m is the number of repeats in the index. -Nick Johnson On Tue, Oct 12, 2010 at 4:18 PM, Jeff Schwartz jefftschwa...@gmail.comwrote: Hi Nick. It doesn't appear from the orignal poster's example that even if he had 5000 entries (the max) in his list property that that would cause an exploding index. Can you elaborate. On Tue, Oct 12, 2010 at 10:59 AM, Nick Johnson (Google) nick.john...@google.com wrote: Hi Eric, This is a classic 'exploding' index. With 10 elements in the list, this index will try to generate 10*9*8*7*6 = 30,240 index entries - far above the limit. It's not possible to build this index. Consider restructuring your data, or using merge join queries. -Nick Johnson On Tue, Oct 12, 2010 at 3:37 PM, Eric Ka Ka Ng ngk...@gmail.comwrote: I would like to build an index for the following entity Class entityA (db.Model): X db.StringListProperty() Y db.IntegerProperty() Z db.IntegerProperty() in index.yaml - kind: entityA properties: - name: Y - name: X - name: X - name: X - name: X - name: X - name: Z direction: desc but the index cannot be built and is always resulted in Error State Thx now we can vaccum the 'error' index now without being stuck in the building state and have to send emails to G to ask to mark the index as 'Error' But I have tried several times to vaccum the index, then re-build, but always fail May i know why it keeps in failing? The StringList property X usually contains 5-10 strings. Best Regards, Eric -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to
[google-appengine] Serious problem caused process to exit... but where are my logs?
Hi there, My app is doing great locally. It deploys fine. Now I get this error in my logs everytime I try to execute a couple of updates to the datastore in the same transaction. In fact, I know part of my code executes OK since one of the updates makes it to the datastore but the other does not. And it's not busting the 30 seconds limit as this transaction takes less than a second locally. Each entity is part of a different entity group... but I had disabled transactions when I was using jpa. I have recently switch from java/ jpa to Objectify. I am also not using transactions and should be able to update different entity groups within the same transaction. My question really is, how come you don't get any log output when the request causes the process to exit? Makes it pretty hard to find the issue. I have tried updating different entities, calling the task directly instead of having the task queue api do it, removed security, changed the http error codes returned, catching errors and logging them somehow... and a bunch of other things. What would really help would be running with full debug logging on and getting my log output. Thanks for an otherwise incredible platform, Remy -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: How about JavaScript v8 as the next language for Google App Engine?
big +1 from my side. express.js is a great example of a not complete but complete-sih framework on top of node. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: TransientError on adding task queue, yet the status page says no error.
I've seen this periodically. The pattern that seems to be percolating around the list is to always wrap a task insertion (if important) in an try/except, and resubmit the job on failure. Myself, I try to insert once in the except block, and then throw up my hands forever. That appears to be work so far. On Sep 22, 8:34 pm, Raymond C. windz...@gmail.com wrote: Anyone else having the same problem? or its just me? Most of my task queue insertions are failing with below error, they were working fine for several weeks. The traces are as below: File /base/python_runtime/python_lib/versions/1/google/appengine/ext/ deferred/deferred.py, line 241, in defer return task.add(queue, transactional=transactional) File /base/python_runtime/python_lib/versions/1/google/appengine/api/ labs/taskqueue/taskqueue.py, line 563, in add return Queue(queue_name).add(self, transactional=transactional) File /base/python_runtime/python_lib/versions/1/google/appengine/api/ labs/taskqueue/taskqueue.py, line 617, in add self.__AddTasks(tasks, transactional) File /base/python_runtime/python_lib/versions/1/google/appengine/api/ labs/taskqueue/taskqueue.py, line 645, in __AddTasks raise self.__TranslateError(e.application_error, e.error_detail) TransientError -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] NeedIndexError: The built-in indices are not efficient enough (related to large result set for self merge-join?)
Thx Robert. Your reference is always good materials for me to study. Would study it and see if can come up with any solution to solve my problem following Alfred's optimization! And if still cannot, would bring up the qeustion again ;) Best Regards, Eric On 12 October 2010 00:48, Robert Kluin robert.kl...@gmail.com wrote: Hey Eric, Perhaps you can come up with a method to identify the 'best' search parameters when deciding how to filter your items. You should also check out Alfred Fuller's IO talk, next gen queries on app engine. http://code.google.com/events/io/2010/sessions/next-gen-queries-appengine.html They are now starting to support some of those items. From the 1.3.8 (pre)release notes: - Removed limits on zigzag merge-join queries. Therefore the error The built-in indices are not efficient enough for this query and your data. Please add a composite index for this query. will no longer be thrown in many cases, enabling more types of exploratory queries without indexes. Maybe some of Alfred's optimizations can help you. Robert On Mon, Oct 11, 2010 at 05:35, Eric Ka Ka Ng ngk...@gmail.com wrote: From the slide Building scalable, complex apps on App Engine by Brett Slatkin http://dl.google.com/io/2009/pres/W_0415_Building_Scalable_Complex_App_Engines.pdf we learn techniques for building scalable services by using list and self merge-join. self merge-join is very useful to me as it does not require building additional custom indexes (the hard limit of 200 custom indexes is always a headache for me with my application, and always have to use many different techniques to reduce the custom indexes usage) however, recently i hit a problem NeedIndexError: The built-in indices are not efficient enough for this query and your data. Please add a composite index for this query. My model is very simple, just like this class Searchable(db.Model): phrases = db.StringListProperty() the query is query = searchable.all().filter('phrases =', word1).filter('phrases =', word2).fetch(1000) It works fine before. But now fail with some cases. I re-read slides, and find that about merge-join performance Scales with number of filters and size of result set Best for queries with fewer results (less than 100) i guess very likely, now when my application scales with more data (let say Searchable now stores 1 million records), when doing the query and if the result set is large (how large is large? 100?), i would then hit this 'built-in indices are not efficient enough' problem. the query works fine if the returned result set is much smaller this is my guessing. so my questions are 1. is my guess correct / making sense? what other cases would cause 'built-in indices are not efficient enough' problem? are there any others share similar experiences? 2. is there any hard number, or formula, we can use to estimate by how large the result set is, would hit the 'built-in indices are not efficient enough' problem? other than size of result set, would there be other factor matter? (number of self merge-join, number of properties, number of indexed properties etc.) 3. the most important question, any solution / workaround for that ? (other than building extra custom indexes which is not quite feasible for me as i'm already having 200 indexes) appreciate for any comments / suggestions. - eric ng -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] index [X, X, X, Y, Z] built failed and always be resulted in Error state
Nick is saying that the StringListProperty has values like: A, B, C. If you also have another property, say an IntegerProperty with value 7, and you create a compound index that contains the IntegerProperty and the StringListProperty you will be OK because the index will look like: 7-A 7-B 7-C However, if the index contains the IntegerProperty and the StringListProperty twice the index looks something like: 7-A-A 7-A-B 7-A-C 7-B-B 7-B-C 7-C-C The issue is NOT from one multiple valued (ie ListProperty) being in a compound index once, but rather from one or more multiple valued properties appearing more than one time in the same compound index. Robert On Tue, Oct 12, 2010 at 12:00, Jeff Schwartz jefftschwa...@gmail.com wrote: So a list property with 5000 entries in it that is not part of any compound index then would not in itself create an exploding index situation, but would if it were part of a compound index, correct? Nor would querying on a list property with 5000 entries not part of any compound index using an equality filer, correct? The reason I am asking all these questions is because I've implemented a fanout solution based on Brett Slatkin's youtube video - http://www.youtube.com/watch?v=AgaL6NGpkB8 - and was concerned that the principle's he promoted in the video using entitie with list properties as indexes to their parents aren't now problematic or considered bad practice or that changes have been made to the datastore where employing his technique would not create exploding indexes. Thanks in advance and for bearing with my many questions. Jeff On Tue, Oct 12, 2010 at 11:41 AM, Nick Johnson (Google) nick.john...@google.com wrote: Hi Jeff, On Tue, Oct 12, 2010 at 4:32 PM, Jeff Schwartz jefftschwa...@gmail.com wrote: Hi Nick, From the docs at http://code.google.com/appengine/docs/java/datastore/queriesandindexes.html#Big_Entities_and_Exploding_Indexes: You could call an index that refers to multiple properties with multiple values an exploding index, because it can get very large with just a few values. I believe the key words in the above are multiple properties with multiple values however I cannot see anywhere where there is a compound index using multiple properties that include the list property which according to the above is what would/could create an exploding index. I should preface this by saying that I am not a python developer so if there is something in the yaml file that does create this situation I apologize but would you mind pointing out what in the yaml file is? The same (list) property multiple times has the same effect as using multiple different properties. When you include a list in a custom index, all the values of the list must be indexed; when you include multiple lists, or multiple instances of the same list, all unique combinations of values must be indexed. -Nick Johnson Thanks in advance. On Tue, Oct 12, 2010 at 11:22 AM, Nick Johnson (Google) nick.john...@google.com wrote: Hi Jeff, X is a StringListProperty, and it's indexed on multiple times. The number of index entries generated for multiple instances of the same list property is n!/(n-m)!, where n is the number of elements in the list, and m is the number of repeats in the index. -Nick Johnson On Tue, Oct 12, 2010 at 4:18 PM, Jeff Schwartz jefftschwa...@gmail.com wrote: Hi Nick. It doesn't appear from the orignal poster's example that even if he had 5000 entries (the max) in his list property that that would cause an exploding index. Can you elaborate. On Tue, Oct 12, 2010 at 10:59 AM, Nick Johnson (Google) nick.john...@google.com wrote: Hi Eric, This is a classic 'exploding' index. With 10 elements in the list, this index will try to generate 10*9*8*7*6 = 30,240 index entries - far above the limit. It's not possible to build this index. Consider restructuring your data, or using merge join queries. -Nick Johnson On Tue, Oct 12, 2010 at 3:37 PM, Eric Ka Ka Ng ngk...@gmail.com wrote: I would like to build an index for the following entity Class entityA (db.Model): X db.StringListProperty() Y db.IntegerProperty() Z db.IntegerProperty() in index.yaml - kind: entityA properties: - name: Y - name: X - name: X - name: X - name: X - name: X - name: Z direction: desc but the index cannot be built and is always resulted in Error State Thx now we can vaccum the 'error' index now without being stuck in the building state and have to send emails to G to ask to mark the index as 'Error' But I have tried several times to vaccum the index, then re-build, but always fail May i know why it keeps in failing? The StringList property X usually contains 5-10 strings. Best Regards, Eric -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to
Re: [google-appengine] index [X, X, X, Y, Z] built failed and always be resulted in Error state
Thanks, Robert. On Tue, Oct 12, 2010 at 1:02 PM, Robert Kluin robert.kl...@gmail.comwrote: Nick is saying that the StringListProperty has values like: A, B, C. If you also have another property, say an IntegerProperty with value 7, and you create a compound index that contains the IntegerProperty and the StringListProperty you will be OK because the index will look like: 7-A 7-B 7-C However, if the index contains the IntegerProperty and the StringListProperty twice the index looks something like: 7-A-A 7-A-B 7-A-C 7-B-B 7-B-C 7-C-C The issue is NOT from one multiple valued (ie ListProperty) being in a compound index once, but rather from one or more multiple valued properties appearing more than one time in the same compound index. Robert On Tue, Oct 12, 2010 at 12:00, Jeff Schwartz jefftschwa...@gmail.com wrote: So a list property with 5000 entries in it that is not part of any compound index then would not in itself create an exploding index situation, but would if it were part of a compound index, correct? Nor would querying on a list property with 5000 entries not part of any compound index using an equality filer, correct? The reason I am asking all these questions is because I've implemented a fanout solution based on Brett Slatkin's youtube video - http://www.youtube.com/watch?v=AgaL6NGpkB8 - and was concerned that the principle's he promoted in the video using entitie with list properties as indexes to their parents aren't now problematic or considered bad practice or that changes have been made to the datastore where employing his technique would not create exploding indexes. Thanks in advance and for bearing with my many questions. Jeff On Tue, Oct 12, 2010 at 11:41 AM, Nick Johnson (Google) nick.john...@google.com wrote: Hi Jeff, On Tue, Oct 12, 2010 at 4:32 PM, Jeff Schwartz jefftschwa...@gmail.com wrote: Hi Nick, From the docs at http://code.google.com/appengine/docs/java/datastore/queriesandindexes.html#Big_Entities_and_Exploding_Indexes : You could call an index that refers to multiple properties with multiple values an exploding index, because it can get very large with just a few values. I believe the key words in the above are multiple properties with multiple values however I cannot see anywhere where there is a compound index using multiple properties that include the list property which according to the above is what would/could create an exploding index. I should preface this by saying that I am not a python developer so if there is something in the yaml file that does create this situation I apologize but would you mind pointing out what in the yaml file is? The same (list) property multiple times has the same effect as using multiple different properties. When you include a list in a custom index, all the values of the list must be indexed; when you include multiple lists, or multiple instances of the same list, all unique combinations of values must be indexed. -Nick Johnson Thanks in advance. On Tue, Oct 12, 2010 at 11:22 AM, Nick Johnson (Google) nick.john...@google.com wrote: Hi Jeff, X is a StringListProperty, and it's indexed on multiple times. The number of index entries generated for multiple instances of the same list property is n!/(n-m)!, where n is the number of elements in the list, and m is the number of repeats in the index. -Nick Johnson On Tue, Oct 12, 2010 at 4:18 PM, Jeff Schwartz jefftschwa...@gmail.com wrote: Hi Nick. It doesn't appear from the orignal poster's example that even if he had 5000 entries (the max) in his list property that that would cause an exploding index. Can you elaborate. On Tue, Oct 12, 2010 at 10:59 AM, Nick Johnson (Google) nick.john...@google.com wrote: Hi Eric, This is a classic 'exploding' index. With 10 elements in the list, this index will try to generate 10*9*8*7*6 = 30,240 index entries - far above the limit. It's not possible to build this index. Consider restructuring your data, or using merge join queries. -Nick Johnson On Tue, Oct 12, 2010 at 3:37 PM, Eric Ka Ka Ng ngk...@gmail.com wrote: I would like to build an index for the following entity Class entityA (db.Model): X db.StringListProperty() Y db.IntegerProperty() Z db.IntegerProperty() in index.yaml - kind: entityA properties: - name: Y - name: X - name: X - name: X - name: X - name: X - name: Z direction: desc but the index cannot be built and is always resulted in Error State Thx now we can vaccum the 'error' index now without being stuck in the building state and have to send emails to G to ask to mark the index as 'Error' But I have tried several times to vaccum the index, then re-build, but always fail May i know why it
Re: [google-appengine] Why initial_value's type of memcache.incr() atomically change to str?
Looks like it might be a bug, check out _validate_encode_value (line 151) and _decode_value (line 205). The value gets converted to a str when set, but should be decoded to an int when you call get. http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/api/memcache/__init__.py Robert On Tue, Oct 12, 2010 at 06:55, 风笑雪 kea...@gmail.com wrote: Hi, I found a problem when setting initial_value of memcache.incr() and memcache.decr(). See this code: from google.appengine.api import memcache memcache.delete('a') memcache.incr('a', 1, initial_value=0) print type(memcache.get('a')) print (memcache.get('a')) Result: type 'str' 1 I think it should be type 'int' instead of type 'str'. Is it a bug? -- keakon -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Google Apps / Domain verification
I'm trying to setup a new domain name for my GAE app by adding a domain alias to my Google Apps account. I've verified the new domain through a CNAME, but now Google Apps is asking me to setup email for the domain by changing MX records. I don't want to setup email for this domain, I just want to host my GAE app on it. Any ideas on how I can add this domain to Google Apps without dealing with MX records? When I setup my GA account's primary domain, the CNAME was sufficient. I'm not sure why adding a Domain Alias seems to require this extra step. =T= -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Checking if new version is ready to serve only after 8 seconds
I think I will change Deploy and RetryWithBackoff to suit my need... On Oct 12, 11:18 am, Peter Ondruska peter.ondru...@gmail.com wrote: Thanks guys.. On Oct 12, 9:00 am, Tim Hoffman zutes...@gmail.com wrote: Hi I deploy over 3g, all the time. (Its the only type of link I can get where I live). And I can test to the fact , the polling/backoff waiting for the deploy are not really relevant in the scheme of things. It takes much longer to actually send everything ;-). T On Oct 9, 1:54 am, Ikai Lan (Google) ikai.l+gro...@google.com wrote: That's the thing about exponential backoffs: it grows so quickly. It goes from 1-2-4-8-16-32 seconds. That's only 6 HTTP requests, and if you set it to 8 seconds, it'd be only 3 HTTP requests worth of savings. I certainly hope you're not deploying via 3G, as the overhead of 3-6 HTTP requests is tiny compared to deploying an application. That being said, you'll probably want to file a feature request here:http://code.google.com/p/googleappengine/issues/list. It seems like a reasonable enough request to make this a parameter on deploy. It's not a great answer: but if you want to do this with the Python SDK, there's always monkeypatching ... -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger:http://googleappengine.blogspot.com Reddit:http://www.reddit.com/r/appengine Twitter:http://twitter.com/app_engine On Fri, Oct 8, 2010 at 10:27 AM, Peter Ondruska peter.ondru...@gmail.comwrote: Whenever I deploy new version/update to my GAE applications (be it Java or Python--it does not matter) it never happened to me that new version was ready earlier than 8 seconds (mostly 32, 64 or even later). I do not believe it is just me. Can I somehow configure on my side when to check for readiness earliest? And I plan to use 3G connection in near future and those extra requests in first 1+2+4+8 seconds are just waste of bandwidth and data... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Fan-in with materialized views: A sketch
Ben, thanks for your code! I'm trying to understand all this stuff too... Robert, any success with your library? May be you've already done all stuff we are trying to implement... p.s. where is Brett S.:) would like to hear his comments on this On Sep 21, 1:49 pm, Ben pondneverfree...@yahoo.com wrote: Thanks for your insights. I would love feedback on this implementation (Brett S. suggested we send in our code for this)http://pastebin.com/3pUhFdk8 This implementation is for just one materialized view row at a time (e.g. a simple counter, no presence markers). Hopefully putting an ETA on the transactional task will relieve the write pressure, since usually it should be an old update with an out-of-date sequence number and be discarded (the update having already been completed in batch by the fork-join-queue). I'd love to generalize this to do more than one materialized view row but thought I'd get feedback first. Thanks, Ben On Sep 17, 7:30 am, Robert Kluin robert.kl...@gmail.com wrote: Responses inline. On Thu, Sep 16, 2010 at 17:32, Ben pondneverfree...@yahoo.com wrote: I have a question about Brett Slatkin's talk at I/O 2010 on data pipelines. The question is about slide #67 of his pdf, corresponding to minute 51:30 of his talk http://code.google.com/events/io/2010/sessions/high-throughput-data-p... I am wondering what is supposed to happen in the transactional task (bullet point 2c). Would these updates to the materialized view cause you to write too frequently to the entity group containing the materialized view? I think there are really two different approaches you can use to insert your work models. 1) The work models get added to the original entity's group. So, inside of the original transaction you do not write to the entity group containing the materialized view -- so no contention on it. Commit the transaction and proceed to step 3. 2) You kick off a transactional task to insert the work model, or fan-out more tasks to create work models :). Then you proceed to step 3. You can use method 1 if you have only a few aggregates. If you have more aggregates use the second method. I have a library I am almost ready to open source that makes method 2 really easy, so you can have lots of aggregates. I'll post to this group when I release it. And a related question, what happens if there is a failure just after the transaction in bullet #2, but right before the named task gets inserted in bullet #3. In my current implementation I just left out the transactional task (bullet point 2c) but I think that causes me to lose the eventual consistency. Failure between steps 2 and 3 just means _that_ particular update will not try to kick-off, ie insert, the fan-in (aggregation) task. But it might have already been inserted by the previous update, or the next update. However, if nothing else kicks of the fan-in task you will need some periodic cleanup method to catch the update and kick of the fan-in task. Depending on exactly how you implemented step 2 you may not need a transactional task. Robert Thanks! -Ben -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Google Apps / Domain verification
You do not need to change MX records if you do not want Gmail for Apps. The only change required is CNAME-ing your record to ghs.google.com. On Oct 12, 11:20 pm, Tim Jones t...@eff.org wrote: I'm trying to setup a new domain name for my GAE app by adding a domain alias to my Google Apps account. I've verified the new domain through a CNAME, but now Google Apps is asking me to setup email for the domain by changing MX records. I don't want to setup email for this domain, I just want to host my GAE app on it. Any ideas on how I can add this domain to Google Apps without dealing with MX records? When I setup my GA account's primary domain, the CNAME was sufficient. I'm not sure why adding a Domain Alias seems to require this extra step. =T= -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Google Apps / Domain verification
You do not need to enable gmail on your domain or setup the MX records. Robert On Tue, Oct 12, 2010 at 17:20, Tim Jones t...@eff.org wrote: I'm trying to setup a new domain name for my GAE app by adding a domain alias to my Google Apps account. I've verified the new domain through a CNAME, but now Google Apps is asking me to setup email for the domain by changing MX records. I don't want to setup email for this domain, I just want to host my GAE app on it. Any ideas on how I can add this domain to Google Apps without dealing with MX records? When I setup my GA account's primary domain, the CNAME was sufficient. I'm not sure why adding a Domain Alias seems to require this extra step. =T= -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Google Apps / Domain verification
Then I'm not sure what I need to do to setup this domain name. When I try to add my new domain to my GAE app, it says Sorry, you've reached a login page for a domain that isn't using Google Apps. And when I go to my Google Apps Domain Settings page, the status of my new domain just says MX records setup validation in progress... Any suggestions on what my next step should be? =T= On Oct 12, 2010, at 3:10 PM, Robert Kluin wrote: You do not need to enable gmail on your domain or setup the MX records. Robert On Tue, Oct 12, 2010 at 17:20, Tim Jones t...@eff.org wrote: I'm trying to setup a new domain name for my GAE app by adding a domain alias to my Google Apps account. I've verified the new domain through a CNAME, but now Google Apps is asking me to setup email for the domain by changing MX records. I don't want to setup email for this domain, I just want to host my GAE app on it. Any ideas on how I can add this domain to Google Apps without dealing with MX records? When I setup my GA account's primary domain, the CNAME was sufficient. I'm not sure why adding a Domain Alias seems to require this extra step. =T= -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: TransientError on adding task queue, yet the status page says no error.
I am curious if by using deferred instead of tasks you will still have to worry about TransientErrors ? Since the deferred library handles inserting the task... I don't see anywhere in the docs of deferred saying you must worry about TransientError... On the same note I am curious if anybody knows if tasks are added or not when a transient error is thrown. I would assume they aren't but I have seen some references in the group that they are in fact added despite a TransientError.. any word on this? I rely heavily on tasks so it is very important I can be assured that when I add one it will be added... On Oct 12, 11:54 am, John Gardner hux...@gmail.com wrote: I've seen this periodically. The pattern that seems to be percolating around the list is to always wrap a task insertion (if important) in an try/except, and resubmit the job on failure. Myself, I try to insert once in the except block, and then throw up my hands forever. That appears to be work so far. On Sep 22, 8:34 pm, Raymond C. windz...@gmail.com wrote: Anyone else having the same problem? or its just me? Most of my task queue insertions are failing with below error, they were working fine for several weeks. The traces are as below: File /base/python_runtime/python_lib/versions/1/google/appengine/ext/ deferred/deferred.py, line 241, in defer return task.add(queue, transactional=transactional) File /base/python_runtime/python_lib/versions/1/google/appengine/api/ labs/taskqueue/taskqueue.py, line 563, in add return Queue(queue_name).add(self, transactional=transactional) File /base/python_runtime/python_lib/versions/1/google/appengine/api/ labs/taskqueue/taskqueue.py, line 617, in add self.__AddTasks(tasks, transactional) File /base/python_runtime/python_lib/versions/1/google/appengine/api/ labs/taskqueue/taskqueue.py, line 645, in __AddTasks raise self.__TranslateError(e.application_error, e.error_detail) TransientError -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] CPU burn rate 10 times higher today on this error: The API call datastore_v3.Get() was explicitly cancelled.
Hi GAE Team, Our app 'snsanalytics' is experiencing the following error frequently: CancelledError: The API call datastore_v3.Get() was explicitly cancelled. Whenever this error happens, a deferred job eats up 50,000+ cpu_ms. This causes our app is burning CPU at 10+ times higher than normal rate every hour. Could any one help investigate ASAP? Regards, Alan -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Log Parser for downloaded request_logs
Matthew Thank you for building this tool. While this tool helps, in my case, i am looking to download logs per request url, since downloading logs for complete app is not really required to fix specific issues. -V -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: CPU burn rate 10 times higher today on this error: The API call datastore_v3.Get() was explicitly cancelled.
The problem doesn't happen all the time though. To help you better diagnose, we have kept the version 2 of the app 'snsanalytics' unchanged. You can see the problem clearly from the most recent log of version 2. On Tue, Oct 12, 2010 at 6:36 PM, Alan Xing a...@sns.ly wrote: Hi GAE Team, Our app 'snsanalytics' is experiencing the following error frequently: CancelledError: The API call datastore_v3.Get() was explicitly cancelled. Whenever this error happens, a deferred job eats up 50,000+ cpu_ms. This causes our app is burning CPU at 10+ times higher than normal rate every hour. Could any one help investigate ASAP? Regards, Alan -- http://twitter.com/alanxing http://twitter.com/xinghailiang http://facebook.com/alanxing http://facebook.com/xinghailiang http://sns.ly - Your social media mgmt center. http://twitter.com/snsanalytics http://facebook.com/snsanalytics -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: High Latency Issues
I'm getting a fair amount of these. A serious problem was encountered with the process that handled this request, causing it to exit. This is likely to cause a new process to be used for the next request to your application. If you see this message frequently, you should contact the App Engine team. (Error code 203) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.