[appengine-java] Chat Time transcript for January 20, 2010
Two Wednesday back, the App Engine team hosted the latest session of its bimonthly IRC office hours. A transcript of the session and a summary of the topics covered is provided below. The next session will take place this coming Wednesday, February 3rd from 7:00-8:00 p.m. PST in the #appengine channel on irc.freenode.net. - Jason --SUMMARY--- - Q: Should you split a single entity into two or more entities if one or more properties are updated frequently, e.g. per request? A: No, but if you expect to update a single entity or entity group more than once per second, you should consider sharding to avoid datastore contention (http://code.google.com/appengine/articles/scaling/ contention.html). You generally don't need to split entities, although you can do so to save bandwidth if your entities contain several large properties that only need to be accessed occasionally. [9:01-9:06] - Q: Does the 30-second request deadline applied during Blobstore uploads? A: No. In general, time spent by the user doesn't count towards your execution time limit. [9:07-9:08] - Discussion on the maximum length of task queue parameters. [9:10-9:12] - Discussion on bulkloader dump and restore, whether there are any plans to allow for clearing a single datastore kind only, and whether tasks or remote_api is the best mechanism for clearing the datastore [9:16-9:20, 9:24-9:25] - Q: How can one determine the string-encoded key of a given entity for another application, e.g. when updating foreign keys ahead of a bulk upload to another datastore? A: This could change going forward, but for now you can supply the '_app_id_namespace' argument to db.Key.from_path. Or instead of using string-encoded keys, you can use Key.to_xml which provides a transparent encoding. Bear in mind that App Engine currently doesn't allow for referencing entities in other applications, though this could change in the next release. [9:24-9:25, 9:27-9:28, 9:32, 9:35-9:36] - Discussion on whether, when building a master-detail relationship (e.g. orders and items), it's better to store all order information in a single entity or to use an individual entity for each item in the order. [9:27-9:35] - Q: If you're new to App Engine, which runtime should you start with? A: Pick the runtime language most familiar to you to minimize the amount of new learning that you have to do all at once -- even though many use standard APIs, App Engine's various services will take time to master, especially the datastore. [9:35-9:38] - Q: For list properties, is the name of the property stored for each item (e.g. N times for N items) even when its unindexed? A: Yes -- the datastore is schemaless, so a list foo=[bar,baz] is actually stored as two properties with the same name: foo=bar and foo=baz. Hence, each item in the list requires the name to be stored with it. See the article at http://code.google.com/appengine/articles/storage_breakdown.html which explains exactly what gets stored for all entity and index rows. [9:40-9:43, 9:52-9:56, 9:59-10:01] - Discussion of deferred.defer, its ability to schedule tasks to be executed during off-peak hours, and its relation to the core product. [9:41-9:46, 9:50] --FULL TRANSCRIPT--- [09:00am] nickjohnson: Welcome to the fortnightly App Engine developer chat! With me today are Jason and Ikai. [09:00am] nickjohnson: And Ryan [09:00am] ikai_google: Hi everybody! [09:00am] ryan_google: and me! (...something) [09:01am] dennis_tw: I have a question about entities that are changed often (eg: changed on every user action. Like modification time or view count). I'm thinking of separating them out into a separate entity so that that 'high frequency' entity is small and can be moved between the datastore and my app with little overhead. Is that a reasonable entity design? [09:01am] ikai_google: *crickets* [09:02am] nickjohnson: dennis_tw: Optimising to minimize the amount of unwanted data you're moving is a good idea. It doesn't have to relate specifically to how frequently it's updated, though. [09:02am] mcrady: any plans to support stretching of images? [09:02am] ikai_google: dennis_tw: In the past, even outside of App Engine, I've used a design where I do frequent writes to MemCache and flush. Of course, you run the risk of some data loss [09:03am] dennis_tw: cool [09:03am] dennis_tw: I'm also wondering if 'small entities' and 'medium sized entities' have pretty much the same overhead. At what point does it become irrelevant how small the entity is? is there a minimum transfer size? [09:03am] ikai_google: mcrady: Do you mean resizing to a larger image? [09:03am] ryan_google: +1 to ikai and nick's points. i'd also note that entity size, in bytes or number of properties or whatever, generally doesn't matter in these cases, but you definitely want to take chances to coalesce, shard, etc to avoid writing to a single entity (or group)
Re: [appengine-java] UserService and form data
Ikai, Thanks, got it :). Misread your original post. . Too much coffee I guess... Manny On Sun, Jan 31, 2010 at 10:24 AM, Manny S manny.m...@gmail.com wrote: Ikai, Thanks for the suggestion. Problem is I don't want the sign in to be mandatory. Users can post with or without signing in. The form is pretty long and sometimes users see the sign in midway through filling the form and decide to sign in. I guess when the user clicks on sign in I could get whatever the users has filled and cache it in the session? I should be able to do this right? Manny On Sat, Jan 30, 2010 at 12:52 AM, Ikai L (Google) ika...@google.comwrote: What about this: 1. Submit the form to your servlet 2. Check to see if the User is logged in or not 3. If the User is not logged in, serialize the form data and cache it in the session, then redirect the User to a login URL 4. After login, repopulate the form You may also be able to save yourself some steps here by simply requiring login on the form itself. On Thu, Jan 28, 2010 at 10:33 PM, Manny S manny.m...@gmail.com wrote: Question on how to retain form data when the page reloads after user signs in using the userservice - Sample scenario User is entering data into a form User sees sign in button on top right Clicks it and signs in to google account Redirects back to the form page But form data (that the user filled out prior to clicking on sign in) is lost In code I call - userservice.createLoginURL(thisURL) whenever the login URL is clicked. How do I ensure that the form data is retained if the user decides to sign in midway? Any pointers in the right direction would be greatly useful. Thanks, Manny -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] appengine-java-io 0.1.0 released
Congratulations! On Mon, Feb 1, 2010 at 08:49, limcheekin limchee...@vobject.com wrote: Hi, As we known Google App Engine doesn't support Java IO writing operation natively. The VObject AppEngine? Java-IO is an open source project that intend to port gaevfs to java.io API to enable java.io classes support in Google App Engine (GAE) with Metaprogramming capability of the Groovy Programming Language. I would like to announced that appengine-java-io 0.1.0 released on 01 Feb 2010. Please go to http://code.google.com/p/appengine-java-io/wiki/ReleaseNoteZeroDotOneDotZero to view the Release Note. Chee Kin -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Guillaume Laforge Groovy Project Manager Head of Groovy Development at SpringSource http://www.springsource.com/g2one -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] problem with Datastore quotas
Hi all, I have the following problem : my quotas show the following : Total Stored Data26% 0.26 of 1.00 GBytes Okay at the same time Datastore Statistics is showing : Size of all entities: 5 MBytes Breakdown by Property Type Property Type Size Blob 5 MBytes String 14 KBytes Integer 12 KBytes Date/Time4 KBytes User 51 Bytes NULL 24 Bytes Metadata 71 KBytes And I have no idea where is the datastore space disappearing.. my app id is sandcode Does anybody has idea ? I'll appreciate your advices.. Best Regards: Dimitar Makariev -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] problem with Datastore quotas
On the page that shows those details is a link to a doc that explains where the extra space is used. Probably indexes could consume a lot. On 1 Feb 2010, at 16:29, dmakariev wrote: Hi all, I have the following problem : my quotas show the following : Total Stored Data26% 0.26 of 1.00 GBytes Okay at the same time Datastore Statistics is showing : Size of all entities: 5 MBytes Breakdown by Property Type Property Type Size Blob 5 MBytes String 14 KBytes Integer 12 KBytes Date/Time4 KBytes User 51 Bytes NULL 24 Bytes Metadata 71 KBytes And I have no idea where is the datastore space disappearing.. my app id is sandcode Does anybody has idea ? I'll appreciate your advices.. Best Regards: Dimitar Makariev -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] How to monitor restarts of GAE instances
Hi everybody, what kind of monitoring do you use to 1. monitor if an instance restarts. 2. if GAE runs on multiple instances. (A code snipplet would be really cool). I am using GUICE, and I am currently monitoring when the GUICE injector is created - according to my monitoring it is created almost every 2 minutes under load(!) resulting in ultra long loading times and super high CPU consumption. But I want to cross check this - maybe my monitoring is just stupid. Thanks a lot, Raphael -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Check for Entity JDOObjectNotFoundException?
Hi there! Using JDO on GAE/J PersistenceManager pm = PMF.get().getPersistenceManager(); Key key = KeyFactory.createKey(MyEntity.class.getSimpleName(), t...@example.com); MyEntity myEntity = pm.getObjectById(MyEntity.class, key); When there is no entity matching the key, it throws JDOObjectNotFoundException. Is there no other way, than catching the Exception (JDOObjectNotFoundException) - like checking for existence of Entity before calling getObjectById? Regards Chris -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Check for Entity JDOObjectNotFoundException?
Is there no other way, than catching the Exception It's the most efficient way (and JPA does a similar thing, returning null from that method). Obviously you could run a query but then that would have to be constructed and compiled etc. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Questions about JDO/JPA versioning
Thanks for all the answers. On Jan 29, 6:13 pm, datanucleus andy_jeffer...@yahoo.com wrote: Question3: Given that A is annotated with @Version and the version field is protected, should a subclass of A needs to be annotated with @Version too in order to have the version managed by JDO/JPA? No. The field is protected, so is available to subclasses ... -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] jython / modjy FileOutputStream exception
[Riley] I'm trying to get jython running with modjy. Following instructions from http://opensource.xhaus.com/projects/modjy/wiki/ModjyGoogleAppEngine but I get the exceptions below. It looks like python is trying to write an initialization file or something. I know a lot about Java but little about Python. Any ideas? Hmm, that's strange; I'll need more information to figure out what's going on. What version of jython are you using? Are you using the demo modjy app provided with jython? If yes, are you modifying it in anyway, e.g. changing the web.xml, etc? Regards, Alan. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] production memcache grabTail namespace isolation broken
Hi, On production, grabTail returns objects from other namespaces. This is a significant problem that makes grabTail (thus queue-like usage) unusable. I filed this as a bug, with a very simple example here: http://code.google.com/p/googleappengine/issues/detail?id=2706 Can you please look into this? I don't even see a workaround that I can implement in the meantime... Thanks, Viktor -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: GAE For Java with struts2 And I am not able to access development admin console
This occurs with most frameworks that use a filter with a wildcard mapping to dispatch all requests, the Development server's admin Servlets are no longer found, and are instead being processed by the your framework (Struts). filter-mapping filter-nameframeworkDispatcher/filter-name url-pattern/*/url-pattern /filter-mapping I'm unsure on the best way to resolve this - as I'm not from a Java background, I don't know the best way to set up the filter mappings. However, when I required access to the Admin in Development, I simply temporarily commented out the framework's default filter with the wildcard mapping, accessed the Admin, then added the filter again when needing to run the app. Andy On Jan 30, 1:11 pm, MKumar m.kumar1...@gmail.com wrote: Hi, I am using JAVA GAE , Everything is working fine, but I am not able to access the Admin console, which as given in documentation can be accessed usinghttp://localhost:myport/_ah/admin But its giving me following error: HTTP ERROR: 404 There is no Action mapped for namespace /_ah and action name admin. RequestURI=/_ah/admin For your information, I had neither used data storage, or queing etc. in application, But wanted to directly enter some data using admin console. Please give me some idea so I can access the admin console for data entery on development server. Thanks Mrityunjay Kumar -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Object state from a query
Can anyone help, plz? On Feb 1, 3:43 pm, Brandon brandon.s...@gmail.com wrote: Hi, I am using JDO. I know that when querying a class of objects residing in different entity group, the query should not be executed inside a transaction. So, what are the state of the objects in the query result (persistent or detached)? Should they be detached first before used? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Spring MVC - File upload problem
Hi You need the jar file, you can download it from: http://cloud.github.com/downloads/pjesi/springextras/springextras-1.1.jar Hope that helps, Viðar 2010/2/1 java rohit82.j...@gmail.com: Hello I am also using spring with appengine I want to store a image into datastore. I have written same code bean id=multipartResolver class=is.hax.spring.web.multipart.StreamingMultipartResolver /bean But when I compile then I got error java.lang.ClassNotFoundException: is.hax.spring.web.multipart.StreamingMultipartResolver what I should do??? Thanks in advance On Jul 5 2009, 2:06 pm, Viðar Svansson vidarsv...@gmail.com wrote: Hi Here is my configuration using StreamingMultipartResolver in my AppEngine MVC application. In my servlet context file: bean id=multipartResolver class=is.hax.spring.web.multipart.StreamingMultipartResolver /bean Then in my model: private MultipartFile picture; // getters and setters And finally in the contoller: public String uploadPicture(@ModelAttribute(picture) Picture picture) { MultipartFile file = picture.getPicture(); byte[] rawData = file.getBytes(); // ... Hope this helps. Viðar On Thu, Jul 2, 2009 at 11:30 PM, zee zenma...@gmail.com wrote: Is there any example on how to use StreamingMultipartResolver in Spring MVC based web application (actually grails) like sample xml file configuration settings to override default spring behaviour? I am facing the same problem using google app engine because the framework based on Spring MVC uses spring file uploads for multipart form requests. On Jun 25, 5:45 pm, Viðar Svansson vidarsv...@gmail.com wrote: Hi Check outhttp://github.com/pjesi/springextras You can use is.hax.spring.web.multipart.StreamingMultipartResolver I am still working on this so watch for updates. Cheers, Viðar On Mon, Jun 22, 2009 at 5:06 AM, hu itswa...@gmail.com wrote: i got problem when upload file with MVC, and i wanna to know how did u implement this ? On May 13, 5:05 am, pjesi vidarsv...@gmail.com wrote: Hi CommonsMultipartResolver does write to the filesystem, which is not what Google wants. I'm working on a special MultipartResolver for the appengine to resolve this issue. Let me know if you are interested. Cheers Viðar On Apr 20, 9:25 pm, Arjan huij...@gmail.com wrote: Hi all, I am creating a form that allows the user touploadan image. I use Spring MVC. I have configured the mulipart resolved as follows: bean id=multipartResolver class=org.springframework.web.multipart.commons.CommonsMultipartResolver property name=maxUploadSize value=200 / /bean However, this results in an exception because it tries to write a temp file. I know the Commons FileUploadlibrary used support streaming without actually creating a temp file. Can someome tell me how I can configure this in Spring MVC? Thanks in advance! Arjan -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Does precompilation work ?
The next release has a few additional fixes for precompilation stability. However, we also plan to turn precompilation on by default so I'd like to get to the bottom of any failures that you're having (especially if they are reproducible). Can anyone experiencing problems zip up their WAR directory and send it to me privately? I'd like to do some analysis on it. On Sun, Jan 31, 2010 at 10:02 PM, Peter Liu tinyee...@gmail.com wrote: In my experience deployment sometimes fail with variety of reasons. Sometimes it says precompiling starts and eventually time out. The console will suggest you to turn off precompilation, but if you try deploy again, it will work. My suggestion is keep trying. On Jan 30, 11:46 am, tomekc to...@ceszke.com wrote: Hi All, does precompilation work for you ? I'm trying a few times deploy with precompilation but still get an exception: java.io.IOException: Precompilation failed. Consider adding precompilation-enabledfalse/precompilation-enabled to your appengine-web.xml and trying again. Unable to update app: Precompilation failed. Consider adding precompilation-enabledfalse/precompilation-enabled to your appengine-web.xml and trying again. Please see the logs [/var/folders/XC/XCZRRBVPHbSTigao7dR+ek+++TI/-Tmp-/ appcfg8627681542400368916.log] for further information. in that log I have: Unable to update: java.io.IOException: Precompilation failed. Consider adding precompilation-enabledfalse/precompilation-enabled to your appengine-web.xml and trying again. ...cut... Caused by: java.io.IOException: Error posting to URL:http://appengine.google.com/api/appversion/precompile?app_id=Xve... 503 Service Unavailable Try Again (503) An unexpected failure has occurred. Please try again. at com.google.appengine.tools.admin.ServerConnection.send (ServerConnection.java:143) ...cut... 503 Service Unavailable Try Again (503) ...cut I ask because maybe it is temporary problem ? Regards, tomek. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Building Scalable Complex App
Please see the answer i sent out yesterday for a proper solution. Duong BaTien DBGROUPS and BudhNet On Sat, 2010-01-30 at 01:14 -0800, digitalsam007 wrote: I failed to make this code work. It seems that the MessageIndex object is not getting persisted when i call tx.commit(). Any help in this regard will be appreciated. Thanks On Jan 17, 12:26 am, Duong BaTien duong.bat...@gmail.com wrote: Hi: Following is my sketch for the Message and MessageIndex. Please advise and if possible, sketch out the Java version of set intersection from the presentation. Have not time to try anything yet; but the design is very significant for proper data modeling. The JDO version of the presentation uses query.setFilter(receivers == 'foo') which i do not see in the API. public class MessageIndex { // create MessageIndex to the parent Message private Key parent; private String keyKind; private long id; @PrimaryKey @Persistent(ValueStrategy = IdGeneratorStrategy.IDENTITY) private KeyFactory.createKey(parent, keyKind, id) key; @Persistent private ListString receivers; // constructor to create MessageIndex linked to Message public MessageIndex(Key parent, String keyKind, long id) { this.parent = parent; this.keyKind = keyKind; this id = id; } public class Message { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key key @Persistent private String sender; @Persistent private String body; // a user create a message to a list of receivers public static Message createMessage(String userId, String body, ListString receivers) { tx.begin() this.body = body; MessageIndex index = new MessageIndex(key, key,getKind(), key.getId ()); index.setReceivers(receivers); tx.commit(); } // query to get all messages sent to a user foo public ListMessage getMessagesSentToUserId(String userId) { ListMessage results = null tx.begin(); Query q = pm.newQuery(MessageIndex.class); q.addFilter(receivers, Query.FilterOperator.EQUAL, foo); q.setKeyOnly(); // query the Message for all messages filtered by parent of MessageIndex key Query query = pm.newQuery(Message.class); query.addFilter(key, Query.FilterOperator.EQUAL, q.execute ().getParent()); results = (ListMessage) query.execute(); tx.commit(); } } Thanks Duong BaTien DBGROUPS and BudhNet On Wed, 2010-01-13 at 16:26 -0800, Jason (Google) wrote: It's been awhile since I've seen this video, so I'll have to go back to re-watch it in order to answer your question completely, but I'd be interested in seeing what you've sketched out so far and happy to give you my feedback on it. - Jason On Tue, Jan 12, 2010 at 3:52 PM, Duong BaTien duong.bat...@gmail.com wrote: Hi Jason, the author Brett Slatkin and others: 1) For the list property, the video shows simple example of standard 1:n Message Receivers in python and Java. It then demonstrates a very efficient technique of Message and MessageIndex with the query of MessageIndex in python (NO Java from then on). I look at both the low level datastore with Query.setKeyOnly() and the query select id from to get the results of ListKey, then go through the loop to get the receivers. I may miss something, since the python code looks so clean. But once you are in any environment, it is very expensive to switch. This technique has major impact in data modeling. 2) Again for the set intersection, the python code looks so clean. I roughly sketch what need to be done on the Java side with either low level of datastore or with JDO. It is still not elegant as the shown codes. The demos are also in python. Again, the technique of merge join will have very significant impact on data modeling that GAE may surpass other platforms. Haven't put enough time in it yet, I wonder if any one has worked out the demos in Java and suggested best practices in data modeling for a complex large application. The job Brett Slatkin sets out to do has not completed without the Java codes since GAE now supports Java. Thanks Duong BaTien DBGROUPS and BudhNet On Tue, 2010-01-12 at 14:13 -0800, Jason (Google) wrote: Can you be more specific about what you're trying to accomplish so we don't have to search through the video to find what you're looking for? Or, at least provide a
[appengine-java] Re: Object state from a query
JDOHelper.getObjectState(obj) ? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Templating a web site
Hi, I am new to Google app engine, and willing to move from asp.net to Google App Engine / Java. While I have big experience in asp.net, c# i would like to learn GAE. My Qs: 1. How can I use page templates / equivalent to master pages in asp.net ? 2. How can I use multi-lingual texts in a an app. Can you suggest some eBooks that explain the issues above, or/and a good guide for an experienced developer to learn GAE? Thanks in advance, Victor. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] How to build a java desktop client for my hosted app engine application ?
How to build a java desktop client for my hosted app engine application ? I have tried to write a code using google client login using that I could authenticate google account and got auth token. but how should I use this token to authenticate my app engine application thanks Mahendra -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: problem with Datastore quotas
Hi again, I've read the documentation. and cannot find anything related to my datastore growth :(, I'm not using indexes. It is quite simple application, demonstrating the usage of JSF 2. I'm using session, and I've included the SessionCleanupServlet in my web.xml. It is started every hour. So the number of entities in _ah_SESSION is round 40 . My other suspect was the log file. I couldn't find any description about where the log files are stored. But I've updated the version number of the application.. deleted the old one with the big log files, and still the total used datastore quota didn't decrease. my current status is 35% of Total Stored Data and at the same time : Size of all entities 414 KBytes I'll be really happy to know where and how to manage properly the datastore space. Best Regards: Dimitar Makariev On Feb 1, 1:38 am, John Patterson jdpatter...@gmail.com wrote: On the page that shows those details is a link to a doc that explains where the extra space is used. Probably indexes could consume a lot. On 1 Feb 2010, at 16:29, dmakariev wrote: Hi all, I have the following problem : my quotas show the following : Total Stored Data 26% 0.26 of 1.00 GBytes Okay at the same time Datastore Statistics is showing : Size of all entities: 5 MBytes Breakdown by Property Type Property Type Size Blob 5 MBytes String 14 KBytes Integer 12 KBytes Date/Time 4 KBytes User 51 Bytes NULL 24 Bytes Metadata 71 KBytes And I have no idea where is the datastore space disappearing.. my app id is sandcode Does anybody has idea ? I'll appreciate your advices.. Best Regards: Dimitar Makariev -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] NPE compiling a JSP
Hi, my application deploys fine in the local environment but when trying to upload to AppEngine I get this stack trace [1] after compiling about 30 JSP files. The page with the bug is not being logged anywhere (the last one, detail.jsp, compiles OK). I had to guess based on a 'ls -U' order of folders in the filesystem, which took me a while to guess. This is just a heads-up, because a way to increase the verbosity of jspC (adding the -v flag or similar) would be really welcome. Maybe there is one, but I could not find it. Of course, fixing the NPE would also be great :) === [1] (list of previous JSP files omitted...) 19:43:44,908 INFO [JspC] Built File: /WEB-INF/jsp/activity-log/ list.jsp 19:43:44,931 INFO [JspC] Built File: /WEB-INF/jsp/activity-log/ detail.jsp java.lang.NullPointerException at org.apache.jasper.compiler.Validator$ValidateVisitor.getMethod (Validator.java:1269) at org.apache.jasper.compiler.Validator $ValidateVisitor.processSignature(Validator.java:1257) at org.apache.jasper.compiler.Validator$ValidateVisitor.access$400 (Validator.java:295) at org.apache.jasper.compiler.Validator$1FVVisitor.visit (Validator.java:1248) at org.apache.jasper.compiler.ELNode$Function.accept(ELNode.java:122) at org.apache.jasper.compiler.ELNode$Nodes.visit(ELNode.java:193) at org.apache.jasper.compiler.ELNode$Visitor.visit(ELNode.java:234) at org.apache.jasper.compiler.ELNode$Root.accept(ELNode.java:53) at org.apache.jasper.compiler.ELNode$Nodes.visit(ELNode.java:193) at org.apache.jasper.compiler.Validator $ValidateVisitor.validateFunctions(Validator.java:1252) at org.apache.jasper.compiler.Validator$ValidateVisitor.visit (Validator.java:593) at org.apache.jasper.compiler.Node$ELExpression.accept(Node.java:896) at org.apache.jasper.compiler.Node$Nodes.visit(Node.java:2163) at org.apache.jasper.compiler.Node$Visitor.visitBody(Node.java:2213) at org.apache.jasper.compiler.Validator$ValidateVisitor.visit (Validator.java:564) at org.apache.jasper.compiler.Node$NamedAttribute.accept(Node.java: 1810) at org.apache.jasper.compiler.Node$Nodes.visit(Node.java:2163) at org.apache.jasper.compiler.Node$Visitor.visitBody(Node.java:2213) at org.apache.jasper.compiler.Validator$ValidateVisitor.visit (Validator.java:716) at org.apache.jasper.compiler.Node$CustomTag.accept(Node.java:1441) at org.apache.jasper.compiler.Node$Nodes.visit(Node.java:2163) at org.apache.jasper.compiler.Node$Visitor.visitBody(Node.java:2213) at org.apache.jasper.compiler.Node$Visitor.visit(Node.java:2219) at org.apache.jasper.compiler.Node$Root.accept(Node.java:456) at org.apache.jasper.compiler.Node$Nodes.visit(Node.java:2163) at org.apache.jasper.compiler.Validator.validate(Validator.java:1475) at org.apache.jasper.compiler.Compiler.generateJava(Compiler.java: 214) at org.apache.jasper.compiler.Compiler.compile(Compiler.java:470) at org.apache.jasper.JspC.processFile(JspC.java:779) at org.apache.jasper.JspC.execute(JspC.java:908) at com.google.appengine.tools.development.LocalJspC.main (LocalJspC.java:17) Exception in thread main org.apache.jasper.JasperException: java.lang.NullPointerException at org.apache.jasper.JspC.processFile(JspC.java:806) at org.apache.jasper.JspC.execute(JspC.java:908) at com.google.appengine.tools.development.LocalJspC.main (LocalJspC.java:17) Caused by: java.lang.NullPointerException at org.apache.jasper.compiler.Validator$ValidateVisitor.getMethod (Validator.java:1269) at org.apache.jasper.compiler.Validator $ValidateVisitor.processSignature(Validator.java:1257) at org.apache.jasper.compiler.Validator$ValidateVisitor.access$400 (Validator.java:295) at org.apache.jasper.compiler.Validator$1FVVisitor.visit (Validator.java:1248) at org.apache.jasper.compiler.ELNode$Function.accept(ELNode.java:122) at org.apache.jasper.compiler.ELNode$Nodes.visit(ELNode.java:193) at org.apache.jasper.compiler.ELNode$Visitor.visit(ELNode.java:234) at org.apache.jasper.compiler.ELNode$Root.accept(ELNode.java:53) at org.apache.jasper.compiler.ELNode$Nodes.visit(ELNode.java:193) at org.apache.jasper.compiler.Validator $ValidateVisitor.validateFunctions(Validator.java:1252) at org.apache.jasper.compiler.Validator$ValidateVisitor.visit (Validator.java:593) at org.apache.jasper.compiler.Node$ELExpression.accept(Node.java:896) at org.apache.jasper.compiler.Node$Nodes.visit(Node.java:2163) at org.apache.jasper.compiler.Node$Visitor.visitBody(Node.java:2213) at org.apache.jasper.compiler.Validator$ValidateVisitor.visit (Validator.java:564) at org.apache.jasper.compiler.Node$NamedAttribute.accept(Node.java: 1810) at
Re: [appengine-java] Secure JSP
You can do this with deployment descriptions or servlet filters. Here documentation about configuring web.xml to do this: http://code.google.com/appengine/docs/java/config/webxml.html On Sun, Jan 31, 2010 at 2:20 PM, Scott gocards...@gmail.com wrote: I have 1 jsp in my app that I want to secure and make people login with their google accounts in order to see the page. Is this possible and if so how? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Does precompilation work ?
Thanks for your reply. Unfortunately, the problem still occurs. The same error desription: 503 Service Unavailable... If precompilation is set to false then application is deployed without any problems. On Feb 1, 5:02 am, Peter Liu tinyee...@gmail.com wrote: In my experience deployment sometimes fail with variety of reasons. Sometimes it says precompiling starts and eventually time out. The console will suggest you to turn off precompilation, but if you try deploy again, it will work. My suggestion is keep trying. On Jan 30, 11:46 am, tomekc to...@ceszke.com wrote: Hi All, does precompilation work for you ? I'm trying a few times deploy with precompilation but still get an exception: java.io.IOException: Precompilation failed. Consider adding precompilation-enabledfalse/precompilation-enabled to your appengine-web.xml and trying again. Unable to update app: Precompilation failed. Consider adding precompilation-enabledfalse/precompilation-enabled to your appengine-web.xml and trying again. Please see the logs [/var/folders/XC/XCZRRBVPHbSTigao7dR+ek+++TI/-Tmp-/ appcfg8627681542400368916.log] for further information. in that log I have: Unable to update: java.io.IOException: Precompilation failed. Consider adding precompilation-enabledfalse/precompilation-enabled to your appengine-web.xml and trying again. ...cut... Caused by: java.io.IOException: Error posting to URL:http://appengine.google.com/api/appversion/precompile?app_id=Xve... 503 Service Unavailable Try Again (503) An unexpected failure has occurred. Please try again. at com.google.appengine.tools.admin.ServerConnection.send (ServerConnection.java:143) ...cut... 503 Service Unavailable Try Again (503) ...cut I ask because maybe it is temporary problem ? Regards, tomek. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: NPE compiling a JSP
For the record, one of the functions defined in the TLD was missing the function-class and function-signature, and that was causing the NPE. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] production memcache grabTail namespace isolation broken
I'll raise the issue with some other members of the team, but it seems like this is working as expected. A memcache namespace is nothing more than a prefix applied to a memcache key. There's no true partitioning mechanism within memcache. grabTail simply returns the item that would be expired by the LRU mechanism if memcache needed more space. There isn't a different queue per namespace, only a queue for global expirations. On Mon, Feb 1, 2010 at 4:29 AM, phraktle phrak...@gmail.com wrote: Hi, On production, grabTail returns objects from other namespaces. This is a significant problem that makes grabTail (thus queue-like usage) unusable. I filed this as a bug, with a very simple example here: http://code.google.com/p/googleappengine/issues/detail?id=2706 Can you please look into this? I don't even see a workaround that I can implement in the meantime... Thanks, Viktor -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] memcache SerializationException
I have hit the wall in trying to figure out what I am doing wrong in my GAE GWT application. I am trying to use memcache to get a list of serializable objects out of the cache in a service. It seems to put them fine, and I can see them being pulled from cache in the debugger but I end up getting the following exception: This is the call to pull the object out of cache: ListModel ecDevices = (ListModel) cache.findInCache(partitionKey); if(ecDevices != null ecDevices.size() 0){ log.info(Returning devices from cache); return ecDevices; }... SEVERE: [1265057364142000] javax.servlet.ServletContext log: Exception while dispatching incoming RPC call com.google.gwt.user.client.rpc.SerializationException: java.lang.reflect.InvocationTargetException at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamWriter.serializeWithCustomSerializer (ServerSerializationStreamWriter.java:760) at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamWriter.serializeImpl (ServerSerializationStreamWriter.java:723) at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamWriter.serialize (ServerSerializationStreamWriter.java:612) at com.google.gwt.user.client.rpc.impl.AbstractSerializationStreamWriter.writeObject (AbstractSerializationStreamWriter.java:129) at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamWriter $ValueWriter$8.write(ServerSerializationStreamWriter.java:152) at com.google.gwt.user.server.rpc.impl.ServerSerializationStreamWriter.serializeValue (ServerSerializationStreamWriter.java:534) at com.google.gwt.user.server.rpc.RPC.encodeResponse(RPC.java:609) at com.google.gwt.user.server.rpc.RPC.encodeResponseForSuccess (RPC.java:467) at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse (RPC.java:564) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall (RemoteServiceServlet.java:188) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost (RemoteServiceServlet.java:224) at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost (AbstractRemoteServiceServlet.java:62) at javax.servlet.http.HttpServlet.service(HttpServlet.java:713) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: 487) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1093) at com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter (ServeBlobFilter.java:51) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1084) at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter (TransactionCleanupFilter.java:43) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1084) at com.google.appengine.tools.development.StaticFileFilter.doFilter (StaticFileFilter.java:121) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1084) at org.mortbay.jetty.servlet.ServletHandler.handle (ServletHandler.java:360) at org.mortbay.jetty.security.SecurityHandler.handle (SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle (SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle (ContextHandler.java:712) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java: 405) at com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle (DevAppEngineWebAppContext.java:70) at org.mortbay.jetty.handler.HandlerWrapper.handle (HandlerWrapper.java:139) at com.google.appengine.tools.development.JettyContainerService $ApiProxyHandler.handle(JettyContainerService.java:352) at org.mortbay.jetty.handler.HandlerWrapper.handle (HandlerWrapper.java:139) at org.mortbay.jetty.Server.handle(Server.java:313) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java: 506) at org.mortbay.jetty.HttpConnection$RequestHandler.content (HttpConnection.java:844) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:644) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381) at org.mortbay.io.nio.SelectChannelEndPoint.run (SelectChannelEndPoint.java:396) at org.mortbay.thread.BoundedThreadPool$PoolThread.run (BoundedThreadPool.java:442) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:25)
Re: [appengine-java] change timezone of date that is already persisted
The way to do this is to create a series of tasks that update each entity one by one. The question I'd like to ask, however, is why you'd want to do this. It's typically best practice to store time information as UTC and convert as needed for display or computation due to weirdness with timezones, daylight saving time, and so forth. Your data will be much easier to internationalize if it's timezone/locale independent. On Thu, Jan 28, 2010 at 5:46 PM, mar_novice mariocape1...@gmail.com wrote: Hi, Is there a way to change the timezone of the whole app and also change the timezone of the dates that were already in the datastore from UTC to a specific timezone? thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Log4j in production GAE
Hello, Has anyone tried to use Log4j in production GAE? Is it supported? According to Java GAE docs, only java.util.logging is fully supported. See http://code.google.com/appengine/docs/java/runtime.html#Logging [...] Any logging framework (such as log4j) that logs to the output or error streams will work. However, for more fine-grained control of the Admin Console's log level display, the logging framework must use a java.util.logging adapter. [...] On the other hand, I see that log4j.properties is included into every project created by GAE Eclipse plugin. I tried Log4j in a test application and Lo4j is working fine in development environment. But it doesn't (just does nothing) when I deploy my test application into live GAE server. Thanks in advance for any info, Alex -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Spring MVC - File upload problem
this works also for grails! Add the library to your lib directory and add bean id=multipartResolver class=is.hax.spring.web.multipart.StreamingMultipartResolver /bean to your applicationContext.xml For saving the image in a google blob i used the following functions: @Persistent Blob imageBlob byte [] getImage(){ if(imageBlob){ imageBlob.getBytes() }else{ null; } } void setImage(byte [] imageBytes){ imageBlob = new Blob(imageBytes) } -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Log4j in production GAE
I use slf4j! use slf4j-log4j when development! use slf4j-jdk log when GAE Server!(change slf4j-log4j to slf4j-jdk-log before GAE appcfg update war) 2010/2/2 Alex alexanderko...@gmail.com Hello, Has anyone tried to use Log4j in production GAE? Is it supported? According to Java GAE docs, only java.util.logging is fully supported. See http://code.google.com/appengine/docs/java/runtime.html#Logging [...] Any logging framework (such as log4j) that logs to the output or error streams will work. However, for more fine-grained control of the Admin Console's log level display, the logging framework must use a java.util.logging adapter. [...] On the other hand, I see that log4j.properties is included into every project created by GAE Eclipse plugin. I tried Log4j in a test application and Lo4j is working fine in development environment. But it doesn't (just does nothing) when I deploy my test application into live GAE server. Thanks in advance for any info, Alex -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Log4j in production GAE
I also vote for slf4j. If you need a for example, here's my logback.xml file, which lives in the WEB-INF/classes directory: ?xml version=1.0 encoding=UTF-8? !-- pattern%date{-MM-dd HH:mm:ss.SSS z}, %5level: [%thread] %class.%method.%line: %message%n/pattern -- configuration debug=true appender class=ch.qos.logback.core.ConsoleAppender name=RootConsoleAppender layout class=ch.qos.logback.classic.PatternLayout pattern%5level: [%thread] %class.%method.%line: %message%n/pattern /layout /appender logger name=org.springframework.beans level value=warn / /logger logger name=org.springframework.transaction level value=debug / /logger logger name=org.springframework.orm.jdo.JdoTransactionManager level value=debug / /logger !-- logger name=com.google.appengine.repackaged.com.google.common.base.FinalizableReferenceQueue level value=warn / /logger -- root level value=info / appender-ref ref=RootConsoleAppender / /root /configuration ?? wrote: I use slf4j! use slf4j-log4j when development! use slf4j-jdk log when GAE Server!(change slf4j-log4j to slf4j-jdk-log before GAE appcfg update war) 2010/2/2 Alex alexanderko...@gmail.com mailto:alexanderko...@gmail.com Hello, Has anyone tried to use Log4j in production GAE? Is it supported? According to Java GAE docs, only java.util.logging is fully supported. See http://code.google.com/appengine/docs/java/runtime.html#Logging [...] Any logging framework (such as log4j) that logs to the output or error streams will work. However, for more fine-grained control of the Admin Console's log level display, the logging framework must use a java.util.logging adapter. [...] On the other hand, I see that log4j.properties is included into every project created by GAE Eclipse plugin. I tried Log4j in a test application and Lo4j is working fine in development environment. But it doesn't (just does nothing) when I deploy my test application into live GAE server. Thanks in advance for any info, Alex -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- 0x2B | ~0x2b -- Hamlet -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Log4j in production GAE
What I just wrote is probably confusing; I'm using slf4j with logback for logging, not log4j. You don't need a logback.xml configuration file if you're using slf4j with log4j (but you will need a log4j configuration file). I prefer logback because with tomcat you can never be sure that any problems with the logging aren't being caused by tomcat's logging (but since gae isn't using tomcat that shouldn't be an issue here). Also, logback is more actively supported (and it's written by the same guy who wrote log4j). Rusty Wright wrote: I also vote for slf4j. If you need a for example, here's my logback.xml file, which lives in the WEB-INF/classes directory: ?xml version=1.0 encoding=UTF-8? !-- pattern%date{-MM-dd HH:mm:ss.SSS z}, %5level: [%thread] %class.%method.%line: %message%n/pattern -- configuration debug=true appender class=ch.qos.logback.core.ConsoleAppender name=RootConsoleAppender layout class=ch.qos.logback.classic.PatternLayout pattern%5level: [%thread] %class.%method.%line: %message%n/pattern /layout /appender logger name=org.springframework.beans level value=warn / /logger logger name=org.springframework.transaction level value=debug / /logger logger name=org.springframework.orm.jdo.JdoTransactionManager level value=debug / /logger !-- logger name=com.google.appengine.repackaged.com.google.common.base.FinalizableReferenceQueue level value=warn / /logger -- root level value=info / appender-ref ref=RootConsoleAppender / /root /configuration ?? wrote: I use slf4j! use slf4j-log4j when development! use slf4j-jdk log when GAE Server!(change slf4j-log4j to slf4j-jdk-log before GAE appcfg update war) 2010/2/2 Alex alexanderko...@gmail.com mailto:alexanderko...@gmail.com Hello, Has anyone tried to use Log4j in production GAE? Is it supported? According to Java GAE docs, only java.util.logging is fully supported. See http://code.google.com/appengine/docs/java/runtime.html#Logging [...] Any logging framework (such as log4j) that logs to the output or error streams will work. However, for more fine-grained control of the Admin Console's log level display, the logging framework must use a java.util.logging adapter. [...] On the other hand, I see that log4j.properties is included into every project created by GAE Eclipse plugin. I tried Log4j in a test application and Lo4j is working fine in development environment. But it doesn't (just does nothing) when I deploy my test application into live GAE server. Thanks in advance for any info, Alex -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- 0x2B | ~0x2b -- Hamlet -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Log4j in production GAE
Oops; more stumbling. I think it's usually Commons Logging that causes problems with tomcat, and perhaps elsewhere, not log4j. slf4j is a Commons Logging alternative. Rusty Wright wrote: What I just wrote is probably confusing; I'm using slf4j with logback for logging, not log4j. You don't need a logback.xml configuration file if you're using slf4j with log4j (but you will need a log4j configuration file). I prefer logback because with tomcat you can never be sure that any problems with the logging aren't being caused by tomcat's logging (but since gae isn't using tomcat that shouldn't be an issue here). Also, logback is more actively supported (and it's written by the same guy who wrote log4j). Rusty Wright wrote: I also vote for slf4j. If you need a for example, here's my logback.xml file, which lives in the WEB-INF/classes directory: ?xml version=1.0 encoding=UTF-8? !-- pattern%date{-MM-dd HH:mm:ss.SSS z}, %5level: [%thread] %class.%method.%line: %message%n/pattern -- configuration debug=true appender class=ch.qos.logback.core.ConsoleAppender name=RootConsoleAppender layout class=ch.qos.logback.classic.PatternLayout pattern%5level: [%thread] %class.%method.%line: %message%n/pattern /layout /appender logger name=org.springframework.beans level value=warn / /logger logger name=org.springframework.transaction level value=debug / /logger logger name=org.springframework.orm.jdo.JdoTransactionManager level value=debug / /logger !-- logger name=com.google.appengine.repackaged.com.google.common.base.FinalizableReferenceQueue level value=warn / /logger -- root level value=info / appender-ref ref=RootConsoleAppender / /root /configuration ?? wrote: I use slf4j! use slf4j-log4j when development! use slf4j-jdk log when GAE Server!(change slf4j-log4j to slf4j-jdk-log before GAE appcfg update war) 2010/2/2 Alex alexanderko...@gmail.com mailto:alexanderko...@gmail.com Hello, Has anyone tried to use Log4j in production GAE? Is it supported? According to Java GAE docs, only java.util.logging is fully supported. See http://code.google.com/appengine/docs/java/runtime.html#Logging [...] Any logging framework (such as log4j) that logs to the output or error streams will work. However, for more fine-grained control of the Admin Console's log level display, the logging framework must use a java.util.logging adapter. [...] On the other hand, I see that log4j.properties is included into every project created by GAE Eclipse plugin. I tried Log4j in a test application and Lo4j is working fine in development environment. But it doesn't (just does nothing) when I deploy my test application into live GAE server. Thanks in advance for any info, Alex -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- 0x2B | ~0x2b -- Hamlet -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Struts Validator not working.,
Am sure this is something that is known to someone in the community to tell me what am doing wrong. I have the struts validator framework defined. Appengine compiles, but when am trying to run the app in my local-host; The request dispatcher object in validator framework is comin back as a null. Am I missing something, any guidence or insights??? java.lang.NullPointerException at org.apache.struts.chain.commands.servlet.PerformForward.handleAsForward (PerformForward.java:113) at org.apache.struts.chain.commands.servlet.PerformForward.perform (PerformForward.java:96) at org.apache.struts.chain.commands.AbstractPerformForward.execute (AbstractPerformForward.java:54) at org.apache.struts.chain.commands.ActionCommandBase.execute (ActionCommandBase.java:51) at org.apache.commons.chain.impl.ChainBase.execute(ChainBase.java: 190) at org.apache.commons.chain.generic.LookupCommand.execute (LookupCommand.java:304) at org.apache.commons.chain.impl.ChainBase.execute(ChainBase.java: 190) at org.apache.struts.chain.ComposableRequestProcessor.process (ComposableRequestProcessor.java:283) at org.apache.struts.action.ActionServlet.process(ActionServlet.java: 1913) at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java: 449) at javax.servlet.http.HttpServlet.service(HttpServlet.java:693) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: 487) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1093) at com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter (ServeBlobFilter.java:51) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1084) at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter (TransactionCleanupFilter.java:43) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1084) at com.google.appengine.tools.development.StaticFileFilter.doFilter (StaticFileFilter.java:121) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter (ServletHandler.java:1084) at org.mortbay.jetty.servlet.ServletHandler.handle (ServletHandler.java:360) at org.mortbay.jetty.security.SecurityHandler.handle (SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle (SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle (ContextHandler.java:712) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java: 405) at com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle (DevAppEngineWebAppContext.java:70) at org.mortbay.jetty.handler.HandlerWrapper.handle (HandlerWrapper.java:139) at com.google.appengine.tools.development.JettyContainerService $ApiProxyHandler.handle(JettyContainerService.java:352) at org.mortbay.jetty.handler.HandlerWrapper.handle (HandlerWrapper.java:139) at org.mortbay.jetty.Server.handle(Server.java:313) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java: 506) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete (HttpConnection.java:830) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:514) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381) at org.mortbay.io.nio.SelectChannelEndPoint.run (SelectChannelEndPoint.java:396) at org.mortbay.thread.BoundedThreadPool$PoolThread.run (BoundedThreadPool.java:442) -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Spring MVC - File upload problem
Hi I tried it with Grails 1.2.0 and app-engine plugin 0.8.8 but got just an another error: java.lang.NoClassDefFoundError: Could not initialize class org.apache.commons.fileupload.disk.DiskFileItem at org.apache.commons.fileupload.disk.DiskFileItemFactory.createItem (DiskFileItemFactory.java:196) at org.apache.commons.fileupload.FileUploadBase.parseRequest (FileUploadBase.java:358) at org.apache.commons.fileupload.servlet.ServletFileUpload.parseRequest (ServletFileUpload.java:126) at org.springframework.web.multipart.commons.CommonsMultipartResolver.parseRequest (CommonsMultipartResolver.java:155) my controller: def imageInstance = new Image(imageParams) def f = request.getFile('imageData') imageInstance.imageData = f.getBytes() Image.withTransaction { if(imageInstance.save(flush:true)) { flash.message = Image ${imageInstance.id} created redirect(action:show,id:imageInstance.id) } else { render(view:'create',model:[imageInstance:imageInstance]) } } and domain-class: import javax.persistence.*; // import com.google.appengine.api.datastore.Key; @Entity class Image implements Serializable { @Id @GeneratedValue(strategy = GenerationType.IDENTITY) Long id byte[] imageData static constraints = { id visible:false } } It seems like the multipart resolver is not replaced with the new one. I placed the jar file into myApplication/lib and GRAILS_HOME/lib directories. Maybe I didn't place the jar to the right directory? Would you like to tell more about how you got it to work? Markus On 2 helmi, 00:00, Sebastian Cartier sebi.cart...@gmail.com wrote: this works also for grails! Add the library to your lib directory and add bean id=multipartResolver class=is.hax.spring.web.multipart.StreamingMultipartResolver /bean to your applicationContext.xml For saving the image in a google blob i used the following functions: @Persistent Blob imageBlob byte [] getImage(){ if(imageBlob){ imageBlob.getBytes() }else{ null; } } void setImage(byte [] imageBytes){ imageBlob = new Blob(imageBytes) } -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] queue task timeout
I'm using queued tasks to fetch feeds, one feed per task. each task takes about a couple of seconds. but, the tasks get unexpected timeout easily with the following error message. any idea how task timeout happens? is it possible to control the timeout period? thanks, Feb 2, 2010 7:00:54 AM org.quartz.core.JobRunShell run SEVERE: Job feed-fetch.task4 threw an unhandled Exception: com.google.apphosting.api.ApiProxy$ApplicationException: ApplicationError: 2: Received exception executing http method POST against URL http://127.0.0.1:/task: Timeout waiting for connection at com.google.appengine.api.urlfetch.dev.LocalURLFetchService.fetch(LocalURLFetchService.java:223) at com.google.appengine.api.labs.taskqueue.dev.UrlFetchJob.execute(UrlFetchJob.java:53) at com.google.appengine.api.labs.taskqueue.dev.UrlFetchJob.execute(UrlFetchJob.java:46) at org.quartz.core.JobRunShell.run(JobRunShell.java:195) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:520) -aj-- AJ Chen, PhD Chair, Semantic Web SIG, sdforum.org http://web2express.org @web2express on twitter Palo Alto, CA, USA 650-283-4091 *Monitoring social media in real time* -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[google-appengine] Chat Time transcript for January 20, 2010
Two Wednesday back, the App Engine team hosted the latest session of its bimonthly IRC office hours. A transcript of the session and a summary of the topics covered is provided below. The next session will take place this coming Wednesday, February 3rd from 7:00-8:00 p.m. PST in the #appengine channel on irc.freenode.net. - Jason --SUMMARY--- - Q: Should you split a single entity into two or more entities if one or more properties are updated frequently, e.g. per request? A: No, but if you expect to update a single entity or entity group more than once per second, you should consider sharding to avoid datastore contention (http://code.google.com/appengine/articles/scaling/ contention.html). You generally don't need to split entities, although you can do so to save bandwidth if your entities contain several large properties that only need to be accessed occasionally. [9:01-9:06] - Q: Does the 30-second request deadline applied during Blobstore uploads? A: No. In general, time spent by the user doesn't count towards your execution time limit. [9:07-9:08] - Discussion on the maximum length of task queue parameters. [9:10-9:12] - Discussion on bulkloader dump and restore, whether there are any plans to allow for clearing a single datastore kind only, and whether tasks or remote_api is the best mechanism for clearing the datastore [9:16-9:20, 9:24-9:25] - Q: How can one determine the string-encoded key of a given entity for another application, e.g. when updating foreign keys ahead of a bulk upload to another datastore? A: This could change going forward, but for now you can supply the '_app_id_namespace' argument to db.Key.from_path. Or instead of using string-encoded keys, you can use Key.to_xml which provides a transparent encoding. Bear in mind that App Engine currently doesn't allow for referencing entities in other applications, though this could change in the next release. [9:24-9:25, 9:27-9:28, 9:32, 9:35-9:36] - Discussion on whether, when building a master-detail relationship (e.g. orders and items), it's better to store all order information in a single entity or to use an individual entity for each item in the order. [9:27-9:35] - Q: If you're new to App Engine, which runtime should you start with? A: Pick the runtime language most familiar to you to minimize the amount of new learning that you have to do all at once -- even though many use standard APIs, App Engine's various services will take time to master, especially the datastore. [9:35-9:38] - Q: For list properties, is the name of the property stored for each item (e.g. N times for N items) even when its unindexed? A: Yes -- the datastore is schemaless, so a list foo=[bar,baz] is actually stored as two properties with the same name: foo=bar and foo=baz. Hence, each item in the list requires the name to be stored with it. See the article at http://code.google.com/appengine/articles/storage_breakdown.html which explains exactly what gets stored for all entity and index rows. [9:40-9:43, 9:52-9:56, 9:59-10:01] - Discussion of deferred.defer, its ability to schedule tasks to be executed during off-peak hours, and its relation to the core product. [9:41-9:46, 9:50] --FULL TRANSCRIPT--- [09:00am] nickjohnson: Welcome to the fortnightly App Engine developer chat! With me today are Jason and Ikai. [09:00am] nickjohnson: And Ryan [09:00am] ikai_google: Hi everybody! [09:00am] ryan_google: and me! (...something) [09:01am] dennis_tw: I have a question about entities that are changed often (eg: changed on every user action. Like modification time or view count). I'm thinking of separating them out into a separate entity so that that 'high frequency' entity is small and can be moved between the datastore and my app with little overhead. Is that a reasonable entity design? [09:01am] ikai_google: *crickets* [09:02am] nickjohnson: dennis_tw: Optimising to minimize the amount of unwanted data you're moving is a good idea. It doesn't have to relate specifically to how frequently it's updated, though. [09:02am] mcrady: any plans to support stretching of images? [09:02am] ikai_google: dennis_tw: In the past, even outside of App Engine, I've used a design where I do frequent writes to MemCache and flush. Of course, you run the risk of some data loss [09:03am] dennis_tw: cool [09:03am] dennis_tw: I'm also wondering if 'small entities' and 'medium sized entities' have pretty much the same overhead. At what point does it become irrelevant how small the entity is? is there a minimum transfer size? [09:03am] ikai_google: mcrady: Do you mean resizing to a larger image? [09:03am] ryan_google: +1 to ikai and nick's points. i'd also note that entity size, in bytes or number of properties or whatever, generally doesn't matter in these cases, but you definitely want to take chances to coalesce, shard, etc to avoid writing to a single entity (or group)
[google-appengine] User is online/offline
Hello, I seek a best way for know if my user is online and when he is disconnected You have an idea? Thanks! Sahid -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] ancestor/subquery
well someboy can help me. we have tow entity A is parent of B we can perform a filter over B and get in my ej. 3000 ancestor´s keys how can performan other sub query based in this 3000k to apply second filter but not using a For Each and re-query one by one. many thanks alberto -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Best practice for caching frequent datastore reads
Hi all, I am thinking of away to imlement a generic way for my frequently readed data models. The access pattern for these models is that they will be retrieved very frequently but updated maybe once in a month. Think aboiut a game site. Game categories will not and should not be updated at the time they are created. BUt these will be used in my front page so every page request will query the db for all the game categories. So best way is to memcache these entries. So, what you prefere as a best practice for these kind of things? I am thinking of a generic class that takes a model as a parameter and hooks the put() method of it. If model is somehow changed, I will update a dirty flag and retrive the contents from the DS, otherwise get it from the memcache. But not sure, how to hook all db operationsof a model, there are many ways to do that and I am assuming there must be a better way to do this, maybe someone else has already done this work? Is there a generic way to do this? Thanks, -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Is GAE the right choice for e-commerce?
Hi NG We are running our business on AppEngine. (Search Shoes of Prey on Google to find us.) So far, it is going really well. AppEngine is a delight to develop on, and you can't argue with the cost. (I've only been charged $0.01 so far. Although, as I've posted in another thread, the small charge by itself was problematic for me.) :) I had to write the entire ecommerce platform from scratch - which suited me given the very custom nature of our business - however you may not be as keen as me. It really depends on your business. We choose AppEngine primarily because a) 2 of us used to work at Google, and we love AppEngine b) if we start serving a lot of traffic/ bandwidth we want something scalable. As I said earlier as well, the API is really nice and makes developing a breeze. The only thing I wish AppEngine had was HTTPS support for custom domains, and I'm hopeful that one day it will be supported. There are workarounds too. Once AppEngine has HTTPS support and a few open- source ecommerce software packages, it will be a really great platform I would think. Good luck. Mike On Jan 31, 4:31 pm, decuman ngn...@gmail.com wrote: Hello, At the moment I'm thinking of building a small/medium online selling business and before investing more of my time (and money) into GAE based development it would be just great to know some experienced users and expert thoughts and opinions on it. As far as I can see (after reading related books, articles and message group threads) GAE is just a perfect solution for prototyping and building a social networking applications (Blogs, Chats, Galleries, Message boards etc.) which requires tracking a lot of users and storing data... which is great. But there is quite a few (literally none) of nice business/e-commerce applications with decent product galleries, shopping carts and checkout functionality and payment service integration. Also there are lots of complains and negative responses pointing to big GAE functional limitations (which are not going to improve soon or ever) and overall practicability of using it as a reliable e-commerce platform. In particular (PayPal, Google Checkout integration) It wasn't a very deep and thoughtful investigation but the outlined above is just my personal subjective feeling so far. I would really love to know the opinion and thoughts on this matter from GAE experts. Shell I move forward with Google App Engine development or it would be move wise to consider some other solution? Thanks in advance! NG -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: High latency with custom domain
My dedicated server has got a really bad latency again. It is really frustrating. Dedicated Server (Germany) - custom domain 1 xxx (xxx) 0.285 ms 0.233 ms 0.228 ms 2 xe-0-1-0.cr-nashira.cgn4.hosteurope.de (80.237.129.13) 0.405 ms 0.352 ms 0.350 ms 3 xe-0-0-0.cr-polaris.fra1.hosteurope.de (80.237.129.110) 2.927 ms 2.937 ms 2.888 ms 4 de-cix20.net.google.com (80.81.193.108) 4.106 ms 4.043 ms 4.433 ms 5 209.85.255.176 (209.85.255.176) 4.238 ms 209.85.255.178 (209.85.255.178) 4.162 ms 4.248 ms 6 72.14.232.105 (72.14.232.105) 14.235 ms 14.068 ms 72.14.233.104 (72.14.233.104) 13.431 ms 7 216.239.43.90 (216.239.43.90) 91.730 ms 72.14.239.94 (72.14.239.94) 94.268 ms 94.815 ms 8 66.249.95.149 (66.249.95.149) 105.794 ms 216.239.48.69 (216.239.48.69) 111.983 ms 66.249.95.149 (66.249.95.149) 108.316 ms 9 72.14.232.213 (72.14.232.213) 124.573 ms 106.259 ms 107.897 ms 10 209.85.253.157 (209.85.253.157) 117.649 ms 119.529 ms 109.063 ms 11 yw-in-f121.1e100.net (74.125.47.121) 108.389 ms 107.146 ms 107.318 ms Dedicated Server (Germany) - *.appspot.com 1 xxx (xxx) 0.298 ms 0.244 ms 0.203 ms 2 xe-0-1-0.cr-nashira.cgn4.hosteurope.de (80.237.129.13) 0.414 ms 0.365 ms 0.344 ms 3 xe-0-0-0.cr-polaris.fra1.hosteurope.de (80.237.129.110) 2.994 ms 2.921 ms 2.921 ms 4 de-cix20.net.google.com (80.81.193.108) 43.233 ms 3.985 ms 4.202 ms 5 209.85.255.176 (209.85.255.176) 4.563 ms 4.850 ms 209.85.255.178 (209.85.255.178) 17.169 ms 6 209.85.254.112 (209.85.254.112) 4.978 ms 209.85.254.116 (209.85.254.116) 4.411 ms 4.354 ms 7 209.85.254.126 (209.85.254.126) 5.018 ms 209.85.254.134 (209.85.254.134) 4.587 ms 4.167 ms 8 fx-in-f141.1e100.net (74.125.39.141) 4.683 ms 5.028 ms 4.960 ms -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Interactive console -- users.get_current_user() always returns None
You'll need to use the dashboard at appengine.google.com/a/ darkflowstudios.com if you registered with your Google Apps account. On Jan 29, 6:09 pm, BrianDFS br...@darkflowstudios.com wrote: Hmmm, there are no cookies listed for localhost:8080. Maybe I'm misunderstanding how to log in locally then. Another thing I've noticed is that when I log into App Engine via the Google Apps portal athttps://appengine.google.com/a/YOURDOMAIN.COM, it takes me to the /start page where it prompts me to create my first application which then prompts me to verify my account via sms. However, I'm pretty sure I've already verified my account via sms. If I try to enter my phone number anyway, it fails with: The phone number has been sent too many messages or has already been used to confirm an account. On Jan 29, 3:18 pm, Nick Johnson (Google) nick.john...@google.com wrote: On Fri, Jan 29, 2010 at 10:15 PM, BrianDFS br...@darkflowstudios.comwrote: Hi Nick, I'm talking about the interactive development console located at: http://localhost:8080/_ah/admin/interactive No matter what I do or which google account I'm logged into, it never thinks I'm logged in. The local admin console only pays attention to what you logged into on localhost, using the local users stub. You can't log into Google accounts with it. If that's what you're meaning, can you check your browser's cookies for cookies set on localhost:8080, and report what you see there? -Nick Johnson On Jan 29, 12:17 pm, Nick Johnson (Google) nick.john...@google.com wrote: Hi Brian, Which interactive console are you referring to? -Nick On Fri, Jan 29, 2010 at 3:01 PM, BrianDFS br...@darkflowstudios.com wrote: I'm completely new to GAE as of about 3 days ago. I'm working on an app based off some pre-existing source code. Everything seemed to be moving along nicely until yesterday when suddenly it seemed as though I was no longer logged in in my test app. To confirm this I tried the hello world or hello user default program at the interactive console and sure enough, users.get_current_user() always returns None now. I'm tried clearing my browser (Chrome) cache and history, I've also tried other browsers, I'm at a loss. I'm using GAE with my Google Apps account. Any tips, suggestions, or help would be greatly appreciated. Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Could not guess mimetype for favicon
I have changed the mimetypes.py file that is in the Python25 directory. Add a line that handles the ico file extension. 2010/1/31 风笑雪 kea...@gmail.com: Put favicon.ico section before docs section. 2010/1/31 samwyse samw...@gmail.com: I moved the mime_type to the end, but it's still not working: handlers: - url: /docs static_dir: docs - url: /favicon\.ico static_files: docs/favicon.ico upload: docs/favicon.ico mime_type: image/x-icon - url: /robots\.txt static_files: docs/robots.txt upload: docs/robots.txt mime_type: text/plain Scanning files on local disk. Initiating update. Could not guess mimetype for docs/favicon.ico. Using application/ octet-stream. Could not guess mimetype for docs/index_files/themedata.thmx. Using application/octet-stream. Cloning 7 static files. Cloning 13 application files. Deploying new version. On Jan 23, 11:18 pm, 风笑雪 kea...@gmail.com wrote: - url: /favicon\.ico static_files: img/favicon.ico upload: img/favicon.ico mime_type: image/x-icon 2010/1/23samwysesamw...@gmail.com: Using GAE Launcher, I get the following message whenever I deploy my app: Could not guess mimetype for docs/favicon.ico. Using application/ octet-stream. My appl.yaml file contains this: - url: /favicon\.ico mime_type: image/vnd.microsoft.icon static_files: docs/favicon.ico upload: docs/favicon.ico About Launcher... returns this: Your Google App Engine SDK is up to date. Version: release: 1.3.0 timestamp: 1259695715 api_versions: ['1'] Any ideas? Thanks! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: why my datastore nearly be full?
The stored data quota in the dashboard isn't updated in realtime, so you wouldn't see the change immediately even if that is the problem. I'd suspect some buggy indexes or something, though, if that really was your only entity. For one thing, you can't have a single entity larger than 1MB, and it would be tough to use 949 MB in datastore overhead even if you wanted to. On Jan 31, 3:05 am, saintthor saintt...@gmail.com wrote: i deleted the only entity of ltWords. the Total Stored Data is still 95%! who can help me? On 1月30日, 下午8时42分, Sylvain sylvain.viv...@gmail.com wrote: maybe this bug ? http://code.google.com/p/googleappengine/issues/detail?id=631 On Jan 30, 9:27 am, saintthor saintt...@gmail.com wrote: in quota details, i notic my Total Stored Data is 95%. it shouldn' t be so large. in Datastore Statistics, Size of all entities is only 9MB. all that happened from about 4% to 95% is i updated an entity for about 200 times. the class is: class ltWords( db.Model ): WordsDText = db.TextProperty() after this, there is only one entity of ltWords and the WordsDText is no longer than 1MB. why does it take more than 90% of my datastore quota and how to release it? for saintthor.appspot.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Android app to monitor appengine quotas
Hi, I've released a little android app to monitor your app engine quotas. It's called Engine Watch and is available from the android market. See my blog at http://code.neenbedankt.com for more details. I hope it's useful to some of you too :) Hugo -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] How memcache work?
In official documentation I didn't find nothing about memcache transaction. What happens if I access to memcache from different request( this is save? ) , is transactions atomic? This will be great if somebody can explain this. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Datastore WRITE operations are failing (9.30AM US/Central)
agreed On Jan 30, 2:32 pm, mete meteba...@gmail.com wrote: That is a very strange situation especially for someone considering to use the service in production like me. Can google people give details or comment on the issue ? On Jan 30, 5:55 pm, Murali imural...@gmail.com wrote: Its back now. Write operations are going through. (9.55AM) Thanks, Murali On Jan 30, 9:45 am, Murali imural...@gmail.com wrote: App Engine team, At 9.30AM (US/Central), all Data store writes are failing. No exception is thrown, so error handling in the app did not produce any logs to capture. It appears to be successful, but entries added are not found the data store. The problem still persists at 9.45. Status athttp://code.google.com/status/appengineshowseverything normal. Could you please have a look. Thanks, Murali -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: why my datastore nearly be full?
I'm seeing the same thing. I got hardly anything stored in the datastore, but my quota is at 50% right now. I did have some stuff in the memcache however. This morning I flushed memcache and deleted all entities in the datastore but the quota usage is still going up. I'm waiting to see what happens after the daily reset. Hugo On Feb 1, 5:27 pm, Wooble geoffsp...@gmail.com wrote: The stored data quota in the dashboard isn't updated in realtime, so you wouldn't see the change immediately even if that is the problem. I'd suspect some buggy indexes or something, though, if that really was your only entity. For one thing, you can't have a single entity larger than 1MB, and it would be tough to use 949 MB in datastore overhead even if you wanted to. On Jan 31, 3:05 am, saintthor saintt...@gmail.com wrote: i deleted the only entity of ltWords. the Total Stored Data is still 95%! who can help me? On 1月30日, 下午8时42分, Sylvain sylvain.viv...@gmail.com wrote: maybe this bug ? http://code.google.com/p/googleappengine/issues/detail?id=631 On Jan 30, 9:27 am, saintthor saintt...@gmail.com wrote: in quota details, i notic my Total Stored Data is 95%. it shouldn' t be so large. in Datastore Statistics, Size of all entities is only 9MB. all that happened from about 4% to 95% is i updated an entity for about 200 times. the class is: class ltWords( db.Model ): WordsDText = db.TextProperty() after this, there is only one entity of ltWords and the WordsDText is no longer than 1MB. why does it take more than 90% of my datastore quota and how to release it? for saintthor.appspot.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: security in task queue servlets
IMHO you're on a wrong way. Use web.xml contrains and no other things. HttpHeaders are very easy to create and send with a http request. fabrizio On Jan 30, 10:07 pm, Patrick Linskey plins...@gmail.com wrote: ... a curl-sourced request. Is this a safe assumption to rely on? Are there plans to document a reliable way to ensure servlet security in a task queue environment? Is there something else that I'm missing? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Best practice for caching frequent datastore reads
First of all, if you really care about the performance, store those data in static directly and let the app engine directly return it (and update the whole app when you need to update the data). Second, please be aware that caching object may not be the most efficient way. In many cases, caching the final output (typically HTML file or JSON file) in memcache gives you the best performance. I am using this technique in all my projects, and it is working very well. If you still want to cache objects, here is my suggestion. While creating a fancy generic class that automatically performs caching seems clever and elegant, that kind of architecture tends to create more headache in future when the project gets bigger and more engineers are involved. I much prefer to hook get/put accesses explicitly for each class. Here is a piece of code I am using in my project (cacheman is my memcache name space manager where I can easily control the namespace - which is a good practice). SEC_USER = 60*60 # one hour @classmethod def get_user(cls, uid): cache_key = cacheman.user(uid) # which performs (/user/ %s %s uid) user = memcache.get(cache_key) if user: return user user = cls.get_by_key_name(uid) if user: memcache.set(cache_key, user, SEC_USER) return user def put(self): cache_key = cacheman.user(self.key().name()) memcache.set(cache_key, self, SEC_USER) return super(User, self).put() Please notice that I am overriding put() method so that I always call memcache.set, but NOT overriding get_key_by_name (this is just my preferred coding style, which you don't need to follow). Please be aware that you need to just get keys when you perform query, and use the accessor (in my case get_user) to get the actual object. Satoshi On Feb 1, 3:40 am, k3xji sum...@gmail.com wrote: Hi all, I am thinking of away to imlement a generic way for my frequently readed data models. The access pattern for these models is that they will be retrieved very frequently but updated maybe once in a month. Think aboiut a game site. Game categories will not and should not be updated at the time they are created. BUt these will be used in my front page so every page request will query the db for all the game categories. So best way is to memcache these entries. So, what you prefere as a best practice for these kind of things? I am thinking of a generic class that takes a model as a parameter and hooks the put() method of it. If model is somehow changed, I will update a dirty flag and retrive the contents from the DS, otherwise get it from the memcache. But not sure, how to hook all db operationsof a model, there are many ways to do that and I am assuming there must be a better way to do this, maybe someone else has already done this work? Is there a generic way to do this? Thanks, -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] security in task queue servlets
If you have a compelling reason for really locking down the task queue url (and Require Admin login isn't enough), you could create a mechanism that creates a task name for each queued task.. and the task verifies that its name is correct. You could have the task use the X-AppEngine-TaskName header to check its name.. So.. when you add a task to the queue.. you do something like this: taskName = getUniqueTaskName() nameHash = getHash(taskName) taskqueue.add(url= '/myTaskQueue', countdown = 0, name = taskName, params = {'nameHash' : nameHash}) and.. in the first part of the /myTaskQueue code.. you could have it verify that the 'nameHash' param is equal to getHash() of the TaskName you grab from the header.. On Sat, Jan 30, 2010 at 4:07 PM, Patrick Linskey plins...@gmail.com wrote: Hi, I'd like to programmatically ensure that my task queue servlets are only invoked via the task queue. I've got a security constraint in my web.xml, but I'd like to also check in code to avoid any potential mis- configuration in the future. Is there any supported means to do such a check? I tried looking at the contents of the HttpServletRequest (isUserInRole (), getAuthType(), getUserPrincipal(), getRemoteName()), to no avail. I also tried UserServiceFactory.getUserService().isAdmin(), but received an exception informing me that no user was logged in. I can see that there are a number of task queue-specific HTTP headers. Currently, I'm checking that X-AppEngine-TaskRetryCount is present, and if so, assuming that the request has come from the task queue and that it's therefore safe to process. Empirically, it looks like GAE strips out the X-AppEngine-TaskRetryCount header when I specify it in a curl-sourced request. Is this a safe assumption to rely on? Are there plans to document a reliable way to ensure servlet security in a task queue environment? Is there something else that I'm missing? Also, in an ideal world, it'd be nice if request.isUserInRole(admin) would return true at the appropriate times. Thanks, -Patrick -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I define an admin user for my application?
Add/Invite them as developers to the application. Robert On Mon, Feb 1, 2010 at 8:15 AM, Tiago Melo tiagohar...@gmail.com wrote: Hi all. I have a simple app deployed at GAE and I want to restrict some URL's to admin users only. But how can I define which users are administrators? I've searched over the admin console and didn't find anything. Thanks in advance. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: How memcache work?
I believe that the memcache service is implemented on top of this. I could be wrong, though: http://memcached.org/ On Mon, Feb 1, 2010 at 10:31 AM, Satoshi satoshi.nakaj...@gmail.com wrote: There is no transaction mechanism around memcache. Although it is technically possible to use an entity group (or groups) on the datastore to create a transaction mechainsm on top of memcache, it does not make sense to use slower mechanism (datastore) on top of faster mechanism (memcache). Generally speaking, you should use memcache to return a snapshot of data to the client. If you need transactions, go directly to the datastore. Satoshi On Feb 1, 6:35 am, masterzim masterzim@googlemail.com wrote: In official documentation I didn't find nothing about memcache transaction. What happens if I access to memcache from different request( this is save? ) , is transactions atomic? This will be great if somebody can explain this. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Patrick H. Twohig. Namazu Studios P.O. Box 34161 San Diego, CA 92163-4161 Office: 619.862.2890 x100 Cell: 619.453.5075 Twitter: @svm_invictvs IRC: svm_invic...@irc.freenode.net ##java, #android-dev, #iphonedev, #appengine http://www.namazustudios.com/ This communication, and any attachments, shall be considered confidential and proprietary information of Namazu Studios LLC. This message, and attachments, are intended for the listed recipients only. If you are not one of the intended recipients, please destroy all copies of this communication. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] security in task queue servlets
I believe that the App Engine injects the pertinent information from the Users service. For instance, when you call HttpServletRequest.getUserPrinicpal(), you're getting values injected by the UsersService. On Mon, Feb 1, 2010 at 11:02 AM, Eli Jones eli.jo...@gmail.com wrote: If you have a compelling reason for really locking down the task queue url (and Require Admin login isn't enough), you could create a mechanism that creates a task name for each queued task.. and the task verifies that its name is correct. You could have the task use the X-AppEngine-TaskName header to check its name.. So.. when you add a task to the queue.. you do something like this: taskName = getUniqueTaskName() nameHash = getHash(taskName) taskqueue.add(url= '/myTaskQueue', countdown = 0, name = taskName, params = {'nameHash' : nameHash}) and.. in the first part of the /myTaskQueue code.. you could have it verify that the 'nameHash' param is equal to getHash() of the TaskName you grab from the header.. On Sat, Jan 30, 2010 at 4:07 PM, Patrick Linskey plins...@gmail.comwrote: Hi, I'd like to programmatically ensure that my task queue servlets are only invoked via the task queue. I've got a security constraint in my web.xml, but I'd like to also check in code to avoid any potential mis- configuration in the future. Is there any supported means to do such a check? I tried looking at the contents of the HttpServletRequest (isUserInRole (), getAuthType(), getUserPrincipal(), getRemoteName()), to no avail. I also tried UserServiceFactory.getUserService().isAdmin(), but received an exception informing me that no user was logged in. I can see that there are a number of task queue-specific HTTP headers. Currently, I'm checking that X-AppEngine-TaskRetryCount is present, and if so, assuming that the request has come from the task queue and that it's therefore safe to process. Empirically, it looks like GAE strips out the X-AppEngine-TaskRetryCount header when I specify it in a curl-sourced request. Is this a safe assumption to rely on? Are there plans to document a reliable way to ensure servlet security in a task queue environment? Is there something else that I'm missing? Also, in an ideal world, it'd be nice if request.isUserInRole(admin) would return true at the appropriate times. Thanks, -Patrick -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Patrick H. Twohig. Namazu Studios P.O. Box 34161 San Diego, CA 92163-4161 Office: 619.862.2890 x100 Cell: 619.453.5075 Twitter: @svm_invictvs IRC: svm_invic...@irc.freenode.net ##java, #android-dev, #iphonedev, #appengine http://www.namazustudios.com/ This communication, and any attachments, shall be considered confidential and proprietary information of Namazu Studios LLC. This message, and attachments, are intended for the listed recipients only. If you are not one of the intended recipients, please destroy all copies of this communication. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: security in task queue servlets
From what I can tell, it doesn't seem to do anything with the user principal in the case where an unauthenticated servlet fires off a task queue request. The task queue docs indicate that requests from the task queue will be executed with admin rights, but I can't seem to find any way to ensure that this is the case from code. Thanks, -Patrick On Feb 1, 12:22 pm, Patrick Twohig patr...@namazustudios.com wrote: I believe that the App Engine injects the pertinent information from the Users service. For instance, when you call HttpServletRequest.getUserPrinicpal(), you're getting values injected by the UsersService. On Mon, Feb 1, 2010 at 11:02 AM, Eli Jones eli.jo...@gmail.com wrote: If you have a compelling reason for really locking down the task queue url (and Require Admin login isn't enough), you could create a mechanism that creates a task name for each queued task.. and the task verifies that its name is correct. You could have the task use the X-AppEngine-TaskName header to check its name.. So.. when you add a task to the queue.. you do something like this: taskName = getUniqueTaskName() nameHash = getHash(taskName) taskqueue.add(url = '/myTaskQueue', countdown = 0, name = taskName, params = {'nameHash' : nameHash}) and.. in the first part of the /myTaskQueue code.. you could have it verify that the 'nameHash' param is equal to getHash() of the TaskName you grab from the header.. On Sat, Jan 30, 2010 at 4:07 PM, Patrick Linskey plins...@gmail.comwrote: Hi, I'd like to programmatically ensure that my task queue servlets are only invoked via the task queue. I've got a security constraint in my web.xml, but I'd like to also check in code to avoid any potential mis- configuration in the future. Is there any supported means to do such a check? I tried looking at the contents of the HttpServletRequest (isUserInRole (), getAuthType(), getUserPrincipal(), getRemoteName()), to no avail. I also tried UserServiceFactory.getUserService().isAdmin(), but received an exception informing me that no user was logged in. I can see that there are a number of task queue-specific HTTP headers. Currently, I'm checking that X-AppEngine-TaskRetryCount is present, and if so, assuming that the request has come from the task queue and that it's therefore safe to process. Empirically, it looks like GAE strips out the X-AppEngine-TaskRetryCount header when I specify it in a curl-sourced request. Is this a safe assumption to rely on? Are there plans to document a reliable way to ensure servlet security in a task queue environment? Is there something else that I'm missing? Also, in an ideal world, it'd be nice if request.isUserInRole(admin) would return true at the appropriate times. Thanks, -Patrick -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Patrick H. Twohig. Namazu Studios P.O. Box 34161 San Diego, CA 92163-4161 Office: 619.862.2890 x100 Cell: 619.453.5075 Twitter: @svm_invictvs IRC: svm_invic...@irc.freenode.net ##java, #android-dev, #iphonedev, #appengine http://www.namazustudios.com/ This communication, and any attachments, shall be considered confidential and proprietary information of Namazu Studios LLC. This message, and attachments, are intended for the listed recipients only. If you are not one of the intended recipients, please destroy all copies of this communication. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: security in task queue servlets
(and Require Admin login isn't enough) That would be enough, but the only way to do that is to put the limitation in the web.xml file, which is pretty far away from the servlet in question. I want to make sure that someone doesn't accidentally mis-configure the web.xml file to remove the admin restriction. And I'd really rather not parse web.xml and apply all the appropriate rules to do so. Also, I don't know whether or not Google guarantees that the X- AppEngine-TaskName header is stripped from malicious incoming requests. It looks like it does, but that's just based on my observations. Thanks, -Patrick On Feb 1, 11:02 am, Eli Jones eli.jo...@gmail.com wrote: If you have a compelling reason for really locking down the task queue url (and Require Admin login isn't enough), you could create a mechanism that creates a task name for each queued task.. and the task verifies that its name is correct. You could have the task use the X-AppEngine-TaskName header to check its name.. So.. when you add a task to the queue.. you do something like this: taskName = getUniqueTaskName() nameHash = getHash(taskName) taskqueue.add(url = '/myTaskQueue', countdown = 0, name = taskName, params = {'nameHash' : nameHash}) and.. in the first part of the /myTaskQueue code.. you could have it verify that the 'nameHash' param is equal to getHash() of the TaskName you grab from the header.. On Sat, Jan 30, 2010 at 4:07 PM, Patrick Linskey plins...@gmail.com wrote: Hi, I'd like to programmatically ensure that my task queue servlets are only invoked via the task queue. I've got a security constraint in my web.xml, but I'd like to also check in code to avoid any potential mis- configuration in the future. Is there any supported means to do such a check? I tried looking at the contents of the HttpServletRequest (isUserInRole (), getAuthType(), getUserPrincipal(), getRemoteName()), to no avail. I also tried UserServiceFactory.getUserService().isAdmin(), but received an exception informing me that no user was logged in. I can see that there are a number of task queue-specific HTTP headers. Currently, I'm checking that X-AppEngine-TaskRetryCount is present, and if so, assuming that the request has come from the task queue and that it's therefore safe to process. Empirically, it looks like GAE strips out the X-AppEngine-TaskRetryCount header when I specify it in a curl-sourced request. Is this a safe assumption to rely on? Are there plans to document a reliable way to ensure servlet security in a task queue environment? Is there something else that I'm missing? Also, in an ideal world, it'd be nice if request.isUserInRole(admin) would return true at the appropriate times. Thanks, -Patrick -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: security in task queue servlets
I am using web.xml, but I'm unsatisfied with the potential for a future mis-configuration to open up a hole to these servlets. I'm reluctant to rely on an obscure rule tucked away in a shared separately-modifiable deployment descriptor to always be properly configured. HttpHeaders are very easy to create and send with a http request. Agreed, but it's also very easy for the appserver to filter out certain headers (which it looks like Google is doing). If they're doing this intentionally, then the existence of a header can provide a meaningful security validation. -Patrick On Feb 1, 10:43 am, fhtino fht...@gmail.com wrote: IMHO you're on a wrong way. Use web.xml contrains and no other things. HttpHeaders are very easy to create and send with a http request. fabrizio On Jan 30, 10:07 pm, Patrick Linskey plins...@gmail.com wrote: ... a curl-sourced request. Is this a safe assumption to rely on? Are there plans to document a reliable way to ensure servlet security in a task queue environment? Is there something else that I'm missing? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: security in task queue servlets
That's why you have to use the TaskName and Hash method to verify that a Task was added to the queue in an orthodox manner. Unless someone knows the hash scheme you are using for TaskNames.. they will not be able to send in a (TaskName,getHash(TaskName)) pair that would get validated when the task started running.. Let me know if I'm not being clear in my suggestion.. this should do exactly what you want.. it would 100% prevent a person from accidentally visiting the URL and running a task (since it checks for the TaskName header).. and it would pretty much prevent some random person from trying to impersonate a task by sending a Post to the task URL.. They would have to know the correct hash to send along with the TaskName they are spoofing.. On Mon, Feb 1, 2010 at 3:29 PM, Patrick Linskey plins...@gmail.com wrote: (and Require Admin login isn't enough) That would be enough, but the only way to do that is to put the limitation in the web.xml file, which is pretty far away from the servlet in question. I want to make sure that someone doesn't accidentally mis-configure the web.xml file to remove the admin restriction. And I'd really rather not parse web.xml and apply all the appropriate rules to do so. Also, I don't know whether or not Google guarantees that the X- AppEngine-TaskName header is stripped from malicious incoming requests. It looks like it does, but that's just based on my observations. Thanks, -Patrick On Feb 1, 11:02 am, Eli Jones eli.jo...@gmail.com wrote: If you have a compelling reason for really locking down the task queue url (and Require Admin login isn't enough), you could create a mechanism that creates a task name for each queued task.. and the task verifies that its name is correct. You could have the task use the X-AppEngine-TaskName header to check its name.. So.. when you add a task to the queue.. you do something like this: taskName = getUniqueTaskName() nameHash = getHash(taskName) taskqueue.add(url= '/myTaskQueue', countdown = 0, name = taskName, params = {'nameHash' : nameHash}) and.. in the first part of the /myTaskQueue code.. you could have it verify that the 'nameHash' param is equal to getHash() of the TaskName you grab from the header.. On Sat, Jan 30, 2010 at 4:07 PM, Patrick Linskey plins...@gmail.com wrote: Hi, I'd like to programmatically ensure that my task queue servlets are only invoked via the task queue. I've got a security constraint in my web.xml, but I'd like to also check in code to avoid any potential mis- configuration in the future. Is there any supported means to do such a check? I tried looking at the contents of the HttpServletRequest (isUserInRole (), getAuthType(), getUserPrincipal(), getRemoteName()), to no avail. I also tried UserServiceFactory.getUserService().isAdmin(), but received an exception informing me that no user was logged in. I can see that there are a number of task queue-specific HTTP headers. Currently, I'm checking that X-AppEngine-TaskRetryCount is present, and if so, assuming that the request has come from the task queue and that it's therefore safe to process. Empirically, it looks like GAE strips out the X-AppEngine-TaskRetryCount header when I specify it in a curl-sourced request. Is this a safe assumption to rely on? Are there plans to document a reliable way to ensure servlet security in a task queue environment? Is there something else that I'm missing? Also, in an ideal world, it'd be nice if request.isUserInRole(admin) would return true at the appropriate times. Thanks, -Patrick -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] What does Support for mapping operations across datasets mean?
What does Support for mapping operations across datasets mean? (Taken from the Features on Deck section of the current Roadmap: http://code.google.com/appengine/docs/roadmap.html) Thanks! Chris PS: I did a fairly extensive search to figure this out on my own, but came up with very little. Then I asked a respected colleague who was also unsure. I know this is probably painfully obvious to some people, so I apologize. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] What does Support for mapping operations across datasets mean?
I'm *pretty* sure it's a reference to MapReduce? http://en.wikipedia.org/wiki/MapReduce On Mon, Feb 1, 2010 at 4:28 PM, DutrowLLC dutrow...@gmail.com wrote: What does Support for mapping operations across datasets mean? (Taken from the Features on Deck section of the current Roadmap: http://code.google.com/appengine/docs/roadmap.html) Thanks! Chris PS: I did a fairly extensive search to figure this out on my own, but came up with very little. Then I asked a respected colleague who was also unsure. I know this is probably painfully obvious to some people, so I apologize. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Character encoding Problem
Hi, When I deploy my application I see this character: � instead of this one: ü. When I run the app. in development mode the characters are ok. My application gets an xml file from a webservice on our company server. The header of the xml file is: ?xml version=1.0 encoding=utf-8 ? when I open this xml file in a browser, the characters are displayed as they should be. In eclipse, i set all the java-files to text-file encoding utf-8 (properties text file encoding) In the Project-properties, i set the encoding to utf-8 also. In the head of my Html page I have this line: meta http- equiv=content-type content=text/html; charset=UTF-8 What could be wrong ? ? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Receive email from googlemail account
Hi, Is it possible to receive email from googlemail account? Regards, Steve -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Store email in appengine
Hi, appengine allows per documentation only passing email as http and the received email has to be handled by a servlet. What is the best approach to persist the received email in appengine? Regards, STeve -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: why my datastore nearly be full?
We see about 10x overhead, I assume from indexes, in our apps. One app is using 30M for storing entities and metadata, the indexes take another 300M. We see similar usage multipliers in all of our apps. Robert 2010/2/1 Hugo Visser botte...@gmail.com: I'm seeing the same thing. I got hardly anything stored in the datastore, but my quota is at 50% right now. I did have some stuff in the memcache however. This morning I flushed memcache and deleted all entities in the datastore but the quota usage is still going up. I'm waiting to see what happens after the daily reset. Hugo On Feb 1, 5:27 pm, Wooble geoffsp...@gmail.com wrote: The stored data quota in the dashboard isn't updated in realtime, so you wouldn't see the change immediately even if that is the problem. I'd suspect some buggy indexes or something, though, if that really was your only entity. For one thing, you can't have a single entity larger than 1MB, and it would be tough to use 949 MB in datastore overhead even if you wanted to. On Jan 31, 3:05 am, saintthor saintt...@gmail.com wrote: i deleted the only entity of ltWords. the Total Stored Data is still 95%! who can help me? On 1月30日, 下午8时42分, Sylvain sylvain.viv...@gmail.com wrote: maybe this bug ? http://code.google.com/p/googleappengine/issues/detail?id=631 On Jan 30, 9:27 am, saintthor saintt...@gmail.com wrote: in quota details, i notic my Total Stored Data is 95%. it shouldn' t be so large. in Datastore Statistics, Size of all entities is only 9MB. all that happened from about 4% to 95% is i updated an entity for about 200 times. the class is: class ltWords( db.Model ): WordsDText = db.TextProperty() after this, there is only one entity of ltWords and the WordsDText is no longer than 1MB. why does it take more than 90% of my datastore quota and how to release it? for saintthor.appspot.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Best practice for caching frequent datastore reads
Second, please be aware that caching object may not be the most efficient way. In many cases, caching the final output (typically HTML file or JSON file) in memcache gives you the best performance. I am using this technique in all my projects, and it is working very well. That was a very helpful point indeed. Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: why my datastore nearly be full?
Hi to all , I'm experiencing the same problem. I've described it also in : http://groups.google.com/group/google-appengine-java/browse_thread/thread/27d03163d170e6ec/9ad021dee7e82359 Currently my status is : from the dash board : Total Stored Data : 38% (and definitely counting..) Datastore statistics : Size of all entities 414 KBytes from which the main is _ah_SESSION - 86% I don't have any indexes defined. the number of sessions which my application currently keeps are round 40 per hour. Any help regarding this would be appreciated. If anybody from Google guys is interested - my app-id is sandcode Best Regards: Dimitar Makariev On Feb 1, 11:51 am, Robert Kluin robert.kl...@gmail.com wrote: We see about 10x overhead, I assume from indexes, in our apps. One app is using 30M for storing entities and metadata, the indexes take another 300M. We see similar usage multipliers in all of our apps. Robert 2010/2/1 Hugo Visser botte...@gmail.com: I'm seeing the same thing. I got hardly anything stored in the datastore, but my quota is at 50% right now. I did have some stuff in the memcache however. This morning I flushed memcache and deleted all entities in the datastore but the quota usage is still going up. I'm waiting to see what happens after the daily reset. Hugo On Feb 1, 5:27 pm, Wooble geoffsp...@gmail.com wrote: The stored data quota in the dashboard isn't updated in realtime, so you wouldn't see the change immediately even if that is the problem. I'd suspect some buggy indexes or something, though, if that really was your only entity. For one thing, you can't have a single entity larger than 1MB, and it would be tough to use 949 MB in datastore overhead even if you wanted to. On Jan 31, 3:05 am, saintthor saintt...@gmail.com wrote: i deleted the only entity of ltWords. the Total Stored Data is still 95%! who can help me? On 1月30日, 下午8时42分, Sylvain sylvain.viv...@gmail.com wrote: maybe this bug ? http://code.google.com/p/googleappengine/issues/detail?id=631 On Jan 30, 9:27 am, saintthor saintt...@gmail.com wrote: in quota details, i notic my Total Stored Data is 95%. it shouldn' t be so large. in Datastore Statistics, Size of all entities is only 9MB. all that happened from about 4% to 95% is i updated an entity for about 200 times. the class is: class ltWords( db.Model ): WordsDText = db.TextProperty() after this, there is only one entity of ltWords and the WordsDText is no longer than 1MB. why does it take more than 90% of my datastore quota and how to release it? for saintthor.appspot.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Is GAE the right choice for e-commerce?
Hi Mike, Thanks for your reply. It is really helpful and encouraging to know that there are successful e-Commerce models already running good on GAE. Your online shop looks GREAT! Clean and intuitive design with simple cart + payment structure. And yes, it is pretty much the same that I want to do for my service - to write a custom (yet simple) shopping cart and be able to control and change anything quick and easy (as I understand it)... and due do appEngine development simplicity phenomenon I actually enjoy doing it right now, but, as I mentioned above, my main concern is all about the simplicity of adding, controlling, 'rewiring' and, the last but not least, securing clients online payments. I'm pretty sure it is relatively easy for an experienced web programmer (especially Google programmer;) to set it up nicely from the scratch but not that transparent for a beginner (like myself) to make that part running properly without fiddling with different approaches, trials and fails... and true, I wish there were a couple of reliable open-source e-commerce solutions already to investigate and rely on. Also I guess there could be a decent guideline for embedding Google Checkout service... at least it seems natural as it is Google's proprietary online paying system. Thanks, NG On Feb 2, 2:19 am, Mike mickn...@gmail.com wrote: Hi NG We are running our business on AppEngine. (Search Shoes of Prey on Google to find us.) So far, it is going really well. AppEngine is a delight to develop on, and you can't argue with the cost. (I've only been charged $0.01 so far. Although, as I've posted in another thread, the small charge by itself was problematic for me.) :) I had to write the entire ecommerce platform from scratch - which suited me given the very custom nature of our business - however you may not be as keen as me. It really depends on your business. We choose AppEngine primarily because a) 2 of us used to work at Google, and we love AppEngine b) if we start serving a lot of traffic/ bandwidth we want something scalable. As I said earlier as well, the API is really nice and makes developing a breeze. The only thing I wish AppEngine had was HTTPS support for custom domains, and I'm hopeful that one day it will be supported. There are workarounds too. Once AppEngine has HTTPS support and a few open- source ecommerce software packages, it will be a really great platform I would think. Good luck. Mike On Jan 31, 4:31 pm, decuman ngn...@gmail.com wrote: Hello, At the moment I'm thinking of building a small/medium online selling business and before investing more of my time (and money) into GAE based development it would be just great to know some experienced users and expert thoughts and opinions on it. As far as I can see (after reading related books, articles and message group threads) GAE is just a perfect solution for prototyping and building a social networking applications (Blogs, Chats, Galleries, Message boards etc.) which requires tracking a lot of users and storing data... which is great. But there is quite a few (literally none) of nice business/e-commerce applications with decent product galleries, shopping carts and checkout functionality and payment service integration. Also there are lots of complains and negative responses pointing to big GAE functional limitations (which are not going to improve soon or ever) and overall practicability of using it as a reliable e-commerce platform. In particular (PayPal, Google Checkout integration) It wasn't a very deep and thoughtful investigation but the outlined above is just my personal subjective feeling so far. I would really love to know the opinion and thoughts on this matter from GAE experts. Shell I move forward with Google App Engine development or it would be move wise to consider some other solution? Thanks in advance! NG -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: why my datastore nearly be full?
Hi to all , I have more or less the same problem. I've described it also here : http://groups.google.com/group/google-appengine-java/browse_thread/thread/27d03163d170e6ec/9ad021dee7e82359 my current status is : Total Stored Data : 37% While at the same time the Display statistics is : Size of all entities 414 KBytes with 86% used by _ah_SESSION , which is ok I don't have any indexes defined. if the guys from Google want to take a look my app-id is sandcode Best Regards: Dimitar Makariev On Feb 1, 11:51 am, Robert Kluin robert.kl...@gmail.com wrote: We see about 10x overhead, I assume from indexes, in our apps. One app is using 30M for storing entities and metadata, the indexes take another 300M. We see similar usage multipliers in all of our apps. Robert 2010/2/1 Hugo Visser botte...@gmail.com: I'm seeing the same thing. I got hardly anything stored in the datastore, but my quota is at 50% right now. I did have some stuff in the memcache however. This morning I flushed memcache and deleted all entities in the datastore but the quota usage is still going up. I'm waiting to see what happens after the daily reset. Hugo On Feb 1, 5:27 pm, Wooble geoffsp...@gmail.com wrote: The stored data quota in the dashboard isn't updated in realtime, so you wouldn't see the change immediately even if that is the problem. I'd suspect some buggy indexes or something, though, if that really was your only entity. For one thing, you can't have a single entity larger than 1MB, and it would be tough to use 949 MB in datastore overhead even if you wanted to. On Jan 31, 3:05 am, saintthor saintt...@gmail.com wrote: i deleted the only entity of ltWords. the Total Stored Data is still 95%! who can help me? On 1月30日, 下午8时42分, Sylvain sylvain.viv...@gmail.com wrote: maybe this bug ? http://code.google.com/p/googleappengine/issues/detail?id=631 On Jan 30, 9:27 am, saintthor saintt...@gmail.com wrote: in quota details, i notic my Total Stored Data is 95%. it shouldn' t be so large. in Datastore Statistics, Size of all entities is only 9MB. all that happened from about 4% to 95% is i updated an entity for about 200 times. the class is: class ltWords( db.Model ): WordsDText = db.TextProperty() after this, there is only one entity of ltWords and the WordsDText is no longer than 1MB. why does it take more than 90% of my datastore quota and how to release it? for saintthor.appspot.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Interactive console -- users.get_current_user() always returns None
Yes, I'm using https://appengine.google.com/a/darkflowstudios.com and it still redirects me to the /start page to Create an Application. I've added myself to the wait list using the form located at: https://appengine.google.com/waitlist/sms_issues Hoping for a response sometime this week as I have an app waiting to go. Thanks. BTW, I also sorted out my login issues. Was a browser cache issue. On Feb 1, 9:39 am, Wooble geoffsp...@gmail.com wrote: You'll need to use the dashboard at appengine.google.com/a/ darkflowstudios.com if you registered with your Google Apps account. On Jan 29, 6:09 pm, BrianDFS br...@darkflowstudios.com wrote: Hmmm, there are no cookies listed for localhost:8080. Maybe I'm misunderstanding how to log in locally then. Another thing I've noticed is that when I log into App Engine via the Google Apps portal athttps://appengine.google.com/a/YOURDOMAIN.COM, it takes me to the /start page where it prompts me to create my first application which then prompts me to verify my account via sms. However, I'm pretty sure I've already verified my account via sms. If I try to enter my phone number anyway, it fails with: The phone number has been sent too many messages or has already been used to confirm an account. On Jan 29, 3:18 pm, Nick Johnson (Google) nick.john...@google.com wrote: On Fri, Jan 29, 2010 at 10:15 PM, BrianDFS br...@darkflowstudios.comwrote: Hi Nick, I'm talking about the interactive development console located at: http://localhost:8080/_ah/admin/interactive No matter what I do or which google account I'm logged into, it never thinks I'm logged in. The local admin console only pays attention to what you logged into on localhost, using the local users stub. You can't log into Google accounts with it. If that's what you're meaning, can you check your browser's cookies for cookies set on localhost:8080, and report what you see there? -Nick Johnson On Jan 29, 12:17 pm, Nick Johnson (Google) nick.john...@google.com wrote: Hi Brian, Which interactive console are you referring to? -Nick On Fri, Jan 29, 2010 at 3:01 PM, BrianDFS br...@darkflowstudios.com wrote: I'm completely new to GAE as of about 3 days ago. I'm working on an app based off some pre-existing source code. Everything seemed to be moving along nicely until yesterday when suddenly it seemed as though I was no longer logged in in my test app. To confirm this I tried the hello world or hello user default program at the interactive console and sure enough, users.get_current_user() always returns None now. I'm tried clearing my browser (Chrome) cache and history, I've also tried other browsers, I'm at a loss. I'm using GAE with my Google Apps account. Any tips, suggestions, or help would be greatly appreciated. Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Charged $0.01... consequently my credit card was blocked
Mike, Yikes, really sorry about that. You're right, it doesn't make any sense to charge 0.01 (On a positive note, I saw your site linked in another groups post and it's pretty awesome: http://www.shoesofprey.com). I've raised the issue so that we can talk about ways we can reduce the chances that this sort of thing can happen in the future. Can you fill out the billing issues form? http://code.google.com/support/bin/request.py?contact_type=AppEngineBillingSupport Maybe there is something we can do for you. At the very least it'll create a trackable issue for us. On Sat, Jan 30, 2010 at 12:27 PM, Mike mickn...@gmail.com wrote: Hi there Just got off the phone with my bank after spending 10 minutes reactivating my credit card after my first AppEngine credit card charge for $0.01 USD. Banks tend to view these charges as potentially fraudulent. I understand the need to charge for AppEngine, but charging $0.01 a week hardly seems worthwhile. Perhaps the system could aggregate charges to a reasonable amount - say $5, and then charge the card? Alternatively, allow me to prepay a certain amount, so then I can forget about it for a few months. The other issue is that my credit card is in Australia Dollars. I'm probably going to get some sort of foreign currency conversion fee now every time $0.01 is charged. That will probably be many many multiples of the actual charge. Cheers Mike -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] What are my options for generating PDFs dynamically?
I need to be able to generate PDFs dynamically. I also need to be able to parse PDF files to extract certain information. Are there any Python/Java libraries that work within the sandboxing and whitelisting restrictions? Thanks, Daniel -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] How to store very large files (e.g. PDFs)
I need to allow users to upload very large/complex PDFs containing many images. The PDFs are typically 300MB each. Will this be possible with GAE? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: What are my options for generating PDFs dynamically?
On Feb 2, 1:57 pm, Daniel C daniel.c...@gmail.com wrote: I need to be able to generate PDFs dynamically. I also need to be able to parse PDF files to extract certain information. Are there any Python/Java libraries that work within the sandboxing and whitelisting restrictions? I use reportlab to generate PDFs on Appengine. I don't parse PDFs, but their reportlab plus product may help you there. http://www.reportlab.com/software/opensource/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Android app to monitor appengine quotas
can u find way to get orders for paid Android apps from goog checkout? On Feb 1, 11:05 am, Hugo Visser botte...@gmail.com wrote: Hi, I've released a little android app to monitor your app engine quotas. It's called Engine Watch and is available from the android market. See my blog athttp://code.neenbedankt.comfor more details. I hope it's useful to some of you too :) Hugo -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] my links are invalid on Digg
Hi all, Sorry if this post doesn't seems to be appropriate, but I can't submit links to digg, with this error message by digg This link does not appear to be a working link. Please check the URL and try again. A sample link from my GAE site is this http://jamesgae.appspot.com/blog/2010/01/28/blogs-on-the-cloud-creating-your-own-customize-blog Any ideas? Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] How to deal with the _ToDatastoreError(err) problem?
I meet this problem regularly. Anyone has an idea about this problem? Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Character encoding Problem
You've probably already done this, but did you try going to Window Preferences General Workspace in eclipse and setting the Text File Encoding to UTF-8? On Feb 1, 4:12 pm, Jos Muysers josmuys...@gmail.com wrote: Hi, When I deploy my application I see this character: instead of this one: ü. When I run the app. in development mode the characters are ok. My application gets an xml file from a webservice on our company server. The header of the xml file is: ?xml version=1.0 encoding=utf-8 ? when I open this xml file in a browser, the characters are displayed as they should be. In eclipse, i set all the java-files to text-file encoding utf-8 (properties text file encoding) In the Project-properties, i set the encoding to utf-8 also. In the head of my Html page I have this line: meta http- equiv=content-type content=text/html; charset=UTF-8 What could be wrong ? ? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: How to store very large files (e.g. PDFs)
If I understand correctly, there is currently a 50 MB file storage amount using the Blobstore API. More info can be found here: http://googleappengine.blogspot.com/2009/12/app-engine-sdk-130-released-including.html and here: http://code.google.com/appengine/docs/java/blobstore/ On Feb 1, 6:52 pm, Daniel C daniel.c...@gmail.com wrote: I need to allow users to upload very large/complex PDFs containing many images. The PDFs are typically 300MB each. Will this be possible with GAE? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: security in task queue servlets
Hi, That's a great suggestion -- thanks! I don't use task names for anything meaningful currently, so using it as a hashing channel should work great. Now that you've got me thinking about strong encryption vs. just checking local state, I wonder if maybe I should just sign the request payloads. (When I replied earlier, I had only read the first paragraph. I'm going to blame that on the small display on my phone. Meanwhile, my questions to Google still stands: do you make any guarantees about stripping client-provided headers, and do you have any plans to provide an API for checking the current request's roles in a way that works with task queues? -Patrick On Feb 1, 12:39 pm, Eli Jones eli.jo...@gmail.com wrote: That's why you have to use the TaskName and Hash method to verify that a Task was added to the queue in an orthodox manner. Unless someone knows the hash scheme you are using for TaskNames.. they will not be able to send in a (TaskName,getHash(TaskName)) pair that would get validated when the task started running.. Let me know if I'm not being clear in my suggestion.. this should do exactly what you want.. it would 100% prevent a person from accidentally visiting the URL and running a task (since it checks for the TaskName header).. and it would pretty much prevent some random person from trying to impersonate a task by sending a Post to the task URL.. They would have to know the correct hash to send along with the TaskName they are spoofing.. On Mon, Feb 1, 2010 at 3:29 PM, Patrick Linskey plins...@gmail.com wrote: (and Require Admin login isn't enough) That would be enough, but the only way to do that is to put the limitation in the web.xml file, which is pretty far away from the servlet in question. I want to make sure that someone doesn't accidentally mis-configure the web.xml file to remove the admin restriction. And I'd really rather not parse web.xml and apply all the appropriate rules to do so. Also, I don't know whether or not Google guarantees that the X- AppEngine-TaskName header is stripped from malicious incoming requests. It looks like it does, but that's just based on my observations. Thanks, -Patrick On Feb 1, 11:02 am, Eli Jones eli.jo...@gmail.com wrote: If you have a compelling reason for really locking down the task queue url (and Require Admin login isn't enough), you could create a mechanism that creates a task name for each queued task.. and the task verifies that its name is correct. You could have the task use the X-AppEngine-TaskName header to check its name.. So.. when you add a task to the queue.. you do something like this: taskName = getUniqueTaskName() nameHash = getHash(taskName) taskqueue.add(url = '/myTaskQueue', countdown = 0, name = taskName, params = {'nameHash' : nameHash}) and.. in the first part of the /myTaskQueue code.. you could have it verify that the 'nameHash' param is equal to getHash() of the TaskName you grab from the header.. On Sat, Jan 30, 2010 at 4:07 PM, Patrick Linskey plins...@gmail.com wrote: Hi, I'd like to programmatically ensure that my task queue servlets are only invoked via the task queue. I've got a security constraint in my web.xml, but I'd like to also check in code to avoid any potential mis- configuration in the future. Is there any supported means to do such a check? I tried looking at the contents of the HttpServletRequest (isUserInRole (), getAuthType(), getUserPrincipal(), getRemoteName()), to no avail. I also tried UserServiceFactory.getUserService().isAdmin(), but received an exception informing me that no user was logged in. I can see that there are a number of task queue-specific HTTP headers. Currently, I'm checking that X-AppEngine-TaskRetryCount is present, and if so, assuming that the request has come from the task queue and that it's therefore safe to process. Empirically, it looks like GAE strips out the X-AppEngine-TaskRetryCount header when I specify it in a curl-sourced request. Is this a safe assumption to rely on? Are there plans to document a reliable way to ensure servlet security in a task queue environment? Is there something else that I'm missing? Also, in an ideal world, it'd be nice if request.isUserInRole(admin) would return true at the appropriate times. Thanks, -Patrick -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.comgoogle-appengine%2Bunsubscrib e...@googlegroups.com . For more options, visit this group at
[google-appengine] Re: Android app to monitor appengine quotas
Currently my appengine instance is running in free mode so I don't know if I can test that. If it's in the admin console it shouldn't be too hard to add. On Feb 2, 4:07 am, Andrei gml...@gmail.com wrote: can u find way to get orders for paid Android apps from goog checkout? On Feb 1, 11:05 am, Hugo Visser botte...@gmail.com wrote: Hi, I've released a little android app to monitor your app engine quotas. It's called Engine Watch and is available from the android market. See my blog athttp://code.neenbedankt.comformore details. I hope it's useful to some of you too :) Hugo -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: How to deal with the _ToDatastoreError(err) problem?
You have to retry your operations. Take a look at this - http://appengine-cookbook.appspot.com/recipe/autoretry-datastore-timeouts On Feb 2, 6:10 am, lookon areyouloo...@gmail.com wrote: I meet this problem regularly. Anyone has an idea about this problem? Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: How to deal with the _ToDatastoreError(err) problem?
Thanks:-) Have a try. On Feb 2, 3:18 pm, Flips p...@script-network.com wrote: You have to retry your operations. Take a look at this -http://appengine-cookbook.appspot.com/recipe/autoretry-datastore-time... On Feb 2, 6:10 am, lookon areyouloo...@gmail.com wrote: I meet this problem regularly. Anyone has an idea about this problem? Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] How Google calculate datastore usage?
Here is my app: http://reader2twitter.appspot.com In the Dashboard, I see Total Stored Data : 0.70 of 1.00 GBytes. But in the Datastore Statistics, I see Size of all entities : 159 MBytes. I am confused. Can you give me some help? Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.