Re: [google-appengine] java.lang.NoClassDefFoundError: com/google/api/client/extensions/appengine/auth/oauth2/AbstractAppEngineAuthorizationCodeServlet
Hi Vinny Thanks for reply, I am not as much aware with maven.but here wat code I have is using maven only.And I given jar file manually with java build path. Is it the problem. *Amit Rai */ Google Developer 07th Floor,Tower-2, e -City, Electronic City, 1st Phase. Bangalore.560100 / India Extn: 7814, P: (+91) 8050637 782 E: amit@sial.com *www.sigmaaldrich.com*http://www.sigmaaldrich.com/ On Fri, Feb 28, 2014 at 1:15 PM, Vinny P vinny...@gmail.com wrote: On Thu, Feb 27, 2014 at 8:49 AM, Amit Rai amit@sial.com wrote: I have included all the dependency jar file and build path also.And this code is compiled successfully but while running in localhost its giving following error. org.mortbay.util.MultiException[java.lang.NoClassDefFoundError: com/google/api/client/extensions/appengine/auth/oauth2/AbstractAppEngineAuthorizationCodeCallbackServlet, java.lang.NoClassDefFoundError: com/google/api/client/extensions/appengine/auth/oauth2/AbstractAppEngineAuthorizationCodeServlet] Are you including the JAR dependencies via Maven or by manually placing the library JAR? If you're manually placing the JAR file, where did you originally download the file from? Can you open up the JAR and double check that the missing classes are included? - -Vinny P Technology Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to a topic in the Google Groups Google App Engine group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/W-OV_hvqcYs/unsubscribe . To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out. -- This message and any files transmitted with it are the property of Sigma-Aldrich Corporation, are confidential, and are intended solely for the use of the person or entity to whom this e-mail is addressed. If you are not one of the named recipient(s) or otherwise have reason to believe that you have received this message in error, please contact the sender and delete this message immediately from your computer. Any other use, retention, dissemination, forwarding, printing, or copying of this e-mail is strictly prohibited. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out. inline: image/gif
Re: [google-appengine] filter logs by user?
Thanks for your detailed answer, Vinny! We don't use Google authentication but our own, so I guess we are out of luck and have to resort to matching simple log messages. I'm wondering, regular expression wise, what would be the best way to mark each request with the user and company name. The goal would be to have zero accidental matches of other log messages. Maybe something like: *user:bob.harris company:google* So by using '' and '' as well as an identifier I could match requests made by a person or a company. Do you think there is a better way? On Friday, February 28, 2014 8:49:10 AM UTC+1, Vinny P wrote: On Wed, Feb 26, 2014 at 8:35 AM, stephanos stephan...@gmail.comjavascript: wrote: we would like to filter our logs by user as well as by company. The dashboard provides filtering by Labels and explains that these are regular expressions for filtering Apache Combined logs. It also lists a few valid labels including user and identd_user. How can I specify the values for those labels so that App Engine will parse them correctly? Hi Stephan, It's a fairly straightforward regular expression. Here's a simple example: I uploaded a Go application which uses Google Accounts to log in, then logged in with my *vinnyapp* Gmail account. Then I went into logs and searched for my login. Here's an example of the search (note that the *labels* radiobox is selected): http://imgur.com/1qeJ5tZ and here's an example of one of the logs that the search pulled up: http://imgur.com/XteD1FN . Note that my Google account is listed in the logs. If you're using the built-in Users service, the logs should automatically record the user logging in. If you're not, then you'll have to find a different way of recording the current user and inspecting the logs. - -Vinny P Technology Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Have anyone tried to port a java project with heavy framworks, such as struts and spring, to a pure jsp/servlet project?
Hi De Witte, That would happen if you are successful. Imagine building a complex app using app engine. It's pretty difficult to stay away from modules. For example: imagine a social network scenario, where you have long running tasks competing with frontend requests. then put britney spears sending a message and blasting thousands of tasks for push notifications. Modules have two good things: 1) different scaling and memory configuration. 2) separate the code into chunks of smaller packages. (you don't need spring for everything) Although, modules have big drawbacks: 1) impossible to develop without throwing the mouse on the wall sometimes. 2) hate maven integration. (half baked) 3) hate the appengine IDE integration (half baked) 4) hate the dispatch.xml (half baked) thanks rafa On Thu, Feb 27, 2014 at 9:02 AM, de Witte wd.dewi...@gmail.com wrote: We don't use modules, why should you? Do you have a requirement which cannot be done without them? Op donderdag 27 februari 2014 02:58:56 UTC+1 schreef Rafael Sanches: De Witte, Do you have a multi module project? How do you handle that without module? The eclipse plugin doesn't work with that setup. thanks rafa On Wed, Feb 26, 2014 at 3:45 PM, de Witte wd.de...@gmail.com wrote: We have been working with Java and GAE for several years now and so far no problems. AngularJs + ExtJS 4.0 is a good combo for the front end. Although I prefer GWT. Spring+Maven is a headache even without GAE. We have been avoiding it and doing fine. Op maandag 24 februari 2014 03:11:22 UTC+1 schreef Rafael Sanches: Not being updated doesn't mean it doesn't work properly. GAE + java is definitely frustrating, but some of your statements are a bit too broad ;) On Sun, Feb 23, 2014 at 12:44 AM, Tapir tapi...@gmail.com wrote: On Sunday, February 23, 2014 1:47:28 PM UTC+8, Rafael Sanches wrote: Can you try and let us all know? it may be the solution you're looking for. The Google App Engine module for Play has not been updated for 2 years! :( For another one, Objectify , https://code.google.com/p/play -framework-objectify/, the showcase on this page need 15 seconds to load. :( The decision is made, I will never use java for web development any more, for many reasons, such as not support hot deploy, large memory consumption, too many configurations, too many hidden elements, and too slow startup on GAE. On Thu, Feb 20, 2014 at 8:29 PM, Tapir tapi...@gmail.com wrote: On Friday, February 21, 2014 5:39:17 AM UTC+8, Rafael Sanches wrote: Have you tried playframework? They have optimizations to make your life easier when developing. My life is living hell right now when developing with maven and appengine. The play framework solves that by running their own thing. Read more: http://www.playframework.com/documentation/1.0/gaehttp://www.google.com/url?q=http%3A%2F%2Fwww.playframework.com%2Fdocumentation%2F1.0%2Fgaesa=Dsntz=1usg=AFQjCNFObfH5yu11uniN9ReUNUVgFDtbGg I don't think they can bypass the core GAE SDK lib. The problem of GAE Java is at the core GAE SDK lib. On Wed, Feb 19, 2014 at 7:24 PM, Tapir tapi...@gmail.com wrote: On Thursday, February 20, 2014 6:26:52 AM UTC+8, Emanuele Ziglioli wrote: Great stuff hope you don't move to Go too soon! Why? It is a way of no ways. On Thursday, 20 February 2014 03:52:33 UTC+13, Tapir wrote: On Tuesday, February 18, 2014 6:43:56 PM UTC+8, Tapir wrote: On Tuesday, February 18, 2014 4:56:51 PM UTC+8, Rafael Sanches wrote: very interesting... can you tell me how you did that? in a maven build or only via eclipse? neither, I did it manually. ok, the new test result: 1. if I remove all jar files from the war/WEB-INF/lib, the warmup time is about 2.3 seconds. 2. if I put the only the core app engine sdk jar file in war/WEB-INF/lib, but doesn't reference it, the warmup time is about 2.9 seconds. 3. if I put DatastoreService datastore = DatastoreServiceFactory.getDatastoreService(); in the jsp file but do nothing eslse, the warmup time is 3.5 seconds 4. if I put a query in the default jsp file, the warmup time is 4.7 seconds. 5. if I convert query result as list, the warmup time is 6 seconds. made some new tests today. The results are some different. === First, the impact of number of lib jar files: 1. no lib jar files, warm up time = 3 seconds (slower than yesterday) 2. one core lib jar file, no refs, warm up time = 5 seconds (slower than yesterday) 3. four core lib jar files (the ones under sdk user folder), no refs, warm up time = 5.7 seconds 4. fifty-four jar files (used in my projects) , no refs, warm up time = 6.2 seconds So, obviously, the core GAE SDK lib files have a big impact on the warm up time. Second, number of references of core sdk lib. 1. no refs, 3.5 seconds 2. fewer refs, 4.4 seconds, by adding
Re: [google-appengine] Have anyone tried to port a java project with heavy framworks, such as struts and spring, to a pure jsp/servlet project?
sorry, my last phrase was confusing, I meant: That would happen if you are successful building a complex architecture using app engine On Fri, Feb 28, 2014 at 1:00 AM, Rafael mufumb...@gmail.com wrote: Hi De Witte, That would happen if you are successful. Imagine building a complex app using app engine. It's pretty difficult to stay away from modules. For example: imagine a social network scenario, where you have long running tasks competing with frontend requests. then put britney spears sending a message and blasting thousands of tasks for push notifications. Modules have two good things: 1) different scaling and memory configuration. 2) separate the code into chunks of smaller packages. (you don't need spring for everything) Although, modules have big drawbacks: 1) impossible to develop without throwing the mouse on the wall sometimes. 2) hate maven integration. (half baked) 3) hate the appengine IDE integration (half baked) 4) hate the dispatch.xml (half baked) thanks rafa On Thu, Feb 27, 2014 at 9:02 AM, de Witte wd.dewi...@gmail.com wrote: We don't use modules, why should you? Do you have a requirement which cannot be done without them? Op donderdag 27 februari 2014 02:58:56 UTC+1 schreef Rafael Sanches: De Witte, Do you have a multi module project? How do you handle that without module? The eclipse plugin doesn't work with that setup. thanks rafa On Wed, Feb 26, 2014 at 3:45 PM, de Witte wd.de...@gmail.com wrote: We have been working with Java and GAE for several years now and so far no problems. AngularJs + ExtJS 4.0 is a good combo for the front end. Although I prefer GWT. Spring+Maven is a headache even without GAE. We have been avoiding it and doing fine. Op maandag 24 februari 2014 03:11:22 UTC+1 schreef Rafael Sanches: Not being updated doesn't mean it doesn't work properly. GAE + java is definitely frustrating, but some of your statements are a bit too broad ;) On Sun, Feb 23, 2014 at 12:44 AM, Tapir tapi...@gmail.com wrote: On Sunday, February 23, 2014 1:47:28 PM UTC+8, Rafael Sanches wrote: Can you try and let us all know? it may be the solution you're looking for. The Google App Engine module for Play has not been updated for 2 years! :( For another one, Objectify , https://code.google.com/p/play -framework-objectify/, the showcase on this page need 15 seconds to load. :( The decision is made, I will never use java for web development any more, for many reasons, such as not support hot deploy, large memory consumption, too many configurations, too many hidden elements, and too slow startup on GAE. On Thu, Feb 20, 2014 at 8:29 PM, Tapir tapi...@gmail.com wrote: On Friday, February 21, 2014 5:39:17 AM UTC+8, Rafael Sanches wrote: Have you tried playframework? They have optimizations to make your life easier when developing. My life is living hell right now when developing with maven and appengine. The play framework solves that by running their own thing. Read more: http://www.playframework.com/documentation/1.0/gaehttp://www.google.com/url?q=http%3A%2F%2Fwww.playframework.com%2Fdocumentation%2F1.0%2Fgaesa=Dsntz=1usg=AFQjCNFObfH5yu11uniN9ReUNUVgFDtbGg I don't think they can bypass the core GAE SDK lib. The problem of GAE Java is at the core GAE SDK lib. On Wed, Feb 19, 2014 at 7:24 PM, Tapir tapi...@gmail.com wrote: On Thursday, February 20, 2014 6:26:52 AM UTC+8, Emanuele Ziglioli wrote: Great stuff hope you don't move to Go too soon! Why? It is a way of no ways. On Thursday, 20 February 2014 03:52:33 UTC+13, Tapir wrote: On Tuesday, February 18, 2014 6:43:56 PM UTC+8, Tapir wrote: On Tuesday, February 18, 2014 4:56:51 PM UTC+8, Rafael Sanches wrote: very interesting... can you tell me how you did that? in a maven build or only via eclipse? neither, I did it manually. ok, the new test result: 1. if I remove all jar files from the war/WEB-INF/lib, the warmup time is about 2.3 seconds. 2. if I put the only the core app engine sdk jar file in war/WEB-INF/lib, but doesn't reference it, the warmup time is about 2.9 seconds. 3. if I put DatastoreService datastore = DatastoreServiceFactory.getDatastoreService(); in the jsp file but do nothing eslse, the warmup time is 3.5 seconds 4. if I put a query in the default jsp file, the warmup time is 4.7 seconds. 5. if I convert query result as list, the warmup time is 6 seconds. made some new tests today. The results are some different. === First, the impact of number of lib jar files: 1. no lib jar files, warm up time = 3 seconds (slower than yesterday) 2. one core lib jar file, no refs, warm up time = 5 seconds (slower than yesterday) 3. four core lib jar files (the ones under sdk user folder), no refs, warm up time = 5.7 seconds 4. fifty-four jar files (used in my projects) , no refs, warm up time = 6.2 seconds So, obviously,
Re: [google-appengine] Have anyone tried to port a java project with heavy framworks, such as struts and spring, to a pure jsp/servlet project?
This is pretty much our experience of modules too - especially the maven integration feels half-baked. But we have a couple batch operations that run at midnight that require extra memory, and it sucks to have to pay for that all day. I think I've been one of the loudest people complaining about the java/cold start/user facing issue, and yes, it's still a concern. Every so often a request appears to hang, and that's not really acceptable, especially for our low-traffic/high-value-per-request app. But I will also say that it doesn't seem like it has been as much of a problem as it has been in the past - maybe something got smarter? I don't notice the hung requests much anymore, and it's been forever since we've experienced the startup latency 60s thus instances can't get off the ground problem. I would describe our java app as a success story, even if the history has not been perfect. Neither Heroku nor CloudFoundry has been perfect either. Jeff On Fri, Feb 28, 2014 at 1:00 AM, Rafael mufumb...@gmail.com wrote: Hi De Witte, That would happen if you are successful. Imagine building a complex app using app engine. It's pretty difficult to stay away from modules. For example: imagine a social network scenario, where you have long running tasks competing with frontend requests. then put britney spears sending a message and blasting thousands of tasks for push notifications. Modules have two good things: 1) different scaling and memory configuration. 2) separate the code into chunks of smaller packages. (you don't need spring for everything) Although, modules have big drawbacks: 1) impossible to develop without throwing the mouse on the wall sometimes. 2) hate maven integration. (half baked) 3) hate the appengine IDE integration (half baked) 4) hate the dispatch.xml (half baked) thanks rafa On Thu, Feb 27, 2014 at 9:02 AM, de Witte wd.dewi...@gmail.com wrote: We don't use modules, why should you? Do you have a requirement which cannot be done without them? Op donderdag 27 februari 2014 02:58:56 UTC+1 schreef Rafael Sanches: De Witte, Do you have a multi module project? How do you handle that without module? The eclipse plugin doesn't work with that setup. thanks rafa On Wed, Feb 26, 2014 at 3:45 PM, de Witte wd.de...@gmail.com wrote: We have been working with Java and GAE for several years now and so far no problems. AngularJs + ExtJS 4.0 is a good combo for the front end. Although I prefer GWT. Spring+Maven is a headache even without GAE. We have been avoiding it and doing fine. Op maandag 24 februari 2014 03:11:22 UTC+1 schreef Rafael Sanches: Not being updated doesn't mean it doesn't work properly. GAE + java is definitely frustrating, but some of your statements are a bit too broad ;) On Sun, Feb 23, 2014 at 12:44 AM, Tapir tapi...@gmail.com wrote: On Sunday, February 23, 2014 1:47:28 PM UTC+8, Rafael Sanches wrote: Can you try and let us all know? it may be the solution you're looking for. The Google App Engine module for Play has not been updated for 2 years! :( For another one, Objectify , https://code.google.com/p/play-framework-objectify/, the showcase on this page need 15 seconds to load. :( The decision is made, I will never use java for web development any more, for many reasons, such as not support hot deploy, large memory consumption, too many configurations, too many hidden elements, and too slow startup on GAE. On Thu, Feb 20, 2014 at 8:29 PM, Tapir tapi...@gmail.com wrote: On Friday, February 21, 2014 5:39:17 AM UTC+8, Rafael Sanches wrote: Have you tried playframework? They have optimizations to make your life easier when developing. My life is living hell right now when developing with maven and appengine. The play framework solves that by running their own thing. Read more: http://www.playframework.com/documentation/1.0/gae I don't think they can bypass the core GAE SDK lib. The problem of GAE Java is at the core GAE SDK lib. On Wed, Feb 19, 2014 at 7:24 PM, Tapir tapi...@gmail.com wrote: On Thursday, February 20, 2014 6:26:52 AM UTC+8, Emanuele Ziglioli wrote: Great stuff hope you don't move to Go too soon! Why? It is a way of no ways. On Thursday, 20 February 2014 03:52:33 UTC+13, Tapir wrote: On Tuesday, February 18, 2014 6:43:56 PM UTC+8, Tapir wrote: On Tuesday, February 18, 2014 4:56:51 PM UTC+8, Rafael Sanches wrote: very interesting... can you tell me how you did that? in a maven build or only via eclipse? neither, I did it manually. ok, the new test result: 1. if I remove all jar files from the war/WEB-INF/lib, the warmup time is about 2.3 seconds. 2. if I put the only the core app engine sdk jar file in war/WEB-INF/lib, but doesn't reference it, the warmup time is about 2.9 seconds. 3. if I put DatastoreService datastore = DatastoreServiceFactory.getDatastoreService(); in the jsp file but do nothing eslse, the
[google-appengine] Google app engine Spring default-lazy-init=true does not work.
Hi All, I have tried placing the lazy-init=true in bean and default-lazy-init=true beans level but does not stop spanning for the controller mapping. Not sure i am doing something wrong looks like a easy but could not make it work. spring.xml file. beans xmlns=http://www.springframework.org/schema/beans; xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance; xsi:schemaLocation=http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd; default-lazy-init=true bean class= org.springframework.web.servlet.handler.SimpleUrlHandlerMapping property name=mappings props prop key=/hello/*aController/prop prop key=/drive/*driveController/prop prop key=/sheet/*spreedsheet/prop prop key=/myjsonp/*myjsonp/prop /props /property /bean bean id=aController class= com.test.bootstrap.controller.TimeConverterController lazy-init=true / bean id=driveController class= com.test.bootstrap.controller.GoogleDriveIntegration / bean id=spreedsheet class= com.test.bootstrap.controller.GoogleSpreedSheetController / bean id=myjsonp class=com.test.bootstrap.controller.JsonPController / bean id=employeeBean class=com.test.bootstrap.main.Employee lazy-init =true / bean id=addressBean class=com.test.bootstrap.main.Address lazy-init= true / bean id=viewResolver class= org.springframework.web.servlet.view.ResourceBundleViewResolver property name=basename value=views/ /bean /beans Thanks, Pravanajan -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Have anyone tried to port a java project with heavy framworks, such as struts and spring, to a pure jsp/servlet project?
Hi Jeff, I would describe mine as success too. Although, the only point of using appengine was the ease of development and that it would scale as we grew. 1) Ease of development is out of question with a big code base. 2) scale comes at the highest cost. Thanks Rafa On Feb 28, 2014 2:23 AM, Jeff Schnitzer j...@infohazard.org wrote: This is pretty much our experience of modules too - especially the maven integration feels half-baked. But we have a couple batch operations that run at midnight that require extra memory, and it sucks to have to pay for that all day. I think I've been one of the loudest people complaining about the java/cold start/user facing issue, and yes, it's still a concern. Every so often a request appears to hang, and that's not really acceptable, especially for our low-traffic/high-value-per-request app. But I will also say that it doesn't seem like it has been as much of a problem as it has been in the past - maybe something got smarter? I don't notice the hung requests much anymore, and it's been forever since we've experienced the startup latency 60s thus instances can't get off the ground problem. I would describe our java app as a success story, even if the history has not been perfect. Neither Heroku nor CloudFoundry has been perfect either. Jeff On Fri, Feb 28, 2014 at 1:00 AM, Rafael mufumb...@gmail.com wrote: Hi De Witte, That would happen if you are successful. Imagine building a complex app using app engine. It's pretty difficult to stay away from modules. For example: imagine a social network scenario, where you have long running tasks competing with frontend requests. then put britney spears sending a message and blasting thousands of tasks for push notifications. Modules have two good things: 1) different scaling and memory configuration. 2) separate the code into chunks of smaller packages. (you don't need spring for everything) Although, modules have big drawbacks: 1) impossible to develop without throwing the mouse on the wall sometimes. 2) hate maven integration. (half baked) 3) hate the appengine IDE integration (half baked) 4) hate the dispatch.xml (half baked) thanks rafa On Thu, Feb 27, 2014 at 9:02 AM, de Witte wd.dewi...@gmail.com wrote: We don't use modules, why should you? Do you have a requirement which cannot be done without them? Op donderdag 27 februari 2014 02:58:56 UTC+1 schreef Rafael Sanches: De Witte, Do you have a multi module project? How do you handle that without module? The eclipse plugin doesn't work with that setup. thanks rafa On Wed, Feb 26, 2014 at 3:45 PM, de Witte wd.de...@gmail.com wrote: We have been working with Java and GAE for several years now and so far no problems. AngularJs + ExtJS 4.0 is a good combo for the front end. Although I prefer GWT. Spring+Maven is a headache even without GAE. We have been avoiding it and doing fine. Op maandag 24 februari 2014 03:11:22 UTC+1 schreef Rafael Sanches: Not being updated doesn't mean it doesn't work properly. GAE + java is definitely frustrating, but some of your statements are a bit too broad ;) On Sun, Feb 23, 2014 at 12:44 AM, Tapir tapi...@gmail.com wrote: On Sunday, February 23, 2014 1:47:28 PM UTC+8, Rafael Sanches wrote: Can you try and let us all know? it may be the solution you're looking for. The Google App Engine module for Play has not been updated for 2 years! :( For another one, Objectify , https://code.google.com/p/play-framework-objectify/, the showcase on this page need 15 seconds to load. :( The decision is made, I will never use java for web development any more, for many reasons, such as not support hot deploy, large memory consumption, too many configurations, too many hidden elements, and too slow startup on GAE. On Thu, Feb 20, 2014 at 8:29 PM, Tapir tapi...@gmail.com wrote: On Friday, February 21, 2014 5:39:17 AM UTC+8, Rafael Sanches wrote: Have you tried playframework? They have optimizations to make your life easier when developing. My life is living hell right now when developing with maven and appengine. The play framework solves that by running their own thing. Read more: http://www.playframework.com/documentation/1.0/gae I don't think they can bypass the core GAE SDK lib. The problem of GAE Java is at the core GAE SDK lib. On Wed, Feb 19, 2014 at 7:24 PM, Tapir tapi...@gmail.com wrote: On Thursday, February 20, 2014 6:26:52 AM UTC+8, Emanuele Ziglioli wrote: Great stuff hope you don't move to Go too soon! Why? It is a way of no ways. On Thursday, 20 February 2014 03:52:33 UTC+13, Tapir wrote: On Tuesday, February 18, 2014 6:43:56 PM UTC+8, Tapir wrote: On Tuesday, February 18, 2014 4:56:51 PM UTC+8, Rafael Sanches wrote: very
Re: [google-appengine] Re: Best way to update 400,000 entities at once?
This is exactly the sort of task the MapReduce was meant for. It should be really a lot easier than managing the partitioning, error recovery, etc yourself. Take a look at our new docs: https://developers.google.com/appengine/docs/java/dataprocessing/mapreduce_library hopefully they should make it less overwhelming. On Thu, Feb 27, 2014 at 5:04 AM, de Witte wd.dewi...@gmail.com wrote: Use a backend instance and keep it running until done. Or Use two tasks. One for retrieving 1000 keys at the time and a second one to update the entities in a batch of 1000. Done it for 300.000 entities in less than a 20 mins. ~300 tasks Op vrijdag 7 februari 2014 22:43:33 UTC+1 schreef Keith Lea: Hi everyone, I'm a long time App Engine user for my app's backend, but I'm really still a novice about the datastore. I'd like to add a new property (and index) for all entities of a certain type. I have about 400,000 of this type of entity in the datastore, and I'd like to load each one, add a property, and save it back to the datastore. 400,000 times. This will obviously take a long time, so I'd really like to split it up into ~100 tasks that each take 1/100th of the entities (~4,000 entities) and perform this operation. But I really don't know how to do this using queries, and the Java MapReduce library is overwhelmingly complicated. So how can I create 100 tasks that each take a unique chunk of the entities to operate on? Is this called sharding? Is there a way for a task to say give me entity #200,000 thru #204,000? (My entity's keys are strings, which were generated by my application and generally look like 928348-com.example-iOS.) I'm using Java and Objectify btw. Thanks for any help or guidance!! Keith -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: 1.9.0 Pre-Release SDKs are now available.
The new docs are here: https://developers.google.com/appengine/docs/java/dataprocessing/ These are a replacement for the ones on code.google.com Kirill: Your compatibility question is addressed here: https://developers.google.com/appengine/docs/java/dataprocessing/mapreduce_update On Mon, Feb 10, 2014 at 4:50 PM, Kirill Lebedev k.lebe...@electionear.comwrote: Thanks for this information. Actually one of our appengines is already on 1.9.0. And I have a compatibility question based on your notes: We are actively using Java version of MapReduce library. We are still on 0.2 version (Blobstore-based intermediate storage and InMemory Shuffling). We tried to update our system to 0.3 and 0.4 (current SVN state) and it was not successful. More over 0.4 SVN version has import com.google.appengine.api.labs.modules.ModulesService; and import com.google.appengine.api.labs.modules.ModulesServiceFactory; imports in MapReduceJob.java. So how this release will affect MapReduce compatibility? Will 0.2 version still work on 1.9.0. Do you have any plans to publicly release 0.4 version of Java MapReduce that will reflect 1.9.0 changes? It is critical cause our code relies on that libraries. Thanks, Kirill Lebedev вторник, 4 февраля 2014 г., 17:43:41 UTC-8 пользователь Richmond Manzana написал: We want to inform you that the pre-release SDKs for Python, PHP and Java are now available. As previously announcedhttp://google-opensource.blogspot.com/2013/05/a-change-to-google-code-download-service.htmlin a Google code site announcement, new App Engine Binaries are no longer available at: http://code.google.com/p/googleappengine/downloads/list Older binaries will remain available at the code.google.com site. 1.9.0 Pre-release SDKs are now available at these links: App Engine 1.9.0 Java prerelease SDKhttp://commondatastorage.googleapis.com/appengine-sdks%2Ffeatured%2Fappengine-java-sdk-1.9.0_prerelease.zip App Engine 1.9.0 Python prerelease SDKhttp://commondatastorage.googleapis.com/appengine-sdks%2Ffeatured%2Fgoogle_appengine-1.9.0_prerelease.zip App Engine 1.9.0 PHP prerelease SDKhttp://commondatastorage.googleapis.com/appengine-sdks%2Ffeatured%2Fgoogle_appengine-php-sdk-1.9.0_prerelease.zip In the future, please look forward to the finding the latest binaries at https://developers.google.com/appengine/downloads Also, please see the pre-release notes below. Cheers, Richmond Manzana Technical Program Manager Google App Engine App Engine SDK - Pre-Release Notes Version 1.9.0 Python PHP == - New App Engine Application Identifiers must now start with a letter, in addition to the existing requirements that the identifier be 6-30 characters which are letters, numbers, and hyphens, and not start or end with a hyphen. Python == - The size limit on the Search API is now computed and enforced on a per-index basis, rather than for the app as a whole. The per-index limit is now 10GB. There is no fixed limit on the number of indexes, or on the total amount of Search API storage an application may use. - Users now have the ability to embed images in emails via the Content-Id attachment header. https://code.google.com/p/googleappengine/issues/detail?id=965 https://code.google.com/p/googleappengine/issues/detail?id=10503 - Fixed an issue with NDB backup/restore corrupting certain compressed entities. https://code.google.com/p/googleappengine/issues/detail?id=8599 PHP == - The PHP interpreter has been upgraded from PHP 5.4.19 to PHP 5.4.22. - Autoloading is now available in the SDK so developers will no longer need to explicitly require SDK files. - Expanded php.ini setting google_appengine.allow_include_gs_buckets to allow a path filter be included for improved security. - A warning message now appears if an application moves a user uploaded file to a Google Cloud Storage bucket/path. This is due to the fact that code may be included and lead to a local file inclusion vulnerability. - Added API functions CloudStorageTools::getMetadata() and CloudStorageTools::getContentType() for retrieving the metadata and content type of Google Cloud Storage objects. https://code.google.com/p/googleappengine/issues/detail?id=10182 - Fixed an issue with GCS folders not displaying correctly in Developers Console. - Fixed an issue with PHP_SELF and SCRIPT_NAME not being implemented correctly. https://code.google.com/p/googleappengine/issues/detail?id=9989 https://code.google.com/p/googleappengine/issues/detail?id=10478 Java == - Java 6 applications cannot be deployed to Google App Engine from any version of the SDK. Existing Java 6 applications will continue to run. If you are still relying on a Java 6 application in Google App Engine, we strongly encourage you to start
[google-appengine] Show and Tell: Plexi Voice a Windows Phone 8 App with AppEngine backend.
We just put out our Open Beta of Plexi Voice. A GoogleNow/Siri like Conversational Search product. Yes it is for Windows Phone. Yes, it uses Google AppEngine for all of its backend. http://www.windowsphone.com/en-us/store/app/plexivoice/9c09f697-657d-4613-ae 67-7e787121d0a4 The code is ours, full stack, written in Python. We aren't entirely converted to modules, but pretty close. We do some cross app calls, but try to do as much as possible as Modules. We'd move to doing everything in Modules but the cost of migrating data from one App to the other would be like $10k and it isn't worth it at this point. Several people in the past have asked about using Python's Natural Language Tool Kit on AppEngine. 2 years ago we did make a version that fit in an F4 instance, but we don't use that anymore, we have our own NLP Engine, that will run in an F1. For the most part all of our code runs in an F2. Some of our bigger things related to large dictionaries doesn't fit, so we run F2's instead of F1s. We average about 2000 instance hours a day. We have 2 Terabytes of DataStore. For our search engine we crawl about 2 Million pages a day of the Internet. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Show and Tell: Plexi Voice a Windows Phone 8 App with AppEngine backend.
It sounds like a lot of hacks to make it fit in appengine. Can you tell me if you would have developed and maintained this in half of the time if you had just 4 boxes running with 64gb of ram? On Fri, Feb 28, 2014 at 2:19 PM, Brandon Wirtz drak...@digerat.com wrote: We just put out our Open Beta of Plexi Voice. A GoogleNow/Siri like Conversational Search product. Yes it is for Windows Phone. Yes, it uses Google AppEngine for all of its backend. http://www.windowsphone.com/en-us/store/app/plexivoice/9c09f697-657d-4613-ae67-7e787121d0a4 The code is ours, full stack, written in Python. We aren't entirely converted to modules, but pretty close. We do some cross app calls, but try to do as much as possible as Modules. We'd move to doing everything in Modules but the cost of migrating data from one App to the other would be like $10k and it isn't worth it at this point. Several people in the past have asked about using Python's Natural Language Tool Kit on AppEngine. 2 years ago we did make a version that fit in an F4 instance, but we don't use that anymore, we have our own NLP Engine, that will run in an F1. For the most part all of our code runs in an F2. Some of our bigger things related to large dictionaries doesn't fit, so we run F2's instead of F1s. We average about 2000 instance hours a day. We have 2 Terabytes of DataStore. For our search engine we crawl about 2 Million pages a day of the Internet. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
RE: [google-appengine] Show and Tell: Plexi Voice a Windows Phone 8 App with AppEngine backend.
No hacks. We didn't plan to run NLTK forever. We have peaks and valleys and no 4 boxes would not handle the amount of crawling we do. We wouldn't have stuck with the platform if we thought it was slowing down development. It saved us an Infrastructure guy which was huge for a startup. Over 2 years that buys a LOT of hosting. Also with AppEngine we haven't had to worry much about downtime. The outages we have had were reasonably short, and I don't have to get out of bed to deal with. From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Rafael Sent: Friday, February 28, 2014 3:47 PM To: google-appengine Subject: Re: [google-appengine] Show and Tell: Plexi Voice a Windows Phone 8 App with AppEngine backend. It sounds like a lot of hacks to make it fit in appengine. Can you tell me if you would have developed and maintained this in half of the time if you had just 4 boxes running with 64gb of ram? On Fri, Feb 28, 2014 at 2:19 PM, Brandon Wirtz drak...@digerat.com mailto:drak...@digerat.com wrote: We just put out our Open Beta of Plexi Voice. A GoogleNow/Siri like Conversational Search product. Yes it is for Windows Phone. Yes, it uses Google AppEngine for all of its backend. http://www.windowsphone.com/en-us/store/app/plexivoice/9c09f697-657d-4613-ae 67-7e787121d0a4 The code is ours, full stack, written in Python. We aren't entirely converted to modules, but pretty close. We do some cross app calls, but try to do as much as possible as Modules. We'd move to doing everything in Modules but the cost of migrating data from one App to the other would be like $10k and it isn't worth it at this point. Several people in the past have asked about using Python's Natural Language Tool Kit on AppEngine. 2 years ago we did make a version that fit in an F4 instance, but we don't use that anymore, we have our own NLP Engine, that will run in an F1. For the most part all of our code runs in an F2. Some of our bigger things related to large dictionaries doesn't fit, so we run F2's instead of F1s. We average about 2000 instance hours a day. We have 2 Terabytes of DataStore. For our search engine we crawl about 2 Million pages a day of the Internet. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com mailto:google-appengine+unsubscr...@googlegroups.com . To post to this group, send email to google-appengine@googlegroups.com mailto:google-appengine@googlegroups.com . Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com mailto:google-appengine+unsubscr...@googlegroups.com . To post to this group, send email to google-appengine@googlegroups.com mailto:google-appengine@googlegroups.com . Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Log Viewer performance issues
The Log Viewer fetches all the logs for the requests it renders This causes performance issues if you are running tasks with informative logs I've created a feature request about this issue, please star it if you are experiencing browser performance issues too https://code.google.com/p/googleappengine/issues/detail?id=10639 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: 1.8.8 Pre-Release SDKs are now available.
Hi Vinny, I am simply trying to setup and use a soap client and am having the issues posted https://code.google.com/p/googleappengine/issues/detail?id=9858 If we have socket support now, any ideas as to why this does not work? As far as I am aware no one has been able to successfully use soap with GAE. On Wednesday, February 12, 2014 2:59:53 PM UTC+10, Vinny P wrote: On Sat, Feb 8, 2014 at 1:01 AM, cr...@portical.com.au javascript: wrote: Where do I find the Sockets API? I do not see it listed under APIs in the Developers Console. I have billing enabled. The Sockets API isn't a service like the other Google Cloud Platform APIs are (i.e. images, task queue, etc). Having the Sockets API merely means that you can now use low level sockets to connect to external servers rather than using URL Fetch. Since you said you have billing enabled, there's nothing left for you to do. Just start writing socket-using code into your application and it'll work. - -Vinny P Technology Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Announcing a credit for App Engine applications with new custom domains
Thanks for the update. Unfortunately it was pretty much the same message as 4 months ago. If it's not announced by June 26th this year, that's an embarrassment right there. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: 1.8.8 Pre-Release SDKs are now available.
I have successfully used SOAP as a client on appengine using python. I use SUDs, and pre-process the WSDL files effectively pre-loading the file cache. Had to patch SUDS for this to work. This is used to get dynamic shipping quotes from Temando via their SOAP service. T On Saturday, March 1, 2014 7:53:14 AM UTC+8, cr...@portical.com.au wrote: Hi Vinny, I am simply trying to setup and use a soap client and am having the issues posted https://code.google.com/p/googleappengine/issues/detail?id=9858 If we have socket support now, any ideas as to why this does not work? As far as I am aware no one has been able to successfully use soap with GAE. On Wednesday, February 12, 2014 2:59:53 PM UTC+10, Vinny P wrote: On Sat, Feb 8, 2014 at 1:01 AM, cr...@portical.com.au wrote: Where do I find the Sockets API? I do not see it listed under APIs in the Developers Console. I have billing enabled. The Sockets API isn't a service like the other Google Cloud Platform APIs are (i.e. images, task queue, etc). Having the Sockets API merely means that you can now use low level sockets to connect to external servers rather than using URL Fetch. Since you said you have billing enabled, there's nothing left for you to do. Just start writing socket-using code into your application and it'll work. - -Vinny P Technology Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: 1.8.8 Pre-Release SDKs are now available.
On Fri, Feb 28, 2014 at 5:53 PM, cr...@portical.com.au wrote: I am simply trying to setup and use a soap client and am having the issues posted https://code.google.com/p/googleappengine/issues/detail?id=9858 If we have socket support now, any ideas as to why this does not work? As far as I am aware no one has been able to successfully use soap with GAE. Sockets and SOAP are two entirely different concepts. You need to isolate whether the issue is with sockets or the soap client you're using. First of all, check to see if sockets are working properly by using the example sockets implementation: https://github.com/GoogleCloudPlatform/appengine-sockets-python-java-go If that works, then sockets are working correctly on your app ID. From there you can debug the SOAP client and see why it's not working. - -Vinny P Technology Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] filter logs by user?
On Fri, Feb 28, 2014 at 2:44 AM, stephanos stephan.beh...@gmail.com wrote: Thanks for your detailed answer, Vinny! We don't use Google authentication but our own, so I guess we are out of luck and have to resort to matching simple log messages. I'm wondering, regular expression wise, what would be the best way to mark each request with the user and company name. The goal would be to have zero accidental matches of other log messages. Maybe something like: *user:bob.harris company:google* So by using '' and '' as well as an identifier I could match requests made by a person or a company. Do you think there is a better way? No, that sounds like a good plan. The only thing I would add is to make sure the identifier is on it's own log line and perhaps prefix the line with a special, reserved token. For example, you could write this into logging: *ID Token - User:bob Company:google * Then when you parse the logs using your analyzer, you can first look for log lines prefixed with *ID Token*. If you match the prefix, then you know that this is an ID, and then you can parse out the username and the company. This does two things: it simplifies the code in your matcher and also provides a layer of error-checking: without the prefixing token check, a regex might accidentally match some other log information that isn't intended to be ID data. - -Vinny P Technology Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Show and Tell: Plexi Voice a Windows Phone 8 App with AppEngine backend.
On Fri, Feb 28, 2014 at 4:19 PM, Brandon Wirtz drak...@digerat.com wrote: We just put out our Open Beta of Plexi Voice. A GoogleNow/Siri like Conversational Search product. Yes it is for Windows Phone. Yes, it uses Google AppEngine for all of its backend. http://www.windowsphone.com/en-us/store/app/plexivoice/9c09f697-657d-4613-ae67-7e787121d0a4 Hi Brandon, I'm curious what you think about Cortana, Microsoft's version of Siri due out soonhttp://techcrunch.com/2014/02/20/microsofts-cortana-on-windows-8-1-said-to-resemble-siri-replace-bing-system-search/. How do you think it'll stack up to Plexi? Plexi Voice looks terrific; I'll need to borrow my friend's Windows phone and try it out :-) - -Vinny P Technology Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] java.lang.NoClassDefFoundError: com/google/api/client/extensions/appengine/auth/oauth2/AbstractAppEngineAuthorizationCodeServlet
On Fri, Feb 28, 2014 at 2:34 AM, Amit Rai amit@sial.com wrote: I am not as much aware with maven.but here wat code I have is using maven only. I would try disconnecting the project from Maven and manually adding the JAR libraries specified in the Maven build to your project/build path. Somewhere along the line, the appropriate libraries aren't being bundled into the WAR. You'll have to pin down where in the build process that's happening. The easiest way to check Maven's work is to temporarily bypass it by manually supplying the libraries. - -Vinny P Technology Media Advisor Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.