[appengine-java] System.out.println behavior in production
I'm having difficulty debugging my application in production. According to the GAE docs, System.out.println output is directed as log level INFO. My home page generates such output when debugging in my test local environment. But, my deployment to production generates NO output. Using the Admin Console->Log I can see my Requests but NOT my standard output from println(). How can I see this output? Is it just NOT being flushed or is there a config for this? Thanks, Diana --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: System.out.println behavior in production
Yes, I tried all the levels in the Console. It must be the properties
file. I noticed the plugin created the log4j.properties in the src
folder and the contents are below. How do I set the default level in
this file? Thanks.
# A default log4j configuration for log4j users.
#
# To use this configuration, deploy it into your application's WEB-INF/
classes
# directory. You are also encouraged to edit it as you like.
# Configure the console as our one appender
log4j.appender.A1=org.apache.log4j.ConsoleAppender
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%d{HH:mm:ss,SSS} %-5p [%c]
- %m%n
# tighten logging on the DataNucleus Categories
log4j.category.DataNucleus.JDO=WARN, A1
log4j.category.DataNucleus.Persistence=WARN, A1
log4j.category.DataNucleus.Cache=WARN, A1
log4j.category.DataNucleus.MetaData=WARN, A1
log4j.category.DataNucleus.General=WARN, A1
log4j.category.DataNucleus.Utility=WARN, A1
log4j.category.DataNucleus.Transaction=WARN, A1
log4j.category.DataNucleus.Datastore=WARN, A1
log4j.category.DataNucleus.ClassLoading=WARN, A1
log4j.category.DataNucleus.Plugin=WARN, A1
log4j.category.DataNucleus.ValueGeneration=WARN, A1
log4j.category.DataNucleus.Enhancer=WARN, A1
log4j.category.DataNucleus.SchemaTool=WARN, A1
On Sep 18, 10:13 pm, Don Schwarz wrote:
> Are you switch the logging level in the Admin Console to Info? By default
> it only shows errors.
>
> Do you have a logging.properties file? If so, what is your default .level
> set to? System.out would be logged at INFO, so if you are specifying this
> make sure it is INFO or lower.
> Finally, I would recommend using java.util.logging instead of System.out.
> It's a lot more flexible and is very easy to use.
>
> On Fri, Sep 18, 2009 at 10:09 PM, Diana Cruise
> wrote:
>
>
>
>
>
> > I'm having difficulty debugging my application in production.
> > According to the GAE docs, System.out.println output is directed as
> > log level INFO. My home page generates such output when debugging in
> > my test local environment. But, my deployment to production generates
> > NO output. Using the Admin Console->Log I can see my Requests but NOT
> > my standard output from println(). How can I see this output? Is it
> > just NOT being flushed or is there a config for this?
>
> > Thanks, Diana- Hide quoted text -
>
> - Show quoted text -
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---
[appengine-java] Re: System.out.println behavior in production
I found the Logger docs for GAE at:
http://code.google.com/appengine/docs/java/runtime.html#Logging
I'm definitely getting output now from Stdout but I can't seem to get
logger.log() to work. I'll fiddle with it and log a separate issue if
I can't get it to work. Thanks for the help Don!
On Sep 18, 10:27 pm, Diana Cruise wrote:
> Yes, I tried all the levels in the Console. It must be the properties
> file. I noticed the plugin created the log4j.properties in the src
> folder and the contents are below. How do I set the default level in
> this file? Thanks.
>
> # A default log4j configuration for log4j users.
> #
> # To use this configuration, deploy it into your application's WEB-INF/
> classes
> # directory. You are also encouraged to edit it as you like.
>
> # Configure the console as our one appender
> log4j.appender.A1=org.apache.log4j.ConsoleAppender
> log4j.appender.A1.layout=org.apache.log4j.PatternLayout
> log4j.appender.A1.layout.ConversionPattern=%d{HH:mm:ss,SSS} %-5p [%c]
> - %m%n
>
> # tighten logging on the DataNucleus Categories
> log4j.category.DataNucleus.JDO=WARN, A1
> log4j.category.DataNucleus.Persistence=WARN, A1
> log4j.category.DataNucleus.Cache=WARN, A1
> log4j.category.DataNucleus.MetaData=WARN, A1
> log4j.category.DataNucleus.General=WARN, A1
> log4j.category.DataNucleus.Utility=WARN, A1
> log4j.category.DataNucleus.Transaction=WARN, A1
> log4j.category.DataNucleus.Datastore=WARN, A1
> log4j.category.DataNucleus.ClassLoading=WARN, A1
> log4j.category.DataNucleus.Plugin=WARN, A1
> log4j.category.DataNucleus.ValueGeneration=WARN, A1
> log4j.category.DataNucleus.Enhancer=WARN, A1
> log4j.category.DataNucleus.SchemaTool=WARN, A1
>
> On Sep 18, 10:13 pm, Don Schwarz wrote:
>
>
>
> > Are you switch the logging level in the Admin Console to Info? By default
> > it only shows errors.
>
> > Do you have a logging.properties file? If so, what is your default .level
> > set to? System.out would be logged at INFO, so if you are specifying this
> > make sure it is INFO or lower.
> > Finally, I would recommend using java.util.logging instead of System.out.
> > It's a lot more flexible and is very easy to use.
>
> > On Fri, Sep 18, 2009 at 10:09 PM, Diana Cruise
> > wrote:
>
> > > I'm having difficulty debugging my application in production.
> > > According to the GAE docs, System.out.println output is directed as
> > > log level INFO. My home page generates such output when debugging in
> > > my test local environment. But, my deployment to production generates
> > > NO output. Using the Admin Console->Log I can see my Requests but NOT
> > > my standard output from println(). How can I see this output? Is it
> > > just NOT being flushed or is there a config for this?
>
> > > Thanks, Diana- Hide quoted text -
>
> > - Show quoted text -- Hide quoted text -
>
> - Show quoted text -
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---
[appengine-java] How-to view stack dumps in GAE
What does GAE do with standard dumps such as e.printStackTrace()?
Here is the Admin Console Log output in DEBUG mode for my test:
I 09-19 08:14AM 54.605 [app1-widget/1.336449820535414591].:
hello, testMode=false
I 09-19 08:14AM 54.605 org.apache.jsp.test_jsp _jspService: hello2
I 09-19 08:14AM 54.606 [app1-widget/1.336449820535414591].:
hello3
The output above comes from the code snippet below which tests Stdout,
Logger, and then a forced stack dump. As you can see the stack dump
identified with the message "hello4" is NOT in the log output above.
How do I view this stack dump in GAE?
<%@ page contentType="text/html;charset=UTF-8" language="java" %>
<%@ page import="java.util.*,
widget.presentation.common.*"
%>
<%
try {
UserSession userSess = UserSession.get(session, request, response);
System.out.println("hello, testMode=" + userSess.isTestMode());
java.util.logging.Logger logger = java.util.logging.Logger.getLogger
("widget");
logger.log(java.util.logging.Level.INFO, "hello2");
userSess.logInfo("hello3");
if (true) throw new Exception("hello4");
%>
...
<%
} catch (Exception e) { e.printStackTrace(); }
%>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---
[appengine-java] Re: JDO : How to make an unowned reference to an entity if we don't know its key yet ?
Now I'm confused...I thought an "unowned" relation was defined as being outside the Entity Group? If I have entity A and entity B who are both related to each other via entity C, I was under the impression that we had to decide whether C would be part of A's group or B's group, but NOT could not all be in 1 group. Perhaps that is a JDO limitation and you are using the lower level API? For the sake of my discussion I'll assume A "owns" C and B has "unowned" relation with C. According to Max Ross from the Google GAE Persistence Video, I am under the impression that we should denormalize data in order that the application get around the absence of Join capability. So the many-to-many entity C would contain extra data for each FK to A and B such that, for example, if the application is querying C based a particular A.key...then each record found will already contain fields from B instead of just having the key only (ie C.keyB). Otherwise, for each row the app would have to turn around and query B by primaryKey...hence encurring much overhead. Have I interpreted this fashion of persistence handling correctly as you understand it? Are you using this same denormalization in your designs? On Sep 19, 11:08 am, mlenormand wrote: > All the entities we are talking about are in the SAME entity group in > order to create/update them in the same transaction. In addition to > that, we declare an unowned relationship, using "Key" for example. I > want to save all my entities in the same transaction. > > On Sep 19, 5:52 pm, objectuser wrote: > > > > > Because your relationships are unowned (so the objects are in > > different entity groups, right?), you wouldn't be able to save them > > all in the same transaction anyway. > > > So you're right: you'll have to save them, which means multiple > > transactions, and if one of them fails you'll have inconsistent data. > > > On Sep 19, 8:47 am, mlenormand wrote: > > > > Hi Everybody, > > > > Here's my problem with JDO and unowned relationships : > > > > I have several entities of the same kind that are organized in a > > > hierarchical way, maintained with unowned relationships. > > > JDO can generate keys for my entities when they are written to the > > > datastore, but with this option, I can't know my entity's keys until > > > they are saved to the datastore. If I want to make a reference to an > > > entity A from an entity B, it seems to be impossible because at this > > > time I don't know A's key, that will be generated when transaction is > > > committed. > > > > One solution would be to use several transactions, each one would > > > persist an entity, but it's not good because I want to persist all the > > > entities in the same transaction. > > > > I know that defining keys myself could solve the problem, but maybe > > > there is an other solution that would be better ? > > > > Thank you for your help. And thank you to the GAE team for the > > > fantastic work you did !- Hide quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: JDO : How to make an unowned reference to an entity if we don't know its key yet ?
Seems I read you can only perform a Save on a particular Entity once during a transaction...the 2nd would fail. In my scenario above, A and B are in different Entity Groups. So, are your designs in agreement with the Max Ross principle as I have described above? Obviously, this A, B, C relation would be common to most any app. On Sep 19, 12:29 pm, objectuser wrote: > Interesting. If they're all in the same entity group, how about you > save the group without the keys set, then set the keys and save again, > all in the same transaction? > > On Sep 19, 11:08 am, mlenormand wrote: > > > > > All the entities we are talking about are in the SAME entity group in > > order to create/update them in the same transaction. In addition to > > that, we declare an unowned relationship, using "Key" for example. I > > want to save all my entities in the same transaction. > > > On Sep 19, 5:52 pm, objectuser wrote: > > > > Because your relationships are unowned (so the objects are in > > > different entity groups, right?), you wouldn't be able to save them > > > all in the same transaction anyway. > > > > So you're right: you'll have to save them, which means multiple > > > transactions, and if one of them fails you'll have inconsistent data. > > > > On Sep 19, 8:47 am, mlenormand wrote: > > > > > Hi Everybody, > > > > > Here's my problem with JDO and unowned relationships : > > > > > I have several entities of the same kind that are organized in a > > > > hierarchical way, maintained with unowned relationships. > > > > JDO can generate keys for my entities when they are written to the > > > > datastore, but with this option, I can't know my entity's keys until > > > > they are saved to the datastore. If I want to make a reference to an > > > > entity A from an entity B, it seems to be impossible because at this > > > > time I don't know A's key, that will be generated when transaction is > > > > committed. > > > > > One solution would be to use several transactions, each one would > > > > persist an entity, but it's not good because I want to persist all the > > > > entities in the same transaction. > > > > > I know that defining keys myself could solve the problem, but maybe > > > > there is an other solution that would be better ? > > > > > Thank you for your help. And thank you to the GAE team for the > > > > fantastic work you did !- Hide quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: My recommendation: Use Low-Level API instead of JDO/JPA
Could you supply some briefs on your findings? This is exactly the type of feedback we need in this forum to help us who are starting new projects to best decide on what APIs and designs to use. For example, if you could summarize your handling of this simple relation that is in ALL apps: Entity A and B are related to each other via C. So, A is 1-many with C and B is 1-many with C...so C is a many-to-many. Surely, you have this in your app. How do you compare your JDO implementation of these 3 Entities with your low- level API implementation? Perhaps you could include some simple code to illustrate your points. Seems I've only seen very basic examples like Hello World. Really, nothing beyond 1 relation...which is very limited and thus, ALL THE PERSISTENCE CONFUSION around here... Thanks. On Sep 23, 4:40 am, dflorey wrote: > I've just refactored my app to use Low-Level API instead of JDO and > it's much cleaner, works more reliable etc. > If you are not really forced to use JDO/JPA I'd strongly recommend to > use the very simple, elegant and clean Low-Level API. > BigTable simply is not a relational db, so it is highly misleading to > fake a relational wrapper and to me it caused lots of pain. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Type Owned by Two Different Types?
It seems alarming to me that these basic relations are difficult to
code. Lets face it, applications are full of these relations...it
seems to me that too much developer time is required in the
Persistence layer of GAE apps. The Persistence and Presentation
layers needs to be a "no brainer", so more focus can be where it needs
to be - the business layer. Does anyone feel the same?
On Sep 23, 7:51 pm, objectuser wrote:
> Here's some code that reproduces my problem. Before the code, I think
> it's interesting to note that I started making this example in JPA
> (because I had an old test that used JPA) and I was not able to
> reproduce the error. However, after converting to JDO, I get the
> error. I'm not 100% sure the code is exactly the same, however, but
> wanted to note it in case it's helpful to anyone.
>
> LocalServiceTestCase is based on the examples from Google.
>
> Does anyone see a problem with the code or is this a legit bug?
>
> @PersistenceCapable(identityType = IdentityType.APPLICATION,
> detachable = "true")
> public class ParentOfOne {
>
> @PrimaryKey
> @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
> private Long id;
>
> @Persistent
> private SharedChild child;
> ...
>
> }
>
> @PersistenceCapable(identityType = IdentityType.APPLICATION,
> detachable = "true")
> public class ParentOfList {
>
> @PrimaryKey
> @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
> private Long id;
>
> @Persistent
> private List children;
> ...
>
> }
>
> @PersistenceCapable(identityType = IdentityType.APPLICATION,
> detachable = "true")
> public class SharedChild {
>
> @PrimaryKey
> @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
> private Key id;
>
> @Persistent
> private String name;
> ...
>
> }
>
> public class SharedChildTestCase extends LocalServiceTestCase {
>
> private PersistenceManager persistenceManager;
>
> @Before
> public void openPersistenceManager() {
> PersistenceManagerFactory persistenceManagerFactory =
> JDOHelper
>
> .getPersistenceManagerFactory("transactions-optional");
> persistenceManager =
> persistenceManagerFactory.getPersistenceManager
> ();
> }
>
> @After
> public void closePersitenceManager() {
> persistenceManager.close();
> }
>
> @Test
> public void saveParents() {
> ParentOfList pol = new ParentOfList();
> List list = new ArrayList();
> SharedChild c1 = new SharedChild();
> c1.setName("c1");
> list.add(c1);
> SharedChild c2 = new SharedChild();
> c2.setName("c2");
> list.add(c2);
> pol.setChildren(list);
> persistenceManager.makePersistent(pol);
>
> ParentOfOne poo = new ParentOfOne();
> SharedChild c3 = new SharedChild();
> c1.setName("c3");
> poo.setChild(c3);
> persistenceManager.makePersistent(poo);
> }
>
> }
>
> On Sep 22, 9:40 am, objectuser wrote:
>
>
>
> > Thanks, Marton. That's very unfortunate. The limitations still
> > surprise me. Is that documented somewhere? Need to go back and
> > reread that stuff I guess.
>
> > I'll see if I can come up with a simple test case for the collection.
>
> > On Sep 22, 3:28 am, Marton Papp wrote:
>
> > > A class cannot have two different owned relationship to a single
> > > class. For example if I have these two classes:
>
> > > @PersistenceCapable(identityType = IdentityType.APPLICATION)
> > > public class Parent3 {
>
> > > @PrimaryKey
> > > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
> > > private Long id;
>
> > > Child1 child1;
>
> > > Child1 child2;
>
> > > // ... get-set
>
> > > }
>
> > > @PersistenceCapable(identityType = IdentityType.APPLICATION)
> > > public class Child1 {
>
> > > @PrimaryKey
> > > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
> > > private Key id;
>
> > > // ... get-set
>
> > > }
>
> > > then this code will fail with an assertion error:
>
> > > public void testMultiParent2() throws Exception {
>
> > > PersistenceManager pm;
>
> > > Parent3 p3 = null;
> > > Parent3 p3Loaded = null;
>
> > > pm = pmf.getPersistenceManager();
> > > pm.currentTransaction().begin();
>
> > > p3 = new Parent3();
> > > p3.setChild1(new Child1());
> > > p3.setChild2(new Child1());
> > > pm.makePersistent(p3);
> > > pm.currentTransaction().commit();
> > > pm.close();
>
> > > pm = pmf.getPe
[appengine-java] Re: GAE Performance
Shawn, the docs link you site is riddled with numbers (easy to get lost in them and what they truely mean)...which is why I included a simplest of scenarios above, that being to simply add a home addressbook entry attached to a User. Surely someone has a sizeable production system today in GAE that could share load results and real costs. If noone does, then that is also very troubling. Gaurav, I assume too that reading is NOT the problem and by this post am hoping to get real-world numbers to a simple update transaction. But, we need production app feedback from the most popular apps out there. Is there such a list for Java for GAE yet? Surely, there are large production apps by now? On Oct 19, 2:17 am, Gaurav wrote: > GAE performs best for simultaneous read operations. So there could be > virtually any no. > of users reading at the same time, no issue. But when it comes to > making updates > performance degradation is significant. > To get a better understanding of how gae performs under heavy load, I > recommend > this video :http://www.youtube.com/watch?v=AgaL6NGpkB8 > > On Oct 19, 7:09 am, Shawn Brown wrote: > > > > > Hi, > > > > I have read that a particular User can process 1-10 request per > > > second. Is this a limit of the free quota or does paid quota also > > > have this limitation. > > > Where and what did you read? > > > That doesn't seem consistent with the published limits. I guess it > > depends on what they were doing with the > > app.http://code.google.com/appengine/docs/quotas.html > > > Shawn- Hide quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: GAE Performance
Shawn, the 1-10s per Update was sited from Max Ross' I/O Video and I've seen it in various talks/docs along the way... On Oct 19, 11:03 am, Diana Cruise wrote: > Shawn, the docs link you site is riddled with numbers (easy to get > lost in them and what they truely mean)...which is why I included a > simplest of scenarios above, that being to simply add a home > addressbook entry attached to a User. Surely someone has a sizeable > production system today in GAE that could share load results and real > costs. If noone does, then that is also very troubling. > > Gaurav, I assume too that reading is NOT the problem and by this post > am hoping to get real-world numbers to a simple update transaction. > But, we need production app feedback from the most popular apps out > there. Is there such a list for Java for GAE yet? Surely, there are > large production apps by now? > > On Oct 19, 2:17 am, Gaurav wrote: > > > > > GAE performs best for simultaneous read operations. So there could be > > virtually any no. > > of users reading at the same time, no issue. But when it comes to > > making updates > > performance degradation is significant. > > To get a better understanding of how gae performs under heavy load, I > > recommend > > this video :http://www.youtube.com/watch?v=AgaL6NGpkB8 > > > On Oct 19, 7:09 am, Shawn Brown wrote: > > > > Hi, > > > > > I have read that a particular User can process 1-10 request per > > > > second. Is this a limit of the free quota or does paid quota also > > > > have this limitation. > > > > Where and what did you read? > > > > That doesn't seem consistent with the published limits. I guess it > > > depends on what they were doing with the > > > app.http://code.google.com/appengine/docs/quotas.html > > > > Shawn- Hide quoted text - > > > - Show quoted text -- Hide quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: GAE Performance
This is exactly what I'm am talking about...in my case the User and UserAddr are both in the same Entity Group. So, are you saying that my application which has a global presence in GAE can only support 25 simultaneous Users performing this update in under 5 seconds? Again, I take 1-10 requests per second response and go with the avg of 5/s. Add up 25 Users simultaneously hitting this Entity Group and that consumes a full 5 seconds. So, if you have 25 Users doing the same update over and over they will each have about a 5 second response. I know I am wrong because this is way LOW for a Google platform or any other...I just am NOT hearing or seeing numbers that say otherwise. If you clarify for me that this Entity Group performance stat of 1-10/ s is granular to the Row then we're on to something... That would tell me that my scenario above only applies if ALL Users were logged into the same account!!! If the Entity Group performance stat is granular to the Row then that would mean an infinite number of Users would average 5 updates per second. Please tell me this is TRUE! Otherwise, if this Entity Group performance stat of 1-10/s is granular to the whole group (is ALL rows) then the performance is dire as I described originally. Please tell me it isn't so! On Oct 19, 11:10 am, Don Schwarz wrote: > It's 1-10 updates per second per Entity > Group:http://code.google.com/appengine/docs/java/datastore/transactions.htm... > > You need to break your design up into Entity Groups according to which > pieces will need to be updated in a single transaction. In the best case, > each entity can be its own entity group and the only restriction is that you > update each entity no more often than 1-10 times per second. > > For example, it would not be a good idea to store a global counter in one > entity unless you planned to update it no more than 1-10 times per second. > The solution to this is to use sharded counters: > > http://code.google.com/appengine/articles/sharding_counters.html > > On Mon, Oct 19, 2009 at 11:06 AM, Diana Cruise > wrote: > > > > > > > Shawn, the 1-10s per Update was sited from Max Ross' I/O Video and > > I've seen it in various talks/docs along the way... > > > On Oct 19, 11:03 am, Diana Cruise wrote: > > > Shawn, the docs link you site is riddled with numbers (easy to get > > > lost in them and what they truely mean)...which is why I included a > > > simplest of scenarios above, that being to simply add a home > > > addressbook entry attached to a User. Surely someone has a sizeable > > > production system today in GAE that could share load results and real > > > costs. If noone does, then that is also very troubling. > > > > Gaurav, I assume too that reading is NOT the problem and by this post > > > am hoping to get real-world numbers to a simple update transaction. > > > But, we need production app feedback from the most popular apps out > > > there. Is there such a list for Java for GAE yet? Surely, there are > > > large production apps by now? > > > > On Oct 19, 2:17 am, Gaurav wrote: > > > > > GAE performs best for simultaneous read operations. So there could be > > > > virtually any no. > > > > of users reading at the same time, no issue. But when it comes to > > > > making updates > > > > performance degradation is significant. > > > > To get a better understanding of how gae performs under heavy load, I > > > > recommend > > > > this video :http://www.youtube.com/watch?v=AgaL6NGpkB8 > > > > > On Oct 19, 7:09 am, Shawn Brown wrote: > > > > > > Hi, > > > > > > > I have read that a particular User can process 1-10 request per > > > > > > second. Is this a limit of the free quota or does paid quota also > > > > > > have this limitation. > > > > > > Where and what did you read? > > > > > > That doesn't seem consistent with the published limits. I guess it > > > > > depends on what they were doing with the app. > >http://code.google.com/appengine/docs/quotas.html > > > > > > Shawn- Hide quoted text - > > > > > - Show quoted text -- Hide quoted text - > > > > - Show quoted text -- Hide quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: GAE Performance
an infinite number of "distinct" users, that is... On Oct 19, 4:01 pm, Diana Cruise wrote: > This is exactly what I'm am talking about...in my case the User and > UserAddr are both in the same Entity Group. So, are you saying that > my application which has a global presence in GAE can only support 25 > simultaneous Users performing this update in under 5 seconds? > > Again, I take 1-10 requests per second response and go with the avg of > 5/s. Add up 25 Users simultaneously hitting this Entity Group and > that consumes a full 5 seconds. So, if you have 25 Users doing the > same update over and over they will each have about a 5 second > response. > > I know I am wrong because this is way LOW for a Google platform or any > other...I just am NOT hearing or seeing numbers that say otherwise. > > If you clarify for me that this Entity Group performance stat of 1-10/ > s is granular to the Row then we're on to something... That would > tell me that my scenario above only applies if ALL Users were logged > into the same account!!! If the Entity Group performance stat is > granular to the Row then that would mean an infinite number of Users > would average 5 updates per second. Please tell me this is TRUE! > > Otherwise, if this Entity Group performance stat of 1-10/s is granular > to the whole group (is ALL rows) then the performance is dire as I > described originally. Please tell me it isn't so! > > On Oct 19, 11:10 am, Don Schwarz wrote: > > > > > It's 1-10 updates per second per Entity > > Group:http://code.google.com/appengine/docs/java/datastore/transactions.htm... > > > You need to break your design up into Entity Groups according to which > > pieces will need to be updated in a single transaction. In the best case, > > each entity can be its own entity group and the only restriction is that you > > update each entity no more often than 1-10 times per second. > > > For example, it would not be a good idea to store a global counter in one > > entity unless you planned to update it no more than 1-10 times per second. > > The solution to this is to use sharded counters: > > >http://code.google.com/appengine/articles/sharding_counters.html > > > On Mon, Oct 19, 2009 at 11:06 AM, Diana Cruise > > wrote: > > > > Shawn, the 1-10s per Update was sited from Max Ross' I/O Video and > > > I've seen it in various talks/docs along the way... > > > > On Oct 19, 11:03 am, Diana Cruise wrote: > > > > Shawn, the docs link you site is riddled with numbers (easy to get > > > > lost in them and what they truely mean)...which is why I included a > > > > simplest of scenarios above, that being to simply add a home > > > > addressbook entry attached to a User. Surely someone has a sizeable > > > > production system today in GAE that could share load results and real > > > > costs. If noone does, then that is also very troubling. > > > > > Gaurav, I assume too that reading is NOT the problem and by this post > > > > am hoping to get real-world numbers to a simple update transaction. > > > > But, we need production app feedback from the most popular apps out > > > > there. Is there such a list for Java for GAE yet? Surely, there are > > > > large production apps by now? > > > > > On Oct 19, 2:17 am, Gaurav wrote: > > > > > > GAE performs best for simultaneous read operations. So there could be > > > > > virtually any no. > > > > > of users reading at the same time, no issue. But when it comes to > > > > > making updates > > > > > performance degradation is significant. > > > > > To get a better understanding of how gae performs under heavy load, I > > > > > recommend > > > > > this video :http://www.youtube.com/watch?v=AgaL6NGpkB8 > > > > > > On Oct 19, 7:09 am, Shawn Brown wrote: > > > > > > > Hi, > > > > > > > > I have read that a particular User can process 1-10 request per > > > > > > > second. Is this a limit of the free quota or does paid quota also > > > > > > > have this limitation. > > > > > > > Where and what did you read? > > > > > > > That doesn't seem consistent with the published limits. I guess it > > > > > > depends on what they were doing with the app. > > >http://code.google.com/appengine/docs/quotas.html > > > > > > > Shawn- Hide quoted text - > > > > > > - Show quoted text -- Hide quoted text - > > > > > - Show quoted text -- Hide quoted text - > > > - Show quoted text -- Hide quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: GAE Performance
e the entire index to potentially > be locked. > > 2) I also haven't read anything yet about deadlock situations on GAE > which can happen surprisingly easily if you're updating multiple > indexes with enough concurrency and are using page locking. If you > were designing the GAE datastore service, the way to avoid that > situation would be to lock all indexes on each data update in the same > order every time. You'd sacrifice a lot of throughput, but you'd > never hit a deadlock so I suspect they've done something like this > behind the scenes unless people just aren't using GAE heavily enough > yet or the good people of the GAE have used some special sauce in the > datastore service impl. > > So I guess what I'm trying to say is that I don't believe that you > should be satisfied with any particular bit of performance data from > another application because your mileage will almost certainly vary. > I think that If you really want to know how your application would > perform and want to find out before writing the whole app and sharing > it with a billion users, I would recommend a very empirical approach: > > I'd write a sample app with with entity group where entity widths and > indexes are those that you think will be representative of your > deployed application and then add a simple test harness that will: > > a) seed data to a point that you think is representative > b) update and query your data in what you believe will be a worst case > scenario and then record the times > > I think the resulting curve of performance you see will be highly > dependent on how you vary the seed data size and the number of > indexes. Of course there are more dimensions than that, such as the # > of concurrent read operations and the # of concurrent write > operations, that you can vary as well depending on what your > performance requirements are. > > I hope this is somewhat helpful and I also hope that it's not totally > incorrect and misleading since, as I said, it's all rampant > speculation based on somewhat limited publicly available data. > > -Michael > > P.S. Of course, if anyone has data including # of records, #/ > composisiton of indexes, # reads per hour, # writes per hour and > latency per txn, I'd be fascinated to hear about it too! > > On Oct 19, 4:01 pm, Diana Cruise wrote: > > > > > This is exactly what I'm am talking about...in my case the User and > > UserAddr are both in the same Entity Group. So, are you saying that > > my application which has a global presence in GAE can only support 25 > > simultaneous Users performing this update in under 5 seconds? > > > Again, I take 1-10 requests per second response and go with the avg of > > 5/s. Add up 25 Users simultaneously hitting this Entity Group and > > that consumes a full 5 seconds. So, if you have 25 Users doing the > > same update over and over they will each have about a 5 second > > response. > > > I know I am wrong because this is way LOW for a Google platform or any > > other...I just am NOT hearing or seeing numbers that say otherwise. > > > If you clarify for me that this Entity Group performance stat of 1-10/ > > s is granular to the Row then we're on to something... That would > > tell me that my scenario above only applies if ALL Users were logged > > into the same account!!! If the Entity Group performance stat is > > granular to the Row then that would mean an infinite number of Users > > would average 5 updates per second. Please tell me this is TRUE! > > > Otherwise, if this Entity Group performance stat of 1-10/s is granular > > to the whole group (is ALL rows) then the performance is dire as I > > described originally. Please tell me it isn't so! > > > On Oct 19, 11:10 am, Don Schwarz wrote: > > > > It's 1-10 updates per second per Entity > > > Group:http://code.google.com/appengine/docs/java/datastore/transactions.htm... > > > > You need to break your design up into Entity Groups according to which > > > pieces will need to be updated in a single transaction. In the best case, > > > each entity can be its own entity group and the only restriction is that > > > you > > > update each entity no more often than 1-10 times per second. > > > > For example, it would not be a good idea to store a global counter in one > > > entity unless you planned to update it no more than 1-10 times per second. > > > The solution to this is to use sharded counters: > > > >http://code.google.com/appengine/articles/sharding_counters.html > > > > On Mon, Oct 19,
[appengine-java] Photo and Video services
What options do I have in GAE to allow Users to upload, store, and view media (photos, video, audio, etc) from my within my application? Is there a special data type in datastore that would be used to store media? If I store a media item in datastore then how do I display it to the User? In a regular app server, I would do this as a static file reference where I first transfer the data from the database to a file accessible under webroot (if it is not already present). But in GAE the application can NOT create files in the static area due to permissions...so is there another technique to accomplish this? Would videos best be served indirectly via youtube? If so, how do I know they will NOT be removed prematurely by youtube? Thanks. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Photo and Video services
Vince, I was unable to upload small photos...getting error code 500! What is the basic principle behind this solution? It appears you are storing the file in datastore in increments of 1M or less...along the lines of what Raphael was getting at. How do you display a list of images from the datastore on a webpage? On Oct 22, 1:33 pm, Vince Bonfanti wrote: > Hi Diana, > > I've created GaeVFS to solve this problem: > > http://code.google.com/p/gaevfs/ > > You can view a demonstration here: > > http://gaevfs.appspot.com/ > > Note that the current released version (0.3) will only upload about > 2.0MB before timing out; the latest code in SVN will support the full > 10.0MB allowed by Google App Engine. > > There's a feature on the roadmap called "Service for storing and > serving large files," but there's apparently no additional information > available on this feature: > > http://code.google.com/appengine/docs/roadmap.html > > Vince > > > > On Thu, Oct 22, 2009 at 1:55 PM, Diana Cruise > wrote: > > > What options do I have in GAE to allow Users to upload, store, and > > view media (photos, video, audio, etc) from my within my application? > > > Is there a special data type in datastore that would be used to store > > media? If I store a media item in datastore then how do I display it > > to the User? In a regular app server, I would do this as a static > > file reference where I first transfer the data from the database to a > > file accessible under webroot (if it is not already present). But in > > GAE the application can NOT create files in the static area due to > > permissions...so is there another technique to accomplish this? > > > Would videos best be served indirectly via youtube? If so, how do I > > know they will NOT be removed prematurely by youtube? > > > Thanks.- Hide quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Photo and Video services
Thanks Baz. Vince, I hit your demo site and just entered a path like /gaevfs/mypic or /gaevfs/images/mypic, then selected a local small photo. On submitting the form, received the 500 error...perhaps the photo had spaces in the name? On Oct 22, 3:10 pm, Vince Bonfanti wrote: > You were unable to upload a small photo on my demo site, or in your > development environment? I just successfully uploaded an image to the > demo site, which you can download from here: > > http://gaevfs.appspot.com/gaevfs/images/ > > Yes, the images get stored in "blocks" of 1MB or less. GaeVFS > implements a virtual file system, so you refer to the file using URL > links, just as you would for a static file. For example, here's an > image I just uploaded: > > http://gaevfs.appspot.com/gaevfs/images/img4.jpg > > The GaeVfsServlet handles both upload and download, and demonstrates > how this is done: > > http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlant... > > The GaeVfsServlet also handles setting the content type for the response. > > Vince > > > > On Thu, Oct 22, 2009 at 3:04 PM, Diana Cruise > wrote: > > > Vince, I was unable to upload small photos...getting error code 500! > > > What is the basic principle behind this solution? It appears you are > > storing the file in datastore in increments of 1M or less...along the > > lines of what Raphael was getting at. How do you display a list of > > images from the datastore on a webpage? > > > On Oct 22, 1:33 pm, Vince Bonfanti wrote: > >> Hi Diana, > > >> I've created GaeVFS to solve this problem: > > >> http://code.google.com/p/gaevfs/ > > >> You can view a demonstration here: > > >> http://gaevfs.appspot.com/ > > >> Note that the current released version (0.3) will only upload about > >> 2.0MB before timing out; the latest code in SVN will support the full > >> 10.0MB allowed by Google App Engine. > > >> There's a feature on the roadmap called "Service for storing and > >> serving large files," but there's apparently no additional information > >> available on this feature: > > >> http://code.google.com/appengine/docs/roadmap.html > > >> Vince > > >> On Thu, Oct 22, 2009 at 1:55 PM, Diana Cruise > >> wrote: > > >> > What options do I have in GAE to allow Users to upload, store, and > >> > view media (photos, video, audio, etc) from my within my application? > > >> > Is there a special data type in datastore that would be used to store > >> > media? If I store a media item in datastore then how do I display it > >> > to the User? In a regular app server, I would do this as a static > >> > file reference where I first transfer the data from the database to a > >> > file accessible under webroot (if it is not already present). But in > >> > GAE the application can NOT create files in the static area due to > >> > permissions...so is there another technique to accomplish this? > > >> > Would videos best be served indirectly via youtube? If so, how do I > >> > know they will NOT be removed prematurely by youtube? > > >> > Thanks.- Hide quoted text - > > >> - Show quoted text -- Hide quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Photo and Video services
Actually, I tried simple files without spaces also and they failed too. When I hit your photo I noticed execellent response time...have you noticed any particular degradation when displaying lists of photos, for example? On Oct 22, 3:58 pm, Diana Cruise wrote: > Thanks Baz. > > Vince, I hit your demo site and just entered a path like /gaevfs/mypic > or /gaevfs/images/mypic, then selected a local small photo. On > submitting the form, received the 500 error...perhaps the photo had > spaces in the name? > > On Oct 22, 3:10 pm, Vince Bonfanti wrote: > > > > > You were unable to upload a small photo on my demo site, or in your > > development environment? I just successfully uploaded an image to the > > demo site, which you can download from here: > > > http://gaevfs.appspot.com/gaevfs/images/ > > > Yes, the images get stored in "blocks" of 1MB or less. GaeVFS > > implements a virtual file system, so you refer to the file using URL > > links, just as you would for a static file. For example, here's an > > image I just uploaded: > > > http://gaevfs.appspot.com/gaevfs/images/img4.jpg > > > The GaeVfsServlet handles both upload and download, and demonstrates > > how this is done: > > > http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlant... > > > The GaeVfsServlet also handles setting the content type for the response. > > > Vince > > > On Thu, Oct 22, 2009 at 3:04 PM, Diana Cruise > > wrote: > > > > Vince, I was unable to upload small photos...getting error code 500! > > > > What is the basic principle behind this solution? It appears you are > > > storing the file in datastore in increments of 1M or less...along the > > > lines of what Raphael was getting at. How do you display a list of > > > images from the datastore on a webpage? > > > > On Oct 22, 1:33 pm, Vince Bonfanti wrote: > > >> Hi Diana, > > > >> I've created GaeVFS to solve this problem: > > > >> http://code.google.com/p/gaevfs/ > > > >> You can view a demonstration here: > > > >> http://gaevfs.appspot.com/ > > > >> Note that the current released version (0.3) will only upload about > > >> 2.0MB before timing out; the latest code in SVN will support the full > > >> 10.0MB allowed by Google App Engine. > > > >> There's a feature on the roadmap called "Service for storing and > > >> serving large files," but there's apparently no additional information > > >> available on this feature: > > > >> http://code.google.com/appengine/docs/roadmap.html > > > >> Vince > > > >> On Thu, Oct 22, 2009 at 1:55 PM, Diana Cruise > > >> wrote: > > > >> > What options do I have in GAE to allow Users to upload, store, and > > >> > view media (photos, video, audio, etc) from my within my application? > > > >> > Is there a special data type in datastore that would be used to store > > >> > media? If I store a media item in datastore then how do I display it > > >> > to the User? In a regular app server, I would do this as a static > > >> > file reference where I first transfer the data from the database to a > > >> > file accessible under webroot (if it is not already present). But in > > >> > GAE the application can NOT create files in the static area due to > > >> > permissions...so is there another technique to accomplish this? > > > >> > Would videos best be served indirectly via youtube? If so, how do I > > >> > know they will NOT be removed prematurely by youtube? > > > >> > Thanks.- Hide quoted text - > > > >> - Show quoted text -- Hide quoted text - > > > - Show quoted text -- Hide quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Photo and Video services
Yes, that worked ok from another system so it must be an IE7 problem...good catch! On Oct 22, 5:55 pm, Vince Bonfanti wrote: > Can you try a different browser? From the logs it looks like you're > using IE7. I just tried with IE8, Chrome, and Firefox 3 (all on > Windows) and they all worked fine. Something about the path being sent > by the browser is causing the error--if we can narrow it to IE7 then I > can investigate further. > > The only "degradation" I've ever seen is that larger files take longer > to download, which you'd expect. However, I've never done any real > load or stress testing. > > Vince > > > > On Thu, Oct 22, 2009 at 6:07 PM, Diana Cruise > wrote: > > > Actually, I tried simple files without spaces also and they failed > > too. When I hit your photo I noticed execellent response time...have > > you noticed any particular degradation when displaying lists of > > photos, for example? > > > On Oct 22, 3:58 pm, Diana Cruise wrote: > >> Thanks Baz. > > >> Vince, I hit your demo site and just entered a path like /gaevfs/mypic > >> or /gaevfs/images/mypic, then selected a local small photo. On > >> submitting the form, received the 500 error...perhaps the photo had > >> spaces in the name? > > >> On Oct 22, 3:10 pm, Vince Bonfanti wrote: > > >> > You were unable to upload a small photo on my demo site, or in your > >> > development environment? I just successfully uploaded an image to the > >> > demo site, which you can download from here: > > >> > http://gaevfs.appspot.com/gaevfs/images/ > > >> > Yes, the images get stored in "blocks" of 1MB or less. GaeVFS > >> > implements a virtual file system, so you refer to the file using URL > >> > links, just as you would for a static file. For example, here's an > >> > image I just uploaded: > > >> > http://gaevfs.appspot.com/gaevfs/images/img4.jpg > > >> > The GaeVfsServlet handles both upload and download, and demonstrates > >> > how this is done: > > >> > > >> > http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlant... > > >> > The GaeVfsServlet also handles setting the content type for the response. > > >> > Vince > > >> > On Thu, Oct 22, 2009 at 3:04 PM, Diana Cruise > >> > wrote: > > >> > > Vince, I was unable to upload small photos...getting error code 500! > > >> > > What is the basic principle behind this solution? It appears you are > >> > > storing the file in datastore in increments of 1M or less...along the > >> > > lines of what Raphael was getting at. How do you display a list of > >> > > images from the datastore on a webpage? > > >> > > On Oct 22, 1:33 pm, Vince Bonfanti wrote: > >> > >> Hi Diana, > > >> > >> I've created GaeVFS to solve this problem: > > >> > >> http://code.google.com/p/gaevfs/ > > >> > >> You can view a demonstration here: > > >> > >> http://gaevfs.appspot.com/ > > >> > >> Note that the current released version (0.3) will only upload about > >> > >> 2.0MB before timing out; the latest code in SVN will support the full > >> > >> 10.0MB allowed by Google App Engine. > > >> > >> There's a feature on the roadmap called "Service for storing and > >> > >> serving large files," but there's apparently no additional information > >> > >> available on this feature: > > >> > >> http://code.google.com/appengine/docs/roadmap.html > > >> > >> Vince > > >> > >> On Thu, Oct 22, 2009 at 1:55 PM, Diana Cruise > >> > >> wrote: > > >> > >> > What options do I have in GAE to allow Users to upload, store, and > >> > >> > view media (photos, video, audio, etc) from my within my > >> > >> > application? > > >> > >> > Is there a special data type in datastore that would be used to > >> > >> > store > >> > >> > media? If I store a media item in datastore then how do I display > >> > >> > it > >> > >> > to the User? In a regular app server, I would do this as a static > >> > >> > file reference where I first transfer the data from the database to > >> > &
[appengine-java] Re: Photo and Video services
What is the nature of the 10Mb limit again? On Oct 22, 6:38 pm, Diana Cruise wrote: > Yes, that worked ok from another system so it must be an IE7 > problem...good catch! > > On Oct 22, 5:55 pm, Vince Bonfanti wrote: > > > > > Can you try a different browser? From the logs it looks like you're > > using IE7. I just tried with IE8, Chrome, and Firefox 3 (all on > > Windows) and they all worked fine. Something about the path being sent > > by the browser is causing the error--if we can narrow it to IE7 then I > > can investigate further. > > > The only "degradation" I've ever seen is that larger files take longer > > to download, which you'd expect. However, I've never done any real > > load or stress testing. > > > Vince > > > On Thu, Oct 22, 2009 at 6:07 PM, Diana Cruise > > wrote: > > > > Actually, I tried simple files without spaces also and they failed > > > too. When I hit your photo I noticed execellent response time...have > > > you noticed any particular degradation when displaying lists of > > > photos, for example? > > > > On Oct 22, 3:58 pm, Diana Cruise wrote: > > >> Thanks Baz. > > > >> Vince, I hit your demo site and just entered a path like /gaevfs/mypic > > >> or /gaevfs/images/mypic, then selected a local small photo. On > > >> submitting the form, received the 500 error...perhaps the photo had > > >> spaces in the name? > > > >> On Oct 22, 3:10 pm, Vince Bonfanti wrote: > > > >> > You were unable to upload a small photo on my demo site, or in your > > >> > development environment? I just successfully uploaded an image to the > > >> > demo site, which you can download from here: > > > >> > http://gaevfs.appspot.com/gaevfs/images/ > > > >> > Yes, the images get stored in "blocks" of 1MB or less. GaeVFS > > >> > implements a virtual file system, so you refer to the file using URL > > >> > links, just as you would for a static file. For example, here's an > > >> > image I just uploaded: > > > >> > http://gaevfs.appspot.com/gaevfs/images/img4.jpg > > > >> > The GaeVfsServlet handles both upload and download, and demonstrates > > >> > how this is done: > > > >> > > > >> > http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlant... > > > >> > The GaeVfsServlet also handles setting the content type for the > > >> > response. > > > >> > Vince > > > >> > On Thu, Oct 22, 2009 at 3:04 PM, Diana Cruise > > >> > wrote: > > > >> > > Vince, I was unable to upload small photos...getting error code 500! > > > >> > > What is the basic principle behind this solution? It appears you are > > >> > > storing the file in datastore in increments of 1M or less...along the > > >> > > lines of what Raphael was getting at. How do you display a list of > > >> > > images from the datastore on a webpage? > > > >> > > On Oct 22, 1:33 pm, Vince Bonfanti wrote: > > >> > >> Hi Diana, > > > >> > >> I've created GaeVFS to solve this problem: > > > >> > >> http://code.google.com/p/gaevfs/ > > > >> > >> You can view a demonstration here: > > > >> > >> http://gaevfs.appspot.com/ > > > >> > >> Note that the current released version (0.3) will only upload about > > >> > >> 2.0MB before timing out; the latest code in SVN will support the > > >> > >> full > > >> > >> 10.0MB allowed by Google App Engine. > > > >> > >> There's a feature on the roadmap called "Service for storing and > > >> > >> serving large files," but there's apparently no additional > > >> > >> information > > >> > >> available on this feature: > > > >> > >> http://code.google.com/appengine/docs/roadmap.html > > > >> > >> Vince > > > >> > >> On Thu, Oct 22, 2009 at 1:55 PM, Diana Cruise > > >> > >> wrote: > > > >> > >> > What options do I have in GAE to allow Users to upload, store, and > > >> > >> > view media (photos, video, audio, etc) from my within my > > >> > >>
[appengine-java] Re: Photo and Video services
In terms of performance does gaevfs accmmodate http resource timestamp checks to avoid re-downloading repeated requests for the same images? On Oct 22, 8:22 pm, Vince Bonfanti wrote: > That's the most GAE will let you upload in one request (for now). > Again, there's a "Service for storing and serving large files" feature > on the roadmap, but the details of this feature have not be revealed. > > Vince > > > > On Thu, Oct 22, 2009 at 7:40 PM, Diana Cruise > wrote: > > > What is the nature of the 10Mb limit again? > > > On Oct 22, 6:38 pm, Diana Cruise wrote: > >> Yes, that worked ok from another system so it must be an IE7 > >> problem...good catch! > > >> On Oct 22, 5:55 pm, Vince Bonfanti wrote: > > >> > Can you try a different browser? From the logs it looks like you're > >> > using IE7. I just tried with IE8, Chrome, and Firefox 3 (all on > >> > Windows) and they all worked fine. Something about the path being sent > >> > by the browser is causing the error--if we can narrow it to IE7 then I > >> > can investigate further. > > >> > The only "degradation" I've ever seen is that larger files take longer > >> > to download, which you'd expect. However, I've never done any real > >> > load or stress testing. > > >> > Vince > > >> > On Thu, Oct 22, 2009 at 6:07 PM, Diana Cruise > >> > wrote: > > >> > > Actually, I tried simple files without spaces also and they failed > >> > > too. When I hit your photo I noticed execellent response time...have > >> > > you noticed any particular degradation when displaying lists of > >> > > photos, for example? > > >> > > On Oct 22, 3:58 pm, Diana Cruise wrote: > >> > >> Thanks Baz. > > >> > >> Vince, I hit your demo site and just entered a path like /gaevfs/mypic > >> > >> or /gaevfs/images/mypic, then selected a local small photo. On > >> > >> submitting the form, received the 500 error...perhaps the photo had > >> > >> spaces in the name? > > >> > >> On Oct 22, 3:10 pm, Vince Bonfanti wrote: > > >> > >> > You were unable to upload a small photo on my demo site, or in your > >> > >> > development environment? I just successfully uploaded an image to > >> > >> > the > >> > >> > demo site, which you can download from here: > > >> > >> > http://gaevfs.appspot.com/gaevfs/images/ > > >> > >> > Yes, the images get stored in "blocks" of 1MB or less. GaeVFS > >> > >> > implements a virtual file system, so you refer to the file using URL > >> > >> > links, just as you would for a static file. For example, here's an > >> > >> > image I just uploaded: > > >> > >> > http://gaevfs.appspot.com/gaevfs/images/img4.jpg > > >> > >> > The GaeVfsServlet handles both upload and download, and demonstrates > >> > >> > how this is done: > > >> > >> > > >> > >> > http://code.google.com/p/gaevfs/source/browse/trunk/src/com/newatlant... > > >> > >> > The GaeVfsServlet also handles setting the content type for the > >> > >> > response. > > >> > >> > Vince > > >> > >> > On Thu, Oct 22, 2009 at 3:04 PM, Diana Cruise > >> > >> > wrote: > > >> > >> > > Vince, I was unable to upload small photos...getting error code > >> > >> > > 500! > > >> > >> > > What is the basic principle behind this solution? It appears you > >> > >> > > are > >> > >> > > storing the file in datastore in increments of 1M or less...along > >> > >> > > the > >> > >> > > lines of what Raphael was getting at. How do you display a list > >> > >> > > of > >> > >> > > images from the datastore on a webpage? > > >> > >> > > On Oct 22, 1:33 pm, Vince Bonfanti wrote: > >> > >> > >> Hi Diana, > > >> > >> > >> I've created GaeVFS to solve this problem: > > >> > >> > >> http://code.google.com/p/gaevfs/ > > >> > >> > >> You can view a demonstration her
[appengine-java] Re: GAE Performance
Relating to entity groups, how can we determine what entity group each entity belongs to? Using the Data Viewer, I would think we could examine this type of setup info for each entity but I have NOT found how to do that. Thanks! On Oct 23, 1:50 pm, "Jason (Google)" wrote: > Hi Diana. As others have stated, App Engine can write to multiple entity > groups in parallel, so if each User entity is a root entity or is otherwise > placed in a different entity group, then there shouldn't be any issues. > Regarding performance, all apps should generally be able to handle up to 30 > simultaneous dynamic requests assuming a 75ms processing time for each > (average load), for a throughput of 400 qps or so: > > http://code.google.com/appengine/docs/java/runtime.html#Quotas_and_Li... > > If you want any other performance or cost-related numbers, let me know. > > For updates to the same entity or entity group, App Engine uses optimistic > concurrency as opposed to locking. If an entity is already being updated, > then the second request will fail and will automatically get retried on the > server. After consistent failures, an exception will be thrown which you can > catch to either handle gracefully. Datastore writes will fail from time to > time, generally about 0.1 to 0.2 percent of the time, but the failure rate > will be higher when there is contention, i.e. a high rate of simultaneous > writes to the same entity/entity group. > > http://code.google.com/appengine/articles/scaling/contention.html > > - Jason > > On Thu, Oct 22, 2009 at 8:04 AM, Diana Cruise wrote: > > > > > > > I'm glad to hear that the 1-10 requests/second is per User root > > entity...in my case this means that huge number of Users logged in > > around the world should expect sub-second response even if tens of > > thousands clicked the Update button at the same instance in time! > > > The only problem is we do NOT hear from anyone outside of Google to > > confirm performance of large volume for specific applications and what > > the real costs are!!! > > > Regarding deadlock, I hear GAE does NOT both with lock timeouts so as > > soon as a transaction trys to retrieve a record that is already > > locked, it will receive an error and have to retry. > > > On Oct 19, 5:50 pm, "Dr. Flufenstein" > > wrote: > > > Preface: Please note, I'm not speaking for google at all in this note > > > and a lot of what I've written is speculation based on what I've read > > > in various GAE docs as well as some meager knowledge of how relational > > > DBs generally work. And yes, I know datastore isn't a relational DB, > > > but I believe that their indexing implementation likely runs into many > > > of the same problems you have with indexing relational data although > > > that assumption could be completely wrong. > > > > From what I can tell, the update bottleneck you're referring to is for > > > updating what you would often think of as a single record if you were > > > persisting one instance of your User as a single denormalized record > > > in a relational schema. I suspect this bottleneck is due to the > > > datastore architecture and the way that data updates are accumulated > > > (possibly grouped/keyed by PK) in a queue, which is probably read from > > > like a cache if read requests come in before the data has been flushed > > > into the actual storage medium and replicated to the other > > > datacenters. > > > > So if each of your users were updating their own User records, I don't > > > believe you'd experience that limitation which may be an artifact of > > > how those in-memory queue/cache structures are managed/locked during > > > updates (i.e. a new update for a record may be held until it's been > > > flushed from the queue to the storage medium to prevent having to > > > merge/reconcile records in the queue). If they were all updating a > > > single shared record, then I think you'd hit this pretty quick. > > > > Let's say though that your users are updating separate records...as > > > your data size grows, you will probably see your update throughput > > > decrease as other factors become dominant, and I believe this will > > > primarily be dependent on the number and composition of the indexes on > > > your data as well as the number of entities persisted. To me, this is > > > the much riskier unknown because your average index structure is > > > harder to update piecewise in parallel because the index must
[appengine-java] Re: GAE status / availalbility problems?
I'm troubled by this lack of help for GAE apps...I see MANY forum entries with NO reply lately whereas in earlier months there seemed to be more quality responses. Who are the Google reps assisting with this forum? Am I missing something? On Nov 6, 11:47 pm, Dmitry Anipko wrote: > After some more experimentation, it looks like there is something in > the data store that is surfaced by our application: > > if I clean up the store for the application completely, using the web > interface, then for the first ~100 request it works fine, but after > some time, it starts producing those errors I wrote before - it can't > query objects by index (and that problem persists until I clean the > store again). > > The second observation - although I don't know if it is related or not > - is when the store for my app is in such state, and I try to view > objects through the web interface, the web interfaces crashes with the > error 500: > > Server Error > A server error has occurred. > > Return to Applications screen » > > GAE folks, can someone please take a look - this is a blocking issue > for my team, and I don't know how to proceed here without your help. > > Everything works just fine when I run it on a local development > server. > > Thank you. > > On Nov 6, 11:05 am, Diana Cruise wrote: > > > > > I see NO reply from GAE here regarding such a critical issue!!! > > > Is your app still down, what is the status of your app? How are we > > supposed to put an application in production with GAE if such a > > problem can occur with NO response for 3 days? Is there another > > channel to raise such high-priority issues to get proper helpdesk > > response? > > > On Nov 4, 1:58 am, Dmitry Anipko wrote: > > > > It is 11:56 Pacific 11/3/2009, are there any known issues withGAE > > > right now? > > > > My application that has been working for a couple of months now today > > > started producing failures for some queries from the data store > > > (basically an object which has just been stored cannot be retrieved > > > after that by key), and after some time started producing even > > > > > > > > > > 500 Server Error > > > > > > > > > Error: Server Error > > > The server encountered an error and could not complete your > > > request.If the problem persists, please report your problem and > > > mention this error message and the query that caused it. > > > > > > > > > > The same application executes just fine in the localGAEdevelopment > > > environment. Can folks fromGAEcomment if there is any maintenance > > > going on right now or how could I get more details on what causing the > > > failures (again the same code worked for quite some time now, so I > > > think it is unlikely the code is the issue here). Thanks for your help.- > > > Hide quoted text - > > > - Show quoted text -- Hide quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: GAE status / availalbility problems?
Thanks Phil, but this scheduled downtime was advertised for 11/3 at 5pm and was actually between 5:17-6:12. Dmitry clearly states his problem was at 11:56 that day...not to mention, he is still having the problem! We would just like to see more answers in the forum in order to be successful with GAE. On Nov 8, 8:06 pm, philburk wrote: > Your problem started on 11/3. There was a scheduled downtime that day. > It may be related. Here is a thread that announces scheduled downtime. > It is not very well advertised. > > http://groups.google.com/group/google-appengine-downtime-notify/brows... > > Phil (not from Google) > > On Nov 7, 7:29 am, Diana Cruise wrote: > > > > > I'm troubled by this lack of help for GAE apps...I see MANY forum > > entries with NO reply lately whereas in earlier months there seemed to > > be more quality responses. Who are the Google reps assisting with- Hide > > quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Which datastore exceptions are appropriate for Re-try
WSouza, what scenario triggers UnknownException?
On Nov 9, 3:24 am, leszek wrote:
> http://code.google.com/intl/pl/appengine/docs/java/datastore/transact...
>
>
> for (int i = 0; i < NUM_RETRIES; i++) {
> pm.currentTransaction().begin();
>
> ClubMembers members = pm.getObjectById(ClubMembers.class,
> "k12345");
> members.incrementCounterBy(1);
>
> try {
> pm.currentTransaction().commit();
> break;
>
> } catch (JDOCanRetryException ex) {
> if (i == (NUM_RETRIES - 1)) {
> throw ex;
> }
> }
> }
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---
[appengine-java] Re: Which datastore exceptions are appropriate for Re-try
Leszek, thanks much!
On Nov 9, 11:20 am, Diana Cruise wrote:
> WSouza, what scenario triggers UnknownException?
>
> On Nov 9, 3:24 am, leszek wrote:
>
>
>
> >http://code.google.com/intl/pl/appengine/docs/java/datastore/transact...
>
> >
> > for (int i = 0; i < NUM_RETRIES; i++) {
> > pm.currentTransaction().begin();
>
> > ClubMembers members = pm.getObjectById(ClubMembers.class,
> > "k12345");
> > members.incrementCounterBy(1);
>
> > try {
> > pm.currentTransaction().commit();
> > break;
>
> > } catch (JDOCanRetryException ex) {
> > if (i == (NUM_RETRIES - 1)) {
> > throw ex;
> > }
> > }
> > }
> > - Hide quoted text -
>
> - Show quoted text -
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---
[appengine-java] Re: JDO/JPA Snippets That Work - Updating a bidirectional, owned, one-to-many relationship with a new child
Max Ross, I took time to review and enter discussion relating to your
snippet. In that thread above I raised questions, but never heard
back...please have a look.
On Nov 2, 12:55 pm, "Max Ross (Google)"
wrote:
> Looking in the debugger is not a good indication of what is and is not set
> because jdo-enhanced classes perform lazy loading when the fields are
> accessed. If you call a getter method and System.out.println() the result
> is it still null? Feel free to send me your model objects and your
> persistence code with some println()s showing me the problem and I'll have a
> look.
>
> Max
>
>
>
> On Tue, Oct 27, 2009 at 5:28 PM, stanlick wrote:
>
> > Hey Max --
>
> > At what point are you validating this? I am breaking after the commit
> > and also after a subsequent query.execute(). Dude, it is about to
> > make me jump off a bridge.
>
> > P.S. I'm not a good swimmer, so please throw me a clue.
>
> > Scott
>
> > On Oct 27, 12:25 pm, "Max Ross (Google)"
> >
>
> > wrote:
> > > At what point are you checking to see if the parent property of the child
> > is
> > > populated? I see that the newly created Chapter's Book property is
> > > populated when addChapterToBook() completes.
>
> > > Max
>
> > > On Tue, Oct 27, 2009 at 8:31 AM, stanlick wrote:
>
> > > > This code saves the child for me, however, the parent property of the
> > > > child (in your case "book") is null in the child. I have tried
> > > > deliberately setting this child "parent" property myself to the parent
> > > > prior to commit, but nothing seems to work.
>
> > > > On Sep 28, 6:45 pm, Max Ross
> > > >
> > >>
> > > > wrote:
> > > > > Hello again and welcome to Episode 3 of JDO/JPA Snippets That Work.
> > > > Today's
> > > > > episode is called..
>
> > > > > Updating A Bidrectional Owned One-To-Many With A New Child
>
> > > > > All the way back in episode one we demonstrated how to create both a
> > > > parent
> > > > > and a child of a bidirectional, owned, one-to-many relationship
> > > > > at the same time. This week we're going to see how to add a child to
> > an
> > > > > existing parent. We'll use the same model objects we used in episode
> > > > one:
>
> > > > > JPA:
> > > > > @Entity
> > > > > public class Book {
> > > > > @Id
> > > > > @GeneratedValue(strategy=GenerationType.IDENTITY)
> > > > > private Key id;
>
> > > > > private String title;
>
> > > > > @OneToMany(mappedBy = "book", cascade = CascadeType.ALL)
> > > > > private List chapters = new ArrayList();
>
> > > > > // getters and setters
>
> > > > > }
>
> > > > > @Entity
> > > > > public class Chapter {
> > > > > @Id
> > > > > @GeneratedValue(strategy=GenerationType.IDENTITY)
> > > > > private Key id;
>
> > > > > private String title;
> > > > > private int numPages;
>
> > > > > @ManyToOne(fetch = FetchType.LAZY)
> > > > > private Book book;
>
> > > > > // getters and setters
>
> > > > > }
>
> > > > > Now let's assume we've already created a book with a few chapters in
> > the
> > > > > datastore and we want to add a brand new chapter to a Book with a
> > given
> > > > id
> > > > > (we'll assume someone else is creating and closing an EntityManager
> > named
> > > > > 'em' for us):
>
> > > > > public void addChapterToBook(EntityManager em, Key bookKey, Chapter
> > > > chapter)
> > > > > {
> > > > > em.getTransaction().begin();
> > > > > try {
> > > > > Book b = em.find(Book.class, bookKey);
> > > > > if (b == null) {
> > > > > throw new RuntimeException("Book " + bookKey + " not
> > > > found!");
> > > > > }
> > > > > b.getChapters().add(chapter);
> > > > > em.getTransaction().commit();
> > > > > } finally {
> > > > > if (em.getTransaction().isActive()) {
> > > > > em.getTransaction().rollback();
> > > > > }
> > > > > }
>
> > > > > }
>
> > > > > JDO:
>
> > > > > @PersistenceCapable(identityType = IdentityType.APPLICATION,
> > detachable =
> > > > > "true")
> > > > > public class Book {
>
> > > > > @PrimaryKey
> > > > > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
> > > > > private Key id;
>
> > > > > private String title;
>
> > > > > @Persistent(mappedBy = "book")
> > > > > @Element(dependent = "true")
> > > > > @Order(extensions = @Extension(vendorName="datanucleus", key=
> > > > > "list-ordering", value="id asc"))
> > > > > private List chapters = new ArrayList();
>
> > > > > // getters and setters
>
> > > > > }
>
> > > > > @PersistenceCapable(identityType = IdentityType.APPLICATION,
> > detachable =
> > > > > "true")
> > > > > public class Chapter {
> > > > > @PrimaryKey
> > > > > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
> > > > > private Key id;
>
> > > > > private String title;
> > > > > private int numPages;
>
> > > > > @Persistent
> > > > > private Book book;
>
> > > > > // getters and setters
>
[appengine-java] Re: GAE status / availalbility problems?
What's the latest here Dmitry...can GAE handle more than 100 request or not? Please update where you are with this problem. Thanks. On Nov 9, 5:15 pm, Dmitry Anipko wrote: > Following up offline and will update with resolution later. > > > > On Mon, Nov 9, 2009 at 9:50 AM, Jason (Google) wrote: > > As always, I need your application ID in order to help troubleshoot > > app-specific issues. Please provide that and any stack traces or other error > > messages you see in your application's logs. > > > - Jason > > > On Fri, Nov 6, 2009 at 9:47 PM, Dmitry Anipko > > wrote: > > >> After some more experimentation, it looks like there is something in > >> the data store that is surfaced by our application: > > >> if I clean up the store for the application completely, using the web > >> interface, then for the first ~100 request it works fine, but after > >> some time, it starts producing those errors I wrote before - it can't > >> query objects by index (and that problem persists until I clean the > >> store again). > > >> The second observation - although I don't know if it is related or not > >> - is when the store for my app is in such state, and I try to view > >> objects through the web interface, the web interfaces crashes with the > >> error 500: > > >> Server Error > >> A server error has occurred. > > >> Return to Applications screen » > > >> GAE folks, can someone please take a look - this is a blocking issue > >> for my team, and I don't know how to proceed here without your help. > > >> Everything works just fine when I run it on a local development > >> server. > > >> Thank you. > > >> On Nov 6, 11:05 am, Diana Cruise wrote: > >> > I see NO reply from GAE here regarding such a critical issue!!! > > >> > Is your app still down, what is the status of your app? How are we > >> > supposed to put an application in production with GAE if such a > >> > problem can occur with NO response for 3 days? Is there another > >> > channel to raise such high-priority issues to get proper helpdesk > >> > response? > > >> > On Nov 4, 1:58 am, Dmitry Anipko wrote: > > >> > > It is 11:56 Pacific 11/3/2009, are there any known issues withGAE > >> > > right now? > > >> > > My application that has been working for a couple of months now today > >> > > started producing failures for some queries from the data store > >> > > (basically an object which has just been stored cannot be retrieved > >> > > after that by key), and after some time started producing even > > >> > > > >> > > > >> > > 500 Server Error > >> > > > >> > > > >> > > Error: Server Error > >> > > The server encountered an error and could not complete your > >> > > request.If the problem persists, please report your problem and > >> > > mention this error message and the query that caused it. > >> > > > >> > > > > >> > > The same application executes just fine in the localGAEdevelopment > >> > > environment. Can folks fromGAEcomment if there is any maintenance > >> > > going on right now or how could I get more details on what causing the > >> > > failures (again the same code worked for quite some time now, so I > >> > > think it is unlikely the code is the issue here). Thanks for your > >> > > help.- Hide quoted text - > > >> > - Show quoted text -- Hide quoted text - > > - Show quoted text - -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: GAE status / availalbility problems?
Excellent news. How long did it take to resolve this matter and what was the impact to your Users? I assume you would agree we could avoid this type of problem by only using the data types listed here right? http://code.google.com/appengine/docs/java/datastore/dataclasses.html#Core_Value_Types On Nov 18, 3:27 pm, Dmitry Anipko wrote: > Thanks Jason for help. In our case the problem was caused by usage of > byte[] field in a persisted class. Somehow though, it wasn't causing > any problems immediately, but only after a few objects have been > saved. Per Jason, usage of byte[] in JDO on GAE isn't supported. > > > > On Wed, Nov 18, 2009 at 1:00 PM, Diana Cruise > wrote: > > What's the latest here Dmitry...can GAE handle more than 100 request > > or not? Please update where you are with this problem. Thanks. > > > On Nov 9, 5:15 pm, Dmitry Anipko wrote: > >> Following up offline and will update with resolution later. > > >> On Mon, Nov 9, 2009 at 9:50 AM, Jason (Google) wrote: > >> > As always, I need your application ID in order to help troubleshoot > >> > app-specific issues. Please provide that and any stack traces or other > >> > error > >> > messages you see in your application's logs. > > >> > - Jason > > >> > On Fri, Nov 6, 2009 at 9:47 PM, Dmitry Anipko > >> > wrote: > > >> >> After some more experimentation, it looks like there is something in > >> >> the data store that is surfaced by our application: > > >> >> if I clean up the store for the application completely, using the web > >> >> interface, then for the first ~100 request it works fine, but after > >> >> some time, it starts producing those errors I wrote before - it can't > >> >> query objects by index (and that problem persists until I clean the > >> >> store again). > > >> >> The second observation - although I don't know if it is related or not > >> >> - is when the store for my app is in such state, and I try to view > >> >> objects through the web interface, the web interfaces crashes with the > >> >> error 500: > > >> >> Server Error > >> >> A server error has occurred. > > >> >> Return to Applications screen » > > >> >> GAE folks, can someone please take a look - this is a blocking issue > >> >> for my team, and I don't know how to proceed here without your help. > > >> >> Everything works just fine when I run it on a local development > >> >> server. > > >> >> Thank you. > > >> >> On Nov 6, 11:05 am, Diana Cruise wrote: > >> >> > I see NO reply from GAE here regarding such a critical issue!!! > > >> >> > Is your app still down, what is the status of your app? How are we > >> >> > supposed to put an application in production with GAE if such a > >> >> > problem can occur with NO response for 3 days? Is there another > >> >> > channel to raise such high-priority issues to get proper helpdesk > >> >> > response? > > >> >> > On Nov 4, 1:58 am, Dmitry Anipko wrote: > > >> >> > > It is 11:56 Pacific 11/3/2009, are there any known issues withGAE > >> >> > > right now? > > >> >> > > My application that has been working for a couple of months now > >> >> > > today > >> >> > > started producing failures for some queries from the data store > >> >> > > (basically an object which has just been stored cannot be retrieved > >> >> > > after that by key), and after some time started producing even > > >> >> > > > >> >> > > > >> >> > > 500 Server Error > >> >> > > > >> >> > > > >> >> > > Error: Server Error > >> >> > > The server encountered an error and could not complete your > >> >> > > request.If the problem persists, please report your problem and > >> >> > > mention this error message and the query that caused it. > >> >> > > > >> >> > > > > >> >> > > The same application executes just fine in the localGAEdevelopment > >> >> > > environment. Can folks fromGAEcomment if there is any maintenance > >> >> > > going on right now or how co
[appengine-java] Re: GAE status / availalbility problems?
Agreed...the Developer should always work diligently to find the source of the problem - what a revelation! The issue here is if Dmitry had several hundred thousand Users waiting a week or so on this issue to be resolved...how many would he lose as a result? How many new sign-ups would be lost? Negative User experiences will reflect not only on our apps but on GAE too. We are in this together. I think Google will need a paid subscription/ticket model to address escalated tickets related to lucrative production apps. On Nov 19, 1:36 pm, Dmitry Anipko wrote: > After Jason replied, it took a day or so. > > Thank you for pointing that out, yes I think we can agree on that :) > > > > On Wed, Nov 18, 2009 at 11:35 PM, Diana Cruise > wrote: > > Excellent news. How long did it take to resolve this matter and what > > was the impact to your Users? > > > I assume you would agree we could avoid this type of problem by only > > using the data types listed here right? > > >http://code.google.com/appengine/docs/java/datastore/dataclasses.html... > > > On Nov 18, 3:27 pm, Dmitry Anipko wrote: > >> Thanks Jason for help. In our case the problem was caused by usage of > >> byte[] field in a persisted class. Somehow though, it wasn't causing > >> any problems immediately, but only after a few objects have been > >> saved. Per Jason, usage of byte[] in JDO on GAE isn't supported. > > >> On Wed, Nov 18, 2009 at 1:00 PM, Diana Cruise > >> wrote: > >> > What's the latest here Dmitry...can GAE handle more than 100 request > >> > or not? Please update where you are with this problem. Thanks. > > >> > On Nov 9, 5:15 pm, Dmitry Anipko wrote: > >> >> Following up offline and will update with resolution later. > > >> >> On Mon, Nov 9, 2009 at 9:50 AM, Jason (Google) > >> >> wrote: > >> >> > As always, I need your application ID in order to help troubleshoot > >> >> > app-specific issues. Please provide that and any stack traces or > >> >> > other error > >> >> > messages you see in your application's logs. > > >> >> > - Jason > > >> >> > On Fri, Nov 6, 2009 at 9:47 PM, Dmitry Anipko > >> >> > > >> >> > wrote: > > >> >> >> After some more experimentation, it looks like there is something in > >> >> >> the data store that is surfaced by our application: > > >> >> >> if I clean up the store for the application completely, using the web > >> >> >> interface, then for the first ~100 request it works fine, but after > >> >> >> some time, it starts producing those errors I wrote before - it can't > >> >> >> query objects by index (and that problem persists until I clean the > >> >> >> store again). > > >> >> >> The second observation - although I don't know if it is related or > >> >> >> not > >> >> >> - is when the store for my app is in such state, and I try to view > >> >> >> objects through the web interface, the web interfaces crashes with > >> >> >> the > >> >> >> error 500: > > >> >> >> Server Error > >> >> >> A server error has occurred. > > >> >> >> Return to Applications screen » > > >> >> >> GAE folks, can someone please take a look - this is a blocking issue > >> >> >> for my team, and I don't know how to proceed here without your help. > > >> >> >> Everything works just fine when I run it on a local development > >> >> >> server. > > >> >> >> Thank you. > > >> >> >> On Nov 6, 11:05 am, Diana Cruise wrote: > >> >> >> > I see NO reply from GAE here regarding such a critical issue!!! > > >> >> >> > Is your app still down, what is the status of your app? How are we > >> >> >> > supposed to put an application in production with GAE if such a > >> >> >> > problem can occur with NO response for 3 days? Is there another > >> >> >> > channel to raise such high-priority issues to get proper helpdesk > >> >> >> > response? > > >> >> >> > On Nov 4, 1:58 am, Dmitry Anipko wrote: > > >> >> >> > > It is 11:56 Pacific 11/3/2009, are there any kno
[appengine-java] Re: Why is it called "Google App Engine for Java" ?
I'm curious which classes you are referring to that are missing and are NOT considered a stability risk running under a shared app server environment. We are developing in Java with this solution so I don't mind the naming...I have yet to need a class not offered, guess I have been lucky :) So, what would you recommend calling it? On Nov 27, 12:04 pm, jago wrote: > Hi, > > this is not Java! The whitelist is > ridiculous:http://code.google.com/appengine/docs/java/jrewhitelist.html > > There is so much missing I don't even know where to start. Is there an > issue I can star that extends the whitelist to the full Java6 lib? > > This is destroying the Java standard and shouldn't be called Java. > Every other company that tries to do an implementation of Java and > call it Java gets a massive punch from Sun. Why is Google allowed to > use Java6 with a whitelist and still call it Java? -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Why is it called "Google App Engine for Java" ?
Jago...in shared environments you can't let apps launch their own threads, you can't let apps takeover file systems, etc...these are basic principles for shared resources such as GAE and, for example, contradicts J2EE specs such as EJB and so on. Such apps belong on dedicated servers. Also, loading of images to datastore is possible with VFS...as far as editing images, I don't know. Ted... java.lang.Thread, you want to launch new processes from within your app server...that's a job for URLFetch. Anyhow, I was just curious at what you couldn't do...good to share that with the community in case they know something you don't. As far as the naming goes, you may be the first to raise this concern in GAE's existence (2 years or so). On Nov 27, 6:07 pm, marksea wrote: > > In the end all they > > support is Java syntax and a couple of classes. > > I count 1332. Since you're so particular about your terminology, you > may want to look up "a couple." -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Why is it called "Google App Engine for Java" ?
I agree with Ravi and Bryce...we have other fish to fry (threads to answer) and Java is a programming language reference which doesn't imply every library under the sun (pun intended). On Dec 1, 4:19 am, bryce cottam wrote: > the GAE isn't cross-compiling, javac is doing the compiling (the real > java compiler), so it's kind of a moot argument. The point is, the > system runs java, not g-java, not pseudo-java, it's java. It just > doesn't have all the libraries you want. That's all. It's written in > the Java programming language (or Scala, or Groovy JRuby or anything > else that runs on the JVM) and runs on a JVM... it's Java. > > > > On Tue, Dec 1, 2009 at 2:18 AM, Bobby wrote: > > Suppose someone writes an app in C and wishes to cross compile to > > Java. In that scenario, if some C class is unsupported by the cross- > > compiler, then does the C app stop being a C app? No. > > > Bobby > > > On Dec 1, 4:10 am, Bobby wrote: > >> Maybe for terminology's sake it would make sense to detach the GAE > >> from any language associations and just indicate that there are cross- > >> compilers from Java/Python to GAE. Then there would be no question on > >> whether it's "technically Java" - one does write in Java, it's just > >> that the cross-compiler has its own requirements and limitations. Is > >> that any better? :) > > >> Bobby > > >> On Nov 30, 2:58 pm, Nicolas Melendez > >> wrote: > > >> > if we are going to collaborate each other, we need the right names > >> > because > >> > communication is critical. > > >> > On Mon, Nov 30, 2009 at 4:37 PM, Jess Evans > >> > wrote: > >> > > There are innovators who "attempt" to advance the field and be the > >> > > first. > >> > > They appeal to the desperate and the early adopters with nothing to > >> > > loose > >> > > and much to gain. > > >> > > There are guardians with deep experience who strive to ensure > >> > > reliability > >> > > and predictability. They appeal to those upon whom many are dependent; > >> > > those with everything to lose and little to gain. > > >> > > Both camps are invaluable. > > >> > > Incorrectly categorizing a technology, or failing to understand who > >> > > you're > >> > > employed to represent, are personal problems. > > >> > > On Nov 30, 2009 12:22 PM, "Jeff Schnitzer" wrote: > > >> > > I, for one, am sick and tired of Sun's domineering, suffocating > >> > > stranglehold on what is and isn't Java. GAE is a breath of fresh air. > > >> > > JavaME and JavaEE also impose a variety of limitations on Java. > >> > > What's the difference between those and GAE? The difference is that > >> > > Sun got a committee of big company representatives with their own > >> > > vested interests in a closed room and argued about how "the official > >> > > specs" should work for years... and ultimately produced a bunch of > >> > > crap that is barely useful to anyone. Just *try* out Websphere > >> > > sometime, I dare you. > > >> > > For the first time ever, someone has produced a shared application > >> > > service model for Java that's even easier than PHP. This could never > >> > > have come out of the JCP. > > >> > > If this is "destroying java", then GOOD RIDDANCE! > > >> > > Jeff > > >> > > -- You received this message because you are subscribed to the Google > >> > > Groups "Google App Engine fo... > > >> > > -- > >> > > You received this message because you are subscribed to the Google > >> > > Groups > >> > > "Google App Engine for Java" group. > >> > > To post to this group, send email to > >> > > google-appengine-j...@googlegroups.com. > >> > > To unsubscribe from this group, send email to > >> > > google-appengine-java+unsubscr...@googlegroups.com > >> > > . > >> > > For more options, visit this group at > >> > >http://groups.google.com/group/google-appengine-java?hl=en. > > > -- > > > You received this message because you are subscribed to the Google Groups > > "Google App Engine for Java" group. > > To post to this group, send email to google-appengine-j...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine-java+unsubscr...@googlegroups.com. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine-java?hl=en.- Hide quoted > > text - > > - Show quoted text - -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: NullPointerException while running a new app
Help anyone...the key here is the stack trace but it points to a line in the generated .java file that came from the .jsp. I am also getting a null pointer but when I check the line number using the generated .java on my local system, it does NOT make sense. Hence, the line number in production is NOT matching the line numbers locally!!! How do we affectively trouble-shoot a JSP stack dump on GAE? On Mar 13, 3:07 pm, anjolight wrote: > Hi I am getting the same NullPointerException. > > The challenge is that my app runs fine in my local eclipse but it > gives me this error when it's deployed to app engine. And the error > gives me very little information as to what went wrong. > > What's the best way to debug the nullpointerexception with JSP? > > On Feb 24, 5:05 pm, "Ikai L (Google)" wrote: > > > > > What's your application ID? > > > On Fri, Feb 19, 2010 at 12:05 AM, Alberty Pascal > > wrote: > > > > Hi all, > > > > while updating my application, I got the following stack for all > > > request made on it. > > > I try to deploy a really simple new application (created with Eclipse > > > and its plugin) and got the same stack. > > > > Any idea ? > > > > Thanks > > > > Pascal > > > > java.lang.NullPointerException > > > at > > > com.google.appengine.runtime.Request.process-7d335e6a040b96cc(Request.java) > > > at org.apache.jsp.index_jsp._jspService(index_jsp.java:44) > > > at > > > org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:94) > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) > > > at > > > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: > > > 487) > > > at > > > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java: > > > 362) > > > at > > > org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java: > > > 216) > > > at > > > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java: > > > 181) > > > at > > > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java: > > > 712) > > > at > > > org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java: > > > 405) > > > at > > > org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:268) > > > at > > > org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:126) > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:693) > > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) > > > at > > > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: > > > 487) > > > at org.mortbay.jetty.servlet.ServletHandler > > > $CachedChain.doFilter(ServletHandler.java:1093) > > > at > > > > com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java: > > > 97) > > > at org.mortbay.jetty.servlet.ServletHandler > > > $CachedChain.doFilter(ServletHandler.java:1084) > > > at > > > > com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java: > > > 35) > > > at org.mortbay.jetty.servlet.ServletHandler > > > $CachedChain.doFilter(ServletHandler.java:1084) > > > at > > > > com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java: > > > 43) > > > at org.mortbay.jetty.servlet.ServletHandler > > > $CachedChain.doFilter(ServletHandler.java:1084) > > > at > > > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java: > > > 360) > > > at > > > org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java: > > > 216) > > > at > > > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java: > > > 181) > > > at > > > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java: > > > 712) > > > at > > > org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java: > > > 405) > > > at > > > > com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java: > > > 238) > > > at > > > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java: > > > 139) > > > at org.mortbay.jetty.Server.handle(Server.java:313) > > > at > > > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java: > > > 506) > > > at org.mortbay.jetty.HttpConnection > > > $RequestHandler.headerComplete(HttpConnection.java:830) > > > at > > > > com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java: > > > 76) > > > at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381) > > > at > > > > com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java: > > > 135) > > > at > > > com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java: > > > 235) > > > at com.google.apphosting.base.RuntimePb$EvaluationRuntime > >
[appengine-java] Re: How to upload pics in appengine java
Also this guy Vince has a package that not only helps with uploads but also then allowing your app to generate pages that use the photos including delta checks - really neat stuff! See GAEVFS mentioned in this thread. Please post your results so the rest of us can benefit from your experience. http://groups.google.com/group/google-appengine-java/browse_thread/thread/a4ee0a7c54453586/b1f2984b43c920a1?hl=en#b1f2984b43c920a1 On Mar 14, 12:51 am, seleronm wrote: > Hi, > > I think that the following links are > useful.http://stackoverflow.com/questions/1513603/ > > Please try. > thanks. > > > > >AFAIK almost any file can be apart of the docs that you upload (images, css, > >html, js, jsp, ico). I haven't had any trouble uploading images. > > >Are you talking about accepting images from an input form? In that case you > >can store them as a blob type in the datastore. > > >Hope this helps > > >On Sun, Mar 14, 2010 at 12:12 AM, nag wrote: > > >> Hi > >> I am trying to build online reg form > >> can u help me how to upload image files to app engine. > > >> First: is there any way to upload images in app engine? > > >> thanks > >> Nag > > >> -- > >> You received this message because you are subscribed to the Google Groups > >> "Google App Engine for Java" group. > >> To post to this group, send email to > >> google-appengine-j...@googlegroups.com. > >> To unsubscribe from this group, send email to > >> google-appengine-java+unsubscr...@googlegroups.com >> 2bunsubscr...@googlegroups.com> > >> . > >> For more options, visit this group at > >>http://groups.google.com/group/google-appengine-java?hl=en. > > >-- > >-Pav > > >-- > >You received this message because you are subscribed to the Google Groups > >"Google App Engine for Java" group. > >To post to this group, send email to google-appengine-j...@googlegroups.com. > >To unsubscribe from this group, send email to google-appengine-java+ > >unsubscr...@googlegroups.com. > >For more options, visit this group athttp://groups.google.com/group/google- > >appengine-java?hl=en. > > >---html-part included links--- > >mailto:nagarjuna...@gmail.com > >mailto:google-appengine-java@googlegroups.com > >mailto:google-appengine-java%2bunsubscr...@googlegroups.com > >http://groups.google.com/group/google-appengine-java?hl=en- Hide quoted text > >- > > - Show quoted text - -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
