Re: [appengine-java] Re: Why should app startup times be a problem.
Warmed instances make absolutely no sense on the GAE! It's supposed to be an invisible, infinite, platform without the notion of ram, cpu's, drives or instances - it goes completely against the heart and soul of the project. That just makes us an Amazon. It's boggled my mind from day 1 why instances aren't loaded in the background. I always assumed it would be addressed shortly - but instead we've decided to completely break the model. Why? I hate to over-simply, but it really doesn't seem like it's that difficult to make sure requests only go to ready/warm servers. I bet you could even solve 99% of the cases by simply waiting 3 minutes before serving a request. Of course, I still love the GAE :) Baz -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
Shawn, By the same token google could cut all their hardware in half and double page loads, check out this formula: NumberOfRequests / HardwareQuality = PageLoad Therefore, Google should power the entire GAE infrastructure with a netbook - yes each page would take 2.5 decades to load, but imagine the cost savings! The main point of my post is the model, not the money. I am more than willing to pay for AppEngine as I love it and think it's the future. The issue is that I don't want to pay and manage "instances" because in my opinion that goes against the model. I would prefer to pay a higher rate (or the equivalent money some other way) without the knowledge of instances. With all that said, and re-reading some other posts, I may actually be wrong here... To be sure, if I pay for a warmed instance and, say, my site gradually becomes more and more popular and gradually goes from 1 instance to 11 - have I received the benefit of a warmed instance at every step of the way? Is my twelfth instance now the "warm" one? Cheers, Baz -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Why should app startup times be a problem.
Great information, Ikai. I really feel that "instances" should be completely avoided in concept and language on the GAE. What if the feature was simply an enable/disable deal called "Warm Scale". If it were enabled, then your *next* instance would always be warm, regardless of how many instances you already had. This would be most noticeable and suitable for low QPS production apps that are constantly going from 0 to 1 instances (as you mentioned), but it could still be important for others, say, for a super-high-profile site, or a situation where your QPS is right at the threshold of instances and oscillating back and forth between two instances. Whatever the situation, if the solution were generalized like that, and most importantly not tied to a SPECIFIC NUMBER of instances, it would be up to the user to decide how important it was for them and whether to enable it. Cheers, Baz -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: App Engine and Spring slow start up
> > I do not understand why there is not at least a way to keep the apps > running. I would even pay for that. On deck is the ability to pay for and have reserved a warmed instance: http://code.google.com/appengine/docs/roadmap.html (3rd bullet) Baz -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Facebook/social website design problem
I think you're best bet is to do all the work in the write, and have each
profile's page constantly updated and ready for an extremely simple read.
For one, this will significantly reduce your costs, because everyone reads
more than they write. Two, by the time you get big enough to surpass the 30
second limit, you will probably be able to use "Background servers capable
of running for longer than 30s" (
http://code.google.com/appengine/docs/roadmap.html)
Cheers,
Baz
On Tue, May 18, 2010 at 8:43 AM, Ravi wrote:
> Thanks Li for looking into it.
> For me 1-2 minute delay is fine, problem is like contains query(for
> finding friend's data), i can pass max 30 user id, so basically on one
> page i will be able to show max 30.
> But in other case where wall type data need to be saved queries need
> to perform faster and need to break the limit of 30 as i would like to
> get latest activity from all of my friends rather then first 30
> friends. So if i have 300 friends i may need to run such query 10
> times in the batch of 30 nad each time need to go through activties
> and sort them out which will be very time consuming and sure user will
> not like the response time.
> I am thinking why not do it at the write time that my updates goes to
> all of friend's domain but again in one 30 second request i will try
> to create/update my domain(user Data) + 300(assuming i have 300
> friends) other domains. I am not sure how it will perform.
>
> But i would love to see some solution where write is less and still
> read is faster like any other site(facebook/twitter etc)
>
>
> Ravi.
>
>
> On May 18, 4:31 pm, Yiming Li wrote:
> > This is an interesting problem, I am not an expert on database design,
> > especially key-value based datastore, but I will try to answer this
> > question.
> >
> > For me, the design like
> > Class User{
> > Key id;
> > String name;
> > String email;
> > List friends.
> > ...etc}
> >
> > makes perfect sense, and as you said, you can do more query on
> > something like "UserActivity" or "TimeBecomeFriends", with
> > "user1Id= or user2Id=". approach. I am also fine with
> > it.
> >
> > How big is your application? I think a good design is relevant to the
> > scalability, and we are always trading storage space with query time.
> >
> > Another thing maybe useful is memcache, using it may lead to a gain in
> > performance, although it may bring a bit data inconsistency, however
> > I think it's ok. For example, it should be fine if you see your
> > friends' latest post 1 minute later.
> >
> > Maybe there is some paper talking about datastore design for scalable
> > applications? Not sure.
> >
> > Thanks
> >
> >
> >
> >
> >
> > On Tue, May 18, 2010 at 7:58 AM, Ravi wrote:
> > > Hi,
> > > I am trying to design a website like Facebook, where user will have
> > > friends and will have wall where they can see there friend's updates
> > > etc. While designing it on GAE i got confused with what will be the
> > > best approach like data duplicate, data read time or data write time
> > > etc
> >
> > > Friend's Design:
> > > I thought there will be User and User will be friend with another
> > > User.
> >
> > > Class User{
> > > Key id;
> > > String name;
> > > String email;
> > > ...etc
> > > }
> >
> > > Now i definitely can not have owened relatioship with Friends means I
> > > can not have List friends in my User Class as it will not be
> > > same domain(User) data, so i thought ok i will have List or
> > > List User Ids as Friends
> > > Class User{
> > > Key id;
> > > String name;
> > > String email;
> > > List friends.
> > > ...etc
> > > }
> > > And confusion/problem begins.
> > > First a Friendship is supposed to be from both side, so it should be
> > > defined only once but in this design both Friends will have each
> > > other's id in there friend list. Basically its not a typical
> > > relationship. What if i want to keep friendship info like when it
> > > started, who introduced etc. Then i may need to create a new Friend
> > > class with two fields as user1Id and user2Id and query this class with
> > > user1Id= or user2Id=. But not sure about this
> > > approach.
> > > Second i am doing data duplicate(but i am fine with it if it can be
> > > proved as best solution).
> > >
[appengine-java] Advice for looping through 100K records and updating multiple entities on each iteration
Hello, I have a non-user triggered process that queries the datastore for a dataset of about 100k records, then loops through each record to update two other entities in the datastore. What would be a good way to go about doing this? I'm concerned about timing out and/or consuming more resources than I need to. Task queue? Cheers. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Frustrations with Performance
For a work-around you can cron a hit to your site every minute to keep it live. On Wed, Jul 7, 2010 at 11:01 AM, Jake wrote: > Hey, > > Are you getting frequent instance restarts? Check the log for loading > requests. This has been known to happen on sites with little-to-no > traffic. > > Solution? None, but if this issue is affecting you, star this: > http://code.google.com/p/googleappengine/issues/detail?id=2931 > > Jake > > On Jul 7, 11:54 am, praseed wrote: > > Folks, > > > > I hope I have the right audience. > > > > I am an Independent Software Vendor (read: small software business) > > and I have developed a GWT/GAE product. > > > > Things were good during development (plugin et al). I was able to get > > features developed quickly. > > > > However, the application performance these days has been dicey to say > > the least. > > > > A simple click of a button tends to get 5 to 6 seconds to respond at > > times. Dont have any concurrent users right now. The Appspot dashboard > > does not show any issues and I am well below the quota. > > > > I have upgraded to the latest GAE engine. > > > > The App is absolutely horrendous to use due to this. As an ISV, I > > cannot demo this to my prospective customers and I cant "market" the > > cloud availability, Google performance etc.., -- coz that seems to be > > the downfall of my app. > > > > The App performs way better on my laptop. The moment it is deployed, > > it is bloody slow. > > > > I am aware of the current datastore issues at Google. Is that the sole > > reason for this , almost pervasive issue? > > > > Google AppEngine for Business has been announced. Any time tables? > > Will it address these issues? > > > > Can someone address this? How about other developers? Can anyone chime > > in here... > > > > Cheers > > Strawman > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > google-appengine-j...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
