[google-appengine] Re: Globally monotonic counter
On Jul 16, 10:35 pm, Andy Freeman wrote: > > I'm starting to think that the > > "GAE takes > > care of the messy details of distributed systems programming" claim is > > a bit overstated... > > Global clock consistency requires very expensive clocks accessible > from every server with known latency (and even that's a bit dodgy). > AFAIK, GAE doesn't provide that, but who does? > > GAE doesn't do the impossible, but also doesn't say that it does. WRT > the latter, would you really prefer otherwise? But that's just it -- in many places it's claimed that GAE makes it all a cakewalk. From the datastore docs: """ Storing data in a scalable web application can be tricky. A user could be interacting with any of dozens of web servers at a given time, and the user's next request could go to a different web server than the one that handled the previous request. All web servers need to be interacting with data that is also spread out across dozens of machines, possibly in different locations around the world. Thanks to Google App Engine, you don't have to worry about any of that. App Engine's infrastructure takes care of all of the distribution, replication and load balancing of data behind a simple API—and you get a powerful query engine and transactions as well. """ You could argue that that's not claiming to do the impossible, but "you don't have to worry about any of that" is certainly not true. Nowhere in the documentation is there a discussion of the kinds of subtle gotchas that you need to be aware of when programming for this kind of system. It's all just "golly isn't this so gosh-darn easy!" You have to go digging to find the article on transaction isolation where you find out that your queries can return results that, um, don't match your queries. And AFAICT you *do* have to worry about subsequent requests being handled by different servers, since there doesn't seem to be any guarantee that the datastore writes made in one request will be seen in the next. Memcache doesn't have transactions, so it seems like guaranteeing coherence with the datastore is tricky. I worked in a distributed systems group for many years, so I know that many of these problems are simply inherent to distributed systems. It doesn't disturb me that they exist. What bothers me is the way these issues are broadly *ignored* by GAE's documentation. If I wasn't a bit savvy about distributed systems I probably wouldn't have realized that clock skew could cause problems, and nothing I read in GAE's docs would have helped me figure it out. So no, I don't want GAE to claim to do the impossible, I want them to *stop* claiming to do the impossible. I would love to see some articles about the pitfalls of the system and how to avoid them or mitigate them. The transaction isolation article is great in that respect -- I hope people at Google are planning more along those lines. Cheers, -n8 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: how fast is google app engine
> Unfortunatley I don't see it as a good indication as to how fast it > will run any gae application > as such a benchmark doesn't take into account things like, network > access, file I/O, datastore access etc > which are all crucial to how a web app performs. In adition trying to Yes, you are right. But, what I am trying to reveal with this website is just one factor out of those many variables. Just computing power, or CPU performance. I have updated the website with Pystone value from my local machine as a reference value, so that people can have an idea how much computing power GAE cloud environment have compared to contemporary CPU. Another goal of the website is to catch the trend of computing power with the hope that it will improve in time. Cosmos Web Solutions --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: how fast is google app engine
> Unfortunatley I don't see it as a good indication as to how fast it > will run any gae application > as such a benchmark doesn't take into account things like, network > access, file I/O, datastore access etc > which are all crucial to how a web app performs. In adition trying to Yes, you are right. But, what I am trying to reveal with this website is just one factor out of those many variables. Just computing power, or CPU performance. I have updated the website with Pystone value from my local machine as a reference value, so that people can have an idea how much computing power GAE cloud environment have compared to contemporary CPU. Another goal of the website is to catch the trend of computing power in time with the hope that it will improve in time. Cosmos Web Solutions --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Eating one's own dog food
> > If you need > more, try to create multiple account, schedule roster among them, sync your > data among them. This is the solutions. If terms and conditions of use of > GAE allows, there will be open souce projects for GAE clustering. I'm pretty sure that the GAE TOS forbid that "solution". On Jul 16, 7:26 pm, Juguang XIAO wrote: > Dogfooding, as wikipedia names it, is ideological, may not be practical. > People, as well as company, lives practically for surviving first then > chasing dreams. > > I have seen the positive movement from Google, did so much for developers. > People may take it for granted, thinking the leader should do more. I am > content with GAE, as it offers some free and exciting stuffs. I cannot ask > Google to give more, unless he decided so. People can be happy when they are > grateful. > > Technically speaking, I do not think Google offers its best to developers. > It will be too costy to do so. Their mainstream businesses need to be > maintained, and I guess each business unit has its own authority and freedom > to do thing in their own way. Core businesses and technologies need to be > protected. If you are not happy, go for Microsoft. ;-) > > I do not believe, people can run their serious business without serious pay. > 6.5 hour per day CUP time is enough for casual applications. If you need > more, try to create multiple account, schedule roster among them, sync your > data among them. This is the solutions. If terms and conditions of use of > GAE allows, there will be open souce projects for GAE clustering. > > Juguang > > > > > > On Fri, Jul 17, 2009 at 10:03 AM, GenghisOne wrote: > > > So it looks like there's an updated Google App Engine roadmap and > > guess what...no mention of full-text search. > > > Doesn't that strike anyone as a bit odd? How can an emerging cloud > > computing platform not effectively address full-text search? And > > what's really odd is the absolute silence from Google...quite frankly, > > I don't get it. > > > On Jul 16, 12:28 pm, Bryan wrote: > > > This is a very interesting discussion. I would like to see some input > > > from Google. > > > > On Jul 15, 10:20 am, richard emberson > > > wrote: > > > > > I understand that BigTable is behind GAE, but my concern is > > > > more with GAE performance and quotas. If GAE had existed > > > > when Larry and Sergey were developing their pagerack > > > > algorithm, would they have used GEA for evaluation? > > > > I have my doubts. They would quickly reach quota limits, > > > > way before they knew if they had a viable idea. > > > > > Richard > > > > > Tony wrote: > > > > > Though I realize this is not exactly what you're asking, the concept > > > > > of GAE is that it exposes some of the infrastructure that all Google > > > > > applications rely on (i.e. Datastore) for others to use. So, in a > > > > > sense, Google's various applications were using App Engine before App > > > > > Engine existed. As far as I know, every Google service runs on the > > > > > same homogeneous infrastructure, which is part of what makes it so > > > > > reliable (and why the only available languages are Python and Java, > > > > > languages used internally at Google). > > > > > > But I don't work there, so maybe I'm completely off-base. > > > > > > On Jul 15, 12:53 pm, richard emberson > > > > > wrote: > > > > >> Eating one's own dog foodhttp:// > > en.wikipedia.org/wiki/Eating_one's_own_dog_food > > > > >> or in this case: > > > > >> Using one's own cloud. > > > > > >> Amazon' cloud is based upon the IT technology they use > > > > >> within Amazon. > > > > >> Salesforce.com's Force.com offering is what they used to > > > > >> build their CRM system. > > > > > >> These cloud vendors "Eat their own dog food". > > > > > >> If a cloud vendor does not use their cloud offering for > > > > >> their other products and/or internal systems, one > > > > >> would have to assume that the cloud is viewed as > > > > >> a technology ghetto within their own corporation - good > > > > >> enough for others but not for ourselves. > > > > > >> So, concerning the Google App Engine, are other groups > > > > >> within Google clamoring to port or build their offerings > > > > >> on top of the App Engine? If so, please be specific, what > > > > >> Google products and infrastructure and what are the schedules > > > > >> for their hosting on GAE? > > > > > >> Is the GAE group supporting the Google Docs group as they > > > > >> move to use GAE? How about gmail, will the Google Gmail > > > > >> group be relying on GAE support? I have not seen emails > > > > >> from either of those internal Google groups on the GAE > > > > >> mailing list. Lastly, when will Google search be supported > > > > >> by the GAE group; > > > > > >> Will those groups have to live under the same quota restrictions > > > > >> while they evaluate using GAE? If not, why not? If they >
[google-appengine] Re: Eating one's own dog food
> If GAE had existed > when Larry and Sergey were developing their pagerack > algorithm, would they have used GEA for evaluation? No, but that has nothing to do with quotas. GAE is pretty far from a reasonable platform for a search engine. > They would quickly reach quota limits, > way before they knew if they had a viable idea. If GAE's free quota is not sufficient to evaluate your idea, perhaps you should evaluate it using a service that provides sufficent free quota. On Jul 15, 10:20 am, richard emberson wrote: > I understand that BigTable is behind GAE, but my concern is > more with GAE performance and quotas. If GAE had existed > when Larry and Sergey were developing their pagerack > algorithm, would they have used GEA for evaluation? > I have my doubts. They would quickly reach quota limits, > way before they knew if they had a viable idea. > > Richard > > > > > > Tony wrote: > > Though I realize this is not exactly what you're asking, the concept > > of GAE is that it exposes some of the infrastructure that all Google > > applications rely on (i.e. Datastore) for others to use. So, in a > > sense, Google's various applications were using App Engine before App > > Engine existed. As far as I know, every Google service runs on the > > same homogeneous infrastructure, which is part of what makes it so > > reliable (and why the only available languages are Python and Java, > > languages used internally at Google). > > > But I don't work there, so maybe I'm completely off-base. > > > On Jul 15, 12:53 pm, richard emberson > > wrote: > >> Eating one's own dog > >> foodhttp://en.wikipedia.org/wiki/Eating_one's_own_dog_food > >> or in this case: > >> Using one's own cloud. > > >> Amazon' cloud is based upon the IT technology they use > >> within Amazon. > >> Salesforce.com's Force.com offering is what they used to > >> build their CRM system. > > >> These cloud vendors "Eat their own dog food". > > >> If a cloud vendor does not use their cloud offering for > >> their other products and/or internal systems, one > >> would have to assume that the cloud is viewed as > >> a technology ghetto within their own corporation - good > >> enough for others but not for ourselves. > > >> So, concerning the Google App Engine, are other groups > >> within Google clamoring to port or build their offerings > >> on top of the App Engine? If so, please be specific, what > >> Google products and infrastructure and what are the schedules > >> for their hosting on GAE? > > >> Is the GAE group supporting the Google Docs group as they > >> move to use GAE? How about gmail, will the Google Gmail > >> group be relying on GAE support? I have not seen emails > >> from either of those internal Google groups on the GAE > >> mailing list. Lastly, when will Google search be supported > >> by the GAE group; > > >> Will those groups have to live under the same quota restrictions > >> while they evaluate using GAE? If not, why not? If they > >> are unreasonable for an internal evaluation, what makes them > >> reasonable for an external evaluation? > > >> Evaluating whether or not GAE should be used for a particular > >> application is not FREE even if one gets a very small slice > >> of GAE resources with which to do the evaluation. > >> Tens or hundreds of hours go into determine if GAE has > >> the right characteristics and quotas that limit how fast one > >> can work makes it worse. (Yes one can $$ for higher quotas, > >> but during the evaluation phase $$ is out of the question.) > > >> Richard Emberson > > >> -- > >> Quis custodiet ipsos custodes > > -- > Quis custodiet ipsos custodes- Hide quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Globally monotonic counter
> I'm starting to think that the > "GAE takes > care of the messy details of distributed systems programming" claim is > a bit overstated... Global clock consistency requires very expensive clocks accessible from every server with known latency (and even that's a bit dodgy). AFAIK, GAE doesn't provide that, but who does? GAE doesn't do the impossible, but also doesn't say that it does. WRT the latter, would you really prefer otherwise? On Jul 16, 3:47 pm, n8gray wrote: > Thanks for the advice, Nick. I'd still like to know more about the > consistency model though. For example, I wonder if there's any > guarantee that two transactions on different entity groups executed by > one process in a given order will be observed in the same order. I > suspect the answer is no. I'm starting to think that the "GAE takes > care of the messy details of distributed systems programming" claim is > a bit overstated... > > Cheers, > -n8 > > On Jul 14, 2:27 am, "Nick Johnson (Google)" > wrote: > > > > > Hi Nathan, > > > Your best options are either to keep track of one event stream per > > game, or to use system time, and 'rewind' the timestamp a bit to > > capture any missed events, as you suggest. Global monotonic counters > > aren't very practical in large distributed systems. > > > -Nick Johnson- Hide quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Globally monotonic counter
> I think I can live without uniqueness as > long as timestamps don't go backwards. Timestamps on a given server probably don't go backwards, but there's no guarantee about the relationship between clocks on different servers. On Jul 13, 11:16 pm, n8gray wrote: > Hi Albert, > > Thanks for the suggestion. I think I can live without uniqueness as > long as timestamps don't go backwards. But I think my problem still > exists with a sharded counter. Having a counter is certainly better > than using datetime.utcnow() and lets me assign an order to all events > that have been generated, but that's not really the problem. The > tricky part is deciding, on the client end, which events to request > based on the events you've received. When the client asks for "all > events after time T" it gets some "last" event with a new timestamp > S. But I don't think you can trust this S because there might be some > other event in some corner of GAE with an earlier timestamp that > hasn't yet been observed by the server that answered the client's > request. > > I guess the root of the problem is that I know that transactions on > entity groups give me the ACID properties but when it comes to > operations outside of transactions I have no idea what the consistency > model is. Has this been described somewhere? > > Thanks, > -n8 > > On Jul 13, 7:06 pm, Albert wrote: > > > > > Hi! > > > This is a quick suggestion. > > > How about using a global counter (just like your title suggests). You > > can use a sharded global counter to facilitate your counting. > > > And use that counter as a "timestamp" / bookmark. > > > On every "event", you read from the global counter, use that value as > > your "timestamp", and then increment the global counter. > > > I'm not sure of it's implications, though. I'm not also sure if it > > actually guarantees uniqueness of "timestamps" when two events happen > > almost at the same time. > > > Perhaps you can get an idea from this. > > > Enjoy!- Hide quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: CPU time for trivial datastore roundtrip
More testing reveals that requesting by key costs 12 api_cpu_ms per result returned, plain and simple. No bonus for batching whatsoever. So, to sum up my testing so far: Getting (using knowledge of keys) seems to take 12 ms * results. Querying seems to take 17.5 ms + 13.2 ms * results. Inserting seems to take 3204 ms + 345 ms * results. Deleting seems to take about the same as inserting. // Anthony --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Eating one's own dog food
Hi there. I did not pay anything for any services, inc. gmail, google docs, gmaps, to Google.:-) She got big pays by someone else. For GAE as hosting provider, i agreed that for exceeding quota it may be more expensive than other providers. However, different people have different ways to calcuate the cost. As for me as an individual, I am not thinking to pay for it at this moment. :-p Juguang On Fri, Jul 17, 2009 at 10:38 AM, GenghisOne wrote: > > Hi Juguang > > Your words provide a nice counter-balance to those who are not so > happy with App Engine. Yes, you are correct in reminding us all that > we should be grateful for something free. And yes App Engine has > likely provided a whole bunch of young, inquiring minds with an > amazing opportunity to tinker and learn on a global scale. > > But for some strange reason, I'm also reminded of that old > saying...you get what you pay for. > > > On Jul 16, 7:26 pm, Juguang XIAO wrote: > > Dogfooding, as wikipedia names it, is ideological, may not be practical. > > People, as well as company, lives practically for surviving first then > > chasing dreams. > > > > I have seen the positive movement from Google, did so much for > developers. > > People may take it for granted, thinking the leader should do more. I am > > content with GAE, as it offers some free and exciting stuffs. I cannot > ask > > Google to give more, unless he decided so. People can be happy when they > are > > grateful. > > > > Technically speaking, I do not think Google offers its best to > developers. > > It will be too costy to do so. Their mainstream businesses need to be > > maintained, and I guess each business unit has its own authority and > freedom > > to do thing in their own way. Core businesses and technologies need to be > > protected. If you are not happy, go for Microsoft. ;-) > > > > I do not believe, people can run their serious business without serious > pay. > > 6.5 hour per day CUP time is enough for casual applications. If you need > > more, try to create multiple account, schedule roster among them, sync > your > > data among them. This is the solutions. If terms and conditions of use of > > GAE allows, there will be open souce projects for GAE clustering. > > > > Juguang > > > > > > > > On Fri, Jul 17, 2009 at 10:03 AM, GenghisOne > wrote: > > > > > So it looks like there's an updated Google App Engine roadmap and > > > guess what...no mention of full-text search. > > > > > Doesn't that strike anyone as a bit odd? How can an emerging cloud > > > computing platform not effectively address full-text search? And > > > what's really odd is the absolute silence from Google...quite frankly, > > > I don't get it. > > > > > On Jul 16, 12:28 pm, Bryan wrote: > > > > This is a very interesting discussion. I would like to see some > input > > > > from Google. > > > > > > On Jul 15, 10:20 am, richard emberson > > > > wrote: > > > > > > > I understand that BigTable is behind GAE, but my concern is > > > > > more with GAE performance and quotas. If GAE had existed > > > > > when Larry and Sergey were developing their pagerack > > > > > algorithm, would they have used GEA for evaluation? > > > > > I have my doubts. They would quickly reach quota limits, > > > > > way before they knew if they had a viable idea. > > > > > > > Richard > > > > > > > Tony wrote: > > > > > > Though I realize this is not exactly what you're asking, the > concept > > > > > > of GAE is that it exposes some of the infrastructure that all > Google > > > > > > applications rely on (i.e. Datastore) for others to use. So, in > a > > > > > > sense, Google's various applications were using App Engine before > App > > > > > > Engine existed. As far as I know, every Google service runs on > the > > > > > > same homogeneous infrastructure, which is part of what makes it > so > > > > > > reliable (and why the only available languages are Python and > Java, > > > > > > languages used internally at Google). > > > > > > > > But I don't work there, so maybe I'm completely off-base. > > > > > > > > On Jul 15, 12:53 pm, richard emberson < > richard.ember...@gmail.com> > > > > > > wrote: > > > > > >> Eating one's own dog foodhttp:// > > > en.wikipedia.org/wiki/Eating_one's_own_dog_food > > > > > >> or in this case: > > > > > >> Using one's own cloud. > > > > > > > >> Amazon' cloud is based upon the IT technology they use > > > > > >> within Amazon. > > > > > >> Salesforce.com's Force.com offering is what they used to > > > > > >> build their CRM system. > > > > > > > >> These cloud vendors "Eat their own dog food". > > > > > > > >> If a cloud vendor does not use their cloud offering for > > > > > >> their other products and/or internal systems, one > > > > > >> would have to assume that the cloud is viewed as > > > > > >> a technology ghetto within their own corporation - good > > > > > >> enough for others but not for ourselves. > > > > > > > >> So, concerning the Google App Engine, are other groups > > > > > >> w
[google-appengine] Re: Datastore Utility
I don't think there is any schema in the datastore. Your db tables are defined as classes in Python or Java. Once you store data, you can see it in the data-viewer. But if, for example, you delete all instances of a kind (a table?) then the structure for that table is no longer shown in the data viewer. Also, you might define 10 attributes or columns for a data table, but you might only store 5 of them. DataViewer will only show the 5 columns stored. A person could presumably write a program to interrogate the datastore, and show a list of all kinds (tables) and the names of all columns current stored. There might be a way to show the indexes - but I haven't explored that at all. Neal Walters --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Eating one's own dog food
Hi Juguang Your words provide a nice counter-balance to those who are not so happy with App Engine. Yes, you are correct in reminding us all that we should be grateful for something free. And yes App Engine has likely provided a whole bunch of young, inquiring minds with an amazing opportunity to tinker and learn on a global scale. But for some strange reason, I'm also reminded of that old saying...you get what you pay for. On Jul 16, 7:26 pm, Juguang XIAO wrote: > Dogfooding, as wikipedia names it, is ideological, may not be practical. > People, as well as company, lives practically for surviving first then > chasing dreams. > > I have seen the positive movement from Google, did so much for developers. > People may take it for granted, thinking the leader should do more. I am > content with GAE, as it offers some free and exciting stuffs. I cannot ask > Google to give more, unless he decided so. People can be happy when they are > grateful. > > Technically speaking, I do not think Google offers its best to developers. > It will be too costy to do so. Their mainstream businesses need to be > maintained, and I guess each business unit has its own authority and freedom > to do thing in their own way. Core businesses and technologies need to be > protected. If you are not happy, go for Microsoft. ;-) > > I do not believe, people can run their serious business without serious pay. > 6.5 hour per day CUP time is enough for casual applications. If you need > more, try to create multiple account, schedule roster among them, sync your > data among them. This is the solutions. If terms and conditions of use of > GAE allows, there will be open souce projects for GAE clustering. > > Juguang > > > > On Fri, Jul 17, 2009 at 10:03 AM, GenghisOne wrote: > > > So it looks like there's an updated Google App Engine roadmap and > > guess what...no mention of full-text search. > > > Doesn't that strike anyone as a bit odd? How can an emerging cloud > > computing platform not effectively address full-text search? And > > what's really odd is the absolute silence from Google...quite frankly, > > I don't get it. > > > On Jul 16, 12:28 pm, Bryan wrote: > > > This is a very interesting discussion. I would like to see some input > > > from Google. > > > > On Jul 15, 10:20 am, richard emberson > > > wrote: > > > > > I understand that BigTable is behind GAE, but my concern is > > > > more with GAE performance and quotas. If GAE had existed > > > > when Larry and Sergey were developing their pagerack > > > > algorithm, would they have used GEA for evaluation? > > > > I have my doubts. They would quickly reach quota limits, > > > > way before they knew if they had a viable idea. > > > > > Richard > > > > > Tony wrote: > > > > > Though I realize this is not exactly what you're asking, the concept > > > > > of GAE is that it exposes some of the infrastructure that all Google > > > > > applications rely on (i.e. Datastore) for others to use. So, in a > > > > > sense, Google's various applications were using App Engine before App > > > > > Engine existed. As far as I know, every Google service runs on the > > > > > same homogeneous infrastructure, which is part of what makes it so > > > > > reliable (and why the only available languages are Python and Java, > > > > > languages used internally at Google). > > > > > > But I don't work there, so maybe I'm completely off-base. > > > > > > On Jul 15, 12:53 pm, richard emberson > > > > > wrote: > > > > >> Eating one's own dog foodhttp:// > > en.wikipedia.org/wiki/Eating_one's_own_dog_food > > > > >> or in this case: > > > > >> Using one's own cloud. > > > > > >> Amazon' cloud is based upon the IT technology they use > > > > >> within Amazon. > > > > >> Salesforce.com's Force.com offering is what they used to > > > > >> build their CRM system. > > > > > >> These cloud vendors "Eat their own dog food". > > > > > >> If a cloud vendor does not use their cloud offering for > > > > >> their other products and/or internal systems, one > > > > >> would have to assume that the cloud is viewed as > > > > >> a technology ghetto within their own corporation - good > > > > >> enough for others but not for ourselves. > > > > > >> So, concerning the Google App Engine, are other groups > > > > >> within Google clamoring to port or build their offerings > > > > >> on top of the App Engine? If so, please be specific, what > > > > >> Google products and infrastructure and what are the schedules > > > > >> for their hosting on GAE? > > > > > >> Is the GAE group supporting the Google Docs group as they > > > > >> move to use GAE? How about gmail, will the Google Gmail > > > > >> group be relying on GAE support? I have not seen emails > > > > >> from either of those internal Google groups on the GAE > > > > >> mailing list. Lastly, when will Google search be supported > > > > >> by the GAE group; > > > > > >> Will those groups have to live under the same quota restrictions > > > > >> while they evaluate
[google-appengine] Re: Eating one's own dog food
Dogfooding, as wikipedia names it, is ideological, may not be practical. People, as well as company, lives practically for surviving first then chasing dreams. I have seen the positive movement from Google, did so much for developers. People may take it for granted, thinking the leader should do more. I am content with GAE, as it offers some free and exciting stuffs. I cannot ask Google to give more, unless he decided so. People can be happy when they are grateful. Technically speaking, I do not think Google offers its best to developers. It will be too costy to do so. Their mainstream businesses need to be maintained, and I guess each business unit has its own authority and freedom to do thing in their own way. Core businesses and technologies need to be protected. If you are not happy, go for Microsoft. ;-) I do not believe, people can run their serious business without serious pay. 6.5 hour per day CUP time is enough for casual applications. If you need more, try to create multiple account, schedule roster among them, sync your data among them. This is the solutions. If terms and conditions of use of GAE allows, there will be open souce projects for GAE clustering. Juguang On Fri, Jul 17, 2009 at 10:03 AM, GenghisOne wrote: > > So it looks like there's an updated Google App Engine roadmap and > guess what...no mention of full-text search. > > Doesn't that strike anyone as a bit odd? How can an emerging cloud > computing platform not effectively address full-text search? And > what's really odd is the absolute silence from Google...quite frankly, > I don't get it. > > > > On Jul 16, 12:28 pm, Bryan wrote: > > This is a very interesting discussion. I would like to see some input > > from Google. > > > > On Jul 15, 10:20 am, richard emberson > > wrote: > > > > > I understand that BigTable is behind GAE, but my concern is > > > more with GAE performance and quotas. If GAE had existed > > > when Larry and Sergey were developing their pagerack > > > algorithm, would they have used GEA for evaluation? > > > I have my doubts. They would quickly reach quota limits, > > > way before they knew if they had a viable idea. > > > > > Richard > > > > > Tony wrote: > > > > Though I realize this is not exactly what you're asking, the concept > > > > of GAE is that it exposes some of the infrastructure that all Google > > > > applications rely on (i.e. Datastore) for others to use. So, in a > > > > sense, Google's various applications were using App Engine before App > > > > Engine existed. As far as I know, every Google service runs on the > > > > same homogeneous infrastructure, which is part of what makes it so > > > > reliable (and why the only available languages are Python and Java, > > > > languages used internally at Google). > > > > > > But I don't work there, so maybe I'm completely off-base. > > > > > > On Jul 15, 12:53 pm, richard emberson > > > > wrote: > > > >> Eating one's own dog foodhttp:// > en.wikipedia.org/wiki/Eating_one's_own_dog_food > > > >> or in this case: > > > >> Using one's own cloud. > > > > > >> Amazon' cloud is based upon the IT technology they use > > > >> within Amazon. > > > >> Salesforce.com's Force.com offering is what they used to > > > >> build their CRM system. > > > > > >> These cloud vendors "Eat their own dog food". > > > > > >> If a cloud vendor does not use their cloud offering for > > > >> their other products and/or internal systems, one > > > >> would have to assume that the cloud is viewed as > > > >> a technology ghetto within their own corporation - good > > > >> enough for others but not for ourselves. > > > > > >> So, concerning the Google App Engine, are other groups > > > >> within Google clamoring to port or build their offerings > > > >> on top of the App Engine? If so, please be specific, what > > > >> Google products and infrastructure and what are the schedules > > > >> for their hosting on GAE? > > > > > >> Is the GAE group supporting the Google Docs group as they > > > >> move to use GAE? How about gmail, will the Google Gmail > > > >> group be relying on GAE support? I have not seen emails > > > >> from either of those internal Google groups on the GAE > > > >> mailing list. Lastly, when will Google search be supported > > > >> by the GAE group; > > > > > >> Will those groups have to live under the same quota restrictions > > > >> while they evaluate using GAE? If not, why not? If they > > > >> are unreasonable for an internal evaluation, what makes them > > > >> reasonable for an external evaluation? > > > > > >> Evaluating whether or not GAE should be used for a particular > > > >> application is not FREE even if one gets a very small slice > > > >> of GAE resources with which to do the evaluation. > > > >> Tens or hundreds of hours go into determine if GAE has > > > >> the right characteristics and quotas that limit how fast one > > > >> can work makes it worse. (Yes one can $$ for higher quotas, > > > >> but during the evaluation phase $$ is
[google-appengine] Re: GAE got it right, but Wave?
Hi Andy I think it's a little premature to suggest that Google got it perfectly right with App Engine. I think they got a few things right, but there's still some things they need to improve -- reliability, security, auditability, and searchability to name a few. Also, I sincerely hope that Google's other projects, especially the big audacious ones -- Docs, Wave and most recently the OS project -- don't starve App Engine of the resources it needs. On Jul 15, 11:10 pm, Andrew Badera wrote: > GAE went live to cheers from happy developers who could quickly tinker away > at the new technology. Wave has certainly been hyped, but all the early > enthusiasts seem to be angry/frustrated/resigned at this point due to lack > of developer accounts and/or communication from the Wave team. > > How did GAE get it so right and Wave get it so wrong? > > Thanks- > - Andy Badera > - and...@badera.us > - Google me:http://www.google.com/search?q=andrew+badera > - This email is: [ ] bloggable [x] ask first [ ] private --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Application Updates but Does Not Serve New Files?
Nick and I emailed privately, but for anyone else who is interested, CSS files were not being updated. This occurred in both FF and IE with flushed browser caches. I suppose it's possible these were being cached somewhere in the Internet in between me and Google, but it eventually resolved itself whatever the cause. On Jul 15, 1:32 am, "Nick Johnson (Google)" wrote: > Hi Devel63, > > We need to know what files you're serving that appear to be old, and > what they should be serving. My money is on a client-side caching > issue, but it's impossible to tell without more details. > > -Nick Johnson > > On Tue, Jul 14, 2009 at 11:29 PM, Devel63 wrote: > > > Today I am suddenly unable to effectively update my app. > > > appcfg.py appears to update the app, the serving version is set > > correctly, but the old files are being served. > > > The app name is judysapps-qa > > > Any ideas? > > -- > Nick Johnson, App Engine Developer Programs Engineer > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration > Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Eating one's own dog food
So it looks like there's an updated Google App Engine roadmap and guess what...no mention of full-text search. Doesn't that strike anyone as a bit odd? How can an emerging cloud computing platform not effectively address full-text search? And what's really odd is the absolute silence from Google...quite frankly, I don't get it. On Jul 16, 12:28 pm, Bryan wrote: > This is a very interesting discussion. I would like to see some input > from Google. > > On Jul 15, 10:20 am, richard emberson > wrote: > > > I understand that BigTable is behind GAE, but my concern is > > more with GAE performance and quotas. If GAE had existed > > when Larry and Sergey were developing their pagerack > > algorithm, would they have used GEA for evaluation? > > I have my doubts. They would quickly reach quota limits, > > way before they knew if they had a viable idea. > > > Richard > > > Tony wrote: > > > Though I realize this is not exactly what you're asking, the concept > > > of GAE is that it exposes some of the infrastructure that all Google > > > applications rely on (i.e. Datastore) for others to use. So, in a > > > sense, Google's various applications were using App Engine before App > > > Engine existed. As far as I know, every Google service runs on the > > > same homogeneous infrastructure, which is part of what makes it so > > > reliable (and why the only available languages are Python and Java, > > > languages used internally at Google). > > > > But I don't work there, so maybe I'm completely off-base. > > > > On Jul 15, 12:53 pm, richard emberson > > > wrote: > > >> Eating one's own dog > > >> foodhttp://en.wikipedia.org/wiki/Eating_one's_own_dog_food > > >> or in this case: > > >> Using one's own cloud. > > > >> Amazon' cloud is based upon the IT technology they use > > >> within Amazon. > > >> Salesforce.com's Force.com offering is what they used to > > >> build their CRM system. > > > >> These cloud vendors "Eat their own dog food". > > > >> If a cloud vendor does not use their cloud offering for > > >> their other products and/or internal systems, one > > >> would have to assume that the cloud is viewed as > > >> a technology ghetto within their own corporation - good > > >> enough for others but not for ourselves. > > > >> So, concerning the Google App Engine, are other groups > > >> within Google clamoring to port or build their offerings > > >> on top of the App Engine? If so, please be specific, what > > >> Google products and infrastructure and what are the schedules > > >> for their hosting on GAE? > > > >> Is the GAE group supporting the Google Docs group as they > > >> move to use GAE? How about gmail, will the Google Gmail > > >> group be relying on GAE support? I have not seen emails > > >> from either of those internal Google groups on the GAE > > >> mailing list. Lastly, when will Google search be supported > > >> by the GAE group; > > > >> Will those groups have to live under the same quota restrictions > > >> while they evaluate using GAE? If not, why not? If they > > >> are unreasonable for an internal evaluation, what makes them > > >> reasonable for an external evaluation? > > > >> Evaluating whether or not GAE should be used for a particular > > >> application is not FREE even if one gets a very small slice > > >> of GAE resources with which to do the evaluation. > > >> Tens or hundreds of hours go into determine if GAE has > > >> the right characteristics and quotas that limit how fast one > > >> can work makes it worse. (Yes one can $$ for higher quotas, > > >> but during the evaluation phase $$ is out of the question.) > > > >> Richard Emberson > > > >> -- > > >> Quis custodiet ipsos custodes > > > -- > > Quis custodiet ipsos custodes --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Quota changes keep on getting canceled by Google checkout
Hi Dominic, I notice that you filled out the billing issues form so if it OK with you we will continue to discuss over email. Thank you, Jeff On Wed, Jul 15, 2009 at 12:25 PM, dplouffe wrote: > > Hi all, > > I want to increase my quotas and do so from the Billing Settings > section of the admin interface. I go through the Google Checkout > process without any issues. When I check my inbox I get two emails, > one confirming my new order and another one stating that my order has > been canceled. There doesn't seem to be any reasons in the email or > in my Billing History as per why that is. > > Anyone can help me find out why my order keeps on getting canceled? > > Cheers, > > Dominic > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Application Main Logs not working
Hi Richard, Could you tell us your app ID and one of the request which resulted in a 500 error? I'd be happy to look into this but there are quite a few things which could cause a 500 error, including typos in source code and such. Thank you, Jeff On Wed, Jul 15, 2009 at 11:38 AM, richard emberson < richard.ember...@gmail.com> wrote: > > > When I send in a request that ought to log something > at the INFO level, nothing shows up in the Log viewer. > > When I make some requests I get a 500 response code, > internal server error, but nothing shows up in the > Log viewer. So, I can not tell what the problem is. > > Richard > -- > Quis custodiet ipsos custodes > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Globally monotonic counter
Thanks for the advice, Nick. I'd still like to know more about the consistency model though. For example, I wonder if there's any guarantee that two transactions on different entity groups executed by one process in a given order will be observed in the same order. I suspect the answer is no. I'm starting to think that the "GAE takes care of the messy details of distributed systems programming" claim is a bit overstated... Cheers, -n8 On Jul 14, 2:27 am, "Nick Johnson (Google)" wrote: > Hi Nathan, > > Your best options are either to keep track of one event stream per > game, or to use system time, and 'rewind' the timestamp a bit to > capture any missed events, as you suggest. Global monotonic counters > aren't very practical in large distributed systems. > > -Nick Johnson --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: CPU time for trivial datastore roundtrip
On Jul 16, 12:45 pm, Alexander Trakhimenok wrote: > This is expected behavior and nature of GAE data store. > > This is the beauty that the time will consistent and have direct > relations to number of items retrieved and not depend on number of > records stored. I understand that it is important to have a consistent response time, and maybe a charge of of api_cpu_ms = x * results + y could make sense, but as it currently is, it's crazy. It's claiming 13.2 ms /per result/. It's saying that the overhead of a request is roughly the same as fetching one extra result. I would have expected x to be lower and y to be higher. Even more than that, it's saying it costs Google's datastore 1000 ms to retrieve a 75-result indexed query. So I guess this means batching really doesn't matter. No point in trying to do one longer request when two shorter ones can be done. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Eating one's own dog food
This is a very interesting discussion. I would like to see some input from Google. On Jul 15, 10:20 am, richard emberson wrote: > I understand that BigTable is behind GAE, but my concern is > more with GAE performance and quotas. If GAE had existed > when Larry and Sergey were developing their pagerack > algorithm, would they have used GEA for evaluation? > I have my doubts. They would quickly reach quota limits, > way before they knew if they had a viable idea. > > Richard > > > > Tony wrote: > > Though I realize this is not exactly what you're asking, the concept > > of GAE is that it exposes some of the infrastructure that all Google > > applications rely on (i.e. Datastore) for others to use. So, in a > > sense, Google's various applications were using App Engine before App > > Engine existed. As far as I know, every Google service runs on the > > same homogeneous infrastructure, which is part of what makes it so > > reliable (and why the only available languages are Python and Java, > > languages used internally at Google). > > > But I don't work there, so maybe I'm completely off-base. > > > On Jul 15, 12:53 pm, richard emberson > > wrote: > >> Eating one's own dog > >> foodhttp://en.wikipedia.org/wiki/Eating_one's_own_dog_food > >> or in this case: > >> Using one's own cloud. > > >> Amazon' cloud is based upon the IT technology they use > >> within Amazon. > >> Salesforce.com's Force.com offering is what they used to > >> build their CRM system. > > >> These cloud vendors "Eat their own dog food". > > >> If a cloud vendor does not use their cloud offering for > >> their other products and/or internal systems, one > >> would have to assume that the cloud is viewed as > >> a technology ghetto within their own corporation - good > >> enough for others but not for ourselves. > > >> So, concerning the Google App Engine, are other groups > >> within Google clamoring to port or build their offerings > >> on top of the App Engine? If so, please be specific, what > >> Google products and infrastructure and what are the schedules > >> for their hosting on GAE? > > >> Is the GAE group supporting the Google Docs group as they > >> move to use GAE? How about gmail, will the Google Gmail > >> group be relying on GAE support? I have not seen emails > >> from either of those internal Google groups on the GAE > >> mailing list. Lastly, when will Google search be supported > >> by the GAE group; > > >> Will those groups have to live under the same quota restrictions > >> while they evaluate using GAE? If not, why not? If they > >> are unreasonable for an internal evaluation, what makes them > >> reasonable for an external evaluation? > > >> Evaluating whether or not GAE should be used for a particular > >> application is not FREE even if one gets a very small slice > >> of GAE resources with which to do the evaluation. > >> Tens or hundreds of hours go into determine if GAE has > >> the right characteristics and quotas that limit how fast one > >> can work makes it worse. (Yes one can $$ for higher quotas, > >> but during the evaluation phase $$ is out of the question.) > > >> Richard Emberson > > >> -- > >> Quis custodiet ipsos custodes > > -- > Quis custodiet ipsos custodes --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Optimal tree structure with datastore
Hi Nick, i just found my model will not work efficiently. The problem is not not position update, there will be max few tens of nodes in subtree, so i just have to update them, and even if enough big distance between positions will be chosen, lets say 1, i can just do simple checks between two nodes, (e.g. average value from positions which are around) and change the position in my node, this will mean few selects with 2 entries, and one update. Eg: B, C, D are sub nodes of A pB (position of B) = 1 000 000, because it was the first node created under B pC = 1 010 000 pD = 1 020 000 nov - reoder to get D, B, C will with some few queries and checks result with: pB = 1 000 000 (unchanged) pC = 1 010 000 (unchanged) pA = (1 000 000 - 0) / 2 = 500 000 (position updated) [0 is there because A is the first node] This will give me a possibility for quick changed in position. Of course there may be a conflict if i would like to `move` node between two nodes (Left, Right) and when pR - pL = 0 (difference between two possitions). This may lead to update of position of multiple nodes, but probably only one 2 nodes will be updated. But for this, there may be cron checking difference between positions, and... But i'm not expecting hundreds of positional moves anyway... The problem is somewhere else - lies in parents[] property. If there will be a move on lets say second level, and moved node have 5 000 descendants, parents property of all descendants must be updated, this will probably not be fast enough, or on other words, it will be very expensive. Hmm, i need something better here... But, anyway thanks for your help! Peter. Of course, there might be a conflict when the On Jul 15, 3:17 pm, "Nick Johnson (Google)" wrote: > On Tue, Jul 14, 2009 at 8:35 PM, Peter Cicman wrote: > > > Hi Nick, > > > no, level and position are not identical. > > > The idea behind it is to use level for node to know how deep in tree > > it exists, so root nodes have level=0, node with one parent have > > level=1, node with three parents (A-B-C-D) have level=3, etc.. This > > should allow simple move in path.. WHERE level ><... > > > Position will be used just for sorting. It will start in every subtree > > with zero value, and will be incremented when new node is added, so it > > will be possible to reorder node position in subtree, something like > > this should be possible: > > Ah. The distinction wasn't clear initially. Given that 'position' in > your example appears to apply to all nodes of a given depth, how will > you efficiently calculate and update this? Inserting a new leftmost > leaf node would require renumbering every other node's position. > > > > > > > > > BEFORE: > > A(0) level = 0 > > / | \ > > B(0) C(1) D(2) level = 1 > > / \ > > E(0) F(1) level = 2 > > > (Note: number in brackets represents position) > > > AFTER (node B, C reorder): > > > A(0) level = 0 > > / | \ > > C(0) B(1) D(2) level = 1 > > / \ > > E(0) F(1) level = 2 > > > What do you think about it? There will be probably maximum few > > hundreds of levels. Does it fits for datastore? > > Yes, though a 'few hundred' is getting to the point where the > ListProperty of ancestors will take a significant amount of time to > update. > > > > > Do i have to create index on parents property? If yes, size of this > > index will probably grow nonlinear with growing amount of levels. And > > i have to put also position, level and yet another two or three > > properties under index. What is the maximum index size? I did found > > just one information about it in documentation - `its huge`, but what > > does huge exactly mean? > > You only have to index the properties if you want to execute queries > against them that use more than one property and involve an inequality > or a sort order. See the indexing documentation for more details. The > indexing is limited by index entries per entity, not size - and the > limit is in the low thousands. > > > > > My hopefully last question :) > > > Is there somewhere some paper about deeper explanation of datastore? > > It would be nice to know how it exactly works in deeper level, how are > > the query counters working (i had some problem there also), etc... > > Reading the Bigtable paper would be a good start. > > -Nick Johnson > > > > > > > > > Thanks a lot! > > Peter. > > > On Jul 14, 11:47 am, "Nick Johnson (Google)" > > wrote: > >> Hi Peter, > > >> On Mon, Jul 13, 2009 at 6:56 PM, Peter Cicman wrote: > > >> > Hi Nick, hi Vince, i did come up with something like this yesterday: > > >> > class Tree(db.Expando): > >> > parents = ListProperty() > >> > level = IntegerProperty() > >> > position = IntegerProperty() > > >> > properties: > >> > parents: sorted list of all parent keys, sorting order top/down, > >> > so parents[-1] gives object direct parent > >> > level: nested level, for ro
[google-appengine] Re: CPU time for trivial datastore roundtrip
Hi Alexander, I have a code which updates the table with 7 fields, one of them being an image, on a certain condition, which would take place once in two days, and then retrieves around 25 records from that table and displays them in an html page which uses Dajngo script. All together, I have one conditional put and one retrieve statement. This is taking me around 5000 cpu_ms. Could you please help me reduce this. Thanks in advance, Shanmukh :) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: CPU time for trivial datastore roundtrip
This is expected behavior and nature of GAE data store. This is the beauty that the time will consistent and have direct relations to number of items retrieved and not depend on number of records stored. -- Alex On Jul 15, 4:14 am, Anthony Mills wrote: > On Jul 14, 6:48 am, Anthony Mills wrote: > > > I'll do some testing later tonight about what happens when > > REQUEST_SIZE is 1, 5, 10, 20, etc., and report back. > > Ok, here are a selection of the results of my tests. "rs" controls the > number of results asked for. > > /service?a=g&t=m&rs=1 200 32ms 39cpu_ms 31api_cpu_ms > /service?a=g&t=m&rs=2 200 44ms 50cpu_ms 44api_cpu_ms > /service?a=g&t=m&rs=3 200 56ms 64cpu_ms 57api_cpu_ms > /service?a=g&t=m&rs=4 200 101ms 82cpu_ms 70api_cpu_ms > /service?a=g&t=m&rs=5 200 53ms 90cpu_ms 84api_cpu_ms > /service?a=g&t=m&rs=6 200 50ms 109cpu_ms 97api_cpu_ms > /service?a=g&t=m&rs=7 200 41ms 117cpu_ms 110api_cpu_ms > /service?a=g&t=m&rs=8 200 57ms 136cpu_ms 123api_cpu_ms > /service?a=g&t=m&rs=9 200 43ms 144cpu_ms 136api_cpu_ms > /service?a=g&t=m&rs=10 200 62ms 174cpu_ms 150api_cpu_ms > /service?a=g&t=m&rs=20 200 47ms 306cpu_ms 282api_cpu_ms > /service?a=g&t=m&rs=50 200 76ms 705cpu_ms 678api_cpu_ms > /service?a=g&t=m&rs=70 200 86ms 974cpu_ms 942api_cpu_ms > > Interesting. The variable api_cpu_ms climbs very ... predictably. In > fact, it would seem: > > api_cpu_ms = 13.2 * results + 17.5 > > ... rounded off to the nearest millisecond. At least for queries. I > haven't tried straight key fetches. > > This is really disturbing. > > // Anthony --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Doing Appengine settings programmatically
Have a look at how App Engine Launcher works http://googleappengine.blogspot.com/2008/05/app-engine-launcher-for-mac-os-x.html but pretty sure it just wraps appcfg.py You could just wrap appcfg.cmd On 16/07/2009, vijay wrote: > > Ping!! > > On Wed, Jul 15, 2009 at 7:48 PM, vijay wrote: > > HI All, > > I am writing an application in Java which will create a website for my > users using some custom fields. I want that the website built and this > application will be deployed on App Engine. Assuming that My user trust my > application and I can have their username and password enterd to my > applications setting page, how to do all Appengine settings > programmatically. Are their any API available for this. > > The reason I want to do it this way is that the user can directly handle > their app engine interaction and can pay for usage over freebie quota. > > > > > > Thanks > > Vijay > > > > > -- Barry - www.nearby.org.uk - www.geograph.org.uk - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Doing Appengine settings programmatically
Ping!! On Wed, Jul 15, 2009 at 7:48 PM, vijay wrote: > HI All,I am writing an application in Java which will create a website for > my users using some custom fields. I want that the website built and this > application will be deployed on App Engine. Assuming that My user trust my > application and I can have their username and password enterd to my > applications setting page, how to do all Appengine settings > programmatically. Are their any API available for this. > The reason I want to do it this way is that the user can directly handle > their app engine interaction and can pay for usage over freebie quota. > > Thanks > Vijay > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Looking for solution for bigtable problem (which can be solved in relational db easily)
Yes, I was also at the stage of creating a counter for each combination. But it will indeed be a challenge. As users will be able to vote for favourite and disliked books. Users can also revoke their votes which could mean a lot of counters to update. So imagine I have 10 favourite books and 10 disliked books. So there will at least a counter of 1 for each combination of favourite and disliked books. When I remove on the favourite books from my list, I have to update all the combinations. It's not only a challenge to implement it, but also to keep it consistent in case of failure. The more the challenge it will be to implement the more likely there will be a bug... In case of a relational database I would compromise at real-time accurately I would just run a relational query once a hour and store the results for all combinations in cache. With a relational query I could join the tables easily. In this way I would also not care about data inconsistency as it could not happen. I'm still open for any other ideas and it's interesting to discuss about bigtable vs more tradiitional relational databases. Maybe there is a moment in the future I will see the light of bigtable, but not at the moment to be honestly. Thanks again, Marcel On Jul 16, 7:05 pm, Dan wrote: > What you describe sounds like a dynamic grouping, which is challenging > for any datastore. By challenging, I mean it's difficult to deliver > quickly and accurately. If you're willing to deal with some hassle and > compromise a bit on (at least one of) those two objectives, then it > should be possible to get something working. > > Here's a suggestion: > store a list of books purchased in each user's "row" as a > listproperty. then when you want to show a list of books that were > purchased by customers that bought a specific book, you could use the > = operator to get a list of those listproperties. This approach has > some challenges, like the 1000-row limit, dealing with duplicate > references to books. And indices that use list properties can get very > large, so you'd need to use caution. > > another approach is to create a separate table to contain the co- > purchase relationship. It would contain 1 row for each relationship. > So if customer A bought books 1,2,3 and customer B bought books 2,3,4 > then the table of relationships would have these rows: > first_book second_book > 1 2 > 2 3 > 1 3 > 2 4 > (I might be mistaken, but I think this is similar to how the datastore > indexes list properties, by specifying all permutations.) > To make things easier to look up, you could add a field to indicate > how many times the co-occurrence happened, like this: > first_book second_book count > 1 2 1 > 2 3 2 <-- 2 customers have purchased both 2 and 3 > 1 3 1 > 2 4 1 > then when you wanted to know what books have been co-purchased, you > would run 2 queries on this table, looking for relationships which > match on the first_book and then on the second_book. You would then > have a list of books, and would need to check to see which ones got > the most votes. > > In either case, you're going to have to deal with the challenge of a > large number of intermediate results that need to be ranked/filtered. > You could try to avoid this problem by adding the vote results to the > relationship table and putting it into an index, like this: > first_book second_book count first_book_vote_score > second_book_vote_score > ... but then you would need to keep that score information up-to-date, > which can be a challenge, too. For a book voting situation, you could > probably relax the accuracy and timeliness goals and do a batch update > of the vote results in the relationship table, perhaps using a > "updated_timestamp" field in the users' purchases and the books' votes > tables. > > hth > Dan > > On Jul 16, 1:54 am, Marcel Overdijk wrote: > > > > > I have a very simple model. Users and Books. > > Every user can pick a list of favourite and disliked books. > > > Now I like to present an public page with most favourite books. > > Thi is no problem, I could create a counter (perhaps a sharded > > counter). > > > But when I click a specific book I want to display it details + (and > > now comes the difficult part) a list of books which Users also voted > > for who vote for the particular book displayed. > > > Similar to Amazon's "People who bought this book, also bought..." > > > With a relational db query this would be solved very easily, but I > > don't see an effective way to this with big table. > > > Any ideas? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~---
[google-appengine] Re: More Indexing Woes
Hi Jeff, Thank you very much for your prompt response! All is looking good! Cheers! On Jul 16, 11:24 am, "Jeff S (Google)" wrote: > Hi Blair, > > I've reset thisquotafor you. > > Happy coding, > > Jeff > > On Jul 16, 8:09 am, Blair Kelsie wrote: > > > Hi Jeff, > > > We also have the "Data Indices Count" warning message. > > > Could you please reset ourindexquota? > > > application: steprep > > > Thank you, > > > Blair > > VendAsta Technologies Inc. > > > On Jul 14, 11:48 am, "Jeff S (Google)" wrote: > > > > Hi Scott, > > > > Apologies for not getting back to you earlier. We were aware of theindex > > > issues with your application and yourindexquotashould now be reset. Could > > > you give vacuuming your indexes another try? Please allow a couple of > > > hours > > > for the indexes to be removed and let me know if you there are still > > > issues. > > > > Thank you, > > > > Jeff > > > > On Mon, Jul 13, 2009 at 3:21 PM, Scott wrote: > > > > > Hi, > > > > > I posted about a week ago, but some of the issues were never resolved, > > > > and there doesn't seem to be any response on that thread, so I'll try > > > > a new one. > > > > > My application has indexes that are in a constant state of error. They > > > > don't respond to vacuuming. I'd like to have them deleted before > > > > attempting to build more indexes which I need for the features I'm > > > > working on next in my application. So time is of the essence. > > > > > Also, I just got a 'Your application is exceeding aquota: Datastore > > > > Indices Count' message on my dashboard, but I have only about a couple > > > > dozen indexes, including the ones that are in an error state. > > > > > Thanks > > > > > -Scott > > > > application: skrit --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: More Indexing Woes
Hi Blair, I've reset this quota for you. Happy coding, Jeff On Jul 16, 8:09 am, Blair Kelsie wrote: > Hi Jeff, > > We also have the "Data Indices Count" warning message. > > Could you please reset our index quota? > > application: steprep > > Thank you, > > Blair > VendAsta Technologies Inc. > > On Jul 14, 11:48 am, "Jeff S (Google)" wrote: > > > Hi Scott, > > > Apologies for not getting back to you earlier. We were aware of the index > > issues with your application and your index quota should now be reset. Could > > you give vacuuming your indexes another try? Please allow a couple of hours > > for the indexes to be removed and let me know if you there are still issues. > > > Thank you, > > > Jeff > > > On Mon, Jul 13, 2009 at 3:21 PM, Scott wrote: > > > > Hi, > > > > I posted about a week ago, but some of the issues were never resolved, > > > and there doesn't seem to be any response on that thread, so I'll try > > > a new one. > > > > My application has indexes that are in a constant state of error. They > > > don't respond to vacuuming. I'd like to have them deleted before > > > attempting to build more indexes which I need for the features I'm > > > working on next in my application. So time is of the essence. > > > > Also, I just got a 'Your application is exceeding a quota: Datastore > > > Indices Count' message on my dashboard, but I have only about a couple > > > dozen indexes, including the ones that are in an error state. > > > > Thanks > > > > -Scott > > > application: skrit --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Your application is exceeding a quota: Datastore Indices Count
Hi Roman, I've reset this quota so you should be all set. Apologies for the inconvenience. Happy coding, Jeff On Jul 16, 9:48 am, Roman wrote: > Hi, I got the same issue and it is blocking the application. Please > help. > > App id is roman-baglay > > Thank you! > > On Jun 30, 10:59 pm, Alex Epshteyn > wrote: > > > I should add that thisquotamessage doesn't seem to be impacting > > theapplicationin any way (i.e. there are no actualquotadenial errors). > > > On Jul 1, 12:49 am, Alex Epshteyn > > wrote: > > > > I just started seeing this message as well today after I enabled > > > billing on my app (id = typeracerdata) > > > > On Jun 29, 8:20 pm, "Jeff S (Google)" wrote: > > > > > Thisquotahas been reset, apologies for the inconvenience. > > > > > Happy coding, > > > > > Jeff > > > > > On Mon, Jun 29, 2009 at 1:35 PM, gae123 wrote: > > > > > > I see this with my app with appid: neatschool after I uploaded a new > > > > > version of the app that I have not made the current version yet. Could > > > > > you please reset thecount? > > > > > > Thanks- Hide quoted text - > > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Your application is exceeding a quota: Datastore Indices Count
I got a problem "Your application is exceeding a quota: Datastore Indices Count" Application id is roman-baglay Please help! Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Mapping www subdomain results in 'The requested URL / was not found on this server.'
I've followed the instructions to get my domain name, purchased on GoDaddy, to point to my app at http://fsmainsite.appspot.com/. The problem is, when I log into Google Apps, and go to my settings for this application, click on the link to http://www.mydomain.com, it opens up a new page which displays the URL http://www.fsmainsite.appspot.com/ in the address bar which is incorrect. The app is hosted at http://fsmainsite.appspot.com. Any help is appreciated. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: More Indexing Woes
Hi Jeff, We also have the "Data Indices Count" warning message. Could you please reset our index quota? application: steprep Thank you, Blair VendAsta Technologies Inc. On Jul 14, 11:48 am, "Jeff S (Google)" wrote: > Hi Scott, > > Apologies for not getting back to you earlier. We were aware of the index > issues with your application and your index quota should now be reset. Could > you give vacuuming your indexes another try? Please allow a couple of hours > for the indexes to be removed and let me know if you there are still issues. > > Thank you, > > Jeff > > On Mon, Jul 13, 2009 at 3:21 PM, Scott wrote: > > > Hi, > > > I posted about a week ago, but some of the issues were never resolved, > > and there doesn't seem to be any response on that thread, so I'll try > > a new one. > > > My application has indexes that are in a constant state of error. They > > don't respond to vacuuming. I'd like to have them deleted before > > attempting to build more indexes which I need for the features I'm > > working on next in my application. So time is of the essence. > > > Also, I just got a 'Your application is exceeding a quota: Datastore > > Indices Count' message on my dashboard, but I have only about a couple > > dozen indexes, including the ones that are in an error state. > > > Thanks > > > -Scott > > application: skrit --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Your application is exceeding a quota: Datastore Indices Count
Hi, I got the same issue and it is blocking the application. Please help. App id is roman-baglay Thank you! On Jun 30, 10:59 pm, Alex Epshteyn wrote: > I should add that thisquotamessage doesn't seem to be impacting > theapplicationin any way (i.e. there are no actualquotadenial errors). > > On Jul 1, 12:49 am, Alex Epshteyn > wrote: > > > > > I just started seeing this message as well today after I enabled > > billing on my app (id = typeracerdata) > > > On Jun 29, 8:20 pm, "Jeff S (Google)" wrote: > > > > Thisquotahas been reset, apologies for the inconvenience. > > > > Happy coding, > > > > Jeff > > > > On Mon, Jun 29, 2009 at 1:35 PM, gae123 wrote: > > > > > I see this with my app with appid: neatschool after I uploaded a new > > > > version of the app that I have not made the current version yet. Could > > > > you please reset thecount? > > > > > Thanks- Hide quoted text - > > - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: CANNOT receive SNS :(
I am also in China. I can get GAE activation SMS. You just need to wait for more hours. 2009/7/16 Nick Johnson (Google) > > Hi 陈永仁, > > I've activated your account. > > -Nick Johnson > > On Thu, Jul 16, 2009 at 11:53 AM, 陈永仁 wrote: > > > > I want to create a Google App engine Account, but I cannot receive > > SNS in my country-china mainland. Some of my friends said that the > > china mobile company does not support Google App Engine any more. > > > > so I have to ask Google App Engine Team for help > > > > Is there any other way of verifying my account? > > > > Thank you > > > > Best Regard > > > > John Anderson > > > > > > > > > > > -- > Nick Johnson, App Engine Developer Programs Engineer > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration > Number: 368047 > > > > -- = Juguang XIAO Beijing, China --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Looking for solution for bigtable problem (which can be solved in relational db easily)
What you describe sounds like a dynamic grouping, which is challenging for any datastore. By challenging, I mean it's difficult to deliver quickly and accurately. If you're willing to deal with some hassle and compromise a bit on (at least one of) those two objectives, then it should be possible to get something working. Here's a suggestion: store a list of books purchased in each user's "row" as a listproperty. then when you want to show a list of books that were purchased by customers that bought a specific book, you could use the = operator to get a list of those listproperties. This approach has some challenges, like the 1000-row limit, dealing with duplicate references to books. And indices that use list properties can get very large, so you'd need to use caution. another approach is to create a separate table to contain the co- purchase relationship. It would contain 1 row for each relationship. So if customer A bought books 1,2,3 and customer B bought books 2,3,4 then the table of relationships would have these rows: first_book second_book 1 2 2 3 1 3 2 4 (I might be mistaken, but I think this is similar to how the datastore indexes list properties, by specifying all permutations.) To make things easier to look up, you could add a field to indicate how many times the co-occurrence happened, like this: first_book second_book count 1 2 1 2 3 2 <-- 2 customers have purchased both 2 and 3 1 3 1 2 4 1 then when you wanted to know what books have been co-purchased, you would run 2 queries on this table, looking for relationships which match on the first_book and then on the second_book. You would then have a list of books, and would need to check to see which ones got the most votes. In either case, you're going to have to deal with the challenge of a large number of intermediate results that need to be ranked/filtered. You could try to avoid this problem by adding the vote results to the relationship table and putting it into an index, like this: first_book second_book count first_book_vote_score second_book_vote_score ... but then you would need to keep that score information up-to-date, which can be a challenge, too. For a book voting situation, you could probably relax the accuracy and timeliness goals and do a batch update of the vote results in the relationship table, perhaps using a "updated_timestamp" field in the users' purchases and the books' votes tables. hth Dan On Jul 16, 1:54 am, Marcel Overdijk wrote: > I have a very simple model. Users and Books. > Every user can pick a list of favourite and disliked books. > > Now I like to present an public page with most favourite books. > Thi is no problem, I could create a counter (perhaps a sharded > counter). > > But when I click a specific book I want to display it details + (and > now comes the difficult part) a list of books which Users also voted > for who vote for the particular book displayed. > > Similar to Amazon's "People who bought this book, also bought..." > > With a relational db query this would be solved very easily, but I > don't see an effective way to this with big table. > > Any ideas? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: phpAdmin clone for Google's datastore
There is a tool called "Data Viewer" in the GAE Admin Console. Have you seen it? It displays data rows (10 at a time I think), allows you to scroll forward, allow you enter your own GQL command, and allows you to update and delete. The only trick is that it only works with data already stored. So for example, if you define two tables (classes inheriting from db.Model), in your code, and only store one of them, you will only see that one table (that actually has data stored) in the Data Viewer. If later, you delete all those rows, you will no longer see that table/patter in the viewer, until you store a new row in your code. Neal Walters --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: bulkloader raises "no module named appengine.base"
Hi Nick,
Thanks for the reply.
I managed to upload data by changing the base class of Region model
from BaseModel of Django helper to the native db.model and removing
BaseModel import from the models.py.
Is it not possible with Django Helper?
--
onur
On Jul 16, 5:05 pm, "Nick Johnson (Google)"
wrote:
> Hi omat,
>
> It looks like you're importing, probably indirectly, your Django app,
> which is attempting to set itself up inside the bulk loader. You need
> to make sure that when you import your models, you don't also import
> the Django parts of your app.
>
> -Nick Johnson
>
>
>
> On Thu, Jul 16, 2009 at 2:42 PM, omat wrote:
>
> > Hi,
>
> > I am using Django 1.1 with App Engine Helper for Django. My
> > application runs fine, but when I try to upload data using bulkloader,
> > "no module named appengine.base" is raised.
>
> > I issue the command in the root directory of the project (where the
> > app.yaml resides):
>
> > $ appcfg.py upload_data --config_file=profile/loaders.py --
> > filename=profile/initial_data/region.csv --kind=Region --url=http://
> > localhost:8082/remote_api ./
>
> > It seems to get going at the first 2 lines, as it responds:
>
> > Uploading data records.
> > [INFO ] Logging to bulkloader-log-20090716.162701
>
> > Then a "no module named appengine.base" is raised. The full traceback
> > is as follows:
>
> > Traceback (most recent call last):
> > File "/usr/local/bin/appcfg.py", line 60, in
> > run_file(__file__, globals())
> > File "/usr/local/bin/appcfg.py", line 57, in run_file
> > execfile(script_path, globals_)
> > File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> > GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> > google/appengine/tools/appcfg.py", line 2303, in
> > main(sys.argv)
> > File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> > GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> > google/appengine/tools/appcfg.py", line 2294, in main
> > result = AppCfgApp(argv).Run()
> > File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> > GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> > google/appengine/tools/appcfg.py", line 1458, in Run
> > self.action(self)
> > File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> > GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> > google/appengine/tools/appcfg.py", line 2182, in __call__
> > return method()
> > File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> > GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> > google/appengine/tools/appcfg.py", line 2071, in PerformUpload
> > run_fn(args)
> > File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> > GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> > google/appengine/tools/appcfg.py", line 1984, in RunBulkloader
> > sys.exit(bulkloader.Run(arg_dict))
> > File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> > GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> > google/appengine/tools/bulkloader.py", line 4354, in Run
> > return _PerformBulkload(arg_dict)
> > File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> > GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> > google/appengine/tools/bulkloader.py", line 4219, in _PerformBulkload
> > LoadConfig(config_file)
> > File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> > GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> > google/appengine/tools/bulkloader.py", line 3992, in LoadConfig
> > ('', 'r', imp.PY_SOURCE))
> > File "parliament/loaders.py", line 9, in
> > import models
> > File "/Users/omat/workspace/vekilim/gae/parliament/models.py", line
> > 1, in
> > from appengine_django.models import BaseModel
> > File "/Users/omat/workspace/projetakvimi/gae/appengine_django/
> > models.py", line 24, in
> > from django.db.models.fields import Field
> > File "/Users/omat/workspace/projetakvimi/gae/django/db/__init__.py",
> > line 34, in
> > (settings.DATABASE_ENGINE, ", ".join(map(repr,
> > available_backends)), e_user)
> > django.core.exceptions.ImproperlyConfigured: 'appengine' isn't an
> > available database backend. Available options are: 'dummy', 'mysql',
> > 'oracle', 'postgresql', 'postgresql_psycopg2', 'sqlite3'
> > Error was: No module named appengine.base
>
> > Any help would be appreciated greatly.
>
> > Thanks,
> > omat
>
> --
> Nick Johnson, App Engine Developer Programs Engineer
> Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
> Number: 368047
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email
[google-appengine] Re: How does the GAE infrastructure support this?
On Thu, Jul 16, 2009 at 4:27 PM, gpesti wrote: > > Hi Nick, > > Thanks for the fast reply. > > I am testing my understanding: > 1. There is no way to track bandwidth and storage. Not on a per-request basis, but you can certainly track your overall usage. > 2. I need to implement my own user mgmt (registration, ...) and then I > have to track my users' requests within my application through > get_request_cpu_usage(). You can use the Users API for user management if you're happy with requiring users to sign in with Google accounts. If you want to account for individual users' resource usage, then you need to track it per-user, yes. -Nick Johnson > I am daydreaming: if an "out of the box" user management was provided > by the GAE infrastructure (based on e.g. OpenID?) and if it was bound > to a bit more elaborate resource usage tracking service... This would > be an ideal platform for SaaS developers: per user reporting/billing/ > quotas. > > Anyway, thanks for the info: > Gabor > > On Jul 16, 4:13 pm, "Nick Johnson (Google)" > wrote: >> Hi gpesti, >> >> Currently the only way we have for an app to track its own resource >> usage is to use the quota API: >> >> from google.appengine.api import quota >> megacycles = quota.get_request_cpu_usage() >> >> Currently, this is the only available quota you can do this for, and >> the returned figure does not include CPU usage due to API calls. We >> intend to expand the available information in the near future. >> >> Depending on the nature of your computation intensive requests - eg, >> what sort of work they do, and if it can be batch processed offline, >> rather than as part of the response to a user's request - you may want >> to consider using the task queue API, or shelling out to another >> service such as Amazon's EC2 to do the processor-intensive work. >> >> -Nick Johnson >> >> >> >> >> >> On Thu, Jul 16, 2009 at 2:25 PM, gpesti wrote: >> >> > Hi All, >> >> > I have a traditional WEB application with the following features: >> > - the provided services are very computation-intensive >> > - the usage is subscription-based, you need to register and log in to >> > access the services >> > - there are free and paid versions of the services, the key >> > differentiating factor is that paid requests have higher priority, can >> > consume more resources, etc than the free ones. >> > - there is a flat monthly fee for the paid services. >> >> > Beyond the obvious (scaling…) limitations the other main reason this >> > application needs to be modified/recreated is the flat fee model. To >> > provide a real pay-as-you-go experience it would be good to track the >> > resource utilization on a per customer basis and bill accordingly. >> > Setting per user quotas would be a plus, too. >> >> > I understand the GAE pricing model is based on the resource >> > utilization of the application as a whole, but are there any out of >> > box platform services that an application could use for internal >> > resource usage management? >> >> > Thanks, >> > Gabor >> >> -- >> Nick Johnson, App Engine Developer Programs Engineer >> Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration >> Number: 368047 > > > -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: How does the GAE infrastructure support this?
Hi Nick, Thanks for the fast reply. I am testing my understanding: 1. There is no way to track bandwidth and storage. 2. I need to implement my own user mgmt (registration, ...) and then I have to track my users' requests within my application through get_request_cpu_usage(). I am daydreaming: if an "out of the box" user management was provided by the GAE infrastructure (based on e.g. OpenID?) and if it was bound to a bit more elaborate resource usage tracking service... This would be an ideal platform for SaaS developers: per user reporting/billing/ quotas. Anyway, thanks for the info: Gabor On Jul 16, 4:13 pm, "Nick Johnson (Google)" wrote: > Hi gpesti, > > Currently the only way we have for an app to track its own resource > usage is to use the quota API: > > from google.appengine.api import quota > megacycles = quota.get_request_cpu_usage() > > Currently, this is the only available quota you can do this for, and > the returned figure does not include CPU usage due to API calls. We > intend to expand the available information in the near future. > > Depending on the nature of your computation intensive requests - eg, > what sort of work they do, and if it can be batch processed offline, > rather than as part of the response to a user's request - you may want > to consider using the task queue API, or shelling out to another > service such as Amazon's EC2 to do the processor-intensive work. > > -Nick Johnson > > > > > > On Thu, Jul 16, 2009 at 2:25 PM, gpesti wrote: > > > Hi All, > > > I have a traditional WEB application with the following features: > > - the provided services are very computation-intensive > > - the usage is subscription-based, you need to register and log in to > > access the services > > - there are free and paid versions of the services, the key > > differentiating factor is that paid requests have higher priority, can > > consume more resources, etc than the free ones. > > - there is a flat monthly fee for the paid services. > > > Beyond the obvious (scaling…) limitations the other main reason this > > application needs to be modified/recreated is the flat fee model. To > > provide a real pay-as-you-go experience it would be good to track the > > resource utilization on a per customer basis and bill accordingly. > > Setting per user quotas would be a plus, too. > > > I understand the GAE pricing model is based on the resource > > utilization of the application as a whole, but are there any out of > > box platform services that an application could use for internal > > resource usage management? > > > Thanks, > > Gabor > > -- > Nick Johnson, App Engine Developer Programs Engineer > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration > Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Webservice authentication (Python)
Nick, I tried running the following code, having added
save_cookies=True
-
def passwdFunc():
return ('my_email','my_passwd')
rpcServer = appengine_rpc.HttpRpcServer
('myapp.appspot.com',passwdFunc,None,'myAppName',save_cookies=True)
blah = rpcServer.Send('/')
-
It still gives me a 302 error:
-
Server: myapp.appspot.com
Sending HTTP request:
POST /? HTTP/1.1
Host: myapp.appspot.com
X-appcfg-api-version: 1
Content-type: application/octet-stream
Got http error, this is try #1
Got 302 redirect. Location:
https://www.google.com/accounts/ServiceLogin?service=ah&continue=http://myapp.appspot.com/_ah/login%3Fcontinue%3Dhttp://myapp.appspot.com/ On Wed, Jul 15, 2009 at 9:55 PM, epb wrote:
>
> > On Jul 15, 4:05 pm, Tony wrote:
> >> The response has the "Set-cookie" header set, which will cause the
> >> user's browser to save the cookie and then present it on the next
> >> request (after redirected by the 302). In my code I've opted not to
> >> follow the redirect, and extracted the cookie myself, because it's the
> >> urlfetch service doing the request, not the user. You can then either
> >> return a response to the user with a "Set-cookie" HTTP header (causing
> >> their browser to save the cookie), or handle it some other way (return
> >> it in the body and set the cookie with Javascript, for example).
>
> >> I think I misunderstood your original question, though, and you're
> >> looking for something different. You want to get an authorization
> >> cookie and then use it to make repeated requests with urlfetch, not
> >> with a browser? If that's the case, you're going to want to capture
> >> the "Set-cookie" header from the second response, and supply that in
> >> future requests (setting the "Cookie" header for urlfetch).
>
> > Yes, that is exactly what I want :) My client app. is not browser-
> > based. I guess I'll just use your method then.. appengine_rpc must be
> > intended for browser apps only, as it does nothing to capture the
> > auth. cookie. I could of course extend the appengine_rpc module to
> > capture the cookie, but the module uses urllib2.OpenerDirector.open()
> > to open URLs and this is perhaps not the way to go in my case? I am
> > not sure what the difference is between urlfetch() and open() it
> > seems like I can get the headers (and hereby the cookie) by using info
> > () on the response from open().
>
> appengine_rpc is intended for any Python app. It captures the cookie
> by using a CookieJar, which does the capturing/sending of the cookie.
>
> -Nick Johnson
>
>
>
>
>
> >> Basically, urlfetch will follow redirects but it won't handle cookies
> >> automatically - so what's happening is it's ignoring the "Set-cookie"
> >> header and following the redirect, and being denied because it's not
> >> supplying a cookie.
>
> >> On Jul 15, 2:58 pm, epb wrote:
>
> >> > I can see why Tony's version would work. His "algorithm" has two
> >> > steps:
>
> >> > 1. Get the authorization token using ClientLogin (which I also managed
> >> > to do).
> >> > 2. Use the uri "servername/_ah/login" to get the auth. cookie.
>
> >> > The appengine_rpc module seems to do authentication in a similar way:
>
> >> > A. Try to access the app. This results in a redirect to a location
> >> > that starts withhttps://www.google.com/accounts/ServiceLogin
> >> > B. Get a auth. token (like step 1 above)
> >> > C. Use auth. token to get auth. cookie.
> >> > D. Try to access the app. again (this is where it fails in my case...)
>
> >> > Anyway, step C is performed using the function below:
>
> >> > ---
>
> >> > def _GetAuthCookie(self, auth_token):
> >> > """Fetches authentication cookies for an authentication token.
>
> >> > Args:
> >> > auth_token: The authentication token returned by ClientLogin.
>
> >> > Raises:
> >> > HTTPError: If there was an error fetching the authentication
> >> > cookies.
> >> > """
> >> > continue_location = "http://localhost/";
> >> > args = {"continue":
[google-appengine] Re: Can't manage the application
Thanks Nick. It works. But why is the reason I can't use normal way to log in? Just curious. Cheers On Thu, Jul 16, 2009 at 3:04 PM, Nick Johnson (Google) wrote: > > Hi Itcstyle, > > Are you logging in using the account you created the app with, or one > on the domain you just set up? If the former, is it an Apps domain? > You may need to log in at http://appengine.google.com/a/yourdomain . > > -Nick Johnson > > On Thu, Jul 16, 2009 at 2:54 PM, ltcstyle wrote: >> >> Hi, >> >> I have a problem with google app engine dash board. >> >> I created a google app and an admin email account. >> Then I register a google app engine use my admin email in the newly >> created domain. >> >> I can upload the code, and it all runs ok, but when I try to log in to >> manage the application, google app engine doesn't show me that app >> under my account. >> And I tried http://appengine.google.com/dashboard?&app_id=**, it >> doesn't help. >> >> Could anyone help? >> >> Cheers >> >> > >> > > > > -- > Nick Johnson, App Engine Developer Programs Engineer > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration > Number: 368047 > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] phpAdmin clone for Google's datastore
Most databases include tools for executing adhoc queries and inserting or updating data. Most of the time using decent GUI tools like phpAdmin or TOAD. Those tools aid developers and sys admins in their daily job. I'm really missing such a editor for GAE. I think it should be available in the GAE administrator console. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: No SMS support for my country
I've activated your account. -Nick Johnson On Thu, Jul 16, 2009 at 2:24 PM, Munkhbaatar wrote: > > Hello, I am from Mongolia, > > I have a problem with "Verify Your Account by SMS" page. No SMS > support for my country. > > Is there any other way of verifying my account? > > Thanks. > > > > -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: How does the GAE infrastructure support this?
Hi gpesti, Currently the only way we have for an app to track its own resource usage is to use the quota API: from google.appengine.api import quota megacycles = quota.get_request_cpu_usage() Currently, this is the only available quota you can do this for, and the returned figure does not include CPU usage due to API calls. We intend to expand the available information in the near future. Depending on the nature of your computation intensive requests - eg, what sort of work they do, and if it can be batch processed offline, rather than as part of the response to a user's request - you may want to consider using the task queue API, or shelling out to another service such as Amazon's EC2 to do the processor-intensive work. -Nick Johnson On Thu, Jul 16, 2009 at 2:25 PM, gpesti wrote: > > Hi All, > > I have a traditional WEB application with the following features: > - the provided services are very computation-intensive > - the usage is subscription-based, you need to register and log in to > access the services > - there are free and paid versions of the services, the key > differentiating factor is that paid requests have higher priority, can > consume more resources, etc than the free ones. > - there is a flat monthly fee for the paid services. > > Beyond the obvious (scaling…) limitations the other main reason this > application needs to be modified/recreated is the flat fee model. To > provide a real pay-as-you-go experience it would be good to track the > resource utilization on a per customer basis and bill accordingly. > Setting per user quotas would be a plus, too. > > I understand the GAE pricing model is based on the resource > utilization of the application as a whole, but are there any out of > box platform services that an application could use for internal > resource usage management? > > Thanks, > Gabor > > > > -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: CANNOT receive SNS :(
Hi 陈永仁, I've activated your account. -Nick Johnson On Thu, Jul 16, 2009 at 11:53 AM, 陈永仁 wrote: > > I want to create a Google App engine Account, but I cannot receive > SNS in my country-china mainland. Some of my friends said that the > china mobile company does not support Google App Engine any more. > > so I have to ask Google App Engine Team for help > > Is there any other way of verifying my account? > > Thank you > > Best Regard > > John Anderson > > > > -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: More Indexing Woes
The indexes have deleted now, thanks! On Jul 16, 4:49 am, Jeffrey Rosen wrote: > I am getting the "Your application is exceeding a quota: Datastore > Indices Count" error too for application "jeffr". I am not totally > sure what it means. > > On Jul 14, 10:48 am, "Jeff S (Google)" wrote: > > > Hi Scott, > > > Apologies for not getting back to you earlier. We were aware of the index > > issues with your application and your index quota should now be reset. Could > > you give vacuuming your indexes another try? Please allow a couple of hours > > for the indexes to be removed and let me know if you there are still issues. > > > Thank you, > > > Jeff > > > On Mon, Jul 13, 2009 at 3:21 PM, Scott wrote: > > > > Hi, > > > > I posted about a week ago, but some of the issues were never resolved, > > > and there doesn't seem to be any response on that thread, so I'll try > > > a new one. > > > > My application has indexes that are in a constant state of error. They > > > don't respond to vacuuming. I'd like to have them deleted before > > > attempting to build more indexes which I need for the features I'm > > > working on next in my application. So time is of the essence. > > > > Also, I just got a 'Your application is exceeding a quota: Datastore > > > Indices Count' message on my dashboard, but I have only about a couple > > > dozen indexes, including the ones that are in an error state. > > > > Thanks > > > > -Scott > > > application: skrit --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Clustering the data and Indices
Hi Mark, You only need to use Entity groups if you need to update multiple entities in a single transaction. Using them will not speed up insertion or querying. Using separate 'account group' and 'account' entity types, with a ReferenceProperty to connect an Account to its AccountGroup is probably your best bet here. -Nick Johnson On Wed, Jul 15, 2009 at 6:01 PM, Mark Jones wrote: > > I'm expecting lots of incoming data (about 80-90 million rows per day) > that needs to be broken across many "accounts" and will largely be > retrieved as a group on the account. > > I'm wanting to understand how to set up my entities and entity groups > so that the indexes are clustered around account in such a way that > ingestion and query are both fast. > > A total hierarchy looks like: > > Account Group - Queries across a group of accounts where the # of > accounts is small, and queries are few (other than to list the > accounts) > > Account - Date bounded queries filtered by one or two fields > > What is the correct structure for this kind of datastore. > > > > -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: bulkloader raises "no module named appengine.base"
Hi omat,
It looks like you're importing, probably indirectly, your Django app,
which is attempting to set itself up inside the bulk loader. You need
to make sure that when you import your models, you don't also import
the Django parts of your app.
-Nick Johnson
On Thu, Jul 16, 2009 at 2:42 PM, omat wrote:
>
> Hi,
>
> I am using Django 1.1 with App Engine Helper for Django. My
> application runs fine, but when I try to upload data using bulkloader,
> "no module named appengine.base" is raised.
>
>
> I issue the command in the root directory of the project (where the
> app.yaml resides):
>
> $ appcfg.py upload_data --config_file=profile/loaders.py --
> filename=profile/initial_data/region.csv --kind=Region --url=http://
> localhost:8082/remote_api ./
>
>
> It seems to get going at the first 2 lines, as it responds:
>
> Uploading data records.
> [INFO ] Logging to bulkloader-log-20090716.162701
>
>
> Then a "no module named appengine.base" is raised. The full traceback
> is as follows:
>
> Traceback (most recent call last):
> File "/usr/local/bin/appcfg.py", line 60, in
> run_file(__file__, globals())
> File "/usr/local/bin/appcfg.py", line 57, in run_file
> execfile(script_path, globals_)
> File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> google/appengine/tools/appcfg.py", line 2303, in
> main(sys.argv)
> File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> google/appengine/tools/appcfg.py", line 2294, in main
> result = AppCfgApp(argv).Run()
> File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> google/appengine/tools/appcfg.py", line 1458, in Run
> self.action(self)
> File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> google/appengine/tools/appcfg.py", line 2182, in __call__
> return method()
> File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> google/appengine/tools/appcfg.py", line 2071, in PerformUpload
> run_fn(args)
> File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> google/appengine/tools/appcfg.py", line 1984, in RunBulkloader
> sys.exit(bulkloader.Run(arg_dict))
> File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> google/appengine/tools/bulkloader.py", line 4354, in Run
> return _PerformBulkload(arg_dict)
> File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> google/appengine/tools/bulkloader.py", line 4219, in _PerformBulkload
> LoadConfig(config_file)
> File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
> GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
> google/appengine/tools/bulkloader.py", line 3992, in LoadConfig
> ('', 'r', imp.PY_SOURCE))
> File "parliament/loaders.py", line 9, in
> import models
> File "/Users/omat/workspace/vekilim/gae/parliament/models.py", line
> 1, in
> from appengine_django.models import BaseModel
> File "/Users/omat/workspace/projetakvimi/gae/appengine_django/
> models.py", line 24, in
> from django.db.models.fields import Field
> File "/Users/omat/workspace/projetakvimi/gae/django/db/__init__.py",
> line 34, in
> (settings.DATABASE_ENGINE, ", ".join(map(repr,
> available_backends)), e_user)
> django.core.exceptions.ImproperlyConfigured: 'appengine' isn't an
> available database backend. Available options are: 'dummy', 'mysql',
> 'oracle', 'postgresql', 'postgresql_psycopg2', 'sqlite3'
> Error was: No module named appengine.base
>
>
>
> Any help would be appreciated greatly.
>
>
> Thanks,
> omat
>
>
> >
>
--
Nick Johnson, App Engine Developer Programs Engineer
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
Number: 368047
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] How does the GAE infrastructure support this?
Hi All, I have a traditional WEB application with the following features: - the provided services are very computation-intensive - the usage is subscription-based, you need to register and log in to access the services - there are free and paid versions of the services, the key differentiating factor is that paid requests have higher priority, can consume more resources, etc than the free ones. - there is a flat monthly fee for the paid services. Beyond the obvious (scaling…) limitations the other main reason this application needs to be modified/recreated is the flat fee model. To provide a real pay-as-you-go experience it would be good to track the resource utilization on a per customer basis and bill accordingly. Setting per user quotas would be a plus, too. I understand the GAE pricing model is based on the resource utilization of the application as a whole, but are there any out of box platform services that an application could use for internal resource usage management? Thanks, Gabor --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Can't manage the application
Hi all, I created a google app and used the admin account to create the App Engine app. But then, I can't access the App Engine dashboard. http://appengine.google.com/dashboard?&app_id=speed-cupid Can any googler help? Cheers --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Can't manage the application
Hi Itcstyle, Are you logging in using the account you created the app with, or one on the domain you just set up? If the former, is it an Apps domain? You may need to log in at http://appengine.google.com/a/yourdomain . -Nick Johnson On Thu, Jul 16, 2009 at 2:54 PM, ltcstyle wrote: > > Hi, > > I have a problem with google app engine dash board. > > I created a google app and an admin email account. > Then I register a google app engine use my admin email in the newly > created domain. > > I can upload the code, and it all runs ok, but when I try to log in to > manage the application, google app engine doesn't show me that app > under my account. > And I tried http://appengine.google.com/dashboard?&app_id=**, it > doesn't help. > > Could anyone help? > > Cheers > > > > -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: No SMS support for my country
Hello, I am from Mongolia, I have a problem with "Verify Your Account by SMS" page. No SMS support for my country. Is there any other way of verifying my account? Thanks. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] CANNOT receive SNS :(
I want to create a Google App engine Account, but I cannot receive SNS in my country-china mainland. Some of my friends said that the china mobile company does not support Google App Engine any more. so I have to ask Google App Engine Team for help Is there any other way of verifying my account? Thank you Best Regard John Anderson --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Persisting Objects - Objects as primary keys?
I am using the Eclipse Plugin to develope for the GAE. I would like to
persist an object, using following source code:
@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class Test {
@Persistent(primaryKey = "true", valueStrategy =
IdGeneratorStrategy.IDENTITY)
private Long id;
@Persistent
private TestObject obj;
public Test() {
this.obj = new TestObject();
}
}
@PersistenceCapable(embeddedOnly = "true")
public class TestObject {
@Persistent
private String test;
public TestObject() {
this.test = "test";
}
}
I am using the persistence manager with following code:
Test t = new Test();
PersistenceManager pm = JDOPersistenceManagerFactory.get
().getPersistenceManager();
try {
pm.makePersistent(t);
} finally { pm.close(); }
However the source code which can be seen above does not persist
anything but the id. My final goal is to have the TestObject as a
primary key, with either the id as a field of the object or even
better no specific id in the TestObject, but simply use the existing
fields of the object as unique idetification. Is that possible at all?
For debugging purposes it would be great to know where the persistent
data is stored in case the app is run as a local web application. So
far I could not find anything that looked like a lokal datastore.
Any help is greatly appreciated!
Hacci
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Can't manage the application
Hi, I have a problem with google app engine dash board. I created a google app and an admin email account. Then I register a google app engine use my admin email in the newly created domain. I can upload the code, and it all runs ok, but when I try to log in to manage the application, google app engine doesn't show me that app under my account. And I tried http://appengine.google.com/dashboard?&app_id=**, it doesn't help. Could anyone help? Cheers --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Task Queue Exception: What does it mean?
I got a "Transient Error" when putting something into the Task Queue. I'll catch this exception and deal with it going forward, but I figured since the taskqueue api is still 'beta', you'd want to know that this error message is less than helpful!!! Traceback (most recent call last): File "/base/python_lib/versions/1/google/appengine/ext/webapp/ __init__.py", line 501, in __call__ handler.get(*groups) File "/base/data/home/apps/mesonstreaming/1.334940745140132970/ main.py", line 348, in get taskqueue.add(url='/admin/checkStarted?session=%s' % session.key (), method='GET', countdown='30') File "/base/python_lib/versions/1/google/appengine/api/labs/ taskqueue/taskqueue.py", line 637, in add return Task(*args, **kwargs).add() File "/base/python_lib/versions/1/google/appengine/api/labs/ taskqueue/taskqueue.py", line 495, in add return Queue(queue_name).add(self) File "/base/python_lib/versions/1/google/appengine/api/labs/ taskqueue/taskqueue.py", line 563, in add self.__TranslateError(e) File "/base/python_lib/versions/1/google/appengine/api/labs/ taskqueue/taskqueue.py", line 592, in __TranslateError raise TransientError(error.error_detail) TransientError --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] bulkloader raises "no module named appengine.base"
Hi,
I am using Django 1.1 with App Engine Helper for Django. My
application runs fine, but when I try to upload data using bulkloader,
"no module named appengine.base" is raised.
I issue the command in the root directory of the project (where the
app.yaml resides):
$ appcfg.py upload_data --config_file=profile/loaders.py --
filename=profile/initial_data/region.csv --kind=Region --url=http://
localhost:8082/remote_api ./
It seems to get going at the first 2 lines, as it responds:
Uploading data records.
[INFO] Logging to bulkloader-log-20090716.162701
Then a "no module named appengine.base" is raised. The full traceback
is as follows:
Traceback (most recent call last):
File "/usr/local/bin/appcfg.py", line 60, in
run_file(__file__, globals())
File "/usr/local/bin/appcfg.py", line 57, in run_file
execfile(script_path, globals_)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/appcfg.py", line 2303, in
main(sys.argv)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/appcfg.py", line 2294, in main
result = AppCfgApp(argv).Run()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/appcfg.py", line 1458, in Run
self.action(self)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/appcfg.py", line 2182, in __call__
return method()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/appcfg.py", line 2071, in PerformUpload
run_fn(args)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/appcfg.py", line 1984, in RunBulkloader
sys.exit(bulkloader.Run(arg_dict))
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/bulkloader.py", line 4354, in Run
return _PerformBulkload(arg_dict)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/bulkloader.py", line 4219, in _PerformBulkload
LoadConfig(config_file)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/
GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/
google/appengine/tools/bulkloader.py", line 3992, in LoadConfig
('', 'r', imp.PY_SOURCE))
File "parliament/loaders.py", line 9, in
import models
File "/Users/omat/workspace/vekilim/gae/parliament/models.py", line
1, in
from appengine_django.models import BaseModel
File "/Users/omat/workspace/projetakvimi/gae/appengine_django/
models.py", line 24, in
from django.db.models.fields import Field
File "/Users/omat/workspace/projetakvimi/gae/django/db/__init__.py",
line 34, in
(settings.DATABASE_ENGINE, ", ".join(map(repr,
available_backends)), e_user)
django.core.exceptions.ImproperlyConfigured: 'appengine' isn't an
available database backend. Available options are: 'dummy', 'mysql',
'oracle', 'postgresql', 'postgresql_psycopg2', 'sqlite3'
Error was: No module named appengine.base
Any help would be appreciated greatly.
Thanks,
omat
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Webservice authentication (Python)
On Wed, Jul 15, 2009 at 9:55 PM, epb wrote:
>
> On Jul 15, 4:05 pm, Tony wrote:
>> The response has the "Set-cookie" header set, which will cause the
>> user's browser to save the cookie and then present it on the next
>> request (after redirected by the 302). In my code I've opted not to
>> follow the redirect, and extracted the cookie myself, because it's the
>> urlfetch service doing the request, not the user. You can then either
>> return a response to the user with a "Set-cookie" HTTP header (causing
>> their browser to save the cookie), or handle it some other way (return
>> it in the body and set the cookie with Javascript, for example).
>>
>> I think I misunderstood your original question, though, and you're
>> looking for something different. You want to get an authorization
>> cookie and then use it to make repeated requests with urlfetch, not
>> with a browser? If that's the case, you're going to want to capture
>> the "Set-cookie" header from the second response, and supply that in
>> future requests (setting the "Cookie" header for urlfetch).
>
> Yes, that is exactly what I want :) My client app. is not browser-
> based. I guess I'll just use your method then.. appengine_rpc must be
> intended for browser apps only, as it does nothing to capture the
> auth. cookie. I could of course extend the appengine_rpc module to
> capture the cookie, but the module uses urllib2.OpenerDirector.open()
> to open URLs and this is perhaps not the way to go in my case? I am
> not sure what the difference is between urlfetch() and open() it
> seems like I can get the headers (and hereby the cookie) by using info
> () on the response from open().
appengine_rpc is intended for any Python app. It captures the cookie
by using a CookieJar, which does the capturing/sending of the cookie.
-Nick Johnson
>
>>
>> Basically, urlfetch will follow redirects but it won't handle cookies
>> automatically - so what's happening is it's ignoring the "Set-cookie"
>> header and following the redirect, and being denied because it's not
>> supplying a cookie.
>>
>> On Jul 15, 2:58 pm, epb wrote:
>>
>> > I can see why Tony's version would work. His "algorithm" has two
>> > steps:
>>
>> > 1. Get the authorization token using ClientLogin (which I also managed
>> > to do).
>> > 2. Use the uri "servername/_ah/login" to get the auth. cookie.
>>
>> > The appengine_rpc module seems to do authentication in a similar way:
>>
>> > A. Try to access the app. This results in a redirect to a location
>> > that starts withhttps://www.google.com/accounts/ServiceLogin
>> > B. Get a auth. token (like step 1 above)
>> > C. Use auth. token to get auth. cookie.
>> > D. Try to access the app. again (this is where it fails in my case...)
>>
>> > Anyway, step C is performed using the function below:
>>
>> > ---
>>
>> > def _GetAuthCookie(self, auth_token):
>> > """Fetches authentication cookies for an authentication token.
>>
>> > Args:
>> > auth_token: The authentication token returned by ClientLogin.
>>
>> > Raises:
>> > HTTPError: If there was an error fetching the authentication
>> > cookies.
>> > """
>> > continue_location = "http://localhost/";
>> > args = {"continue": continue_location, "auth": auth_token}
>> > login_path = os.environ.get("APPCFG_LOGIN_PATH", "/_ah")
>> > req = self._CreateRequest("%s://%s%s/login?%s" %
>> > (self.scheme, self.host, login_path,
>> > urllib.urlencode(args)))
>> > try:
>> > response = self.opener.open(req)
>> > except urllib2.HTTPError, e:
>> > response = e
>> > if (response.code != 302 or
>> > response.info()["location"] != continue_location):
>> > raise urllib2.HTTPError(req.get_full_url(), response.code,
>> > response.msg,
>> > response.headers, response.fp)
>> > self.authenticated = True
>>
>> > --
>>
>> > It seems to me, that we do nothing with the response in this
>> > function?? Shouldn't we save the cookie in the response like Tony's
>> > does above, and then use it when we try to log in again?
>>
>> > On Jul 15, 1:06 pm, epb wrote:
>>
>> > > Thanks for your answers.
>>
>> > > As I understand Nick's response, I only need to use appengine_rpc.py
>> > > for the entire process. I tried the following:
>>
>> > > ---
>>
>> > > def passwdFunc():
>> > > return ('my_email','my_passwd')
>>
>> > > rpcServer = appengine_rpc.HttpRpcServer
>> > > ('myapp.appspot.com',passwdFunc,None,'myAppName')
>> > > blah = rpcServer.Send('/')
>>
>> > > ---
>>
>> > > This gave me a 302 error and the following log:
>>
>> > > ---
>>
>> > > Server: myapp.appspot.com
>> > > Sending HTTP request:
>> > > POST /? HTTP/1.1
>> > > Host: myapp.appspot.com
>> > > X-appcfg-api-version: 1
>> > > Content-type: application/octet-stream
>>
>> > > Got http error, this is try #1
>> > > Got 302 redirect.
>> > > Location:https://www.google.com/acco
[google-appengine] Re: No SMS support for my country
Hi Andres, I've manually activated your account. -Nick Johnson 2009/7/15 Andrés Mejía : > > Hello, > > I tried to create an application in the App Engine but couldn't verify > my account since there's no SMS support for my country. > > Is there any other way of verifying my account? > > Thanks a lot. > > > > -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: simultaneous dynamic requests, max blob size, multi-blob?
Hi Jeff, On Thu, Jul 16, 2009 at 3:33 AM, Jeff Enderwick wrote: > > 1) I saw that 30 was listed somewhere as the max simultaneous dynamic > requests for an app. Is this really true? Even if I am paying, and I > have a very popular app? I see the math indicating that one can retire > 400/sec with 75ms latency, but I am sure that GOOG-internal apps > aren't held to this limit of 30. Is this really a hard limit? It's not a hard limit, no. If your app scales large enough, this will be increased. 400 QPS is a very busy app, though. > > 2) Great to see that "Service for storing and serving large files" is > on deck. Can y'all give any indication whether this will be > accomplished by making blobs bigger (or adding some sort of uberblob)? > Any indications on new max size would be great. It will definitely be capable of storing blobs larger than the current 10MB limit. For more details, you'll have to wait for the service to be released. -Nick Johnson > > 3) Along those lines, has anyone implemented something like multi-blob > fragmentation/reassembly for storing & serving things greater than > 1MB? > > Thanks! > Jeff > > > > -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Looking for solution for bigtable problem (which can be solved in relational db easily)
I have a very simple model. Users and Books. Every user can pick a list of favourite and disliked books. Now I like to present an public page with most favourite books. Thi is no problem, I could create a counter (perhaps a sharded counter). But when I click a specific book I want to display it details + (and now comes the difficult part) a list of books which Users also voted for who vote for the particular book displayed. Similar to Amazon's "People who bought this book, also bought..." With a relational db query this would be solved very easily, but I don't see an effective way to this with big table. Any ideas? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: More Indexing Woes
I am getting the "Your application is exceeding a quota: Datastore Indices Count" error too for application "jeffr". I am not totally sure what it means. On Jul 14, 10:48 am, "Jeff S (Google)" wrote: > Hi Scott, > > Apologies for not getting back to you earlier. We were aware of the index > issues with your application and your index quota should now be reset. Could > you give vacuuming your indexes another try? Please allow a couple of hours > for the indexes to be removed and let me know if you there are still issues. > > Thank you, > > Jeff > > > > On Mon, Jul 13, 2009 at 3:21 PM, Scott wrote: > > > Hi, > > > I posted about a week ago, but some of the issues were never resolved, > > and there doesn't seem to be any response on that thread, so I'll try > > a new one. > > > My application has indexes that are in a constant state of error. They > > don't respond to vacuuming. I'd like to have them deleted before > > attempting to build more indexes which I need for the features I'm > > working on next in my application. So time is of the essence. > > > Also, I just got a 'Your application is exceeding a quota: Datastore > > Indices Count' message on my dashboard, but I have only about a couple > > dozen indexes, including the ones that are in an error state. > > > Thanks > > > -Scott > > application: skrit --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
