[google-appengine] Re: DB lmitations: how to use multiple criteria on many columns? URGENT!
> Neither of these will work; you can't use inequality filters on more > than one property. Thanks for the heads up on that. I thought I was missing something obvious as the question seemed too straighforward and I've not had to do such queries yet. For those looking for more info on queries and indexes, the relevant documentation is here: http://code.google.com/appengine/docs/python/datastore/queriesandindexes.html#Restrictions_on_Queries -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Blobstore Upload w/Textarea problems
I'm performing a POST to a Blobstore URL with a textarea field (and several other fields). The textarea field is getting converted to MIME quoted-printable format. Is there something I am doing wrong or is this an issue with App Engine? I am using multipart/form-data POST as instructed by the documentation for Blobstore uploads, and my page is UTF-8. I am able to work around this issue by decoding the string using the Python quopri module. Also, please note this does not happen on my dev server, only in production. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: What's a better choice - updating a record or overwriting?
@alon yes I'm managing my keys. I had read updating an entity was expensive (now I'm not sure if by updating, they meant "put"). Probably as @Tim says, updating would mean a get and then a put. So, I guess if I'm managing my keys and I don't care about old data, doing a "put" would be a good option. -N On Sep 20, 5:18 pm, Tim Hoffman wrote: > Alon > > On Sep 20, 8:07 pm, alon wrote:> actually i read the > basic rule of app engine is that PUT is the most > > resource extensive job you can do and should avoid it at all cause :) > > Correct. (well actually a big query is way more expensive than > putting one entity ;-) > > > Also correct me if im wrong, but in order to update you need to query > > to get the object itself, how can you update an object without pulling > > the data before hand? or are you managing your keys? > > There is no update in appengine, you get() or put() . An update > is get(), modify properties, put() writing the whole entity on each > put. > > And yes you would need to manage keys if you are just doing puts, > hence my question "do you care about old data" > > > Come to think of it. overwrite will require the same. im guessing it > > wont make a difference in performance. > > overwrite is just a put, but get + put is more expensive. Especially > if you don't care about the old data. > > T > > > > > On Sep 19, 2:41 pm, nischalshetty wrote: > > > > Ah, that's a good point. Ok, so if I don't care about the existing > > > data, I can simple overwrite them without worrying. I hope its the > > > case if say I have like 50 million entities? > > > > -N > > > > On Sep 19, 4:16 pm, Tim Hoffman wrote: > > > > > Hi > > > > > That all depends on if there is any data in the old entity that you > > > > want to copy to the overwritten > > > > entity. If not it will be cheaper to do a put rather than a get > > > > followed by a put. > > > > > T > > > > > On Sep 19, 2:03 pm, nischalshetty wrote: > > > > > > When I started out, I made a few of my entities without the appengine > > > > > "Key" as primary key i.e. made them with numeric primary keys. > > > > > > Then I moved on to using "Key" as a primary key for my entities. In > > > > > the documentation it says if we try to persist an entity with an > > > > > already existing key, it'll overwrite the existing record. > > > > > > So, my question is, to update my entities, should I just overwrite > > > > > them or should I do the traditional, get entity, modify it and then > > > > > call the update method to persist it? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Multiple Domains for one app, even if you don't own the domain, or if other domain is already registered with Google Apps
You can't point the naked domain at an App Engine app at all. However, App Engine doesn't control your DNS settings. You set CNAMEs for the subdomains you want handled by App Engine or other Google Apps services to ghs.google.com, and setup CNAME or A records for the other addresses in your domain to point to wherever you want them to go. On Sep 20, 6:02 pm, nacho wrote: > What about the inverse case? > > Let's say i have mydomain.com poiting to my appengine app and i want > to use something.mydomain.com poiting to another server? How can i do > it? > > On 20 sep, 05:11, adams wrote: > > > > > I've been playing around with this kind of scenario as well. As far as > > I can tell you don't have to do anything from your side, except give > > them your app engine id. Your client needs to have a Google Apps > > account for their domain name. They can then add your app engine > > application to their Google Apps account and point any URL for that > > domain to your app engine application. They would do this by: > > > 1. Logging into their Google Apps account > > 2. Click on "Add more services" > > 3. Enter app id on the Google App Engine section under "Other > > services" > > > I am not 100% sure the above is true, but it seems to have worked for > > one of my Google Apps domains. > > > On Sep 17, 10:05 pm, NickFranceschina > > wrote: > > > > I promise I have been searching... but I haven't been unable to find > > > the correct answer. Thus I turn to you fine folks. > > > > So lets say I have my super fancy new killer app-engine app: > > > http://mykillerapp.com > > > which is my custom domain pointed to: > > > http://mykillerapp.appspot.com > > > > and then because I'm a genius salesman, I convince Coca-Cola that they > > > need my app on their site. They write me a huge check with lots of > > > zeros and say "OK, lets set it up athttp://awesome.coca-cola.com"; > > > > how can I make that happen? the only way i can see is to add coca- > > > cola.com to my google apps for domains account (verifying it with > > > CNAME or .HTML file)... but yeah, Coke isn't gonna let me do that cuz > > > it's obviously their domain. > > > > or what if I sell it to JoeSchmoe.com who already has a google apps > > > account... I can't add his domain to my google apps because google > > > tells me "this domain is already registered" (or something like that) > > > > SOOO... is there a way for me to only add awesome.coca-cola.com to my > > > google apps domain so I can link the app to it? or setup > > > awesome.JoeSchmoe.com if JoeSchmoe.com is already a google apps > > > customer? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Web Service with XmlRPC
Hi everyone How to create web service in GAE using python with XmlRPC. Show me some code. Thank you Tsolmon -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] 118 languages set-up help - python
using google app in python - not tipify, Can anyone offer any decent info / help on how to get 118 working so i can build translations. i know tipify does this. Can i use the 118 module in my non tipify app. How do i do this. I want to be able to get translations from code and also through django template. Regards Martin Webb -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Multiple Domains for one app, even if you don't own the domain, or if other domain is already registered with Google Apps
What about the inverse case? Let's say i have mydomain.com poiting to my appengine app and i want to use something.mydomain.com poiting to another server? How can i do it? On 20 sep, 05:11, adams wrote: > I've been playing around with this kind of scenario as well. As far as > I can tell you don't have to do anything from your side, except give > them your app engine id. Your client needs to have a Google Apps > account for their domain name. They can then add your app engine > application to their Google Apps account and point any URL for that > domain to your app engine application. They would do this by: > > 1. Logging into their Google Apps account > 2. Click on "Add more services" > 3. Enter app id on the Google App Engine section under "Other > services" > > I am not 100% sure the above is true, but it seems to have worked for > one of my Google Apps domains. > > On Sep 17, 10:05 pm, NickFranceschina > wrote: > > > I promise I have been searching... but I haven't been unable to find > > the correct answer. Thus I turn to you fine folks. > > > So lets say I have my super fancy new killer app-engine app: > > http://mykillerapp.com > > which is my custom domain pointed to: > > http://mykillerapp.appspot.com > > > and then because I'm a genius salesman, I convince Coca-Cola that they > > need my app on their site. They write me a huge check with lots of > > zeros and say "OK, lets set it up athttp://awesome.coca-cola.com"; > > > how can I make that happen? the only way i can see is to add coca- > > cola.com to my google apps for domains account (verifying it with > > CNAME or .HTML file)... but yeah, Coke isn't gonna let me do that cuz > > it's obviously their domain. > > > or what if I sell it to JoeSchmoe.com who already has a google apps > > account... I can't add his domain to my google apps because google > > tells me "this domain is already registered" (or something like that) > > > SOOO... is there a way for me to only add awesome.coca-cola.com to my > > google apps domain so I can link the app to it? or setup > > awesome.JoeSchmoe.com if JoeSchmoe.com is already a google apps > > customer? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Precompilation failures
Whenever I attempt a deploy of my Python app, it fails with: Cloned 2200 files. Precompilation starting. Rolling back the update. Error 500: --- begin server output --- 500 Server Error Error: Server Error The server encountered an error and could not complete your request.If the problem persists, please report your problem and mention this error message and the query that caused it. --- end server output --- I think it has something to do with the precompilation step because if I attempt to deploy the app several more times it finishes successfully (presumably after the compile completes). Is there a way to increase the timeout for precompilation? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Storage Quota issue w/ Total Stored Data
Yesterday I nearly hit the daily Storage Quota for Total Stored Data, was at .93 GB. Using the Datastore Viewer I deleted all of my entities yesterday. Today I was expecting the Total Stored Data quota to be at 0, but it's still at .93 GB. Wondering what's is going on? The GAE storage quota isn't tied to storage amounts w/ Gmail and Google Docs? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Solving cold start problems with stale-while-revalidate?
Hi everybody! We already have a reverse proxy in GAE that caches responses with Cache-control:public. I was wondering if we can also use the stale- while-revalidate option to prevent cold starts. According to the spec, that option allows the proxy to return a stale version while fetching the new one: http://www.igvita.com/2008/10/07/asynchronous-http-cache-validations/ http://tools.ietf.org/html/draft-nottingham-http-stale-while-revalidate-01 My questions: 1) Does GAE reverse proxy support stale-while-revalidate? 2) Do you think that strategy can minimize cold start impacts? Thanks Sérgio -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Scaleability and the magic 1000ms request time
I did another count across all of our urls. 1 hour, 1.04p - 2.04p log time, 2010-09-20 DeadlineExceededErrors: 157 10-second timeouts: 179 Given the high rate of 10-second timeouts, can I assume that I am over the 1000ms threshhold? These errors represent almost all of the errors from our system. j On Sep 20, 2:59 pm, Jason C wrote: > For one of my urls, for a 1 hour period (12.50p to 1.50p log time > 2010-09-20), I saw 89 DeadlineExceededErrors and 101 of the 10-second > timeouts. (appid: steprep) > > WRT DeadlineExceededError - these requests are normally _well_ below > the 30s boundary. > > j > > On Sep 20, 1:24 pm, Kenneth wrote: > > > > > We're seeing a lot of 10 second and deadline errors today. Nothing > > like last week but it is still pretty bad today. > > > There are 21 non-task 10 second errors and 5 30 second deadline errors > > in the past 8 hours. The deadline errors are on calls that would > > normally take <500ms. > > > On Sep 20, 5:26 pm, Jason C wrote: > > > > I _do_ believe I'm seeing otherwise - in the form of lots of deadline- > > > related errors on large spike jobs (e.g., mapreduce and other > > > continuation-styled jobs). > > > > Do you have any suggestions how I could measure this? > > > > j > > > > On Sep 20, 9:55 am, "Ikai Lan (Google)" > > > wrote:> Task Queues and cron jobs should not. We encourage small tasks, > > > but in > > > > general tasks that take several seconds to run should not impact your > > > > autoscaling. If you're seeing otherwise, please let us know. > > > > > -- > > > > Ikai Lan > > > > Developer Programs Engineer, Google App Engine > > > > Blogger:http://googleappengine.blogspot.com > > > > Reddit:http://www.reddit.com/r/appengine > > > > Twitter:http://twitter.com/app_engine > > > > > On Mon, Sep 20, 2010 at 11:33 AM, Jason C > > > > wrote: > > > > > Ikai, > > > > > > Do you have a definitive answer on whether or not task/cron requests > > > > > count towards the 1000ms threshold? There seems to be some confusion > > > > > and counter-evidence here. > > > > > > Including our cron/task requests, we run at 1500-2000ms / request. > > > > > This is largely because we have LOTS of taskqueue items and we tend to > > > > > do a fair amount of work in them. Further, when we do large spike jobs > > > > > (e.g., mapreduce), we see lots of deadline-related errors. > > > > > > What is the best way to know if we're above or below this threshold? > > > > > (appid: steprep) > > > > > > j > > > > > > On Sep 16, 7:41 pm, "Jan Z/ Hapara" wrote: > > > > > > Hi Ikai - the behavior we are seeing suggests the "offline" tasks > > > > > > are > > > > > > subject to the same 1000msec rule as external requests. > > > > > > > Queuing up a number of tasks reliably results in the "Request was > > > > > > aborted after waiting too long to attempt to service your request" > > > > > > error - which is actually fine, BUT, the appengine kicks in the > > > > > > back- > > > > > > off algorithm. > > > > > > > This results in tasks that cycle for 20+ generations, with mean time > > > > > > between run attempts of 19hr+. > > > > > > > How do we know the 1000 msec rule is in effect? > > > > > > > The situation improves drastically if we introduce a large number of > > > > > > "no-op" tasks that complete in ~40 msec and skew the averages. > > > > > > > J > > > > > > > On Sep 17, 2:05 am, "Ikai Lan (Google)" > > > > > > > > > > > > > wrote: > > > > > > > > Jason, I think your situation is fine. Offline tasks have the > > > > > > > property > > > > > that, > > > > > > > unlike user-facing tasks, do not require instant execution. If you > > > > > schedule > > > > > > > an offline task for "now", that actually means "when there's > > > > > > > capacity" > > > > > and > > > > > > > App Engine can allocate idle capacity to process your request. > > > > > > > Thus, > > > > > the > > > > > > > need to spin up additional instances is unnecessary in most > > > > > > > cases. Are > > > > > you > > > > > > > seeing that your tasks are backed up? > > > > > > > > On Thu, Sep 16, 2010 at 12:56 PM, bFlood > > > > > > > wrote: > > > > > > > > "which in turn affects the capacity available for running > > > > > > > > offline > > > > > > > > tasks" - so, if you have a low volume site, you won't get that > > > > > > > > many > > > > > > > > instances for your tasks? likewise, if you have some user facing > > > > > > > > requests that go longer then 1000ms (by design or otherwise), > > > > > > > > the > > > > > > > > instances available for your tasks are impacted? or am I > > > > > > > > confused? > > > > > > > > > On Sep 16, 8:44 am, "Nick Johnson (Google)" > > > > > > > > > > > > > > > > wrote: > > > > > > > > > Hi Jason, > > > > > > > > > > The same appservers are used to serve user-facing and offline > > > > > traffic. > > > > > > > > The > > > > > > > > > volume of user-facing traffic (that is below the latency > > > > > > > > > threshold) > > > > > you > > > > > > > > > serve d
[google-appengine] Re: Scaleability and the magic 1000ms request time
For one of my urls, for a 1 hour period (12.50p to 1.50p log time 2010-09-20), I saw 89 DeadlineExceededErrors and 101 of the 10-second timeouts. (appid: steprep) WRT DeadlineExceededError - these requests are normally _well_ below the 30s boundary. j On Sep 20, 1:24 pm, Kenneth wrote: > We're seeing a lot of 10 second and deadline errors today. Nothing > like last week but it is still pretty bad today. > > There are 21 non-task 10 second errors and 5 30 second deadline errors > in the past 8 hours. The deadline errors are on calls that would > normally take <500ms. > > On Sep 20, 5:26 pm, Jason C wrote: > > > > > I _do_ believe I'm seeing otherwise - in the form of lots of deadline- > > related errors on large spike jobs (e.g., mapreduce and other > > continuation-styled jobs). > > > Do you have any suggestions how I could measure this? > > > j > > > On Sep 20, 9:55 am, "Ikai Lan (Google)" > > wrote:> Task Queues and cron jobs should not. We encourage small tasks, but > > in > > > general tasks that take several seconds to run should not impact your > > > autoscaling. If you're seeing otherwise, please let us know. > > > > -- > > > Ikai Lan > > > Developer Programs Engineer, Google App Engine > > > Blogger:http://googleappengine.blogspot.com > > > Reddit:http://www.reddit.com/r/appengine > > > Twitter:http://twitter.com/app_engine > > > > On Mon, Sep 20, 2010 at 11:33 AM, Jason C > > > wrote: > > > > Ikai, > > > > > Do you have a definitive answer on whether or not task/cron requests > > > > count towards the 1000ms threshold? There seems to be some confusion > > > > and counter-evidence here. > > > > > Including our cron/task requests, we run at 1500-2000ms / request. > > > > This is largely because we have LOTS of taskqueue items and we tend to > > > > do a fair amount of work in them. Further, when we do large spike jobs > > > > (e.g., mapreduce), we see lots of deadline-related errors. > > > > > What is the best way to know if we're above or below this threshold? > > > > (appid: steprep) > > > > > j > > > > > On Sep 16, 7:41 pm, "Jan Z/ Hapara" wrote: > > > > > Hi Ikai - the behavior we are seeing suggests the "offline" tasks are > > > > > subject to the same 1000msec rule as external requests. > > > > > > Queuing up a number of tasks reliably results in the "Request was > > > > > aborted after waiting too long to attempt to service your request" > > > > > error - which is actually fine, BUT, the appengine kicks in the back- > > > > > off algorithm. > > > > > > This results in tasks that cycle for 20+ generations, with mean time > > > > > between run attempts of 19hr+. > > > > > > How do we know the 1000 msec rule is in effect? > > > > > > The situation improves drastically if we introduce a large number of > > > > > "no-op" tasks that complete in ~40 msec and skew the averages. > > > > > > J > > > > > > On Sep 17, 2:05 am, "Ikai Lan (Google)" > > > > > > > > > > > wrote: > > > > > > > Jason, I think your situation is fine. Offline tasks have the > > > > > > property > > > > that, > > > > > > unlike user-facing tasks, do not require instant execution. If you > > > > schedule > > > > > > an offline task for "now", that actually means "when there's > > > > > > capacity" > > > > and > > > > > > App Engine can allocate idle capacity to process your request. Thus, > > > > the > > > > > > need to spin up additional instances is unnecessary in most cases. > > > > > > Are > > > > you > > > > > > seeing that your tasks are backed up? > > > > > > > On Thu, Sep 16, 2010 at 12:56 PM, bFlood > > > > > > wrote: > > > > > > > "which in turn affects the capacity available for running offline > > > > > > > tasks" - so, if you have a low volume site, you won't get that > > > > > > > many > > > > > > > instances for your tasks? likewise, if you have some user facing > > > > > > > requests that go longer then 1000ms (by design or otherwise), the > > > > > > > instances available for your tasks are impacted? or am I confused? > > > > > > > > On Sep 16, 8:44 am, "Nick Johnson (Google)" > > > > > > > > > > > > > > wrote: > > > > > > > > Hi Jason, > > > > > > > > > The same appservers are used to serve user-facing and offline > > > > traffic. > > > > > > > The > > > > > > > > volume of user-facing traffic (that is below the latency > > > > > > > > threshold) > > > > you > > > > > > > > serve determines how many appservers we provision for your > > > > application, > > > > > > > > which in turn affects the capacity available for running offline > > > > (task > > > > > > > queue > > > > > > > > and cron) tasks. > > > > > > > > > -Nick Johnson > > > > > > > > > On Thu, Sep 16, 2010 at 1:41 PM, Jason C < > > > > jason.a.coll...@gmail.com> > > > > > > > wrote: > > > > > > > > > The number of instances that App Engine makes available to > > > > > > > > > your > > > > > > > > > application depends on if you keep your average request time > > > > under > > > > > > > > > 1000ms for user-facing requests. > > >
[google-appengine] Re: Exploding Indexes - Update?
Wow, thanks Alfred, this is good info.
> - Remove the sort order
> - Restrict the # of tag= in your query
OR (choose one) or AND (we have to do both)?
If I just have to remove the sort order from the query and manipulate
the keys value then I can live with that. But if I have to also
restrict the number of tags to 5 then I guess I'll have to wait until
you complete your work since our spec requires up to 10 tags.
Error message when uploading data through the remote API:
google.appengine.api.datastore_errors.BadRequestError: Too many
indexed
properties for entity: app: "sampleapp",path < Element {type:
"Report",id: 6001 }>
This index put it over the limit: entity_type: "Report",ancestor:
false,
Property { name: "tags", direction: ASCENDING},Property { name:
"tags",
direction: ASCENDING},Property { name: "tags", direction:
ASCENDING},
Property { name: "tags", direction: ASCENDING},Property { name:
"approval_time", direction: DESCENDING}
On Sep 20, 11:52 am, Alfred Fuller
wrote:
> Hi,
>
> This is the quintessential exploding index problem that I am fixing. The
> problem is that we don't currently support zigzag merge join and sorting at
> the same time. You have several options to address this:
>
> - Remove the sort order
> - Slightly worse results until you add it back when I add support for
> zigzag + sort
> - Encode the sort order in the key_name
> - since zigzag results are sorted in key order you can prefix the
> key_name with a value that will replicate that order (key_names
> are sorted
> by lexicographically). This will only work if all the entities
> in your query
> either have no parent or the same parent.
> - Restrict the # of tag= in your query
> - If you try to sort today you have a upper bound of 5 tags per query
> (which in turn restricts you to 5 tags per entity). You can get more
> tags/entity if you further restrict the # of tags in the query.
>
> - Alfred
>
> On Mon, Sep 20, 2010 at 10:08 AM, DevShop wrote:
> > > Do you have 2 list properties? A single list property won't cause
> > exploding
> > > indexes. Can you post the whole property?
>
> > Thanks for your help!
>
> > Do you mean "Can you post the whole class?" ? Here it is, the list
> > property that explodes is the first property:
>
> > class Report(db.Model):
>
> > # List of tags for this report, i.e. ['seattle', 'rockroll_rep',
> > 'rock', 'zztop', 'qwest_field'].
> > tags = db.ListProperty(basestring, required=True,
> > validator=are_valid_tags)
>
> > status = db.StringProperty(required=True, choices = ['PENDING',
> > 'APPROVED', 'DECLINED', 'FLAGGED'], default = 'PENDING')
>
> > # If status is PENDING, the reason is detailed here
> > pending_reason = db.TextProperty(default='')
>
> > # If status is FLAGGED, the reason is detailed here
> > flagged_reason = db.TextProperty(default='')
>
> > # The report, first line is the headline.
> > text = db.TextProperty(default='')
>
> > # URL with more info about this report.
> > url = db.StringProperty(default=None, validator=is_valid_url)
>
> > # Editor that approved this report. Could be None if report
> > submitted
> > from an 'APPROVED' reporter.
> > editor = db.ReferenceProperty(Reporter,
> > collection_name='approved_reports', default=None)
>
> > # Date/time of insertion of this report into the system.
> > creation_time = db.DateTimeProperty(auto_now_add=True)
>
> > # Date/time of approval of this report, might be the same as
> > creation_time if it was automatically approved.
> > approval_time = db.DateTimeProperty(auto_now_add=True)
>
> > On Sep 20, 9:35 am, "Ikai Lan (Google)"
> >
>
> > wrote:
> > > Do you have 2 list properties? A single list property won't cause
> > exploding
> > > indexes. Can you post the whole property?
>
> > > --
> > > Ikai Lan
> > > Developer Programs Engineer, Google App Engine
> > > Blogger:http://googleappengine.blogspot.com
> > > Reddit:http://www.reddit.com/r/appengine
> > > Twitter:http://twitter.com/app_engine
>
> > > On Mon, Sep 20, 2010 at 11:56 AM, DevShop wrote:
> > > > Ikai, thanks for the prompt response.
>
> > > > Here's our situation: records on entity 'Report' are tagged with
> > > > multiple keywords ('tags') like: 'music', 'event', 'downtown',
> > > > 'zz_top', 'concert', 'g_arena', 'september', '2010'. People might
> > > > search for events downtown in September like this:
>
> > > > city.domain.com/tags/event/september/2010/downtown
>
> > > > So we are using a 'tags = db.ListProperty(basestring, required=True,
> > > > validator=are_valid_tags)' in our table with queries like:
>
> > > > reports = db.Query(Report).filter('tags =', subdomain)
> > > > for tag in tags:
> > > > reports = reports.filter('tags =', tag)
> > > > reports = reports.order('-approval_time').fetch(int(25))
>
> > > > Advice and/or insight would be much appr
[google-appengine] Re: DB lmitations: how to use multiple criteria on many columns? URGENT!
Hi,
I´ve come with a solution to a similar problem I had.
What I did was this:
Lets say you can have MAX 6 rooms, so, on one field you store the
actual number of rooms and in other a list of "room queries" que
entity fits:
I'll explain myself:
class Entity{
Long rooms;
ArrayList roomQueries;
}
Now you need a code for every possible query for a room range i.e.
"14" (from 1 to 4), "23" (from 2 to 3), "1" (exactly 1). Then you
calculate the ranges that fit your number of rooms and store them in
"roomQueries".
Having all this set, you can query with a equality filter
i.e .filter(roomQueries = 56).
Another way of implementing this (actually the one im using) is
assigning every "slot" a power of 2, then a range query would be the
sum of all "slots" values involved
Using the above case, you'd have
1 room -- > 1
2 rooms -- > 2
3 rooms -- > 4
4 rooms -- > 8
5 rooms -- > 16
6 rooms -- > 32
the the query for 2 to 3 rooms would be = 6, 2 to 5 = 30.. and so on.
Hope this can help you :)
On Sep 19, 2:11 am, "GAE-framework.googlecode.com"
wrote:
> Hillo GAE developers!
>
> I have a serious situation with datastore. I need select data from
> model with multiple criteria. I have a House model, where exists
> fields: cost, rooms, floor. I need filter all houses, where cost >
> 5 and < 15, and also rooms > = 3 and < 5, and also floor >= 1
> and < 3.
>
> How can I select this dataset? As I see, the Datastore says that I can
> use filter criteria with >, <, >=, <= and != only on one column. But I
> reallly need use this rules on multiple columns.
>
> Please, help me!!!
> How to implement my task with Datastore limitations?
>
> Thank you. I really need your help.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] 118n languages set/up help need python
using google app in python - not tipify, Can anyone offer any decent info / help on how to get 118n working so i can build translations. i know tipify does this. Can i use the 118 module in my non tipify app. How do i do this. I want to be able to get translations from code and also through django template Regards Martin Webb The information contained in this email is confidential and may contain proprietary information. It is meant solely for the intended recipient. Access to this email by anyone else is unauthorised. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted in reliance on this, is prohibited and may be unlawful. No liability or responsibility is accepted if information or data is, for whatever reason corrupted or does not reach its intended recipient. No warranty is given that this email is free of viruses. The views expressed in this email are, unless otherwise stated, those of the author -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Scaleability and the magic 1000ms request time
We're seeing a lot of 10 second and deadline errors today. Nothing like last week but it is still pretty bad today. There are 21 non-task 10 second errors and 5 30 second deadline errors in the past 8 hours. The deadline errors are on calls that would normally take <500ms. On Sep 20, 5:26 pm, Jason C wrote: > I _do_ believe I'm seeing otherwise - in the form of lots of deadline- > related errors on large spike jobs (e.g., mapreduce and other > continuation-styled jobs). > > Do you have any suggestions how I could measure this? > > j > > On Sep 20, 9:55 am, "Ikai Lan (Google)" > wrote:> Task Queues and cron jobs should not. We encourage small tasks, but in > > general tasks that take several seconds to run should not impact your > > autoscaling. If you're seeing otherwise, please let us know. > > > -- > > Ikai Lan > > Developer Programs Engineer, Google App Engine > > Blogger:http://googleappengine.blogspot.com > > Reddit:http://www.reddit.com/r/appengine > > Twitter:http://twitter.com/app_engine > > > On Mon, Sep 20, 2010 at 11:33 AM, Jason C wrote: > > > Ikai, > > > > Do you have a definitive answer on whether or not task/cron requests > > > count towards the 1000ms threshold? There seems to be some confusion > > > and counter-evidence here. > > > > Including our cron/task requests, we run at 1500-2000ms / request. > > > This is largely because we have LOTS of taskqueue items and we tend to > > > do a fair amount of work in them. Further, when we do large spike jobs > > > (e.g., mapreduce), we see lots of deadline-related errors. > > > > What is the best way to know if we're above or below this threshold? > > > (appid: steprep) > > > > j > > > > On Sep 16, 7:41 pm, "Jan Z/ Hapara" wrote: > > > > Hi Ikai - the behavior we are seeing suggests the "offline" tasks are > > > > subject to the same 1000msec rule as external requests. > > > > > Queuing up a number of tasks reliably results in the "Request was > > > > aborted after waiting too long to attempt to service your request" > > > > error - which is actually fine, BUT, the appengine kicks in the back- > > > > off algorithm. > > > > > This results in tasks that cycle for 20+ generations, with mean time > > > > between run attempts of 19hr+. > > > > > How do we know the 1000 msec rule is in effect? > > > > > The situation improves drastically if we introduce a large number of > > > > "no-op" tasks that complete in ~40 msec and skew the averages. > > > > > J > > > > > On Sep 17, 2:05 am, "Ikai Lan (Google)" > > > > > > > > > wrote: > > > > > > Jason, I think your situation is fine. Offline tasks have the property > > > that, > > > > > unlike user-facing tasks, do not require instant execution. If you > > > schedule > > > > > an offline task for "now", that actually means "when there's capacity" > > > and > > > > > App Engine can allocate idle capacity to process your request. Thus, > > > the > > > > > need to spin up additional instances is unnecessary in most cases. Are > > > you > > > > > seeing that your tasks are backed up? > > > > > > On Thu, Sep 16, 2010 at 12:56 PM, bFlood wrote: > > > > > > "which in turn affects the capacity available for running offline > > > > > > tasks" - so, if you have a low volume site, you won't get that many > > > > > > instances for your tasks? likewise, if you have some user facing > > > > > > requests that go longer then 1000ms (by design or otherwise), the > > > > > > instances available for your tasks are impacted? or am I confused? > > > > > > > On Sep 16, 8:44 am, "Nick Johnson (Google)" > > > > > > wrote: > > > > > > > Hi Jason, > > > > > > > > The same appservers are used to serve user-facing and offline > > > traffic. > > > > > > The > > > > > > > volume of user-facing traffic (that is below the latency > > > > > > > threshold) > > > you > > > > > > > serve determines how many appservers we provision for your > > > application, > > > > > > > which in turn affects the capacity available for running offline > > > (task > > > > > > queue > > > > > > > and cron) tasks. > > > > > > > > -Nick Johnson > > > > > > > > On Thu, Sep 16, 2010 at 1:41 PM, Jason C < > > > jason.a.coll...@gmail.com> > > > > > > wrote: > > > > > > > > The number of instances that App Engine makes available to your > > > > > > > > application depends on if you keep your average request time > > > under > > > > > > > > 1000ms for user-facing requests. > > > > > > > > > Ikai Lan (I believe) said that taskqueue and cron job requests > > > > > > > > do > > > not > > > > > > > > count against this boundary. Ikai also said that this boundary > > > was in > > > > > > > > place because longer requests were bad for the ecosystem. > > > > > > > > > Since taskqueue and cron job requests do not count against this > > > > > > > > boundary, in order for them to not be bad for the ecosystem, I'm > > > > > > > > guessing that they are served from a different set of servers > > > than > > > > > > > > user-facing requests are. > > > > > > > > > We (a
[google-appengine] Index Stuck Building for 65 Hours (over 2.5 days)
Can someone from Google please un-stick this Building index on my app? On Friday night at around 10 PM EST (about 65 hours ago), I uploaded a new index.yaml with a new index defined on three properties. This index has been in "Building" status since then.. and at some point on Saturday night, the stats for "Queued", "Running", "Completed", and "Total" disappeared from the Datastore Indexes "Status" box. I do not want to delete this index, vacuum and just restart the entire process since I have no idea how long the re-build will take. And, I have no way of knowing if I'll just be in this same position, attempting to get someone to help get this index build done. My app-id is "me-finance" Thanks for help. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Exploding Indexes - Update?
Hi,
This is the quintessential exploding index problem that I am fixing. The
problem is that we don't currently support zigzag merge join and sorting at
the same time. You have several options to address this:
- Remove the sort order
- Slightly worse results until you add it back when I add support for
zigzag + sort
- Encode the sort order in the key_name
- since zigzag results are sorted in key order you can prefix the
key_name with a value that will replicate that order (key_names
are sorted
by lexicographically). This will only work if all the entities
in your query
either have no parent or the same parent.
- Restrict the # of tag= in your query
- If you try to sort today you have a upper bound of 5 tags per query
(which in turn restricts you to 5 tags per entity). You can get more
tags/entity if you further restrict the # of tags in the query.
- Alfred
On Mon, Sep 20, 2010 at 10:08 AM, DevShop wrote:
> > Do you have 2 list properties? A single list property won't cause
> exploding
> > indexes. Can you post the whole property?
>
> Thanks for your help!
>
> Do you mean "Can you post the whole class?" ? Here it is, the list
> property that explodes is the first property:
>
> class Report(db.Model):
>
># List of tags for this report, i.e. ['seattle', 'rockroll_rep',
> 'rock', 'zztop', 'qwest_field'].
> tags = db.ListProperty(basestring, required=True,
> validator=are_valid_tags)
>
> status = db.StringProperty(required=True, choices = ['PENDING',
> 'APPROVED', 'DECLINED', 'FLAGGED'], default = 'PENDING')
>
># If status is PENDING, the reason is detailed here
>pending_reason = db.TextProperty(default='')
>
># If status is FLAGGED, the reason is detailed here
>flagged_reason = db.TextProperty(default='')
>
># The report, first line is the headline.
>text = db.TextProperty(default='')
>
># URL with more info about this report.
>url = db.StringProperty(default=None, validator=is_valid_url)
>
># Editor that approved this report. Could be None if report
> submitted
> from an 'APPROVED' reporter.
>editor = db.ReferenceProperty(Reporter,
> collection_name='approved_reports', default=None)
>
># Date/time of insertion of this report into the system.
>creation_time = db.DateTimeProperty(auto_now_add=True)
>
># Date/time of approval of this report, might be the same as
> creation_time if it was automatically approved.
>approval_time = db.DateTimeProperty(auto_now_add=True)
>
>
> On Sep 20, 9:35 am, "Ikai Lan (Google)"
>
> >
> wrote:
> > Do you have 2 list properties? A single list property won't cause
> exploding
> > indexes. Can you post the whole property?
> >
> > --
> > Ikai Lan
> > Developer Programs Engineer, Google App Engine
> > Blogger:http://googleappengine.blogspot.com
> > Reddit:http://www.reddit.com/r/appengine
> > Twitter:http://twitter.com/app_engine
> >
> > On Mon, Sep 20, 2010 at 11:56 AM, DevShop wrote:
> > > Ikai, thanks for the prompt response.
> >
> > > Here's our situation: records on entity 'Report' are tagged with
> > > multiple keywords ('tags') like: 'music', 'event', 'downtown',
> > > 'zz_top', 'concert', 'g_arena', 'september', '2010'. People might
> > > search for events downtown in September like this:
> >
> > > city.domain.com/tags/event/september/2010/downtown
> >
> > > So we are using a 'tags = db.ListProperty(basestring, required=True,
> > > validator=are_valid_tags)' in our table with queries like:
> >
> > > reports = db.Query(Report).filter('tags =', subdomain)
> > > for tag in tags:
> > > reports = reports.filter('tags =', tag)
> > > reports = reports.order('-approval_time').fetch(int(25))
> >
> > > Advice and/or insight would be much appreciated!
> >
> > > On Sep 20, 7:59 am, "Ikai Lan (Google)"
> > >
> >
> >
> > > wrote:
> > > > In general, we do not release ETAs for features. Alfred's
> enhancements
> > > are
> > > > coming in multiple small feature releases over the coming months.
> >
> > > > It'd be more practical to post what's causing an exploding index, as
> > > there
> > > > are probably ways to design around it and still meet your needs.
> >
> > > > --
> > > > Ikai Lan
> > > > Developer Programs Engineer, Google App Engine
> > > > Blogger:http://googleappengine.blogspot.com
> > > > Reddit:http://www.reddit.com/r/appengine
> > > > Twitter:http://twitter.com/app_engine
> >
> > > > On Mon, Sep 20, 2010 at 10:49 AM, DevShop
> wrote:
> > > > > At the Google IO Conference in May, Alfred Fuller talked about his
> > > > > solution for the exploding indexes problem.
> >
> > > > > I cannot deploy my app due to an exploding index, just seeding the
> > > > > database with initial data triggers the problem.
> >
> > > > > Any updates as to when the solution will be deployed by the App
> Engine
> > > > > team?
> >
> > > > > --
> > > > > You received this message because you
[google-appengine] Re: Exploding Indexes - Update?
> Do you have 2 list properties? A single list property won't cause exploding
> indexes. Can you post the whole property?
Thanks for your help!
Do you mean "Can you post the whole class?" ? Here it is, the list
property that explodes is the first property:
class Report(db.Model):
# List of tags for this report, i.e. ['seattle', 'rockroll_rep',
'rock', 'zztop', 'qwest_field'].
tags = db.ListProperty(basestring, required=True,
validator=are_valid_tags)
status = db.StringProperty(required=True, choices = ['PENDING',
'APPROVED', 'DECLINED', 'FLAGGED'], default = 'PENDING')
# If status is PENDING, the reason is detailed here
pending_reason = db.TextProperty(default='')
# If status is FLAGGED, the reason is detailed here
flagged_reason = db.TextProperty(default='')
# The report, first line is the headline.
text = db.TextProperty(default='')
# URL with more info about this report.
url = db.StringProperty(default=None, validator=is_valid_url)
# Editor that approved this report. Could be None if report submitted
from an 'APPROVED' reporter.
editor = db.ReferenceProperty(Reporter,
collection_name='approved_reports', default=None)
# Date/time of insertion of this report into the system.
creation_time = db.DateTimeProperty(auto_now_add=True)
# Date/time of approval of this report, might be the same as
creation_time if it was automatically approved.
approval_time = db.DateTimeProperty(auto_now_add=True)
On Sep 20, 9:35 am, "Ikai Lan (Google)"
wrote:
> Do you have 2 list properties? A single list property won't cause exploding
> indexes. Can you post the whole property?
>
> --
> Ikai Lan
> Developer Programs Engineer, Google App Engine
> Blogger:http://googleappengine.blogspot.com
> Reddit:http://www.reddit.com/r/appengine
> Twitter:http://twitter.com/app_engine
>
> On Mon, Sep 20, 2010 at 11:56 AM, DevShop wrote:
> > Ikai, thanks for the prompt response.
>
> > Here's our situation: records on entity 'Report' are tagged with
> > multiple keywords ('tags') like: 'music', 'event', 'downtown',
> > 'zz_top', 'concert', 'g_arena', 'september', '2010'. People might
> > search for events downtown in September like this:
>
> > city.domain.com/tags/event/september/2010/downtown
>
> > So we are using a 'tags = db.ListProperty(basestring, required=True,
> > validator=are_valid_tags)' in our table with queries like:
>
> > reports = db.Query(Report).filter('tags =', subdomain)
> > for tag in tags:
> > reports = reports.filter('tags =', tag)
> > reports = reports.order('-approval_time').fetch(int(25))
>
> > Advice and/or insight would be much appreciated!
>
> > On Sep 20, 7:59 am, "Ikai Lan (Google)"
> >
>
> > wrote:
> > > In general, we do not release ETAs for features. Alfred's enhancements
> > are
> > > coming in multiple small feature releases over the coming months.
>
> > > It'd be more practical to post what's causing an exploding index, as
> > there
> > > are probably ways to design around it and still meet your needs.
>
> > > --
> > > Ikai Lan
> > > Developer Programs Engineer, Google App Engine
> > > Blogger:http://googleappengine.blogspot.com
> > > Reddit:http://www.reddit.com/r/appengine
> > > Twitter:http://twitter.com/app_engine
>
> > > On Mon, Sep 20, 2010 at 10:49 AM, DevShop wrote:
> > > > At the Google IO Conference in May, Alfred Fuller talked about his
> > > > solution for the exploding indexes problem.
>
> > > > I cannot deploy my app due to an exploding index, just seeding the
> > > > database with initial data triggers the problem.
>
> > > > Any updates as to when the solution will be deployed by the App Engine
> > > > team?
>
> > > > --
> > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "Google App Engine" group.
> > > > To post to this group, send email to google-appengine@googlegroups.com
> > .
> > > > To unsubscribe from this group, send email to
> > > > google-appengine+unsubscr...@googlegroups.com
> >
>
> > > > .
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/google-appengine?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Exploding Indexes - Update?
Do you have 2 list properties? A single list property won't cause exploding
indexes. Can you post the whole property?
--
Ikai Lan
Developer Programs Engineer, Google App Engine
Blogger: http://googleappengine.blogspot.com
Reddit: http://www.reddit.com/r/appengine
Twitter: http://twitter.com/app_engine
On Mon, Sep 20, 2010 at 11:56 AM, DevShop wrote:
> Ikai, thanks for the prompt response.
>
> Here's our situation: records on entity 'Report' are tagged with
> multiple keywords ('tags') like: 'music', 'event', 'downtown',
> 'zz_top', 'concert', 'g_arena', 'september', '2010'. People might
> search for events downtown in September like this:
>
> city.domain.com/tags/event/september/2010/downtown
>
> So we are using a 'tags = db.ListProperty(basestring, required=True,
> validator=are_valid_tags)' in our table with queries like:
>
> reports = db.Query(Report).filter('tags =', subdomain)
> for tag in tags:
> reports = reports.filter('tags =', tag)
> reports = reports.order('-approval_time').fetch(int(25))
>
> Advice and/or insight would be much appreciated!
>
>
> On Sep 20, 7:59 am, "Ikai Lan (Google)"
>
> >
> wrote:
> > In general, we do not release ETAs for features. Alfred's enhancements
> are
> > coming in multiple small feature releases over the coming months.
> >
> > It'd be more practical to post what's causing an exploding index, as
> there
> > are probably ways to design around it and still meet your needs.
> >
> > --
> > Ikai Lan
> > Developer Programs Engineer, Google App Engine
> > Blogger:http://googleappengine.blogspot.com
> > Reddit:http://www.reddit.com/r/appengine
> > Twitter:http://twitter.com/app_engine
> >
> > On Mon, Sep 20, 2010 at 10:49 AM, DevShop wrote:
> > > At the Google IO Conference in May, Alfred Fuller talked about his
> > > solution for the exploding indexes problem.
> >
> > > I cannot deploy my app due to an exploding index, just seeding the
> > > database with initial data triggers the problem.
> >
> > > Any updates as to when the solution will be deployed by the App Engine
> > > team?
> >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups
> > > "Google App Engine" group.
> > > To post to this group, send email to google-appengine@googlegroups.com
> .
> > > To unsubscribe from this group, send email to
> > > google-appengine+unsubscr...@googlegroups.com
>
> >
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/google-appengine?hl=en.
> >
> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Scaleability and the magic 1000ms request time
I _do_ believe I'm seeing otherwise - in the form of lots of deadline- related errors on large spike jobs (e.g., mapreduce and other continuation-styled jobs). Do you have any suggestions how I could measure this? j On Sep 20, 9:55 am, "Ikai Lan (Google)" wrote: > Task Queues and cron jobs should not. We encourage small tasks, but in > general tasks that take several seconds to run should not impact your > autoscaling. If you're seeing otherwise, please let us know. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blogger:http://googleappengine.blogspot.com > Reddit:http://www.reddit.com/r/appengine > Twitter:http://twitter.com/app_engine > > > > On Mon, Sep 20, 2010 at 11:33 AM, Jason C wrote: > > Ikai, > > > Do you have a definitive answer on whether or not task/cron requests > > count towards the 1000ms threshold? There seems to be some confusion > > and counter-evidence here. > > > Including our cron/task requests, we run at 1500-2000ms / request. > > This is largely because we have LOTS of taskqueue items and we tend to > > do a fair amount of work in them. Further, when we do large spike jobs > > (e.g., mapreduce), we see lots of deadline-related errors. > > > What is the best way to know if we're above or below this threshold? > > (appid: steprep) > > > j > > > On Sep 16, 7:41 pm, "Jan Z/ Hapara" wrote: > > > Hi Ikai - the behavior we are seeing suggests the "offline" tasks are > > > subject to the same 1000msec rule as external requests. > > > > Queuing up a number of tasks reliably results in the "Request was > > > aborted after waiting too long to attempt to service your request" > > > error - which is actually fine, BUT, the appengine kicks in the back- > > > off algorithm. > > > > This results in tasks that cycle for 20+ generations, with mean time > > > between run attempts of 19hr+. > > > > How do we know the 1000 msec rule is in effect? > > > > The situation improves drastically if we introduce a large number of > > > "no-op" tasks that complete in ~40 msec and skew the averages. > > > > J > > > > On Sep 17, 2:05 am, "Ikai Lan (Google)" > > > > > > > wrote: > > > > > Jason, I think your situation is fine. Offline tasks have the property > > that, > > > > unlike user-facing tasks, do not require instant execution. If you > > schedule > > > > an offline task for "now", that actually means "when there's capacity" > > and > > > > App Engine can allocate idle capacity to process your request. Thus, > > the > > > > need to spin up additional instances is unnecessary in most cases. Are > > you > > > > seeing that your tasks are backed up? > > > > > On Thu, Sep 16, 2010 at 12:56 PM, bFlood wrote: > > > > > "which in turn affects the capacity available for running offline > > > > > tasks" - so, if you have a low volume site, you won't get that many > > > > > instances for your tasks? likewise, if you have some user facing > > > > > requests that go longer then 1000ms (by design or otherwise), the > > > > > instances available for your tasks are impacted? or am I confused? > > > > > > On Sep 16, 8:44 am, "Nick Johnson (Google)" > > > > > wrote: > > > > > > Hi Jason, > > > > > > > The same appservers are used to serve user-facing and offline > > traffic. > > > > > The > > > > > > volume of user-facing traffic (that is below the latency threshold) > > you > > > > > > serve determines how many appservers we provision for your > > application, > > > > > > which in turn affects the capacity available for running offline > > (task > > > > > queue > > > > > > and cron) tasks. > > > > > > > -Nick Johnson > > > > > > > On Thu, Sep 16, 2010 at 1:41 PM, Jason C < > > jason.a.coll...@gmail.com> > > > > > wrote: > > > > > > > The number of instances that App Engine makes available to your > > > > > > > application depends on if you keep your average request time > > under > > > > > > > 1000ms for user-facing requests. > > > > > > > > Ikai Lan (I believe) said that taskqueue and cron job requests do > > not > > > > > > > count against this boundary. Ikai also said that this boundary > > was in > > > > > > > place because longer requests were bad for the ecosystem. > > > > > > > > Since taskqueue and cron job requests do not count against this > > > > > > > boundary, in order for them to not be bad for the ecosystem, I'm > > > > > > > guessing that they are served from a different set of servers > > than > > > > > > > user-facing requests are. > > > > > > > > We (appid: steprep) have a number of external machines that also > > hit > > > > > > > our urls. While we make every effort to keep user-facing requests > > > > > > > quick and responsive, we often use many seconds serving the > > requests > > > > > > > that are built for external machines (by design). > > > > > > > > It has only just struck me this morning that this could be having > > a > > > > > > > bad (perhaps dramatic) impact on our overall scaleability. > > > > > > > > First off, is it true that cron and taskqueue items are
[google-appengine] Re: Problem with django? or appengine
Yes quite possible. What is strange, during all this problems the copy
of this app on different appengine app (I guess that also on different
cloud server but I'm not sure) was working ok. After all I moved my
data and redirected the traffic to this copy and it looks fine now.
On Sep 20, 6:03 pm, Blixt wrote:
> Yeah, probably the same thing described
> inhttp://groups.google.com/group/google-appengine/browse_thread/thread/...
>
> Our application also uses Django 1.1 via use_library.
>
> On Sep 20, 2:23 pm, getoryk wrote:
>
>
>
>
>
>
>
> > I'm using django 1.1 via use_library and today I'm getting lot of
> > DedadlineExcedded Errors (or simply Server error without any
> > traceback). Not always, only from time to time - there are periods of
> > good work. Here is the example of traceback - not sure what to do
> > about this:
>
> > :
> > Traceback (most recent call last):
> > File "/base/data/home/apps/bike-reg/230.344933350703321288/
> > djangomain.py", line 71, in
> > main()
> > File "/base/data/home/apps/bike-reg/230.344933350703321288/
> > djangomain.py", line 68, in main
> > util.run_wsgi_app(application)
> > File "/base/python_runtime/python_lib/versions/1/google/appengine/
> > ext/webapp/util.py", line 97, in run_wsgi_app
> > run_bare_wsgi_app(add_wsgi_middleware(application))
> > File "/base/python_runtime/python_lib/versions/1/google/appengine/
> > ext/webapp/util.py", line 115, in run_bare_wsgi_app
> > result = application(env, _start_response)
> > File "/base/python_runtime/python_lib/versions/third_party/
> > django-1.1/django/core/handlers/wsgi.py", line 241, in __call__
> > response = self.get_response(request)
> > File "/base/python_runtime/python_lib/versions/third_party/
> > django-1.1/django/core/handlers/base.py", line 133, in get_response
> > receivers =
> > signals.got_request_exception.send(sender=self.__class__,
> > request=request)
> > File "/base/python_runtime/python_lib/versions/third_party/
> > django-1.1/django/dispatch/dispatcher.py", line 166, in send
> > response = receiver(signal=self, sender=sender, **named)
> > File "/base/data/home/apps/bike-reg/230.344933350703321288/
> > djangomain.py", line 58, in log_exception
> > logging.exception('Exception in request: %s: %s', cls.__name__,
> > err)
> > File "/base/python_runtime/python_dist/lib/python2.5/logging/
> > __init__.py", line 1301, in exception
> > apply(error, (msg,)+args, {'exc_info': 1})
> > File "/base/python_runtime/python_dist/lib/python2.5/logging/
> > __init__.py", line 1294, in error
> > apply(root.error, (msg,)+args, kwargs)
> > File "/base/python_runtime/python_dist/lib/python2.5/logging/
> > __init__.py", line 1015, in error
> > apply(self._log, (ERROR, msg, args), kwargs)
> > File "/base/python_runtime/python_dist/lib/python2.5/logging/
> > __init__.py", line 1101, in _log
> > self.handle(record)
> > File "/base/python_runtime/python_dist/lib/python2.5/logging/
> > __init__.py", line , in handle
> > self.callHandlers(record)
> > File "/base/python_runtime/python_dist/lib/python2.5/logging/
> > __init__.py", line 1148, in callHandlers
> > hdlr.handle(record)
> > File "/base/python_runtime/python_dist/lib/python2.5/logging/
> > __init__.py", line 655, in handle
> > self.emit(record)
> > File "/base/python_runtime/python_lib/versions/1/google/appengine/
> > api/app_logging.py", line 70, in emit
> > message = self._AppLogsMessage(record)
> > File "/base/python_runtime/python_lib/versions/1/google/appengine/
> > api/app_logging.py", line 83, in _AppLogsMessage
> > message = self.format(record).replace("\n", NEWLINE_REPLACEMENT)
> > File "/base/python_runtime/python_dist/lib/python2.5/logging/
> > __init__.py", line 630, in format
> > return fmt.format(record)
> > File "/base/python_runtime/python_dist/lib/python2.5/logging/
> > __init__.py", line 426, in format
> > record.exc_text = self.formatException(record.exc_info)
> > File "/base/python_runtime/python_dist/lib/python2.5/logging/
> > __init__.py", line 398, in formatException
> > traceback.print_exception(ei[0], ei[1], ei[2], None, sio)
> > File "/base/python_runtime/python_dist/lib/python2.5/traceback.py",
> > line 125, in print_exception
> > print_tb(tb, limit, file)
> > File "/base/python_runtime/python_dist/lib/python2.5/traceback.py",
> > line 69, in print_tb
> > line = linecache.getline(filename, lineno, f.f_globals)
> > File "/base/python_runtime/python_dist/lib/python2.5/linecache.py",
> > line 14, in getline
> > lines = getlines(filename, module_globals)
> > File "/base/python_runtime/python_dist/lib/python2.5/linecache.py",
> > line 40, in getlines
> > return updatecache(filename, module_globals)
> > File "/base/python_runtime/python_dist/lib/python2.5/linecache.py",
> > line 129, in updatecache
> > lines = fp.readlines()
--
You received this message because you are subscribed to the Google
[google-appengine] Re: Problem with django? or appengine
Yeah, probably the same thing described in
http://groups.google.com/group/google-appengine/browse_thread/thread/9945f9e2a50c2395
Our application also uses Django 1.1 via use_library.
On Sep 20, 2:23 pm, getoryk wrote:
> I'm using django 1.1 via use_library and today I'm getting lot of
> DedadlineExcedded Errors (or simply Server error without any
> traceback). Not always, only from time to time - there are periods of
> good work. Here is the example of traceback - not sure what to do
> about this:
>
> :
> Traceback (most recent call last):
> File "/base/data/home/apps/bike-reg/230.344933350703321288/
> djangomain.py", line 71, in
> main()
> File "/base/data/home/apps/bike-reg/230.344933350703321288/
> djangomain.py", line 68, in main
> util.run_wsgi_app(application)
> File "/base/python_runtime/python_lib/versions/1/google/appengine/
> ext/webapp/util.py", line 97, in run_wsgi_app
> run_bare_wsgi_app(add_wsgi_middleware(application))
> File "/base/python_runtime/python_lib/versions/1/google/appengine/
> ext/webapp/util.py", line 115, in run_bare_wsgi_app
> result = application(env, _start_response)
> File "/base/python_runtime/python_lib/versions/third_party/
> django-1.1/django/core/handlers/wsgi.py", line 241, in __call__
> response = self.get_response(request)
> File "/base/python_runtime/python_lib/versions/third_party/
> django-1.1/django/core/handlers/base.py", line 133, in get_response
> receivers =
> signals.got_request_exception.send(sender=self.__class__,
> request=request)
> File "/base/python_runtime/python_lib/versions/third_party/
> django-1.1/django/dispatch/dispatcher.py", line 166, in send
> response = receiver(signal=self, sender=sender, **named)
> File "/base/data/home/apps/bike-reg/230.344933350703321288/
> djangomain.py", line 58, in log_exception
> logging.exception('Exception in request: %s: %s', cls.__name__,
> err)
> File "/base/python_runtime/python_dist/lib/python2.5/logging/
> __init__.py", line 1301, in exception
> apply(error, (msg,)+args, {'exc_info': 1})
> File "/base/python_runtime/python_dist/lib/python2.5/logging/
> __init__.py", line 1294, in error
> apply(root.error, (msg,)+args, kwargs)
> File "/base/python_runtime/python_dist/lib/python2.5/logging/
> __init__.py", line 1015, in error
> apply(self._log, (ERROR, msg, args), kwargs)
> File "/base/python_runtime/python_dist/lib/python2.5/logging/
> __init__.py", line 1101, in _log
> self.handle(record)
> File "/base/python_runtime/python_dist/lib/python2.5/logging/
> __init__.py", line , in handle
> self.callHandlers(record)
> File "/base/python_runtime/python_dist/lib/python2.5/logging/
> __init__.py", line 1148, in callHandlers
> hdlr.handle(record)
> File "/base/python_runtime/python_dist/lib/python2.5/logging/
> __init__.py", line 655, in handle
> self.emit(record)
> File "/base/python_runtime/python_lib/versions/1/google/appengine/
> api/app_logging.py", line 70, in emit
> message = self._AppLogsMessage(record)
> File "/base/python_runtime/python_lib/versions/1/google/appengine/
> api/app_logging.py", line 83, in _AppLogsMessage
> message = self.format(record).replace("\n", NEWLINE_REPLACEMENT)
> File "/base/python_runtime/python_dist/lib/python2.5/logging/
> __init__.py", line 630, in format
> return fmt.format(record)
> File "/base/python_runtime/python_dist/lib/python2.5/logging/
> __init__.py", line 426, in format
> record.exc_text = self.formatException(record.exc_info)
> File "/base/python_runtime/python_dist/lib/python2.5/logging/
> __init__.py", line 398, in formatException
> traceback.print_exception(ei[0], ei[1], ei[2], None, sio)
> File "/base/python_runtime/python_dist/lib/python2.5/traceback.py",
> line 125, in print_exception
> print_tb(tb, limit, file)
> File "/base/python_runtime/python_dist/lib/python2.5/traceback.py",
> line 69, in print_tb
> line = linecache.getline(filename, lineno, f.f_globals)
> File "/base/python_runtime/python_dist/lib/python2.5/linecache.py",
> line 14, in getline
> lines = getlines(filename, module_globals)
> File "/base/python_runtime/python_dist/lib/python2.5/linecache.py",
> line 40, in getlines
> return updatecache(filename, module_globals)
> File "/base/python_runtime/python_dist/lib/python2.5/linecache.py",
> line 129, in updatecache
> lines = fp.readlines()
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Exploding Indexes - Update?
Ikai, thanks for the prompt response.
Here's our situation: records on entity 'Report' are tagged with
multiple keywords ('tags') like: 'music', 'event', 'downtown',
'zz_top', 'concert', 'g_arena', 'september', '2010'. People might
search for events downtown in September like this:
city.domain.com/tags/event/september/2010/downtown
So we are using a 'tags = db.ListProperty(basestring, required=True,
validator=are_valid_tags)' in our table with queries like:
reports = db.Query(Report).filter('tags =', subdomain)
for tag in tags:
reports = reports.filter('tags =', tag)
reports = reports.order('-approval_time').fetch(int(25))
Advice and/or insight would be much appreciated!
On Sep 20, 7:59 am, "Ikai Lan (Google)"
wrote:
> In general, we do not release ETAs for features. Alfred's enhancements are
> coming in multiple small feature releases over the coming months.
>
> It'd be more practical to post what's causing an exploding index, as there
> are probably ways to design around it and still meet your needs.
>
> --
> Ikai Lan
> Developer Programs Engineer, Google App Engine
> Blogger:http://googleappengine.blogspot.com
> Reddit:http://www.reddit.com/r/appengine
> Twitter:http://twitter.com/app_engine
>
> On Mon, Sep 20, 2010 at 10:49 AM, DevShop wrote:
> > At the Google IO Conference in May, Alfred Fuller talked about his
> > solution for the exploding indexes problem.
>
> > I cannot deploy my app due to an exploding index, just seeding the
> > database with initial data triggers the problem.
>
> > Any updates as to when the solution will be deployed by the App Engine
> > team?
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Scaleability and the magic 1000ms request time
Task Queues and cron jobs should not. We encourage small tasks, but in general tasks that take several seconds to run should not impact your autoscaling. If you're seeing otherwise, please let us know. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Mon, Sep 20, 2010 at 11:33 AM, Jason C wrote: > Ikai, > > Do you have a definitive answer on whether or not task/cron requests > count towards the 1000ms threshold? There seems to be some confusion > and counter-evidence here. > > Including our cron/task requests, we run at 1500-2000ms / request. > This is largely because we have LOTS of taskqueue items and we tend to > do a fair amount of work in them. Further, when we do large spike jobs > (e.g., mapreduce), we see lots of deadline-related errors. > > What is the best way to know if we're above or below this threshold? > (appid: steprep) > > j > > On Sep 16, 7:41 pm, "Jan Z/ Hapara" wrote: > > Hi Ikai - the behavior we are seeing suggests the "offline" tasks are > > subject to the same 1000msec rule as external requests. > > > > Queuing up a number of tasks reliably results in the "Request was > > aborted after waiting too long to attempt to service your request" > > error - which is actually fine, BUT, the appengine kicks in the back- > > off algorithm. > > > > This results in tasks that cycle for 20+ generations, with mean time > > between run attempts of 19hr+. > > > > How do we know the 1000 msec rule is in effect? > > > > The situation improves drastically if we introduce a large number of > > "no-op" tasks that complete in ~40 msec and skew the averages. > > > > J > > > > On Sep 17, 2:05 am, "Ikai Lan (Google)" > > > > > > wrote: > > > > > > > > > Jason, I think your situation is fine. Offline tasks have the property > that, > > > unlike user-facing tasks, do not require instant execution. If you > schedule > > > an offline task for "now", that actually means "when there's capacity" > and > > > App Engine can allocate idle capacity to process your request. Thus, > the > > > need to spin up additional instances is unnecessary in most cases. Are > you > > > seeing that your tasks are backed up? > > > > > On Thu, Sep 16, 2010 at 12:56 PM, bFlood wrote: > > > > "which in turn affects the capacity available for running offline > > > > tasks" - so, if you have a low volume site, you won't get that many > > > > instances for your tasks? likewise, if you have some user facing > > > > requests that go longer then 1000ms (by design or otherwise), the > > > > instances available for your tasks are impacted? or am I confused? > > > > > > On Sep 16, 8:44 am, "Nick Johnson (Google)" > > > > > wrote: > > > > > Hi Jason, > > > > > > > The same appservers are used to serve user-facing and offline > traffic. > > > > The > > > > > volume of user-facing traffic (that is below the latency threshold) > you > > > > > serve determines how many appservers we provision for your > application, > > > > > which in turn affects the capacity available for running offline > (task > > > > queue > > > > > and cron) tasks. > > > > > > > -Nick Johnson > > > > > > > On Thu, Sep 16, 2010 at 1:41 PM, Jason C < > jason.a.coll...@gmail.com> > > > > wrote: > > > > > > The number of instances that App Engine makes available to your > > > > > > application depends on if you keep your average request time > under > > > > > > 1000ms for user-facing requests. > > > > > > > > Ikai Lan (I believe) said that taskqueue and cron job requests do > not > > > > > > count against this boundary. Ikai also said that this boundary > was in > > > > > > place because longer requests were bad for the ecosystem. > > > > > > > > Since taskqueue and cron job requests do not count against this > > > > > > boundary, in order for them to not be bad for the ecosystem, I'm > > > > > > guessing that they are served from a different set of servers > than > > > > > > user-facing requests are. > > > > > > > > We (appid: steprep) have a number of external machines that also > hit > > > > > > our urls. While we make every effort to keep user-facing requests > > > > > > quick and responsive, we often use many seconds serving the > requests > > > > > > that are built for external machines (by design). > > > > > > > > It has only just struck me this morning that this could be having > a > > > > > > bad (perhaps dramatic) impact on our overall scaleability. > > > > > > > > First off, is it true that cron and taskqueue items are served on > a > > > > > > different set of servers? If so, is there any way to designate > that a > > > > > > particular url is being requested by a machine and can be routed > to > > > > > > this alternate set (of presumably slower) servers (e.g., a > request > > > > > > header)? > > > > > > > > If I'm way off on all of this, and if taskqueue and cron jobs are > > > > > > served from the same set of servers,
[google-appengine] Re: Scaleability and the magic 1000ms request time
Ikai, Do you have a definitive answer on whether or not task/cron requests count towards the 1000ms threshold? There seems to be some confusion and counter-evidence here. Including our cron/task requests, we run at 1500-2000ms / request. This is largely because we have LOTS of taskqueue items and we tend to do a fair amount of work in them. Further, when we do large spike jobs (e.g., mapreduce), we see lots of deadline-related errors. What is the best way to know if we're above or below this threshold? (appid: steprep) j On Sep 16, 7:41 pm, "Jan Z/ Hapara" wrote: > Hi Ikai - the behavior we are seeing suggests the "offline" tasks are > subject to the same 1000msec rule as external requests. > > Queuing up a number of tasks reliably results in the "Request was > aborted after waiting too long to attempt to service your request" > error - which is actually fine, BUT, the appengine kicks in the back- > off algorithm. > > This results in tasks that cycle for 20+ generations, with mean time > between run attempts of 19hr+. > > How do we know the 1000 msec rule is in effect? > > The situation improves drastically if we introduce a large number of > "no-op" tasks that complete in ~40 msec and skew the averages. > > J > > On Sep 17, 2:05 am, "Ikai Lan (Google)" > wrote: > > > > > Jason, I think your situation is fine. Offline tasks have the property that, > > unlike user-facing tasks, do not require instant execution. If you schedule > > an offline task for "now", that actually means "when there's capacity" and > > App Engine can allocate idle capacity to process your request. Thus, the > > need to spin up additional instances is unnecessary in most cases. Are you > > seeing that your tasks are backed up? > > > On Thu, Sep 16, 2010 at 12:56 PM, bFlood wrote: > > > "which in turn affects the capacity available for running offline > > > tasks" - so, if you have a low volume site, you won't get that many > > > instances for your tasks? likewise, if you have some user facing > > > requests that go longer then 1000ms (by design or otherwise), the > > > instances available for your tasks are impacted? or am I confused? > > > > On Sep 16, 8:44 am, "Nick Johnson (Google)" > > > wrote: > > > > Hi Jason, > > > > > The same appservers are used to serve user-facing and offline traffic. > > > The > > > > volume of user-facing traffic (that is below the latency threshold) you > > > > serve determines how many appservers we provision for your application, > > > > which in turn affects the capacity available for running offline (task > > > queue > > > > and cron) tasks. > > > > > -Nick Johnson > > > > > On Thu, Sep 16, 2010 at 1:41 PM, Jason C > > > wrote: > > > > > The number of instances that App Engine makes available to your > > > > > application depends on if you keep your average request time under > > > > > 1000ms for user-facing requests. > > > > > > Ikai Lan (I believe) said that taskqueue and cron job requests do not > > > > > count against this boundary. Ikai also said that this boundary was in > > > > > place because longer requests were bad for the ecosystem. > > > > > > Since taskqueue and cron job requests do not count against this > > > > > boundary, in order for them to not be bad for the ecosystem, I'm > > > > > guessing that they are served from a different set of servers than > > > > > user-facing requests are. > > > > > > We (appid: steprep) have a number of external machines that also hit > > > > > our urls. While we make every effort to keep user-facing requests > > > > > quick and responsive, we often use many seconds serving the requests > > > > > that are built for external machines (by design). > > > > > > It has only just struck me this morning that this could be having a > > > > > bad (perhaps dramatic) impact on our overall scaleability. > > > > > > First off, is it true that cron and taskqueue items are served on a > > > > > different set of servers? If so, is there any way to designate that a > > > > > particular url is being requested by a machine and can be routed to > > > > > this alternate set (of presumably slower) servers (e.g., a request > > > > > header)? > > > > > > If I'm way off on all of this, and if taskqueue and cron jobs are > > > > > served from the same set of servers, I'm not sure how the "bad for the > > > > > ecosystem" argument holds, and perhaps Google should revisit this > > > > > 1000ms boundary condition altogether. > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > Groups > > > > > "Google App Engine" group. > > > > > To post to this group, send email to google-appengine@googlegroups.com > > > . > > > > > To unsubscribe from this group, send email to > > > > > google-appengine+unsubscr...@googlegroups.com > > > > e...@googlegroups.com> > > e...@googlegroups.com> > > > > > . > > > > > For more options, visit this group at > > > > >http://groups.google.com/group/google-appengine?hl=en. > > > > > -- > > > > Nick Johnso
[google-appengine] Re: DB lmitations: how to use multiple criteria on many columns? URGENT!
On Sep 20, 4:22 am, adams wrote:
> I believe you can do this in two ways:
>
> Either using GQL
> (see:http://code.google.com/appengine/docs/python/datastore/gqlreference.html)
>
> eg: SELECT * FROM House WHERE cost > 5000 AND cost < 15 AND rooms
>
> >= 3... etc
>
> Or by chaining your filters on your Model class (http://
> code.google.com/appengine/docs/python/datastore/
> queryclass.html#Query_filter):
>
> eg: House.filter('cost >', 5000).filter('cost <',
> 15).filter('rooms >= ', 3)... etc
Neither of these will work; you can't use inequality filters on more
than one property.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Exploding Indexes - Update?
In general, we do not release ETAs for features. Alfred's enhancements are coming in multiple small feature releases over the coming months. It'd be more practical to post what's causing an exploding index, as there are probably ways to design around it and still meet your needs. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Mon, Sep 20, 2010 at 10:49 AM, DevShop wrote: > At the Google IO Conference in May, Alfred Fuller talked about his > solution for the exploding indexes problem. > > I cannot deploy my app due to an exploding index, just seeding the > database with initial data triggers the problem. > > Any updates as to when the solution will be deployed by the App Engine > team? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Entities are created without asking for it while Unit testing
No, the datastore should be cleared on each test run. What do your setup and
teardown methods look like?
--
Ikai Lan
Developer Programs Engineer, Google App Engine
Blogger: http://googleappengine.blogspot.com
Reddit: http://www.reddit.com/r/appengine
Twitter: http://twitter.com/app_engine
On Mon, Sep 20, 2010 at 6:26 AM, Killian wrote:
> Ok, so after a few hours of going crazy, I finally found out what was
> going on I think ! :-)
>
> As it turns out, when JUNIT is run, a second "appengine-generated"
> folder instance is created at the root of the project with a parallel
> datastore.
> So, the good news is that the solution is simple, I simply need to
> delete these two directories each time. However, just for the sake, my
> question now is, was that intentionally done? Or is it a setting I
> need to place somewhere in order for the Junit tests to use the same
> datastore as the application?
>
>
>
> On Sep 19, 2:43 pm, Killian wrote:
> > Hi Lads,
> >
> > This is something has been driving me mad for the last few days. It's
> > the weirdest issue I've ever come across app engine since I started
> > using it a while back.
> >
> > In a nutshell, what' happening is that when I run the simple code
> > below, which is supposed to get an object by Id within the datastore,
> > if it doesn't find it, it simply creates an new entity of this kind
> > and returns it (and of course later on makes it persistent).
> >
> > Now this code works perfectly when I'm running the app, however, when
> > I'm unit testing it, a LocaleCache object is always returned from the
> > datastore even if no entity was ever created. I tried so many
> > different debugging ideas to try and figure out what was going on and
> > really I don't know. Even if I delete the datastore file before
> > running the tests, although no LocaleCache entity was ever created or
> > stored, when I call " pm.getObjectById(LocaleCache.class, cacheKey);"
> > a LocaleCache object is always returned.
> > This is so weird ! And as I said, the strangest is that it only
> > happens when the code is run as a unit test.
> >
> > Any ideas of suggestions of what is going on would be really
> > appreciated as this issue is slowing us down a lot.
> >
> > Here is the sample code I'm referring to.
> >
> > Thanks :-)
> >
> > private LocaleCache getCache() {
> > LocaleCache cache =null;
> > Key cacheKey = keyFactory.createLocaleCacheKey(locale,
> > rightNow, range);
> > try {
> > cache = pm.getObjectById(LocaleCache.class, cacheKey);
> > } catch (JDOObjectNotFoundException e) {
> > cache = new LocaleCache();
> > cache.setUpCache(range);
> > cache.setKey(cacheKey);
> > }
> > return cache;
> > }
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Exploding Indexes - Update?
At the Google IO Conference in May, Alfred Fuller talked about his solution for the exploding indexes problem. I cannot deploy my app due to an exploding index, just seeding the database with initial data triggers the problem. Any updates as to when the solution will be deployed by the App Engine team? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Request for more than 10 app limit
Can you fill out this form? http://code.google.com/support/bin/request.py?contact_type=AppEngineBillingSupport It'd be helpful if you can explain in that link your need. On Sun, Sep 19, 2010 at 4:02 AM, tomodian wrote: > Hi. I've already reached the limit of 10 apps per account. > Would you please raise this? > > Regards. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] One to may relation ship problem
The error is here in the stack trace:
javax.jdo.JDOFatalUserException: Error in meta-data for
com.moviediary.datastore.Movie.actresses: Class
com.moviediary.datastore.Movie has multiple relationship fields of
type com.moviediary.datastore.Person: actresses and actor. This is
not yet supported.
You can only have one list with multiple child classes. Instead, store Lists
of keys.
On Sat, Sep 18, 2010 at 9:47 AM, laden wrote:
> Hi All,
>
> I am Nutan developing an application using Google App Engine and Java.
>
>
> @PersistenceCapable(identityType = IdentityType.APPLICATION)
> public class Movie {
>
>@PrimaryKey
>@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
>private Key movieId;
>
>@Persistent
>private String movieName;
>
>@Persistent
>private Language language;
>
>@Persistent(mappedBy="movie")
>private List actor;
>
> @Persistent(mappedBy="movie")
>private List actresses;
> }
>
> This is Movie data class where actor is a list.
> Where person class is below
>
> @PersistenceCapable(identityType = IdentityType.APPLICATION)
> public class Person {
>
>@PrimaryKey
>@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
> private Key personId;
>
>@Persistent
>private String personName;
>
>@Persistent
>private String gender;
>
>@Persistent
>private int categoryId;
>
>@Persistent
>private Movie movie;
> }
>
> But when I'm trying to save movie using the following code
>
>Country c = new Country();
>c.setCountryName("India");
>Movie m = new Movie();
>m.setMovieName("Athadu");
>m.setCountry(c);
>m.setRating("5");
>Person p = new Person();
>p.setPersonName("Mahesh Babu");
>p.setGender("Male");
>ArrayList actors = new ArrayList();
>actors.add(p);
>m.setActor(actors);
>PersistenceManager pm = PMF.get().getPersistenceManager();
>try {
>pm.makePersistent(m);
>} finally {
>pm.close();
>}
>
> The following error is coming.
> javax.jdo.JDOFatalUserException: Error in meta-data for
> com.moviediary.datastore.Movie.actresses: Class
> com.moviediary.datastore.Movie has multiple relationship fields of
> type com.moviediary.datastore.Person: actresses and actor. This is
> not yet supported.
>at
>
> org.datanucleus.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:
> 354)
>at
>
> org.datanucleus.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:
> 674)
>at
>
> org.datanucleus.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:
> 694)
>at com.moviediary.MovieDiary2Servlet.doGet(MovieDiary2Servlet.java:
> 32)
>at javax.servlet.http.HttpServlet.service(HttpServlet.java:693)
>at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
>at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:
> 511)
>at org.mortbay.jetty.servlet.ServletHandler
> $CachedChain.doFilter(ServletHandler.java:1166)
>at
>
> com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:
> 58)
>at org.mortbay.jetty.servlet.ServletHandler
> $CachedChain.doFilter(ServletHandler.java:1157)
>at
>
> com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:
> 43)
>at org.mortbay.jetty.servlet.ServletHandler
> $CachedChain.doFilter(ServletHandler.java:1157)
>at
>
> com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:
> 122)
>at org.mortbay.jetty.servlet.ServletHandler
> $CachedChain.doFilter(ServletHandler.java:1157)
>at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:
> 388)
>at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:
> 216)
>at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:
> 182)
>at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:
> 765)
>at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:
> 418)
>at
>
> com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:
> 70)
>at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:
> 152)
>at com.google.appengine.tools.development.JettyContainerService
> $ApiProxyHandler.handle(JettyContainerService.java:349)
>at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:
> 152)
>at org.mortbay.jetty.Server.handle(Server.java:326)
>at
> org.mortbay.jetty.HttpConne
Re: [google-appengine] Re: JDO query cursor usage questions
Yes, storing the cursor locally would be how you do it. If you weren't using GWT, you'd be rendering a link to the next page with the start cursor in the body of the HTML document, which would be functionally equivalent. If you used ranges and offsets, you'd have to store page state anyway client side. On Sun, Sep 19, 2010 at 8:38 AM, Rick Horowitz wrote: > I'm using a GWT UI. Is the recommended best practice to cache already > received results in the client, and track the currently displayed page > in the client, thus allowing to use the cached results when the user > is displaying a page already retrieved from the server and going to > the server only when the user pages forward? > > On Sep 17, 9:39 pm, Robert Kluin wrote: > > The cursor works sort of like a like a pointer, or a bookmark, in the > > index. When you run the query it advances the pointer forwards. It > > does not simply operate via offsets on the result set. Looks like the > > range specifies how many records you want back, i.e. how far you want > > it to advance the cursor. > > > > The current implementation of cursors do not allow you to run > > backwards. There was talk about cursors supporting that feature, but > > I have not heard about it six Google IO. > > > > Robert > > > > > > > > On Fri, Sep 17, 2010 at 16:57, Rick Horowitz wrote: > > > I'm finding the JDO cursor usage documentation a bit confusing. > > > According to the docs athttp:// > code.google.com/appengine/docs/java/datastore/queriesandindexe... > > > > > To get the initial page of results: > > > > > query.setRange(0, 20); // which makes sense > > > > > Then to get the second page of results, it again uses: > > > > > query.setRange(0, 20); // which is confusing. Shouldn't it be > > > query.setRange(20, 40) ? > > > > > In addition, does the cursor allow one to specify the previous page to > > > enable support for a "Previous Page" button in the user interface? > > > > > Thanks. Any clarification would be very much appreciated. > > > > > Rick > > > > > -- > > > You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > > > To post to this group, send email to google-appengine@googlegroups.com > . > > > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > > > For more options, visit this group athttp:// > groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Strange problem with user ids involving gmail.com
I've never heard of this issue before. We need the @gmail.com portion in
case folks log in with their Google Apps accounts.
It might be a bit more inconvenient, but can you store the user ID instead?
UserProperty should probably be deprecated in favor of this - we considered
forward migrating this but were afraid it would break backwards
compatibility.
On Fri, Sep 17, 2010 at 4:59 PM, gwstuff wrote:
> Hello,
>
> We have been facing a really strange problem involving the
> UserProperty() property. We have an entity type PR:
>
> class PR(db.Model):
>user = db.UserProperty()
>...
>
> The way we retrieve these entities is through queries of this kind:
>
> pr = PR.gql('WHERE user=:1',current_user)
>
> ...where current_user is fetched using the call
> users.get_current_user()
>
> This works 99% of the time, but every now and then a user can't get in
> even though he has a PR record with his user tacked to it. AFAIK, this
> only happens to people with @gmail.com addresses. Further
> investigation revealed that this is because the property in the
> database has its email() field set to the user id without
> '@gmail.com'. So USER(email='x.y') == USER('email='x...@gmail.com')
> evaluates to False.
>
> The funny thing is that even when we reset all such addresses and add
> the @gmail.com suffix, this suffix disappears periodically, and the
> problem comes back and locks these people out. We do not manually
> construct user objects at any time - we always call get_current_user()
> so our hope was that whatever the convention is would stay consistent
> across all of our operations.
>
> Has anyone faced this issue, or either way, have any suggestions?
>
> Sincerely,
> Z
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: DB lmitations: how to use multiple criteria on many columns? URGENT!
I believe you can do this in two ways:
Either using GQL (see:
http://code.google.com/appengine/docs/python/datastore/gqlreference.html)
eg: SELECT * FROM House WHERE cost > 5000 AND cost < 15 AND rooms
>= 3... etc
Or by chaining your filters on your Model class (http://
code.google.com/appengine/docs/python/datastore/
queryclass.html#Query_filter):
eg: House.filter('cost >', 5000).filter('cost <',
15).filter('rooms >= ', 3)... etc
Note you can't use OR in GQL.
P.S. Putting URGENT in your message title generally guarantees a slow
response on message boards... :-)
On Sep 19, 9:11 am, "GAE-framework.googlecode.com"
wrote:
> Hillo GAE developers!
>
> I have a serious situation with datastore. I need select data from
> model with multiple criteria. I have a House model, where exists
> fields: cost, rooms, floor. I need filter all houses, where cost >
> 5 and < 15, and also rooms > = 3 and < 5, and also floor >= 1
> and < 3.
>
> How can I select this dataset? As I see, the Datastore says that I can
> use filter criteria with >, <, >=, <= and != only on one column. But I
> reallly need use this rules on multiple columns.
>
> Please, help me!!!
> How to implement my task with Datastore limitations?
>
> Thank you. I really need your help.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: DB lmitations: how to use multiple criteria on many columns? URGENT!
I'm assuming that rooms and floor are integers, that is, you want the entries which have rooms=3 and 4, and floor = 1 and 2. With this assumption you one can construct a query which only have one inequality: SELECT * FROM house WHERE cost > 5 AND cost < 15 AND rooms IN (3, 4) AND floor IN (1, 2) Worth noting is that this query is actually executed as 4 queries (one for each combination of IN-parameters). Apart from the additional performance, AppEngine limits the number of "subqueries" like this to 25 (if I remember correctly). So you can't scale this to any number of rooms/floors if you need to. On 19 Sep, 08:11, "GAE-framework.googlecode.com" wrote: > Hillo GAE developers! > > I have a serious situation with datastore. I need select data from > model with multiple criteria. I have a House model, where exists > fields: cost, rooms, floor. I need filter all houses, where cost > > 5 and < 15, and also rooms > = 3 and < 5, and also floor >= 1 > and < 3. > > How can I select this dataset? As I see, the Datastore says that I can > use filter criteria with >, <, >=, <= and != only on one column. But I > reallly need use this rules on multiple columns. > > Please, help me!!! > How to implement my task with Datastore limitations? > > Thank you. I really need your help. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Problem with django? or appengine
I'm using django 1.1 via use_library and today I'm getting lot of
DedadlineExcedded Errors (or simply Server error without any
traceback). Not always, only from time to time - there are periods of
good work. Here is the example of traceback - not sure what to do
about this:
:
Traceback (most recent call last):
File "/base/data/home/apps/bike-reg/230.344933350703321288/
djangomain.py", line 71, in
main()
File "/base/data/home/apps/bike-reg/230.344933350703321288/
djangomain.py", line 68, in main
util.run_wsgi_app(application)
File "/base/python_runtime/python_lib/versions/1/google/appengine/
ext/webapp/util.py", line 97, in run_wsgi_app
run_bare_wsgi_app(add_wsgi_middleware(application))
File "/base/python_runtime/python_lib/versions/1/google/appengine/
ext/webapp/util.py", line 115, in run_bare_wsgi_app
result = application(env, _start_response)
File "/base/python_runtime/python_lib/versions/third_party/
django-1.1/django/core/handlers/wsgi.py", line 241, in __call__
response = self.get_response(request)
File "/base/python_runtime/python_lib/versions/third_party/
django-1.1/django/core/handlers/base.py", line 133, in get_response
receivers =
signals.got_request_exception.send(sender=self.__class__,
request=request)
File "/base/python_runtime/python_lib/versions/third_party/
django-1.1/django/dispatch/dispatcher.py", line 166, in send
response = receiver(signal=self, sender=sender, **named)
File "/base/data/home/apps/bike-reg/230.344933350703321288/
djangomain.py", line 58, in log_exception
logging.exception('Exception in request: %s: %s', cls.__name__,
err)
File "/base/python_runtime/python_dist/lib/python2.5/logging/
__init__.py", line 1301, in exception
apply(error, (msg,)+args, {'exc_info': 1})
File "/base/python_runtime/python_dist/lib/python2.5/logging/
__init__.py", line 1294, in error
apply(root.error, (msg,)+args, kwargs)
File "/base/python_runtime/python_dist/lib/python2.5/logging/
__init__.py", line 1015, in error
apply(self._log, (ERROR, msg, args), kwargs)
File "/base/python_runtime/python_dist/lib/python2.5/logging/
__init__.py", line 1101, in _log
self.handle(record)
File "/base/python_runtime/python_dist/lib/python2.5/logging/
__init__.py", line , in handle
self.callHandlers(record)
File "/base/python_runtime/python_dist/lib/python2.5/logging/
__init__.py", line 1148, in callHandlers
hdlr.handle(record)
File "/base/python_runtime/python_dist/lib/python2.5/logging/
__init__.py", line 655, in handle
self.emit(record)
File "/base/python_runtime/python_lib/versions/1/google/appengine/
api/app_logging.py", line 70, in emit
message = self._AppLogsMessage(record)
File "/base/python_runtime/python_lib/versions/1/google/appengine/
api/app_logging.py", line 83, in _AppLogsMessage
message = self.format(record).replace("\n", NEWLINE_REPLACEMENT)
File "/base/python_runtime/python_dist/lib/python2.5/logging/
__init__.py", line 630, in format
return fmt.format(record)
File "/base/python_runtime/python_dist/lib/python2.5/logging/
__init__.py", line 426, in format
record.exc_text = self.formatException(record.exc_info)
File "/base/python_runtime/python_dist/lib/python2.5/logging/
__init__.py", line 398, in formatException
traceback.print_exception(ei[0], ei[1], ei[2], None, sio)
File "/base/python_runtime/python_dist/lib/python2.5/traceback.py",
line 125, in print_exception
print_tb(tb, limit, file)
File "/base/python_runtime/python_dist/lib/python2.5/traceback.py",
line 69, in print_tb
line = linecache.getline(filename, lineno, f.f_globals)
File "/base/python_runtime/python_dist/lib/python2.5/linecache.py",
line 14, in getline
lines = getlines(filename, module_globals)
File "/base/python_runtime/python_dist/lib/python2.5/linecache.py",
line 40, in getlines
return updatecache(filename, module_globals)
File "/base/python_runtime/python_dist/lib/python2.5/linecache.py",
line 129, in updatecache
lines = fp.readlines()
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Multiple Domains for one app, even if you don't own the domain, or if other domain is already registered with Google Apps
I've been playing around with this kind of scenario as well. As far as I can tell you don't have to do anything from your side, except give them your app engine id. Your client needs to have a Google Apps account for their domain name. They can then add your app engine application to their Google Apps account and point any URL for that domain to your app engine application. They would do this by: 1. Logging into their Google Apps account 2. Click on "Add more services" 3. Enter app id on the Google App Engine section under "Other services" I am not 100% sure the above is true, but it seems to have worked for one of my Google Apps domains. On Sep 17, 10:05 pm, NickFranceschina wrote: > I promise I have been searching... but I haven't been unable to find > the correct answer. Thus I turn to you fine folks. > > So lets say I have my super fancy new killer app-engine app: > http://mykillerapp.com > which is my custom domain pointed to: > http://mykillerapp.appspot.com > > and then because I'm a genius salesman, I convince Coca-Cola that they > need my app on their site. They write me a huge check with lots of > zeros and say "OK, lets set it up athttp://awesome.coca-cola.com"; > > how can I make that happen? the only way i can see is to add coca- > cola.com to my google apps for domains account (verifying it with > CNAME or .HTML file)... but yeah, Coke isn't gonna let me do that cuz > it's obviously their domain. > > or what if I sell it to JoeSchmoe.com who already has a google apps > account... I can't add his domain to my google apps because google > tells me "this domain is already registered" (or something like that) > > SOOO... is there a way for me to only add awesome.coca-cola.com to my > google apps domain so I can link the app to it? or setup > awesome.JoeSchmoe.com if JoeSchmoe.com is already a google apps > customer? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Vacuum index option in appcfg.cmd
U can create a dummy python app.yaml and run appcfg.py vacuum_indexes myapp/ for a java app. This should delete all created indexes. On Sep 19, 8:57 pm, todu wrote: > The command is . appcfg.py vacuum_indexes myapp/ for python works > well. > > If you are using complicated Java then maybe Larry Ellison can help :) > > On Sep 19, 5:45 am, NM wrote: > > > > > Hi, I have two indexes in an error state. I got the message to vacuum > > the index at the time of deploying the app from eclipse. However, the > > appcfg.cmd tool does not seem to have the vacuum_indexes option. How > > do I solve this issue? > > > Thanks, > > Nirmallya -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: application size limit
The limit is 3000 files, and it's documented at http://code.google.com/appengine/docs/java/runtime.html#Quotas_and_Limits On Sep 18, 5:00 am, Michael wrote: > Dear All: > > It looks like there is a certain hard limit in the number of files in > appengine applications at around 1000 files. I added an icon library > to my application and got this: > > --- > update: > [java] Reading application configuration data... > [java] Sep 18, 2010 10:43:34 AM > com.google.apphosting.utils.config.AppEngineWebXmlReader > readAppEngineWebXml > [java] INFO: Successfully processed war/WEB-INF/appengine-web.xml > [java] 2010-09-18 10:43:34.342:INFO::Logging to STDERR via > org.mortbay.log.StdErrLog > [java] Sep 18, 2010 10:43:35 AM > com.google.apphosting.utils.config.AbstractConfigXmlReader > readConfigXml > [java] INFO: Successfully processed war/WEB-INF/web.xml > [java] Sep 18, 2010 10:43:35 AM > com.google.apphosting.utils.config.AbstractConfigXmlReader > readConfigXml > [java] INFO: Successfully processed war/WEB-INF/cron.xml > [java] Beginning server interaction for gpsessentials... > [java] 0% Creating staging directory > [java] 5% Scanning for jsp files. > [java] 20% Scanning files on local disk. > [java] 25% Scanned 250 files. > [java] 28% Scanned 500 files. > [java] 31% Scanned 750 files. > [java] 33% Scanned 1000 files. > [java] 34% Scanned 1250 files. > [java] 35% Initiating update. > [java] 36% Cloning 35 static files. > [java] 37% Cloning 1319 application files. > [java] 37% Cloned 100 files. > [java] 37% Cloned 200 files. > [java] 37% Cloned 300 files. > [java] 37% Cloned 400 files. > [java] 37% Cloned 500 files. > [java] 37% Cloned 600 files. > [java] 37% Cloned 700 files. > [java] 37% Cloned 800 files. > [java] 37% Cloned 900 files. > [java] 37% Cloned 1000 files. > [java] 37% Cloned 1100 files. > [java] 37% Cloned 1200 files. > [java] 37% Cloned 1300 files. > [java] 40% Uploading 12 files. > [java] 52% Uploaded 3 files. > [java] 61% Uploaded 6 files. > [java] 68% Uploaded 9 files. > [java] 73% Uploaded 12 files. > [java] 77% Initializing precompilation... > [java] 90% Deploying new version. > [java] 95% Rolling back the update. > [java] > [java] java.io.IOException: Error posting to > URL:https://appengine.google.com/api/appversion/deploy?app_id=gpsessentia... > [java] 400 Bad Request > [java] > [java] Client Error (400) > [java] The request is invalid for an unspecified reason. > [java] > [java] Unable to update app: Error posting to > URL:https://appengine.google.com/api/appversion/deploy?app_id=gpsessentia... > [java] 400 Bad Request > [java] > [java] Client Error (400) > [java] The request is invalid for an unspecified reason. > [java] > [java] Please see the logs [/var/folders/UR/URWJm8otEueOL91TvJPa1E > +++TI/-Tmp-/appcfg5307073489146939726.log] for further information. > > BUILD FAILED > /Users/ms/Development/Projects/GpsEssentials/ShareServer/build.xml:81: > The following error occurred while executing this line: > /Applications/Development/eclipse/plugins/ > com.google.appengine.eclipse.sdkbundle.1.3.7_1.3.7.v201008311427/ > appengine-java-sdk-1.3.7/config/user/ant-macros.xml:57: Java returned: > 1 > > Total time: 4 minutes 25 seconds > Michael:ShareServer ms$ cat /var/folders/UR/URWJm8otEueOL91TvJPa1E++ > +TI/-Tmp-/appcfg5307073489146939726.log > Unable to update: > java.io.IOException: Error posting to > URL:https://appengine.google.com/api/appversion/deploy?app_id=gpsessentia... > 400 Bad Request > > Client Error (400) > The request is invalid for an unspecified reason. > > at > com.google.appengine.tools.admin.ServerConnection.send(ServerConnection.jav a: > 149) > at > com.google.appengine.tools.admin.ServerConnection.post(ServerConnection.jav a: > 82) > at > com.google.appengine.tools.admin.AppVersionUpload.send(AppVersionUpload.jav a: > 582) > at > com.google.appengine.tools.admin.AppVersionUpload.deploy(AppVersionUpload.j > ava: > 541) > at > com.google.appengine.tools.admin.AppVersionUpload.commit(AppVersionUpload.j > ava: > 504) > at > com.google.appengine.tools.admin.AppVersionUpload.doUpload(AppVersionUpload > .java: > 129) > at > com.google.appengine.tools.admin.AppAdminImpl.update(AppAdminImpl.java: > 56) > at com.google.appengine.tools.admin.AppCfg > $UpdateAction.execute(AppCfg.java:547) > at com.google.appengine.tools.admin.AppCfg.(AppCfg.java:138) > at com.google.appengine.tools.admin.AppCfg.(AppCfg.java:61) > at com.google.appengine.tools.admin.AppCfg.main(AppCfg.java:57) > com.google.appengine.tools.admin.AdminException: Unable to update app: > Error posting to > URL:https://appengine.google.com/api/appversion/depl
[google-appengine] Index Stuck Building for 60 Hours
I created a new index in my index.yaml on Friday night at around 10PM EST. It is still hung up in "Building" state as of this moment (2 and a half days later). The Datastore Indexes page just shows the Status as "Building" with no "Queued", "Running", "Completed" or "Total counts. Can someone at Google please make sure this index finishes building. I don't want to delete this index, vacuum it, then recreate and hope that it finishes building over the next 24 hours (which is how long the last index took to successfully build). My app-id is "me-finance". Thank you. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Static resources not updated?
My minds was stuck on a: myscript-x.y.z.js naming scheme, the solution
you suggest is a off course much simpler to administrate.
Thanks again.
Cheers
Tonny
On Sep 13, 8:56 pm, "Ikai Lan (Google)"
wrote:
> It's not trivial, but it's not hard, either. Here's one approach (of many):
>
> 1. Create a deploy script to wrap the appcfg.py update
> 2. Pre deploy, your script create a Python file that declares a variable
> with the current time
> 3. In your javascript and CSS include statements in your HTML, modify the
> URL like this:
>
> /js/some.js?v={{ GLOBAL_TIMESTAMP }}
>
> (ContextProcessors are key here if you are using Django templates)
>
> Just changing the URL with a meaningless parameter on a static asset should
> be enough to ensure that browsers and proxies force download a new version
> of your asset file.
>
>
>
> On Mon, Sep 13, 2010 at 12:53 AM, Tonny <12br...@gmail.com> wrote:
>
> > On Sep 9, 8:29 pm, "Ikai L (Google)" wrote:
> > > Are you using a YAML file configuration for static assets? There's a
> > place
> > > to set expires times. At any rate, next time this happens, take note of
> > the
> > > headers returned and post them.
> > I have this for the files in affected:
> > - url: /s/x.y
> > static_dir: static/my.library
>
> > where my.library contains javascript files.
>
> > I'll post the header the next time i see the issue.
>
> > > As a best practice, in your URLs to your static assets, you should have
> > > cache busters since this'll also cover the case of folks with weird local
> > > browser cache settings.
> > Yeah, I know. So far I have gotten away with including the files
> > directly.
> > Since I'm writting "raw" Mootools based javascript with nothing but a
> > home brewed build system,
> > cache busting i not a trivial task - if someone reading this post can
> > bust this perception i would like
> > to hear :-).
>
> > > On Thu, Sep 9, 2010 at 12:49 AM, Tonny <12br...@gmail.com> wrote:
> > > > This could what was happening, since the new files was available after
> > > > 5-10 minuttes (as mentioned i used curl to look). I have no customized
> > > > expire times though.
>
> > > > On Sep 8, 10:26 pm, Peter Petrov wrote:
> > > > > This generally happens when you set a custom expiration time for
> > > > > static files. This causes Google's frontend servers to cache the
> > files
> > > > > for up to the specified time, and they don't update it even if you
> > > > > push a new version in the mean time. This is true both for
> > > > > *.appspot.com and for custom domains. You can take a look at the
> > HTTP
> > > > > headers returned by your Google Frontend server, e.g. using Firebug,
> > > > > to see when the next refresh will happen.
>
> > > > > If you don't set custom expiration for static files, the frontend
> > > > > servers seem to fetch the resources every time (or at least very
> > > > > frequently). On the other hand, the browsers' caching behaviour is
> > > > > then undefined. In any case, you should use cache-busters.
>
> > > > > On Sep 8, 4:26 pm, Tonny <12br...@gmail.com> wrote:
>
> > > > > > I did and uploading 10 minuttes, but my static resources has not
> > > > > > changed when i load them through the webserver?
>
> > > > > > Does anybody have similar problems currently?
>
> > > > > > Regards
> > > > > > Tonny
>
> > > > --
> > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "Google App Engine" group.
> > > > To post to this group, send email to google-appengine@googlegroups.com
> > .
> > > > To unsubscribe from this group, send email to
> > > > google-appengine+unsubscr...@googlegroups.com > > > e...@googlegroups.com> > e...@googlegroups.com>
> > > > .
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/google-appengine?hl=en.
>
> > > --
> > > Ikai Lan
> > > Developer Programs Engineer, Google App Engine
> > > Blog:http://googleappengine.blogspot.com
> > > Twitter:http://twitter.com/app_engine
> > > Reddit:http://www.reddit.com/r/appengine
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Multiple Domains for one app, even if you don't own the domain, or if other domain is already registered with Google Apps
Hi Nick, What you'd do is have the company in question sign up for Google Apps on the main domain (even if they don't activate any of the services on it), then add your App Engine app on a subdomain from within the Apps control panel. -Nick Johnson On Fri, Sep 17, 2010 at 8:05 PM, NickFranceschina < nickfrancesch...@gmail.com> wrote: > I promise I have been searching... but I haven't been unable to find > the correct answer. Thus I turn to you fine folks. > > So lets say I have my super fancy new killer app-engine app: > http://mykillerapp.com > which is my custom domain pointed to: > http://mykillerapp.appspot.com > > > and then because I'm a genius salesman, I convince Coca-Cola that they > need my app on their site. They write me a huge check with lots of > zeros and say "OK, lets set it up at http://awesome.coca-cola.com"; > > > how can I make that happen? the only way i can see is to add coca- > cola.com to my google apps for domains account (verifying it with > CNAME or .HTML file)... but yeah, Coke isn't gonna let me do that cuz > it's obviously their domain. > > or what if I sell it to JoeSchmoe.com who already has a google apps > account... I can't add his domain to my google apps because google > tells me "this domain is already registered" (or something like that) > > > SOOO... is there a way for me to only add awesome.coca-cola.com to my > google apps domain so I can link the app to it? or setup > awesome.JoeSchmoe.com if JoeSchmoe.com is already a google apps > customer? > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: What's a better choice - updating a record or overwriting?
Alon On Sep 20, 8:07 pm, alon wrote: > actually i read the basic rule of app engine is that PUT is the most > resource extensive job you can do and should avoid it at all cause :) > Correct. (well actually a big query is way more expensive than putting one entity ;-) > Also correct me if im wrong, but in order to update you need to query > to get the object itself, how can you update an object without pulling > the data before hand? or are you managing your keys? > There is no update in appengine, you get() or put() . An update is get(), modify properties, put() writing the whole entity on each put. And yes you would need to manage keys if you are just doing puts, hence my question "do you care about old data" > Come to think of it. overwrite will require the same. im guessing it > wont make a difference in performance. > overwrite is just a put, but get + put is more expensive. Especially if you don't care about the old data. T > On Sep 19, 2:41 pm, nischalshetty wrote: > > > > > > > > > Ah, that's a good point. Ok, so if I don't care about the existing > > data, I can simple overwrite them without worrying. I hope its the > > case if say I have like 50 million entities? > > > -N > > > On Sep 19, 4:16 pm, Tim Hoffman wrote: > > > > Hi > > > > That all depends on if there is any data in the old entity that you > > > want to copy to the overwritten > > > entity. If not it will be cheaper to do a put rather than a get > > > followed by a put. > > > > T > > > > On Sep 19, 2:03 pm, nischalshetty wrote: > > > > > When I started out, I made a few of my entities without the appengine > > > > "Key" as primary key i.e. made them with numeric primary keys. > > > > > Then I moved on to using "Key" as a primary key for my entities. In > > > > the documentation it says if we try to persist an entity with an > > > > already existing key, it'll overwrite the existing record. > > > > > So, my question is, to update my entities, should I just overwrite > > > > them or should I do the traditional, get entity, modify it and then > > > > call the update method to persist it? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: What's a better choice - updating a record or overwriting?
actually i read the basic rule of app engine is that PUT is the most resource extensive job you can do and should avoid it at all cause :) Also correct me if im wrong, but in order to update you need to query to get the object itself, how can you update an object without pulling the data before hand? or are you managing your keys? Come to think of it. overwrite will require the same. im guessing it wont make a difference in performance. On Sep 19, 2:41 pm, nischalshetty wrote: > Ah, that's a good point. Ok, so if I don't care about the existing > data, I can simple overwrite them without worrying. I hope its the > case if say I have like 50 million entities? > > -N > > On Sep 19, 4:16 pm, Tim Hoffman wrote: > > > > > Hi > > > That all depends on if there is any data in the old entity that you > > want to copy to the overwritten > > entity. If not it will be cheaper to do a put rather than a get > > followed by a put. > > > T > > > On Sep 19, 2:03 pm, nischalshetty wrote: > > > > When I started out, I made a few of my entities without the appengine > > > "Key" as primary key i.e. made them with numeric primary keys. > > > > Then I moved on to using "Key" as a primary key for my entities. In > > > the documentation it says if we try to persist an entity with an > > > already existing key, it'll overwrite the existing record. > > > > So, my question is, to update my entities, should I just overwrite > > > them or should I do the traditional, get entity, modify it and then > > > call the update method to persist it? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 500 Server Error
please provide more details. its hard to tell whats the issue. Java,python? local server or remote? do you have some exception you can paste? On Sep 18, 6:10 am, damian wrote: > I had functioning web server. However, for last two-three days > whenever I am trying to get to my web server I am getting 500 Server > Error. (Error: Server Error). What I need to do to reset this or get > rid of this error? > > Thank you > Damian -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Multiple Domains for one app, even if you don't own the domain, or if other domain is already registered with Google Apps
Google App Engine uses google apps for hosting your app. it does not let you host subdomains. What you can do is try to register mydomain.cocacola.com as a google app for your domain. i just tried it and it works fine. you can register subdomains under domains that are already signed up with google apps. Then do the standard procedure. maybe that will work? On Sep 17, 9:05 pm, NickFranceschina wrote: > I promise I have been searching... but I haven't been unable to find > the correct answer. Thus I turn to you fine folks. > > So lets say I have my super fancy new killer app-engine app: > http://mykillerapp.com > which is my custom domain pointed to: > http://mykillerapp.appspot.com > > and then because I'm a genius salesman, I convince Coca-Cola that they > need my app on their site. They write me a huge check with lots of > zeros and say "OK, lets set it up athttp://awesome.coca-cola.com"; > > how can I make that happen? the only way i can see is to add coca- > cola.com to my google apps for domains account (verifying it with > CNAME or .HTML file)... but yeah, Coke isn't gonna let me do that cuz > it's obviously their domain. > > or what if I sell it to JoeSchmoe.com who already has a google apps > account... I can't add his domain to my google apps because google > tells me "this domain is already registered" (or something like that) > > SOOO... is there a way for me to only add awesome.coca-cola.com to my > google apps domain so I can link the app to it? or setup > awesome.JoeSchmoe.com if JoeSchmoe.com is already a google apps > customer? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Very bad site performance for the last week
The performance of www.simpleeditions.com (application: simpleeditionsapp) has been very bad for the last week. We're getting DeadlineExceededErrors often now, saying it took 33 seconds before timing out. The datastore queries are not complicated and they're not fetching a lot of data. A week ago it was much better. It doesn't even seem as if it's datastore related. Here are some example tracebacks: --- : Traceback (most recent call last): File "/base/data/home/apps/simpleeditionsapp/stable. 344781513275803132/app.py", line 41, in import simpleeditions.urls File "/base/data/home/apps/simpleeditionsapp/stable. 344781513275803132/simpleeditions/urls.py", line 21, in from simpleeditions import view File "/base/data/home/apps/simpleeditionsapp/stable. 344781513275803132/simpleeditions/view.py", line 30, in from django.utils import simplejson File "/base/python_runtime/python_lib/versions/third_party/ django-1.1/django/utils/simplejson/__init__.py", line 140, in from django.utils.simplejson.encoder import JSONEncoder File "/base/python_runtime/python_lib/versions/third_party/ django-1.1/django/utils/simplejson/encoder.py", line 2, in --- : Traceback (most recent call last): File "/base/data/home/apps/simpleeditionsapp/stable. 344781513275803132/app.py", line 41, in import simpleeditions.urls File "/base/data/home/apps/simpleeditionsapp/stable. 344781513275803132/simpleeditions/urls.py", line 21, in from simpleeditions import view File "/base/data/home/apps/simpleeditionsapp/stable. 344781513275803132/simpleeditions/view.py", line 21, in import base64 --- : Traceback (most recent call last): File "/base/data/home/apps/simpleeditionsapp/stable. 344781513275803132/app.py", line 41, in import simpleeditions.urls File "/base/data/home/apps/simpleeditionsapp/stable. 344781513275803132/simpleeditions/urls.py", line 21, in from simpleeditions import view File "/base/data/home/apps/simpleeditionsapp/stable. 344781513275803132/simpleeditions/view.py", line 30, in from django.utils import simplejson File "/base/python_runtime/python_lib/versions/third_party/ django-1.1/django/__init__.py", line 1, in VERSION = (1, 1, 0, 'final', 0) --- This is really putting us off from using Google App Engine for serious projects... it just isn't usable at all for a site with a user base, they would just leave a site behaving like this. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Entities are created without asking for it while Unit testing
Ok, so after a few hours of going crazy, I finally found out what was
going on I think ! :-)
As it turns out, when JUNIT is run, a second "appengine-generated"
folder instance is created at the root of the project with a parallel
datastore.
So, the good news is that the solution is simple, I simply need to
delete these two directories each time. However, just for the sake, my
question now is, was that intentionally done? Or is it a setting I
need to place somewhere in order for the Junit tests to use the same
datastore as the application?
On Sep 19, 2:43 pm, Killian wrote:
> Hi Lads,
>
> This is something has been driving me mad for the last few days. It's
> the weirdest issue I've ever come across app engine since I started
> using it a while back.
>
> In a nutshell, what' happening is that when I run the simple code
> below, which is supposed to get an object by Id within the datastore,
> if it doesn't find it, it simply creates an new entity of this kind
> and returns it (and of course later on makes it persistent).
>
> Now this code works perfectly when I'm running the app, however, when
> I'm unit testing it, a LocaleCache object is always returned from the
> datastore even if no entity was ever created. I tried so many
> different debugging ideas to try and figure out what was going on and
> really I don't know. Even if I delete the datastore file before
> running the tests, although no LocaleCache entity was ever created or
> stored, when I call " pm.getObjectById(LocaleCache.class, cacheKey);"
> a LocaleCache object is always returned.
> This is so weird ! And as I said, the strangest is that it only
> happens when the code is run as a unit test.
>
> Any ideas of suggestions of what is going on would be really
> appreciated as this issue is slowing us down a lot.
>
> Here is the sample code I'm referring to.
>
> Thanks :-)
>
> private LocaleCache getCache() {
> LocaleCache cache =null;
> Key cacheKey = keyFactory.createLocaleCacheKey(locale,
> rightNow, range);
> try {
> cache = pm.getObjectById(LocaleCache.class, cacheKey);
> } catch (JDOObjectNotFoundException e) {
> cache = new LocaleCache();
> cache.setUpCache(range);
> cache.setKey(cacheKey);
> }
> return cache;
> }
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: DB lmitations: how to use multiple criteria on many columns? URGENT!
You will need to do this within your application. So narrow down your initial dataset with a filter on one column, then use application logic to process the results to find records with your required values. Cheers Greg. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Cannot access Admin Interface on Localhost
Hello, on local server you have a fake authentication which allows simulation of admin login. If you open your front end with your browser, you should see a login box showing "t...@examle.com" as login name, with a "sign in as administrator" checkbox. You must sign in as admin if you want to see the pages you marked as restricted in your web.xml. No-login access is only for the local admin console (_ah/admin), but for anything you explicitly restrict you must sign in as administrator. Regards Lorenzo On Sep 18, 2:29 am, sbaechler wrote: > Hi > > I just set up AppEngine with the YouTube Direct App. When I run it and > try to access the admin interface on localhost I get a 403 FORBIDDEN > error. In the tutorial the guy said that there was no user > authentication on the local server. > The front end seems to work. The template is displayed correctly. > > The Log-message on the console: > 18.09.2010 00:11:24 > com.google.apphosting.utils.jetty.AppEngineAuthentication > $AppEngineUserRealm isUserInRole > INFO: Checking if principal is in role admin > > When I publish the app and try to access the admin interface online > there is a server error. > > I have a Google Apps account and the app is running under this > account. Could this be an issue? > > Best Regards > Simon -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Auto generated keys vs UUID
Hi,
You can use allocate_id_range (see
http://code.google.com/appengine/docs/python/datastore/functions.html) to
allocate IDs to cover the range that you imported, to prevent the datastore
automatically allocating any IDs in that range.
-Nick Johnson
2010/9/18 Jaganathan (ஜ௧நாதன்)
> Hi Nick and others
>
> Finally, I tried my app on the imported DB today. The import was done as
> per the technique mentioned in the previous mail.
>
> As expected, it threw the following error, when I initiated an action that
> creates a new entity:
>
> *the id allocated for a new entity was already in use, please try
> again*Traceback (most recent call last):
>
>
> [some trace truncated]
> File
> "/base/data/home/apps/notesonatree/1-5.344881683710780408/notesonatree/core/models.py",
> line 111, in create_note
> new_note.put()
> File
> "/base/python_runtime/python_lib/versions/1/google/appengine/ext/db/__init__.py",
> line 893, in put
> return datastore.Put(self._entity, rpc=rpc)
> File
> "/base/python_runtime/python_lib/versions/1/google/appengine/api/datastore.py",
> line 293, in Put
> raise _ToDatastoreError(err)
> InternalError: the id allocated for a new entity was already in use, please
> try again
>
>
> :D . Is there a way to work around this? Is this a work in progress?
>
> Also try to answer my previous mail if possible.
>
> Thanks
> Jagan
>
> 2010/9/13 Jaganathan (ஜ௧நாதன்)
>
> Hi Nick and others
>>
>> Thank you for your response.
>>
>> More inline below ...
>>
>> On Thu, Sep 2, 2010 at 8:17 PM, Nick Johnson (Google) <
>> nick.john...@google.com> wrote:
>>
>>> Hi Jagan,
>>>
>>> On Tue, Aug 31, 2010 at 9:46 AM, Jagan wrote:
>>>
Hi Google, Nick Johnson and others
Does Google recommend using UUID (UUID4 to be specific) as key names
for entities in GAE instead of auto generated numeric ids?
Does this hold good in all use cases in the universe?
>>>
>>> This depends entirely on your use-case. Unless you expect an extremely
>>> high insertion rate of new entities (eg, >=100QPS) with no natural key name,
>>> it's not necessary to use a UUID. UUIDs take more space than regular integer
>>> IDs.
>>>
>>>
And is this the reason why import export of auto generates IDs are not
yet supported in BulkUpload? Should I wait till this is supported or
should I refactor my code to use UUID key names?
>>>
>>> It's possible to do this using the bulkloader, but requires you to
>>> override a method to generate key names.
>>>
>>
>> Were you speaking about the old bulkloader? From looking at the code
>> at\
>> google\appengine\tools\bulkloader.py, I think that might not work,
>> because even if we return a Key constructed with the id, it is later doing
>> loader.create_entity with the key name. Here is the snippet that I am
>> relating to:
>>
>> Line 1260-1267:
>> for line_number, values in rows:
>> key = loader.generate_key(line_number, values)
>> if isinstance(key, datastore.Key):
>> parent = key.parent()
>> key = key.name()
>> else:
>> parent = None
>> entity = loader.create_entity(values, key_name=key, parent=parent)
>>
>>
>> Anyways, I tried the following technique in the new bulkloader:
>>
>> - kind: XYZ
>> connector: csv
>> connector_options:
>> property_map:
>> - property: __key__
>> external_name: key
>> *import_transform: transform.create_foreign_key('XYZ', True)*
>> export_transform: transform.key_id_or_name_as_string
>>
>> It is working. Is this good enough?
>>
>> But* *I have one general doubt here. I read in one forum that, with these
>> techniques, the *sequence number* (?) that GAE maintains for the next new
>> generated id wont be incremented / updated, and so when our app tries
>> to create a new entity, its auto generated id might clash. Is this true? Is
>> there any link that describes how auto increment ids work on App Engine?
>>
>>
>>>
>>>
Is there a demo page that compares between the bulk read/write
performance between entities that have 32-bit numeric ids versus xxx-
bit UUID key names?
>>>
>>> The performance should be roughly the same - but the space used will be
>>> significantly larger in the case of UUIDs.
>>>
>>> -Nick Johnson
>>>
>>>
Some home work that I did:
http://groups.google.com/group/google-appengine/browse_thread/thread/bc868376ffa37619/db9bda21c95ea806?lnk=gst&q=uuid#db9bda21c95ea806
http://groups.google.com/group/google-appengine/browse_thread/thread/a6fcce84837c34b/d4c9d5b3c7869222?lnk=gst&q=uuid#d4c9d5b3c7869222
http://groups.google.com/group/google-appengine/browse_thread/thread/7dedb7d65bdf4f/37f2733fd9f572b9?lnk=gst&q=uuid#37f2733fd9f572b9
Thanks
Jagan
--
You received this message because you are subscribed to the Google
Groups "Google App Engine" group.
To post to this group, send email to google-appeng...@g
