Re: [google-appengine] datastore with ~500 million rows?
If you consider for a moment that *all* of the rows for all of our applications are stored in one big logical BigTable, an extra 500 million is probably a drop in the bucket. On the other hand, GAE has a limit of 1MB for any single entity. 3.5TB / 500M rows is about 7MB per entity, so that's not going to work. Also realize that 3.5TB will over $500/mo just for storage. Jeff On Mon, Apr 19, 2010 at 11:59 AM, sunsu wrote: > Hi guys, > > I'm wondering if its possible for me to create a datastore on Google's > App engine that could feasibly grow to over 500 million rows > (entries). It would be all text (json blobs really), so I don't think > each entry would take up all that much space. I'm thinking if it were > to actually grow to be that big, It would be about 3.5 terabytes of > data (??). > > To give a little background: I'm thinking about making a game in which > the universe is made of a series of tilemaps. In order to create it at > the scale that I would like, I'll need to (eventually...because the > world would grow as people explored) store data for about 500 million > tilemaps :D. > > Would querying a datastore with 500 million rows even work relatively > quickly on Google's App Engine (assuming it was properly indexed)? > > Thanks for the input, > Sunsu > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Project name availability
i'm not from Google, but from my experience that's not the case, they are independent (for example, while there's a project on code.google.com for datanucleus-appengine there is no http://datanucleus-appengine.appspot.com/) regards 2010/4/19 Florent Georges > Hi, > > I tried to create a new project on Google App Engine, but the name I > chose was not available anymore. That surprised me a bit as this is a > rather uncommon name. Then I thought that maybe the name of projects > on Google Code were reserved automatically on the App Engine (I have a > project on Google Code with the same name). > > I dug into the help and the archives, but I did not find anything on > that subject. > > Is there any chance the name I am after is reserved because of the > namesake project on Google Code? > > Regards, > > -- > Florent Georges > http://fgeorges.org/ > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Indifference will be the downfall of mankind, but who cares?... http://blog.opsiland.info/ -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Pagination
On Apr 19, 8:27 pm, Ulrich wrote: > kamathln wrote: > Why don't you just generate a few links (something like page 1,2,3, last > page) in the first request and then generate additional links if a user > goes to page 2 or 3 (or last page), so you do not have to generate them > in the background with a task. /me slaps forehead. Yeah. That should do :-D May be my mind was thinking in terms of Ajax. I have no exact clue. As I said in my first post, this is a blurred idea. Or I can say the gist of the idea. I am not able to get my head around for exact implementation details, hence I posted here. > -Ulrich -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Sign in to Google App Engine Programmatically
Hi, I'm trying to log in to Google App Engine with my Google Account, and I was wondering if it's possible to do this programmatically. I want to retrieve all the App ID that I have in Google App Engine (https://appengine.google.com/) through my application (written in Java). What kind of authentication would be needed to do this? Can I use ClientLogin for this? Thanks!! -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Project name availability
Hi, I tried to create a new project on Google App Engine, but the name I chose was not available anymore. That surprised me a bit as this is a rather uncommon name. Then I thought that maybe the name of projects on Google Code were reserved automatically on the App Engine (I have a project on Google Code with the same name). I dug into the help and the archives, but I did not find anything on that subject. Is there any chance the name I am after is reserved because of the namesake project on Google Code? Regards, -- Florent Georges http://fgeorges.org/ -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] datastore with ~500 million rows?
Hi guys, I'm wondering if its possible for me to create a datastore on Google's App engine that could feasibly grow to over 500 million rows (entries). It would be all text (json blobs really), so I don't think each entry would take up all that much space. I'm thinking if it were to actually grow to be that big, It would be about 3.5 terabytes of data (??). To give a little background: I'm thinking about making a game in which the universe is made of a series of tilemaps. In order to create it at the scale that I would like, I'll need to (eventually...because the world would grow as people explored) store data for about 500 million tilemaps :D. Would querying a datastore with 500 million rows even work relatively quickly on Google's App Engine (assuming it was properly indexed)? Thanks for the input, Sunsu -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Python script Google Authentication for a GAE Application
Hi Everyone, I'm having a hard time figuring this out and I'm hoping someone can point me in the right direction for what to read. I've written an appengine app, a servlett, that will accept data from a get request, check if the user is signed in to a google account and then store the data from the request in the datastore (with a User as part of the model). All pretty standard. I'd like for a python script to be able to automate the data input (think returning the results of s...@home) but it's important that the python client be able to authenticate itself via a google account. I'm at a bit of a loss for where to even start... these seem like good places, but I don't even know enough to be sure which, if any, will work: http://code.google.com/apis/accounts/docs/OAuthForInstalledApps.html -- seems like the best shot but they say as a log in for 'one or more google service' which I'm not sure my app counts as.. http://code.google.com/appengine/docs/python/tools/uploadingdata.html -- I think this is not so good as I would have to give every user too lenient permissions for my app I've looked through the gdata source code as well trying to authenticate with a generic service, but I'm not sure if gdata authentication is universal enough to work with an arbitrary appengine app. Ideally if someone can get me started or point me at the correct place that would be great. I'd rather not do this over email or xmpp, but I'm considering those options. Thank you anyone for your help!, Michael -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Python Documentation Missing?
I was aware of that page but was looking for some more detailed API documentation for example: PyDocs for google.appengine.ext.db. My form data is not saving properly. What document would tell me about the API methods available and the expected input/output values to help me debug this? J On Mon, Apr 19, 2010 at 9:24 AM, Nick Johnson (Google) < nick.john...@google.com> wrote: > Hi, > > The Python API is documented here: > http://code.google.com/appengine/docs/python/ > > -Nick Johnson > > > On Mon, Apr 19, 2010 at 4:04 PM, codingJoe wrote: > >> Where can I find PyDocs for google.appengine.ext.db? >> >> My round-trip edits on entities with the reference property aren't >> working properly. I can't seem to figure out why, find any lower >> level documentation that might help me figure it out. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appeng...@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com >> . >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> >> > > > -- > Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. > :: Registered in Dublin, Ireland, Registration Number: 368047 > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: > 368047 > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] is it possible to use stored records with JDO from another own applications.
Hello All. I would like to know How to handle BIGTABLE records with JDO from another own application. Both application are own my account. i could not find the way in official GAE documentation. if you have a time to help me, please tell me solutions or URL. I'm sorry about my poor English skill. Must be need your imagination what I want to say. Am I making a sense? I hope you understand what i wanna say... -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Stuck indexes.
Hi all, I have a problem, there are few indexes in my GAE application in frozen state (building). They have beed building few days already. Can you drop them or help somehow with it? Application name: sidentdv Thank you. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Implementing a follower system similar to Twitter?
Mark - im working on a similar system myself. I agree that using a string in
the user class is potentially going to throw issues on a large scale - app as
it would need sharding? also running into 10,000 of followers could get heavy.
I have made an implementation using a simple relationship model. and it can be
used for anything. Friendships, followers, related images anything you like. I
am not 100% sure it is the best model or even built correctly but i was
considering posting it for comments. Its still in working so the code is not
100% tested ect - but it may give you some ideas.
Any comments on this are much appreciated - as i have said i have looked at
list properties but i can only imagine they would need sharding etc etc.
I have not added any mcache support but this will be added to the finish class
- i will post my code if anyone is interested.
!/usr/bin/env python
#
# Copyright 2007 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from google.appengine.ext import db
import google.appengine.ext.db
from google.appengine.api import memcache
import shards
class collection(db.Model):
"""
Stores relationships - This could be achieved using a
ListProperty(db.ReferenceProperty) but that may
run into issues as it is possible that many "sessions" could be appending
keys at the same time. Also a list
property may have a performance issue if their are millions of siblings
following for one entity.
further-more we can mchache our keys for even faster performance when we
check if keys are siblings of parent urls.
I feel that this approach simple
it may be - may be a better approach for long term performance?
The collection object stores a domain, url and a key
domain, url, key
Where the domain and url describe the object and the key is the related
sibling.
domian='friend'
url='martin'
key=some instance key
long_url = 'friend/martin'
stored_key ='friend/martin/[some key]'
We can store friendships by storing one side of the friendship as the url
and the siblings (friends) as keys.
the query to test if a sibling is a friend of a parent - is VERY simple:
stored_key=make_key(domian, url,key)
exists = collection.get_by_key_name(stored_key)
This works as when we store a key the we store the key name as the
domian+url+key; this is done using the make_key function
We can memcache this - again using the stored_key as the cache key for
super quick requests - when we are using the collection for say friendships
user pages will need to know if the current user is a friend of the page
owner - mcache will respond super fast in this scenario.
Counters.
We use our shard class for keeping count of how many instances are related
to the url (ie how many friends to an entity)
counter_key=long_url+'/cnt'
shards.increment(counter_key)
(note the format of our key )
Getting a list of all the keys that are a sibling: (so all the friends of
an entity)
long_url=make_url('friends','martin') # 'friends/martin'
all_siblings = db.GqlQuery('SELECT * FROM collection WHERE url = :1',
long_url).get(limit) #paging removed
We use a simple query that filters the url and returns the keys. As the
keys are strings we don't need to worry about
the data-store loading 'related instances'. Once we get our keys - we can
create a short list of keys [] that can then make one
call to the datastore and load the instances by key.
This in theory should be super-quick as only the instances say 10 at a time
are loaded for for queries where potentialy thousands of
siblings may be present
#Make a list of keys so that we can load our instances in a flash - in real
world this might be 10 at a time (see limit in above query)
li=[]
for instance in keys:
li.append(instance.key)
#get a list of instances for the keys
return db.get(li)
The next question we could ask the datastore is:
who am i a sibling of (who am i friends to)
This can be returned using another simple query;
all_instances = db.GqlQuery('SELECT * FROM collection WHERE key = :1 AND
domain = ;2', sibling_key, 'friends').get(limit)
note we use 'friends' as our domain as what to only return keys where the
relationship is of a friendship
Re: [google-appengine] Re: Counting query results > 1000
"yet" ;-) Jeff On Mon, Apr 19, 2010 at 1:46 PM, Patrick Twohig wrote: > For whatever it's worth, just getting all the keys with a fetch limit of the > largest possible value for an integer seems to work. I have approximately > 10,000 entities and counting the keys seems to work and not hit a deadline > exception. > > Cheers! > > On Sat, Apr 10, 2010 at 7:31 PM, 风笑雪 wrote: >> >> def count(model, limit): >> result = model.count(limit) >> if limit > 1000 and result == 1000: >> result = len(model.all(keys_only=True).fetch(limit)) >> return result >> >> -- >> keakon >> >> >> >> 2010/4/10 Patrick Twohig : >> > I can't use a cursor, it has to happen in one request. I tried using a >> > sharded counter to provide an estimation of ranking, but that has proven >> > unsuccessful. >> > >> > On Thu, Apr 8, 2010 at 11:55 PM, Tim Hoffman wrote: >> >> >> >> Hi >> >> >> >> You will need to use a cursor combined with a keys_only query (for >> >> efficiency) and keep >> >> resubmitting the cursor until you get all the results. >> >> >> >> If you plan to do this in appengine process rather than via the >> >> remote_api you will probably need to use a chain of tasks >> >> to achieve the result if you have a lot of entities. >> >> >> >> In python it looks something like the following (I don't do any work >> >> with java on app engine) >> >> >> >> def count(query): >> >> i = 0 >> >> while True: >> >> result = query.fetch(1000) >> >> i = i + len(result) >> >> if len(result) < 1000: >> >> break >> >> cursor = query.cursor() >> >> query.with_cursor(cursor) >> >> return i >> >> >> >> myquery = mymodels.MyModel.all(keys_only=True) >> >> >> >> x = count(myquery) >> >> >> >> Hope this helps >> >> >> >> Rgds >> >> >> >> T >> >> >> >> >> >> On Apr 9, 7:36 am, Patrick Twohig wrote: >> >> > I'm trying to figure out a way to count entities matching a certain >> >> > criteria. I was thinking of doing a key-only query and using >> >> > PreparedQuery.countEntities() (Java). However, when I do that, it >> >> > always >> >> > returns a value no higher than 1000. Is there a way to get an >> >> > accurate >> >> > count of all entities matching a query without having to do something >> >> > like >> >> > >> >> > int count = datastore.prepare( query.setKeysOnly() >> >> > ).asList(FetchOptions.Builder.limit(Integer.MAX_VALUE)).size() ? >> >> > >> >> > Thanks, >> >> > Patrick. >> >> > >> >> > -- >> >> > Patrick H. Twohig. >> >> > >> >> > Namazu Studios >> >> > P.O. Box 34161 >> >> > San Diego, CA 92163-4161 >> >> > >> >> > -- >> >> > You received this message because you are subscribed to the Google >> >> > Groups "Google App Engine" group. >> >> > To post to this group, send email to >> >> > google-appeng...@googlegroups.com. >> >> > To unsubscribe from this group, send email to >> >> > google-appengine+unsubscr...@googlegroups.com. >> >> > For more options, visit this group >> >> > athttp://groups.google.com/group/google-appengine?hl=en. >> >> >> >> -- >> >> You received this message because you are subscribed to the Google >> >> Groups >> >> "Google App Engine" group. >> >> To post to this group, send email to google-appeng...@googlegroups.com. >> >> To unsubscribe from this group, send email to >> >> google-appengine+unsubscr...@googlegroups.com. >> >> For more options, visit this group at >> >> http://groups.google.com/group/google-appengine?hl=en. >> >> >> > >> > >> > >> > -- >> > Patrick H. Twohig. >> > >> > Namazu Studios >> > P.O. Box 34161 >> > San Diego, CA 92163-4161 >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "Google App Engine" group. >> > To post to this group, send email to google-appeng...@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com. >> > For more options, visit this group at >> > http://groups.google.com/group/google-appengine?hl=en. >> > >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appeng...@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > > > -- > Patrick H. Twohig. > > Namazu Studios > P.O. Box 34161 > San Diego, CA 92163-4161 > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post
[google-appengine] System Performance
I have code that is running poorly all of a sudden today. Normally it runs well, i've listed the normal and the current stats below. But 2 our of 3 today are running like this "200 22765ms 22234cpu_ms 18617api_cpu_ms " With no other activity on the app I can refresh the page and 2 out of 3 run pooryly and the 1 out of 3 runs within normal ranges. I know it's not my app. I'm about to see some spike in my app's usage and random Deadline Exceeded errors and other random errors don't look good on my part. I'm also getting deadline exceeded errors and random odd errors like "The API call user.CreateLoginURL() was explicitly cancelled." Normal "200 16957ms 2750cpu_ms 1291api_cpu_ms" Currently: "200 22765ms 22234cpu_ms 18617api_cpu_ms " -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Google IO Tickets
I unfortunately will not be able to attend the Google IO event due to unforeseen circumstances and have to sadly put my Google IO ticket up for sale on eBay if you are interested. http://bit.ly/bS14xR Regards, Kim On Mar 30, 5:17 pm, Koen Bok wrote: > Sorry to ask for this here, but we decided we wanted to go to Google > IO last week and are looking for two tickets. Anyone knows a way to > still get them? > > We have a pretty big AppEngine project (enstore.com) and we would hate > to miss out on this :-/ -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Effectively Parallelizing Fetches (with pictures, yay!)
Hi All,
As I understand it, the process of performing a single fetch (call to
get()) from the dastastore using a key basically involves finding the host
housing the entity, opening a socket, fetching the data, and then cleaning
up the connection. So to fetch something like 30 entities from the
datastore, you're repeating the process 30 times over in serial, each time
incurring whatever overhead is involved. I also read that if you perform
bulk fetches, (ie passing multiple keys at once) you can eliminate a great
deal of that overhead. In one of the videos I watched from Google I/0 2009,
the presenter (whose name I forget - d'oh) said that performing a bulk fetch
actually performs the fetches in parallel from the data store and you shoudl
see requests noticeably faster.
Currently I have a few situations where the app performs many fetches from
the data store in serially, rather than in bulk, and I believe it is the
result of these requests being extremely slow and CPU intensive. Where
possible, I put into place as much bulk fetches as I can but I'm a little
stuck in a few places.
I'm basing the fetch latency on today's numbers --
http://code.google.com/status/appengine/detail/datastore/2010/04/19.
Anomalies aside, It looks like the get latency somewhere between 80ms and
160ms, let's spit difference and just say that it's 120 milliseconds.
Additionally, the query latency is somewhere between 250ms and 500ms.
Splitting the difference, that's 375ms. I'm just going to use those numbers
as a ballpark estimate for fetching multiple entities from the data store,
feel free to correct me if any of my reasoning is flawed or incorrect.
Example 1: http://imagepaste.nullnetwork.net/viewimage.php?id=830
Given the above example, I'm assuming that if I performed an ancestor query
with Foo("A") as the ancestor it would effectively bulk-fetch the entire
entity group. I could then use the result of that query to get the data I
need. That would make the fetch from the datastore one query, 375
milliseconds versus (7entities * 160ms) or 1120ms. So long as you need 3
or more entities (3 * 160) it would stand to reason that you're just better
off just fetching the whole thing. In some simple tests I did, that seemed
to be the case, the query approach was faster, and that's great if you know
everything is in the same entity group.
Example 2: http://imagepaste.nullnetwork.net/viewimage.php?id=831
Given the above example, none of the entities are in the same entity group,
but I would want to try to perform bulk fetches wherever possible. I would
first fetch Foo("A"). I would then see that it has two key properties
pointing to Bar("B") and Bar("C"), perform a fetch of those two entities at
once. Finally, I would see that Bar("B") and Bar("C") each reference two
more entities -- Baz("D"), Baz("E"), Baz("F"), and Baz("G") for a total of
four. In the worst case, I would fetch each entity individually taking,
once again, 1120ms. In the best case and I perform 3 fetches, (fetch A
first, then fetch B and C, then lastly fetch D, E, F, and G), it would be
more in the neighborhood of 480 milliseconds. It's still an improvement
over fetching each entity individually, but not much.
So I was thinking of ways to improve this, the second example in particular,
because I have a few places in my app where that exact thing is happening.
Right now it's actually implemented with individual fetches, but it backed
by memcache in many circumstances so that definitely helps.
So given that, here's my questions...
- When serializing the objects, would it be worthwhile adding some sort
of metadata in the entity that would tell me what other entities it
references (either directly or indirectly) so that I could fetch the whole
thing with one or two API calls? I was thinking that an entity could have
child entities with all the keys it references directly or indirectly. This
would be a huge pain to implement, and I'm not sure it would make a
noticeable performance boost.
- Is there something "under the covers" of the API that actually makes
more efficient usage of resources that I don't know about?
- Is there something in the API that I don't know about that could make
the second example faster w/o much effort?
- Is my design just bad and I should figure out a better way of doing
it? If so, how would I go about doing that?
Alright, that's all for now.
Thanks,
Patrick.
--
Patrick H. Twohig.
Namazu Studios
P.O. Box 34161
San Diego, CA 92163-4161
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Counting query results > 1000
For whatever it's worth, just getting all the keys with a fetch limit of the largest possible value for an integer seems to work. I have approximately 10,000 entities and counting the keys seems to work and not hit a deadline exception. Cheers! On Sat, Apr 10, 2010 at 7:31 PM, 风笑雪 wrote: > def count(model, limit): >result = model.count(limit) >if limit > 1000 and result == 1000: >result = len(model.all(keys_only=True).fetch(limit)) >return result > > -- > keakon > > > > 2010/4/10 Patrick Twohig : > > I can't use a cursor, it has to happen in one request. I tried using a > > sharded counter to provide an estimation of ranking, but that has proven > > unsuccessful. > > > > On Thu, Apr 8, 2010 at 11:55 PM, Tim Hoffman wrote: > >> > >> Hi > >> > >> You will need to use a cursor combined with a keys_only query (for > >> efficiency) and keep > >> resubmitting the cursor until you get all the results. > >> > >> If you plan to do this in appengine process rather than via the > >> remote_api you will probably need to use a chain of tasks > >> to achieve the result if you have a lot of entities. > >> > >> In python it looks something like the following (I don't do any work > >> with java on app engine) > >> > >> def count(query): > >> i = 0 > >> while True: > >> result = query.fetch(1000) > >> i = i + len(result) > >> if len(result) < 1000: > >> break > >> cursor = query.cursor() > >> query.with_cursor(cursor) > >> return i > >> > >> myquery = mymodels.MyModel.all(keys_only=True) > >> > >> x = count(myquery) > >> > >> Hope this helps > >> > >> Rgds > >> > >> T > >> > >> > >> On Apr 9, 7:36 am, Patrick Twohig wrote: > >> > I'm trying to figure out a way to count entities matching a certain > >> > criteria. I was thinking of doing a key-only query and using > >> > PreparedQuery.countEntities() (Java). However, when I do that, it > >> > always > >> > returns a value no higher than 1000. Is there a way to get an > accurate > >> > count of all entities matching a query without having to do something > >> > like > >> > > >> > int count = datastore.prepare( query.setKeysOnly() > >> > ).asList(FetchOptions.Builder.limit(Integer.MAX_VALUE)).size() ? > >> > > >> > Thanks, > >> > Patrick. > >> > > >> > -- > >> > Patrick H. Twohig. > >> > > >> > Namazu Studios > >> > P.O. Box 34161 > >> > San Diego, CA 92163-4161 > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups "Google App Engine" group. > >> > To post to this group, send email to > google-appeng...@googlegroups.com. > >> > To unsubscribe from this group, send email to > >> > google-appengine+unsubscr...@googlegroups.com > . > >> > For more options, visit this group > >> > athttp://groups.google.com/group/google-appengine?hl=en. > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Google App Engine" group. > >> To post to this group, send email to google-appeng...@googlegroups.com. > >> To unsubscribe from this group, send email to > >> google-appengine+unsubscr...@googlegroups.com > . > >> For more options, visit this group at > >> http://groups.google.com/group/google-appengine?hl=en. > >> > > > > > > > > -- > > Patrick H. Twohig. > > > > Namazu Studios > > P.O. Box 34161 > > San Diego, CA 92163-4161 > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com > . > > For more options, visit this group at > > http://groups.google.com/group/google-appengine?hl=en. > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Patrick H. Twohig. Namazu Studios P.O. Box 34161 San Diego, CA 92163-4161 -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Comcast not accepting outgoing emails
A link always helps: http://code.google.com/p/googleappengine/issues/detail?id=3114 Jeff On Mon, Apr 19, 2010 at 11:05 AM, emur...@conceptuamath.com wrote: > I've added a separate issue for the fact that the reply-address has an > illegal character that causes some email host to reject the email. it > issue number 3114. Please Star this issue and add a comment so it > get's attention. > > Ed > > > On Apr 19, 9:12 am, Jody Belka wrote: >> There is indeed a serious. Google is including an _ as part of a dns >> label in (I assume) the return path. An _ is not valid as part of a >> dns label. The only valid things are letters, numbers, and a dash (-). >> This is something that Google should urgently fix, as any servers that >> check the return path for validity will reject mail. >> >> On 19 April 2010 15:38, Mabuse wrote: >> >> >> >> >> >> > Hello, >> > I've the same issue. The german mail provider web.de rejects all mails >> > from the app engine. >> > It seems there is a syntax error in the mail header. See: >> >http://code.google.com/p/googleappengine/issues/detail?id=1800#c15 >> > It's an blocker for serious apps with GAE. >> >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Google App Engine" group. >> > To post to this group, send email to google-appeng...@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com> > e...@googlegroups.com> >> > . >> > For more options, visit this group at >> >http://groups.google.com/group/google-appengine?hl=en. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appeng...@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group >> athttp://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Facebook ranking friend time consuming
Use a single batch get() instead of issuing lots of separate queries.
It is best to think of the datastore as a big distributed hashmap
rather than an RDBMS.
Batch gets are easy to do with Objectify (a JDO alternative for GAE).
You can still do batch gets with JDO but the syntax is tortured. It
looks something like this:
Query q = pm.newQuery("select from " + Book.class.getName() + " where
:keys.contains(key));
Jeff
On Sat, Apr 17, 2010 at 9:22 PM, Martin de la torre wrote:
> Hello i am doing a jsp where i make one call to facebook to get the logged
> client , another to get their friends and
> and for each friend i look him on the datastore .Finaly i show on html the
> raking of the friends.
>
> for (int i=0;i {
> String amigo=friends.getString(i);
> String query = "select from " + Jugador.class.getName()+" where id
> == "+amigo;
> List jugadores = (List)
> pm.newQuery(query).execute();
> if(!jugadores.isEmpty())
> jugadoresAmigos.add(jugadores.get(0));
> }
>
> It seems that this process is very time consuming and sometimes the jsp
> does not load because it exeds the 30 seconds limit.
> is there a way to optimize it?
>
> here i read that it is good to have a lot of small requests but i did not
> figure how can i do that
> http://wubbahed.com/2009/03/01/google-appengine-and-facebook-applications-10-things-i-wish-i-had-known/
>
> thanks
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Source code for AppCfg.java
Also, do you know if the source will be made available anytime soon? Thanks! On Apr 17, 3:15 pm, joetz7 wrote: > Hi, thanks for the reply! > > I'm trying to make it so that I can put password as part of the > argument to the script (appcfg.cmd). Right now it only allows me to > put the e-mail as part of the argument. > > joetz7 > > On Apr 16, 1:43 pm, Toby Reyelts wrote: > > > > > > > Sorry, the source isn't available for this yet, but we're working on it. > > What extensions were you thinking of? > > > On Fri, Apr 16, 2010 at 11:45 AM, joetz7 wrote: > > > So, does anyone know if AppCfg.java is available for download? > > > > Thanks! > > > > On Apr 14, 6:21 pm, joetz7 wrote: > > > > Hi all, > > > > > I was wondering if the code for AppCfg.java is available anywhere. > > > > I'm trying to modify it a bit to fit my application, however I can't > > > > seem to find the source code for it. (Only can find the AppCfg.class > > > > inside com.google.appengine.tools.admin) > > > > > Any help would be greatly appreciated!! > > > > Thanks! > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "Google App Engine" group. > > > To post to this group, send email to google-appeng...@googlegroups.com. > > > To unsubscribe from this group, send email to > > > google-appengine+unsubscr...@googlegroups.com > > e...@googlegroups.com> > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Comcast not accepting outgoing emails
I've added a separate issue for the fact that the reply-address has an illegal character that causes some email host to reject the email. it issue number 3114. Please Star this issue and add a comment so it get's attention. Ed On Apr 19, 9:12 am, Jody Belka wrote: > There is indeed a serious. Google is including an _ as part of a dns > label in (I assume) the return path. An _ is not valid as part of a > dns label. The only valid things are letters, numbers, and a dash (-). > This is something that Google should urgently fix, as any servers that > check the return path for validity will reject mail. > > On 19 April 2010 15:38, Mabuse wrote: > > > > > > > Hello, > > I've the same issue. The german mail provider web.de rejects all mails > > from the app engine. > > It seems there is a syntax error in the mail header. See: > >http://code.google.com/p/googleappengine/issues/detail?id=1800#c15 > > It's an blocker for serious apps with GAE. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Comcast not accepting outgoing emails
Yes, I think that is my issue. Here is the return header from an email that was successfully received by yahoo: Return-Path: <34vdlsw0jb-ooazoqbfgmymftsymux.oayqygdygdkmtaa@1rf- h_fbpz96xcrx.apphosting.bounces.google.com> note: @1rf-h_fbpz96xcrx. The underscore is illegal. I think that is why many hosts are rejecting my emails. The receiver is never notified in any way. On Apr 19, 9:12 am, Jody Belka wrote: > There is indeed a serious. Google is including an _ as part of a dns > label in (I assume) the return path. An _ is not valid as part of a > dns label. The only valid things are letters, numbers, and a dash (-). > This is something that Google should urgently fix, as any servers that > check the return path for validity will reject mail. > > On 19 April 2010 15:38, Mabuse wrote: > > > > > > > Hello, > > I've the same issue. The german mail provider web.de rejects all mails > > from the app engine. > > It seems there is a syntax error in the mail header. See: > >http://code.google.com/p/googleappengine/issues/detail?id=1800#c15 > > It's an blocker for serious apps with GAE. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Random, but persistant error
for what is worth, I use a Mac and recently it got a Java update. Thinking on it, I changed the run settings for my project to use the older 1.5 JRE and it seems to have mitigated this issue a bit. 2010/4/18 mandar khadilk > I get the same error many times on my local dev instance. > > I restart the App and then it goes away. > This error needs to be debugged more. > > > On Apr 13, 1:23 pm, Juan González wrote: > > Hi all, > >I have a random issue when testing my application locally. From > > time to time, when I use PersistenceManager.getObjectById() I receive > > the following exception: > > > > java.lang.UnsupportedOperationException > > at > > > org.datanucleus.store.appengine.EntityUtils.getPropertyName(EntityUtils.jav > a: > > 62) > > at > > > org.datanucleus.store.appengine.DatastoreFieldManager.getPropertyName(Datas > toreFieldManager.java: > > 1073) > > at > > > org.datanucleus.store.appengine.DatastoreFieldManager.fetchObjectField(Data > storeFieldManager.java: > > 309) > > at > > > org.datanucleus.state.AbstractStateManager.replacingObjectField(AbstractSta > teManager.java: > > 1197) > > at > > > org.ugr.rtpstat.server.orm.UsuarioRegistrado.jdoReplaceField(UsuarioRegistr > ado.java) > > at > > > org.ugr.rtpstat.server.orm.UsuarioRegistrado.jdoReplaceFields(UsuarioRegist > rado.java) > > at > > > org.datanucleus.state.JDOStateManagerImpl.replaceFields(JDOStateManagerImpl > .java: > > 2772) > > at > > > org.datanucleus.state.JDOStateManagerImpl.replaceFields(JDOStateManagerImpl > .java: > > 2791) > > at > > > org.datanucleus.store.appengine.DatastorePersistenceHandler.fetchObject(Dat > astorePersistenceHandler.java: > > 466) > > at > > > org.datanucleus.state.JDOStateManagerImpl.validate(JDOStateManagerImpl.java > : > > 4263) > > at > > org.datanucleus.ObjectManagerImpl.findObject(ObjectManagerImpl.java: > > 2444) > > at > > > org.datanucleus.jdo.JDOPersistenceManager.getObjectById(JDOPersistenceManag > er.java: > > 1671) > > at > > > org.datanucleus.jdo.JDOPersistenceManager.getObjectById(JDOPersistenceManag > er.java: > > 1767) > > at > > > org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManager.getObjec > tById(DatastoreJDOPersistenceManager.java: > > 73) > > at > > > org.ugr.rtpstat.server.RtpstatServiceImpl.getRegisteredUser(RtpstatServiceI > mpl.java: > > 88) > > at > > > org.ugr.rtpstat.server.RtpstatServiceImpl.getRegisteredUser(RtpstatServiceI > mpl.java: > > 79) > > at > > > org.ugr.rtpstat.server.RtpstatServiceImpl.isRegisteredUser(RtpstatServiceIm > pl.java: > > 204) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: > > 39) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImp > l.java: > > 25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > > > com.google.appengine.tools.development.agent.runtime.Runtime.invoke(Runtime > .java: > > 100) > > at > > com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java: > > 562) > > at > > > com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServi > ceServlet.java: > > 188) > > at > > > com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServi > ceServlet.java: > > 224) > > at > > > com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(Abstract > RemoteServiceServlet.java: > > 62) > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:713) > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) > > at > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: > > 511) > > at org.mortbay.jetty.servlet.ServletHandler > > $CachedChain.doFilter(ServletHandler.java:1166) > > at > > > com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFi > lter.java: > > 51) > > at org.mortbay.jetty.servlet.ServletHandler > > $CachedChain.doFilter(ServletHandler.java:1157) > > at > > > com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(Trans > actionCleanupFilter.java: > > 43) > > at org.mortbay.jetty.servlet.ServletHandler > > $CachedChain.doFilter(ServletHandler.java:1157) > > at > > > com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFile > Filter.java: > > 122) > > at org.mortbay.jetty.servlet.ServletHandler > > $CachedChain.doFilter(ServletHandler.java:1157) > > at > > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java: > > 388) > > at > > org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java: > > 216) > > at > > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java: > > 182) > > at
[google-appengine] Re: Questions about request logs (their circular buffer, its size etc.)
Thanks Nick, that's nice and clear now. Do you know if there's any upper limit on the frequency with which logs can be downloaded? Cheers, On Apr 16, 2:03 pm, "Nick Johnson (Google)" wrote: > Hi Peter, > > Both are true: We keep 90 days of request logs, but only a fixed amount of > application logging. > > -Nick Johnson > > > > > > On Thu, Apr 15, 2010 at 7:11 PM, peterk wrote: > > Hey Ikai, thanks for the reply. > > > Are you sure about the 90 day thing? I was reading a thread here where > > Nick Johnson suggested it was a fixed buffer size: > > >http://groups.google.com/group/google-appengine/browse_thread/thread/... > > > In that case, the person asking seemed to be only able to keep 20 > > minutes of log data before old logs started to be overwritten (a busy > > site I guess!). > > > Has something changed? Could you check out what the case actually is? > > > If I might, I'd like to request if log related 'quotas' could be > > exposed and published, and expandable with money if possible! :) > > They're quite critical to an app I'm trying to plan, and while the > > current setup seems to be OK for my needs, the blindness to what's > > happening under the covers and what might happen going forward is a > > wee bit disconcerting. > > > Cheers, > > > On Apr 15, 6:48 pm, "Ikai L (Google)" wrote: > > > The last time I checked, we guaranteed 90 days of logs regardless of > > size. > > > This is subject to change, however. I don't know what size we will set, > > but > > > like many of our other products, it'll likely start at a number we think > > is > > > reasonable based on overall usage and trend upward. > > > > Anything is possible with regards to billed features - we just can't give > > an > > > ETA unless it's explicitly stated on the roadmap. > > > > On Wed, Apr 14, 2010 at 12:27 PM, peterk wrote: > > > > Reading some previous discussions and things alluded to therein, some > > > > questions cropped up. > > > > > I'm considering an application that quite fundamentally requires > > > > access to all its request logs. > > > > > So I guess I'd be recommended to use appcfg.py to periodically pull > > > > down the logs and append them to a txt file, or do whatever else I > > > > need to do to persist them. > > > > > In previous discussion I saw Google folks say that logs are stored in > > > > a circular buffer. Meaning if you had enough requests, they'd > > > > overwrite older requests. Meaning that if you had enough requests or > > > > were downloading the logs sufficiently infrequently, you might lose > > > > logs. > > > > > So question number 1: how big is the circular buffer for each > > > > application? > > > > > 2: Changes to that buffer size could detrimentally impact on my > > > > application if revised downward (silently in particular), so are there > > > > any guarantees that this buffer size won't be reduced in the future? > > > > > 3: Is there any possibility that this buffer could be exposed via the > > > > quota system so that an app could purchase a larger buffer size if it > > > > were helpful? > > > > > I'm flashing forward to a busy site and somewhat worrying about how > > > > frequently I'll have to download the logs, and whether it would be > > > > feasible to download them with a certain frequency anyway (e.g. if the > > > > log download time is quite long - as I've found it can be). Any > > > > answers or advice would be very welcome. Thanks! > > > > > -- > > > > You received this message because you are subscribed to the Google > > Groups > > > > "Google App Engine" group. > > > > To post to this group, send email to google-appengine@googlegroups.com > > . > > > > To unsubscribe from this group, send email to > > > > google-appengine+unsubscr...@googlegroups.com > > > e...@googlegroups.com> > e...@googlegroups.com> > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/google-appengine?hl=en. > > > > -- > > > Ikai Lan > > > Developer Programs Engineer, Google App Enginehttp:// > > googleappengine.blogspot.com|http://twitter.com/app_engine > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. > > -- > Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: > Registered in Dublin, Ireland, Registration Number: 368047 > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: > 368047 > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsub
Re: [google-appengine] Re: Comcast not accepting outgoing emails
There is indeed a serious. Google is including an _ as part of a dns label in (I assume) the return path. An _ is not valid as part of a dns label. The only valid things are letters, numbers, and a dash (-). This is something that Google should urgently fix, as any servers that check the return path for validity will reject mail. On 19 April 2010 15:38, Mabuse wrote: > Hello, > I've the same issue. The german mail provider web.de rejects all mails > from the app engine. > It seems there is a syntax error in the mail header. See: > http://code.google.com/p/googleappengine/issues/detail?id=1800#c15 > It's an blocker for serious apps with GAE. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Google IO 2010 Conference
I unfortunately will not be able to attend the Google IO event due to unforeseen circumstances and have to sadly put my Google IO ticket up for sale on eBay if you are interested. http://bit.ly/bS14xR Regards, Kim -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] [Java] Blobstore without servlet
Hi there. I'm using GAE with the Play! Framework which don't provide acces to any servletRequest object. So I was wondering if there any way to use the Blobstore without servlet ? In my code I have my data as byte[], I juste need to store it directly in the blobstore. Thanks in advance. Regards, Michaël. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Problem with Google App Email and Google App Engine
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hallo, I'm using Google Apps for my domain and added an email address to Google Apps. I changed the email address of my Google Account (which is connected with Google App Engine) to my new Google Apps powered email. Every Google Service works fine with the new email, but when I try to update an app of my Google Account, App Engine says I don't have the permissions to modify the app: Application: gaeapp123; version: 1. Server: appengine.google.com. Scanning files on local disk. Scanned 500 files. Scanned 1000 files. Initiating update. Password for d3f3n...@d3f3nd3r.com: Error 403: --- begin server output --- You do not have permission to modify this app (app_id=u'gaeapp123'). - --- end server output --- Any ideas? Thanks -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iQIcBAEBAgAGBQJLzGdVAAoJECGFYnroAHFZVPcP/A1f5TUjadIZ6/ls7YNuxLHk YuRs7+eaf60WkZNADSMdMNMPDTr51ML4GukegRgjnR5XMVoaxgoErBwgTmoRTNtF gJiwIAUmHxtw7gkWSFoHIrjroeOa3FQweoFLCqgCP/hJS5P8O+KN2J8mq3N0sAz0 W3KVvRtId2khguZWKwCS0fc4p+ollY1LnHFd9ulyLMO02r8MhgqA9nQ2C8mtoVp7 0eOzQFD4ZXLZ/EBjDzJhTWcTqi1d65xNPcsfyXw98s6Le6PtchsRYmCfiAGWB5FE qD6J7z0dCz6GLHeeraV3LrhI0+HjYZOUcW0KkMkInoO/np1csRmEgP/kNP9EhNJS NsekYerx0KtpXS9ZpnFbLwUHBV8pASBam9b7lTg7Ula5Pu4XY00Dn+5ow/LWwVUm jsN8QY1HfiT+Y2zPC3FSJcyqN41yuUuBdcOC3tEKSwek7Z5KSij/ed9RoL1suaIW f8pwV/l4avhFF0eNDGFOjdCqSg4IKFGCAIBb+g+2aE0bXFs097HeE54RLBEdr6df 1y7EOWNUHo6Totu8DW98/04HslDeNB0msCu3zm3DSNdk6xqRhz946rn2qFV0n1mV kzVNqGz2aZobYI7KOHDzyHhAnCi6723Wma/z9BpPOXJzg4lFWJaIKidOEOsNX6UK 76m66AwO0Ndb0eCpuZt1 =/9z8 -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Comcast not accepting outgoing emails
Hello, I've the same issue. The german mail provider web.de rejects all mails from the app engine. It seems there is a syntax error in the mail header. See: http://code.google.com/p/googleappengine/issues/detail?id=1800#c15 It's an blocker for serious apps with GAE. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Unable to start Google AppEngine Launcher.
I get the error C:\DOCUME~1\ruben\LOCALS~1\Temp \WER8bc6.dir00\GoogleAppEngineLauncher.exe.mdmp C:\DOCUME~1\ruben\LOCALS~1\Temp\WER8bc6.dir00\appcompat.txt What is going on. It used to work fine. After a s/w update my machine had, i experienced this issue. I tried repair, remove and install again procedures. Any clues? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Pagination
kamathln wrote: See, with cursors, you can do basic pagination of entries (not the content of the entries themselves. Like if you have 200 entries in the datastore for an entity type, you can show 10 at a time and using the cursor, you can generate a "next" link. But that in turn poses a problem: you can only generate a "next" link, and not "previous" link. So my idea for the solution is to go over the remaining results in a quick way, and cache the cursors that point to first page, second page and so on.. in a csv in memcache entry(session managed, of course). Now using this memcache entry, you can generate a whole array of page links. The next problem that arises is that if the database has grown too big, then the csv generation will be slow(as it has to jump a lot of times). So the next idea is , instead of generating all cursors in the main cgi , the main cgi generates only first few set of page links, and then triggers a task queue entry to continue from there. By the time the user would have read the first page and clicks on "next page", the task would have generated few more entries. As the set of cursors for pagination is in memcache, you can use it even in your first page, if you are using AJAX (guessing, not that sure.) Why don't you just generate a few links (something like page 1,2,3, last page) in the first request and then generate additional links if a user goes to page 2 or 3 (or last page), so you do not have to generate them in the background with a task. -Ulrich -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Python Documentation Missing?
Hi, The Python API is documented here: http://code.google.com/appengine/docs/python/ -Nick Johnson On Mon, Apr 19, 2010 at 4:04 PM, codingJoe wrote: > Where can I find PyDocs for google.appengine.ext.db? > > My round-trip edits on entities with the reference property aren't > working properly. I can't seem to figure out why, find any lower > level documentation that might help me figure it out. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Python Documentation Missing?
A couple places I've looked (Unsuccessfully) 1. Looking for documentation from the AppEngine perspective... Try clicking on the PyDocs for google.appengine.ext.db link here. Dead. http://gdata-python-client.googlecode.com/svn/trunk/pydocs/gdata.alt.appengine.html 2. Ok, lets look from the gdata-python-client perspective: found: http://gdata-python-client.googlecode.com/svn/trunk/pydocs/ no mention of google.appengine.ext.db -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Python Documentation Missing?
Where can I find PyDocs for google.appengine.ext.db? My round-trip edits on entities with the reference property aren't working properly. I can't seem to figure out why, find any lower level documentation that might help me figure it out. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Implementing a follower system similar to Twitter?
Hi,
I'm building a service similar to twitter, where users can follow one
another. My user class looks like this:
@PersistenceCapable
class User {
@PrimaryKey
private String mUsername;
}
I'm thinking to store the follower/followee relationships in an
intermediate class, and am not sure if this makes sense, or if there
is a better way to do it. I'm thinking of the case where I'm viewing
another user's info page, and want to follow them. Then I'd create a
new object like this:
@PersistenceCapable
class Relationship {
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key mKey;
@Persistent
private String mFollowerUsername;
@Persistent
private String mFolloweeUsername;
}
I can then query for the first N records above when needing to show a
list of followers, and page as necessary. Is there a better pattern
for doing this? I was reading this post:
http://groups.google.com/group/google-appengine/browse_thread/thread/18bddc764f2647b9/d567b638a201fde8?lnk=gst&q=follower#d567b638a201fde8
where Brett suggested storing the follower/followee user ids directly
in the User object (perhaps as an ArrayList). In some sense
this would be easier but if those lists have 10,000 entries each,
won't it take a long time to load the User object? Loading user
objects probably is a frequent action in my case. Also to support
adding and deletion of followers, I'd need to search the
ArrayList for duplicates (guess I could use a set
instead.
Not sure if this question belongs here, it seems pretty JDO specific,
but I'm wondering what the best way to do this is so it scales well on
app engine,
Thank you
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Datastore Transaction problems in App Engine 1.3.2
Same problem here. Is that a problem with nested transactions? Anyone could comment, please? Thanks much, Maciej On 10 April 2010 23:27, AdamM wrote: > Hi, > > I am developing an application, that operates on multiple entity > groups in multiple datastore transactions. > > I first create a PersistenceManager and open a transaction on it.I > make an object persistent using this transaction, but I do not commit > it. After this I create a new PersistenceManager with a transaction > from the same factory, I make persistent an object to a different > entity group. Again I do not commit. At the end I will commit the > transaction after each other. > > There is a foreign key relationship between the 2 objects (I store the > Object with primary key first). > > The second transaction fails to commit with an Exception: > > NestedThrowablesStackTrace: > java.lang.IllegalStateException: Attempted to deregister a transaction > that is not currently registered. > at > com.google.appengine.api.datastore.TransactionStackImpl.remove(TransactionStackImpl.java: > 61) > at > com.google.appengine.api.datastore.TransactionImpl.commit(TransactionImpl.java: > 71) > at > org.datanucleus.store.appengine.DatastoreTransaction.commit(DatastoreTransaction.java: > 61) > at > org.datanucleus.store.appengine.DatastoreXAResource.commit(DatastoreXAResource.java: > 88) > (I do not get this error, if I leave the persistence of the object out > from the 2nd transaction.) > > I was running this code both from JUnit Tests and on Local App Engine > instance. > > This was working with 1.2.6 or 1.3.0 but not with 1.3.2. > > I also have a JUnit Test to test concurrent modification, I used to > get JDOCanRetryException on concurrent changes, but now with the new > version I am getting: > > java.util.ConcurrentModificationException: too much contention on > these datastore entities. please try again. > at > com.google.appengine.api.datastore.DatastoreApiHelper.translateError(DatastoreApiHelper.java: > 37) > at > com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(DatastoreApiHelper.java: > 63) > at > com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(TransactionImpl.java: > 42) > at > com.google.appengine.api.datastore.TransactionImpl.makeSyncCall(TransactionImpl.java: > 56) > at > com.google.appengine.api.datastore.TransactionImpl.commit(TransactionImpl.java: > 66) > > Has anything changed in 1.3.2? > > Please help. > > Regards, > > Adam > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Maciej Machulak email: maciej.machu...@gmail.com tel: +48 602 45 31 44 tel: +44 7999 606 767 -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Redirecting domain (not google apps) to appengine
On Mon, Apr 19, 2010 at 12:03 PM, Fredrik Bonander < carl.fredrik.bonan...@gmail.com> wrote: > Hi Nick, > > I've read the article. But if I register my existing domain w/ google apps > won't the rest of subdomains stop working? Wouldn't the domain be moved to > Google? > No - only the things you explicitly CNAME to the appropriate domain will change in functionality. Google does not have the power to 'take over' your domain any more than anyone else does. -Nick Johnson > ..fredrik > > > > On 19 apr 2010, at 12.55, Nick Johnson (Google) wrote: > > Hi Fredrik, > > Please follow the instructions here: > http://code.google.com/appengine/docs/domain.html > > -Nick Johnson > > On Mon, Apr 19, 2010 at 10:23 AM, Fredrik Bonander < > carl.fredrik.bonan...@gmail.com> wrote: > >> Hi, >> >> I'm building a application that supports different domains. A small CMS >> that supports different domains. >> >> But what I can't figure out is how to redirect other domains that's >> outside google apps. I have a domain at google apps, that work's perfectly. >> When I create a cname that points at either my appid.appspot.com or >> www.appsdomain.com it just goes to google.com. >> >> What do I need to do so the other domains point to my appengine >> application. >> >> ..fredrik >> >> >> >> -- >> Fredrik Bonander >> carl.fredrik.bonan...@gmail.com >> +46 70 943 5441 >> >> *- **the infinite power of the creative min**d - * >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appeng...@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com >> . >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > > > -- > Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. > :: Registered in Dublin, Ireland, Registration Number: 368047 > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: > 368047 > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > -- > Fredrik Bonander > carl.fredrik.bonan...@gmail.com > +46 70 943 5441 > > *- **the infinite power of the creative min**d - * > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Redirecting domain (not google apps) to appengine
Hi Nick, I've read the article. But if I register my existing domain w/ google apps won't the rest of subdomains stop working? Wouldn't the domain be moved to Google? ..fredrik On 19 apr 2010, at 12.55, Nick Johnson (Google) wrote: > Hi Fredrik, > > Please follow the instructions here: > http://code.google.com/appengine/docs/domain.html > > -Nick Johnson > > On Mon, Apr 19, 2010 at 10:23 AM, Fredrik Bonander > wrote: > Hi, > > I'm building a application that supports different domains. A small CMS that > supports different domains. > > But what I can't figure out is how to redirect other domains that's outside > google apps. I have a domain at google apps, that work's perfectly. > When I create a cname that points at either my appid.appspot.com or > www.appsdomain.com it just goes to google.com. > > What do I need to do so the other domains point to my appengine application. > > ..fredrik > > > > -- > Fredrik Bonander > carl.fredrik.bonan...@gmail.com > +46 70 943 5441 > > - the infinite power of the creative mind - > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > > -- > Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: > Registered in Dublin, Ireland, Registration Number: 368047 > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: > 368047 > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- Fredrik Bonander carl.fredrik.bonan...@gmail.com +46 70 943 5441 - the infinite power of the creative mind - -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] datastore design for hierarchical data.
Hi, See my answer to your question on Stack Overflow: http://stackoverflow.com/questions/2662212/database-design-in-google-app-engine/2663049#2663049 In general, please refrain from cross-posting here and on SO, unless you've posted to one and haven't received any satisfactory answers after a reasonable interval. -Nick Johnson On Sun, Apr 18, 2010 at 1:29 PM, gops wrote: > hi , i am designing a simple project based to do list. the idea is to > define tasks under project ( no workflow - just "task is completed" or > not is required. ) in a hirarchial way. i.e. each task has multiple > task and that task may have other multiple task. a project can be said > to be completed if all task under that project are completed. , i > tought of using refrenceproeperty to create hirarchy , but could not > figure out easy way ( which do not take more than 30 seconds to find > all the children of a project and check weather it is completed or > not ) . to detect if project is complete or not. how to design > database for such job ? and also , if i need to copy the project in > order to define another project , how to copy hierarchical data ? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Return list of dicts instead of object
Many thanks, this should do nicely. I have already written a hack-job
implementation to get around it, but will soon change it to something
that makes use of this.
On 16 Apr., 10:23, hawkett wrote:
> If you use the (undocumented) underlying datastore API you will get
> datastore objects as lists of dictionary-like objects. It is the
> higher level datastore API that converts these objects into Models.
> Consequently, using this API is slightly faster as well. For example
>
> from google.appengine.api import datastore
>
> q = datastore.Query("SomeKind", {"name =": "someName"}, None, False,
> True, None) # Kind, query, ?, keys_only, compile, cursor
> q_results = q.Run()
>
> for result in q_results:
> print result.key()
> print result['name']
>
> The only mechanism I have found to understand this API is to review
> the app engine code. The documentation in the code is pretty
> reasonable, but generally no examples.
>
> On Apr 15, 1:17 pm, Lynge wrote:
>
>
>
>
>
> > Hi, I am using an Expando class and I want to sort it.
> > It needs to be sorted according to a dynamically assigned property so
> > it has o be done at runtime.
>
> > I am using python and it could be done easily with something like:
>
> > dates = sorted(datelist, key=itemgetter('comp_date'))
>
> > But since the objects returned from the datastore are not iterable
> > this is impossible.
>
> > So I thought that I would just create my own list of dicts and then I
> > can iterate and sort it, but alas that is not possible either.
>
> > I can iterate over the primary just fine
>
> > for date in datesfromdb:
> > datelist.append(dict(date))
>
> > but no matter how I try to get the data out of the date object I have
> > no luck.
> > I dont want to name all the properties of the model since that would
> > make it a big pain to update anything, but how then do I do it?
>
> > Any help would be appreciated.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group
> athttp://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Redirecting domain (not google apps) to appengine
Hi Fredrik, Please follow the instructions here: http://code.google.com/appengine/docs/domain.html -Nick Johnson On Mon, Apr 19, 2010 at 10:23 AM, Fredrik Bonander < carl.fredrik.bonan...@gmail.com> wrote: > Hi, > > I'm building a application that supports different domains. A small CMS > that supports different domains. > > But what I can't figure out is how to redirect other domains that's outside > google apps. I have a domain at google apps, that work's perfectly. > When I create a cname that points at either my appid.appspot.com or > www.appsdomain.com it just goes to google.com. > > What do I need to do so the other domains point to my appengine > application. > > ..fredrik > > > > -- > Fredrik Bonander > carl.fredrik.bonan...@gmail.com > +46 70 943 5441 > > *- **the infinite power of the creative min**d - * > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Redirecting domain (not google apps) to appengine
Hi, I'm building a application that supports different domains. A small CMS that supports different domains. But what I can't figure out is how to redirect other domains that's outside google apps. I have a domain at google apps, that work's perfectly. When I create a cname that points at either my appid.appspot.com or www.appsdomain.com it just goes to google.com. What do I need to do so the other domains point to my appengine application. ..fredrik -- Fredrik Bonander carl.fredrik.bonan...@gmail.com +46 70 943 5441 - the infinite power of the creative mind - -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: datastore design for hierarchical data.
I agree with this approach. It works well with tree data containing fewer than 1000 nodes. It's also the amount of entities that appengine can fetch in one query. Here's some example code which I'm using for a forum, it may help you get started: http://gist.github.com/370861 It's running at: http://www.caterpi.com/forum On Apr 18, 8:03 am, Tim Hoffman wrote: > Here is the approach I would probably take unless you have many > hundreds of child tasks. > > I would keep a list of keys of all children of each project/task. > (You can then do a db.get and get all immediate children with single > call) > and each child has a reference to it's parent. In addition in each > item keep a list of all parents keys back to the root. That way you > can search for all children > in part of the tree/subtree > > (This approach is especially useful if you have multiple child types) > as back ref sets > are kind specific. > > Keep a running count of complete children at each level, rather than > trying to calculate the current state in a single big function, (keep > these subsidiary values up to date through tasks) > > T > > On Apr 18, 8:29 pm, gops wrote: > > > > > hi , i am designing a simple project based to do list. the idea is to > > define tasks under project ( no workflow - just "task is completed" or > > not is required. ) in a hirarchial way. i.e. each task has multiple > > task and that task may have other multiple task. a project can be said > > to be completed if all task under that project are completed. , i > > tought of using refrenceproeperty to create hirarchy , but could not > > figure out easy way ( which do not take more than 30 seconds to find > > all the children of a project and check weather it is completed or > > not ) . to detect if project is complete or not. how to design > > database for such job ? and also , if i need to copy the project in > > order to define another project , how to copy hierarchical data ? > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Appstats caused java OutOfMemoryError - heap
Same for me. On 18 avr, 22:45, aloo wrote: > I'm experiencing the same issue - were you able to resolve? > > On Mar 31, 5:17 pm, Jake wrote: > > > > > I have a rather large and intense sorting process that runs > > periodically. When I enabled Appstats, this process started to fail > > with the following trace. > > > When I disabled Appstats, it went back to normal. > > > I really liked Appstats, too. sigh. > > > 03-31 02:05PM 56.204 > > Uncaught exception from servlet > > java.lang.OutOfMemoryError: Java heap space > > at > > com.google.appengine.repackaged.com.google.protobuf.ByteString.toByteArray( > > ByteString.java: > > 173) > > at > > com.google.appengine.repackaged.com.google.protobuf.CodedOutputStream.write > > BytesNoTag(CodedOutputStream.java: > > 356) > > at > > com.google.appengine.repackaged.com.google.protobuf.CodedOutputStream.write > > Bytes(CodedOutputStream.java: > > 200) > > at com.google.appengine.api.memcache.MemcacheServicePb > > $MemcacheSetRequest$Item.writeTo(MemcacheServicePb.java:1568) > > at > > com.google.appengine.repackaged.com.google.protobuf.CodedOutputStream.write > > GroupNoTag(CodedOutputStream.java: > > 333) > > at > > com.google.appengine.repackaged.com.google.protobuf.CodedOutputStream.write > > Group(CodedOutputStream.java: > > 172) > > at com.google.appengine.api.memcache.MemcacheServicePb > > $MemcacheSetRequest.writeTo(MemcacheServicePb.java:1980) > > at > > com.google.appengine.repackaged.com.google.protobuf.AbstractMessageLite.toB > > yteArray(AbstractMessageLite.java: > > 36) > > at > > com.google.appengine.api.memcache.MemcacheServiceImpl.makeSyncCall(Memcache > > ServiceImpl.java: > > 170) > > at > > com.google.appengine.api.memcache.MemcacheServiceImpl.putAll(MemcacheServic > > eImpl.java: > > 420) > > at > > com.google.appengine.api.memcache.MemcacheServiceImpl.putAll(MemcacheServic > > eImpl.java: > > 459) > > at > > com.google.appengine.tools.appstats.MemcacheWriter.persist(MemcacheWriter.j > > ava: > > 264) > > at > > com.google.appengine.tools.appstats.MemcacheWriter.commit(MemcacheWriter.ja > > va: > > 177) > > at > > com.google.appengine.tools.appstats.AppstatsFilter.doFilter(AppstatsFilter. > > java: > > 94) > > at org.mortbay.jetty.servlet.ServletHandler > > $CachedChain.doFilter(ServletHandler.java:1157) > > at > > com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlo > > bUploadFilter.java: > > 97) > > at org.mortbay.jetty.servlet.ServletHandler > > $CachedChain.doFilter(ServletHandler.java:1157) > > at > > com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionF > > ilter.java: > > 35) > > at org.mortbay.jetty.servlet.ServletHandler > > $CachedChain.doFilter(ServletHandler.java:1157) > > at > > com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(Trans > > actionCleanupFilter.java: > > 43) > > at org.mortbay.jetty.servlet.ServletHandler > > $CachedChain.doFilter(ServletHandler.java:1157) > > at > > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java: > > 388) > > at > > org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java: > > 216) > > at > > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java: > > 182) > > at > > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java: > > 765) > > at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java: > > 418) > > at > > com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionH > > andlerMap.java: > > 238) > > at > > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java: > > 152) > > at org.mortbay.jetty.Server.handle(Server.java:326) > > at > > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java: > > 542) > > at org.mortbay.jetty.HttpConnection > > $RequestHandler.headerComplete(HttpConnection.java:923) > > at > > com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequ > > estParser.java: > > 76) > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Request per Second limitation (PLEASE)
Hi Jairo, This depends entirely on what quota users were going over. You need to check your admin console to determine this. If it's a paid quota, you need to allocate more budget to it. If it's an unpaid quota, contact us again and we'll see what we can do. -Nick Johnson On Mon, Apr 19, 2010 at 2:33 AM, Jairo Vasquez wrote: > Hi Nick, > > I had to revert the change (a new feature that increased requests) > because users were getting "Over quota error" once a while (1 of 6 > tries more or less). > > That was Thursday. > > My appID: paymentez > > If you can search something about it maybe I could release our new > feature again before trying another solution for that feature. > > Thanks again for your help. > > On Apr 16, 8:02 am, "Nick Johnson (Google)" > wrote: > > Hi Jairo, > > > > On Thu, Apr 15, 2010 at 10:48 PM, Jairo Vasquez >wrote: > > > > > Nick, > > > > > I'm getting "Limited" in Requests and Secure Requests Quotas. Also > > > pages are showing "Quota error" (not all times). > > > > What do you mean by "I'm getting 'limited'"? What are you observing, > > precisely? > > > > > > > > > I've tried to reduce requests. I just though you told me everything > > > would scale :S > > > > What is your average request latency, and what is your App ID? > > > > -Nick Johnson > > > > > > > > > > > > > > > > > Please what can I do. > > > > > On Apr 14, 12:21 pm, "Nick Johnson (Google)" > > > wrote: > > > > Hi Jairo, > > > > > > If your average latency is low (below 1 second), you should be able > to > > > scale > > > > up to any request load, as long as you provide the budget to support > it. > > > > > > -Nick Johnson > > > > > > On Wed, Apr 14, 2010 at 5:37 PM, Jairo Vasquez < > jairo.vasq...@gmail.com > > > >wrote: > > > > > > > Hi, > > > > > > > I am managing 650 request per second. Can I start to manage 1200 > > > > > request per second? > > > > > > > I need to know what my limits are, what can I do. > > > > > > > Should I use another solution or can GAE support this load? > > > > > > > Thanks > > > > > > > -- > > > > > Jairo Vasquez Moreno > > > > > Mentez Developer > > > > >www.mentez.com > > > > > Medellin - Colombia > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > Groups > > > > > "Google App Engine" group. > > > > > To post to this group, send email to > google-appengine@googlegroups.com > > > . > > > > > To unsubscribe from this group, send email to > > > > > google-appengine+unsubscr...@googlegroups.com e...@googlegroups.com> > > e...@googlegroups.com> > > > > > . > > > > > For more options, visit this group at > > > > >http://groups.google.com/group/google-appengine?hl=en. > > > > > > -- > > > > Nick Johnson, Developer Programs Engineer, App Engine Google Ireland > Ltd. > > > :: > > > > Registered in Dublin, Ireland, Registration Number: 368047 > > > > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration > > > Number: > > > > 368047 > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Google App Engine" group. > > > To post to this group, send email to google-appengine@googlegroups.com > . > > > To unsubscribe from this group, send email to > > > google-appengine+unsubscr...@googlegroups.com e...@googlegroups.com> > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > > > -- > > Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. > :: > > Registered in Dublin, Ireland, Registration Number: 368047 > > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration > Number: > > 368047 > > > > -- > > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > > For more options, visit this group athttp:// > groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.
