[google-appengine] Re: remote api, when?
Ryan mentioned it in the chat (see the transcript here in the group), he thinks it should be working already and there is an article coming soon. Nik On 21 Ян, 11:06, dobee berndd...@gmail.com wrote: hi all there is an undocumented remote api client in the 1.1.8 sdk which i tried, but it seems the protocoll buffer implementation is not available on appengine, just in the sdk. this error is raised when calling theremote_apihandler type 'exceptions.ImportError': cannot import name remote_api_pb Traceback (most recent call last): File /base/python_lib/versions/1/google/appengine/ext/remote_api/ handler.py, line 50, in module from google.appengine.ext.remote_apiimport remote_api_pb File /base/python_lib/versions/1/google/appengine/ext/remote_api/ __init__.py, line 72, in module from google.appengine.ext.remote_apiimport remote_api_pb type 'exceptions.ImportError': cannot import name remote_api_pb does anybody know when this feature will be enabled? thx in advance, BErnd --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Streaming support?
I don't think GAE will be usable for video streaming anytime soon. The platform is oriented towards doing stuff in small chunks, which doesn't fit video streaming. There is support for long running processes in development, but it is not even on the roadmap. Nik On 20 Ян, 12:15, alokiN okilje...@gmail.com wrote: Hi all. In the 1.7 SDK version docs, there were explicit notes indicating that the streaming was not supported. The new version indicates only that a certain type of streaming isn't supported because no data can be sent to the user before the handler exits. Does this mean that there is a way to use GAE for video streaming? If so, does anyone have some hints about all that: docs/links/anything. Thanks.. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] What means Deployment ???
Hello and good day to everybody Sorry by my english I have a question: What means Deployments It appear under Quota Details menu and the end of all of the page This number is the avalilable possible deployment a day ? Thanks With no more Bye bye --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: What means Deployment ???
On Thu, Jan 22, 2009 at 18:09, Yeradis yera...@gmail.com wrote: Hello and good day to everybody Sorry by my english I have a question: What means Deployments It appear under Quota Details menu and the end of all of the page This number is the avalilable possible deployment a day ? Thanks With no more Bye bye It's the number of times you can deploy your app to appengine's servers with appcfg.py each day. -- Joe --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] django + GAE + app-engine-patch + i18n (+ Mac)
Anybody have luck with this combo: django + GAE + app-engine-patch + i18n ? I'm on Mac OS X 10.5. First I had to get the gettext package from Fink. But now still when I run manage.py makemessages -l en, I get: Error: errors happened while running xgettext on __init__.py /bin/sh: xgettext: command not found Where would I get xgettext (what is it even), and how can I get i18n working? Thank you. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Caching using global variables
Hi there, I've been playing with caching using global variables (see http://tack.appspot.com/ for my plaything) and would like some input on the following approach: http://paste.blixt.org/3381 You would use the above code like this: def get_recent_posts(): if cache.recent_posts.is_current(): return cache.recent_posts.data posts = [] # ... Populate 'posts' here ... cache.recent_posts.set(posts) return posts I'm sure lots of you have experimented with caching in global variables. How often does the running instance shut down (requiring the cache to be rebuilt)? How many instances does it create during heavy load (each instance has separate variables of course, while memcache shares its data across all instances.) I've found that using global variables instead of memcache is considerably faster when reloading the same data a few times, but I cannot currently test this on a larger scale, which is why I'm turning to you guys. Thanks, Andreas Blixt --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Caching using global variables
The big difference between in-memory caching and memcache caching is that memcache data is shared between all of your running instances, but in-memory caching is on a per-instance basis. On Jan 22, 9:52 am, Blixt andreasbl...@gmail.com wrote: Hi there, I've been playing with caching using global variables (seehttp://tack.appspot.com/ for my plaything) and would like some input on the following approach:http://paste.blixt.org/3381 You would use the above code like this: def get_recent_posts(): if cache.recent_posts.is_current(): return cache.recent_posts.data posts = [] # ... Populate 'posts' here ... cache.recent_posts.set(posts) return posts I'm sure lots of you have experimented with caching in global variables. How often does the running instance shut down (requiring the cache to be rebuilt)? How many instances does it create during heavy load (each instance has separate variables of course, while memcache shares its data across all instances.) I've found that using global variables instead of memcache is considerably faster when reloading the same data a few times, but I cannot currently test this on a larger scale, which is why I'm turning to you guys. Thanks, Andreas Blixt --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Too many Indexes? runs fine on desk server not in production? how to debug?
Happy to debug but how? (well maybe not happy but willing)
I'm a real newbie at all this, but I did hack together a very modified
version on the bloog blog project which is now live at
http://tombrander.appspot.com/ It runs fine except that I can't post
even moderately long blog posts, other examples using this code have
longer posts than I, I have not (knowingly) gone near any of the code
involved in this area. On the post that is now 1st on the blog if I
add even one more word I generate the following error. Can someone
give me some advice how to begin the debug process, I'm new to both
app-engine and Python so don't have a clue as to how to start the
debug process, particularly since it works on my local server? Left a
message on the Bloog group but it is pretty low activity so no
response.
Error message:
01-21 07:50PM 40.818
Too many indexed properties for entity: app: tombrander,path
Element {type: Article,id: 6}
This index put it over the limit: entity_type: Article,ancestor:
false,Property { name: __searchable_text_index, direction:
ASCENDING},Property { name: __searchable_text_index, direction:
ASCENDING},Property { name: published, direction: DESCENDING}
Traceback (most recent call last):
File /base/python_lib/versions/1/google/appengine/ext/webapp/
__init__.py, line 500, in __call__
handler.post(*groups)
File /base/data/home/apps/tombrander/1.330872506639541542/handlers/
restful.py, line 109, in redirect_if_needed
self.put(*args, **kwargs)
File /base/data/home/apps/tombrander/1.330872506639541542/utils/
authorized.py, line 69, in check_login
handler_method(self, *args, **kwargs)
File /base/data/home/apps/tombrander/1.330872506639541542/handlers/
bloog/blog.py, line 496, in put
process_article_edit(handler = self, permalink = permalink)
File /base/data/home/apps/tombrander/1.330872506639541542/handlers/
bloog/blog.py, line 191, in process_article_edit
article.put()
File /base/python_lib/versions/1/google/appengine/ext/db/
__init__.py, line 657, in put
return datastore.Put(self._entity)
File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 162, in Put
raise _ToDatastoreError(err)
File /base/python_lib/versions/1/google/appengine/api/
datastore.py, line 1637, in _ToDatastoreError
raise errors[err.application_error](err.error_detail)
BadRequestError: Too many indexed properties for entity: app:
tombrander,path Element {type: Article,id: 6}
This index put it over the limit: entity_type: Article,ancestor:
false,Property { name: __searchable_text_index, direction:
ASCENDING},Property { name: __searchable_text_index, direction:
ASCENDING},Property { name: published, direction: DESCENDING}
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: is gaeutilities sessions the only 3rd party session manager?
I think it's a case of it's been that way for so long I haven't realized I need to change it. The put on every write to update the last activity, I mean. I'll look into modifying it so that it only writes when the session token changes. As for google though, every time I load up my igoogle page, I have cookie called SID for the www.google.com that changes each page view. This is really as it should be for real security, as even the 15 second window I have configured by default it more that large enough for someone to write a script very quickly to sniff and grab the cookie to be used to hijack the session. Modifying the last_activity to update less frequently is a good idea, I will try to get that in soon. On Jan 22, 2:08 pm, jeremy jeremy.a...@gmail.com wrote: Bowman - I think my issue with gaeutilities may largely be that the default settings seem excessive. Looking at the cookies google.com hands its visitors, their SID cookie lasts a good decade and seems to rotate every few hours, not seconds. So at the moment i'm thinking I'll see if changing the default satisfies me. Also, at the moment there's a datastore put() on every session-using request to (at a minimum) update the last activity time stamp. But if my SESSION_TOKEN_TTL is something like 1 hour, then actually i only need to update the last activity time stamp if the last timestamp was more than an hour ago. That would significantly reduce the amount of datastore writes, no? re: I'm just trying to figure out the value in the signed cookie approach, because if I can figure out a way for it to make sense I would consider moving gaeutilities to that approach. : Do you mean storing all session data in the clients cookie and using a signature to authenticate it? That approach makes me nervous, although I can imagine it's actually quite effective. You just store the session data in plain text but then also include a signature = SHA1 (session_data + secret_key_only_the_server_knows). That way the server can confirm it was the one who committed the values (and not the client messing with their own cookies). On Jan 21, 8:33 pm, bowman.jos...@gmail.com bowman.jos...@gmail.com wrote: Does beaker store all session information as cookies? I'm just trying to figure out the value in the signed cookie approach, because if I can figure out a way for it to make sense I would consider moving gaeutilities to that approach. gaeutilities stores only a temporary session token in the browser cookie store, all information is stored server side for security. Since I only have the one cookie, which is basically an id to find the session server side with, I don't see a way for this approach to keep a session from being hijacked. As in the end the 'hash-value' string could be hijacked and reused by another browser. The performance issue with gaeutililites, and why Jeremy is looking for options, is the approach I've taken to securing the session id requires frequent put() operations. I've seen some other blogs who've mentioned this is a performance bottleneck for gae, but I haven't been able to come up with another approach that will not sacrifice reliability or performance. Basically I rotate the session id in the cookie every 5 seconds (this length is configurable). I store the current plus previous 2 tokens in a ListProperty in order to manage sites using AJAX type functionality. A session token is generally good for 15 seconds with this approach. Longer if there are not interim requests generating new tokens, as a new token is only generated when the 5 second TTL is hit. So you come back 2 minutes later, and the session token is still valid for that request, just regenerated on the request. As you can imagine, this is a lot of put operations for each session. I deemed just using memcache to not be a viable alternative because it's documented that it's a volatile storage, it can be cleared at any time. I do use memcache for speeding up reads to the session data, limiting read access to the datastore. I'm definitely open to suggestions for a method to do this without all the puts. The key being the reliability and security are more important than performance. In the end it's cheaper to pay for the CPU usage than the risks by allowing users to have their sessions easily hijacked, depending on the application, or having sessions randomly lost due to the storage medium not being permanent enough. On Jan 21, 6:27 pm, jeremy jeremy.a...@gmail.com wrote: thanks for the suggestions. does beaker really work out of the box with gae? On Jan 21, 1:06 am, Ian Bicking i...@colorstudy.com wrote: On Tue, Jan 20, 2009 at 10:40 PM, jeremy jeremy.a...@gmail.com wrote: can anyone recommend / mention a session manager other than the one in gaeutilities? Beaker works with GAE:http://beaker.groovie.org/ -- Ian Bicking |
[google-appengine] Re: Caching using global variables
Maybe you can use both your approach+memcache. Look for stuff in the local cache first. If not found, go to memcache, if not found, then load from the datastore. This can be neatly hidden in a decorator. See: http://appengine-cookbook.appspot.com/recipe/decorator-to-getset-from-the-memcache-automatically -Mahmoud On Jan 22, 11:13 am, Blixt andreasbl...@gmail.com wrote: Yup, I know, that's what brings forth my questions about the large- scale behavior of in-memory caching. Will the instances be recreated too often for it to be more effective than memcache? Will there be too many instances at a time for it to be more effective? I also rewrote the code to use proper programming conventions, before someone tells me off for my unconventional code ;) The new code is here:http://paste.blixt.org/3382 - Andreas Blixt On Jan 22, 5:08 pm, Tony Arkles t.ark...@gmail.com wrote: The big difference between in-memory caching and memcache caching is that memcache data is shared between all of your running instances, but in-memory caching is on a per-instance basis. On Jan 22, 9:52 am, Blixt andreasbl...@gmail.com wrote: Hi there, I've been playing with caching using global variables (seehttp://tack.appspot.com/ for my plaything) and would like some input on the following approach:http://paste.blixt.org/3381 You would use the above code like this: def get_recent_posts(): if cache.recent_posts.is_current(): return cache.recent_posts.data posts = [] # ... Populate 'posts' here ... cache.recent_posts.set(posts) return posts I'm sure lots of you have experimented with caching in global variables. How often does the running instance shut down (requiring the cache to be rebuilt)? How many instances does it create during heavy load (each instance has separate variables of course, while memcache shares its data across all instances.) I've found that using global variables instead of memcache is considerably faster when reloading the same data a few times, but I cannot currently test this on a larger scale, which is why I'm turning to you guys. Thanks, Andreas Blixt --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Indexes aren't being used by the server [was: Re: can't deploy index.yaml]
finally got it to really work by removing the - name __searchable_text_index line from index.yaml there sure seem to be a lot of undocumented features in this process On Jan 22, 7:33 am, jones34 ljw1...@gmail.com wrote: Argh. Spoke too soon. The deploy now _seems_ to work, but I still get the index not found error when I try to run the app on the server. ** Here's the index.yaml: indexes: - kind: Note properties: - name: create_date_time direction: desc - name: __searchable_text_index ** Here's the transcript from the upload: *** *** Running appfg.py with the following flags: --no_cookies --email=lawrence.wh...@deathrayresearch.com --passin update Scanning files on local disk. Initiating update. Password for lawrence.wh...@deathrayresearch.com: Cloning 4 static files. Cloning 3 application files. Closing update. Uploading index definitions. If deploy fails you might need to 'rollback' manually. The Make Symlinks... menu option can help with command-line work. *** appcfg.py has finished with exit code 0 *** *** Here's the traceback I get: *** Traceback (most recent call last): File /base/python_lib/versions/1/google/appengine/ext/webapp/ __init__.py, line 498, in __call__ handler.get(*groups) File /base/data/home/apps/3323808/1.330880695773925261/main.py, line 46, in get notes = notes_query.fetch(20) File /base/python_lib/versions/1/google/appengine/ext/db/ __init__.py, line 1378, in fetch raw = self._get_query().Get(limit, offset) File /base/python_lib/versions/1/google/appengine/api/ datastore.py, line 938, in Get return self._Run(limit, offset)._Next(limit) File /base/python_lib/versions/1/google/appengine/api/ datastore.py, line 887, in _Run str(exc) + '\nThis query needs this index:\n' + yaml) NeedIndexError: no matching index found. This query needs this index: - kind: Note properties: - name: create_date_time direction: desc What's weird In the console I see two versions of each index Entity and Indexes Status Note __searchable_text_index ▲ , create_date_time ▼ Serving create_date_time ▼ , __searchable_text_index ▲ Serving I believe this is because I reversed the order they're listed in the index.yaml, but if they're there why aren't they being found? Can I delete one of them somehow? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Chinese characters caus es ServerError 网页中有汉字，就会产生错误
(I edited the subject, many English speakers may not know this post is
written in Chinese)
I have tried this on Linux if I have this script in GB2312, I got no
error but a warning:
WARNING 2009-01-22 21:55:14,652 __init__.py] Response written is
not UTF-8: 'utf8' codec can't decode byte 0xa3 in position 35:
unexpected code byte
So I guess that is possible that your python script is not in UTF-8
but GB2312 since you are using Simplified Chinese. If you save it as
UTF-8, you shouldn't have any problem with Chinese.
If you still want to retain the encoding, you can modify you code
self.response.out.write('htmlbodypHello, have no thing，有了汉字就不
灵了？/p/body/html'.decode('GB2312'))
You have to decode every output statement.
On Jan 22, 5:53 pm, 网页中有汉字，就会产生错误，不知是啥原因 hjma...@gmail.com wrote:
this code right in local development environment, but error on update
to server.
import wsgiref.handlers
from google.appengine.ext import webapp
class MainPage(webapp.RequestHandler):
def get(self):
self.response.headers['Content-Type'] = 'text/html; charset=utf-8'
self.response.out.write('htmlbodypHello, have no thing，有了汉字就不
灵了？/p/body/html')
def main():
application = webapp.WSGIApplication(
[('/', MainPage)],
debug=True)
wsgiref.handlers.CGIHandler().run(application)
if __name__ == __main__:
main()
Error: Server Error
The server encountered an error and could not complete your request.
If the problem persists, please report your problem and mention this
error message and the query that caused it.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: How expensive are queries
Using get(key) is faster then a Model.gql().fetch() or Model.all().fetch(). db.get(key_list) is faster. But still you have to go to the datastore and that takes time. But getting data from memcache is very fast. Why don't you cache the result from the template rendering? The only extra Python code is to keep track of objects in a dictionary to make sure I have only one copy of every object. Do you have to merge the results from a couple of queries? I don't know how efficient Django as a framework is. I only use the template rendering. And that is not very expensive. 2009/1/22 Roman roman.budzianow...@gmail.com: There is not much going on in the Python code. Based on URL I retrieve a few objects and setup a context for a template. I memcache either objects or keys (assuming that get(key) is as fast as getting an object from memcache). The only extra Python code is to keep track of objects in a dictionary to make sure I have only one copy of every object. I am using django dispatch loop and noticed that a redirect request alone (too add a /) may take 500 ms. A request that results in all objects or keys coming from memcache (say 10) still can take sometimes 1600ms, which gives me a red warning (I think 1500 is the threshold). I wonder how accurate is the time sampling. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: is gaeutilities sessions the only 3rd party session manager?
Ok. I actually modified Session.__init__ locally to do the
last_activity on sid rotation (i also refactored it a bit to reduce
repeated code blocks). Regarding google.com's SID cookie - i'm not
seeing the sid update within minutes. I'm not sure why yours rotates
so quickly, but it's something entirely configurable in your code so
it shouldn't matter. Anyway, here's my version of Session.__init__ :
def __init__(self, cookie_path=DEFAULT_COOKIE_PATH,
cookie_name=COOKIE_NAME,
session_expire_time=SESSION_EXPIRE_TIME,
clean_check_percent=CLEAN_CHECK_PERCENT,
integrate_flash=INTEGRATE_FLASH, check_ip=CHECK_IP,
check_user_agent=CHECK_USER_AGENT,
set_cookie_expires=SET_COOKIE_EXPIRES,
session_token_ttl=SESSION_TOKEN_TTL):
Initializer
Args:
cookie_name: The name for the session cookie stored in the
browser.
session_expire_time: The amount of time between requests
before the
session expires.
clean_check_percent: The percentage of requests the will
fire off a
cleaning routine that deletes stale session data.
integrate_flash: If appengine-utilities flash utility should
be
integrated into the session object.
check_ip: If browser IP should be used for session
validation
check_user_agent: If the browser user agent should be used
for
sessoin validation.
set_cookie_expires: True adds an expires field to the cookie
so
it saves even if the browser is closed.
session_token_ttl: Number of sessions a session token is
valid
for before it should be regenerated.
self.cookie_path = cookie_path
self.cookie_name = cookie_name
self.session_expire_time = session_expire_time
self.clean_check_percent = clean_check_percent
self.integrate_flash = integrate_flash
self.check_user_agent = check_user_agent
self.check_ip = check_ip
self.set_cookie_expires = set_cookie_expires
self.session_token_ttl = session_token_ttl
# make sure the page is not cached in the browser
self.no_cache_headers()
Check the cookie and, if necessary, create a new one.
self.cache = {}
self.sid = None
string_cookie = os.environ.get('HTTP_COOKIE', '')
self.cookie = Cookie.SimpleCookie()
self.output_cookie = Cookie.SimpleCookie()
self.cookie.load(string_cookie)
dirty = False
# check for existing cookie
if self.cookie.get(cookie_name):
self.sid = self.cookie[cookie_name].value
self.session = self._get_session() # will return None if
sid expired
else:
self.sid = self.new_sid()
self.session = None
if self.session is None:
# start a new session if there is None.
self.sid = self.new_sid()
self.session = _AppEngineUtilities_Session()
if 'HTTP_USER_AGENT' in os.environ:
self.session.ua = os.environ['HTTP_USER_AGENT']
else:
self.session.ua = None
if 'REMOTE_ADDR' in os.environ:
self.session.ip = os.environ['REMOTE_ADDR']
else:
self.session.ip = None
self.session.sid = [self.sid]
dirty = True
else:
# check the age of the token to determine if a new one
# is required
duration = datetime.timedelta
(seconds=self.session_token_ttl)
session_age_limit = datetime.datetime.now() - duration
if self.session.last_activity session_age_limit:
self.sid = self.new_sid()
if len(self.session.sid) 2:
self.session.sid.remove(self.session.sid[0])
self.session.sid.append(self.sid)
dirty = True
else:
self.sid = self.session.sid[-1]
self.output_cookie[cookie_name] = self.sid
self.output_cookie[cookie_name]['path'] = cookie_path
if set_cookie_expires:
self.output_cookie[cookie_name]['expires'] =
self.session_expire_time
self.cache['sid'] = pickle.dumps(self.sid)
if dirty:
self.session.put()
#self.cookie.output()
print self.output_cookie.output()
# fire up a Flash object if integration is enabled
if self.integrate_flash:
import flash
self.flash = flash.Flash(cookie=self.cookie)
# randomly delete old stale sessions in the datastore (see
# CLEAN_CHECK_PERCENT variable)
if random.randint(1, 100) CLEAN_CHECK_PERCENT:
self._clean_old_sessions()
On Jan 22, 4:38 pm, bowman.jos...@gmail.com
bowman.jos...@gmail.com wrote:
I think it's a case of it's been that way for so long I
[google-appengine] Re: is gaeutilities sessions the only 3rd party session manager?
I've gone with a different approach that currently achieves similar
results, that's now available in the trunk. A new variable,
last_activity_update has been added. It's the amount of seconds that
needs to pass before that field needs to be updated by doing a put().
It defaults to 60 seconds, which of course is longer than the duration
before a put is required to update the session token with the default
settings.
This will allow developers who wish to lengthen their
session_token_ttl to a larger interval to still get their
last_activity update in, useful for session timing. It too is
customizable so for developers who have no use for this field can set
it to a large enough number to be irrelevant.
I'm trying to flush out an idea I have to limit the amount of writes
for the token even further, but am still researching it. If I figure
it out I will get it in and do another release. Otherwise I will
release what's there now. Before any release I want to go over the
refactoring you did as it does look more efficient than what I
currently have, thanks.
On Jan 22, 6:31 pm, jeremy jeremy.a...@gmail.com wrote:
Ok. I actually modified Session.__init__ locally to do the
last_activity on sid rotation (i also refactored it a bit to reduce
repeated code blocks). Regarding google.com's SID cookie - i'm not
seeing the sid update within minutes. I'm not sure why yours rotates
so quickly, but it's something entirely configurable in your code so
it shouldn't matter. Anyway, here's my version of Session.__init__ :
def __init__(self, cookie_path=DEFAULT_COOKIE_PATH,
cookie_name=COOKIE_NAME,
session_expire_time=SESSION_EXPIRE_TIME,
clean_check_percent=CLEAN_CHECK_PERCENT,
integrate_flash=INTEGRATE_FLASH, check_ip=CHECK_IP,
check_user_agent=CHECK_USER_AGENT,
set_cookie_expires=SET_COOKIE_EXPIRES,
session_token_ttl=SESSION_TOKEN_TTL):
Initializer
Args:
cookie_name: The name for the session cookie stored in the
browser.
session_expire_time: The amount of time between requests
before the
session expires.
clean_check_percent: The percentage of requests the will
fire off a
cleaning routine that deletes stale session data.
integrate_flash: If appengine-utilities flash utility should
be
integrated into the session object.
check_ip: If browser IP should be used for session
validation
check_user_agent: If the browser user agent should be used
for
sessoin validation.
set_cookie_expires: True adds an expires field to the cookie
so
it saves even if the browser is closed.
session_token_ttl: Number of sessions a session token is
valid
for before it should be regenerated.
self.cookie_path = cookie_path
self.cookie_name = cookie_name
self.session_expire_time = session_expire_time
self.clean_check_percent = clean_check_percent
self.integrate_flash = integrate_flash
self.check_user_agent = check_user_agent
self.check_ip = check_ip
self.set_cookie_expires = set_cookie_expires
self.session_token_ttl = session_token_ttl
# make sure the page is not cached in the browser
self.no_cache_headers()
Check the cookie and, if necessary, create a new one.
self.cache = {}
self.sid = None
string_cookie = os.environ.get('HTTP_COOKIE', '')
self.cookie = Cookie.SimpleCookie()
self.output_cookie = Cookie.SimpleCookie()
self.cookie.load(string_cookie)
dirty = False
# check for existing cookie
if self.cookie.get(cookie_name):
self.sid = self.cookie[cookie_name].value
self.session = self._get_session() # will return None if
sid expired
else:
self.sid = self.new_sid()
self.session = None
if self.session is None:
# start a new session if there is None.
self.sid = self.new_sid()
self.session = _AppEngineUtilities_Session()
if 'HTTP_USER_AGENT' in os.environ:
self.session.ua = os.environ['HTTP_USER_AGENT']
else:
self.session.ua = None
if 'REMOTE_ADDR' in os.environ:
self.session.ip = os.environ['REMOTE_ADDR']
else:
self.session.ip = None
self.session.sid = [self.sid]
dirty = True
else:
# check the age of the token to determine if a new one
# is required
duration = datetime.timedelta
(seconds=self.session_token_ttl)
session_age_limit = datetime.datetime.now() - duration
if self.session.last_activity session_age_limit:
[google-appengine] Marzia Please Help with Account ID Problems!
Hi Marzia, I cannot successfully invite developers to an app using Google Apps authentication: when the invited developer clicks on the Apps confirmation link, they get an error page. If instead of inviting a developer from the Apps domain to which I restricted the app, I invite a developer from a different Apps domain, it works! It's possible I unwittingly screwed up this whole account. One app accepts Google IDs, and the other accepts Google App IDs. But I just realized that the although the entire account is owned by a standard Google ID, I've been using as a developer login an account that exists both as a Google ID and as a Google Apps ID. I sure could use some help straightening this out! If we could take this offline, I can give you the account IDs and email addresses involved. Thanks, Dan --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Is there some sort of cookie issued by google apps can take advantage of for sessions?
There's been some ongoing discussion about the approach I and others have been taking to session management in our appengine applications. I always rank security over performance, but with how heavy datastore writes are, this can be problematic and eventually expensive for applications. I've been thinking though, since users can log in with their Google accounts using the User API google offers, I was wondering if there was a layer to this that could be tied into for all applications, whether they choose to implement the full stack for user management? Since we can host our own domains, I'm assuming that Google has figured out a way to tie their own cookies into being readable through the stack somehow? While I respect the fact you may not want to go into detail how the full process works, I was wondering if the User API could be expanded to allow applications to whatever identifier you're using client side to uniquely identify browser sessions? This would then allow the various developers working on their own session implementations to build off of that to maintain session state, and gain the security of a real revolving session token that doesn't require a put. A full API for session data management doesn't need to be provided, thought it would be nice. Just access to a token that I'm assuming somehow exists. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: is gaeutilities sessions the only 3rd party session manager?
Hmm, I'm not sure what session timing is.
I have an idea to reduce writes. Instead of updating the sid of every
session individually, give each session a random value between 0 and
C, and have one application-wide value R randomized every
session_token_ttl seconds to an integer between 0 and C, then hand the
client the value of this as a token:
t = (session_id+R)%C
then when a client hands the server a token, you can compute
session_id = (t-R)%C
(you can store the last 3 values of R as is done now for each sessions
sid)
I'm pretty sure there's no analysis attack that would allow a client
to figure out either R at any moment or their own (constant)
session_id. But, i could be wrong about that :\ ... The advantage
would be you're only updating a single datastore entity every
session_token_ttl.
On Jan 22, 9:24 pm, bowman.jos...@gmail.com
bowman.jos...@gmail.com wrote:
I've gone with a different approach that currently achieves similar
results, that's now available in the trunk. A new variable,
last_activity_update has been added. It's the amount of seconds that
needs to pass before that field needs to be updated by doing a put().
It defaults to 60 seconds, which of course is longer than the duration
before a put is required to update the session token with the default
settings.
This will allow developers who wish to lengthen their
session_token_ttl to a larger interval to still get their
last_activity update in, useful for session timing. It too is
customizable so for developers who have no use for this field can set
it to a large enough number to be irrelevant.
I'm trying to flush out an idea I have to limit the amount of writes
for the token even further, but am still researching it. If I figure
it out I will get it in and do another release. Otherwise I will
release what's there now. Before any release I want to go over the
refactoring you did as it does look more efficient than what I
currently have, thanks.
On Jan 22, 6:31 pm, jeremy jeremy.a...@gmail.com wrote:
Ok. I actually modified Session.__init__ locally to do the
last_activity on sid rotation (i also refactored it a bit to reduce
repeated code blocks). Regarding google.com's SID cookie - i'm not
seeing the sid update within minutes. I'm not sure why yours rotates
so quickly, but it's something entirely configurable in your code so
it shouldn't matter. Anyway, here's my version of Session.__init__ :
def __init__(self, cookie_path=DEFAULT_COOKIE_PATH,
cookie_name=COOKIE_NAME,
session_expire_time=SESSION_EXPIRE_TIME,
clean_check_percent=CLEAN_CHECK_PERCENT,
integrate_flash=INTEGRATE_FLASH, check_ip=CHECK_IP,
check_user_agent=CHECK_USER_AGENT,
set_cookie_expires=SET_COOKIE_EXPIRES,
session_token_ttl=SESSION_TOKEN_TTL):
Initializer
Args:
cookie_name: The name for the session cookie stored in the
browser.
session_expire_time: The amount of time between requests
before the
session expires.
clean_check_percent: The percentage of requests the will
fire off a
cleaning routine that deletes stale session data.
integrate_flash: If appengine-utilities flash utility should
be
integrated into the session object.
check_ip: If browser IP should be used for session
validation
check_user_agent: If the browser user agent should be used
for
sessoin validation.
set_cookie_expires: True adds an expires field to the cookie
so
it saves even if the browser is closed.
session_token_ttl: Number of sessions a session token is
valid
for before it should be regenerated.
self.cookie_path = cookie_path
self.cookie_name = cookie_name
self.session_expire_time = session_expire_time
self.clean_check_percent = clean_check_percent
self.integrate_flash = integrate_flash
self.check_user_agent = check_user_agent
self.check_ip = check_ip
self.set_cookie_expires = set_cookie_expires
self.session_token_ttl = session_token_ttl
# make sure the page is not cached in the browser
self.no_cache_headers()
Check the cookie and, if necessary, create a new one.
self.cache = {}
self.sid = None
string_cookie = os.environ.get('HTTP_COOKIE', '')
self.cookie = Cookie.SimpleCookie()
self.output_cookie = Cookie.SimpleCookie()
self.cookie.load(string_cookie)
dirty = False
# check for existing cookie
if self.cookie.get(cookie_name):
self.sid = self.cookie[cookie_name].value
self.session = self._get_session() # will return None if
sid expired
else:
[google-appengine] Re: is gaeutilities sessions the only 3rd party session manager?
By session timing I was referring to how long a session is valid for. For example, a session is valid for 2 hours. This means the session is valid for last_activity + 2 hours. Completely separate from the session token process. So, if you leave the site and come back 90 minutes later, what happens is. Session token is handed to server. Session token is verified as a valid session token, because now - last_activity is less than 120 minutes. Your session token is regenerated because it's older than 5 seconds. Because of that put operation your last_activity gets updated as well. What I see as a concern with your approach is what happens when the server wide variable R gets out of sync with someone's version that was crypted based off of it? The original reason the 3 valid token set Now, one thought I've had is that really, sessions should be used for logged in users in most cases. So one thing I will add sooner rather than later is a class that will help you determine if you need to use session at all. class session_manager() def session_exists(): Check and see if the browser has a session cookie. Note, because of the expire time set on the cookie by session, may be all you ever need. def delete_session_cookie(): Clears the session cookie. Useful for if you check to see if a session cookie exists, but the user is not logged in so you want to clear it. I'm using appenginepatch myself, with the django middleware bundled with gaeutilities. I'll see what I can do about patching this to work with this class in place when I do it. Other than that, some approaches I'm considering. 1: I've made a request seeing if Google has some sort of session id functionality they can give us. I think this should be possible based off the fact they offer their user api. There has to be something in the browser the application can access to tie that browser to the logged in user session. 2: Going with memcache for the session token, with a datastore fallback. Confirm memcache is working using the method to return the memcache stats. If it is, then only use memcache. If it is down, use the datastore. Of course when it goes down this would null out all active sessions, which is undesirable. I have already seen a case where memcache was down for applications, so this is a realistic possibility. 3: A hybrid datastore/memcache solution with 2 cookies.Since there is the 15 second window already. Store every third token in the database and as an alternate cookie. Say session_id, and ds_session_id. Include in the value a count, value--count, split on -- to get the token value and the count. If the count is 2, reset to 1 and set up the ds_session_id cookie and store the value in the datastore. This would be a write every 15 seconds through constant refreshes. Lengthening the session token ttl will help reduce the amount writes, but will increase the attack vector of spoofing. Also, so many people are behind NAT's and/or using DHCP clients, I may turn off IP verification by default, though will leave the option in to use it. On Jan 22, 11:12 pm, jeremy jeremy.a...@gmail.com wrote: Hmm, I'm not sure what session timing is. I have an idea to reduce writes. Instead of updating the sid of every session individually, give each session a random value between 0 and C, and have one application-wide value R randomized every session_token_ttl seconds to an integer between 0 and C, then hand the client the value of this as a token: t = (session_id+R)%C then when a client hands the server a token, you can compute session_id = (t-R)%C (you can store the last 3 values of R as is done now for each sessions sid) I'm pretty sure there's no analysis attack that would allow a client to figure out either R at any moment or their own (constant) session_id. But, i could be wrong about that :\ ... The advantage would be you're only updating a single datastore entity every session_token_ttl. On Jan 22, 9:24 pm, bowman.jos...@gmail.com bowman.jos...@gmail.com wrote: I've gone with a different approach that currently achieves similar results, that's now available in the trunk. A new variable, last_activity_update has been added. It's the amount of seconds that needs to pass before that field needs to be updated by doing a put(). It defaults to 60 seconds, which of course is longer than the duration before a put is required to update the session token with the default settings. This will allow developers who wish to lengthen their session_token_ttl to a larger interval to still get their last_activity update in, useful for session timing. It too is customizable so for developers who have no use for this field can set it to a large enough number to be irrelevant. I'm trying to flush out an idea I have to limit the amount of writes for the token even further, but am still researching it. If I figure it out I will get it in and do another
[google-appengine] Re: is gaeutilities sessions the only 3rd party session manager?
What I see as a concern with your approach is what happens when the server wide variable R gets out of sync with someone's version that was crypted based off of it? The original reason the 3 valid token set that's why i mention that you can store the last 3 values of R as is done now for each sessions sid - so all 3 would be tried as is done now with the sid list on each session entity. you could also count how often R has been randomized and hand this iteration index to the client as part of the token. i'm not sure about going primarily with memcache - isn't memcache designed only to be a caching layer? memcache isn't volatile in the sense of being either up or down. rather, it throws out stored data randomly as far as the developer is concerned as load increases. On Jan 23, 1:37 am, bowman.jos...@gmail.com bowman.jos...@gmail.com wrote: By the way, I really am not concerned with analysis attacks. It's sniffing/spoofing attacks that are most common for session hijacking. I simply sniff the network and find out what the name and value of the cookie are, and what user agent you are sending. I then duplicate those 2 things and if I'm behind the same NAT as you, I have your session loaded up in my browser. If I'm any good a social hacking, I set my page to auto refresh and then distract you by talking to you until I have your full session by rotating the session tokens past the point of where the one in your browser is invalid, and more than likely the application will make log back in, without logging me out. This is where you may want to consider tieing the session directly to a user account, so a user can only be logged in once at any time, and logging invalidates the current log in if it exists, ie: active_session field on your user model. Just some late night thoughts when I really should be asleep. On Jan 22, 11:12 pm, jeremy jeremy.a...@gmail.com wrote: Hmm, I'm not sure what session timing is. I have an idea to reduce writes. Instead of updating the sid of every session individually, give each session a random value between 0 and C, and have one application-wide value R randomized every session_token_ttl seconds to an integer between 0 and C, then hand the client the value of this as a token: t = (session_id+R)%C then when a client hands the server a token, you can compute session_id = (t-R)%C (you can store the last 3 values of R as is done now for each sessions sid) I'm pretty sure there's no analysis attack that would allow a client to figure out either R at any moment or their own (constant) session_id. But, i could be wrong about that :\ ... The advantage would be you're only updating a single datastore entity every session_token_ttl. On Jan 22, 9:24 pm, bowman.jos...@gmail.com bowman.jos...@gmail.com wrote: I've gone with a different approach that currently achieves similar results, that's now available in the trunk. A new variable, last_activity_update has been added. It's the amount of seconds that needs to pass before that field needs to be updated by doing a put(). It defaults to 60 seconds, which of course is longer than the duration before a put is required to update the session token with the default settings. This will allow developers who wish to lengthen their session_token_ttl to a larger interval to still get their last_activity update in, useful for session timing. It too is customizable so for developers who have no use for this field can set it to a large enough number to be irrelevant. I'm trying to flush out an idea I have to limit the amount of writes for the token even further, but am still researching it. If I figure it out I will get it in and do another release. Otherwise I will release what's there now. Before any release I want to go over the refactoring you did as it does look more efficient than what I currently have, thanks. On Jan 22, 6:31 pm, jeremy jeremy.a...@gmail.com wrote: Ok. I actually modified Session.__init__ locally to do the last_activity on sid rotation (i also refactored it a bit to reduce repeated code blocks). Regarding google.com's SID cookie - i'm not seeing the sid update within minutes. I'm not sure why yours rotates so quickly, but it's something entirely configurable in your code so it shouldn't matter. Anyway, here's my version of Session.__init__ : def __init__(self, cookie_path=DEFAULT_COOKIE_PATH, cookie_name=COOKIE_NAME, session_expire_time=SESSION_EXPIRE_TIME, clean_check_percent=CLEAN_CHECK_PERCENT, integrate_flash=INTEGRATE_FLASH, check_ip=CHECK_IP, check_user_agent=CHECK_USER_AGENT, set_cookie_expires=SET_COOKIE_EXPIRES, session_token_ttl=SESSION_TOKEN_TTL): Initializer Args: cookie_name: The name for the session cookie stored
[google-appengine] Re: remote api, when?
wow, nice to hear ... this feature really rocks! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: remote api, when?
and thanks for your mention~ On Thu, Jan 22, 2009 at 11:20 PM, dobee berndd...@gmail.com wrote: wow, nice to hear ... this feature really rocks! -- Stay hungry,Stay foolish. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
