While working on getting Joomla to work without core file hacks on App 
Engine, I ran into some trouble where it attempts to build paths based on 
the path the index.php file is in.

Specifically, it tries to send media files[images] to JPATH_BASE.'/media', 
cache files to JPATH_BASE.'/cache', and some log files to JPATH_BASE.'/log',

Rather then running with all that code imported from google cloud storage, 
I instead setup a stream morpher/wrapper.  By default it can self register 
itself to handle all streams beginning with ggsm:// and it can be assigned 
a set of simple pattern matching rules to access a different stream.

IE for any file that begins with ggsm://log/$pathname it will instead 
access ggslog://$pathname 
For anything going to ggsm://cache/$pathname it will instead use 
gggs://bucketname/$pathname

Where ggslog is the protocol string assigned to a simple syslog wrapper so 
all data sent to a log file will instead be routed to syslog and use the 
google logging api.

By the same token, gggs is routed to my own custom copy of the google cloud 
storage stream handler - the only changes there being that instead of a 
final class, my stream class is a public class that can be extended for 
other functionality - and I added append mode support for opening 
files[which I discovered on App Engine requires using the 
GoogleStorageClient classes in order to retrieve the data to append - you 
can't open a read stream to a path if you are in the middle of opening a 
write stream].
 
I've dumped the classes into a github repo if anyone else needs them:
https://github.com/garyamort/Stream-Morpher

One thing I'd like to do is expand the Google Cloud Storage class to use 
APC caching[as long as it is available] in addition to memcached.  I am 
thinking that for APC caching I would limit it to only 1 second - the files 
I am using don't generally change more quickly then once a second, and if 
they do it's ok when multiple instances are running for seperate instances 
to have stale data.  Since a single user will generally be routed to the 
same instance, anything cached in APC for them will be available there 
immediately - while memcached can provide up to 15 minutes of caching as a 
fallback, and if those fail the slower load from GS can be done.

Also it is relatively simple to support file locking and such via GS 
metadata.  Since each request has a unique request id assigned to it, it's 
easy to store GS metadata with the request id and time of file lock.  The 
lock function can limit honoring of locks to a small timeframe[say 5 
seconds] and if the lock was performed more then 5 seconds ago - there are 
a number of special metadata update parameters that can be set so that it 
will only update if the current data matches the previous data.

IE:  lockRequestId=1,lockedAt=12/12/12 12:12:12

The second request could try to update the meta data to:
lockRequestId=2, lockedAt=12/12/12 12:22/01 only if 
lockRequestId=1,lockedAt=12/12/12 12:12:12

So with competing requests attempting file locks, only the first one gets 
the lock - the second will fail and have to wait again.


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

Reply via email to