Another technique for cache busting is URL rewriting within the path to the resources, which has some benefits over the query string usage in certain cases.
The Python AE runtime supports URL mapping for static resources which can make this even less painful and allows long expire times - so for example you could generate all static URLs like this: /static/[app version from environment]/css/foo.css and have it map to the static/your/foo.css withint your appengine upload This has an advantage in the case of css because then all your resources referenced from the CSS (as long as they're using relative URLs under the same static directory) are also affected by the path versionion. So a background-image: url(../images/bar.jpeg) ; declaration in the foo.css above would cause the browser to request the resource with the injected app version /static/[app version]/ images/bar.jpeg. This would not happen with the query string cache busting technique unless you willing to parse your css dynamically (and then you're not using the static file appengine servers). Using this technique you can feel free to set your static file caching headers to large values (months to a year) to get a better client experience without sacrificing the ability to update your app at will (your app version id will change everytime you upload, which creates a new URL and new set of cacheable objects to the browser) Unfortunately, the Java AE runtime does not have a similar support for static file URL rewriting - I've done a feature request here (PLEASE STAR! :) ) http://code.google.com/p/googleappengine/issues/detail?id=2070 So, to get the same behavior and peformance benefit of hitting the AE static file servers, you'd have to physically create a versioned directory during the build process before uploading to appengine . You could do the same thing for the python as well, but the URL mapping feature is so easy to use I've never wanted to do so. - Donovan On Jun 3, 2:32 pm, "Ikai L (Google)" <ika...@google.com> wrote: > Tim, > > As a general rule, web developers should *always* tag assets with a version > string. Besides our caching infrastructure (which I am still investigating), > it's the single most reliable way to invalidate browser, web accelerator, > ISP and proxy caches, which may or may not respect the headers you set. > Headers aren't a guarantee anybody downstream will respect your invalidation > strategy - headers are a way to, in the best case scenario, reduce HTTP > calls to your assets server. This is why most web frameworks will append at > least a timestamp or hash to asset tags. The simplest way is to do this: > > "/js/somefile.js?v=123" > > You won't have to rename the file - most caching mechanisms key on the > unmodified URL string. > > I know this might be painful in the short term, but trust me, this will save > you a lot of headaches in the long term when users' computers start doing > crazy things with cached assets. In the meantime, I'll try to figure out > what's going on, but I haven't yet been able to reproduce the issue in my > own applications no matter what I do. At the very least, I hope to provide > an explanation of expected behavior. > > > > On Wed, Jun 2, 2010 at 4:37 PM, Tim Hoffman <zutes...@gmail.com> wrote: > > Hi Ikai > > > Here are bits of our app.yaml > > > application: psc-dev1 > > version: 1-0-2 > > runtime: python > > api_version: 1 > > > default_expiration: "4d 5h" > > > - url: /images > > static_dir: images > > expiration: "30d" > > > - url: /css > > static_dir: css > > expiration: "30d" > > > - url: /js > > static_dir: js > > expiration: "30d" > > > The main difference is we have explicit expiration set on the static > > handlers. > > > Ikae one thing you didn't show in the your headers was the Etag. As > > I understand > > it, the Etag value is being set as a side affect of the deploying new > > versions. > > It seems the reverse proxy for app domain maps isn't honoring the Etag > > for cache control. > > > This must be a fairly recent phenomenon as we have never experienced > > this issue > > withwww.polytechnic.wa.edu.aubefore. (I can't show you it in action > > at the moment as > > we have serious site problems trying to run the new version). > > > We can't very well wait for the cache's to expire. (I think we will > > reduce the time > > somewhat, but even waiting a few days for the cache to expire won't > > work. > > The site is broken without the new js/css, so I can't leave the new > > version active whilst > > waiting for things to expire) > > > As to the notion of using a "cache-buster" that will mean effectively > > changing > > css/js paths in our templates and re-deploying the app. I suppose we > > will have to > > go with path at the moment. I am in Australia and I had gone to bed > > (midnight) > > before you guys cam online, and I need to get this new version > > deployed. However > > I don't believe this is a good solution. It effectively means the > > developers (me and a few > > other people) need to manage a changing version string on the end of > > all css/js static urls > > for each deployment, (if these change) each time we want to deploy/ > > update our code. > > > Thanks > > > Tim > > > On Jun 3, 4:38 am, "Ikai L (Google)" <ika...@google.com> wrote: > > > I can't reproduce this. Here's my YAML file: > > > > application: ikailan-com > > > version: 1 > > > runtime: python > > > api_version: 1 > > > default_expiration: "1d" > > > > handlers: > > > - url: / > > > script: main.py > > > > - url: /assets > > > static_dir: assets > > > > Can you guys post your app.yaml? > > > > On Wed, Jun 2, 2010 at 1:04 PM, Ikai L (Google) <ika...@google.com> > > wrote: > > > > > In the meantime, I'll investigate whether these headers are being > > > > implicitly or incorrectly set. > > > > > On Wed, Jun 2, 2010 at 12:37 PM, J <j.si...@earlystageit.com> wrote: > > > > >> Thanks, Ikai, for your help. It is much appreciated. > > > > >> We'll use a cache buster for now. > > > > >> On Jun 2, 3:27 pm, "Ikai L (Google)" <ika...@google.com> wrote: > > > >> > Okay, looks like the Google Front-End is kicking in to cache your > > stuff > > > > >> > Server Google Frontend > > > >> > Content-Length 2834 > > > >> > Age 41 > > > >> > Cache-Control public, max-age=600 > > > > >> > Are you guys setting this header anywhere? Unfortunately, there's no > > way > > > >> to > > > >> > invalidate items in the frontend cache, so you'll have to use a > > cache > > > >> buster > > > >> > or wait for the TTL to expire. > > > > >> > On Wed, Jun 2, 2010 at 12:22 PM, J <j.si...@earlystageit.com> > > wrote: > > > >> > > That's interesting, Rafael. I wonder why it serves correctly for > > you. > > > > >> > > They are still different from my PoV. > > > > >> > > The bad side headers: > > > >> > > Etag "74EQOA" > > > >> > > Date Tue, 01 Jun 2010 18:12:57 GMT > > > >> > > Expires Sat, 05 Jun 2010 02:12:57 GMT > > > >> > > Content-Type text/css > > > >> > > Content-Encoding gzip > > > >> > > Server Google Frontend > > > >> > > Content-Length 2820 > > > >> > > Cache-Control public, max-age=288000 > > > >> > > Age 90134 > > > >> > > X-XSS-Protection 0 > > > > >> > > The good side headers: > > > >> > > Etag "7xGL5w" > > > >> > > Date Wed, 02 Jun 2010 19:13:00 GMT > > > >> > > Expires Wed, 02 Jun 2010 19:12:04 GMT > > > >> > > Content-Type text/css > > > >> > > Content-Encoding gzip > > > >> > > Server Google Frontend > > > >> > > Content-Length 2834 > > > >> > > Age 41 > > > >> > > Cache-Control public, max-age=600 > > > > >> > > Short of using a cache-buster, is there a setting I can use on GAE > > or > > > >> > > in my app.yaml file? > > > > >> > > On Jun 2, 3:01 pm, "Ikai L (Google)" <ika...@google.com> wrote: > > > >> > > > I wonder if there is some layer of the infrastructure that is > > > >> performing > > > >> > > > caching without you guys having opted-in, hence the reason why a > > > >> > > > cache-buster like ?v=something works. Can you guys confirm? Also > > - > > > >> are > > > >> > > you > > > >> > > > guys setting any headers? Which headers get returned? > > > > >> > > > I've personally always used cache busters for the simple reason > > that > > > >> > > > sometimes users using web-accelerators or with aggressive ISP or > > > >> caching > > > >> > > > settings tend to key off the URL. Can you guys use this > > workaround > > > >> in the > > > >> > > > meantime? > > > > >> > > > On Wed, Jun 2, 2010 at 11:57 AM, Rafael Sierra < > > > >> rafaeljs...@gmail.com > > > >> > > >wrote: > > > > >> > > > > On Wed, Jun 2, 2010 at 3:37 PM, J <j.si...@earlystageit.com> > > > >> wrote: > > > >> > > > > > To reproduce the problem, go to > > > >> > > > >http://qa.connectscholar.com/stylesheets/caSkin.css > > > >> > > > > > and also tohttp:// > > > >> charityaxis-qa.appspot.com/stylesheets/caSkin.css. > > > >> > > > > > Both URLs point to the same file but one returns the old > > content > > > >> and > > > >> > > > > > appspot.com returns the new content. > > > > >> > > > > Same file here: > > > > >> > > > > Hidan:~ sdm$ curlhttp:// > > > >> > > charityaxis-qa.appspot.com/stylesheets/caSkin.css> 1 > > > >> > > > > % Total % Received % Xferd Average Speed Time Time > > > >> Time > > > >> > > > > Current > > > >> > > > > Dload Upload Total Spent > > > >> Left > > > >> > > > > Speed > > > >> > > > > 100 10941 0 10941 0 0 6146 0 --:--:-- > > 0:00:01 > > > >> > > --:--:-- > > > >> > > > > 31918 > > > >> > > > > Hidan:~ sdm$ curlhttp:// > > > >> qa.connectscholar.com/stylesheets/caSkin.css> > > > >> > > 2 > > > >> > > > > % Total % Received % Xferd Average Speed Time Time > > > >> Time > > > >> > > > > Current > > > >> > > > > Dload Upload Total Spent > > > >> Left > > > >> > > > > Speed > > > >> > > > > 100 10941 0 10941 0 0 2777 0 --:--:-- > > 0:00:03 > > > >> > > --:--:-- > > > >> > > > > 3854 > > > >> > > > > Hidan:~ sdm$ md5 1 > > > >> > > > > MD5 (1) = 8f5ef511be1a03fd722223337c334933 > > > >> > > > > Hidan:~ sdm$ md5 2 > > > >> > > > > MD5 (2) = 8f5ef511be1a03fd722223337c334933 > > > > >> > > > > > On Jun 2, 1:18 pm, "Ikai L (Google)" <ika...@google.com> > > wrote: > > > >> > > > > >> Thanks for bringing this up, Tim. Anyone else seeing this > > > >> problem? > > > >> > > If > > > >> > > > > so, > > > >> > > > > >> please post details. Are you guys setting any kind of cache > > > >> headers? > > > > >> > > > > >> I'm going to try to reproduce these issues, so any > > information > > > >> will > > > >> > > be > > > >> > > > > >> helpful. > > > > >> > > > > >> On Wed, Jun 2, 2010 at 9:14 AM, J < > > j.si...@earlystageit.com> > > > >> wrote: > > > >> > > > > >> > Thanks, Tim, for letting me know I'm not going insane. > > > > >> > > > > >> > To the Google guys, this problem seems to have started > > > >> yesterday > > > >> > > > > >> > afternoon (1:30-ish PM, EDT) in case it helps you track > > down > > > >> the > > > >> > > > > >> > cause. > > > > >> > > > > >> > On Jun 2, 8:19 am, Tim Hoffman <zutes...@gmail.com> > > wrote: > > > >> > > > > >> > > Hi > > > > >> > > > > >> > > We have been trying to deploy a major revision of one > > of > > > >> our > > > >> > > apps. > > > > >> > > > > >> > > Under the specific version 2-0-0.latest... all the new > > > >> css/js > > > >> > > and > > > >> > > > > >> > > static images are available. > > > > >> > > > > >> > > When we make the new version default, the css, js and > > > >> static > > > >> > > files > > > >> > > > > are > > > >> > > > > >> > > accessible via <appid>.appspot.com > > > > >> > > > > >> > > However when accessing the same site via the google > > apps > > > >> domain > > > >> > > > > >> > > mapping we are still > > > >> > > > > >> > > getting the old versions css, js and static images. > > > > >> > > > > >> > > I have tried removing the apps domain mapping and > > re-adding > > > >> it > > > >> > > with > > > >> > > > > no > > > >> > > > > >> > > affect. > > > > >> > > > > >> > > We have had to revert to the earlier version as all > > access > > > >> to > > > >> > > the > > > >> > > > > site > > > >> > > > > >> > > is via the apps domain and > > > >> > > > > >> > > its not working with the latest version missing the new > > > >> css/js. > > ... > > read more » -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
