[appengine-java] Re: Caching using static fields
Thanks leszek, I agree that keeping an indicator in memcache is much better than keeping the data itself there. I was trying to avoid checking memcache on every request, but I guess there is no way around that :) On Oct 19, 10:11 am, leszek wrote: > What about using memcache as keeping "cache version counter" ? When > update is needed than this counter is increased. Every requests keeps > local number and at the beginning compare local counter against > memcache counter. If not equal than refresh local cache and local > cache number. > If memcache counter if not available (expired) than assume it as equal > 0 and behave accordingly. It could mean unnecessary cache refreshing > from time to time if memcache expires. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Caching using static fields
Thanks Peter, As I wrote in my original post, I'm trying to avoid a situation where incoming requests trigger reloading of data files into memory. Since I expect very high traffic I'll have to put some locking mechanism to prevent simultaneous loads. Anyways, I think this becomes a moot point since AFAIK there isn't way to deploy a single file without having GAE restart my app. right ? On Oct 19, 9:41 am, Peter Hulsen wrote: > I suppose you can keep, per instance, a static lastUpdated (Date) > field with your static data. Now, when you access the data first look > at lastUpdated, if this is longer than a certain time period reload > the data before using the static data. You more or less rebuild > memcache, but in this way you cannot share data between instances. > > Peter > > On 18 okt, 17:22, Raviv Pavel wrote: > > > A reply from a Google rep on this matter would be much appreciated. > > > On Oct 16, 9:55 pm, Raviv Pavel wrote: > > > > I'm afraid you don't understand the problem. > > > The issue is how to "notify" all instances of the app that they should > > > reload their in-memory cached data. > > > Cron jobs and tasks send request so asingleinstance of the app, not > > > to all of them. > > > > On Oct 16, 5:21 pm, Jean Meurtrier wrote: > > > > > I don't exactly understand what you want to do, but you write you > > > > cannot use a timer and you need to be triggered by incoming requests. > > > > Why not use the cron jobs or the task queues? > > > > > JMT > > > > > On Oct 15, 9:58 pm,RavivPavel wrote: > > > > > > Most web apps I build need to cache data which falls into two > > > > > categories: > > > > > > 1. Configuration and lookup lists such as IP-to-country and > > > > > configuration - loaded fromfileinto static member and reloaded when > > > > > afilechange is detected. > > > > > 2. User generated lists such as tags, product list, etc - loaded from > > > > > DB into static fields, and either periodically reloaded, or broadcast > > > > > change to all servers (e.g. UDP) > > > > > > Using static fields gives me better performance than memcache when > > > > > accessing long lookup lists on every request, > > > > > > I'm not sure how to approach this in GAE especially since I can't use > > > > > a timer to watch afileor periodically reload from DB. The options I > > > > > see are: > > > > > > For scenario 1: > > > > > a. Redeploying the app with the new static config and lookup files - a > > > > > bit risky since I can accidentydeploynew code. > > > > > b. Periodically reload thefile- Since there is no Timer, it will > > > > > have to be triggered by incoming requests. Ugly :( > > > > > > For scenario 2: > > > > > a. Load from Datastore into memcache and invalidate upon update - > > > > > Slower than static member and may drain quota fast since I need to > > > > > serve around 20 million req/hour at peek times. > > > > > b. somehow issue a request to every active instance of my app to > > > > > invalidate its static cache - Is this possible? > > > > > > Any advice would be much appreciated. > > > > > >Raviv. > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Caching using static fields
A reply from a Google rep on this matter would be much appreciated. On Oct 16, 9:55 pm, Raviv Pavel wrote: > I'm afraid you don't understand the problem. > The issue is how to "notify" all instances of the app that they should > reload their in-memory cached data. > Cron jobs and tasks send request so a single instance of the app, not > to all of them. > > On Oct 16, 5:21 pm, Jean Meurtrier wrote: > > > I don't exactly understand what you want to do, but you write you > > cannot use a timer and you need to be triggered by incoming requests. > > Why not use the cron jobs or the task queues? > > > JMT > > > On Oct 15, 9:58 pm,RavivPavel wrote: > > > > Most web apps I build need to cache data which falls into two > > > categories: > > > > 1. Configuration and lookup lists such as IP-to-country and > > > configuration - loaded from file into static member and reloaded when > > > a file change is detected. > > > 2. User generated lists such as tags, product list, etc - loaded from > > > DB into static fields, and either periodically reloaded, or broadcast > > > change to all servers (e.g. UDP) > > > > Using static fields gives me better performance than memcache when > > > accessing long lookup lists on every request, > > > > I'm not sure how to approach this in GAE especially since I can't use > > > a timer to watch a file or periodically reload from DB. The options I > > > see are: > > > > For scenario 1: > > > a. Redeploying the app with the new static config and lookup files - a > > > bit risky since I can accidenty deploy new code. > > > b. Periodically reload the file - Since there is no Timer, it will > > > have to be triggered by incoming requests. Ugly :( > > > > For scenario 2: > > > a. Load from Datastore into memcache and invalidate upon update - > > > Slower than static member and may drain quota fast since I need to > > > serve around 20 million req/hour at peek times. > > > b. somehow issue a request to every active instance of my app to > > > invalidate its static cache - Is this possible? > > > > Any advice would be much appreciated. > > > >Raviv. > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
