Hi Kim, Can you please provide your app ID? Also, how are you accessing your app? Only via appspot.com, or also via a Google Apps domain you've mapped to the app? The more details you can provide about your load testing - especially the actual code - the easier this will be to diagnose.
-Nick Johnson On Tue, Jun 23, 2009 at 2:42 PM, Kim Riber <kimsteenri...@gmail.com> wrote: > > Yes. There is no other activity when running step 3. > The two values I get back, are some of the last ones I write to the > cache (I return all the values I write). > So it looks a lot like the write threads are writing to two instances > of memcache, and when I start reading back, I see the 2 different > values from the 2 insances. > When waiting for an hour or so, when I run step 3 it returns only one > value again. > If I run the heavy load task (and spawn more instances), running step > 3 will start reporting the 2 values again. Like the cluster with the > second memcache instance is getting fired up, and reporting it's old > value. > the code for step 3 is something like this: > > def read_memcache() > val = memcache.get('my_test_key') > return val > > I could try to set up an application with the 3 steps. > > Cheers > Kim > > On Jun 23, 2:52 pm, "Nick Johnson (Google)" <nick.john...@google.com> > wrote: > > Hi Kim, > > > > Are you able to send us the code you use for step 3? And are you certain > > nothing is changing the memcache concurrently with step 3? > > > > > > > > On Tue, Jun 23, 2009 at 1:44 PM, Kim Riber <kimsteenri...@gmail.com> > wrote: > > > > > Hi Nick > > > > > I run the test in 3 steps (with half a minute in between): > > > 1. Heavy load process to spawn appinstances > > > 2. Write a lot of random values to the same key > > > 3. Read the key from multiple threads. > > > I can repeat 3rd step, and still get the same result (mostly 2 > > > different values) > > > > > It seems like I hit 2 different memcache servers with different view > > > on what is in the cache at that key > > > > > >Also, are you running this against the dev_appserver, or in > production? > > > How do I see that? > > > We are running towards .appspot.com > > > > If you are testing against the local development server, that's the > > dev_appserver. If you're testing code running on appspot.com, that's > > production. > > > > -Nick Johnson > > > > > > > > > <http://appspot.com> > > > > > -Kim > > > > > On Jun 23, 12:39 pm, "Nick Johnson (Google)" <nick.john...@google.com> > > > wrote: > > > > Hi Kim, > > > > > > It's not clear from your description exactly how you're performing > your > > > > tests. Without extra information, the most likely explanation would > be > > > that > > > > you're seeing a race condition in your code, where the key is > modified > > > > between subsequent requests to the memcache API. > > > > > > Also, are you running this against the dev_appserver, or in > production? > > > > > > -Nick Johnson > > > > > > On Tue, Jun 23, 2009 at 7:18 AM, Kim Riber <kimsteenri...@gmail.com> > > > wrote: > > > > > > > Just made another test, to confirm the behavior I see. > > > > > This example is much simpler, and simply has 10 threads writing > random > > > > > values to memcahce to the same key. > > > > > I would expect the last value written to be the one left in > memcache. > > > > > When afterwards, having 4 threads reading 10 times from that same > key, > > > > > they return 2 different values. > > > > > This only happens if I prior to the writing threads, run some heavy > > > > > tasks, to force gae to spawn more app instances. > > > > > It seems like each server cluster might have its own memcache, > > > > > independant from each other. I hope this is not true. From a thread > > > > > from Ryan > > > > > > > > http://groups.google.com/group/google-appengine/browse_thread/thread/. > > > .. > > > > > he states that > > > > > > > >as for the datastore, and all other current stored data APIs like > > > > > >memcache, there is a single, global view of data. we go to great > > > > > >lengths to ensure that these APIs are strongly consistent. > > > > > > > Regards > > > > > Kim > > > > > > > On Jun 17, 8:51 pm, Kim Riber <kimsteenri...@gmail.com> wrote: > > > > > > To clarify a bit: > > > > > > > > one thread from our server runs one loop with a unique id. > > > > > > each requests stores a value in memcache and returns that value. > In > > > > > > the following request, the memcache is queried if the value just > > > > > > written, is in the cache. > > > > > > This sometimes fail. > > > > > > > > My fear is that it is due to the requests changing to another app > > > > > > instance and then suddently getting wrong data. > > > > > > > > instance 1 +++++ +++++ > > > > > > instance 2 -- > > > > > > > > Hope this clears out the example above a bit > > > > > > > > Cheers > > > > > > Kim > > > > > > > > On Jun 17, 7:52 pm, Kim Riber <kimsteenri...@gmail.com> wrote: > > > > > > > > > Hi, > > > > > > > I'm experiencing some rather strange behavior from memcache. I > > > think > > > > > > > I'm getting different data back from memcache using the same > key > > > > > > > The issue I see is that when putting load on our application, > even > > > > > > > simple memcache queries are starting to return inconsistant > data. > > > When > > > > > > > running the same request from multiple threads, I get different > > > > > > > results. > > > > > > > I've made a very simple example, that runs fine on 1-200 > threads, > > > but > > > > > > > if I put load on the app (with some heavier requests) just > before I > > > > > > > run my test, I see different values coming back from memcache > using > > > > > > > the same keys. > > > > > > > > > def get_new_memcahce_value(key, old_value): > > > > > > > old_val = memcache.get(key) > > > > > > > new_val = uuid.uuid4().get_hex() > > > > > > > reply = 'good' > > > > > > > if old_val and old_value != "": > > > > > > > if old_val != old_value: > > > > > > > reply = 'fail' > > > > > > > new_val = old_value > > > > > > > else: > > > > > > > if not memcache.set(key, new_val): > > > > > > > reply = 'set_fail' > > > > > > > else: > > > > > > > reply = 'new' > > > > > > > if not memcache.set(key,new_val): > > > > > > > reply = 'set_fail' > > > > > > > return (new_value, reply) > > > > > > > > > and from a server posting requests: > > > > > > > > > def request_loop(id): > > > > > > > key = "test:key_%d" % id > > > > > > > val, reply = get_new_memcahce_value(key, "") > > > > > > > for i in range(20): > > > > > > > val,reply = get_new_memcahce_value(key, val) > > > > > > > > > Is memcache working localy on a cluster of servers, and if an > > > > > > > application is spawned over more clusters, memcache will not > > > > > > > propergate data to the other clusters? > > > > > > > > > I hope someone can clarify this, since I can't find any post > > > regarding > > > > > > > this issue. > > > > > > > > > Is there some way to get the application instance ID, so I can > do > > > some > > > > > > > more investigation on the subject? > > > > > > > > > Thanks > > > > > > > Kim > > > > > > -- > > > > Nick Johnson, App Engine Developer Programs Engineer > > > > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration > > > Number: > > > > 368047 > > > > -- > > Nick Johnson, App Engine Developer Programs Engineer > > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration > Number: > > 368047 > > > -- Nick Johnson, App Engine Developer Programs Engineer Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047 --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---