No, they are not in the same request. get_multi is called within 5
minutes after set_multi is called. User object is firly small (less
than 1k) and total number of items in memcache is less than 50k, so it
won't be more than 100MB. User object contains strings, integers and
integer list. No blobs, nor string list. Again, all set_multi call
seems success, no error, no exception occurred.
Thanks,
Kazuho
On Dec 3, 1:14 pm, "Ikai L (Google)" <ika...@google.com> wrote:
> Are you doing the get_multi and set_multi within the same request, or are
> the items disappearing afterwards? There's a 100mb limit to the number of
> items you can be storing in Memcache, but I'm not sure this is what you're
> hitting - the least recently used objects would be getting expired. What
> kind of data is stored in the User object?
>
> ---------- Forwarded message ----------
> From: naan <kaz...@gmail.com>
> Date: Thu, Dec 3, 2009 at 12:51 PM
> Subject: [google-appengine] Re: Weird memcached issue
> To: Google App Engine <google-appengine@googlegroups.com>
>
> I try to store 100 users object per request. memcache.set_multi seems
> working (return an empty array which means all data stored
> successfully), but get_multi returns only 9-15 of them. It worked
> yesterday morning, then stopped working afternoon. it seems that I
> can't store new user data into memcache, but still can get old cached
> data with get_multi.
>
> The test code you gave me works fine. And my code works fine too on
> another GAE instance. Hmm.... Does call memcache.flush_all() will
> solve the issue?
>
> On Dec 3, 12:04 pm, "Ikai L (Google)" <ika...@google.com> wrote:
> > What is being returned by get_multi? None or empty dictionary? Is a
> > namespace being specified?
>
> > Here's some test code that is working for me when deployed. What might you
> > be doing differently from this?
>
> > from google.appengine.ext import webapp
> > from google.appengine.ext.webapp.util import run_wsgi_app
> > from google.appengine.ext import db
>
> > from google.appengine.api import memcache
>
> > class MyThing(db.Model):
> > name = db.StringProperty()
>
> > def __str__(self):
> > return "MyThing: %s" % self.name
>
> > class MemcacheTest(webapp.RequestHandler):
> > def get(self):
> > stats = memcache.get_stats()
>
> > println(self, "<b>Cache Hits:%s</b><br>" % stats['hits'])
> > println(self, "<b>Cache Misses:%s</b><br><br>" % stats['misses'])
>
> > memcache.set("data", "My data")
> > data = memcache.get("data")
> > println(self, "Set 'data' -> 'My data'. Get 'data' -> " + data)
>
> > key_range = range(1000)
> > mapping = { }
>
> > for i in key_range:
> > mapping["key%d" % i] = "value %d" % i
>
> > memcache.set_multi(mapping)
> > println(self, "Set %d values using set_multi" % len(mapping))
>
> > cached_mapping = memcache.get_multi(mapping.keys())
> > println(self, "Retrieved %d values using get_multi" %
> > len(cached_mapping))
>
> > namespace = "synccache"
> > println(self, "Testing namespaces with namespace '%s'" % namespace)
> > memcache.set_multi(mapping, namespace=namespace)
> > println(self, "Set %d values using set_multi in namespace '%s'" % (
> > len(mapping), namespace ))
>
> > cached_mapping = memcache.get_multi(mapping.keys(),
> namespace=namespace)
> > println(self, "Retrieved %d values using get_multi in namespace '%s'"
> %
> > ( len(cached_mapping), namespace ))
>
> > println(self, "Testing objects")
> > obj_map = {}
> > for i in key_range:
> > thing = MyThing(name="thing %d" % i)
> > obj_map[thing.name] = thing
>
> > namespace = "things"
> > println(self, "Testing namespaces with namespace '%s'" % namespace)
> > memcache.set_multi(obj_map, namespace=namespace)
> > println(self, "Set %d values using set_multi in namespace '%s'" % (
> > len(mapping), namespace ))
>
> > cached_mapping = memcache.get_multi(obj_map.keys(),
> namespace=namespace)
> > println(self, "Retrieved %d values using get_multi in namespace '%s'"
> %
> > ( len(cached_mapping), namespace ))
>
> > for key in cached_mapping:
> > println(self, str(obj_map[key]))
>
> > def println(handler, string):
> > handler.response.out.write(string + "<br/>")
>
> > application = webapp.WSGIApplication([ ('/cachetest', MemcacheTest),
> > ], debug=True)
>
> > def main():
> > run_wsgi_app(application)
>
> > if __name__ == "__main__":
> > main()
>
> > On Thu, Dec 3, 2009 at 11:09 AM, naan <kaz...@gmail.com> wrote:
> > > I'm using memcache.set_multi() / add_multi() for caching DataStore
> > > data. Code is something like following:
>
> > > users = User.all().query(....).fetch(...)
>
> > > mapping = {}
> > > for u in users:
> > > mapping[str(u.key())] = u
>
> > > memcache.set_multi(mapping, namespace='synccache')
>
> > > It seems that set_multi/add_multi returns correct value and there's no
> > > error logs, error messages, nor exception. However, I can't retrieve
> > > them with memcache.get_multi().
>
> > > Thanks,
> > > Kazuho
>
> > > On Dec 3, 10:46 am, "Ikai L (Google)" <ika...@google.com> wrote:
> > > > We've just tried this and it seems to work for us. What are you
> storing
> > > in
> > > > Memcache? How are you generating the keys?
>
> > > > On Thu, Dec 3, 2009 at 10:31 AM, Ikai L (Google) <ika...@google.com>
> > > wrote:
>
> > > > > Is there an error message when you try to store data? Do you see any
> > > > > information in the logs?
>
> > > > > On Wed, Dec 2, 2009 at 5:49 PM, naan <kaz...@gmail.com> wrote:
>
> > > > >> My app id is echofonsync and the issue remains. I can get a result
> > > > >> from memcache.get_stats() now, but can not store new data. Thanks.
>
> > > > >> On Dec 2, 5:05 pm, "Ikai L (Google)" <ika...@google.com> wrote:
> > > > >> > Is this issue still occurring for you? If so, let me know your
> > > > >> application
> > > > >> > ID.
>
> > > > >> > Michael, let me know if this is causing you problems in
> production.
> > > I
> > > > >> can
> > > > >> > attempt to migrate your application to a different pool, but this
> is
> > > not
> > > > >> > something I'd advise if you're running in production and things
> are
> > > > >> working
> > > > >> > fine.
>
> > > > >> > On Wed, Dec 2, 2009 at 4:26 PM, naan <kaz...@gmail.com> wrote:
> > > > >> > > Hi,
>
> > > > >> > > This happens for me too. I can't store any data into memcache.
> > > Also, I
> > > > >> > > can't get any results with memcache.get_stats(). The function
> > > returns
> > > > >> > > None.
>
> > > > >> > > On Dec 2, 1:45 pm, Michael <m...@mzlab.net> wrote:
> > > > >> > > > The problem with consistency is: the same key maps to the
> > > different
> > > > >> > > > values (when I'm hitting another memcached server), so
> randomly
> > > I'm
> > > > >> > > > getting old value from memcache and it produces weird effects
> on
> > > my
> > > > >> > > > application (like new message appearing, though they were
> marked
> > > as
> > > > >> > > > read).
>
> > > > >> > > > Is it going to be fixed soon?
>
> > > > >> > > > On Dec 2, 11:33 pm, Michael <m...@mzlab.net> wrote:
>
> > > > >> > > > > At the time of the first post it was going for about 10
> > > minutes.
>
> > > > >> > > > > It's still happening.
>
> > > > >> > > --
>
> > > > >> > > You received this message because you are subscribed to the
> Google
> > > > >> Groups
> > > > >> > > "Google App Engine" group.
> > > > >> > > To post to this group, send email to
> > > > >> google-appeng...@googlegroups.com.
> > > > >> > > To unsubscribe from this group, send email to
> > > > >> > > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com>
> <google-appengine%2bunsubscr...@googlegroups.com<google-appengine%252bunsubscr...@googlegroups.com>
>
> > > <google-appengine%2bunsubscr...@googlegroups.com<google-appengine%252bunsubscr...@googlegroups.com>
> <google-appengine%252bunsubscr...@googlegroups.com<google-appengine%25252bunsubscr...@googlegroups.com>
>
> > > > >> <google-appengine%2bunsubscr...@googlegroups.com<google-appengine%252bunsubscr...@googlegroups.com>
> <google-appengine%252bunsubscr...@googlegroups.com<google-appengine%25252bunsubscr...@googlegroups.com>
>
> > > <google-appengine%252bunsubscr...@googlegroups.com<google-appengine%25252bunsubscr...@googlegroups.com>
> <google-appengine%25252bunsubscr...@googlegroups.com<google-appengine%2525252bunsubscr...@googlegroups.com>
>
> > > > >> > > .
> > > > >> > > For more options, visit this group at
> > > > >> > >http://groups.google.com/group/google-appengine?hl=en.
>
> > > > >> > --
> > > > >> > Ikai Lan
> > > > >> > Developer Programs Engineer, Google App Engine
>
> > > > >> --
>
> > > > >> You received this message because you are subscribed to the Google
> > > Groups
> > > > >> "Google App Engine" group.
> > > > >> To post to this group, send email to
> > > google-appeng...@googlegroups.com.
> > > > >> To unsubscribe from this group, send email to
> > > > >> google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com>
> <google-appengine%2bunsubscr...@googlegroups.com<google-appengine%252bunsubscr...@googlegroups.com>
>
> > > <google-appengine%2bunsubscr...@googlegroups.com<google-appengine%252bunsubscr...@googlegroups.com>
> <google-appengine%252bunsubscr...@googlegroups.com<google-appengine%25252bunsubscr...@googlegroups.com>
>
> > > > >> .
> > > > >> For more options, visit this group at
> > > > >>http://groups.google.com/group/google-appengine?hl=en.
>
> > > > > --
> > > > > Ikai Lan
> > > > > Developer Programs Engineer, Google App Engine
>
> > > > --
> > > > Ikai Lan
> > > > Developer Programs Engineer, Google App Engine
>
> > > --
>
> > > You received this message because you are subscribed to the Google
> Groups
> > > "Google App Engine" group.
> > > To post to this group, send email to google-appeng...@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com>
> <google-appengine%2bunsubscr...@googlegroups.com<google-appengine%252bunsubscr...@googlegroups.com>
>
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/google-appengine?hl=en.
>
> > --
> > Ikai Lan
> > Developer Programs Engineer, Google App Engine
>
> --
>
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To
>
> ...
>
> read more »
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
