[google-appengine] Re: memcache set succeeds but immediate get fails. Pls help
> > memcache.set() does not set if id already present.
Huh? I don't see that in the documentation. Why do you think that it
is true?
memcache.set is described as "Sets a key's value, regardless of
previous contents in cache."
memcache.add is described as "Sets a key's value, if and only if the
item is not already in memcache."
http://code.google.com/appengine/docs/python/memcache/functions.html
On Feb 7, 1:32 pm, observer247 wrote:
> Thanks Eli ! The cache time was the issue.
>
> memcache.set() does not set if id already present. So I am using
> delete and add.
> I cannot be sure id is present, memcache could be deleted because of
> memory pressure from app engine, right ?
>
> On Feb 7, 10:18 am, Eli Jones wrote:
>
>
>
> > One minor thing I noticed.. why not use memcache.set() instead of
> > memcache.delete(), memcache.add()?
>
> > On Sun, Feb 7, 2010 at 6:22 AM, observer247 wrote:
> > > This is my code:
>
> > > ret = memcache.add(key=mykey, value=qList, time=
> > > 60*60*24*30)
> > > logging.critical("Created cache batch %s Passed %s" %
> > > (mykey, str(ret)))
>
> > > qList = memcache.get(mykey)
>
> > > For some reason, qList is None ! I have logged all values and qList is
> > > a non empty list. Check code below where I print a lot of info in the
> > > logs.
>
> > > Detailed code here:
>
> > > def MY_QC_MAX(): return 3
> > > def MY_QC_SIZE(): return 200
>
> > > def createBatchMyModels():
> > > import random
> > > for n in range(MY_QC_MAX()):
> > > bnum = n + 1
> > > mykey = "qkey_batch_"+str(bnum)
> > > qQ = MyModel.all(keys_only=True).filter('approved',
> > > True)
> > > if bnum > 1:
> > > qQ = qQ.filter('__key__ >', last_key)
> > > rows = qQ.fetch(MY_QC_SIZE())
> > > tot = len(rows)
> > > if tot < MY_QC_SIZE():
> > > logging.critical("Not enough MyModels for
> > > batch %u, got %u" % (bnum, tot))
> > > if tot == 0:
> > > return
> > > last_key = rows[tot - 1]
> > > # create the qList
> > > qList = list()
> > > logging.critical("Added %u rows into key %s" % (tot,
> > > mykey))
> > > tmpc = 0
> > > for r in rows:
> > > if tmpc == 0:
> > > logging.critical("elem %u into key %s"
> > > % (r.id(), mykey))
> > > tmpc = tmpc + 1
> > > qList.append(r.id())
>
> > > for elem in qList:
> > > logging.info("key %s elem is %u" % (mykey,
> > > elem))
> > > memcache.delete(mykey)
> > > ret = memcache.add(key=mykey, value=qList, time=
> > > 60*60*24*30)
> > > logging.critical("Created cache batch %s Passed %s" %
> > > (mykey, str(ret)))
>
> > > qList = memcache.get(mykey)
> > > if qList is None:
> > > logging.critical(".. getNextMyModel: Did not
> > > find key %s" % mykey)
> > > else:
> > > logging.critical(".. LEN : %u" % len(qList))
>
> > > Sample log:
> > > .
> > > 02-07 03:15AM 05.240 key qkey_batch_1 elem is 13108
> > > C 02-07 03:15AM 05.250 Created cache batch qkey_batch_1 Passed True
> > > C 02-07 03:15AM 05.253 .. getNextQuestion: Did not find key
> > > qkey_batch_1
> > > C 02-07 03:15AM 05.339 Added 200 rows into key qkey_batch_2
> > > ...
>
> > > Can anyone pls help !
>
> > > --
> > > You received this message because you are subscribed to the Google Groups
> > > "Google App Engine" group.
> > > To post to this group, send email to google-appeng...@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > > google-appengine+unsubscr...@googlegroups.com > > e...@googlegroups.com>
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/google-appengine?hl=en.- Hide quoted text -
>
> - Show quoted text -
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: memcache set succeeds but immediate get fails. Pls help
set(key, value, time=0, min_compress_len=0, namespace=None)
Sets a key's value, regardless of previous contents in cache.
It sets it.. regardless of previous contents in cache.
Do a quick test.. you'll see.
http://code.google.com/appengine/docs/python/memcache/functions.html
On Sun, Feb 7, 2010 at 4:32 PM, observer247 wrote:
> Thanks Eli ! The cache time was the issue.
>
> memcache.set() does not set if id already present. So I am using
> delete and add.
> I cannot be sure id is present, memcache could be deleted because of
> memory pressure from app engine, right ?
>
>
>
> On Feb 7, 10:18 am, Eli Jones wrote:
> > One minor thing I noticed.. why not use memcache.set() instead of
> > memcache.delete(), memcache.add()?
> >
> >
> >
> > On Sun, Feb 7, 2010 at 6:22 AM, observer247 wrote:
> > > This is my code:
> >
> > >ret = memcache.add(key=mykey, value=qList, time=
> > > 60*60*24*30)
> > >logging.critical("Created cache batch %s Passed %s" %
> > > (mykey, str(ret)))
> >
> > >qList = memcache.get(mykey)
> >
> > > For some reason, qList is None ! I have logged all values and qList is
> > > a non empty list. Check code below where I print a lot of info in the
> > > logs.
> >
> > > Detailed code here:
> >
> > > def MY_QC_MAX(): return 3
> > > def MY_QC_SIZE(): return 200
> >
> > > def createBatchMyModels():
> > >import random
> > >for n in range(MY_QC_MAX()):
> > >bnum = n + 1
> > >mykey = "qkey_batch_"+str(bnum)
> > >qQ = MyModel.all(keys_only=True).filter('approved',
> > > True)
> > >if bnum > 1:
> > >qQ = qQ.filter('__key__ >', last_key)
> > >rows = qQ.fetch(MY_QC_SIZE())
> > >tot = len(rows)
> > >if tot < MY_QC_SIZE():
> > >logging.critical("Not enough MyModels for
> > > batch %u, got %u" % (bnum, tot))
> > >if tot == 0:
> > >return
> > >last_key = rows[tot - 1]
> > ># create the qList
> > >qList = list()
> > >logging.critical("Added %u rows into key %s" % (tot,
> > > mykey))
> > >tmpc = 0
> > >for r in rows:
> > >if tmpc == 0:
> > >logging.critical("elem %u into key %s"
> > > % (r.id(), mykey))
> > >tmpc = tmpc + 1
> > >qList.append(r.id())
> >
> > >for elem in qList:
> > >logging.info("key %s elem is %u" % (mykey,
> > > elem))
> > >memcache.delete(mykey)
> > >ret = memcache.add(key=mykey, value=qList, time=
> > > 60*60*24*30)
> > >logging.critical("Created cache batch %s Passed %s" %
> > > (mykey, str(ret)))
> >
> > >qList = memcache.get(mykey)
> > >if qList is None:
> > >logging.critical(".. getNextMyModel: Did not
> > > find key %s" % mykey)
> > >else:
> > >logging.critical(".. LEN : %u" % len(qList))
> >
> > > Sample log:
> > > .
> > > 02-07 03:15AM 05.240 key qkey_batch_1 elem is 13108
> > > C 02-07 03:15AM 05.250 Created cache batch qkey_batch_1 Passed True
> > > C 02-07 03:15AM 05.253 .. getNextQuestion: Did not find key
> > > qkey_batch_1
> > > C 02-07 03:15AM 05.339 Added 200 rows into key qkey_batch_2
> > > ...
> >
> > > Can anyone pls help !
> >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups
> > > "Google App Engine" group.
> > > To post to this group, send email to google-appengine@googlegroups.com
> .
> > > To unsubscribe from this group, send email to
> > > google-appengine+unsubscr...@googlegroups.com e...@googlegroups.com>
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/google-appengine?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: memcache set succeeds but immediate get fails. Pls help
I believe the issue is that memcache is non-blocking; the write may
not complete before you try to read, and instead of blocking waiting
for the write to finish memcache returns a None value to the read
request.
On Feb 7, 6:22 am, observer247 wrote:
> This is my code:
>
> ret = memcache.add(key=mykey, value=qList, time=
> 60*60*24*30)
> logging.critical("Created cache batch %s Passed %s" %
> (mykey, str(ret)))
>
> qList = memcache.get(mykey)
>
> For some reason, qList is None ! I have logged all values and qList is
> a non empty list. Check code below where I print a lot of info in the
> logs.
>
> Detailed code here:
>
> def MY_QC_MAX(): return 3
> def MY_QC_SIZE(): return 200
>
> def createBatchMyModels():
> import random
> for n in range(MY_QC_MAX()):
> bnum = n + 1
> mykey = "qkey_batch_"+str(bnum)
> qQ = MyModel.all(keys_only=True).filter('approved',
> True)
> if bnum > 1:
> qQ = qQ.filter('__key__ >', last_key)
> rows = qQ.fetch(MY_QC_SIZE())
> tot = len(rows)
> if tot < MY_QC_SIZE():
> logging.critical("Not enough MyModels for
> batch %u, got %u" % (bnum, tot))
> if tot == 0:
> return
> last_key = rows[tot - 1]
> # create the qList
> qList = list()
> logging.critical("Added %u rows into key %s" % (tot,
> mykey))
> tmpc = 0
> for r in rows:
> if tmpc == 0:
> logging.critical("elem %u into key %s"
> % (r.id(), mykey))
> tmpc = tmpc + 1
> qList.append(r.id())
>
> for elem in qList:
> logging.info("key %s elem is %u" % (mykey,
> elem))
> memcache.delete(mykey)
> ret = memcache.add(key=mykey, value=qList, time=
> 60*60*24*30)
> logging.critical("Created cache batch %s Passed %s" %
> (mykey, str(ret)))
>
> qList = memcache.get(mykey)
> if qList is None:
> logging.critical(".. getNextMyModel: Did not
> find key %s" % mykey)
> else:
> logging.critical(".. LEN : %u" % len(qList))
>
> Sample log:
> .
> 02-07 03:15AM 05.240 key qkey_batch_1 elem is 13108
> C 02-07 03:15AM 05.250 Created cache batch qkey_batch_1 Passed True
> C 02-07 03:15AM 05.253 .. getNextQuestion: Did not find key
> qkey_batch_1
> C 02-07 03:15AM 05.339 Added 200 rows into key qkey_batch_2
> ...
>
> Can anyone pls help !
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: memcache set succeeds but immediate get fails. Pls help
Thanks Eli ! The cache time was the issue.
memcache.set() does not set if id already present. So I am using
delete and add.
I cannot be sure id is present, memcache could be deleted because of
memory pressure from app engine, right ?
On Feb 7, 10:18 am, Eli Jones wrote:
> One minor thing I noticed.. why not use memcache.set() instead of
> memcache.delete(), memcache.add()?
>
>
>
> On Sun, Feb 7, 2010 at 6:22 AM, observer247 wrote:
> > This is my code:
>
> > ret = memcache.add(key=mykey, value=qList, time=
> > 60*60*24*30)
> > logging.critical("Created cache batch %s Passed %s" %
> > (mykey, str(ret)))
>
> > qList = memcache.get(mykey)
>
> > For some reason, qList is None ! I have logged all values and qList is
> > a non empty list. Check code below where I print a lot of info in the
> > logs.
>
> > Detailed code here:
>
> > def MY_QC_MAX(): return 3
> > def MY_QC_SIZE(): return 200
>
> > def createBatchMyModels():
> > import random
> > for n in range(MY_QC_MAX()):
> > bnum = n + 1
> > mykey = "qkey_batch_"+str(bnum)
> > qQ = MyModel.all(keys_only=True).filter('approved',
> > True)
> > if bnum > 1:
> > qQ = qQ.filter('__key__ >', last_key)
> > rows = qQ.fetch(MY_QC_SIZE())
> > tot = len(rows)
> > if tot < MY_QC_SIZE():
> > logging.critical("Not enough MyModels for
> > batch %u, got %u" % (bnum, tot))
> > if tot == 0:
> > return
> > last_key = rows[tot - 1]
> > # create the qList
> > qList = list()
> > logging.critical("Added %u rows into key %s" % (tot,
> > mykey))
> > tmpc = 0
> > for r in rows:
> > if tmpc == 0:
> > logging.critical("elem %u into key %s"
> > % (r.id(), mykey))
> > tmpc = tmpc + 1
> > qList.append(r.id())
>
> > for elem in qList:
> > logging.info("key %s elem is %u" % (mykey,
> > elem))
> > memcache.delete(mykey)
> > ret = memcache.add(key=mykey, value=qList, time=
> > 60*60*24*30)
> > logging.critical("Created cache batch %s Passed %s" %
> > (mykey, str(ret)))
>
> > qList = memcache.get(mykey)
> > if qList is None:
> > logging.critical(".. getNextMyModel: Did not
> > find key %s" % mykey)
> > else:
> > logging.critical(".. LEN : %u" % len(qList))
>
> > Sample log:
> > .
> > 02-07 03:15AM 05.240 key qkey_batch_1 elem is 13108
> > C 02-07 03:15AM 05.250 Created cache batch qkey_batch_1 Passed True
> > C 02-07 03:15AM 05.253 .. getNextQuestion: Did not find key
> > qkey_batch_1
> > C 02-07 03:15AM 05.339 Added 200 rows into key qkey_batch_2
> > ...
>
> > Can anyone pls help !
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
