Re: [Django] #32831: Some cache tests are flaky (e.g. cache.tests.BaseMemcachedTests.test_touch)

[Django]

#32831: Some cache tests are flaky (e.g. 
cache.tests.BaseMemcachedTests.test_touch)
-+-
 Reporter:  Chris Jerdonek   |Owner:  Wassef
 Type:   |  Ben Ahmed
  Cleanup/optimization   |   Status:  assigned
Component:  Core (Cache system)  |  Version:  dev
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Wassef Ben Ahmed):

 * has_patch:  0 => 1

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/010701908e6db82f-40176842-a8ac-4e48-9336-9a20d4f011a3-00%40eu-central-1.amazonses.com.

Re: [Django] #32831: Some cache tests are flaky (e.g. cache.tests.BaseMemcachedTests.test_touch)

[Django]

#32831: Some cache tests are flaky (e.g. 
cache.tests.BaseMemcachedTests.test_touch)
-+-
 Reporter:  Chris Jerdonek   |Owner:  Wassef
 Type:   |  Ben Ahmed
  Cleanup/optimization   |   Status:  assigned
Component:  Core (Cache system)  |  Version:  dev
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Comment (by Wassef Ben Ahmed):

 to my understanding,

 so far we don't know which tests exactly are flaky.
 mainly the cases mentioned above, and anything that uses a shared medium.

 as the @Simon Charette suggested this could be done by figuring out where
 to set locks, which he also pointed out that it might get tricky depending
 on how djangoci works (I have no clue)
 we might need to handle lock depending on the platform running the test or
 use something like "portalocker" which covers cross-platform locks but
 forces the adoption of a new dependency?

 expanding on a previous answer by @Chris Jerdonek on setting assertion
 retries,  why not set it at a unit level instead of individual assertion?

 how you'd typically do with some pytest extensions...  And we'll be
 assured the whole test passed without having to reason line-by-line on
 what went wrong?

 implementation goes something like:
 {{{
 from functools import wraps
 def retry(retries=3, delay=1):
 def decorator(func):
 @wraps(func)
 def wrapper(*args, **kwargs):
 attempts = 0
 while attempts < retries:
 try:
 return func(*args, **kwargs)
 except AssertionError as e:
 attempts += 1
 if attempts >= retries:
 raise
 time.sleep(delay)
 return wrapper
 return decorator
 }}}

 only catching/retrying on "AssertionError" so other exceptions are not
 permitted.

 which I'm still not sure about? (maybe a shared medium causes you to pluck
 a None or something...)
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/010701908cfed5b1-af272b91-343c-4ba6-88f8-7255d19ed15a-00%40eu-central-1.amazonses.com.

Re: [Django] #32831: Some cache tests are flaky (e.g. cache.tests.BaseMemcachedTests.test_touch)

[Django]

#32831: Some cache tests are flaky (e.g. 
cache.tests.BaseMemcachedTests.test_touch)
-+-
 Reporter:  Chris Jerdonek   |Owner:  Wassef
 Type:   |  Ben Ahmed
  Cleanup/optimization   |   Status:  assigned
Component:  Core (Cache system)  |  Version:  dev
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Wassef Ben Ahmed):

 * owner:  (none) => Wassef Ben Ahmed
 * status:  new => assigned

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/010701906af41f0e-242f7888-9bea-434e-a615-2b0f0c4651d0-00%40eu-central-1.amazonses.com.

Re: [Django] #32831: Some cache tests are flaky (e.g. cache.tests.BaseMemcachedTests.test_touch)

[Django]

#32831: Some cache tests are flaky (e.g. 
cache.tests.BaseMemcachedTests.test_touch)
--+
 Reporter:  Chris Jerdonek|Owner:  (none)
 Type:  Cleanup/optimization  |   Status:  new
Component:  Core (Cache system)   |  Version:  dev
 Severity:  Normal|   Resolution:
 Keywords:| Triage Stage:  Accepted
Has patch:  0 |  Needs documentation:  0
  Needs tests:  0 |  Patch needs improvement:  0
Easy pickings:  0 |UI/UX:  0
--+
Comment (by Simon Charette):

 I get a sense that a lot of these failures are due to tests using their
 same global store at the same time as Chris pointed out. A clear sign of
 that is `cache.clear` is being called at `setUp` time.

 A solution here could be to run these tests cases in they own separate key
 space (e.g. with a cache key prefix or with two distinct memcached
 instances) or to serialize them to ensure that no tests share the same
 resource (e.g. using a `multiprocessing.lock` acquired in `setUp` and
 released in `tearDown`).

 There might be more too it though as I'm unsure what the djangci setup
 looks like. Is there one memcached / Redis instance per job or are they
 shared amongst them?
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018ed8b2a1ac-357710da-898f-47af-bf80-a84c9e7b21f6-00%40eu-central-1.amazonses.com.

Re: [Django] #32831: Some cache tests are flaky (e.g. cache.tests.BaseMemcachedTests.test_touch)

[Django]

#32831: Some cache tests are flaky (e.g. 
cache.tests.BaseMemcachedTests.test_touch)
--+
 Reporter:  Chris Jerdonek|Owner:  (none)
 Type:  Cleanup/optimization  |   Status:  new
Component:  Core (Cache system)   |  Version:  dev
 Severity:  Normal|   Resolution:
 Keywords:| Triage Stage:  Accepted
Has patch:  0 |  Needs documentation:  0
  Needs tests:  0 |  Patch needs improvement:  0
Easy pickings:  0 |UI/UX:  0
--+
Comment (by Natalia Bidart):

 Another one, different but similar:
 {{{
 ...
 Using shuffle seed: 5526860306 (generated)
 ...
 ==
 ERROR [0.003s]: test_lru_incr (cache.tests.LocMemCacheTests.test_lru_incr)
 --
 Traceback (most recent call last):
   File "/home/jenkins/workspace/main-
 random/database/spatialite/label/focal/python/python3.12/django/test/utils.py",
 line 446, in inner
 return func(*args, **kwargs)
^
   File "/home/jenkins/workspace/main-
 random/database/spatialite/label/focal/python/python3.12/tests/cache/tests.py",
 line 1433, in test_lru_incr
 self.assertEqual(cache.incr(key), key + 1)
  ^^^
   File "/home/jenkins/workspace/main-
 
random/database/spatialite/label/focal/python/python3.12/django/core/cache/backends/locmem.py",
 line 70, in incr
 raise ValueError("Key '%s' not found" % key)
 ValueError: Key ':1:0' not found

 --
 Ran 17776 tests in 1566.550s

 FAILED (errors=1, skipped=1161, expected failures=5)
 }}}
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018ed35b4901-bbb9a641-9277-41dd-b4e0-0ecdb13248f2-00%40eu-central-1.amazonses.com.

Re: [Django] #32831: Some cache tests are flaky (e.g. cache.tests.BaseMemcachedTests.test_touch)

[Django]

#32831: Some cache tests are flaky (e.g. 
cache.tests.BaseMemcachedTests.test_touch)
--+
 Reporter:  Chris Jerdonek|Owner:  (none)
 Type:  Cleanup/optimization  |   Status:  new
Component:  Core (Cache system)   |  Version:  dev
 Severity:  Normal|   Resolution:
 Keywords:| Triage Stage:  Accepted
Has patch:  0 |  Needs documentation:  0
  Needs tests:  0 |  Patch needs improvement:  0
Easy pickings:  0 |UI/UX:  0
--+
Comment (by Natalia Bidart):

 Cache flaky test in main-random:
 {{{
  > git init /home/jenkins/workspace/main-
 random/database/spatialite/label/focal/python/python3.12 # timeout=10
 ...
 Using shuffle seed: 5744808359 (generated)
 ...
 ==
 FAIL [0.004s]: test_lru_get (cache.tests.LocMemCacheTests.test_lru_get)
 --
 Traceback (most recent call last):
   File "/home/jenkins/workspace/main-
 random/database/spatialite/label/focal/python/python3.12/django/test/utils.py",
 line 446, in inner
 return func(*args, **kwargs)
^
   File "/home/jenkins/workspace/main-
 random/database/spatialite/label/focal/python/python3.12/tests/cache/tests.py",
 line 1406, in test_lru_get
 self.assertEqual(cache.get(key), key)
 AssertionError: None != 0

 --
 Ran 17769 tests in 1613.662s

 FAILED (failures=1, skipped=1160, expected failures=5)
 }}}
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018e8ac310c8-e1f62106-db07-44ee-bece-8b3d1b2f80e2-00%40eu-central-1.amazonses.com.

Re: [Django] #32831: Some cache tests are flaky (e.g. cache.tests.BaseMemcachedTests.test_touch)

[Django]

#32831: Some cache tests are flaky (e.g. 
cache.tests.BaseMemcachedTests.test_touch)
--+
 Reporter:  Chris Jerdonek|Owner:  (none)
 Type:  Cleanup/optimization  |   Status:  new
Component:  Core (Cache system)   |  Version:  dev
 Severity:  Normal|   Resolution:
 Keywords:| Triage Stage:  Accepted
Has patch:  0 |  Needs documentation:  0
  Needs tests:  0 |  Patch needs improvement:  0
Easy pickings:  0 |UI/UX:  0
--+

Comment (by Natalia Bidart):

 I started some debugging efforts [https://forum.djangoproject.com/t/flaky-
 cache-tests/26990/4 in this forum post] and landed a fix for `test_touch`
 when using [https://github.com/django/django/pull/17614 the filebased
 cache backend].

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018d31278eec-18930814-b6bb-4462-a7a4-6c9d22552132-00%40eu-central-1.amazonses.com.

Re: [Django] #32831: Some cache tests are flaky (e.g. cache.tests.BaseMemcachedTests.test_touch)

[Django]

#32831: Some cache tests are flaky (e.g. 
cache.tests.BaseMemcachedTests.test_touch)
--+
 Reporter:  Chris Jerdonek|Owner:  (none)
 Type:  Cleanup/optimization  |   Status:  new
Component:  Core (Cache system)   |  Version:  dev
 Severity:  Normal|   Resolution:
 Keywords:| Triage Stage:  Accepted
Has patch:  0 |  Needs documentation:  0
  Needs tests:  0 |  Patch needs improvement:  0
Easy pickings:  0 |UI/UX:  0
--+
Changes (by Chris Jerdonek):

 * owner:  Chris Jerdonek => (none)
 * status:  assigned => new


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.1f960a33d0ae708e58c1fbafd8917098%40djangoproject.com.

Re: [Django] #32831: Some cache tests are flaky (e.g. cache.tests.BaseMemcachedTests.test_touch)

[Django]

#32831: Some cache tests are flaky (e.g. 
cache.tests.BaseMemcachedTests.test_touch)
-+-
 Reporter:  Chris Jerdonek   |Owner:  Chris
 Type:   |  Jerdonek
  Cleanup/optimization   |   Status:  assigned
Component:  Core (Cache system)  |  Version:  dev
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Chris Jerdonek):

 Another flaky test failure: `test_forever_timeout()` at this line--
 
https://github.com/django/django/blob/1697098795707bd39501d9ceddd3d9be93f8d549/tests/cache/tests.py#L606
 (again with `pull-requests-bionic/database=spatialite,label=bionic-
 pr,python=python3.8`).

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.4320119183ae38dc609046380ef8414d%40djangoproject.com.

Re: [Django] #32831: Some cache tests are flaky (e.g. cache.tests.BaseMemcachedTests.test_touch)

[Django]

#32831: Some cache tests are flaky (e.g. 
cache.tests.BaseMemcachedTests.test_touch)
-+-
 Reporter:  Chris Jerdonek   |Owner:  Chris
 Type:   |  Jerdonek
  Cleanup/optimization   |   Status:  assigned
Component:  Core (Cache system)  |  Version:  dev
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Chris Jerdonek):

 * owner:  nobody => Chris Jerdonek
 * status:  new => assigned


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.5f7781c805348159d349d4ee3fcd04af%40djangoproject.com.

Re: [Django] #32831: Some cache tests are flaky (e.g. cache.tests.BaseMemcachedTests.test_touch) (was: Flaky test in CI: cache.tests.BaseMemcachedTests.test_touch)

[Django]

#32831: Some cache tests are flaky (e.g. 
cache.tests.BaseMemcachedTests.test_touch)
--+
 Reporter:  Chris Jerdonek|Owner:  nobody
 Type:  Cleanup/optimization  |   Status:  new
Component:  Core (Cache system)   |  Version:  dev
 Severity:  Normal|   Resolution:
 Keywords:| Triage Stage:  Accepted
Has patch:  0 |  Needs documentation:  0
  Needs tests:  0 |  Patch needs improvement:  0
Easy pickings:  0 |UI/UX:  0
--+

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/067.0777fc21608ed3ac913bf06aad44b949%40djangoproject.com.