[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Opened https://bugs.python.org/issue45128

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I'm planning to revert PR 28185 because this is blocking the release

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

This seem to have caused several errors that have manifested on the release of 
3.10.0rc2:

test test_multiprocessing_fork failed -- Traceback (most recent call last):
  File "/tmp/tmpu30qfjpr/installation/lib/python3.10/unittest/mock.py", line 
1239, in _dot_lookup
return getattr(thing, comp)
AttributeError: module 'multiprocessing' has no attribute 'shared_memory'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File 
"/tmp/tmpu30qfjpr/installation/lib/python3.10/test/_test_multiprocessing.py", 
line 3818, in test_shared_memory_basics
with unittest.mock.patch(
  File "/tmp/tmpu30qfjpr/installation/lib/python3.10/unittest/mock.py", line 
1422, in __enter__
self.target = self.getter()
  File "/tmp/tmpu30qfjpr/installation/lib/python3.10/unittest/mock.py", line 
1609, in 
getter = lambda: _importer(target)
  File "/tmp/tmpu30qfjpr/installation/lib/python3.10/unittest/mock.py", line 
1252, in _importer
thing = _dot_lookup(thing, comp, import_path)
  File "/tmp/tmpu30qfjpr/installation/lib/python3.10/unittest/mock.py", line 
1242, in _dot_lookup
return getattr(thing, comp)
AttributeError: module 'multiprocessing' has no attribute 'shared_memory'

0:09:11 load avg: 0.71 [231/427/1] test_multiprocessing_forkserver -- 
test_multiprocessing_fork failed (1 error) in 1 min 11 sec
test test_multiprocessing_forkserver failed -- Traceback (most recent call 
last):
  File "/tmp/tmpu30qfjpr/installation/lib/python3.10/unittest/mock.py", line 
1239, in _dot_lookup
return getattr(thing, comp)
AttributeError: module 'multiprocessing' has no attribute 'shared_memory'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File 
"/tmp/tmpu30qfjpr/installation/lib/python3.10/test/_test_multiprocessing.py", 
line 3818, in test_shared_memory_basics
with unittest.mock.patch(
  File "/tmp/tmpu30qfjpr/installation/lib/python3.10/unittest/mock.py", line 
1422, in __enter__
self.target = self.getter()
  File "/tmp/tmpu30qfjpr/installation/lib/python3.10/unittest/mock.py", line 
1609, in 
getter = lambda: _importer(target)
  File "/tmp/tmpu30qfjpr/installation/lib/python3.10/unittest/mock.py", line 
1252, in _importer
thing = _dot_lookup(thing, comp, import_path)
  File "/tmp/tmpu30qfjpr/installation/lib/python3.10/unittest/mock.py", line 
1242, in _dot_lookup
return getattr(thing, comp)
AttributeError: module 'multiprocessing' has no attribute 'shared_memory'

0:11:11 load avg: 0.93 [232/427/2] test_multiprocessing_main_handling -- 
test_multiprocessing_forkserver failed (1 error) in 2 min
0:11:18 load avg: 1.09 [233/427/2] test_multiprocessing_spawn
test test_multiprocessing_spawn failed -- Traceback (most recent call last):
  File "/tmp/tmpu30qfjpr/installation/lib/python3.10/unittest/mock.py", line 
1239, in _dot_lookup
return getattr(thing, comp)
AttributeError: module 'multiprocessing' has no attribute 'shared_memory'

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Nikita Sobolev]

Nikita Sobolev  added the comment:

Closing and moving to https://bugs.python.org/issue45125

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[miss-islington]

miss-islington  added the comment:


New changeset 6b5aea2dc1bf7e117d40d6c9035d5c13124fd968 by Miss Islington (bot) 
in branch '3.10':
bpo-45052: Unskips a failing `test_shared_memory_basics` test (GH-28182)
https://github.com/python/cpython/commit/6b5aea2dc1bf7e117d40d6c9035d5c13124fd968


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Nikita Sobolev]

Nikita Sobolev  added the comment:

I think that adding extra tests and docs are two separate new tasks. I will
open them today.

Thanks a lot, everyone!

пн, 6 сент. 2021 г. в 19:56, Steve Dower :

>
> Steve Dower  added the comment:
>
> The test fix looks good to me. That resolves this issue, yes? The other
> work is going on elsewhere?
>
> --
> nosy:  -miss-islington
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Steve Dower]

Steve Dower  added the comment:

The test fix looks good to me. That resolves this issue, yes? The other work is 
going on elsewhere?

--
nosy:  -miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 9.0 -> 10.0
pull_requests: +26613
pull_request: https://github.com/python/cpython/pull/28185

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Steve Dower]

Steve Dower  added the comment:


New changeset 19871fce3b74fc3f37e334a999e00d0ef65a8f1e by Nikita Sobolev in 
branch 'main':
bpo-45052: Unskips a failing `test_shared_memory_basics` test (GH-28182)
https://github.com/python/cpython/commit/19871fce3b74fc3f37e334a999e00d0ef65a8f1e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Nikita Sobolev]

Change by Nikita Sobolev :


--
keywords: +patch
pull_requests: +26611
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28182

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Nikita Sobolev]

Nikita Sobolev  added the comment:

Ok, that's what I was able to find.

Original BPO: https://bugs.python.org/issue35813
Original PRs (sources + tests):
- https://github.com/python/cpython/pull/11664 it was a little bit "rushed"
- https://github.com/python/cpython/pull/11816 follow up

Related: https://bugs.python.org/issue38169

1. The test fails because `sms.size != sms2.size`
2. The `sms2` object is a pickle-unpickle copy of `sms`
3. The `__reduce__` method defined in `SharedMemory` returns `(name=self.name, 
create=False, size=self.size)` tuple
4. So, when `sms2` is constructed, `size` is ignored, because `create` is 
`False`

Moreover, even when `size` is passed with `create=True` the docs say that the 
final amount of allocated memory is platform specific (no details on the exact 
platforms): it can be larger or equal. Link: 
https://github.com/python/cpython/blame/main/Doc/library/multiprocessing.shared_memory.rst#L61
 

So, this statement makes `self.assertEqual(sms.size, sms2.size)` potentially 
flaky.

After reading all this my suggestion is:
1. Remove `self.assertEqual(sms.size, sms2.size)` line, since this behavior is 
not guaranteed to work
2. Add docs about how `pickle` works with `SharedMemory` / `ShareableList`. 
Right now it is not clear. This should be just an explanation of the current 
status-quo
3. Add more tests on unpickled `Shared*` objects. Including overflowing set 
memory limits on unpickled objects and other implemetation details

I will start with `1.` right now, later I can work on `3.`.
But, I feel like someone more knowledgeable should take `2.` (or at least guide 
me, since `SharedMemory` is not something I use everyday).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Eryk Sun]

Eryk Sun  added the comment:

> It may be a bug in the constructor of SharedMemory. It ignores 
> the size argument on Windows.

The `size` argument is always ignored when `create` is false, on all platforms, 
not that I understand the reason for it. The difference compared to POSIX is 
that the size of a mapping that's backed by the paging file is rounded up to a 
multiple of the page size in Windows (e.g. 4 KiB).

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

It may be a bug in the constructor of SharedMemory. It ignores the size 
argument on Windows.

--
components: +Library (Lib), Windows
nosy: +davin, paul.moore, pitrou, serhiy.storchaka, steve.dower, tim.golden, 
zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Nikita Sobolev]

Nikita Sobolev  added the comment:

I would like to work on this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45052] WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows

[Nikita Sobolev]

New submission from Nikita Sobolev :

While working on https://github.com/python/cpython/pull/28060 we've noticed 
that 
`test.test_multiprocessing_spawn.WithProcessesTestSharedMemory.test_shared_memory_basics`
 fails on Windows:

```
==
FAIL: test_shared_memory_basics 
(test.test_multiprocessing_spawn.WithProcessesTestSharedMemory)
--
Traceback (most recent call last):
  File "D:\a\cpython\cpython\lib\test\_test_multiprocessing.py", line 3794, in 
test_shared_memory_basics
self.assertEqual(sms.size, sms2.size)
^
AssertionError: 512 != 4096
```

For now it is ignored.

Related issue: https://bugs.python.org/issue45042

--
components: Tests
messages: 400646
nosy: sobolevn
priority: normal
severity: normal
status: open
title: WithProcessesTestSharedMemory.test_shared_memory_basics fails on Windows
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com