[Python-Dev] Re: review for doctest fix: bpo-35753

[Alfred Perlstein]

On 11/6/20 6:13 PM, Steven D'Aprano wrote:

For the benefit of others, the problem is that
`unittest.mock.call.__wrapped__` generates a new object, which in turn
has a dynamic `__wrapped__` attribute, which does the same, thus
generating an infinite chain of *distinct* proxies.

Being distinct proxy objects defeats the loop detection algorithm of
`inspect.unwrap`, which raises ValueError when the recursion limit is
reached, and that breaks doctest.

(I am explicitly stating that here because I spent an embarassingly long
time misunderstanding the nature of the bug, then more time digging into
the PR and bug track issues to understand what it was, rather than what
it isn't. Maybe I can save anyone else from my misunderstanding.)

I'm not convinced that this should be fixed by catching the ValueError
inside doctest. Or at least, not *just* by doing so. There's a deeper
problem that should be fixed, outside of doctest. Looking at the
similar issue here:

https://bugs.python.org/issue25532

`mock.call` has broken other functions in the past, and will probably
continue to do so in the future.

I don't think this infinite chain is intentional, I think it just
happens by accident, which makes this a bug in `call`. I think.

Michael Foord (creator of mock, if I recall correctly) suggested
blacklisting `__wrapped__` from the proxying:

https://bugs.python.org/issue25532#msg254726

which I think is the right solution, rather than touching doctest.

Michael also said he wasn't happy with an arbitrary limit on the depth
of proxies, but I would say that limiting the depth to
sys.getrecursionlimit() is not arbitrary and should avoid or at least
mitigate the risk of infinite loops and/or memory exhaustion in the
general case of arbitrary attribute lookups:


 py> a = unittest.mock.call
 py> for i in range(5):  # for arbitrary large values of 5
 ... a = a.wibble
 ...
 py> a
 wibble.wibble.wibble.wibble.wibble


I'm not a mock expert, but I guess such mock dynamic lookups should be
limited to the recursion limit. Currently they will loop forever or
until you run out of memory.

Setting `call.__wrapped__` to None seems to directly fix the problem
with doctest:

 [steve@susan test]$ cat demo2.py
 """
 Run doctest on this module.
 """
 
 from unittest.mock import call

 call.__wrapped__ = None

 [steve@susan test]$ python3.9 -m doctest -v demo2.py
 1 items had no tests:
 demo2
 0 tests in 1 items.
 0 passed and 0 failed.
 Test passed.


but I don't know if that will break any uses of `mock.call`.

Another fix (untested) would be to explicitly test for mocked call:

 inspect.unwrap(val, stop=lambda obj: isinstance(obj, unittest.mock._Call))

but I don't like that much.

I agree.  It would be nice to fix mock.call, however that would leave a 
gap in doctest which it can fail for strange objects that behave 
similarly for no good reason.


Source: https://bugs.python.org/issue25532#msg273272

> Nick Coghlan (ncoghlan)
> unittest.mock.Mock is the only case of that in the standard library, 
but it's far from the only Python mocking library out there, and we 
should give a clear exception in such cases, rather than eating up all 
the memory on the machine.


Doctest existed for a very long time ~2001 through 2014 without the 
"vulnerable" (sorry, just calling it that) call to "unwrap" in place, 
hence the unwrap itself is not really key to doctest working or not.


Adding unwrap without the check for unwrap failing is a bug on par with 
the unwrap problem with mock.call itself.


Based on defensive coding and that doctest should not fail in difficult 
to understand ways I think it makes sense to me to fix both mock.call 
AND doctest, not only one.


-Alfred

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/T6ZS2YMOIAJIZCU443MQPEC2UHNY26W6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Ideas for improving the contribution experience

[Inada Naoki]

On Sat, Oct 17, 2020 at 7:31 AM Tal Einat  wrote:
>
>
> 4. Encourage core devs to dedicate some of their time to working through 
> issues/PRs which are "ignored" or "stalled". This would require first 
> generating reliable lists of issues/PRs in such states. This could be in 
> various forms, such as predefined GitHub/b.p.o. queries, a dedicated 
> web-page, a periodic message similar to b.p.o.'s "weekly summary" email, or 
> dedicated tags/labels for issues/PRs. (Perhaps prioritize "stalled" over 
> "ignored".)
>

I looked random pull requests during the core sprint and I have merged
dozen pull requests.
But it is difficult to find any pull requests I can do anything. See this list:

open + awaiting merge: https://github.com/python/cpython/labels/awaiting%20merge
open + awaiting core review:
https://github.com/python/cpython/labels/awaiting%20merge

* Some PR is "owned" by some core developer already. We just need to
wait them. (Although assignee is not set in most case)
* Some PR is under discussion in b.p.o. issue.
* Some PR requires expertise at some areas.

etc...

Significant effort is required to find PRs I can help. My ideas:

* Recommend to set "assignee" more aggressive.
  * "Assign me" when you will review/work on it later.
  * Check "assign to me" when you have time.
  * Feel free to un-assign.
* Add "waiting who's help" in the issue title.
  * E.g. "needs compiler experts", "xml", "http", "mail", etc...

It will help to find PRs I can help in my spare time.

Regards,



--
Inada Naoki  
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/U66XTG6YWPWRQZMFVRQJ3PPLPWWCEEKR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Distro packagers: PEP 615 and the tzdata dependency

[Petr Viktorin]

On 11/24/20 7:50 PM, Brett Cannon wrote:
If enough people were interested we could create a "Distributors" 
category on discuss.python.org .


I'd join :)



On Tue, Nov 24, 2020 at 9:08 AM Tianon Gravi > wrote:


 > I'd love to have an easy way to keep them in the loop.

I'm one of the maintainers on
https://github.com/docker-library/python

(which is what results in https://hub.docker.com/_/python
), and I'd
love to have an easy way to keep myself in the loop too! O:)

Is there a lower-frequency mailing list where things like this are
normally posted that I could follow?
(I don't want to be a burden, although we'd certainly really love to
have more upstream collaboration on that repo -- we do our best to
represent upstream as correctly/accurately as possible, but we're not
experts!)

 > would it make sense to add a packaging section to our
documentation or
 > to write an informational PEP?

FWIW, I love the idea of an explicit "packaging" section in the docs
(or a PEP), but I've maintained that for other projects before and
know it's not always easy or obvious. :)

♥,
- Tianon
   4096R / B42F 6819 007F 00F8 8E36  4FD4 036A 9C25 BF35 7DD4

PS. thanks doko for giving me a link to this thread! :D

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/66HPNHT576JKSFOQXJTCACX5JRNERMWV/
Code of Conduct: http://python.org/psf/codeofconduct/