[issue47188] ncurses: *** buffer overflow detected ***: terminated with -D_FORTIFY_SOURCE=3
Martin Liška added the comment: So the issue is ncurses, closing here. -- stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue47188> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47188] ncurses: *** buffer overflow detected ***: terminated with -D_FORTIFY_SOURCE=3
New submission from Martin Liška :
Note -D_FORTIFY_SOURCE=3 will come newly with GCC12. So I noticed the following
error:
demo.py:
```python
import curses
curses.initscr()
curses.unget_wch('a')
```
Error message:
*** buffer overflow detected ***: terminated
Backtrace:
#0 __pthread_kill_implementation (threadid=,
signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1 0x77d1e1e3 in __pthread_kill_internal (signo=6, threadid=) at pthread_kill.c:78
#2 0x77cce306 in __GI_raise (sig=sig@entry=6) at
../sysdeps/posix/raise.c:26
#3 0x77cb7813 in __GI_abort () at abort.c:79
#4 0x77d111b7 in __libc_message (action=action@entry=do_abort,
fmt=fmt@entry=0x77e573cf "*** %s ***: terminated\n") at
../sysdeps/posix/libc_fatal.c:155
#5 0x77db630a in __GI___fortify_fail (msg=msg@entry=0x77e57375
"buffer overflow detected") at fortify_fail.c:26
#6 0x77db48b6 in __GI___chk_fail () at chk_fail.c:28
#7 0x77db5be8 in __wcrtomb_chk (s=s@entry=0xaae440 "\376\271\255",
wchar=wchar@entry=97 L'a', ps=ps@entry=0x7fffd4f0, buflen=buflen@entry=1)
at wcrtomb_chk.c:31
#8 0x77a18b31 in wcrtomb (__ps=, __wchar=, __s=, __s=, __wchar=,
__ps=) at /usr/include/bits/wchar2.h:402
#9 unget_wch_sp (sp=0xab0920, wch=97 L'a') at
../ncurses/./widechar/lib_unget_wch.c:89
#10 0x77a18b61 in unget_wch (wch=) at
../ncurses/./widechar/lib_unget_wch.c:113
#11 0x77a55be5 in _curses_unget_wch (module=, ch='a') at
/home/marxin/Programming/cpython/Modules/_cursesmodule.c:4497
#12 0x006f6669 in cfunction_vectorcall_O (func=, args=0x77b355b0,
nargsf=, kwnames=0x0) at Objects/methodobject.c:512
#13 0x0042d0e8 in _PyObject_VectorcallTstate (kwnames=0x0,
nargsf=, args=, callable=, tstate=)
at ./Include/cpython/abstract.h:114
#14 PyObject_Vectorcall (kwnames=0x0, nargsf=, args=, callable=) at ./Include/cpython/abstract.h:123
#15 call_function (kwnames=0x0, oparg=, pp_stack=, bounds=0x7fffd640, tstate=0xa70520) at Python/ceval.c:5379
#16 _PyEval_EvalFrameDefault (tstate=, f=,
throwflag=) at Python/ceval.c:3772
So as seen __wcrtomb_chk is called with buflen == 1 and the function aborts if:
size_t
__wcrtomb_chk (char *s, wchar_t wchar, mbstate_t *ps, size_t buflen)
{
/* We do not have to implement the full wctomb semantics since we
know that S cannot be NULL when we come here. */
if (buflen < MB_CUR_MAX)
__chk_fail ();
return __wcrtomb (s, wchar, ps);
}
Where MB_CUR_MAX == 6.
So the question is if the issue is in libcurses library (that is compiler with
-D_FORTIFY_SOURCE=3), or in Modules/_cursesmodule.c?
------
messages: 416495
nosy: Martin Liška
priority: normal
severity: normal
status: open
title: ncurses: *** buffer overflow detected ***: terminated with
-D_FORTIFY_SOURCE=3
___
Python tracker
<https://bugs.python.org/issue47188>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43818] Email does not apply policy to multipart messages with defects
Martin Dengler added the comment: Possible related to https://github.com/python/cpython/pull/32137 and https://bugs.python.org/issue43323 -- ___ Python tracker <https://bugs.python.org/issue43818> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43818] Email does not apply policy to multipart messages with defects
Change by Martin Dengler : -- nosy: +mdengler ___ Python tracker <https://bugs.python.org/issue43818> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43323] UnicodeEncodeError: surrogates not allowed when parsing invalid charset
Change by Martin Dengler : -- nosy: +mdengler ___ Python tracker <https://bugs.python.org/issue43323> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46913] UBSAN: test_ctypes, test_faulthandler and test_hashlib are failing
Martin Panter added the comment: The ctypes overflow is probably the same as described in Issue 28169 and Issue 15119 -- nosy: +martin.panter ___ Python tracker <https://bugs.python.org/issue46913> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46880] zipfile library doesn't extract windows zip files properly on linux
Martin Fischer added the comment: Can you attach such a .zip file so that others can reproduce the bug? -- nosy: +push-f ___ Python tracker <https://bugs.python.org/issue46880> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46884] [doc] msilib.rst uses data directive to document modules
Martin Fischer added the comment: ([2] was meant to be an online link: https://docs.python.org/3.9/py-modindex.html#cap-m) -- ___ Python tracker <https://bugs.python.org/issue46884> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46884] [doc] msilib.rst uses data directive to document modules
Change by Martin Fischer : -- keywords: +patch pull_requests: +29744 stage: -> patch review pull_request: https://github.com/python/cpython/pull/31620 ___ Python tracker <https://bugs.python.org/issue46884> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46884] [doc] msilib.rst uses data directive to document modules
New submission from Martin Fischer : As per [1] the py:data directive describes data in a module. It should not be used for submodules, that's what the module directive is for. A side-effect that this is causing is that msilib.schema, msilib.sequence and msilib.text do not show up in the Python Module Index[2] as they should. [1]: https://www.sphinx-doc.org/en/master/usage/restructuredtext/domains.html [2]: file:///home/martin/repos-contrib/cpython/Doc/build/html/py-modindex.html#cap-m -- assignee: docs@python components: Documentation messages: 414209 nosy: docs@python, push-f priority: normal severity: normal status: open title: [doc] msilib.rst uses data directive to document modules type: enhancement versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46884> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46879] [doc] incorrect sphinx object names
Change by Martin Fischer : -- keywords: +patch pull_requests: +29739 stage: -> patch review pull_request: https://github.com/python/cpython/pull/31615 ___ Python tracker <https://bugs.python.org/issue46879> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46879] [doc] incorrect sphinx object names
New submission from Martin Fischer : API members documented in sphinx have an object name, which allow the documentation to be linked from other projects. Sphinx calculates the object name by prefixing the current module name to the directive argument, e.g: .. module:: foo .. function:: bar.baz becomes foo.bar.baz. Since these anchors aren't displayed in the documentation, some mistakes have crept in, namely the Python stdlib documentation currently contains the objects: * asyncio.asyncio.subprocess.DEVNULL * asyncio.asyncio.subprocess.PIPE * asyncio.asyncio.subprocess.STDOUT * asyncio.asyncio.subprocess.Process * multiprocessing.sharedctypes.multiprocessing.Manager * xml.etree.ElementTree.xml.etree.ElementInclude As can be observed in the URL fragments: https://docs.python.org/3/library/asyncio-subprocess.html#asyncio.asyncio.subprocess.Process https://docs.python.org/3/library/multiprocessing.html#multiprocessing.sharedctypes.multiprocessing.Manager https://docs.python.org/3/library/xml.etree.elementtree.html#xml.etree.ElementTree.xml.etree.ElementInclude.default_loader I have a patch, prepared, I'll send a PR straight away. -- assignee: docs@python components: Documentation messages: 414192 nosy: docs@python, push-f priority: normal severity: normal status: open title: [doc] incorrect sphinx object names versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46879> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46877] [doc] unittest.doModuleCleanups() does not exist
Change by Martin Fischer : -- title: unittest.doModuleCleanups() does not exist -> [doc] unittest.doModuleCleanups() does not exist ___ Python tracker <https://bugs.python.org/issue46877> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46877] unittest.doModuleCleanups() does not exist
New submission from Martin Fischer : The unittest documentation[1] describes unittest.doModuleCleanups(). That function however doesn't exist since it's only in the unittest.case module and not re-exported in the unittest module (unlike addModuleCleanup). So I think either the documentation should be corrected or doModuleCleanups should be re-exported in unittest/__init__.py to match the documentation. [1]: https://docs.python.org/3.8/library/unittest.html -- assignee: docs@python components: Documentation messages: 414177 nosy: docs@python, lisroach, michael.foord, push-f priority: normal severity: normal status: open title: unittest.doModuleCleanups() does not exist type: enhancement versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue46877> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46740] Improve Telnetlib's throughput
Change by Martin Kirchgessner : -- keywords: +patch pull_requests: +29577 stage: -> patch review pull_request: https://github.com/python/cpython/pull/31449 ___ Python tracker <https://bugs.python.org/issue46740> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46756] Incorrect authorization check in urllib.request
Martin Panter added the comment: Maybe the same as Issue 42766? -- nosy: +martin.panter ___ Python tracker <https://bugs.python.org/issue46756> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46738] Allow http.server to emit HTML 5
Change by Martin Panter : -- superseder: -> Generate HTML 5 with SimpleHTTPRequestHandler.list_directory ___ Python tracker <https://bugs.python.org/issue46738> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46740] Improve Telnetlib's throughput
New submission from Martin Kirchgessner : While using `telnetlib` I sometimes received unusually "large" messages (around 1Mb) from another process on the same machine, and was surprised `read_until` took more than a second. After instrumenting I discovered such messages were received at roughly 500kbyte/s. I think this low throughput comes from two implementation details: - `Telnet.fill_rawq` is calling `self.sock.recv(50)`, whereas 4096 is now recommended - the `Telnet.process_rawq` method is transferring from raw queue to cooked queue by appending byte per byte. For the latter, transferring by slices looks much faster (I'm measuring at least 5x). I'm preparing a PR. -- components: Library (Lib) messages: 413195 nosy: martin_kirch priority: normal severity: normal status: open title: Improve Telnetlib's throughput type: resource usage versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue46740> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46537] zipfile crash on windows, detected during pip install pyodbc-4.0.32-cp310-cp310-win_amd64.whl
Martin Forster added the comment: The wheel file was corrupt, due to git taking care of "CRLF/LF" conversion. Sorry for the fuss -- resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue46537> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46537] zipfile crash on windows, detected during pip install pyodbc-4.0.32-cp310-cp310-win_amd64.whl
Martin Forster added the comment: i opened a issue at pyodbc as well, https://github.com/mkleehammer/pyodbc/issues/1015 -- ___ Python tracker <https://bugs.python.org/issue46537> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46537] zipfile crash on windows, detected during pip install pyodbc-4.0.32-cp310-cp310-win_amd64.whl
Martin Forster added the comment: it tried one built on my own, and the one from https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyodbc -- ___ Python tracker <https://bugs.python.org/issue46537> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46537] zipfile crash on windows, detected during pip install pyodbc-4.0.32-cp310-cp310-win_amd64.whl
New submission from Martin Forster : pyth10n 3.10.1, windows x64 install of yodbc-4.0.32-cp310-cp310-win_amd64.whl crashes. pos: 74196 pos: 74226 pos: -3 ERROR: Could not install packages due to an OSError. Traceback (most recent call last): File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\commands\install.py", line 390, in run installed = install_given_reqs( File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\req\__init__.py", line 73, in install_given_reqs requirement.install( File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\req\req_install.py", line 758, in install install_wheel( File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 794, in install_wheel _install_wheel( File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 637, in _install_wheel file.save() File "C:\git\idfunctiondb\venv\lib\site-packages\pip\_internal\operations\install\wheel.py", line 404, in save with self._zip_file.open(zipinfo) as f: File "C:\Python310\lib\zipfile.py", line 1520, in open fheader = zef_file.read(sizeFileHeader) File "C:\Python310\lib\zipfile.py", line 743, in read self._file.seek(self._pos) OSError: [Errno 22] Invalid argument Please note i added the line 742 with the content: print(str(self) + ' pos: ' + str(self._pos) ) to help to find the root cause. it Looks like that file.tell, returns a negative number, which kill file.seek. -- components: Windows messages: 411750 nosy: Martin-Forster, paul.moore, steve.dower, tim.golden, zach.ware priority: normal severity: normal status: open title: zipfile crash on windows, detected during pip install pyodbc-4.0.32-cp310-cp310-win_amd64.whl type: crash versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue46537> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46517] Review exception handling in urllib
Martin Panter added the comment:
The linked code is for urllib.parse.urlencode, looking something like
try:
if len(query) and not isinstance(query[0], tuple):
raise TypeError
except TypeError:
ty, va, tb = sys.exc_info()
raise TypeError("not a valid non-string sequence "
"or mapping object").with_traceback(tb)
I guess it raises twice so that the error message is not duplicated in the
code. The author probably also wants the TypeError initially raised from the
"len(query)" and "query[0]" operations to get the same "not a valid . . ."
message.
Regarding the OSError, originally it was catching socket.error and raising
IOError. I guess someone only wanted the caller to have catch IOError and not
need to import the socket module. Later these exception types became aliases of
each other.
Anyway, the URLopener class is documented as deprecated, so is it really worth
changing anything in that?
--
nosy: +martin.panter
___
Python tracker
<https://bugs.python.org/issue46517>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46375] io.BytesIO does not have peek()
Marcel Martin added the comment: I opened a PR, but I now wonder whether the missing peek() is by design. First, I noticed that instead of using BytesIO directly, I can wrap the instance in an io.BufferedReader, which does have peek(). (It’s just a bit inconvenient.) The second thing is that BytesIO is currently documented to inherit from BufferedIOBase, but if peek() is implemented, one could argue that BytesIO now should inherit from BufferedReader because it then has all the methods. And that seems to great a change from my perspective. I’ll defer to someone more knowledgeable and do not mind at all if this issue is closed without action. -- ___ Python tracker <https://bugs.python.org/issue46375> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46375] io.BytesIO does not have peek()
Change by Marcel Martin : -- keywords: +patch pull_requests: +28996 stage: -> patch review pull_request: https://github.com/python/cpython/pull/30808 ___ Python tracker <https://bugs.python.org/issue46375> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46375] io.BytesIO does not have peek()
New submission from Marcel Martin : It would be great to be able to use peek() on BytesIO objects. I have a function that gets passed a file-like object and uses peek() on it. This works for nearly all types of files relevant in my library, except BytesIO instances (which I use during testing), for which I need to add a small workaround using tell() and seek(). -- components: Library (Lib) messages: 410552 nosy: marcelm priority: normal severity: normal status: open title: io.BytesIO does not have peek() type: enhancement versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue46375> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46352] Steps To Do Arlo Setup
Change by Jacob Martin : -- components: Installation files: Arlo camera setup.jpg nosy: jacobmartin717 priority: normal severity: normal status: open title: Steps To Do Arlo Setup type: security versions: Python 3.8 Added file: https://bugs.python.org/file50558/Arlo camera setup.jpg ___ Python tracker <https://bugs.python.org/issue46352> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46319] datetime.utcnow() should return a timezone aware datetime
Martin Panter added the comment: Perhaps this is a duplicate of Issue 12756? -- nosy: +martin.panter superseder: -> datetime.datetime.utcnow should return a UTC timestamp ___ Python tracker <https://bugs.python.org/issue46319> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40059] Provide a toml module in the standard library
Martin Reboredo added the comment: > Not officially, no. But I'm personally not going to bring it forward right > now. If someone else wants to formulate a complete proposal for the SC on > this then they are definitely welcome to! You will need to address where the > code is coming from, why that code should be used, what's the API, etc. > > The only reason the SC is mentioned here is there will be a discussion about > how to maintain the stdlib, but it simply hasn't happened yet. You don't have > to wait for it and asking for a TOML module might actually force the issue. I've been working on a document detailing the API implementation (as a PEP draft) see https://gitlab.com/YakoYakoYokuYoku/pep-toml. Although I've yet to write the code I'll to move forward with it. -- ___ Python tracker <https://bugs.python.org/issue40059> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46062] In IDLE, 'File > Save As' seems not to allow creation of a new folder
New submission from Martin Whitehead : The problem: In both the IDLE shell and the editor, 'File > Save As' seems not to allow creation of a new folder. My system: IDLE v3.8.10 (64 bit) Python 3.8.10 Tk version 8.6.10 Linus Mint 20.2 Uma (64 bit), which is based on Ubuntu 20.04 LTS (Focal Fossa) -- assignee: terry.reedy components: IDLE messages: 408437 nosy: suffolkpunch, terry.reedy priority: normal severity: normal status: open title: In IDLE, 'File > Save As' seems not to allow creation of a new folder versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue46062> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40059] Provide a toml module in the standard library
Martin Reboredo added the comment: A new python-ideas mail thread was created for this, you can check it out at https://mail.python.org/archives/list/python-id...@python.org/thread/IWJ3I32A4TY6CIVQ6ONPEBPWP4TOV2V7/. -- nosy: +YakoYakoYokuYoku ___ Python tracker <https://bugs.python.org/issue40059> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: Thanks for your definitive answer, this is what I was waiting for. I understand and I totally agree that subclassing is the way to go to make traceback more flexible. Would you mind linking the other issues concerning the general improvement of traceback? -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: Irit, would you be able to take a look at the patch? --- I found another application scenario for the patch: In Numpy and Pandas, the assert_* functions in testing have a __tracebackhide__ local that advises pytest to hide the frame. With the patch in place, we could at least not display any locals if __tracebackhide__ is present. -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: Thanks Joe! > 1. The changes are sufficient to let the user make things work the way it is > requested that they work and anyone who does not start using the new > format_locals parameter will get the old behavior. That was my goal :) > 2. A side comment: I just noticed that the docstring for > FrameSummary.__init__ does not document the filename, lineno, and name > parameters. Perhaps this could be fixed at the same time? I agree and already updated the pull request. > 3. The new parameter format_locals is slightly confusingly named, because it > does not format a single string from all the locals but instead gives a > dictionary of strings with one string for each local. I think format_locals is OK here (local*s*: plural), but I'm open to better suggestions. > 4. I personally think it would be better to include something like the > example value for format_locals as an extra attribute of the traceback module > so everybody doesn't have to write the same function. That said, the > documented example is sufficient. I sort of agree, but I also don't know what such a general-purpose function would entail. Suggestions? > 5. It is not clear to me why this stringify-ing of the locals can't be done > in StackSummary._extract_from_extended_frame_gen. It passes the dictionary > of locals and also the function to transform it to FrameSummary. It would > make more sense to transform it first. I suppose there might be some user > somewhere who is directly calling FrameSummary so the interface needs to stay. I agree! In principle, FrameSummary has been little more than a container without (much) logic of its own. Memory efficiency requires that that FrameSummary does not hold references to the original locals, thats why repr() was used. I think as well that it would have been better to keep FrameSummary as a mere container and do the conversion of the locals elsewhere. But at this point, this shouldn't be changed anymore. > 6. I fantasize that the new format_locals parameter would have access to the > frame object. In order for this to happen, the frame object would have to be > passed to FrameSummary instead of the 3 values (locals, name, filename) that > are extracted from it. I think the current interface of FrameSummary was > designed to interoperate with very old versions of Python. Oh well. Do you have a use case for that? I have no clue what you would do with the frame object at this point. > 7. If anyone wanted to ever refactor FrameSummary (e.g., to enable my fantasy > in point #6 just above), it becomes harder after this. This change has the > effect of exposing details of the interface of FrameSummary to users of > StackSummary.extract and TracebackException. The four parameters that the > new parameter format_locals takes are most of the parameters of FrameSummary. Previously, I totally misread your request to "not just the local variable values, but also the name of the local variable and maybe also the stack frame it is in". This is why I included filename, lineno and name as parameters to format_locals which now seems totally useless to me and I'll remove them (if nobody objects). -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: I improved the example in traceback.rst to illustrate how format_locals works. -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: Just to avoid misunderstandings: My pull request is not at all about silencing exceptions. It is about customizing the output of the traceback module. (Just like the introduction of capture_locals previously: #22936) (-X capture_locals, on the other hand, is a hypothetical idea that might help with debugging, but we don't have to discuss this now.) My main argument for the proposed change is that traceback is useless in certain situations, because capture_locals is not "robust" (it *might* raise exceptions; I could handle these in the calling code but that would again hide the stack trace that I was about to investigate) and might dump sensitive data like passwords into logfiles (msg237320, msg237323). Nothing is taken away or hidden, but flexibility is added. (The only other option to resolve these issues would be to re-implement much of the current functionality of traceback in a third-party module, which would lead to maintainability problems and fragmentation.) -- nosy: +rbcollins ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: I see two scenarious discussed here: Scenario 1 (Offline / Batch-Mode): A system runs user-supplied jobs that may fail. In the case of an error, the system shouldn't crash but rather store a maximally helpful message and carry on with the next job. Most likely, the relevant information is the situation that lead to the error in the first place, not that repr() has also gone wrong as a result. This could be a a system to automatically test the homework code of your students. Or, like in my case, a framework that runs experiments. You would very likely want a system that does not crash if a __repr__ is wrongly implemented but prints a readable traceback and carries on with the next job. Here, format_locals could be used to fall back to a different representation of an object if repr() fails. This is the use case that my pull request tries to address primarily. Scenario 2 (Online / Write, Test, Repeat): A user frequently rewrites and executes their code until the desired outcome appears. Errors inevitably happen. In this case, a maximally helpful stack trace is needed. Again, the relevant information is the situation that lead to the error in the first place, not that repr() has also gone wrong as a result. Yes, it would be hard for unexperienced users to come up with StackSummary.extract(format_locals=...) by themselves. But, the correct way would be to use a debugger anyway, not some hand-written code to print exceptions on the fly. However, if your students receive a template file where they fill in missing function bodies etc, there might already be a substantial amount of code which they don't understand at first, nor do they need to care. Therefore, a piece of code that formats exceptions nicely would hurt here. I think the most useful change for Scenario 2 (if, for some reason, a proper debugger is not an option) could be to add a command line option (e.g. -X capture_locals) so that a complete stack trace is printed if an exception bubbles up to interpreter level. (Here, it wouldn't hurt to handle exceptions in repr() because the interpreter already stopped exection anyway.) This option would be very easy to teach to inexperienced users. My pull request would provide preparation for this more extensive change. -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39228] traceback.FrameSummary does not handle exceptions from `repr()`
Martin added the comment: I submitted a pull request for the related issue43656: https://github.com/python/cpython/pull/29299 It introduces a format_locals parameter that enables the customized formatting of a FrameSummary. This way, a user of traceback could provide a function that handles exceptions within __repr__ if necessary. -- ___ Python tracker <https://bugs.python.org/issue39228> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41544] multiprocessing.dummy.Process lacks daemon parameter
Martin added the comment: Could someone have a look at my pull request? I have trouble with the tests. -- ___ Python tracker <https://bugs.python.org/issue41544> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41544] multiprocessing.dummy.Process lacks daemon parameter
Change by Martin : -- nosy: +davin, pitrou type: -> behavior ___ Python tracker <https://bugs.python.org/issue41544> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: Could the participants of this issue please have a look at my pull request: https://github.com/python/cpython/pull/29299 What do you like, what don't you like? Does it work for your use case? -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45665] Problems caused by isinstance(list[int], type) returning True
Martin Rueckl added the comment:
Sorry for the noise:
- Literal['a', 'b'],
- TypeVar('T')
Behave like Optional/Union
--
___
Python tracker
<https://bugs.python.org/issue45665>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45665] Problems caused by isinstance(list[int], type) returning True
Martin Rueckl added the comment:
One thing that probably should be considered in this context:
isinstance(arg, type) == issubclass(type(arg), type)
Holds True for arg in (Optional[X], Union[X, Y]). Both sides evaluate to False.
(v3.10.0)
While I still think both sides evaluating to True would be more intuitive, this
supports the proposed change.
Small test snippet:
```
from typing import Dict, List, Set, Tuple, Optional, Union
import pytest
@pytest.mark.parametrize('arg', [
list, List[int], list[int],
dict, Dict[str, int], dict[str, int],
set, Set[int], set[int],
tuple, Tuple[str, int], tuple[str, int],
Optional[int],
Union[int, str]
])
def test_invariant(arg):
same = isinstance(arg, type) == issubclass(type(arg), type)
result = "Check" if same else "Failed"
print(f"\n{result}: Testing: {arg=} with {type(arg)=}: {isinstance(arg,
type)=} <> {issubclass(type(arg), type)=}")
assert same
```
Any other commonly used annotations that could be added to the checklist?
--
nosy: +martinitus
___
Python tracker
<https://bugs.python.org/issue45665>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23597] Allow easy display of local variables in log messages?
Martin added the comment:
In this pull request[1] I introduced a `format_locals` which can be provided to
customize the serialization of the local variables of each frame. This could be
used to filter out variables with certain names, for example. In this context,
you could use a certain naming convention for variables with sensitive
information and filter them out:
def _format_locals(filename, lineno, name, locals):
return {k: repr(v) for k,v in locals.items() if not
k.endsswith("_sensitive")}
traceback.TracebackException.from_exception(e, capture_locals=True,
format_locals=format_locals).format()
(This should be exactly what Robert was suggesting.)
[1] https://github.com/python/cpython/pull/29299
--
keywords: +patch
message_count: 3.0 -> 4.0
nosy: +moi90
nosy_count: 4.0 -> 5.0
pull_requests: +27572
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/29299
___
Python tracker
<https://bugs.python.org/issue23597>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Change by Martin : -- pull_requests: +27563 stage: resolved -> patch review pull_request: https://github.com/python/cpython/pull/29299 ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: Once again a very good summary, thanks Joe! > it would be quite useful if this function parameter was given not just the > local variable values, but also the name of the local variable and maybe also > the stack frame it is in So this would be something like `format_locals(filename, lineno, name, locals) -> dict[str,str]` that could be used inside FrameSummary.__init__. I like that! I wouldn't bother with detecting un-repr-able objects in Python itself, but if the above `format_locals` was implemented, you could easily prepare such a formatter for your students that hides `self` et al. and/or catch exceptions during `repr` to your liking. What is needed to get an OK for such a change? -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45438] inspect not capturing type annotations created by __class_getitem__
Martin Rueckl added the comment: Just my two cents as a new contributor but long time user: - isinstance(list[int], type) returning False seems incredibly un-intuitive to me. I always see generics (e.g. list without type parameter) as higher kinded types, where passing a type argument via [] turns the hkt into a concrete type. - Backporting to 3.9 and 3.10 should be no issue - I am not deep enough in pythons type system to judge whether my PR is consistent/good. I really just tried out what works and "feels consistent". -- ___ Python tracker <https://bugs.python.org/issue45438> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: Irit, I'm unsure about the wording. Something like ":meth:`__repr__` should always succeed, even if errors prevent a full description of the object."? "... even if the object is only partly initialized."? Andrei, I think you can not simply omit the first argument. For virtually all methods, `self` is an important value for debugging. It could be omitted for __init__, but that would break consistency. Also, __init__ could call other methods that help initialize the object and therefore also deal with partly initialized objects. I really think one important part of the solution to this problem is making people aware, that under some rare conditions, repr might receive a partly initialized object. I'm also still convinced that the other part of the solution is a way to customize the behavior of StackSummary.extract and friends. Context: I have written a tool (https://github.com/moi90/experitur) to run batches of (machine learning) experiments (so it is not interactive). In case of an error, the traceback is dumped into a file, together with the respective local variables (using `traceback.TracebackException.from_exception(...).format()`). I want this to succeed *in any case*, even if I was too ignorant to implement correct `__repr__`s in my experiment code (which is mostly on-time-use, so why should I care). In the end, this whole problem only affects users of `capture_locals=True`, so we could just change the semantics of this parameter a bit: False: do nothing, True: use repr (as before), : use this callable to convert the value to a string. This way, we could avoid adding an additional parameter to many of the methods in traceback AND give users an easy way to customize exception formatting for easy debugging. -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45438] inspect not capturing type annotations created by __class_getitem__
Martin Rueckl added the comment: I just created a PR and signed the contributor agreement. Waiting for it to update :-) Comments welcome! -- ___ Python tracker <https://bugs.python.org/issue45438> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45438] inspect not capturing type annotations created by __class_getitem__
Change by Martin Rueckl : -- keywords: +patch pull_requests: +27476 stage: -> patch review pull_request: https://github.com/python/cpython/pull/29212 ___ Python tracker <https://bugs.python.org/issue45438> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45438] inspect not capturing type annotations created by __class_getitem__
Martin Rueckl added the comment: Can confirm for 3.9.7 as well. -- nosy: +martinitus ___ Python tracker <https://bugs.python.org/issue45438> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: > Can we determine if they came from an initialized object or from object in > the middle of initialization? That would be very nice because inside __init__ is the only place where we have to deal with partly initialized objects. But I think Python does not provide a way to detect this state. Although I would very much like having FrameSummary robust to any kind error, I acknowledge that this might not be possible. It might be frustrating for beginners, but I think the only way to "fix" this, is by having people implement their `repr`s correctly. The documentation currently says[1]: > This is typically used for debugging, so it is important that the > representation is information-rich and unambiguous. It should be added that __repr__ might be used to display partly initialized objects during debugging and therefore should deal with these gracefully. [1] https://docs.python.org/3/reference/datamodel.html#object.__repr__ -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45585] Gzipping subprocess output produces invalid .gz file
Martin Panter added the comment: The subprocess module only uses the file object to get a file handle by calling the "fileno" method. See Issue 19992 about documenting this. For Python to compress the output of the child process, you would need a pipe. Gzip file objects provide the "fileno" method, but it just returns the underlying file descriptor. Data written to that file descriptor would normally already be compressed by Python and goes straight to the OS. There is also Issue 24358 opened about whether "fileno" should be implemented. -- nosy: +martin.panter ___ Python tracker <https://bugs.python.org/issue45585> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] TracebackException or StackSummary.extract with capture_locals=True fail to catch exceptions raised by repr() on value of frame local variable in FrameSummary.__init__.
Martin added the comment: Thanks, Joe, this is a very good summary of all the problems. Another idea to fix this could be an additional parameter "repr=repr" to StackSummary.extract. This way, the default behavior is not changed, but the user is allowed to supply their own repr (safe_repr for example). -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45544] Close 2to3 issues and list them here
Change by Martin DeMello : -- nosy: +martindemello nosy_count: 4.0 -> 5.0 pull_requests: +27384 pull_request: https://github.com/python/cpython/pull/10618 ___ Python tracker <https://bugs.python.org/issue45544> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45523] Python3 ThreadingHTTPServer fails to send chunked encoded response
Martin Panter added the comment: Looks like you forgot to encode the length of ten in hexadecimal. I don't think the HTTP server module has any special handling for chunked responses, so this up to the user and isn't a bug in Python. -- nosy: +martin.panter resolution: -> not a bug stage: -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue45523> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45387] GzipFile.write should be buffered
Change by Marcel Martin : -- nosy: +marcelm ___ Python tracker <https://bugs.python.org/issue45387> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45349] configparser.ConfigParser: 2 newlines at end of file (EOF)
Martin Panter added the comment: Looks like the same as Issue 32917. I presume there are two newlines at the end of the file because there are two newlines following every config section. IMO this is a minor cosmetic annoyance, just like writing a key with an empty value gets you a trailing space after the equals sign at the end of the line. Not worth changing as a bug fix, and not worth a special option, but maybe okay to change if the code is simple and it doesn't harm compatibility. -- nosy: +martin.panter resolution: -> duplicate stage: -> resolved status: open -> closed superseder: -> ConfigParser writes a superfluous final blank line ___ Python tracker <https://bugs.python.org/issue45349> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45170] tarfile missing cross-directory checking
Martin Panter added the comment: Issue 21109 has been open for a while and is the same as this, if I am not mistaken. -- nosy: +martin.panter resolution: -> duplicate stage: -> resolved status: open -> closed superseder: -> tarfile: Traversal attack vulnerability ___ Python tracker <https://bugs.python.org/issue45170> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45147] Typo in "What's New In Python 3.10" documentation
Ross Martin added the comment: I would like to fix this, if it hasnt been assigned already. -- nosy: +MrRBM97 ___ Python tracker <https://bugs.python.org/issue45147> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44663] Possible bug in datetime utc
Paul Martin added the comment: The difference between the two is the difference between your local time and utc. datetime.now(timezone.utc) This returns the current time in utc and is timezone aware. So the timestamp can figure out the seconds since epoch taking into account the timezone. datetime.utcnow() Returns the current utc time but is not timezone aware. When timestamp method is run, it is interpreted as a local timestamp. This is explained in the docs but perhaps it should be made clearer that datetime.utcnow().timestamp() is incorrect and would cause bugs. I'm not sure about changing the behaviour of utcnow to return a timezone-aware dt as is it could cause hard to detect bugs in existing code. But I did have issues recently where I was using utcnow until I went back and read the docs and changed to datetime.now(timezone.utc). So it's probably a common trap to fall into. >From the docs: " Naive datetime instances are assumed to represent local time # Note There is no method to obtain the POSIX timestamp directly from a naive datetime instance representing UTC time. If your application uses this convention and your system timezone is not set to UTC, you can obtain the POSIX timestamp by supplying tzinfo=timezone.utc: timestamp = dt.replace(tzinfo=timezone.utc).timestamp() or by calculating the timestamp directly: timestamp = (dt - datetime(1970, 1, 1)) / timedelta(seconds=1)" Warning Because naive datetime objects are treated by many datetime methods as local times, it is preferred to use aware datetimes to represent times in UTC. As such, the recommended way to create an object representing the current time in UTC is by calling datetime.now(timezone.utc). " -- nosy: +primal ___ Python tracker <https://bugs.python.org/issue44663> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43124] [security] smtplib multiple CRLF injection
Martin Ortner added the comment: > This bug report starts with "a malicious user with direct access to > `smtplib.SMTP(..., local_hostname, ..)", which is a senseless supposition. > Anyone with "access to" the SMTP object could just as well be talking > directly to the SMTP server and do anything they want that SMTP itself allows. Let's not argue about the phrasing and settle on the fact that I am not a native English speaker which might be the root cause of the confusion. The core of the issue is that this *unexpected side-effect* may be security-relevant. Fixing it probably takes less time than arguing about phrasing, severity, or spending time describing exploitation scenarios for a general-purpose library that should protect the underlying protocol from injections. Be kind, I come in peace. -- ___ Python tracker <https://bugs.python.org/issue43124> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41539] print blocks with multiprocessing and buffered output
Martin added the comment: Thanks for the pointer, @pitrou! -- ___ Python tracker <https://bugs.python.org/issue41539> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41539] print blocks with multiprocessing and buffered output
Martin added the comment: Yes, I think it should at least be documented. But then it practically says: "Do not use print in your library because it might be used in a threading context" This sounds unacceptable to me. It would be great to "just make it work". > I debugged it a bit and I think the race may be between your print statement > and the util._flush_std_streams() Why would print deadlock with a flush? -- ___ Python tracker <https://bugs.python.org/issue41539> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44146] Format string fill not handling brace char
Martin Panter added the comment:
Another workaround:
>>> '{:{brace}>10d}'.format(5, brace='{')
'{5'
--
nosy: +martin.panter
___
Python tracker
<https://bugs.python.org/issue44146>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44228] [urllib] Error with handling of urllib.parse.quote. Terminates halfway
Martin Panter added the comment:
I presume this is because you are running a Unix shell, and it's nothing to do
with Python. Look up how quoting and variable substitution with dollar signs $
works.
$ set -o nounset
$ python3 -c "import urllib.parse; print
(urllib.parse.quote('ab$cd#hij$klm'))"
bash: cd: unbound variable
Works fine in the Python interpreter:
>>> import urllib.parse; print (urllib.parse.quote('ab$cd#hij$klm'))
ab%24cd%26efg%23hij%24klm
--
nosy: +martin.panter
resolution: -> not a bug
stage: -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.org/issue44228>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] StackSummary.format fails if repr(value) fails
Martin added the comment: Thanks for the explanations. I think this issue can be closed then. -- title: StackSummary.format fails if str(value) fails -> StackSummary.format fails if repr(value) fails ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44115] Improve conversions for fractions
Change by Martin Teichmann : -- keywords: +patch pull_requests: +24705 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26064 ___ Python tracker <https://bugs.python.org/issue44115> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44115] Improve conversions for fractions
New submission from Martin Teichmann : Currently, fraction.Fractions can be generated from floats via their as_integer_ratio method. This had been extended to also work with the Decimals class. It would be more generic - and IMHO more Pythonic - to just allow any data type, as long as it has an as_integer_ratio method. As an example, this allows numpy floats to be converted into fractions. -- components: Library (Lib) messages: 393507 nosy: Martin.Teichmann priority: normal severity: normal status: open title: Improve conversions for fractions type: enhancement versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue44115> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37334] Add a cancel method to asyncio Queues
Martin Teichmann added the comment: This is a years old issue, unfortunately it never got neither merged nor rejected. I just rebased it and hope somebody could finish the review. -- versions: +Python 3.11 -Python 3.8 ___ Python tracker <https://bugs.python.org/issue37334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44007] urlparse("host:123", "http") inconsistent between 3.8 and 3.9
Martin Panter added the comment: I suspect this comes from Issue 27657. Consider how similar URLs like tel:123 or javascript:123 should be parsed. -- nosy: +martin.panter ___ Python tracker <https://bugs.python.org/issue44007> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37788] fix for bpo-36402 (threading._shutdown() race condition) causes reference leak
Martin Panter added the comment: Anselm's pull request PR 15175 looked hopeful to me. I've been using those changes in our builds at work for a while. -- ___ Python tracker <https://bugs.python.org/issue37788> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39228] traceback.FrameSummary does not handle exceptions from `repr()`
Martin added the comment: pdb uses vanilla repr as well: https://github.com/python/cpython/blob/f3ab670fea75ebe177e3412a5ebe39263cd428e3/Lib/pdb.py#L1180 -- ___ Python tracker <https://bugs.python.org/issue39228> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39228] traceback.FrameSummary does not handle exceptions from `repr()`
Martin added the comment: > Even correctly raised code can raise exceptions such as MemoryError, > RecursionError and KeybordInterrupt. For me, this is an argument in favor of the change (although KeybordInterrupt and probably a couple more should not be caught). traceback is used in contexts where an error already occured and it should do its best to help the user find the cause for it. It does not help much if itself then fails because some values are not repr'eable. What does pdb do in this case? -- ___ Python tracker <https://bugs.python.org/issue39228> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39228] traceback.FrameSummary does not handle exceptions from `repr()`
Martin added the comment: > As is true for most special methods, it is a bug for __repr__ methods to > raise. Is this codified anywhere? I only learned about that in this bug report. -- nosy: +moi90 ___ Python tracker <https://bugs.python.org/issue39228> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] StackSummary.format fails if str(value) fails
Martin added the comment: I can't find any mention of this in the documentation: https://docs.python.org/3/reference/datamodel.html#object.__repr__ https://docs.python.org/3/library/functions.html#repr I think this should be mentioned somewhere. -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] StackSummary.format fails if str(value) fails
Martin added the comment: I didn't know repr is supposed to always succeed. Does the documentation mention this? -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] StackSummary.format fails if str(value) fails
Martin added the comment: Yes, it is repr in FrameSummary.__init__. -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] StackSummary.format fails if str(value) fails
Change by Martin : -- keywords: +patch pull_requests: +23812 stage: -> patch review pull_request: https://github.com/python/cpython/pull/25062 ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] StackSummary.format fails if str(value) fails
Martin added the comment: I have to correct myself: The conversion to string already happens during construction, in `FrameSummary.__init__`: https://github.com/python/cpython/blob/4827483f47906fecee6b5d9097df2a69a293a85c/Lib/traceback.py#L273 The issue remains as severe and the fix remains as easy. -- ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43656] StackSummary.format fails if str(value) fails
New submission from Martin : With `capture_locals=True`, `StackSummary.format` prints the local variables for every frame: https://github.com/python/cpython/blob/4827483f47906fecee6b5d9097df2a69a293a85c/Lib/traceback.py#L440 This will fail, however, if string conversion fails. StackSummary.format should be robust towards such possibilities. An easy fix would be a utility function: ``` def try_str(x): try: return str(x) except: return "" ``` -- messages: 389679 nosy: moi90 priority: normal severity: normal status: open title: StackSummary.format fails if str(value) fails type: enhancement versions: Python 3.10, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue43656> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34915] LWPCookieJar.save() creates *.lwp file in 644 mode
Martin Panter added the comment: I don't have a strong opinion, but it does seem a sensible change that matches the high-level nature of the "cookiejar" module, with low risk of users relying on the current file permissions. On the other hand, the "curl" command seems to use the default mode when creating a cookies file (in Netscape a.k.a. Mozilla format): $ curl --cookie-jar cookies https://www.google.com/ [. . .] $ ls -l cookies -rw-r--r-- 1 vadmium vadmium 418 Mar 14 17:12 cookies The MozillaCookieJar class also seems to use the default file mode. I suppose it should be changed as well as the LWP class. -- components: -SSL ___ Python tracker <https://bugs.python.org/issue34915> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43411] wm_manage fails with ttk.Frame
Martin Cooper added the comment: I think you are confusing the perspective of the implementor with that of the typical developer _using_ the toolkit without reading through its source code. In my tkinter applications, I pretty much always use ttk widgets where they are available. This is largely because of the consistent background colour (on a Mac, at least). In the case of Frame specifically, a regular Frame has a white background that doesn't go well with ttk widgets, while a ttk.Frame has a grey background. So naturally, when I started trying to implement a pop-out window, I was using ttk.Frame everywhere. Then, when I started getting the error cited above, telling me that ".!frame" was not a frame, I was more than a little perplexed. Because in my view, I *had* a frame. I don't really care what the widget's unique name is, contrary to your suggestion that I'm confused by it. If I had given my ttk.Frame the name "garply", and the error was '".garply" is not manageable: must be a frame, labelframe or toplevel', I would have looked back at my code, and seen that, yep, garply was indeed a ttk.Frame, so what's the problem? It took me a long time to figure out that only a tkinter Frame, and not a ttk Frame, will work with wm_manage() (disregarding labelframe and toplevel in this context). Because as a regular developer using the library, it simply wasn't at all clear that using a ttk.Frame is fundamentally different under the covers, and isn't actually considered to be a frame for the purposes of wm_manage(). Who'd have thought? Seriously. Regular developers naturally assume that ttk components are a kind of specialised version of their non-ttk counterparts, albeit with different options and style support, rather than a parallel world. You noted that "Our ttk doc does not discuss the ttk widgets that are essentially copies of tk widgets except the mostly documented differences". I think it *would* be worthwhile to say something about the fact that these are parallel components, and not derivatives, because, as this issue clearly illustrates, it can be important to understand this. I even read through the tkinter and ttk source code to try to understand what was going on. The doc string for wm_manage() would have been a good place to say something, but there's nothing. I do understand that that is outside of ttk per se, but still, the two are not so separate that a comment couldn't help explain the constraint, and it really is important to the use of this method to understand that a ttk Frame is not a Frame. It's unfortunate that the readable names used in the path aren't different for tkinter and ttk widgets, especially given how different they are. That would have helped. I realise, though, that changing it now is likely not a viable option. In the end it was a hunch, perhaps born out of desperation and a lack of anything else to try, that led me to discover that a ttk Frame is not a Frame. I wasted a lot of time on this. I filed this issue in the hope that some documentation might be added so that others like myself don't also have to waste their time. -- ___ Python tracker <https://bugs.python.org/issue43411> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43292] xml.ElementTree iterparse filehandle left open
Change by Martin Panter : -- type: security -> resource usage ___ Python tracker <https://bugs.python.org/issue43292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43292] xml.ElementTree iterparse filehandle left open
Martin Panter added the comment: Perhaps this can be handled with Issue 25707, which is open for adding an API to close the file, similar to how "os.scandir" iterator implements a context manager and "close" method. -- nosy: +martin.panter superseder: -> Add the close method for ElementTree.iterparse() object ___ Python tracker <https://bugs.python.org/issue43292> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43375] memory leak in threading ?
Martin Panter added the comment: Sounds the same as Issue 37788, which is still open. -- nosy: +martin.panter ___ Python tracker <https://bugs.python.org/issue43375> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43198] Operations on sets more than hundred times less efficient with python3.9 than with previous versions
Eric Martin added the comment: Thank you for letting us know, I am actually not surprised the issue would bugger you and you would change your mind... Thinking further about it and experimenting further, I thought it might not be such an edge case and there could be a significant cost in some applications. Many thanks Raymond for everything you give to the community! -- ___ Python tracker <https://bugs.python.org/issue43198> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43411] wm_manage fails with ttk.Frame
New submission from Martin Cooper : Attempting to use a ttk.Frame with wm_manage() causes a TclError: _tkinter.TclError: window ".!frame" is not manageable: must be a frame, labelframe or toplevel The (Tcl) documentation for wm manage states "Only frame, labelframe and toplevel widgets can be used with this command." One might reasonably expect a ttk.Frame to appropriately fall under this requirement, especially since the name 'frame' is used for them, but it does not. One must use a tk.Frame instead to make this work. At the very least, this needs to be documented. Looking at the error message and seeing it complain that a 'frame' is not one of 'frame', 'labelframe' or 'toplevel' is extremely confusing. There is nothing to lead to the conclusion that a ttk Frame is not a 'frame'. Better than documenting it, of course, would be to make wm_manage actually work properly with a ttk.Frame, as developers would expect. -- components: Tkinter messages: 388161 nosy: mfncooper priority: normal severity: normal status: open title: wm_manage fails with ttk.Frame type: behavior ___ Python tracker <https://bugs.python.org/issue43411> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43198] Operations on sets more than hundred times less efficient with python3.9 than with previous versions
Eric Martin added the comment: Many thanks Raymond for these insights and thorough explanations, I appreciate it very much. Your conclusions are of course most reasonable. For me, I cannot say that it is a real issue in practice. I have teaching notes in which I illustrate with plots the differences in complexity when working with sets versus lists. One such plot showed that the cost of removing an element from a set does not depend on what the element is (which is why I reused the same element again and again) and is orders of magnitude less than for removing an element from the middle of a list. I just found out that something had changed with Python3.9 when I last run the cell of the jupyter notebook in which that plot was produced... -- ___ Python tracker <https://bugs.python.org/issue43198> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43001] python3.9.1 test_embed test_tabnanny failed
Tony Martin Berbel added the comment: My system crashed completely. I reinstalled Ubuntu. Sorry I couldn't help more ... :( ___ MARTIN BERBEL, Tony GSM: +32 (0) 477 / 33.12.48 -- Le mer. 10 févr. 2021 à 04:06, Tony Martin Berbel a écrit : > > Tony Martin Berbel added the comment: > > I had the same error > I ran the make test command with > > But I don't know where to look for the log file > > -- > nosy: +wingarmac > > ___ > Python tracker > <https://bugs.python.org/issue43001> > ___ > -- ___ Python tracker <https://bugs.python.org/issue43001> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43198] Operations on sets more than hundred times less efficient with python3.9 than with previous versions
New submission from Eric Martin :
Run on a MacBook Pro (15-inch, 2019)
2.3 GHz 8-Core Intel Core i9
16 GB 2400 MHz DDR4
Python 3.8.7 (default, Dec 24 2020, 19:07:18)
[Clang 12.0.0 (clang-1200.0.32.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from timeit import timeit
>>> S = set(range(10_000))
>>> timeit('S.remove(5000); S.add(5000)', globals=globals(), number=100_000)
0.0193329169984
$ python3.9
Python 3.9.1 (default, Dec 8 2020, 13:10:53)
[Clang 12.0.0 (clang-1200.0.32.27)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from timeit import timeit
>>> S = set(range(10_000))
>>> timeit('S.remove(5000); S.add(5000)', globals=globals(), number=100_000)
3.851722548998
--
components: Interpreter Core
messages: 386815
nosy: eamartin
priority: normal
severity: normal
status: open
title: Operations on sets more than hundred times less efficient with python3.9
than with previous versions
type: performance
versions: Python 3.9
___
Python tracker
<https://bugs.python.org/issue43198>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43001] python3.9.1 test_embed test_tabnanny failed
Tony Martin Berbel added the comment: I found lastlog and attached it ! -- Added file: https://bugs.python.org/file49800/lastlog ___ Python tracker <https://bugs.python.org/issue43001> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43001] python3.9.1 test_embed test_tabnanny failed
Tony Martin Berbel added the comment: I had the same error I ran the make test command with > But I don't know where to look for the log file -- nosy: +wingarmac ___ Python tracker <https://bugs.python.org/issue43001> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43126] IOBase.readlines(0) behaviour is inconsistent with documentation
New submission from Martin Larralde : The documentation for `IOBase.readlines` states that the `hint` optional argument should be used like so (https://docs.python.org/3/library/io.html#io.IOBase.readlines): > Read and return a list of lines from the stream. hint can be specified to > control the number of lines read: no more lines will be read if the total > size (in bytes/characters) of all lines so far exceeds hint. So in the case where `hint` is null, I would expect `readlines` to return an empty list. However, this is neither the case for `io.BytesIO` nor for `io.FileIO`: ``` import io io.BytesIO(b"abc\ndef\nghi\n").readlines(0) # this evaluates to [b"abc\n", b"def\n", b"ghi\n"] ``` If this is the intended behaviour, the documentation should be updated, otherwise the code from the `io` module should be fixed. -- assignee: docs@python components: Documentation messages: 386484 nosy: althonos2, docs@python priority: normal severity: normal status: open title: IOBase.readlines(0) behaviour is inconsistent with documentation type: behavior versions: Python 3.9 ___ Python tracker <https://bugs.python.org/issue43126> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43124] smtplib multiple CRLF injection
New submission from Martin Ortner :
// reported via PSRT email (see timeline; last contact: Alex/PSRT)
// external reference:
http://consensys.net/diligence/vulnerabilities/private/z5kxjgfmja4offxbrw1miuxwezggajjfswlz9g2hfuh77we5dy727hqy5x9ii43e/
cve:
vendor: python
vendorUrl: https://www.python.org/
authors: tintinweb
affectedVersions: [at least 3.8.3, <=3.7.8, <=3.6.11, <=3.5.9, <=2.7.18]
vulnClass: CWE-93
# Vulnerability Note
## Summary
>Python is a programming language that lets you work more quickly and integrate
>your systems more effectively.
Two CR-LF injection points have been discovered in the Python standard library
for `SMTP` interaction (client perspective) named `smtplib` that may allow a
malicious user with direct access to `smtplib.SMTP(..., local_hostname, ..)` or
`smtplib.SMTP(...).mail(..., options)` to inject a CR-LF control sequence to
inject arbitrary `SMTP` commands into the protocol stream.
The root cause of this is likely to be found in the design of the `putcmd(cmd,
args)` method, that fails to validate that `cmd` nor `args` contains any
protocol control sequences (i.e. `CR-LF`).
It is recommended to reject or encode `\r\n` in `putcmd()` and enforce that
potential multi-line commands call `putcmd()` multiple times to avoid that
malicious input breaks the expected context of the method and hence cause
unexpected behavior. For reference, the `DATA` command (multi-line) would not
be affected by this change as it calls `putcmd()` only once and continues with
directly interacting with the socket to submit the body.
## Details
### Description
The root cause of this (and probably also some earlier reported CR-LF
injections) is the method `putcmd()` in `lib/smtplib.py`[3]. The method is
called by multiple commands and does not validate that neither `cmd` nor `args`
contains any `CRLF` sequences.
```python
def putcmd(self, cmd, args=""):
"""Send a command to the server."""
if args == "":
str = '%s%s' % (cmd, CRLF)
else:
str = '%s %s%s' % (cmd, args, CRLF)
self.send(str)
```
However, the issue was initially found in `mail(..., options)` [4] which fails
to ensure that none of the provided `options` contains `CRLF` characters. The
method only ensures that provides mail addresses are quoted, `optionslist` is
untouched:
```python
self.putcmd("mail", "FROM:%s%s" % (quoteaddr(sender), optionlist))
```
A similar issue was found with `smtplib.SMTP(...,local_hostname)` (and
`helo(name)`, `ehlo(name)`) which may potentially contain `CRLF` sequences and,
therefore, can be used to inject `SMTP` commands.
Here's a snipped of `helo` [5]
```python
def helo(self, name=''):
"""SMTP 'helo' command.
Hostname to send for this command defaults to the FQDN of the local
host.
"""
self.putcmd("helo", name or self.local_hostname)
(code, msg) = self.getreply()
self.helo_resp = msg
return (code, msg)
```
We highly recommend, fixing this issue once and for all directly in `putcmd()`
and enforce that the interface can only send one command at a time, rejecting
arguments that contain `CRLF` sequences or properly encoding them to avoid
injection.
## Proof of Concept
1. set-up a local tcp listener `⇒ nc -l 10001`
2. run the following PoC and replay the server part as outline in 3.
```python
import smtplib
server = smtplib.SMTP('localhost', 10001, "hi\nX-INJECTED") # localhostname
CRLF injection
server.set_debuglevel(1)
server.sendmail("h...@me.com", "y...@me.com", "wazzuuup\nlinetwo")
server.mail("h...@me.com",["X-OPTION\nX-INJECTED-1","X-OPTION2\nX-INJECTED-2"])
# options CRLF injection
```
3. interact with `smtplib`, check for `X-INJECTED`
```
⇒ nc -l 10001
nc -l 10001
220 yo
ehlo hi
X-INJECTED
250-AUTH PLAIN
250
mail FROM:
250 ok
rcpt TO:
250 ok
data
354 End data with .
wazzuuup
linetwo
.
250 ok
mail FROM: X-OPTION
X-INJECTED-1 X-OPTION2
X-INJECTED-2
250 ok
quit
250 ok
```
### Proposed Fix
* enforce that `putcmd` emits exactly one command at a time and encode `\n ->
\\n`.
```diff
diff --git a/Lib/smtplib.py b/Lib/smtplib.py
index e2dbbbc..9c16e7d 100755
--- a/Lib/smtplib.py
+++ b/Lib/smtplib.py
@@ -365,10 +365,10 @@ class SMTP:
def putcmd(self, cmd, args=""):
"""Send a command to the server."""
if args == "":
-str = '%s%s' % (cmd, CRLF)
+str = cmd
else:
-str = '%s %s%s' % (cmd, args, CRLF)
-self.send(str)
+str = '%s %s' % (cmd, args)
+self.send('%s%s' % (str.replace('\n','\\n'), CRLF))
```
## Vendor Response
Vendor response: gone silent
### Timeline
```
JUL/02/2020 - contact psrt; provided details, PoC,
[issue43123] email MIME splitting
New submission from Martin Ortner :
// reported via PSRT email (see timeline)
// external reference:
https://consensys.net/diligence/vulnerabilities/private/jcchhpke7usq8wo45vloy282phwpd9fj41imumhb8varxahz2bf9afw5mcno84gx/
cve:
vendor: python
vendorUrl: https://www.python.org/
authors: tintinweb
affectedVersions: [at least <= 3.8.3, <=3.7.7, <=2.7.18]
vulnClass: CWE-93
# Vulnerability Note
## Summary
>Python is a programming language that lets you work more quickly and integrate
>your systems more effectively.
The python `email.mime` package fails to properly encode or reject `CR-LF`
control sequences in MIME header values allowing for MIME splitting and header
injection attacks.
* `MIMEText[headerKey] = headerValue` - `headerValue` accepts `CR-LF` in the
value, allowing an attacker in control of part of the header value to perform a
MIME splitting attack.
* `MIMEText[headerKey] = headerValue` - `headerKey` is not checked for `CR-LF`
allowing an attacker in control of part of a header key to inject arbitrary
MIME headers.
* `MIMEText.add_header(headerKey, headerValue)` - `headerKey` is not checked
for `CR-LF` allowing an attacker in control of part of a header key to inject
arbitrary MIME headers.
## Details
### MIME-Splitting with `CR-LF` in header value:
* Note: `CR-LF` injection in `To` header pushes an invalid header and may force
a MIME split (depending on the parsing library) pushing following header values
into the body when being parsed with the `email.message_from_string()` method.
```python
# Import the email modules we'll need
from email.mime.text import MIMEText
# Open a plain text file for reading. For this example, assume that
# the text file contains only ASCII characters.
msg = MIMEText("REAL_MSG_BODY_BEGIN\n...\nREAL_MSG_BODY_END")
msg['Subject'] = 'The contents of is this...'
msg['To'] = "TO toaddr...@oststrom.com\r\nX-SPLIT-MSG-TO-BODY\r\n"
msg['From'] = "FROM fromaddr...@oststrom.com"
msg['MyHEader'] = "hi :: hi"
print(msg)
print(repr(msg))
print("=")
import email
msg = email.message_from_string(str(msg))
print(msg)
print("-> FROM: %s" % msg.get("From"))
print("-> TO: %s" % msg["To"])
print("-> MSG: " + repr(msg.get_payload()))
```
Output:
* Output before the `===` is the constructed message
* Output after the `===` is the parsed message
* Note: that after parsing the message some headers end up in the body (`from`,
`myheader`). Note that `msg[from]` is empty.
```
⇒ python3 a.py
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Subject: The contents of is this...
To: TO toaddr...@oststrom.com
X-SPLIT-MSG-TO-BODY
From: FROM fromaddr...@oststrom.com
MyHEader: hi :: hi
REAL_MSG_BODY_BEGIN
...
REAL_MSG_BODY_END
=
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Subject: The contents of is this...
To: TO toaddr...@oststrom.com
X-SPLIT-MSG-TO-BODY
From: FROM fromaddr...@oststrom.com
MyHEader: hi :: hi
REAL_MSG_BODY_BEGIN
...
REAL_MSG_BODY_END
-> FROM: None
-> TO: TO toaddr...@oststrom.com
-> MSG: 'X-SPLIT-MSG-TO-BODY\nFrom: FROM fromaddr...@oststrom.com\nMyHEader:
hi :: hi\n\nREAL_MSG_BODY_BEGIN\n...\nREAL_MSG_BODY_END'
```
### `CR-LF` injection in header keys.
Note: this is unlikely to be exploited, however, there might be scenarios where
part of the header key is exposed to user input. A `CR-LF` character in the
header key should throw instead.
```python
# Import the email modules we'll need
from email.mime.text import MIMEText
# Open a plain text file for reading. For this example, assume that
# the text file contains only ASCII characters.
msg = MIMEText("REAL_MSG_BODY_BEGIN\n...\nREAL_MSG_BODY_END")
# me == the sender's email address
# you == the recipient's email address
msg['Subject'] = 'The contents of is this...'
msg['To'] = "TO toaddr...@oststrom.com"
msg['From'] = "FROM fromaddr...@oststrom.com"
msg['MyHEader'] = "hi :: hi"
msg["m\r\nh"] = "yo"
print(msg)
print(repr(msg))
print("=")
import email
msg = email.message_from_string(str(msg))
msg.add_header("CUSTOM-HEADER: yo\r\n\nX-INJECTED:
injected-header\r\naa","data")
print(msg)
print("-> FROM: %s" % msg.get("From"))
print("-> TO: %s" % msg["To"])
print("-> MSG: " + repr(msg.get_payload()))
```
Output: `h: yo` and `X-INJECTED:` are injected
```
⇒ python3 a.py
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Subject: The contents of is this...
To: TO toaddr...@oststrom.com
From: FROM fromaddr...@oststrom.com
MyHEade
[issue37788] fix for bpo-36402 (threading._shutdown() race condition) causes reference leak
Change by Martin Panter : -- keywords: +3.7regression ___ Python tracker <https://bugs.python.org/issue37788> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43050] threading timer memory leak
Martin Panter added the comment: Perhaps this is caused by Issue 37788. Python 3.7.4 introduced a leak for any thread that doesn't get its "join" method called. Timer is a subclass of Thread, so to confirm, see if calling "timer.join()" after "cancel" will make the leak go away. -- nosy: +martin.panter resolution: -> duplicate superseder: -> fix for bpo-36402 (threading._shutdown() race condition) causes reference leak type: performance -> resource usage ___ Python tracker <https://bugs.python.org/issue43050> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42874] running configure on Solaris 10 gives grep "illegal option" errors
Martin Wheatley added the comment: I'll do the test it happens with a plain ./configure Many thanks Martin On Tue, 12 Jan 2021 at 15:11, Paul Ganssle wrote: > > Paul Ganssle added the comment: > > This particular grep statement is used to validate the `tzpath` variable. > Apparently it is easy enough to achieve what I was going for using vanilla > grep with no options, so I've created GH-24200 to fix the issue. > > I notice that there are other uses of `-q` and `-E` in the configure file, > but presumably those are on more optional paths. > > @martin.wheatley.home: Can you check to see if GH-24200 fixes your issue? > > Also, can you clarify whether this happens with a plain `./configure` > invocation, or are you specifying `./configure > --with-tzpath='/usr/share/zoneinfo:/usr/lib/zoneinfo:/usr/share/lib/zoneinfo:/etc/zoneinfo'` > yourself? > > -- > versions: +Python 3.10 > > ___ > Python tracker > <https://bugs.python.org/issue42874> > ___ > -- ___ Python tracker <https://bugs.python.org/issue42874> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42874] running configure on Solaris 10 gives grep "illegal option" errors
Martin Wheatley added the comment: I'll do the test it happens with a plain ./configure Many thanks Martin On Tue, 12 Jan 2021 at 15:11, Paul Ganssle wrote: > > Paul Ganssle added the comment: > > This particular grep statement is used to validate the `tzpath` variable. > Apparently it is easy enough to achieve what I was going for using vanilla > grep with no options, so I've created GH-24200 to fix the issue. > > I notice that there are other uses of `-q` and `-E` in the configure file, > but presumably those are on more optional paths. > > @martin.wheatley.home: Can you check to see if GH-24200 fixes your issue? > > Also, can you clarify whether this happens with a plain `./configure` > invocation, or are you specifying `./configure > --with-tzpath='/usr/share/zoneinfo:/usr/lib/zoneinfo:/usr/share/lib/zoneinfo:/etc/zoneinfo'` > yourself? > > -- > versions: +Python 3.10 > > ___ > Python tracker > <https://bugs.python.org/issue42874> > ___ > -- ___ Python tracker <https://bugs.python.org/issue42874> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
