Change by Inada Naoki :
--
keywords: +patch
pull_requests: +28615
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/30409
___
Python tracker
<https://bugs.python.org/issu
nsion module will work
well in Python 3.11.
> I would implement it also for a C extension that uses CPython < 3.10.
> How can I achieve this?
See PyModule_GetState() to have per-interpreter module state instead
of static variables.
https://docs.python.org/3/c-api/module.html#c.PyModule_GetSt
ovements.
If Python changed its GC to mark-and-sweep, PyObject_GC_IsTracked()
can return true always.
Regards,
--
Inada Naoki
--
https://mail.python.org/mailman/listinfo/python-list
U pip
>
> Can't venv have an option for doing this automatically or, better, a
> config file where you can put commands that will be launched every
> time after you create a venv?
> --
> https://mail.python.org/mailman/listinfo/python-list
--
Inada Naoki
--
https://mail.p
has special optimization which tightly coupled with
current CPython implementation.
So you need to use private APIs for MAINTAIN_TRACKING.
But PyObject_GC_Track() is a public API.
Regards,
--
Inada Naoki
--
https://mail.python.org/mailman/listinfo/python-list
or optimization based on *current* Python internals.
That's why it is not a public API. If we expose it as public API, it
makes harder to change Python's GC internals.
--
Inada Naoki
--
https://mail.python.org/mailman/listinfo/python-list
Inada Naoki added the comment:
UTF-8 mode is not enabled by default. So locale encoding is still the default
encoding.
--
nosy: +methane
___
Python tracker
<https://bugs.python.org/issue46
Inada Naoki added the comment:
Nice catch.
> if ((kind & _odict_ITER_KEYS) && (kind &_odict_ITER_VALUES))
You can reduce one branch by
```
#define _odict_ITER_ITEMS (_odict_ITER_KEYS|_odict_ITER_VALUES)
...
if (kind & _odict_ITER_ITEMS == _odict_ITER_ITEM
Inada Naoki added the comment:
That's too bad.
We can not compare two Unicode by pointer even if both are interned anymore...
It was a nice optimization.
--
___
Python tracker
<https://bugs.python.org/issue46
ibilities.
* You set the wrong PYTHONHOME
PYTHONHOME is very rarely useful. It shouldn't be used if you can
not solve this kind of problem.
* Your Python installation is broken.
Some files are deleted or overwritten. You need to *clean* install
Python again.
Bets,
--
Inada Naoki
--
https://mail.python.org/mailman/listinfo/python-list
Inada Naoki added the comment:
Should `_PyUnicode_EqualToASCIIId()` support comparing two unicode from
different interpreter??
--
nosy: +methane
___
Python tracker
<https://bugs.python.org/issue46
Change by Inada Naoki :
--
versions: +Python 3.11 -Python 3.10, Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.org/issue23882>
___
___
Python-bug
Change by Inada Naoki :
--
pull_requests: +27982
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/29745
___
Python tracker
<https://bugs.python.org/issu
Inada Naoki added the comment:
The another error I found is already reported as #42868.
--
___
Python tracker
<https://bugs.python.org/issue38625>
___
___
Pytho
Change by Inada Naoki :
--
resolution: -> fixed
stage: -> resolved
status: open -> closed
versions: +Python 3.8, Python 3.9
___
Python tracker
<https://bugs.python.or
Inada Naoki added the comment:
I confirmed that this bug is fixed, but I found another error.
--
___
Python tracker
<https://bugs.python.org/issue38625>
___
___
Inada Naoki added the comment:
Is this bug fixed by #26730?
--
___
Python tracker
<https://bugs.python.org/issue38625>
___
___
Python-bugs-list mailin
Inada Naoki added the comment:
When I am trying to understand this issue, I see this segfault.
https://gist.github.com/methane/1b83e2abc6739017e0490c5f70a27b52
I am not sure this segfault is caused by this issue or not. If this is
unrelated, I will create another issue
Change by Inada Naoki :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Inada Naoki added the comment:
New changeset 0a4c82ddd34a3578684b45b76f49cd289a08740b by Inada Naoki in branch
'main':
bpo-45475: Revert `__iter__` optimization for GzipFile, BZ2File, and LZMAFile.
(GH-29016)
https://github.com/python/cpython/commit/0a4c82ddd34a3578684b45b76f49cd289a08740b
Change by Inada Naoki :
--
keywords: +patch
pull_requests: +27290
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/29016
___
Python tracker
<https://bugs.python.org/issu
Inada Naoki added the comment:
>>> ll = [l for l in gzip.GzipFile(filename='data/UTF-8-test_for_gzip.txt.gz')]
This is bad code pattern because you don't close the file explicitly.
Actually, the error caused by the optimization thet iter(GzipFile) returns
underlaying faster
Inada Naoki added the comment:
See this comment.
https://github.com/python/cpython/pull/5241#discussion_r162765765
--
___
Python tracker
<https://bugs.python.org/issue32
Inada Naoki added the comment:
Pros:
Faster (about 3~5%) faster function creation, when function don't have
annotations.
When function has annotation, function creation is much slower so performance
gain become tiny.
Cons:
Somewhat backward incompatible:
```
>>> def fo
Change by Inada Naoki :
--
keywords: +patch
pull_requests: +27206
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/28704
___
Python tracker
<https://bugs.python.org/issu
New submission from Inada Naoki :
Move setting `func.__doc__` from PyFunction_New() to __doc__ descriptor, for
faster function creation.
This issue is spin-off of bpo-36521.
--
components: Interpreter Core
messages: 403786
nosy: methane
priority: normal
severity: normal
status: open
Inada Naoki added the comment:
Although I still feel reducing 16% tuples is attractive, no one support the
idea.
I leave this as-is for now, and will go to lazy-loading docstring (maybe,
co_linetable too) later.
--
resolution: -> rejected
stage: patch review -> resolved
Change by Inada Naoki :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.11
___
Python tracker
<https://bugs.python.or
Change by Inada Naoki :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Inada Naoki added the comment:
New changeset ad970e8623523a8656e8c1ff4e1dff3423498a5a by Inada Naoki in branch
'main':
bpo-29410: Change the default hash algorithm to SipHash13. (GH-28752)
https://github.com/python/cpython/commit/ad970e8623523a8656e8c1ff4e1dff3423498a5a
Inada Naoki added the comment:
> I recommend that you add SipHash-1-3 as an additional algorithm and make it
> the default. The removal of --with-hash-algorithm=siphash24 should go through
> regular deprecation cycle of two Python versions.
I am not sure its worth enough. Adding
Inada Naoki added the comment:
> I know that it's not a popular opinion, but I don't think that this denial of
> service (DoS) is important. IMO there are enough other ways to crash a
> server. Moreover, the initial attack vector was a HTTP request with tons of
>
Inada Naoki added the comment:
I am not sure this is worth doing.
Microbenchmarks:
## import time
```
$ main/opt/bin/pyperf command main/opt/bin/python3 -c 'import typing,asyncio'
.
command: Mean +- std dev: 49.6 ms +- 0.1 ms
$ siphash13/opt/bin/pyperf command siphash13
Change by Inada Naoki :
--
pull_requests: +27097
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/28752
___
Python tracker
<https://bugs.python.org/issu
Inada Naoki added the comment:
I can not confirm performance regression between 33ec88ac and 23ae2c3b.
```
Performance version: 1.0.2
Python version: 3.10.0a7+ (64-bit) revision 33ec88ac81
Report on Linux-5.4.0-81-generic-x86_64-with-glibc2.31
Number of logical CPUs: 8
Start date: 2021-10-06
Inada Naoki added the comment:
Lazy filling func.__doc__ has only 3~5% performance gain. And it has small
backward incompatibility.
```
>>> def foo(): "foo"
...
>>> def bar(): "bar"
...
>>> bar.__code__ = foo.__code__
>>> bar.__doc__
'
Change by Inada Naoki :
--
pull_requests: +27065
pull_request: https://github.com/python/cpython/pull/28704
___
Python tracker
<https://bugs.python.org/issue36
Inada Naoki added the comment:
> Do you have any explanation of this?
I think its because current PyFunction_New tries to get docstring always.
See this pull request (lazy-func-doc).
https://github.com/python/cpython/pull/28704
lazy-func-doc is faster than co-docstring and remove-docstr
Inada Naoki added the comment:
> So overhead is around 2%. And this 2% is problem only for "creating function
> with annotation, without docstring, never called, in loop" situation.
My bad, "creating function with docstring, without annotation, nevercalle
Inada Naoki added the comment:
And as a bonus, creating function without docstring is little faster.
```
$ cpython/release/bin/pyperf timeit --duplicate=100 "def f(): pass"
.
Mean +- std dev: 62.5 ns +- 1.2 ns
$ load-none-remove-docstring/release/bin/pyp
Inada Naoki added the comment:
> The difference 5.1 ns is the cost of additional LOAD_CONST. It is around 8%
> (but can be 12% or 2%). The cost of setting docstring externally will be the
> same.
I don't have bare metal machine for now so I don't know why annotation is so
slow.
Inada Naoki added the comment:
> For the sqlalchemy example: the saving in co_consts is about 1.6k (200
> pointers), but an increase in bytecode size of 2.4k.
Please see number of co_constatns tuples. (d) saved 1307 tuples compared to (b).
`sys.getsizeof(())` is 40 on 64bit machine. S
Inada Naoki added the comment:
My machine at the office (used for benchmarking) is hanged up and I need to go
to the office to reboot. So I don't have benchmark machine for now.
Please prioritize LOAD_NONE/LOAD_COMMON_CONST than this. It is hard to maintain
merged branches. Merging
Inada Naoki added the comment:
I used this tool to count co_const size and numbers.
https://github.com/faster-cpython/tools/pull/6
Target is asyncio in the main branch.
main (b34dd58f):
Total: 31 files; 1,068 code objects; 12,741 lines; 39,208 opcodes; 3,880 total
size of co_consts; 738
Inada Naoki added the comment:
> I'm not in favor of (c) co_doc either any more (for the reasons you state). I
> would go for (b), a CO_DOCSTRING flag plus co_consts[0]. I expect that
> co_consts sharing to be a very minor benefit, but you could easily count this
> with another
Inada Naoki added the comment:
> There is a clear disadvantage in moving the docstring from the function's
> code object to the enclosing code object:
>
> Docstrings are rarely looked at (relative to other operations on functions).
> Inner functions and comprehensions are cre
Inada Naoki added the comment:
Mark, would you take a look, please?
--
___
Python tracker
<https://bugs.python.org/issue36521>
___
___
Python-bugs-list mailin
Inada Naoki added the comment:
FYI, This issue is duplicate of https://bugs.python.org/issue34093, and I had
made two pull requests to solve the issue.
--
nosy: +methane
___
Python tracker
<https://bugs.python.org/issue45
Inada Naoki added the comment:
I think pydantic approach is the best practice.
See https://pydantic-docs.helpmanual.io/usage/postponed_annotations/
--
nosy: +methane
___
Python tracker
<https://bugs.python.org/issue39
Inada Naoki added the comment:
It is fixed in Python 3.9.
* https://github.com/python/cpython/pull/14008
* https://bugs.python.org/issue37111
Python 3.8 is "security" status for now. Non-security issues won't be fixed.
Please migrate to Python 3.9.
--
nosy: +methane
Inada Naoki added the comment:
> I am still not convinced that it's a good idea to put the docstring in the
> surrounding code object. I'd like to be able to see it when I introspect a
> code object, not just when introspecting a function object (I may be
> analyzing code on
Change by Inada Naoki :
--
keywords: +patch
pull_requests: +26577
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/28138
___
Python tracker
<https://bugs.python.org/issu
Inada Naoki added the comment:
I'm +0 to this.
Since this alias is not a big maintenance burden, I think we can use
deprecation period longer than minimum 2 releases.
In sdist files of the top5000 pypi packages, I found 18 packages using it
including false positive (e.g. mypi just have
Change by Inada Naoki :
Added file: https://bugs.python.org/file50255/unused_trimmed.txt
___
Python tracker
<https://bugs.python.org/issue45056>
___
___
Python-bug
Change by Inada Naoki :
Added file: https://bugs.python.org/file50254/unused_310rc1.txt
___
Python tracker
<https://bugs.python.org/issue45056>
___
___
Python-bugs-list m
Change by Inada Naoki :
Added file: https://bugs.python.org/file50253/unused_39.txt
___
Python tracker
<https://bugs.python.org/issue45056>
___
___
Python-bugs-list m
Change by Inada Naoki :
Added file: https://bugs.python.org/file50252/dump_unused_consts.py
___
Python tracker
<https://bugs.python.org/issue45056>
___
___
Python-bug
Inada Naoki added the comment:
@Pablo This issue seems an optimization rather than bugfix.
But since Python 3.10 produces more unused constants than 3.9, this is a
regression.
GH-28109 removes not all unused constants, but only trailing unused constants.
I think it is enough to fix
Change by Inada Naoki :
--
keywords: +patch
pull_requests: +26551
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/28109
___
Python tracker
<https://bugs.python.org/issu
Inada Naoki added the comment:
I found anther example for unused constants.
The `readall()` function.
https://github.com/python/cpython/blob/863154c9292e70c5a8a1a3f22ef4ee42e2304281/Lib/_pyio.py#L663-L675
Python 3.9.5:
co_consts = ('Read until EOF, using multiple read() call.', None)
Python
Inada Naoki added the comment:
I don't want all frozen header files to be committed in git repository.
Can't we just build them during build process?
--
___
Python tracker
<https://bugs.python.org/issue45
Inada Naoki added the comment:
I think LOAD_CONST None + RETURN_VALUE is added here. And it removed by
optimize_cfg().
https://github.com/python/cpython/blob/793f55bde9b0299100c12ddb0e6949c6eb4d85e5/Python/compile.c#L7795-L7797
I don't know how easy to remove this unnecessary None
Inada Naoki added the comment:
This is WIP pull request. https://github.com/methane/cpython/pull/35
Some tests are failing because of bpo-36521.
--
___
Python tracker
<https://bugs.python.org/issue36
New submission from Inada Naoki :
Python 3.10 compiler adds None to co_consts even when None is not used at all.
```
$ cat x1.py
def foo():
"docstring"
return 42
import dis
dis.dis(foo)
print(foo.__code__.co_consts)
$ python3.9 x1.py
3 0 LOAD_CONST
Inada Naoki added the comment:
I grepped top 5000 downloaded packages and I can not find any real use of
PyFunction_New(WithQualName).
So I don't know what is current workflow of PyFunction_New.
My current wip implementation adds new API (e.g. PyFunction_NewWithDoc()).
Old API keep using
Inada Naoki added the comment:
> I think that would require a change in the signature of PyFunction_New.
I don't think so. For example, func_annotation don't require changing
PyFunction_New().
```
case TARGET(MAKE_FUNCTION): {
PyObject *codeobj =
Inada Naoki added the comment:
> If docstring is not in co_consts, all co_consts are empty tuple. The empty
> tuple is nearly zero-cost because its a singleton.
My wrong. Two setters will have `(None,)` tuple. But such tuple can be merged
at compile time for now. And "commo
Inada Naoki added the comment:
> Why all the hating on docstrings? What have docstrings done wrong?
Oh, I don't hate docstrings. I just want to move it from code object to
function object.
Remove docstring during unmarshal is your idea, not mine.
My main motivation is reducing code s
Change by Inada Naoki :
--
nosy: +methane
___
Python tracker
<https://bugs.python.org/issue45020>
___
___
Python-bugs-list mailing list
Unsubscribe:
Inada Naoki added the comment:
> This would actually make it harder to strip docstrings e.g. during
> unmarshalling, since you don't know which constants refer to docstrings.
We can not strip class docstring anyway.
One idea to strip docstring during startup: Add new opcode only for s
Inada Naoki added the comment:
> You'd just be moving the problem though -- the docstring would have be
> included in the co_consts array of the surrounding code object instead of the
> function object.
As far as I know, surrounding code objects (e.g. global, class body) will be
Inada Naoki added the comment:
I'd like to remove docstring from code object at all.
func.__doc__ can be set by MAKE_FUNCTION or STORE_ATTR.
Pros are:
* Code objects can be bit smaller than adding co_doc.
* Many code objects don't have docstrings. (e.g. lambdas, somprehensions, and
PEP
Change by Inada Naoki :
--
resolution: fixed ->
stage: resolved ->
status: closed -> open
versions: +Python 3.11
___
Python tracker
<https://bugs.python.or
Inada Naoki added the comment:
Should I resurrect only function implementation and symbol?
Or may I resurrect definitions in header files too?
--
___
Python tracker
<https://bugs.python.org/issue41
Change by Inada Naoki :
--
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.org/issue25702>
___
___
Pyth
Change by Inada Naoki :
--
nosy: +methane
___
Python tracker
<https://bugs.python.org/issue44609>
___
___
Python-bugs-list mailing list
Unsubscribe:
Change by Inada Naoki :
--
pull_requests: +25967
pull_request: https://github.com/python/cpython/pull/27437
___
Python tracker
<https://bugs.python.org/issue41
Inada Naoki added the comment:
Oh, I didn't notice that APIs deprecated in Python 3.0 are included in stable
ABIs defined at Python 3.2!
--
___
Python tracker
<https://bugs.python.org/issue41
Inada Naoki added the comment:
Is this issue duplicate of https://bugs.python.org/issue32615 and
https://bugs.python.org/issue1402289 ?
--
nosy: +methane
___
Python tracker
<https://bugs.python.org/issue44
Inada Naoki added the comment:
Note that it was reverted only in 3.9 branch.
--
___
Python tracker
<https://bugs.python.org/issue36050>
___
___
Python-bug
Change by Inada Naoki :
--
keywords: +patch
pull_requests: +25938
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/27405
___
Python tracker
<https://bugs.python.org/issu
Change by Inada Naoki :
--
nosy: +methane
___
Python tracker
<https://bugs.python.org/issue38278>
___
___
Python-bugs-list mailing list
Unsubscribe:
Inada Naoki added the comment:
There is a macro benchmark we regularly used.
https://pyperformance.readthedocs.io/
You can try micro benchmark for most common use cases with pyperf like this:
```
$ pyperf timeit -s "def f(**kwargs): pass" -- "f(a=1,b=2,c=3)"
Mean +- std
Inada Naoki added the comment:
And https://dev.to/methane/don-t-omit-init-py-3hga too.
--
nosy: +methane
___
Python tracker
<https://bugs.python.org/issue44
Change by Inada Naoki :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.11
___
Python tracker
<https://bugs.python.or
Inada Naoki added the comment:
New changeset 58248d94379b202ccce3e45b1d1830ca47683273 by Srinivas Reddy
Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి) in branch 'main':
bpo-41137: Use utf-8 encoding while reading .pdbrc files (GH-21263)
https://github.com/python/cpython/commit
Inada Naoki added the comment:
iterating whole over the dict is O(n) where n is the historical max size of the
dict.
On the other hand, there are no guarantee about `next(iter(d))` is O(1). The
guarantee is O(n) at worst case. And your example is the worst case. So this is
not a bug
Inada Naoki added the comment:
Can https://bugs.python.org/issue32623 be a fix (or mitigation) of this issue?
--
___
Python tracker
<https://bugs.python.org/issue44
Inada Naoki added the comment:
I see. But Python 3.8 is now security fix mode.
Let's revert the optimization in the 3.9 branch.
--
___
Python tracker
<https://bugs.python.org/issue42
Inada Naoki added the comment:
Because this is SSL issue. HTTPS is not the only user of SSL.
So we should try to fix SSL issue before reverting GH-12698.
--
___
Python tracker
<https://bugs.python.org/issue42
Inada Naoki added the comment:
Another upstream issue: https://github.com/mesonbuild/meson/issues/8263
--
___
Python tracker
<https://bugs.python.org/issue44
Inada Naoki added the comment:
Please write a link to the original issue next time.
https://github.com/mesonbuild/meson/issues/8916
--
___
Python tracker
<https://bugs.python.org/issue44
Change by Inada Naoki :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Inada Naoki added the comment:
>> PS > $OutputEncoding = [System.Text.Encoding]::GetEncoding("UTF-8")
> FYI, $OutputEncoding in PowerShell has nothing to do with the python.exe and
> more.com processes, nor the console session to which they're attached.
Inada Naoki added the comment:
I confirmed this fixes the mojibake:
```
PS > $OutputEncoding = [System.Text.Encoding]::GetEncoding("UTF-8")
PS > [System.Console]::OutputEncoding = $OutputEncoding
PS > [System.Console]::InputEncodin
Inada Naoki added the comment:
> In Windows, pydoc uses the old "more.com" pager with a temporary file that's
> encoded with the default encoding, which is the process active codepage (i.e.
> "ansi" or "mbcs"), unless UTF-8 mode is enabled. The
Inada Naoki added the comment:
Do you use PowerShell?
Please run this command and paste the output.
```
PS> $OutputEncoding
PS> [System.Console]::OutputEncoding
```
--
nosy: +methane
___
Python tracker
<https://bugs.python.org/i
Inada Naoki added the comment:
Can we close this issue?
--
___
Python tracker
<https://bugs.python.org/issue44206>
___
___
Python-bugs-list mailing list
Unsub
Change by Inada Naoki :
--
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed
___
Python tracker
<https://bugs.python.or
Inada Naoki added the comment:
New changeset 35be1f3602c8c66199d8c0a2f2805115864235fb by Miss Islington (bot)
in branch '3.10':
bpo-44256: Do not expose _functools._list_elem_type (GH-26416)
https://github.com/python/cpython/commit/35be1f3602c8c66199d8c0a2f2805115864235fb
101 - 200 of 3179 matches
Mail list logo
