[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Change by Barry A. Warsaw : -- nosy: +barry ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: In https://github.com/python/importlib_metadata/issues/324, I painstakingly created a library to perform robust performance monitoring of behaviors in the library and specifically added tests that capture the performance of entry_points against

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

HEY GOOGLE added the comment: Addes aggreement -- components: +Build nosy: +hectorizdaone type: -> security ___ Python tracker ___

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Miro Hrončok added the comment: The test_entry_points_by_index test also fails on Fedora. See issue44451. -- nosy: +hroncok ___ Python tracker ___

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: The build error is tracked in issue43298. -- ___ Python tracker ___ ___ Python-bugs-list

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: I managed to put together a Dockerfile that seemingly has the build tools installed (https://github.com/jaraco/jaraco.windows/blob/d2edad2e2af9d469189d7ac6a14a4ba6f6270348/Dockerfile). When I attempt to build CPython, however, it fails with this error:

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: In [this Dockerfile](https://github.com/jaraco/jaraco.windows/blob/feature/vs-2/Dockerfile), I've attempted to install Visual Studio, but without success. Docker fails to build on [line

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Anthony Sottile added the comment: they are directly coupled which is why I commented here the api redesign forces O(N) lookups and O(N) constructions which directly impact performance causing the regression -- resolution: fixed -> status: closed -> open

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: As mentioned in msg394775, I'd like to decouple the performance concerns from the original incompatibility. I recognize that performance regressions are in their own way a form of incompatibility, but there have been a lot of changes to entry points with

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Anthony Sottile added the comment: here's the performance regressions, they affect any callers of `distributions()` and are even worse on callers of the new apis. a call to distributions() is about 3x slower than in 3.9 here is the setup I am using: virtualenv venv39 -ppython3.9

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: Thanks Steve for the feedback. > If it's a warning, it could also be that the warning is being triggered > somewhere else first. Since tests run in a random order, you'll want to look > for how reliable the failure is. It is a warning, but it seems

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Steve Dower added the comment: The appx layout is also the only one in CI that actually uses an installed layout - all the rest run tests from the source tree. So it could be related to that. If it's a warning, it could also be that the warning is being triggered somewhere else first.

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: The line where the failure occurs is the point where it's checking that the warning was issued. The fact that a StopIteration is raised indicates that no warnings were caught. I can think of a couple of scenarios where that could happen: - That warning is

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Ken Jin added the comment: The new test `test_entry_points_by_index` (added in c34ed08d975fb7daa7b329f7c631647782290393 ) seems to fail on some windows buildbots:

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: New changeset d0991e2db3bb932e2411ee9dca54fd69ff2611c4 by Miss Islington (bot) in branch '3.10': bpo-44246: Remove note about access by index now that a compatibility shim is offered. (GH-26472) (#26473)

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

miss-islington added the comment: New changeset 7207203e1d71e4bf65e5b4991f60e7dc1e35e813 by Miss Islington (bot) in branch '3.10': [3.10] bpo-44246: Restore compatibility in entry_points (GH-26468) (GH-26471) https://github.com/python/cpython/commit/7207203e1d71e4bf65e5b4991f60e7dc1e35e813

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Change by miss-islington : -- pull_requests: +25068 pull_request: https://github.com/python/cpython/pull/26473 ___ Python tracker ___

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: New changeset 78d9a9b1904f0e1d9db1e941c19782f4f5a881d4 by Jason R. Coombs in branch 'main': bpo-44246: Remove note about access by index now that a compatibility shim is offered. (GH-26472)

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Change by Jason R. Coombs : -- pull_requests: +25067 pull_request: https://github.com/python/cpython/pull/26472 ___ Python tracker ___

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: The compatibility concerns are addressed with PR 26468 and some of the performance concerns may be mitigated with PR 26467. As I mentioned before, if there are continuing performance concerns, please raise them separately (preferably with

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: New changeset c34ed08d975fb7daa7b329f7c631647782290393 by Jason R. Coombs in branch 'main': bpo-44246: Restore compatibility in entry_points (GH-26468) https://github.com/python/cpython/commit/c34ed08d975fb7daa7b329f7c631647782290393 --

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

miss-islington added the comment: New changeset d1480ad2f5d5f02ecbe4b4091e8c428ddfa39ff6 by Miss Islington (bot) in branch '3.10': bpo-44246: Entry points performance improvements. (GH-26467) https://github.com/python/cpython/commit/d1480ad2f5d5f02ecbe4b4091e8c428ddfa39ff6 --

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Change by miss-islington : -- pull_requests: +25064 pull_request: https://github.com/python/cpython/pull/26469 ___ Python tracker ___

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: New changeset 410b70d39d9d77384f8b8597560f6731530149ca by Jason R. Coombs in branch 'main': bpo-44246: Entry points performance improvements. (GH-26467) https://github.com/python/cpython/commit/410b70d39d9d77384f8b8597560f6731530149ca --

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Change by Jason R. Coombs : -- pull_requests: +25063 pull_request: https://github.com/python/cpython/pull/26468 ___ Python tracker ___

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Change by Jason R. Coombs : -- pull_requests: +25062 pull_request: https://github.com/python/cpython/pull/26467 ___ Python tracker ___

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Anthony Sottile added the comment: it does not, it restores apis but in a way which requires a huge performance hit to avoid deprecation warnings it also still has the 2-500x performance regression I've stated above -- ___ Python tracker

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: importlib_metadata 4.4 restores compatibility for the reported concerns. I'll merge those into CPython later. -- ___ Python tracker ___

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Anthony Sottile added the comment: I have also shown that the performance is indeed not better in the nominal case, as demonstrated in the first case -- ___ Python tracker

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Anthony Sottile added the comment: the toil is still present, the existing, good apis are deprecated and the new, bad apis are slow -- and the odd subclasses are still present -- ___ Python tracker

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: There are known performance concerns. I recommend to set those aside for now or move them to a separate issue because (a) The performance is theoretically better in the nominal case because it avoids a sort/group operation. (b) There are known performance

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Anthony Sottile added the comment: oops, tiny typo in those code examples, they should say `group=` instead of `name=` -- though the performance is unchanged: (first example) ```console $ ./venv39/bin/python t.py 0.6641988754272461 $ ./venv310/bin/python t.py 1.3172023296356201 ``` (second

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Anthony Sottile added the comment: the `.select(...)` api is at least twice as slow as indexing as well: setup: ``` virtualenv venv39 -p python3.9 venv39/bin/pip install flake8 pytest pytest-randomly virtualenv venv39 -p python3.10 venv310/bin/pip install flake8 pytest pytest-randomly ```

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Anthony Sottile added the comment: also https://github.com/miurahr/aqtinstall/issues/221 (this links to importlib-metadata tracker, not sure how you missed it) -- ___ Python tracker

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Anthony Sottile added the comment: I also need `.sort(key=...)` for what it's worth, the error in this issue was just the first encountered I think my only satisfactory outcome would be: - the original api returns actual dicts - the sub-api returns actual lists - the new select is

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: Thanks Zac for your input. > Just chiming in with a plea to slow down the rate of changes to > importlib.metadata - I understand that you want to tidy up the API, but even > deprecations cause substantial work downstream. It would be difficult to go much

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Zac Hatfield-Dodds added the comment: Just chiming in with a plea to slow down the rate of changes to importlib.metadata - I understand that you want to tidy up the API, but even deprecations cause substantial work downstream. Would it really be so bad to support the older APIs until they

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: > The standard library has clear rules regarding how previously working > interfaces should be deprecated, and this changeset is violating those. At no > point was documented that relying on the list/dict trait of the existing > interface is not part of the

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: > there's the tuple subclass which pretends to be a dict. There's no tuple subclass that's pretending to be a dict. It overrides __getitem__ for convenience. It never claims to support Mapping. > mypy complains about incorrect types in overrides for both.

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

gaborjbernat added the comment: After reading through the points here, I must say I agree with Anthony here. The standard library has clear rules regarding how previously working interfaces should be deprecated, and this changeset is violating those. At no point was documented that relying

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Change by Dominic Davis-Foster : -- nosy: +domdfcoding ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Anthony Sottile added the comment: > Which types are sneaky and look like built-in types but do not act like them? well for starters, there's the tuple subclass which pretends to be a dict. but it violates substitutability for both `tuple` and `Mapping` so it's not useful in either

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

miss-islington added the comment: New changeset 59f9594f9018450101079eb47e06e3c44cfeee36 by Miss Islington (bot) in branch '3.10': [3.10] bpo-44246: Update What's New for importlib.metadata. (GH-26408) (GH-26415)

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: New changeset 28f12c9f4f39d283d823d81e311d863526dfdb54 by Jason R. Coombs in branch 'main': bpo-44246: Update What's New for importlib.metadata. (#26408) https://github.com/python/cpython/commit/28f12c9f4f39d283d823d81e311d863526dfdb54 --

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Change by miss-islington : -- nosy: +miss-islington nosy_count: 2.0 -> 3.0 pull_requests: +25009 pull_request: https://github.com/python/cpython/pull/26415 ___ Python tracker

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: Although it feels the topic has shifted from the original concern (an unintentional incompatibility) to the broader topic of the API change to entry_points generally, I'm happy to address your comments: > - I don't think they were discussed thoroughly, and

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Change by Jason R. Coombs : -- keywords: +patch pull_requests: +25001 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26408 ___ Python tracker ___

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: In the What's New, I used :func:`importlib.metadata.entry_points`, but that doesn't seem to resolve to the docs for the function. I need to figure out how to link to the entry_points anchor that's there. -- ___

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Anthony Sottile added the comment: personally I think they should be reverted -- they were not committed in spirit with the backwards compatibility policy: https://www.python.org/dev/peps/pep-0387/#making-incompatible-changes - I don't think they were discussed thoroughly, and when

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: Yes, perhaps the What's New could be refreshed. There is a Compatibility Note in the docs for `entry_points` about the deprecated usage (https://docs.python.org/3.10/library/importlib.metadata.html?highlight=importlib%20metadata#entry-points). I want the

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Anthony Sottile added the comment: the "what's new" mentions nothing of this break, nor the aggressive deprecation warnings, nor the various shifted interfaces (such as this one): https://docs.python.org/3.10/whatsnew/3.10.html#importlib-metadata --

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Jason R. Coombs added the comment: This backward incompatibility was unintentionally introduced in importlib_metadata 3.6 (https://importlib-metadata.readthedocs.io/en/latest/history.html#v3-6-0, released Feb 23) and was previously reported in

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Karthikeyan Singaravelan added the comment: Seems this was reported in https://github.com/python/importlib_metadata/issues/300 . Closed in https://github.com/python/importlib_metadata/commit/5ca9bc7dcf73d72260486afb28dadf5e532cf657. -- nosy: +xtreak

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

Change by Karthikeyan Singaravelan : -- nosy: +jaraco ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue44246] 3.10 beta 1: breaking change in importlib.metadata entry points

New submission from Anthony Sottile : this is breaking code that's unfortunately out of my control (vendor) -- also it looks really wrong ```python import importlib.metadata print('looks like a list:') print(importlib.metadata.distribution('pip').entry_points) print('first item:')