[issue32309] Implement asyncio.run_in_executor shortcut

Yury Selivanov added the comment: > > > > IMO, I think it would be a bit more clear to just explicitly call it > "threads" or "max_threads", as that explains what it's effectively doing. > While "concurrency" is still a perfectly correct way of describing the > behavior, I think it might be

[issue37903] IDLE Shell sidebar.

Guido van Rossum added the comment: Strong +1. I should have done this 20 years ago. -- nosy: +gvanrossum ___ Python tracker ___

[issue37607] segfault running code in jupyter on macOS 10.14.5 - crashed on child side of fork pre-exec

wesinator <13hu...@gmail.com> added the comment: Thanks ned - should I report issue to libcurl ? -- ___ Python tracker ___ ___

[issue37903] IDLE Shell sidebar.

Terry J. Reedy added the comment: My previous response was to comments up to msg355812. To answer Raymond's subsequent questions: The user will enter python code the same as before. Currently, the indents and resulting appearance of the code on Shell screen is different from (and worse,

[issue32309] Implement asyncio.run_in_executor shortcut

Kyle Stanley added the comment: > def __init__(self, concurrency=None): Minor clarification: the default should probably be None, which would effectively set the default maximum number of threads to min(32, os.cpu_count() + 4), once it's passed to ThreadPoolExecutor. --

[issue32309] Implement asyncio.run_in_executor shortcut

Kyle Stanley added the comment: > Number of OS threads to spawn. Ah I see, so this would correspond with the "max_workers" argument of ThreadPoolExecutor then, correct? If so, we could pass this in the __init__ for ThreadPool: def __init__(self, concurrency): ... self._executor =

[issue38630] subprocess.Popen.send_signal() should poll the process

Giampaolo Rodola' added the comment: -1 about the PR solution to suppress ProcessLookupError in case the process is gone. In psutil I solved the “pid reused problem” by using process creation time to identify a process uniquely (on start). A decorator can be used to protect the sensibile

[issue32309] Implement asyncio.run_in_executor shortcut

Yury Selivanov added the comment: >> async with asyncio.ThreadPool(concurrency=10) as pool: > I'm definitely on board with the usage of an async context manager and the > functionality shown in the example, but I'm not sure that I entirely > understand what the "concurrency" kwarg in

[issue32309] Implement asyncio.run_in_executor shortcut

Kyle Stanley added the comment: > I believe behavior occurs within shutdown_default_executor(), correct? > Specifically, within for ThreadPoolExecutor when executor.shutdown(wait=True) > is called and all of the threads are joined without a timeout, it simply > waits for each thread to

[issue37892] IDLE Shell: isolate user code input

Terry J. Reedy added the comment: One of the related changes is to add multiple options for saving the shell history. For editing, one wants code without prompts and output either omitting or commented out. For doctests, one wants code with '>>> ' and '... ' prompts and unmarked output.

[issue32309] Implement asyncio.run_in_executor shortcut

Kyle Stanley added the comment: > async with asyncio.ThreadPool(concurrency=10) as pool: I'm definitely on board with the usage of an async context manager and the functionality shown in the example, but I'm not sure that I entirely understand what the "concurrency" kwarg in

[issue37903] IDLE Shell sidebar.

Raymond Hettinger added the comment: Thanks. Can you also show a script running with F5 showing a restart, the output from the script and a subsequent interactive session that shows the inspecting variables. Also show what an exception and SyntaxError looks like. --

[issue37903] IDLE Shell sidebar.

Tal Einat added the comment: To help get a feel for what the current PR looks like, I'm attaching a screenshot (taken on Windows 10). -- Added file: https://bugs.python.org/file48690/screenshot.png ___ Python tracker

[issue38500] Provide a way to get/set PyInterpreterState.frame_eval without needing to access interpreter internals

Dino Viehland added the comment: Adding the getter/setters seems perfectly reasonable to me, and I agree they should be underscore prefixed as well. -- ___ Python tracker

[issue38631] Replace Py_FatalError() with regular Python exceptions

STINNER Victor added the comment: (I have a few more functions that I would like to patch, so I keep the issue open as place holder.) -- ___ Python tracker ___

[issue37903] IDLE Shell sidebar.

Terry J. Reedy added the comment: As stated in my second opening sentence, this issue "implements the first phase of #37892", which gives the context and specific rationale. There are 3 separate parts to a Python Shell. 1. Code entry, indentation, display, and copying. This issue fixes

[issue32309] Implement asyncio.run_in_executor shortcut

Yury Selivanov added the comment: Few thoughts: 1. I like the idea of having a context manager to create a thread pool. It should be initialized in a top-level coroutine and the passed down to other code, as in: async def main(): async with asyncio.ThreadPool(concurrency=10) as pool:

[issue38500] Provide a way to get/set PyInterpreterState.frame_eval without needing to access interpreter internals

Eric Snow added the comment: It depends on how you look at the degree to which you are interacting with the runtime. This is a fairly low-level hook into the runtime. So arguably if you are using this API then you should specify being a "core" extension. That said, getting that clever

[issue37903] IDLE Shell sidebar.

Raymond Hettinger added the comment: > By the way, how popular are doctests these days? Arguably, Sphinx has made them more popular than ever. For me, they are essential. And as long as doctest remains a non-deprecated module, we have to retain support. For the record, I'm opposed to

[issue38430] Memory leak in ThreadPoolExecutor + run_in_executor

Yury Selivanov added the comment: > It is a more complex solution but definitely 100% backward compatible; plus > the solution we can prepare people for removing the deprecated code > eventually. Yeah. Do you think it's worth it bothering with this old low-level API instead of making a

[issue32309] Implement asyncio.run_in_executor shortcut

Kyle Stanley added the comment: > Also in this case run awaits and returns the result. Yury suggested earlier > just to return the future and not await. Yeah that's roughly what my initial version was doing. I'm personally leaning a bit more towards returning the future rather than the

[issue38656] mimetypes for python 3.7.5 fails to detect matroska video

toonn added the comment: The result is the same for 3.7.4, on my mac. -- ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue37903] IDLE Shell sidebar.

Tal Einat added the comment: > This is marked for 3.7 through 3.9. Do you intend to backport this PR? IDLE generally has a different backporting policy: We generally backport all IDLE development to all versions of Python that aren't in security-only or unmaintained status, except for 2.7.

[issue37903] IDLE Shell sidebar.

Tal Einat added the comment: Stephen (Zero), thanks for the clarification. FWIW I'm -1 on the ability to customize the prompts in the sidebar. It would complicate the implementation while reducing the straightforwardness and simplicity of IDLE. Also, anyone who *really* wants to customize

[issue37903] IDLE Shell sidebar.

Tal Einat added the comment: > After the PR, can someone easily reproduce the interactive prompt sessions > in the tutorial and maindocs. Yes, except with prompts (both >>> and ...) in the sidebar rather than inline. > Will it look the same? No, see above. > Would they be able to build

[issue37903] IDLE Shell sidebar.

Raymond Hettinger added the comment: After the PR, can someone easily reproduce the interactive prompt sessions in the tutorial and maindocs. Will it look the same? Would they be able to build doctest examples in the IDLE shell? This is marked for 3.7 through 3.9. Do you intend to

[issue38430] Memory leak in ThreadPoolExecutor + run_in_executor

Andrew Svetlov added the comment: I thought about returning a special subclass. Future has too rich API: get_loop(), done(), result() etc. What's about returning the proxy object with future instance embedded; the object raises DeprecationWarning for everythin except __repr__, __del__ and

[issue38430] Memory leak in ThreadPoolExecutor + run_in_executor

Yury Selivanov added the comment: > The change is slightly not backward compatible but Yeah, that's my main problem with converting `loop.run_in_executor()` to a coroutine. When I attempted doing that I discovered that there's code that expects the method to return a Future, and so expects

[issue38657] String format for hexadecimal notation breaks padding with alternative form

Vedran Čačić added the comment: The width doesn't mean "the number of bits", it means "the width of the field". In every other case too: * when we format negative numbers, width includes the minus sign * when we format decimal numbers, width includes decimal point (or comma) * when we format

[issue38664] await execution order leads to throw or bad syntax

Andrew Svetlov added the comment: What patch are you talking about? For me, patch means something special; e.g. working fork of CPython where the proposed idea is implemented. I read await coro()[key] as 1. Function named coro() returns a dict-like object 2. An element under key is selected

[issue37527] Timestamp conversion on windows fails with timestamps close to EPOCH

Paul Ganssle added the comment: Ah, my mistake. The examples all use `datetime.fromtimestamp`, so I didn't notice that it was failing only on the `timestamp` side. Re-opening, thanks! -- resolution: duplicate -> status: closed -> open superseder: [Windows] datetime.fromtimestamp(t)

[issue38657] String format for hexadecimal notation breaks padding with alternative form

Change by Mark Dickinson : -- nosy: +mark.dickinson ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36439] Inconsistencies with datetime.fromtimestamp(t) when t < 0

Steve Dower added the comment: I've emailed some colleagues to see what they can add here. Ultimately, the Windows CRT is just doing arithmetic, since POSIX time formats do not resemble Windows time formats at all, so it's all emulation. So if we replaced it with our own calculation I don't

[issue38665] Crash when running SQLAlchemy with pyodbc

New submission from Dave Johansen : We're using SQLAlchemy 1.3.10 with pyodbc 4.0.27 in the python:3.7.5-alpine docker image to connect to a MySQL 13.0.5026.0 database and it's crashing with the following error: python: malloc.c:2406: sysmalloc: Assertion `(old_top == initial_top (av) &&

[issue38664] await execution order leads to throw or bad syntax

New submission from rmlibre : If a coroutine returns a sequence, a user cannot write this clean code: >>> await coro()[index] Or, >>> await coro()[slice] Or, >>> await coro()[key] This raises a TypeError("'coroutine' object is not subscriptable"). To solve this on the user side, one must add

[issue38662] Decouple ensurepip from pip's internals using runpy

Pradyun Gedam added the comment: Note that the patch uses the fact that: ``` def foo(): try: raise Exception("returned") except Exception as e: print("except") return e.args[0] finally: print("finally") print(foo()) ``` will print: ``` except

[issue38663] Untokenize does not round-trip ws before bs-nl

New submission from Edward K Ream : Tested on 3.6. tokenize.untokenize does not round-trip whitespace before backslash-newlines outside of strings: from io import BytesIO import tokenize # Round tripping fails on the second string. table = ( r''' print\ ("abc") ''', r''' print \

[issue38662] Decouple ensurepip from pip's internals using runpy

Change by Pradyun Gedam : -- keywords: +patch pull_requests: +16543 stage: -> patch review pull_request: https://github.com/python/cpython/pull/17029 ___ Python tracker ___

[issue38662] Decouple ensurepip from pip's internals using runpy

New submission from Pradyun Gedam : Inspired by https://github.com/python/cpython/pull/16782#discussion_r335285656 This changes ensurepip's current assumption that pip would expose a "main" function, which can be invoked to run pip. I've selected all actively supported Python versions for

[issue38657] String format for hexadecimal notation breaks padding with alternative form

Eric V. Smith added the comment: Now that I re-read this, maybe it was a documentation request, not a functional change? I'd be okay with documenting the existing behavior, so I'll re-open this and change the type. Patches welcome. -- components: +Documentation resolution: rejected

[issue37527] Timestamp conversion on windows fails with timestamps close to EPOCH

Ma Lin added the comment: issue29097 fixed bug in `datetime.fromtimestamp()`. But this issue is about `datetime.timestamp()`, not fixed yet. -- ___ Python tracker ___

[issue38159] PyState_AddModule docs should say that it's not necessary to call it.

Change by miss-islington : -- pull_requests: +16542 pull_request: https://github.com/python/cpython/pull/17027 ___ Python tracker ___

[issue38159] PyState_AddModule docs should say that it's not necessary to call it.

Change by miss-islington : -- pull_requests: +16541 pull_request: https://github.com/python/cpython/pull/17026 ___ Python tracker ___

[issue38159] PyState_AddModule docs should say that it's not necessary to call it.

Petr Viktorin added the comment: New changeset 9bc94eca0c69a551f928692364a99e9b67c4a45b by Petr Viktorin in branch 'master': bpo-38159: Clarify documentation of PyState_AddModule (GH-16101) https://github.com/python/cpython/commit/9bc94eca0c69a551f928692364a99e9b67c4a45b --

[issue38661] Changes to tkinter result in (unexpected) widget map call return changes wrt. 3.7

Change by Serhiy Storchaka : -- nosy: +serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36439] Inconsistencies with datetime.fromtimestamp(t) when t < 0

Paul Ganssle added the comment: This has been coming up in a few different contexts lately, so I think it would be really good if we could get some sort of fix for it. One option is to implement our own versions of these APIs for use in Windows, but a thought occurred to me recently: we

[issue38631] Replace Py_FatalError() with regular Python exceptions

Eric Snow added the comment: FWIW, I agree. :) -- nosy: +eric.snow ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue38233] datetime.datetime.fromtimestamp have different behaviour on windows and mac

Paul Ganssle added the comment: Changing the superceder here as I think #36439 matches better than #37527. -- nosy: +p-ganssle resolution: duplicate -> status: closed -> open superseder: Timestamp conversion on windows fails with timestamps close to EPOCH -> Inconsistencies with

[issue37527] Timestamp conversion on windows fails with timestamps close to EPOCH

Paul Ganssle added the comment: This indeed seems to be a duplicate of 29097, which is fixed in Python 3.7, so we can close this bug. Thank you for your report Dschoni, and thank you for finding the duplicate Ma Lin! -- resolution: -> duplicate stage: -> resolved status: open ->

[issue38661] Changes to tkinter result in (unexpected) widget map call return changes wrt. 3.7

New submission from Jose Salvatierra : Hello! I've encountered what might be a bug. Up till now we had some working code that did this: ``` maps = self.style.map('TCombobox') if maps: self.style.map('DateEntry', **maps) ``` Modifying a custom style to mimic the map of another. This has

[issue38654] `urllib.request.Request` uses mutable value as default value

Steven D'Aprano added the comment: > Also, if some new Python coders saw `[]` or `{}` being used as default > values in the standard library, they might think “I’ll do it too since > the standard library does it”. Great! Having Python coders learn good progamming skills from the standard

[issue24159] Misleading TypeError when pickling bytes to a file opened as text

Борис Верховский added the comment: As I said in issue38226, the error message you get when you try to pickle.load() a file opened in "r" mode instead of "rb" mode, UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte is also confusing and can be

[issue38226] pickle.dump and load error message when file isn't opened in binary mode are not useful

Change by Борис Верховский : -- resolution: -> duplicate stage: -> resolved status: open -> closed ___ Python tracker ___ ___

[issue38659] enum classes cause slow startup time

Ethan Furman added the comment: I was just looking at this problem, and creating a bare-bones, no safety belts version for use in the stdlib (no patch yet) which decreases Enum creation from 14x slower to only 6x slower. (Comparing to a class with simple attributes.) Not sure if that's

[issue38657] String format for hexadecimal notation breaks padding with alternative form

Eric V. Smith added the comment: int.__format__ inherits this from %-formatting, which inherits it from C's printf. There's no way we're going to change this at this point: the breakage would be too great. So, I'm going to reject this. -- assignee: -> eric.smith nosy: +eric.smith

[issue38226] pickle.dump and load error message when file isn't opened in binary mode are not useful

Serhiy Storchaka added the comment: This looks like a duplicate of issue24159. -- nosy: +serhiy.storchaka ___ Python tracker ___

[issue37903] IDLE Shell sidebar.

Stephen Paul Chappell added the comment: Zero: "not to have them added as text as is usual in a terminal window" taleinat: "removing prompts from the shell window's text widget" Zero: "print the values of ps1 and ps2 in the proposed ShellIO subclass" taleinat: "separate sidebar

[issue32309] Implement asyncio.run_in_executor shortcut

Paul Martin added the comment: Good points. I made a mistake in run Should be: async def run(self, func, *args, **kwargs): call = functools.partial(func, *args, **kwargs) return await self._loop.run_in_executor(self._executor, call) Also in this case run awaits and

[issue30385] Segfault on OSX with 3.6.1

ThePokestarFan added the comment: A pythonic way to do it is `os.environ[“no_proxy”]=“*”` -- nosy: +ThePokestarFan ___ Python tracker ___

[issue32309] Implement asyncio.run_in_executor shortcut

Kyle Stanley added the comment: Actually, I think it would be better to move the functionality of loop.shutdown_default_executor() to a new private method loop._shutdown_executor() that takes an executor argument rather than shutting down the default one. This could be used in both

[issue13829] exception error in _scproxy.so when called after fork

ThePokestarFan added the comment: Still present in python 3.8 and issue 38658. Another workaround is in https://bugs.python.org/issue30385#msg293958 -- nosy: +ThePokestarFan versions: +Python 3.8 ___ Python tracker

[issue32309] Implement asyncio.run_in_executor shortcut

Kyle Stanley added the comment: > thread = threading.Thread(target=loop._do_shutdown, args=(executor,future)) Correction: > thread = threading.Thread(target=_do_shutdown, args=(loop, executor,future)) Also, it might make more sense to rename _do_shutdown() to _do_executor_shutdown() to

[issue38658] Python Program crashes when running in fore and back ground

ThePokestarFan added the comment: Thank you. Closing this issue. -- resolution: -> duplicate stage: -> resolved status: open -> closed ___ Python tracker ___

[issue32309] Implement asyncio.run_in_executor shortcut

Kyle Stanley added the comment: Also, I agree with Paul's idea of initializing the ThreadPoolExecutor in the __init__ instead of __aenter__, that makes more sense now that I think about it. -- ___ Python tracker

[issue32309] Implement asyncio.run_in_executor shortcut

Kyle Stanley added the comment: > I don't think changing the default executor is a good approach. What happens, > if two or more thread pools are running at the same time? In that case they > will use the same default executor anyway, so creating a new executor each > time seems like a

[issue37903] IDLE Shell sidebar.

Tal Einat added the comment: Stephen, perhaps what you're suggesting is out of context for this issue? This issue is about removing prompts from the shell window's text widget, replacing them with a separate sidebar showing where prompts and user input were. This will make IDLE's shell less

[issue32309] Implement asyncio.run_in_executor shortcut

Andrew Svetlov added the comment: Paul's version looks better. Two notes: 1. get_running_loop() should be used instead of get_event_loop() 2. There is no `await executer.shutdown()` API, the method is synchronous. -- ___ Python tracker

[issue9495] argparse unittest tracebacks are confusing if an error is raised when not expected

alclarks added the comment: I'm a newcomer looking to contribute - would it be a good idea for me to update one of the attached diffs and raise a pull request? -- nosy: +alclarks ___ Python tracker

[issue32309] Implement asyncio.run_in_executor shortcut

Paul Martin added the comment: Run method should be: async def run(self, func, *args, **kwargs): call = functools.partial(func, *args, **kwargs) return await self._loop.run_in_executor(None, call) -- ___ Python tracker

[issue32309] Implement asyncio.run_in_executor shortcut

Paul Martin added the comment: I don't think changing the default executor is a good approach. What happens, if two or more thread pools are running at the same time? In that case they will use the same default executor anyway, so creating a new executor each time seems like a waste.

[issue32309] Implement asyncio.run_in_executor shortcut

Andrew Svetlov added the comment: `run` should be awaitable method, see #38430 -- ___ Python tracker ___ ___ Python-bugs-list

[issue38660] Checking if two regexes are equal should test if they are functionally equivalent

Serhiy Storchaka added the comment: Python's regular expressions are not actually regular. Lookarounds and groups make things more complex. Even it it is possible to build an ambiguous graph, its size and the time of building can be too large for practical use. Dealing with unicode is only

[issue32309] Implement asyncio.run_in_executor shortcut

Kyle Stanley added the comment: So, here's a prototype implementation of asyncio.ThreadPool that would function exactly as Yury described, but I'm not convinced about the design. Usually, it seems preferred to separate the context manager from the rest of the class (as was done with

[issue38226] pickle.dump and load error message when file isn't opened in binary mode are not useful

Change by Борис Верховский : -- type: -> enhancement ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue38660] Checking if two regexes are equal should test if they are functionally equivalent

Борис Верховский added the comment: I saw two Python regexes, one derived from a regex in the standard library. There was a comment saying that they're interchangeable and I wanted to check if they were actually the same without having to read what all the regex symbols mean. Plus a true

[issue38657] String format for hexadecimal notation breaks padding with alternative form

Serhiy Storchaka added the comment: Yes, the width is the width of the formatted value, not the number of digits. What is your proposition? -- nosy: +serhiy.storchaka ___ Python tracker

[issue38660] Checking if two regexes are equal should test if they are functionally equivalent

Serhiy Storchaka added the comment: These two regexes are not the same. >>> re.compile('([-_.a-zA-Z0-9]+)').match('ä') >>> re.compile(r'([-\w.]+)').match('ä') As Ammar said checking that two regexes always matches the same is very difficult problem. It is the problem of determining if two

[issue38660] Checking if two regexes are equal should test if they are functionally equivalent

Ammar Askar added the comment: The notion of equivalent regular expressions does exist but is way more complicated than the simple example you described. For example: r"a|b" is the same as r"[ab]", r"^aa*$" is the same as r"^a+$" Implementing this properly would probably require a

[issue33187] Document ElementInclude (XInclude) support in ElementTree

Change by Stefan Behnel : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue38660] Checking if two regexes are equal should test if they are functionally equivalent

New submission from Борис Верховский : re.compile('([-_.a-zA-Z0-9]+)') == re.compile(r'([-\w.]+)') should return True because those are the same regex (\w is a-z, A-Z, 0-9 and the underscore). If you want to check if two regexes are identical you would compare the original strings, before