[issue44115] Improve conversions for fractions
Change by Martin Teichmann : -- keywords: +patch pull_requests: +24705 stage: -> patch review pull_request: https://github.com/python/cpython/pull/26064 ___ Python tracker <https://bugs.python.org/issue44115> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44115] Improve conversions for fractions
New submission from Martin Teichmann : Currently, fraction.Fractions can be generated from floats via their as_integer_ratio method. This had been extended to also work with the Decimals class. It would be more generic - and IMHO more Pythonic - to just allow any data type, as long as it has an as_integer_ratio method. As an example, this allows numpy floats to be converted into fractions. -- components: Library (Lib) messages: 393507 nosy: Martin.Teichmann priority: normal severity: normal status: open title: Improve conversions for fractions type: enhancement versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue44115> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37334] Add a cancel method to asyncio Queues
Martin Teichmann added the comment: This is a years old issue, unfortunately it never got neither merged nor rejected. I just rebased it and hope somebody could finish the review. -- versions: +Python 3.11 -Python 3.8 ___ Python tracker <https://bugs.python.org/issue37334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37334] Add a cancel method to asyncio Queues
Martin Teichmann added the comment: I do not think that exposing the lists of futures does any good. I cannot come up with a semantics that could be implemented with that other than the one proposed. Also I think that close() or cancel() is something a reader intuitively understands, while looping over a list of futures is not immediately obvious. -- ___ Python tracker <https://bugs.python.org/issue37334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37334] Add a cancel method to asyncio Queues
Martin Teichmann added the comment: Yes, in the one-producer-many-consumers situation on can indeed to the trick with the None. But this is just a clumsy hack, cancelling the tasks is IMHO more in line with asyncio. In the many-producers-one-consumer scenario this does not work. The one dead consumer cannot feed back to the producers. Sure, there are still many hacks imaginable, but a closing or cancelling the queue is a very clear way of doing things. As for the naming: I personally don't care, close() or cancel() are both fine with me. -- ___ Python tracker <https://bugs.python.org/issue37334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37334] Add a cancel method to asyncio Queues
Martin Teichmann added the comment: Hi Andrew, I still don't get your point. First, this is an extension to the asyncio library, so concurrency is not an issue. And sure, if you call random methods of an object without any reason the outcome won't be anything useful, why, in your example, should the one task call close at all? There is, however, a strong use case: If you have many producers but just one consumer and the consumer stops consuming, it should cancel all producers waiting on the queue. The same way, if you have one producer but many consumers, once the single producer stops producing, it should cancel all waiting consumers. In these situations, the outcome is very clear. Whether it should be possible to "resurrect" a closed/cancelled queue I don't care much, as I neither see a use case in doing so nor a compelling argument why it should be artificially prohibited. So I simply went for the most simple code. The proposed code does something very simple, something a user can grasp. That is IMHO a better protection for users than some code trying to artificially stop users from harming themselves. -- ___ Python tracker <https://bugs.python.org/issue37334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37334] Add a cancel method to asyncio Queues
Martin Teichmann added the comment: Given the reactions I gather "close" is a better name for the method, so I changed it accordingly. In the current implementation, items that had been put on the queue but not processed yet still get processed after the close, and I think this is the desired behavior. I added a test such that this won't unexpectedly change in the future. To be precise, the current implementation of the queue does not even put new items on the queue if there is already a waiting consumer. The item will directly be handed over to said consumer, which may hang around on the event loop for a bit longer, but during this time the item is not in the queue. This also answers the questions about catching the CancelledError: if there are waiting consumers, there is nothing on the queue, so the problem of processing leftover items does not exist. The same holds for the task_done. As for the "why don't I just cancel the task?", well, if you know it. There may be many consumer or producer tasks waiting for their turn. Sure, you can keep a list of all those tasks. But that's exactly the point of the proposed change: the Queue already knows all the waiting tasks, no need to keep another list up-to-date! -- ___ Python tracker <https://bugs.python.org/issue37334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37334] Add a cancel method to asyncio Queues
Martin Teichmann added the comment: I also thought about `.close()` but then found `.cancel()` more intuitive. But intuition is not universal, so I am open to any wording. -- ___ Python tracker <https://bugs.python.org/issue37334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37334] Add a cancel method to asyncio Queues
Change by Martin Teichmann : -- keywords: +patch pull_requests: +14061 stage: -> patch review pull_request: https://github.com/python/cpython/pull/14227 ___ Python tracker <https://bugs.python.org/issue37334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37334] Add a cancel method to asyncio Queues
New submission from Martin Teichmann : When working with queues, it is not uncommon that at some point the producer stops producing data for good, or the consumer stops consuming, for example because a network connection broke down or some user simply closed the session. In this situation it is very useful to simply cancel all the waiting getters and putters. A simple method can do that, Queue.cancel. -- components: asyncio messages: 346023 nosy: Martin.Teichmann, asvetlov, yselivanov priority: normal severity: normal status: open title: Add a cancel method to asyncio Queues type: enhancement versions: Python 3.8 ___ Python tracker <https://bugs.python.org/issue37334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33413] asyncio.gather should not use special Future
Martin Teichmann <martin.teichm...@gmail.com> added the comment: I looked a bit into the details, and found that bpo-30048 created the described weird behavior. There they fixed the problem that a cancel is ignored if a coroutine manages to cancel its own task and return immediately. As shown in the discussion there, this is actually something happening in real code, and is a valid use case. They fixed that by setting a CancelledError as an exception raised by the task, but did not cancel that task (they could have, I tested it, it would pass all tests). But this is just a side show of the fact that we have now four different beasts that can be awaited, and behave differently: coroutines, Futures, Tasks, and _GatheringFutures. I think we should consolidate that. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33413> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33413] asyncio.gather should not use special Future
Change by Martin Teichmann <martin.teichm...@gmail.com>: -- keywords: +patch pull_requests: +6388 stage: -> patch review ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33413> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33413] asyncio.gather should not use special Future
New submission from Martin Teichmann <martin.teichm...@gmail.com>: asyncio.gather() returns a _GatheringFuture, which inherits from asyncio.Future. This is weird in current asyncio, as futures are supposed to be created with loop.create_future(). So I tried to reimplement gather() without this weird special future. I succeeded, yet I stumbled over weird inconsistencies with cancellation. There are three cases: - coroutines have no special notion of cancellation, they treat CancelledError as any other exception - futures have a clear distinction between exceptions and cancellation: future.set_exception(CancelledError()) is different from future.cancel(), as only for the latter future.cancelled() is True. This is used in the _GatheringFuture: it is cancelled() only if it got cancelled via future.cancel(), if its children gets cancelled it may set_exception(CancelledError()), but it will not be cancelled itself. - Tasks consider raising a CancelledError always as a cancellation, whether it actually got cancelled or the wrapped coroutine raised CancelledError for whatever other reason. There is one exception: if the coroutine manages to return immediately after being cancelled, it raises a CancelledError, but task.cancelled() is false. So if a coroutine ends in current_task().cancel() return the current task raises a CancelledError, but task.cancelled() is false. I consider the last exception actually a bug, but it allows me to make my inheritance-free gather() look to the outside exactly like it used to be. -- messages: 316085 nosy: Martin.Teichmann priority: normal severity: normal status: open title: asyncio.gather should not use special Future type: behavior versions: Python 3.8 ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue33413> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30306] release arguments of contextmanager
Martin Teichmann added the comment: Hi, so where do we go from here? I could write the proposed allow_recreation flag, but IMHO this adds only dead code, as it is of no need at all. I don't think code gets more clear by adding dead code... I opted for one line more of comment, I think that helps much more. Plus the additional tests I wrote in the PR I think it would be good in the Python core. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30306> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30306] release arguments of contextmanager
Martin Teichmann added the comment: I personally prefer the current situation. The problem is the following: when an @contextmanager is first called, we don't know yet whether the user wants to use it directly, or as a decorator, so we have to have some kind of hybrid class. Once it's used as a decorator, we need to recreate a context manager every time the decorate function is called. Then we need exactly the direct context manager mentioned above. This is why we recreate it with __class__. In my MR on github I simplify that by not recreating the entire context manager, but only the generator function. I think this is the most clear way, as it shows what's going on in a confined area of code. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30306> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30306] release arguments of contextmanager
Changes by Martin Teichmann <martin.teichm...@gmail.com>: -- pull_requests: +1602 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30306> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30306] release arguments of contextmanager
New submission from Martin Teichmann: The arguments of a function which was decorated to be a context manager are stored inside the context manager, and are thus kept alive. This is a memory leak. Example: @contextmanager def f(a): do_something_with(a) a = None # should release the memory yield if this is now called with something big, say with f(something_really_huge): pass then this something_really_huge is kept alive during the with statement, even though the function explicitly let go of it. -- components: Library (Lib) messages: 293234 nosy: Martin.Teichmann priority: normal severity: normal status: open title: release arguments of contextmanager type: resource usage versions: Python 3.7 ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30306> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29432] wait_for(gather(...)) logs weird error message
Martin Teichmann added the comment: I added a solution to this problem. I just silence the bad error message by overwriting _GatheringFuture.__del__ to do nothing. This may have undesired side effects, though. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29432> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29432] wait_for(gather(...)) logs weird error message
New submission from Martin Teichmann:
when waiting for a gather that times out, everything works as expected, yet a
weird error message is logged. To give a minimal example:
import asyncio
@asyncio.coroutine
def main():
try:
sleep = asyncio.sleep(0.2)
yield from asyncio.wait_for(asyncio.gather(sleep),
timeout=0.1)
except asyncio.TimeoutError:
print("timed out: fine")
yield from asyncio.sleep(0.1)
asyncio.get_event_loop().run_until_complete(main())
This outputs:
timed out: fine
_GatheringFuture exception was never retrieved
future: <_GatheringFuture finished exception=CancelledError()>
concurrent.futures._base.CancelledError
As you can see, I used the pre-3.5 syntax so I could test whether it works on
older systems. No, it doesn't.
I wrote a unit test for this problem, unfortunately I couldn't solve it yet.
--
components: asyncio
messages: 286858
nosy: Martin.Teichmann, gvanrossum, yselivanov
priority: normal
pull_requests: 24
severity: normal
status: open
title: wait_for(gather(...)) logs weird error message
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7
___
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue29432>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue29432] wait_for(gather(...)) logs weird error message
Changes by Martin Teichmann <martin.teichm...@gmail.com>: -- type: -> behavior ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue29432> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28797] Modifying class __dict__ inside __set_name__
Martin Teichmann added the comment: I personally prefer the first patch, which iterates over a copy of __dict__. Making a copy of a dict is actually a pretty fast operation, I would even expect that it is faster than the proposed alternative, creating tuples. Even if the second approach should be faster, the added code complexity is not worth the effort, as this is a code path where speed shouldn't matter much. -- ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue28797> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23722] During metaclass.__init__, super() of the constructed class does not work
Martin Teichmann added the comment: This is the originial patch rebased such that it applies to the current master. As a detail in the discussion: "__classcell__" is not visible during the execution of the class body, as it is added at the end of the class body. In this regard, it is different from "__qualname__", which is set at the beginning of the class body such that it may be changed. The new __classcell__ does show up, however, in the namespace parameter to the __new__ method of the metaclass. -- Added file: http://bugs.python.org/file44533/classcell.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23722> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27366] PEP487: Simpler customization of class creation
Martin Teichmann added the comment: I looked over the patch once more and found some places where I didn't follow normal coding standards. I changed that, namely now the code returns -1 meaning an exception happened and 0 on success, which is what many functions in typeobject.c do. So I think this patch should be ready. -- Added file: http://bugs.python.org/file43882/pep487.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27366> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27366] PEP487: Simpler customization of class creation
Martin Teichmann added the comment: Thanks for the nice review! I made the proposed changes, see attached patch. I am still waiting for a decision whether type.__setattr__ should call __set_name__ from python-dev, once that's sorted out I'll implement and test the one or the other behavior. -- Added file: http://bugs.python.org/file43854/pep487.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27366> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27366] PEP487: Simpler customization of class creation
Changes by Martin Teichmann <martin.teichm...@gmail.com>: Added file: http://bugs.python.org/file43835/pep487.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27366> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27366] PEP487: Simpler customization of class creation
Changes by Martin Teichmann <martin.teichm...@gmail.com>: Removed file: http://bugs.python.org/file43834/pep487.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27366> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27366] PEP487: Simpler customization of class creation
Changes by Martin Teichmann <martin.teichm...@gmail.com>: Added file: http://bugs.python.org/file43834/pep487.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27366> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27366] PEP487: Simpler customization of class creation
Changes by Martin Teichmann <martin.teichm...@gmail.com>: Removed file: http://bugs.python.org/file43611/pep487a.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27366> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23722] During metaclass.__init__, super() of the constructed class does not work
Changes by Martin Teichmann <martin.teichm...@gmail.com>: Added file: http://bugs.python.org/file43766/pep487.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23722> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23722] During metaclass.__init__, super() of the constructed class does not work
Changes by Martin Teichmann <martin.teichm...@gmail.com>: Removed file: http://bugs.python.org/file43765/pep487.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23722> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23722] During metaclass.__init__, super() of the constructed class does not work
Changes by Martin Teichmann <martin.teichm...@gmail.com>: Removed file: http://bugs.python.org/file38604/patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23722> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23722] During metaclass.__init__, super() of the constructed class does not work
Martin Teichmann added the comment: Currently, a class is created as follows: the compiler turns the class statement into a call to __build_class__. This runs the class body. If __class__ or super() is used within a method of the class, an empty PyCell is created, to be filled later with the class once its done. The class body returns this cell. Then the metaclass is called to create the actual class, and finally the cell is set to whatever the metaclass returns. This has the disadvantage that in the metaclasses __new__ and __init__, __class__ and super() are not set. This is a pity, especially because the two parameter version of super() doesn't work either, as the class is not yet bound to a name. The attached patch lets the compiler add said cell as __classcell__ to the classes namespace, where it will later be taken out by type.__new__ in order to be properly filled. This resembles the approach used for __qualname__, with the difference that __qualname__ is already added at the beginning of the classes body, such that it is visible to the user. This way __class__ will be properly set immediately after it is created, thus all methods are immediately usable, already in a metaclasses __new__ or __init__. This changes the behavior if a metaclass returns another class. currently, __build_class__ will try to set the __class__ in the methods of the class body to whatever __new__ returns, which might be completely unrelated to the classes body. -- keywords: +patch Added file: http://bugs.python.org/file43765/pep487.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23722> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27366] PEP487: Simpler customization of class creation
Changes by Martin Teichmann <martin.teichm...@gmail.com>: Removed file: http://bugs.python.org/file43506/pep487.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27366> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27366] PEP487: Simpler customization of class creation
Martin Teichmann added the comment: This is a C implementation of PEP 487, directly in the Python core -- Added file: http://bugs.python.org/file43611/pep487a.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27366> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27366] PEP487: Simpler customization of class creation
New submission from Martin Teichmann: This is the implementation of PEP 487. It adds a metaclass to types that calls a method on a class once it is subclassed. This way one can customize the creation of classes without the need to write an own metaclass. As a second functionality, it calls a method on each descriptor in a class, such that descriptors know their name. -- components: Library (Lib) files: pep487.patch keywords: patch messages: 269050 nosy: Martin.Teichmann priority: normal severity: normal status: open title: PEP487: Simpler customization of class creation type: enhancement versions: Python 3.6 Added file: http://bugs.python.org/file43506/pep487.patch ___ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue27366> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue24431] StreamWriter.drain is not callable concurrently
New submission from Martin Teichmann: Currently there is an assert statement asserting that no two tasks (asyncio tasks, that is) can use StreamWriter.drain at the same time. This is a weird limitiation, if there are two tasks writing to the same network socket, there is no reason why not both of them should drain the socket after (or before) writing to it. A simple bug fix is attached. -- components: asyncio files: patch messages: 245172 nosy: Martin.Teichmann, gvanrossum, haypo, yselivanov priority: normal severity: normal status: open title: StreamWriter.drain is not callable concurrently type: behavior versions: Python 3.4, Python 3.5, Python 3.6 Added file: http://bugs.python.org/file39681/patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue24431 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23722] During metaclass.__init__, super() of the constructed class does not work
Martin Teichmann added the comment: A note on the implementation: The compiler leaves a __cell__ entry in the class' namespace, which is then filled by type.__new__, and removed from the namespace by the latter. This is the same way it is done for __qualname__. As the patch tampers with the compiler, when testing the patch don't forget to remove old .pyc files, otherwise strange things will happen. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23722 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue23722] During metaclass.__init__, super() of the constructed class does not work
New submission from Martin Teichmann: When a new class is initialized with __init__ in a metaclass, the __class__ cell of the class about to be initialized is not set yet, meaning that super() does not work. This is a known but fixable problem. The attached patch moves the initialization of __class__ from the end of __build_class__ into type.__new__. This avoids the proliferation of methods which don't have the __class__ cell set. -- components: Interpreter Core files: patch messages: 238672 nosy: Martin.Teichmann priority: normal severity: normal status: open title: During metaclass.__init__, super() of the constructed class does not work versions: Python 3.5 Added file: http://bugs.python.org/file38604/patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue23722 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22412] Towards an asyncio-enabled command line
Martin Teichmann added the comment: well, I beg to differ. Again, to your points: to a) You claim that my flag would have to be supported by every python platform. Well, the very same built-in function has another flag, PyCF_ONLY_AST, whose precise meaning, according to the docs, might change with each Python release. And I am not sure if all your mentioned distributions (PyPy, Jython, IronPython, Cython) will ever have an actually compatible support for PyCF_ONLY_AST. While I think that it would be not too hard to support my GENERATOR flag on those platforms (maybe except Cython). to b) This is by far the most important part of my patch, something like this certainly should go into CPython. I am actually of the opinion that something like it is very important even without a) getting accepted. I just looked at the event loop of IPython, what they are doing there is just sheer horror: First they have to cripple each event loop they support (be it Qt, GTK or WX) to accept only some events at a time. Then they use ctypes (yes, ctypes: they cannot do it in Python!) to install this as a hook to PyOS_InputHook, a function so undocumented even google couldn't find docs (but many discussions where people had major problems properly using it). All of this would be solved with an input function which does not block I/O. All the event loops (including the asyncio event loop) would just install a callback on stdin and call my input once they need to. I am certainly open to all kinds of proposals on how it precisely should work, and sure bugs need to be fixed. c) sure, with b) accepted c) is just trivial. It was a mere example of the usage of b). b) was written under the principle write as much as you can in python, only the bare minimum in C which I think is a good idea given that it is so much nicer to program in Python than in C. The fact that it would not work under Windows to me is actually not a point, we would just have to make it run under Windows with different means (unfortunately, I'm not such a Windows expert that I would now which those means would be). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22412 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22412] Towards an asyncio-enabled command line
Martin Teichmann added the comment: Well, so I am giving up, apparenty my work is not wanted here. But not before submitting at least the last version of my patch. I seperated out my _input function into two, start_input and continue_input, which are supposed to work in a loop as in start_input(prompt) while True: r = continue_input() if r is not None: return r This would help at least people like IPython. -- Added file: http://bugs.python.org/file36632/patch3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22412 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22412] Towards an asyncio-enabled command line
Martin Teichmann added the comment: And as a last comment, just for completeness, a complete async console. With it you can do cool things like from asyncio import sleep, async def f(): ...yield from sleep(3) ...print(done) yield from f() [after 3 seconds] done async(f()) [wait another 3 seconds] done Just see how async puts something in the background, while yield from keeps it up, and how the event loop runs even while we're doing nothing! Let's hope other projects out there are more open to making their event loops asyncio-compatible than CPython is - otherwise asyncio will soon cease to exist. -- Added file: http://bugs.python.org/file36633/cl.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22412 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22412] Towards an asyncio-enabled command line
Martin Teichmann added the comment: Hi Guido, thanks for the quick response, so my response to your post: to 1: thanks! to 2: I am trying to put most of the stuff into a 3rd party module, unfortunately I need changes in the python compiler, and given that monkey patching is not so simple in C, changing the compiler would mean to re-write it, I know that has been done, but I figured that starting a project like PyPy or jython is a bit too much for me right now. to 3: I started with the repo head of friday night... apparently I was not fast enough to submit this patch before someone changes the python code to the point that my patch doesn't apply anymore. I'll try to fix that soon. to 4: I'll write some docs. to 5: well, there is a callback function just some lines up which will call future.set_result(None). Maybe that was a bit too slick, I'm open for comments on how to do that better. to 6: the C code I put into readline.c is mostly a merge of readline_until_enter_or_signal (in readline.c) and builtin_input (in bltinmodule.c). You're definitely right that those functions could do with some refactoring, but I didn't dare to since they are full of complicated special code for special platforms, and I have no chance to test the code on a machine which has a completely outdated version of the readline library, so I picked the code I needed. to 6: I thought about writing the input part of my patch as a 3rd party library, but then I saw the state of builtin.input and thought that it would actually be a good idea to shine some light onto the state of that function, it is actually a horrible mess. Just look at all the awful dirty tricks that the people over at IPython have to do to use it (im)properly (I'm talking about IPython.lib.inputhook*). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22412 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22412] Towards an asyncio-enabled command line
Martin Teichmann added the comment: As promised, a new patch now for the current head. Last time I apparently got confused with how hg works, sorry. -- Added file: http://bugs.python.org/file36623/patch2 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22412 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22412] Towards an asyncio-enabled command line
New submission from Martin Teichmann: This patch is supposed to facilitate using the asyncio package on the command line. It contains two things: First, a coroutine version of builtin.input, so that it can be called while a asyncio event loop is running. Secondly, it adds a new flag to builtin.compile which allows to use the yield and yield from statements on the module level, making compile always return a generator. The latter part will enable us to run commands like the following on the command line: from asyncio import sleep yield from sleep(3) (This has been discussed on python-ideas, https://mail.python.org/pipermail/python-ideas/2014-September/029293.html) -- components: Interpreter Core, asyncio files: patch messages: 226887 nosy: Martin.Teichmann, gvanrossum, haypo, yselivanov priority: normal severity: normal status: open title: Towards an asyncio-enabled command line type: enhancement versions: Python 3.5 Added file: http://bugs.python.org/file36617/patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue22412 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20518] multiple inheritance + C extension = possibly unexpected __base__
Martin Teichmann added the comment:
I've been working a bit on a solution to this issue, one proposal
is in the attached patch. The patch adds a new flag to tp_flags,
called Py_TPFLAGS_SOLID, which marks a class as being solid, i.e.
its memory layout is incompatible with its parent layout. C classes
can then set this flag, and cpython will assure that no incompatible
classes are in the same inheritance graph.
Other solutions are certainly thinkable: Eric proposed one should
use the MRO instead of __base__. This is a great idea, why is
cpython itself not doing it? Instead of traversing the inheritance
graph trying to find the solid base, one could simply iterate over
the MRO.
In order to illustrate where the actual problem lies, here another
example:
=
rom PyQt4.QtCore import QObject, QTimer, QTimerEvent
# QObject is the parent of QTimer
class Mixin(QObject):
# this overwrites QObject.timerEvent
def timerEvent(self, e):
print('mixed in')
super().timerEvent(e)
class B(Mixin, QTimer):
pass
class C(QTimer, Mixin):
pass
event = QTimerEvent(0)
b = B()
try:
b.timerEvent(event)
except Exception as e:
print(e)
print('-')
c = C()
c.timerEvent(event)
=
I'm writing a mixin class, that overwrites a method from QObject.
In the end I am calling this method (normally that's done from
within PyQt4). For class B, this mixing in works properly, my
code is executed, but then I am getting an exception: PyQt4 had
made b and instance of QObject, not QTimer, since this is where
the __base__ is pointing to. For class C, my code is never called
because PyQt4 is not cooperating. Sure, I could write wrapper
classes for every Qt class that I want to mix into, but what
would be the point of a mixin class then?
I hope I made my problem a bit clearer.
--
Added file: http://bugs.python.org/file33997/patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20518
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue20518] Weird behavior with multiple inheritance when C classes involved
New submission from Martin Teichmann: Python behaves odd with regards to multiple inheritance and classes written in C. I stumbled over this problem while working with PyQt4, but soon realized that part of the problem is not actually in that library, but is deep down in the CPython core. For better understanding of this post, I still use PyQt4 as an example. For those who don't know PyQt4, it's an excellent Python binding for some C++ library, for this post you only need to know that QTimer is a class that inherits from QObject. The PyQt4 documentation repeatedly insists that it is not possible to inherit more than one of its classes. This is not astonishing, since this is actually a limitation of CPython. What should still be possible is to inherit from two classes if one is the parent of the other. Let me give an example: from PyQt4.QtCore import QObject, QTimer # QObject is the parent of QTimer class A(QObject): pass class B(A, QTimer): pass class C(QTimer, A): pass print(B.__base__, B.__mro__) print(C.__base__, C.__mro__) Both classes B and C technically inherit from both QObject and QTimer, but given that QTimer inherits from QObject, there is no actual multiple inheritance here, from the perspective of a class written in C. But now the problems start. The metaclass of PyQt4 uses the __base__ class attribute to find out which of its classes the new class actually decends from (this is called the best_base in typeobject.c). This is the correct behavior, this is exactly what __base__ is for. Lets see what it contains. For the class B, the second-to-last line prints: class '__main__.A' (class '__main__.B', class '__main__.A', class 'PyQt4.QtCore.QTimer', class 'PyQt4.QtCore.QObject', class 'sip.wrapper', class 'sip.simplewrapper', class 'object') So, __base__ is set to class A. This is incorrect, as PyQt4 now thinks it should create a QObject. The reason is the weird algorithm that typeobject.c uses: it tries to find the most special class that does not change the size of the its instances (called the solid_base). This sounds reasonable at first, because only classes written in C can change the size of their instances. Unfortunately, this does not hold the other way around: in PyQt4, the instances only contain a pointer to the actual data structures, so all instances of all PyQt4 classes have the same size, and __base__ will simply default to __bases__[0]. Now I tried to outsmart this algorithm, why not put the PyQt4 class as the first parent class? This is what the class C is for in my example. And indeed, the last line of my example prints: class 'PyQt4.QtCore.QTimer' (class '__main__.C', class 'PyQt4.QtCore.QTimer', class '__main__.A', class 'PyQt4.QtCore.QObject', class 'sip.wrapper', class 'sip.simplewrapper', class 'object') So hooray, __base__ is set to QTimer, the metaclass will inherit from the correct class! But well, there is a strong drawback: now the MRO, which I print in the same example, has a mixture of Python and PyQt4 classes in it. Unfortunately, the methods of the PyQt4 classes do not call super, so they are uncooperative when it comes to multiple inheritance. This is expected, as they are written in C++, a language that has a weird concept of cooperative multiple inheritance, if it has one at all. So, to conclude: it is sometimes not possible to use python cooperative multiple inheritance if C base classes are involved. This is a bummer. Can we change this behavior? Yes, certainly. The clean way would be to re-write typeobject.c to actually find the best_base in a sane way. This would be easiest if we could just find out somehow whether a class is written in Python or in C, e.g. by adding a tp_flag to PyTypeObject. best_base would then point to the most specialized parent written in C. A simpler solution would be to default to __bases__[-1] for __base__, then we can tell users to simply put their uncooperative base classes last in the list of bases. -- components: Interpreter Core messages: 210291 nosy: Martin.Teichmann priority: normal severity: normal status: open title: Weird behavior with multiple inheritance when C classes involved type: behavior versions: Python 2.7, Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20518 ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
