[issue5906] Risk of confusion in multiprocessing module - daemonic processes
Pascal Chambon added the comment: The latest doc has a quick mention about the fact that daemon is not used in the Unix sens, so it seems fine now B-) https://docs.python.org/3/library/multiprocessing.html?#multiprocessing.Process.daemon """Additionally, these are not Unix daemons or services, they are normal processes that will be terminated (and not joined) if non-daemonic processes have exited.""" My paragraph was just my one attempt at distinguishing concepts, it was never part of the official docs -- ___ Python tracker <https://bugs.python.org/issue5906> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37408] [DOC] Precise that Tarfile "format" argument only concerns writing.
Pascal Chambon added the comment: Looking at tarfile.py, "format" seems only used in addfile() indeed. -- ___ Python tracker <https://bugs.python.org/issue37408> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37408] [DOC] Precise that Tarfile "format" argument only concerns writing.
Pascal Chambon added the comment: My bad, this was a wrongly targeted PR, the real one is here: https://github.com/python/cpython/pull/14389 -- ___ Python tracker <https://bugs.python.org/issue37408> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37408] [DOC] Precise that Tarfile "format" argument only concerns writing.
Pascal Chambon added the comment: PR is on https://github.com/pakal/cpython/pull/1 -- ___ Python tracker <https://bugs.python.org/issue37408> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37408] [DOC] Precise that Tarfile "format" argument only concerns writing.
New submission from Pascal Chambon : According to https://bugs.python.org/issue30661#msg339300 , "format" argument of Tarfile.open() only concerns the writing of files. It's worth mentioning it in the doc, if it's True (confirmation from core maintainers is welcome). -- components: Library (Lib) messages: 346586 nosy: pakal priority: normal severity: normal status: open title: [DOC] Precise that Tarfile "format" argument only concerns writing. ___ Python tracker <https://bugs.python.org/issue37408> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19645] decouple unittest assertions from the TestCase class
Pascal Chambon added the comment: "Suppose failureException is set to TypeError on that TestCase class, how would your assertEquals signal failure to the test runner?" failureException is an artefact from unittest.TestCase. It's only supposed to be used in a TestCase context, with an unittest-compatible runner. If people corrupt it, I guess it's their problem? The point of decoupling is imho that other test runner might use the separate set of assertions. These assertions should raise a sensible default (i.e AssertionError) when encountering troubles, and accepting an alternate class as parameter will allow each test framework to customize the way these assertions behave for it. -- ___ Python tracker <https://bugs.python.org/issue19645> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19645] decouple unittest assertions from the TestCase class
Pascal Chambon added the comment: I don't get it, why would failureException block anything ? The unittest.TestCase API must remain the same anyway, but it could become just a wrapper towards external assertions. For example : class TestCase: assertEqual = wrap(assertions.assert_equal) Where "wrap" for example is some kind of functools.partial() injecting into external assertions a parameter "failure_exception_class". Having all these external assertions take such a parameter (defaulting to AssertionError) would be a great plus for adaptability anyway. -- versions: +Python 3.5 -Python 3.9 ___ Python tracker <https://bugs.python.org/issue19645> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue19645] decouple unittest assertions from the TestCase class
Pascal Chambon added the comment: (Redirected here from https://bugs.python.org/issue37262) I haven't dug the assertThat() idea, but why not make, as a first step, turn assertion methods in TestCase to staticmethods/classmethods, instead of instance methods? Since they (to my knowledge) don't need to access an instance dict, they could be turned into such instance-less methods, and thus be usable from other testing frameworks (like pytest, for those who want to use pytest fixtures and yet benefit from advanced assertions like Django's TestCase's assertions). "failureException" and others are meant to be (sub)class attributes, so no backwards incompatible change should occur (unless someone did really weird things with manually instantiated TestCases). -- nosy: +pakal ___ Python tracker <https://bugs.python.org/issue19645> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37262] Make unittest assertions staticmethods/classmethods
Pascal Chambon added the comment: Indeed I missed this ticket, thanks -- ___ Python tracker <https://bugs.python.org/issue37262> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37262] Make unittest assertions staticmethods/classmethods
New submission from Pascal Chambon : Is there any reasons why assertXXX methods in TestCase are instance methods and not staticmethods/classmethods? Since they (to my knowledge) don't need to access an instance dict, they could be turned into instance-less methods, and thus be usable from other testing frameworks (like pytest, for those who want to use all the power of fixtures and yet benefit from advanced assertions, like Django's TestCase's assertXXX). Am I missing something here? -- components: Tests messages: 345463 nosy: pakal priority: normal severity: normal status: open title: Make unittest assertions staticmethods/classmethods type: enhancement versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue37262> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7659] Attribute assignment on object() instances raises wrong exception
Pascal Chambon added the comment: I guess it can, since retrocompatibility prevents some normalization here. Or is it worth updating the doc about "AttributeError", misleading regarding the type of exception expected in this case ? -- status: pending -> open ___ Python tracker <http://bugs.python.org/issue7659> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue27141] Fix collections.UserList shallow copy
Changes by Pascal Chambon : -- nosy: +pakal ___ Python tracker <http://bugs.python.org/issue27141> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17716] From ... import fails when parent package failed but child module succeeded, yet works in std import case
Pascal Chambon added the comment: Well, since it's a tough decision to make (erasing all children modules when rolling back parent, or instead reconnecting with children on 2nd import of parent), I guess it should be discussed on python-dev first, shouldn't it ? -- ___ Python tracker <http://bugs.python.org/issue17716> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17716] From ... import fails when parent package failed but child module succeeded, yet works in std import case
Pascal Chambon added the comment: (sorry for the long post, but it's a complex issue I guess) I forgot to precise that I have this behaviour with the latest python2.7, as well as python3.3 (I guess other versions behave the same). I agree that having side effects in script imports looks dangerous, but on the other hand it's incredibly handy to use the "script" behaviour of module so that each one initializes/checks himself, rather than relying on the calling of initialization methods from somewhere else (many web frameworks don't even plan such setup scripts actually, I have a django ticket running on that subject just at the moment). Loads of python modules perform such inits (registration of atexit handlers, setup of loggers, of working/temp directories, or even modifying process-level settings.), so even though we're currently adding protection via exception handlers (and checking the idempotency of our imports, crucial points!), I could not guarantee that none of the modules/packages we use won't have such temporary failures (failures that can't be fixed by the web server, because module trees become forever unimportable). With the video and the importlib code, I'm beginning to have a better understanding on the from..import, and I noticed that actually both "import mypkg.module_a" and "from mypkg import module_a" get broken when mypkg raised an exception after successfully loading module_a. It's just that the second form breaks loudly, whereas the first one remains silently corrupted (i.e the variable mypkg.module_a does NOT exist in both cases, so theer are pending AttributeErrors anyway). All comes from the fact that - to talk with "importlib/_bootstrap.py" terms - _gcd_import() assumes everything is loaded and bound when a chain of modules (eg. "mypkg.module_a") is in sys.modules, whereas intermediary bindings (setattr(mypkg, "module_a", module_a)) might have been lost due to an import failure (and the removal of the mypkg module). Hum I wonder, could we just recheck all bindings inside that _gcd_import() ? I guess there would be annoying corner cases with circular imports, i.e we could end up creating these bindings whereas they are just "pending to be done" in parent frames... Issue 17636 might provide a workaround for some cases, but it doesn't fix the root problem of the "rolled back" import (eg. here the absence of binding between mypkg and module_a, whatever the import form that was used). Imagine a tree "mypkg/mypkg2/module.py", if module.py gets well loaded but mypkg and mypkg2 fail, then later, somewhere else in the code, it seems an "import mypkg.mypkg2.module" will SUCCEED even though the module tree is broken, and AttributeErrors are pending. I guess Nick was right (and me wrong), the cleanest solution seems to enforce an invariant saying that a submodule can NOT fully be in sys.modules if his parent is not either loaded or "in the process" of loading it (thus if a binding between parent and child is missing, we're simply in the case of circular dependencies). Said another way, the import system should delete all children modules from sys.modules when aborting the import of a parent package. What do you think about it ? -- ___ Python tracker <http://bugs.python.org/issue17716> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17636] Modify IMPORT_FROM to fallback on sys.modules
Changes by Pascal Chambon : -- nosy: +Pascal.Chambon ___ Python tracker <http://bugs.python.org/issue17636> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17716] From ... import fails when parent package failed but child module succeeded, yet works in std import case
Pascal Chambon added the comment: Thanks for the feedback, I'm gonna read those docs and related issues asap, and check that planned evolutions will actually fix this. just as a side note in the meantime: I dont think that the problem here is the "purge " of sys.modules, the failure is actually located in the semantic difference between the two forms of import statements, that should basically behave the same but do not (hance the interest of these related issues noted above). -- ___ Python tracker <http://bugs.python.org/issue17716> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17716] IMPORTANT - Process corruption on partly failed imports
New submission from Pascal Chambon: Hello, we've encountered several times a very nasty bug on our framework, several times tests or even production code (served by mod_wsgi) ended up in a broken state, where imports like "from . import processing_exceptions", which were NOT in circular imports and were 100% existing submodules, raised exceptions like "ImportError: cannot import name processing_exceptions". Restarting the test/server fixed it, and we never knew what happened. I've crossed several forum threads on similar issues, only recently did I find one which gave a way to reproduce the bug: http://stackoverflow.com/questions/12830901/why-does-import-error-change-to-cannot-import-name-on-the-second-import So here attached is a python2 sample (python3 has the same pb), showing the bug (just run their test_import.py) What happens here, is that a package "mypkg" fails to get imported due to an exception (eg. temporarily failuure of DB), but only AFTER successfully importing a submodule mypkg.module_a. Thus, "mypkg.module_a" IS loaded and stays in sys.modules, but "mypkg" is erased from sys.modules (like the doc on python imports describes it). The next time we try, from within the same application, to import "mypkg", and we cross "from mypkg import module_a" in the mypkg's __init__.py code, it SEEMS that the import system checks sys.modules, and seeing "mypkg.module_a" in it, it THINKS that necessarily mypkg is already initialized and contains a name "module_a" in its global namespace. Thus the "cannot import name processing_exceptions" error. Importing "module_a" as an absolute or relative import changes nothing, however doing "import mypkg.module_a" solves the problem (dunno why). Another workaround is to cleanup sys.modules in mypkg/__init__.py, to ensure that a previously failed attempt at importing the package modules doesn't hinder us. # on top of "mypkg/__init__.py" exceeding_modules = [k for k in sys.modules.keys() if k.startswith("mypkg.")] for k in exceeding_modules: del sys.modules[k] Anyway, I don't know enough python's import internals to understand why, exactly, on second import attempt, the system tries a kind of faulty getattr(mypkg, "module_a"), instead of simply returning sys.modules["mypkg.module_a"] which exists. Could anyone help with that ? That's a very damaging issue, imo, since webserver workers can reach a completely broken state because of that. PS: more generally, I guess python users lack insight on the behaviour of "from xxx import yyy", especially when yyy is both a real submodule of xxx and a variable initialized in xxx/__init__.py (it seems the real module overrides the variable), or when the __all__ list of xxx could prevent the import of a submodule of xxx by not including it. Provided I better understand the workflow of all these stuffs - that have quite moved recently I heard - I'd be willing to summarize it for the python docs. -- components: Interpreter Core files: ImportFailPy2.zip messages: 186738 nosy: Pascal.Chambon priority: normal severity: normal status: open title: IMPORTANT - Process corruption on partly failed imports type: behavior versions: Python 2.7, Python 3.1, Python 3.2, Python 3.3, Python 3.4, Python 3.5 Added file: http://bugs.python.org/file29798/ImportFailPy2.zip ___ Python tracker <http://bugs.python.org/issue17716> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5906] Risk of confusion in multiprocessing module - daemonic processes
Pascal Chambon added the comment: I've just crossed again the doc of the daemon flag (http://docs.python.org/library/multiprocessing.html), and it's still quite confusing to newcomers. daemon The process’s daemon flag, a Boolean value. This must be set before start() is called. The initial value is inherited from the creating process. [1] When a process exits, it attempts to terminate all of its daemonic child processes. [1] this sentence is weird: since daemonic processes are not allowed to have children, isn't this flag always False in a new Process instance ? [2] typo, it meant "all of its NON-daemonic child processes" instead, didn't it ? -- resolution: fixed -> status: closed -> open ___ Python tracker <http://bugs.python.org/issue5906> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1553375] Add traceback.print_full_exception()
Pascal Chambon added the comment: I dont understand, if we use traceback.print_stack(), it's the stack at the exception handling point which will be displayed. In my view, the interesting think was not the stack trace at the point where the exception is being handled, but where the unwinding stopped (i.e, a snapshot of the stack at the moment the exception was caught). I agree that most of the time these stacks are quite close, but if you happen to move the traceback object all around, in misc. treatment functions (or even, if it has been returned by functions to their caller - let's be fool), it can be handy to still be able to output a full exception stack, like if the exception had flowed up to the root of the program. At least that's what'd interest me for debugging. try: myfunction() #<- that's the point of which I'd likle a stack trace except Exception, e: handle_my_exception(e) #<- not of that point, some recursion levels deeper Am I the only one viewing it as this ? -- ___ Python tracker <http://bugs.python.org/issue1553375> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10327] Abnormal SSL timeouts when using socket timeouts - once again
Pascal Chambon added the comment: Allright, it actually looks more like a pathological latency behaviour of my target platforms than a ssl bug... I was mislead by the heavy history of socket.settimeout(), sorry. >_< -- status: open -> closed ___ Python tracker <http://bugs.python.org/issue10327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10327] Abnormal SSL timeouts when using socket timeouts - once again
Pascal Chambon added the comment: Humz on second thought you may be right, now I have some trouble reproducing the bugs (wich have been there since the beginning, though), so it may be that the webservice I call seldom takes 10+ seconds to answer (weird anyway). I've placed timers in the codebase, the pb will eventually surface again. -- ___ Python tracker <http://bugs.python.org/issue10327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10327] Abnormal SSL timeouts when using socket timeouts - once again
Pascal Chambon added the comment: The exception is raised too early, none of my calls takes more than 1-2 seconds and I've a default timeout set at 10s or more. This occurs rather rarely, one or two times on some hundreds of calls. I'll make a little script to try to isolate the pb. -- ___ Python tracker <http://bugs.python.org/issue10327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10327] Abnormal SSL timeouts when using socket timeouts - once again
New submission from Pascal Chambon : On freebsd 8, using python 2.6.6, I've run into the bug already widely dealt with in these reports : http://bugs.python.org/issue1380952 http://bugs.python.org/issue1153016 When using socket timeouts (eg. with socket.setdefaulttimeout()), whatever the timeout I use (eg. 10 seconds), I begin having random "SSLError: The read operation timed out" exceptions in my http calls, via urlopen or 3rd party libraries. Here is an example of traceback ending: ... File "/usr/local/lib/python2.6/site-packages/ZSI-2.0-py2.6.egg/ZSI/client.py", line 349, in ReceiveRaw response = self.h.getresponse() File "/usr/local/lib/python2.6/httplib.py", line 990, in getresponse response.begin() File "/usr/local/lib/python2.6/httplib.py", line 391, in begin version, status, reason = self._read_status() File "/usr/local/lib/python2.6/httplib.py", line 349, in _read_status line = self.fp.readline() File "/usr/local/lib/python2.6/socket.py", line 427, in readline data = recv(1) File "/usr/local/lib/python2.6/ssl.py", line 215, in recv return self.read(buflen) File "/usr/local/lib/python2.6/ssl.py", line 136, in read return self._sslobj.read(len) SSLError: The read operation timed out I've checked the py2.6.6 sources, the patches described in previous reports are still applied (eg. SSL_pending() checks etc.), I have no idea of how so long socket timeouts might interfere with ssl operations... -- components: IO messages: 120498 nosy: arekm, georg.brandl, maltehelmert, pakal, pristine777, tarek-ziade, twouters priority: normal severity: normal status: open title: Abnormal SSL timeouts when using socket timeouts - once again type: behavior versions: Python 2.6 ___ Python tracker <http://bugs.python.org/issue10327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1553375] Add traceback.print_full_exception()
Pascal Chambon added the comment: Indeed I don't understand the following part : +Traceback (most recent call last): + File "testmod.py", line 16, in +{exception_action} + File "testmod.py", line 6, in foo +bar() + File "testmod.py", line 11, in bar +raise Exception +Exception Why does the f_back of the first exception, when chain=True, leads back to the {exception_action} part, in the except: black, instead of the initial foo() call inside the try: block ? -- ___ Python tracker <http://bugs.python.org/issue1553375> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1553375] Add traceback.print_full_exception()
Pascal Chambon added the comment: Is that normal to have two methods "test_full_traceback_is_full" at the same place, in full_traceback.patch / r.david.murray / 2010-08-04 02:32 ? format_exception should have the same semantic as print_exception indeed. -- ___ Python tracker <http://bugs.python.org/issue1553375> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8734] msvcrt get_osfhandle crash on bad FD
Pascal Chambon added the comment: I guess it should, shouldn't it ? -- ___ Python tracker <http://bugs.python.org/issue8734> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8840] truncate() semantics changed in 3.1.2
Pascal Chambon added the comment: Good B-) Woudl it be necessary to update the docstrings too ? -- ___ Python tracker <http://bugs.python.org/issue8840> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8840] io.StringIO: truncate+print disabled in 3.1.2
Pascal Chambon added the comment: The change was announced in http://docs.python.org/dev/whatsnew/2.7.html, but indeed it wasn't advertised in py3k changes - my apologies, I didn't check it was. I agree that the doc should be clarified on several aspects. * The returned value is the new file SIZE indeed (I guess we can still use "file" here, since imo other streams can't be truncated anyway). * Truncate() simply changes the current end-of-file (the word is historical, resize() would have been better - as this has been discussed on mailing lists). * Extending the file size with truncate() or with a write() after end-of-file (that's your sample's case) does, or does not (depending on the platform), fill the empty space with zeroes. Proposal for doc update : Resizes the file to the given size (or the current position), without moving the file pointer. This resizing can extend or reduce the current file size. In case of extension, the content of the new file area depends on the platform (on most systems, additional bytes are zero-filled, on win32 they're undetermined). Returns the new file size. Would it be ok thus ? -- ___ Python tracker <http://bugs.python.org/issue8840> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8765] Tests unwillingly writing unicocde to raw streams
Pascal Chambon added the comment: yes, but the same tests are used for py3k as well, where "xxx" is interpreted as unicode (2to3 tools dont try to guess if a py2k string intended to be a byte string or an unicode one). -- ___ Python tracker <http://bugs.python.org/issue8765> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8765] Tests unwillingly writing unicocde to raw streams
Pascal Chambon added the comment: This would require patching separately py2k and py3k visibly... I'll have a look at it when I have time. -- ___ Python tracker <http://bugs.python.org/issue8765> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8765] Tests unwillingly writing unicocde to raw streams
Pascal Chambon added the comment: Allright, what's the expected behaviour then - implicitly converting unicode to bytes (like C RawFileIO), or raising a typeerror (like buffered streams do) ? -- ___ Python tracker <http://bugs.python.org/issue8765> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8763] py3K bdist_msi wrongly installs itself in ALL python versions
Pascal Chambon added the comment: None at all, a simple python "setup.py bdist_msi". Once I've installed the MSI(and whatever the target python version I chose), launching it again will always trigger a "repair/uninstall" wizard, and the operatiosn I do with it are made on ALL python versions at teh same time (eg., if I uninstall, my library gets properly removed from ALL site-packages directories). That's really weird actually, I've not found mentions of similar errors on the web or bug tracker... -- ___ Python tracker <http://bugs.python.org/issue8763> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8765] Tests unwillingly writing unicocde to raw streams
New submission from Pascal Chambon : In test_fileio, one of the tests wants to ensure writing to closed raw streams fails, but it actually tries to write an unicode string, which should rather lead to an immediate TypeError. Here is a tiny patch to prevent the "double error cause" danger - this test is bugging me because my own I/O library cant pass the stdlib io tests in this case. The initial problem here is that we can't write unicode to a buffered binary stream (TypeError), but we can do it with an unbufferred raw stream - as the C implementation of the latter does string coercion instead of raising TypeError. Shouldn't we unify the behaviour of binary streams in such cases ? -- components: IO files: test_fileio_errclosedonwrite.patch keywords: patch messages: 106053 nosy: pakal priority: normal severity: normal status: open title: Tests unwillingly writing unicocde to raw streams versions: Python 2.7 Added file: http://bugs.python.org/file17400/test_fileio_errclosedonwrite.patch ___ Python tracker <http://bugs.python.org/issue8765> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8763] py3K bdist_msi wrongly installs itself in ALL python versions
Pascal Chambon added the comment: Thansk for the attention, Here it is (I ran bdist_msi with the latest SVN py3k version). -- Added file: http://bugs.python.org/file17399/RSFile-1.0.tar.gz ___ Python tracker <http://bugs.python.org/issue8763> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8763] py3K bdist_msi wrongly installs itself in ALL python versions
New submission from Pascal Chambon : I've created a pure-python package with py3k's bdist_msi, and weirdly, even though the windows installer asks for the target directory, it installs the package in ALL python distributions installed (py26, py27 and py31), which is particularly embarassing since it's a py3k-only pure-python package... -- assignee: tarek components: Distutils files: RSFile-1.0.win32.msi messages: 106042 nosy: pakal, tarek priority: normal severity: normal status: open title: py3K bdist_msi wrongly installs itself in ALL python versions versions: Python 3.1, Python 3.2 Added file: http://bugs.python.org/file17398/RSFile-1.0.win32.msi ___ Python tracker <http://bugs.python.org/issue8763> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6583] 2to3 fails to fix test.test_support
Pascal Chambon added the comment: Sorry to reraise an old issue, but the documentation of "test" module is deceiving on that one : "The test.test_support module has been renamed to test.support in Python 3.0. The 2to3 tool will automatically adapt imports when converting your sources to 3.0.". I lost quite some time on this one... -- nosy: +pakal ___ Python tracker <http://bugs.python.org/issue6583> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8734] msvcrt get_osfhandle crash on bad FD
Changes by Pascal Chambon : Removed file: http://bugs.python.org/file17369/msvcrt_crash.patch ___ Python tracker <http://bugs.python.org/issue8734> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8734] msvcrt get_osfhandle crash on bad FD
Pascal Chambon added the comment: My bad, here is a better patch... -- Added file: http://bugs.python.org/file17382/msvcrt_crash2.patch ___ Python tracker <http://bugs.python.org/issue8734> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8734] msvcrt get_osfhandle crash on bad FD
New submission from Pascal Chambon : In python trunk, _set_invalid_parameter_handler() has been dropped and replaced by custom checking functions, but in msvcrt.get_osfhandle() these checks aren't present, so providing a bad FD leads to a crash. Here is the little fix + a test (added to test_fileio, because I can't find unit tests for the msvcrt builtin, and similar "bad fd" cases are treated there). -- components: IO files: msvcrt_crash.patch keywords: patch messages: 105872 nosy: pakal priority: normal severity: normal status: open title: msvcrt get_osfhandle crash on bad FD type: crash versions: Python 2.7 Added file: http://bugs.python.org/file17369/msvcrt_crash.patch ___ Python tracker <http://bugs.python.org/issue8734> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7640] buffered io seek() buggy
Pascal Chambon added the comment: Hello I advocate the inclusion of this patch to the 2.6 maintenance branch, because currently the io module in this branch (which is still the most recent 2.X version released) is simply broken. People using it will certainly encounter MAJOR file corruptions due to the buggy seek(). Or should we rather backport the much more evoluated io of trunk to py2.6 ? -- ___ Python tracker <http://bugs.python.org/issue7640> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8717] Subprocess.py broken in trunk
New submission from Pascal Chambon : There seems to have been some broken commit on subprocess.py in trunk - the moduel can't be imported due to unexisting argument defaults. Here is a very quick patch, but more inquiry migh tbe necessary to find out what happened. -- files: broken_subprocess.patch keywords: patch messages: 105736 nosy: pakal priority: normal severity: normal status: open title: Subprocess.py broken in trunk versions: Python 2.7 Added file: http://bugs.python.org/file17344/broken_subprocess.patch ___ Python tracker <http://bugs.python.org/issue8717> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7865] io close() swallowing exceptions
Pascal Chambon added the comment: Cool, thanks a lot B-) -- ___ Python tracker <http://bugs.python.org/issue7865> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7865] io close() swallowing exceptions
Pascal Chambon added the comment: Here is a code/test patch which *should* fix the multiple close() case, and ensure flush() raise an error on closed file. All IO test suites pass on my win32 (except test_largefile that I skip due to lack of hdd space). I've noticed that the detach() semantic was quite different between _pyio and _io, by the way: C code raises valueerror when we try to access the stream after detaching it, whereas python lets attribute errors be raised. But maybe this is not so important, as these are programming errors anyway. One thing I'm still wondering : why couldn't we obtain these C extension by cythonizing _pyio ? Are there features that cython lacks, or optimization considerations I'm not aware of ? Cython-generated extensions seem s easier to maintain... -- Added file: http://bugs.python.org/file17132/no_swallow_on_close3.patch ___ Python tracker <http://bugs.python.org/issue7865> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7865] io close() swallowing exceptions
Changes by Pascal Chambon : Removed file: http://bugs.python.org/file17077/no_swallow_on_close2.patch ___ Python tracker <http://bugs.python.org/issue7865> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7865] io close() swallowing exceptions
Pascal Chambon added the comment: >Probably an oversight. Do you want to add some tests? That's WIP > Because it's not an IO error at all. No I/O occurs. You are just using the file wrongly (or the wrong file), hence the ValueError. Then when you try to wrap a non-readable stream into a readable buffered stream (like BufferedRWPair), it should raise a value error as well, but currently it's rather: """if not reader.readable(): raise IOError('"reader" argument must be readable.')""" -- ___ Python tracker <http://bugs.python.org/issue7865> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7865] io close() swallowing exceptions
Pascal Chambon added the comment: I'm quite surprised it wasn't already covered by the test suite :S Anyway I'm quite confused about the semantic which is expected from IO operations... Should a flush on a closed stream fail (at the moment sometimes it does, sometimes doesn't) ? Why is sometimes ValueError used when I'd rather expect an IOError ? -- ___ Python tracker <http://bugs.python.org/issue7865> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7865] io close() swallowing exceptions
Pascal Chambon added the comment: SHould be better this way then B-) -- Added file: http://bugs.python.org/file17077/no_swallow_on_close2.patch ___ Python tracker <http://bugs.python.org/issue7865> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7865] io close() swallowing exceptions
Changes by Pascal Chambon : Removed file: http://bugs.python.org/file17046/release_io_close_exceptions.patch ___ Python tracker <http://bugs.python.org/issue7865> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7865] io close() swallowing exceptions
Pascal Chambon added the comment: Patch and test to stop swallowing exceptions on stream close(), for python SVN trunk. -- keywords: +patch Added file: http://bugs.python.org/file17046/release_io_close_exceptions.patch ___ Python tracker <http://bugs.python.org/issue7865> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7865] io close() swallowing exceptions
Pascal Chambon added the comment: Well, it would break code which currently ignores that it fails, so it's more a benefit than a loss for programmers imo. I doubt the impact will be important though, because the io module is still quite recent, and furthermore errors on the last flush are quite unlikely to happen if previous ones succeeded (except "disk full", I don't see any reason for this to happen). -- ___ Python tracker <http://bugs.python.org/issue7865> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8327] unintuitive behaviour of logging message propagation
Pascal Chambon added the comment: Thanks for the doc patch, if you don't mind I'd just add the paragraph below too, to clarify the fact that logger levels are only "entry points" levels, ignored he rest of the time. There might be slight redundancies with the rest of the (long) documentation, but it's all benefit imo. B-) "In addition to any handlers directly associated with a logger, *all handlers associated with all ancestors of the logger* are called to dispatch the message (unless the *propagate* flag for a logger is set to a false value, at which point the passing to ancestor handlers stops). Note that in this process, the level of ancestor loggers is never considered : only their *propagate* attribute and the levels of their loggers canimpact the treatment on the message being dispatched. The level of a logger is thus only acting as an *entry point barrier*, only able to stop the whole dispatch of a message that enters the system through it." -- ___ Python tracker <http://bugs.python.org/issue8327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8327] unintuitive behaviour of logging message propagation
New submission from Pascal Chambon : Hello Crawling into the logging module, I've just discovered its behaviour was actually far from the one I expected, in a consequent gap that the documentation had left. I thought that depending on the "propagate" parameter of each logger, a message would always try to propagate up the logger hierarchy, and that if that message matched the level of a logger (as well as the global disable() level), then this message would be tested against each handler of that logger. But it's not at all the way it is : actually, only the level of the "entrance" logger is checked ; if the message is blocked by it, it's the end, parent loggers will never hear about it (even thoiugh they might have a lower level set). On the contrary, if that entrance logger accepts the message, then ALL handlers from the hierarchy are snet the message, without caring about the own level of their related logger. That's really not the behaviour I would have expected, considering the global_disable > logger_level > handler_level precedence that the doc implicitly showed. Are there rationales behind this that I'm lacking ? So I'd advocate either (if possible) a patching of the logging system, to reflect that semantic ; or the adding of comprehensive paragraphs to teh doc, to explain what the exact relationship between levels and propagation attribuets are. Just tell me if a patch is conceivable B-) -- components: Library (Lib) messages: 102477 nosy: pakal severity: normal status: open title: unintuitive behaviour of logging message propagation versions: Python 2.7 ___ Python tracker <http://bugs.python.org/issue8327> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1553375] Add traceback.print_full_exception()
Pascal Chambon added the comment: What's the status of this (imo quite useful) new traceback function ? Shall I provide some help ? -- ___ Python tracker <http://bugs.python.org/issue1553375> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue1553375] Add traceback.print_full_exception()
Changes by Pascal Chambon : -- nosy: +pakal ___ Python tracker <http://bugs.python.org/issue1553375> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8162] logging.disable() incoherency
Pascal Chambon added the comment: Thanks you B-) I guess it's not important if messages of severity 0 will allways be ignored with the current semantic ? -- ___ Python tracker <http://bugs.python.org/issue8162> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8162] logging.disable() incoherency
New submission from Pascal Chambon : Hello I see some trouble in the semantic of logging.disable(lvl) : according to the doc (and docstrings), it's the same as a logger.setLevel(lvl) called on all logger, but in reality it doesn't act the same way : when we call logger.setLevel(lvl), log messages at level lvl WILL be logged, whereas with logger.disable(lvl), they will NOT be logged (CF method below from logging/__init__.py). So maybe the best would be to explain that disable() also disable the target level itself, and to set by default this disabling level to -1 (and not 0 as currently), so that by default ALL messages get loged, even those set to level 0. def isEnabledFor(self, level): """ Is this logger enabled for level 'level'? """ if self.manager.disable >= level: return 0 return level >= self.getEffectiveLevel() -- assignee: georg.brandl components: Documentation, Library (Lib) messages: 101215 nosy: georg.brandl, pakal severity: normal status: open title: logging.disable() incoherency type: behavior versions: Python 2.6, Python 2.7 ___ Python tracker <http://bugs.python.org/issue8162> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7865] io close() swallowing exceptions
New submission from Pascal Chambon : The current semantic of io streams is to swallow IOErrors on close(), eg. in _pyio from the trunk, we have each time constructs like: try: self.flush() except IOError: pass # If flush() fails, just give up and in C files : /* If flush() fails, just give up */ if (PyErr_ExceptionMatches(PyExc_IOError)) PyErr_Clear(); I'd rather advocate letting exceptions flow, as users have the right to know if their io operations lost bytes. Below is a test case for these swallowed exceptions. PS : issues like http://bugs.python.org/issue7640 are actually much more crucial, so we shall let them higher priority -- components: IO files: test_error_close.py messages: 98940 nosy: pakal severity: normal status: open title: io close() swallowing exceptions versions: Python 2.5, Python 2.6, Python 2.7, Python 3.1, Python 3.2 Added file: http://bugs.python.org/file16147/test_error_close.py ___ Python tracker <http://bugs.python.org/issue7865> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7640] buffered io seek() buggy
Pascal Chambon added the comment: The patch itself... -- Added file: http://bugs.python.org/file16085/Py26_relative_seek.patch ___ Python tracker <http://bugs.python.org/issue7640> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7640] buffered io seek() buggy
Pascal Chambon added the comment: BufferedRandom's seek() is still borken in the latest svn revision of python2.6 (I haven't checked python2.7 yet), so here is a new, smaller patch, tested with the 6 IO-related test suites this time, on win32 and linux. -- ___ Python tracker <http://bugs.python.org/issue7640> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7640] buffered io seek() buggy
Changes by Pascal Chambon : Removed file: http://bugs.python.org/file15759/seek_cur_buffers_py26.patch ___ Python tracker <http://bugs.python.org/issue7640> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7813] Bug in command-line module launcher
Pascal Chambon added the comment: Here is the patch you mentionned, it fixes my problem and seems unharmful indeed. The little problem is, I've not found related tests (test_pkgutil doesn't deal with the importer part) to ad little checks around compile(). Is that patch worth commiting anyway ? -- keywords: +patch Added file: http://bugs.python.org/file16084/py26_pkgutil_compile.patch ___ Python tracker <http://bugs.python.org/issue7813> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Pascal Chambon added the comment: Argh, I had indeed missed some IO-related tests, hadn't noticed the new test_io docstring: # Tests of io are scattered over the test suite: # * test_bufio - tests file buffering # * test_memoryio - tests BytesIO and StringIO # * test_fileio - tests FileIO # * test_file - tests the file interface # * test_io - tests everything else in the io module # * test_univnewlines - tests universal newline support # * test_largefile - tests operations on a file greater than 2**32 bytes # (only enabled with -ulargefile) Thanks for the commits B-) -- ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7813] Bug in command-line module launcher
Pascal Chambon added the comment: Allright, I guess in these conditions this bugs doesn't require a patch B-) Thanks for the details. -- ___ Python tracker <http://bugs.python.org/issue7813> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7813] Bug of command-line module launcher
New submission from Pascal Chambon : I have a weird behaviour of the interpreter on my python2.6 win32 install : if I launch as __main__ the file joined below, like "python test_rsFileLocking.py", all works fine. But if I put it in my PYTHONPATH (or current working directory), and launch it via the "-m" option of python.exe, I get this unexplanatory traceback below. How comes the syntax analyser is not the same in both cases ? I don't what what, in this file, could give syntax errors in the latter case... C:\Users\Pakal\Desktop\release26-maint\PCbuild>python.exe -m rstest.test_rsFileLocking Traceback (most recent call last): File "C:\Users\Pakal\Desktop\release26-maint\lib\runpy.py", line 104, in _run_module_as_main loader, code, fname = _get_module_details(mod_name) File "C:\Users\Pakal\Desktop\release26-maint\lib\runpy.py", line 85, in _get_module_details code = loader.get_code(mod_name) File "C:\Users\Pakal\Desktop\release26-maint\lib\pkgutil.py", line 275, in get_code self.code = compile(source, self.filename, 'exec') File "C:\Users\Pakal\Desktop\RockSolidTools\rstest\test_rsFileLocking.py", line 391 SyntaxError: invalid syntax -- components: Interpreter Core files: test_rsFileLocking.py messages: 98566 nosy: pakal severity: normal status: open title: Bug of command-line module launcher type: compile error versions: Python 2.6 Added file: http://bugs.python.org/file16057/test_rsFileLocking.py ___ Python tracker <http://bugs.python.org/issue7813> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Pascal Chambon added the comment: Hello Here is the patch for the python trunk, regarding truncate() behaviour. I've tested it on windows and linux, against IO test suites (with -uall), in debug and normal mode. I've also updated some docstrings, and added tests for untested potential errors around buffered read+write streams, when truncating. -- Added file: http://bugs.python.org/file16030/PY27_truncate.patch ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Pascal Chambon added the comment: Thanks a lot B-) I'll make a patch for trunk from now to this W.E. -- ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Pascal Chambon added the comment: Hello Just to notify that I've just tested this patch on a fresh python2.6 SVN checkout, on Ubuntu this time (previously, it was only win32), and it passes all IO-related tests I know. -- ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7720] Errors in tests and C implementation of raw FileIO
Pascal Chambon added the comment: Hum, it seems that in python2.6, the C API for PyArg_ParseTuple isn't yet ready for bytes and bytearrays, is it ? "y"-like argument parsers don't exist, so I guess we can't easily patch the C api on this, only tests (replacing "xxx" by b"xxx")... -- ___ Python tracker <http://bugs.python.org/issue7720> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Changes by Pascal Chambon : Removed file: http://bugs.python.org/file15909/patch26_truncate_pos_refcounts.patch ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Pascal Chambon added the comment: Allright - sorry for the failure - I've cleaned my hdd enough to launch large_file tests too. The thing is - are there platforms available to test a patch against the whole test suite of python, and against several OSes ? I've found no such thing around build bots, and since I'm kind of stuck on win32 without a full compilation environment (sqlite, zlib etc. lack some setup), I miss vision depth concerning testing... Anyway, here is a corrected patch, passing the full io-related test suites on win32 (and visibly ready for unix too). -- Added file: http://bugs.python.org/file15931/patch26_largefile_tested.patch ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7720] Errors in tests and C implementation of raw FileIO
New submission from Pascal Chambon : My own fileio implementation fails against the latests versions of the io test suite, under python2.6, because these now require FileIO objects to accept unicode strings in their write methods, whereas the doc mentions raw streams only deal with bytes/bytearrays. Below is the faulty test (unicode literals or ON). The unicode "xxx" is written to the io.FileIO instance, and the stdlib implementation indeed accepts unicode args (in _fileio.c : "if (!PyArg_ParseTuple(args, "s*", &pbuf)..."). In test_io.py : def test_destructor(self): record = [] class MyFileIO(io.FileIO): def __del__(self): record.append(1) io.FileIO.__del__(self) def close(self): record.append(2) io.FileIO.close(self) def flush(self): record.append(3) io.FileIO.flush(self) f = MyFileIO(test_support.TESTFN, "w") f.write("xxx") del f self.assertEqual(record, [1, 2, 3]) -- components: IO messages: 97910 nosy: pakal severity: normal status: open title: Errors in tests and C implementation of raw FileIO versions: Python 2.6 ___ Python tracker <http://bugs.python.org/issue7720> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Pascal Chambon added the comment: Thanks for the detailed feedback. According to what you said (and to details found in python docs), the current _fileio.truncate is actually quite buggy -> no reference counting ops were done on posobj, even though it's sometimes retrieved from _portable_seek, and returned or not depending on rare error cases (SetEndOfFile failing etc.). Here is a patch fixing all that for python2.6, I've tested it manually in debug mode, and against memoryio/io/fileio tests ; hope it will be OK. As soon as this one is certified, I'll prepare a patch for python2.7 -- Added file: http://bugs.python.org/file15909/patch26_truncate_pos_refcounts.patch ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Changes by Pascal Chambon : Removed file: http://bugs.python.org/file15766/python27_full_truncate_bugfix.patch ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Changes by Pascal Chambon : Removed file: http://bugs.python.org/file15765/python26_full_truncate_patch.patch ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Pascal Chambon added the comment: Allright, I shall fix all this asap. But it seems the C code for truncate is actually buggy in the current 2.6 _fileio.c, around line 680. CF code below : posobj = portable_lseek(fd, posobj, 0); -> don't we lose the reference to the old "posobj" there, doing a memory leak ? if (PyErr_Occurred()) return NULL; -> same thing, we return Null without caring about the posobj reference which should be non-Null there ?? If I've understood a little reference counting, "portable_lseek" returns a reference that we own and must Py_DECREF, isn't that so ? if (posobj == Py_None || posobj == NULL) { /* Get the current position. */ posobj = portable_lseek(fd, NULL, 1); if (posobj == NULL) return NULL; } else { /* Move to the position to be truncated. */ posobj = portable_lseek(fd, posobj, 0); } #if defined(HAVE_LARGEFILE_SUPPORT) pos = PyLong_AsLongLong(posobj); #else pos = PyLong_AsLong(posobj); #endif if (PyErr_Occurred()) return NULL; -- -- ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Pascal Chambon added the comment: Is there anything I can do to help this patch making its way to the trunk ? I guess it'd be better if Python2.7 benefited from it, so that users don't run anymore the risk of relying of this undocumented and non-canonical "truncate" behaviour. Regards, Pascal -- ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7659] Attribute assignment on object() instances raises wrong exception
Pascal Chambon added the comment: OK, eligible to QOTW:D -- ___ Python tracker <http://bugs.python.org/issue7659> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7659] Attribute assignment on object() instances raises wrong exception
Pascal Chambon added the comment: Allright, I suppose it's some kind of optimization ? We get a proper error when we do : >>> object.test=1 Traceback (most recent call last): File "", line 1, in object.test=1 TypeError: can't set attributes of built-in/extension type 'object' >>> So we'd need a similar error when trying to assign to object instances, too... I fear I won't be able to help with finding a patch alas, it's a little too deep into the python core for me :p -- ___ Python tracker <http://bugs.python.org/issue7659> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7659] Problems with attribute assignment on object instances
New submission from Pascal Chambon : It seems we can't assign attributes to "objet" class instances, which don't have a __dict__ : IDLE 2.6.4 >>> a = object() >>> a.abc = 3 Traceback (most recent call last): File "", line 1, in a.abc = 3 AttributeError: 'object' object has no attribute 'abc' >>> This behaviour seems undocumented, and contradicts the documentation on AttributeError -> normally, a TypeError should be raised instead: exception AttributeError Raised when an attribute reference (see Attribute references) or assignment fails. (When an object does not support attribute references or attribute assignments at all, TypeError is raised.) -- components: Interpreter Core messages: 97416 nosy: pakal severity: normal status: open title: Problems with attribute assignment on object instances type: behavior versions: Python 2.6 ___ Python tracker <http://bugs.python.org/issue7659> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Pascal Chambon added the comment: And here is the python trunk patch, covering _Pyio and _io modules (+ corresponding tests). -- Added file: http://bugs.python.org/file15766/python27_full_truncate_bugfix.patch ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Pascal Chambon added the comment: Here is the new patch for py2.6, fixing (hopefully) all the truncate() methods. Note that the python _BytesIO implementation isn't covered by the test suite, as it's shadowed by the C implementation ; but imo we shouldn't care : I've manually tested it once, and anyway the trunk sources don't have this problem. -- Added file: http://bugs.python.org/file15765/python26_full_truncate_patch.patch ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Pascal Chambon added the comment: Whoups - I forgot to bugfix as well the _bytesio classes... let's just forget about the previous patch. -- ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Pascal Chambon added the comment: Hi Here is a patch for the python2.6 _fileio.c module, as well as the corresponding testcase. I'll check the _pyio and C _io trunk modules asap. -- keywords: +patch Added file: http://bugs.python.org/file15763/python26_io_truncate_bugfix.patch ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7640] buffered io seek() buggy
Pascal Chambon added the comment: Well, here is a patch for the seek() methods of io module, in python2.6 maintenance branch. Finally, I've only backported some assertions and the offset stuffs - I'm not comfortable enough about recent io refactorings to do more (it changed pretty quickly while I looked away, actually :p). Tests have been patched too (bufferedrandom wasn't tested as other buffer classes), and to have the whole suite pass, I had to modify testWriteNonBlocking() as well (mock objects returned wrong values - this test is marked as unreliable in sources anyway). If more is needed to patch the py2.6 branch, just tell me B-) -- keywords: +patch Added file: http://bugs.python.org/file15759/seek_cur_buffers_py26.patch ___ Python tracker <http://bugs.python.org/issue7640> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7640] buffered io seek() buggy
Pascal Chambon added the comment: Hum, with a selective merge (tortoiseSVN makes it easy), backporting _pyio should be doable in a decent time. Triaging pertinent tests should be more brain damaging :p I'm looking at this. -- ___ Python tracker <http://bugs.python.org/issue7640> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7640] buffered io seek() buggy
Pascal Chambon added the comment: My bad, I had looked at _buffered_raw_seek, not buffered_seek >_< Indeed, the code is OK in both trunk _io an _pyio modules. And the SEEK_CUR flag (value : 1) seems more than sufficiently tested in test_io actually, for example in the function write_ops() : http://svn.python.org/view/python/trunk/Lib/test/test_io.py?view=markup So concerning python2.6, isn't that just possible to backport _pyio (and its test suite) to it (renamed as io.py) ? They seem to offer the same functionality, except that _pyio is much more robust than io.py (the code around seek(), in particular, is much more complete). Just tell me, else I'll gather a patch for the seek() problem. -- ___ Python tracker <http://bugs.python.org/issue7640> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Pascal Chambon added the comment: Allright, I'll try to work on it as soon as I manage to gather a decent compilation environment on windows... -- ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7640] buffered io seek() buggy
New submission from Pascal Chambon : I've noticed a severe bug in my python 2.6.2 io module, and looking at _buffered_raw_seek in http://svn.python.org/view/python/trunk/Modules/_io/bufferedio.c?view=markup, it seems the bug is still there in the C reimplementation of buffered io classes. The problem is, if we seek a buffered stream with whence=os.SEEK_CUR, that call is directly transmitted to the underlying raw stream, which has not the same file pointer position as the buffered stream. So in the end, the file pointer doesn't get moved at the right absolute offset, but at an offset which depends of the state of the read head buffer or the write buffer. When using os.SEEK_CUR, it would be necessary to call tell() or to manually compute offsets, so that the seek() works as expected. Regards, Pascal -- components: IO messages: 97275 nosy: pakal severity: normal status: open title: buffered io seek() buggy type: behavior versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2 ___ Python tracker <http://bugs.python.org/issue7640> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Pascal Chambon added the comment: Hello I'm currently finalizing the API of my raw io file implementation, but I still have trouble with the trunk implementation of IOBase.truncate(). If I remember well, in the mailing list topic on this subject, GvR noted that this change of behaviour compared to python 2.x was not intended, and that it would be better to get back to the expected behaviour - not touching the file pointer - and to document the method in this way. Are there new elements, advocating a status quo on this matter ? Or shouldn't we add the portable_lseek() call in fileio.c to fix that ? On a separate note, I'm confused about the "at most" phrase in the current documentation : --- truncate(size=None) Truncate the file to at most size bytes. size defaults to the current file position, as returned by tell() --- According to what I've read so far, a succesful truncate() call will always extend/reduce the file until teh desired size, isn't that so on all platforms ? Regards Pascal -- ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7545] IO buffering behaviour not properly documented
Pascal Chambon added the comment: Yep, I knew "full buffering" didn't mean "fill my RAM until crash" :p The only visible problem was the interpretation of positive/negative buffering value, which wasn't the same in doc and in code. But the patch seems to fix up the plot prettily well B-) Thanks -- ___ Python tracker <http://bugs.python.org/issue7545> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7545] IO buffering behaviour not properly documented
New submission from Pascal Chambon : Hello, It seems there is an important difference between the doc of the IO module, and its implementation so far (until todcay trunk revision 76805) "buffering is an optional integer used to set the buffering policy. By default full buffering is on. Pass 0 to switch buffering off (only allowed in binary mode), 1 to set line buffering, and an integer > 1 for full buffering." --> actually full buffering only occurs if a negative buffering parameter is given, else it seems th current value i kept as the buffer size - eg. if we give "3", buffering will be 3 bytes... This seems a fine behaviour to me, so this implementation could just be documented as is. --- Only case of full buffering in the C iomodule : if (buffering < 0) { buffering = DEFAULT_BUFFER_SIZE; #ifdef HAVE_STRUCT_STAT_ST_BLKSIZE { struct stat st; long fileno; PyObject *res = PyObject_CallMethod(raw, "fileno", NULL); if (res == NULL) goto error; fileno = PyInt_AsLong(res); Py_DECREF(res); if (fileno == -1 && PyErr_Occurred()) goto error; if (fstat(fileno, &st) >= 0) buffering = st.st_blksize; } #endif } -- assignee: georg.brandl components: Documentation, IO messages: 96607 nosy: georg.brandl, pakal severity: normal status: open title: IO buffering behaviour not properly documented versions: Python 2.6, Python 2.7 ___ Python tracker <http://bugs.python.org/issue7545> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7022] Doc update for io module
Pascal Chambon added the comment: Thanks a lot B-) -- ___ Python tracker <http://bugs.python.org/issue7022> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7022] Doc update for io module
Pascal Chambon added the comment: Thanks for the memoryview tip - I though I was up-to-date with python's features but obviously I wans't ^^ (I searched through dict views but that wasn't it...) It should be exactly what's needed to replace these unnecessary additional arguments. As for the simple doc update, I was just pointing out the fact that EOF is reached if read methods returns an empty bytes AND we didn't ask for 0 bytes. Just to avoid confusing people : file.read(0) will always return an empty bytes, but it doesnt mean that we're at eof, contarrily to what the doc currently says B-) -- ___ Python tracker <http://bugs.python.org/issue7022> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7022] Doc update for io module
New submission from Pascal Chambon : *Propositions of doc update* *RawIOBase*.read(n: int) -> bytes Read up to n bytes from the object and return them. Fewer than n bytes may be returned if the operating system call returns fewer than n bytes. If 0 bytes are returned, and n was not 0, this indicates end of file. If the object is in non-blocking mode and no bytes are available, the call returns None. : *RawIOBase*.readinto(b: bytearray, [start: int = None], [end: int = None]) -> int start and end are used as slice indexes, so that the bytearray taken into account is actually range = b[start:end] (or b[start:], b[:end] or b[:], depending on the arguments which are not None). Read up to len(range) bytes from the object and store them in b, returning the number of bytes read. Like .read, fewer than len(range) bytes may be read, and 0 indicates end of file if len(range) is not 0. None is returned if a non-blocking object has no bytes available. The length of b is never changed. -- assignee: georg.brandl components: Documentation, IO messages: 93352 nosy: georg.brandl, pakal severity: normal status: open title: Doc update for io module type: feature request versions: Python 3.1, Python 3.2 ___ Python tracker <http://bugs.python.org/issue7022> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue4192] Subprocess error with I/O redirection to Pipes
Changes by Pascal Chambon : -- status: open -> closed ___ Python tracker <http://bugs.python.org/issue4192> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
Pascal Chambon added the comment: Well, I guess it deserve discussion on the pydev mailing lits, that's imo a rather important point, to be documented precisely. Concerning the padding, I guess the semantic doesn't change between the io module and the old file type, i.e : file.truncate([size])¶ Truncate the file’s size. If the optional size argument is present, the file is truncated to (at most) that size. The size defaults to the current position. The current file position is not changed. Note that if a specified size exceeds the file’s current size, the result is platform-dependent: possibilities include that the file may remain unchanged, increase to the specified size as if zero-filled, or increase to the specified size with undefined new content. Availability: Windows, many Unix variants. Having platform-dependent semantics for a so important type is anyway dangerous, I feel. The io module should be rock-solid, with very deeply documented semantics, and implementations that force platforms to conform to these specifications. Eg. I've noticed that all io.FileIO methods raised IOErrors in case of problem, except in one case (wrapping a bad file decsriptor) where it's OSError - that little thing might make big programs crash in an ugly way, if they only caught IOError. I'm currently working on a cross-platform (linux, mac, windows at least) FileIO type, with very strict semantics and extended abilities (file range shared and exclusive locking, with timeouts) ; even though it'll initially be slower than _fileio (most of it is in pure python), maybe it might solve most of these io module problems. I'll keep people informed. -- ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6939] shadows around the io truncate() semantics
New submission from Pascal Chambon : Hello I'm having trouble around the behaviour of the io module's truncate methods, in py3k. If I remember well, under unix and older versions of Python (with other file types), truncate never move the fiel pointer (and had to fake that behaviour under windows as well, by saving/restoring the file pointer). However, I see nothing in the documentation of the io module about that, and studying the _fileio.c code, it seems that now the file pointer is (under all OS) moved to the truncation point, and left there. Maybe it requires discusions on the mailing list, but personally I think that the common unix behaviour (letting the file pointer untouched) would be better for everyone, and that the doc should claim it. Also, additional notes about the behaviour of that truncation (reminding people that on some OS, the padding is zero-filled, on others it's not...) would be great. Current doc of io in py3k : http://docs.python.org/py3k/library/io.html#io.IOBase.truncate truncate(size=None) Truncate the file to at most size bytes. size defaults to the current file position, as returned by tell(). Cheers Pascal -- components: IO messages: 92822 nosy: pakal severity: normal status: open title: shadows around the io truncate() semantics type: behavior versions: Python 2.6, Python 2.7, Python 3.0, Python 3.1, Python 3.2 ___ Python tracker <http://bugs.python.org/issue6939> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6929] Confusion between "write" method of rowiobase and rawfileio
Pascal Chambon added the comment: I forgot to note - yep I was actually improperly looking at the python 2.6 documentation, which is erroneous ocncerning the io module. But the py3k doc seems to summarize it allright B-) -- ___ Python tracker <http://bugs.python.org/issue6929> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6929] Confusion between "write" method of rowiobase and rawfileio
Pascal Chambon added the comment: Allright, then only rawIoBase's documentation is wrong... and I'll have to modify my work-in-progress library to mimic FileIo more accurately. Thansk for teh info B-) -- ___ Python tracker <http://bugs.python.org/issue6929> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue6929] Confusion between "write" method of rowiobase and rawfileio
New submission from Pascal Chambon : It seems the properties of the write methods of these two classes are mixed up in documentation. I've checked the sources, and actually it seems the behviour is inverted : rawiobase streams, which can be pipes or other limited streams, may write less than len(b) bytes in one operation, and return ; whereas rawfileio instances, which write on disk, will never fail writing all the data unless the disk is full (resulting in IOError). It's rawiobase which does one system call, not rawiofile which will try again until all is written, isn't it ? RawIoBase write(b) Write the given bytes or bytearray object, b, to the underlying raw stream and return the number of bytes written (never less than len(b), since if the write fails an IOError will be raised). A BlockingIOError is raised if the buffer is full, and the underlying raw stream cannot accept more data at the moment. RawFileIO : write(b) Write the bytes or bytearray object, b, to the file, and return the number actually written. Only one system call is made, so it is possible that only some of the data is written. -- assignee: georg.brandl components: Documentation messages: 92751 nosy: georg.brandl, pakal severity: normal status: open title: Confusion between "write" method of rowiobase and rawfileio type: behavior versions: Python 3.1 ___ Python tracker <http://bugs.python.org/issue6929> ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com