[Python-Dev] Add calloc-like memory allocators to Python
Hi, We made progress on the following issue and the latest patch (calloc-5.patch) is ready for a review: http://bugs.python.org/issue21233 This issue should help numpy to reuse Python memory allocators to use the new tracemalloc module of Python 3.4. The issue is only for Python 3.5. It was also discussed that numpy might require a memory allocator with a specified alignment for SIMD instructions. This topic is discussed in a different issue: http://bugs.python.org/issue18835 Summary of calloc-5.patch: - add the following functions: * void* PyMem_RawCalloc(size_t nelem, size_t elsize) * void* PyMem_Calloc(size_t nelem, size_t elsize) * void* PyObject_Calloc(size_t nelem, size_t elsize) * PyObject* _PyObject_GC_Calloc(size_t basicsize) - add void* calloc(void *ctx, size_t nelem, size_t elsize) field to the PyMemAllocator structure - optimize bytes(n) and bytearray(n) to allocate objects using calloc() instead of malloc() - update tracemalloc to trace also calloc() - document new functions and add unit tests for the calloc hook (in _testcapi) Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Python 2.7.7. on Windows
Greetings, I've just woken up and noticed Python 2.7.7 is on track to be released, and in a rather unique event contains a few security enhancements in addition to the usual fixes: http://legacy.python.org/dev/peps/pep-0466/ I thought this might be a good time to make a final plea to fix a long-standing security issue in the installer on Windows. By default it installs Python to the root folder, thereby bypassing filesystem permissions: http://bugs.python.org/issue1284316 The main rationale given (for not using the standard %ProgramFiles%) has been that the full path to python is too long to type, and ease of use is more important than the security benefits given by following Windows conventions. However, adding python to the PATH variable is an alternative solution that would result in even fewer keystrokes needing to be typed at a console, while maintaining system security. As this is an installer setting and not a code change, I gather that the opportunities for code breakage should be fewer. Please consider updating this setting to a more secure and friendly default, for 2.7.7 and 3.5+ as well. -Mike ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] API and process questions (sparked by Claudiu Popa on 16104
(1) Should fixes to a docstring go in with a patch, even if they aren't related to the changing functionality? Bytestring compilation has several orthogonal parameters. Most -- but not all -- are documented in the docstring. (Specifically, there is no explanation of the rx parameter which acts as a filter, and no mention that symbolic links to directories are ignored.) It is best if a commit changes one small thing at a time. On the other hand, Nick recently posted that the minimal overhead of a patch commit is about half an hour. Is that overhead enough to override the one-issue-per-patch guideline? (2) The patch adds new functionality to use multiple processes in parallel. The normal parameter values are integers indicating how many processes to use. The parameter also needs two special values -- one to indicate use os.cpu_count, and the other to indicate don't use multiprocessing at all. (A) Is there a Best Practices for this situation, with two odd cases? (B) Claudiu originally copied the API from a similar APU for regrtest. What is the barrier for do it sensibly vs stick with precedent elsewhere? (Apparently regrtest treats any negative number as a request for the cpu_count calculation; I suspect that -5 is more likely to be an escaping error for 5 than it is to be a real request for auto-calculation that just happened to choose -5 instead of -1.) (C) How important is it to keep the API consistent between a top-level CLI command and the internal implementation? At the the moment, the request for cpu_count is handled in the CLI wrapper, and not available to interactive callers. On the other hand, interactive callers could just call cpu_count themselves... (D) How important is it to maintain consistency with other uses of the same tool -- multiprocessing has its own was of requesting auto-calculation. (So someone used to multiprocessing might assume that None meant to auto-calculate, as opposed to don't use multiprocessing at all.) -jJ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] API and process questions (sparked by Claudiu Popa on 16104
On Mon Apr 28 2014 at 11:32:58 AM, Jim J. Jewett jimjjew...@gmail.com wrote: (1) Should fixes to a docstring go in with a patch, even if they aren't related to the changing functionality? It should probably be its own commit. Bytestring compilation has several orthogonal parameters. Most -- but not all -- are documented in the docstring. (Specifically, there is no explanation of the rx parameter which acts as a filter, and no mention that symbolic links to directories are ignored.) It is best if a commit changes one small thing at a time. On the other hand, Nick recently posted that the minimal overhead of a patch commit is about half an hour. Is that overhead enough to override the one-issue-per-patch guideline? That's typical for backported patches, not if something is only going into default. That being said, if the change is truly small and you sneak it in I don't think anyone is going to make you revert the patch to do it as more atomic commits. (2) The patch adds new functionality to use multiple processes in parallel. The normal parameter values are integers indicating how many processes to use. The parameter also needs two special values -- one to indicate use os.cpu_count, and the other to indicate don't use multiprocessing at all. (A) Is there a Best Practices for this situation, with two odd cases? No. In this situation I would consider 0 or -1 for use os.cpu_count' and None for don't use multi-processing. (B) Claudiu originally copied the API from a similar APU for regrtest. What is the barrier for do it sensibly vs stick with precedent elsewhere? (Apparently regrtest treats any negative number as a request for the cpu_count calculation; I suspect that -5 is more likely to be an escaping error for 5 than it is to be a real request for auto-calculation that just happened to choose -5 instead of -1.) Regrtest doesn't count as an API to stick to. =) (C) How important is it to keep the API consistent between a top-level CLI command and the internal implementation? At the the moment, the request for cpu_count is handled in the CLI wrapper, and not available to interactive callers. On the other hand, interactive callers could just call cpu_count themselves... It doesn't hurt, but we typically don't promote CLIs for stdlib modules so there isn't much precedent. (D) How important is it to maintain consistency with other uses of the same tool -- multiprocessing has its own was of requesting auto-calculation. (So someone used to multiprocessing might assume that None meant to auto-calculate, as opposed to don't use multiprocessing at all.) That's a judgment call. If there is already no consistency go with the one that makes the most sense. If consistency exists across the stdlib then stay consistent. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104
On Mon, Apr 28, 2014 at 6:32 PM, Jim J. Jewett jimjjew...@gmail.com wrote: (1) Should fixes to a docstring go in with a patch, even if they aren't related to the changing functionality? Bytestring compilation has several orthogonal parameters. Most -- but not all -- are documented in the docstring. (Specifically, there is no explanation of the rx parameter which acts as a filter, and no mention that symbolic links to directories are ignored.) It is best if a commit changes one small thing at a time. On the other hand, Nick recently posted that the minimal overhead of a patch commit is about half an hour. Is that overhead enough to override the one-issue-per-patch guideline? (2) The patch adds new functionality to use multiple processes in parallel. The normal parameter values are integers indicating how many processes to use. The parameter also needs two special values -- one to indicate use os.cpu_count, and the other to indicate don't use multiprocessing at all. (A) Is there a Best Practices for this situation, with two odd cases? (B) Claudiu originally copied the API from a similar APU for regrtest. What is the barrier for do it sensibly vs stick with precedent elsewhere? (Apparently regrtest treats any negative number as a request for the cpu_count calculation; I suspect that -5 is more likely to be an escaping error for 5 than it is to be a real request for auto-calculation that just happened to choose -5 instead of -1.) (C) How important is it to keep the API consistent between a top-level CLI command and the internal implementation? At the the moment, the request for cpu_count is handled in the CLI wrapper, and not available to interactive callers. On the other hand, interactive callers could just call cpu_count themselves... (D) How important is it to maintain consistency with other uses of the same tool -- multiprocessing has its own was of requesting auto-calculation. (So someone used to multiprocessing might assume that None meant to auto-calculate, as opposed to don't use multiprocessing at all.) Thanks for writing this, Jim. Hopefully, the answers to these questions will solve our disagreements regarding the API. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7.7. on Windows
Mike Miller python-...@mgmiller.net wrote: The main rationale given (for not using the standard %ProgramFiles%) has been that the full path to python is too long to type, and ease of use is more important than the security benefits given by following Windows conventions. C:\Program Files\Python27 contains an empty space in the path. If you want to randomly break build tools for C extensions, then go ahead and change it. Sturla ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104
(2) The patch adds new functionality to use multiple processes in parallel. The normal parameter values are integers indicating how many processes to use. The parameter also needs two special values -- one to indicate use os.cpu_count, and the other to indicate don't use multiprocessing at all. (A) Is there a Best Practices for this situation, with two odd cases? No. In this situation I would consider 0 or -1 for use os.cpu_count' and None for don't use multi-processing. Why would the user care if multiprocessing is used behind the scene? It would be strange for processes=1 to fail if multiprocessing is not available. If you set a default value of 1, then compileall() will work regardless of whether multiprocessing is available. In short: processes = 0: use os.cpu_count() processes == 1 (default): just use normal sequential compiling processes 1: use multiprocessing There's no reason to introduce None. Or am I missing something? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7.7. on Windows
Mike Miller wrote: I thought this might be a good time to make a final plea to fix a long-standing security issue in the installer on Windows. By default it installs Python to the root folder, thereby bypassing filesystem permissions: http://bugs.python.org/issue1284316 This would be an incredibly painful change that would surprise and hurt a lot of people. Mark Hammond's last comment on the linked issue sums up my view as well: My take is still that Python is a tool, not an app. People writing an app they wish to distribute using Python should include Python in their package (ie, not rely on an installed version) and these apps should conform with the guidelines. Yes, it is possible for a non-admin user to install arbitrary packages into a place where an admin user may inadvertently run it, thereby providing escalation of privilege. On the other hand, that applies to a lot of development tools, especially since most users on Windows these days are actually limited administrators - ANYTHING they install could run when they elevate a certain process. I'm all for encouraging apps to redistribute Python (as most do on Windows). In this case, they can easily apply the correct security settings, typically by installing pythonxy.dll with their application and ignoring any global settings (PYTHONPATH, registry keys, etc.). On the other hand, Python from python.org is a tool that should only be installed by those who are prepared to manage it. On Windows it is easy enough to have a second, secured copy for your admin scripts and an unsecured copy for non-admin tasks. The main rationale given (for not using the standard %ProgramFiles%) has been that the full path to python is too long to type, and ease of use is more important than the security benefits given by following Windows conventions. However, adding python to the PATH variable is an alternative solution that would result in even fewer keystrokes needing to be typed at a console, while maintaining system security. As this is an installer setting and not a code change, I gather that the opportunities for code breakage should be fewer. Please consider updating this setting to a more secure and friendly default, for 2.7.7 and 3.5+ as well. I'm not sure what change you are proposing here... doesn't the installer already have an option to add to PATH? I'm sure I keep disabling it. I don't want python.exe on my PATH because I have 10+ versions installed at any one time. I have my own set of aliases because even the py.exe launcher can't handle my setup :) The info we've gotten from our users (Python Tools for Visual Studio) shows that most have two or more versions of Python installed, so having one or both of them in PATH or PATHEXT just adds ambiguity. The other rationales are .pyc generation and package installation. Unlike PATH, these are not red herrings, but they do only apply to developers and not end users (where the developer has done his/her job properly). Speaking as the one who is likely to take over from Martin as the Windows build manager (the only people arguing so far are my employer's lawyers, but I'm winning that argument), I don't see any change that can be made to 2.7.7 apart from adding a link in the installer to a documentation page on how to secure the Python installation. At this point, 2.7.6-2.7.7 should be an incredibly safe upgrade, and there's no way to safely change the default installation location or the ACLs on the install directory. As for 3.5, I have some ideas that I will raise with python-dev once I've had a chance to think through all the issues (think proper per-user installs, redist modules, etc.). More secure installations would certainly be on the table, but practicality very easily beats purity here IMHO. Cheers, Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104
And incidentally, I think that the argument *processes* should be renamed to *workers*, or *jobs* (like in make), and any mention of multiprocessing in the documentation should be removed (if any): multiprocessing is an implementation detail. When I type: make -jN I don't really care that make is using fork() ;-) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104
On Mon, Apr 28, 2014 at 12:56 PM, Charles-François Natali cf.nat...@gmail.com wrote: Why would the user care if multiprocessing is used behind the scene? Err ... that was another set of questions that I forgot to ask. (A) Why bother raising an error if multiprocessing is unavailable? After all, there is a perfectly fine fallback... On other hand, errors should not pass silently. If a user has explicitly asked for multiprocessing, there should be some notice that it didn't happen. And builds are presumably something that a developer will monitor to respond to the Exception. (A1) What sort of Error? I'm inclined to raise the original ImportError, but the patch prefers a ValueError. (B) Assuming the exception, I suppose your question adds a 3rd special case of Whatever the system suggests, and I don't care whether or not it involves multiprocessing. It would be strange for processes=1 to fail if multiprocessing is not available. As Claudiu pointed out, processes=1 should really mean 1 worker process, which is still different from do everything in the main process. I'm not sure that level of control is really worth the complexity, but I'm not certain it isn't. processes = 0: use os.cpu_count() I could understand doing that for 0 or -1; what is the purpose of doing it for both, let alone for -4? Are we at the point where the parameter should just take positive integers or one of a set of specified string values? -jJ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] file objects guarantees
Hi, What's meant exactly by a file object? Let me be more specific: for example, pickle.dump() accepts a file object. Looking at the code, it doesn't check the return value of its write() method. So it assumes that write() should always write the whole data (not partial write). Same thing for read, it assumes there won't be short reads. A sample use case would be passing a socket.makefile() to pickle: it works, because makefile() returns a BufferedReader/Writer which takes care of short read/write. But the documentation just says file object. And if you have a look the file object definition in the glossary: https://docs.python.org/3.5/glossary.html#term-file-object There are actually three categories of file objects: raw binary files, buffered binary files and text files. Their interfaces are defined in the io module. The canonical way to create a file object is by using the open() function. So someone passing e.g. a raw binary file - which doesn't handle short reads/writes - would run into trouble. It's the same thing for e.g. GzipFile, and probably many others. Would it make sense to add a note somewhere? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?
Hello, It's more or less known fact (ref: google) that you can't inherit from multiple generic builtin (as in coded in C) types: class C(dict, list): pass TypeError: multiple bases have instance lay-out conflict However, more detailed googling led me to this page of a book: http://books.google.com.ua/books?id=JnR9hQA3SncCpg=PA104lpg=PA104 , which suggest that there might be adhoc support for some native types to serve as multiple bases together, but it doesn't provide an example. The book is apparently focused on Python2. I tried to look at typeobject.c:best_base(), which throws the quoted error above, and the only special rules I could quickly decipher from it is that it's possible to multi-inherit from a class and its subclass. Intuitively, it can be understood - it makes no sense to do that on the same inheritance level, but can happen with recursive inheritance, and then there's no need for conflict - both classes can be merged into subclass. Indeed, following works: import _collections class Foo(_collections.defaultdict, dict): pass (opposite order gives MRO conflict) So, is that it, or disjoint native types are supported as bases somehow? Also, would someone know if a class-subclass case happens for example in stdlib? As the previous question, https://mail.python.org/pipermail/python-dev/2014-April/134342.html this one is to help set adequate requirements for implementing multiple inheritance in MicroPython. Thanks, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104
2014-04-28 18:29 GMT+01:00 Jim J. Jewett jimjjew...@gmail.com: On Mon, Apr 28, 2014 at 12:56 PM, Charles-François Natali cf.nat...@gmail.com wrote: Why would the user care if multiprocessing is used behind the scene? Err ... that was another set of questions that I forgot to ask. (A) Why bother raising an error if multiprocessing is unavailable? After all, there is a perfectly fine fallback... On other hand, errors should not pass silently. If a user has explicitly asked for multiprocessing, there should be some notice that it didn't happen. And builds are presumably something that a developer will monitor to respond to the Exception. The point I'm making is that he's not asking for multiprocessing, he's asking for parallel build. If you pass 1 (or keep the default value), there's no fallback involved: the code should simply proceed sequentially. (A1) What sort of Error? I'm inclined to raise the original ImportError, but the patch prefers a ValueError. NotImplementedError would maybe make sense. As Claudiu pointed out, processes=1 should really mean 1 worker process, which is still different from do everything in the main process. I'm not sure that level of control is really worth the complexity, but I'm not certain it isn't. I disagree. If you pass job =1 (and not processes = 1), then you don't care whether multiprocessing is available or not: you just do everything in your main process. It would be quite wasteful to create a single child process! processes = 0: use os.cpu_count() I could understand doing that for 0 or -1; what is the purpose of doing it for both, let alone for -4? Are we at the point where the parameter should just take positive integers or one of a set of specified string values? Honestly, as long as it accepts 0, I'm happy. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pep8 reasoning
On Sun, Apr 27, 2014 at 1:40 PM, Barry Warsaw ba...@python.org wrote: On Apr 27, 2014, at 12:34 PM, Chris Barker wrote: foo = long_function_name(var_one, var_two, var_three, var_four) Wow, do you really indent those 42 columns? No -- stupid variable-width font! I don't think anyone should write code with variable width fonts, and I'd rather not do email that way either, but gmail is making it tough these days.. Sorry for the added confusion. This actually is forbidden because you're not using vertical alignment when there is an argument on the first line. You would have to line up `var_two` right under `var_one` to be compliant. yeah -- that was the intent oh well. That is, I find that if the argument list is too long for one line, then splitting it out to only one argument per line is much more readable to me. Sure. The PEP outlines ways to do that. not really -- it allows it: # Aligned with opening delimiter. foo = long_function_name(var_one, var_two, var_three, var_four) but all the examples have more than one variable per line...my point is that I think that should be discouraged. i.e. I think the above should be: # Aligned with opening delimiter. foo = long_function_name(var_one, var_two, var_three, var_four) (done with fixed-width font this time -- but it may not look right in your mail reader..) though I doubt there would consensus on requiring that -- but many of us learn more from examples than the specification, so maybe I'll submit a patch with an example like that. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pep8 reasoning
On Sun, Apr 27, 2014 at 6:34 PM, Steven D'Aprano st...@pearwood.infowrote: I would agree with having at least one example done with one arg per line. Is it really necessary? I think that one-arg-per-line is an obvious variation of the existing example. not really -- a lot of folks learn more from following examples. If all teh examples are the same in some aspect, we'll tend to think that that's intentional, and that is at least the recommended method If the only example given was one-arg-per-line, then the reader might wrongly assume that *only* one arg was allowed. indeed -- but maybe less clear but the the fact that none of the examples show one argument per line, the implication is that that is discouraged. very had to document a widely variable standard! -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7.7. on Windows
On Mon, 28 Apr 2014 19:52:48 +1200 Mike Miller python-...@mgmiller.net wrote: I thought this might be a good time to make a final plea to fix a long-standing security issue in the installer on Windows. By default it installs Python to the root folder, thereby bypassing filesystem permissions: http://bugs.python.org/issue1284316 Regardless of whether this change this or not, we certainly cannot change it in a bugfix version. So, 3.5 at the earliest it would be. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] file objects guarantees
File objects historically have a pretty vague spec. E.g. it's not defined which methods are supported beyond read() and readline() (for readers) or write() (for writers). Short writes shouldn't be allowed (a regular file object's write() doesn't even return a value for this reason). This means that a raw (unbuffered) Short reads *should* be expected, since behavior around these varies widely. If you see code that doesn't expect them please file bugs. The glossary item doesn't provide much guidance for would-be implementers of compliant file file objects, and the interface defined in the io module is too large (e.g. nobody cares to implement readinto()). I think we should clarify that raw (unbuffered) file objects are not safe. I don't care about preventing this explicitly though -- when you see file object you should think duck typing and program accordingly. (Reading and writing are already distinct interfaces; ditto for text vs. bytes.) On Mon, Apr 28, 2014 at 10:54 AM, Charles-François Natali cf.nat...@gmail.com wrote: Hi, What's meant exactly by a file object? Let me be more specific: for example, pickle.dump() accepts a file object. Looking at the code, it doesn't check the return value of its write() method. So it assumes that write() should always write the whole data (not partial write). Same thing for read, it assumes there won't be short reads. A sample use case would be passing a socket.makefile() to pickle: it works, because makefile() returns a BufferedReader/Writer which takes care of short read/write. But the documentation just says file object. And if you have a look the file object definition in the glossary: https://docs.python.org/3.5/glossary.html#term-file-object There are actually three categories of file objects: raw binary files, buffered binary files and text files. Their interfaces are defined in the io module. The canonical way to create a file object is by using the open() function. So someone passing e.g. a raw binary file - which doesn't handle short reads/writes - would run into trouble. It's the same thing for e.g. GzipFile, and probably many others. Would it make sense to add a note somewhere? ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?
On Mon, 28 Apr 2014 20:45:48 +0300 Paul Sokolovsky pmis...@gmail.com wrote: So, is that it, or disjoint native types are supported as bases somehow? Also, would someone know if a class-subclass case happens for example in stdlib? Well, for instance this trivial example works: class C(list, object): pass ... Basically, if two classes have compatible layouts, you can inherit from both at once. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?
Antoine's example works because list inherits from object. The more general rule compatible layout only allows the rarest of cases to work -- basically the two classes involved must have a common base class and one of the classes must not add any C-level fields to that base class's layout. I would never count on this except in cases where this possibility is documented (e.g. when one class is designed to be a mix-in for the other). On Mon, Apr 28, 2014 at 11:24 AM, Antoine Pitrou solip...@pitrou.netwrote: On Mon, 28 Apr 2014 20:45:48 +0300 Paul Sokolovsky pmis...@gmail.com wrote: So, is that it, or disjoint native types are supported as bases somehow? Also, would someone know if a class-subclass case happens for example in stdlib? Well, for instance this trivial example works: class C(list, object): pass ... Basically, if two classes have compatible layouts, you can inherit from both at once. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104
In article CA+OGgf4weBk1NsXBSqi1g7tDE-=7rfkzd5bzn1mxihwsgge...@mail.gmail.com, Jim J. Jewett jimjjew...@gmail.com wrote: As Claudiu pointed out, processes=1 should really mean 1 worker process, which is still different from do everything in the main process. I'm not sure that level of control is really worth the complexity, but I'm not certain it isn't. For regrtest, there is an important difference between do everything in the main process and do one test at a time in a subprocess, namely the inadvertent global side-effects that running some tests have on other tests. The latter option is much safer and gives more reproducible test results. For compileall, I don't think there is a big difference between the two cases such that the regrtest semantics need to be followed. -- Ned Deily, n...@acm.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?
Hello, On Mon, 28 Apr 2014 20:24:58 +0200 Antoine Pitrou solip...@pitrou.net wrote: On Mon, 28 Apr 2014 20:45:48 +0300 Paul Sokolovsky pmis...@gmail.com wrote: So, is that it, or disjoint native types are supported as bases somehow? Also, would someone know if a class-subclass case happens for example in stdlib? Well, for instance this trivial example works: class C(list, object): pass ... Well, it's easy to treat object class as a special-case, null class. So, let's re-formulate questions above with where such native base classes are not 'object'. Basically, if two classes have compatible layouts, you can inherit from both at once. How is compatible layout defined? Or layout for that matter at all? Regards Antoine. -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?
Hi Paul, On Mon, 28 Apr 2014 21:42:02 +0300 Paul Sokolovsky pmis...@gmail.com wrote: Basically, if two classes have compatible layouts, you can inherit from both at once. How is compatible layout defined? Or layout for that matter at all? See Guido's answer. I don't think it's documented anywhere, but you can find the relevant code somewhere in Objects/typeobject.c (it's quite a mouthful, though :-)). (IIRC, layout is determined by tp_basicsize, tp_itemsize, the number of __slots__, and other things perhaps) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pep8 reasoning
On Apr 28, 2014, at 11:12 AM, Chris Barker wrote: No -- stupid variable-width font! I don't think anyone should write code with variable width fonts, and I'd rather not do email that way either, but gmail is making it tough these days.. Ouch. I'm sure it's gmail being helpful the way http://wiki.list.org/x/2IA9 is similarly helpful. Sure. The PEP outlines ways to do that. not really -- it allows it: # Aligned with opening delimiter. foo = long_function_name(var_one, var_two, var_three, var_four) but all the examples have more than one variable per line...my point is that I think that should be discouraged. i.e. I think the above should be: # Aligned with opening delimiter. foo = long_function_name(var_one, var_two, var_three, var_four) (done with fixed-width font this time -- but it may not look right in your mail reader..) Fortunately, my mail stack is sane. :) though I doubt there would consensus on requiring that -- but many of us learn more from examples than the specification, so maybe I'll submit a patch with an example like that. I also very much doubt you'd get consensus on that as a requirement, and I would oppose the PEP taking a stand one way or the other. I'm not sure that the PEP needs an example to illustrate its acceptability. Cheers, -Barry signature.asc Description: PGP signature ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?
On Mon, Apr 28, 2014 at 11:42 AM, Paul Sokolovsky pmis...@gmail.com wrote: Well, it's easy to treat object class as a special-case, null class. But the implementation doesn't, at least not for the question you're asking. So, let's re-formulate questions above with where such native base classes are not 'object'. Basically, if two classes have compatible layouts, you can inherit from both at once. How is compatible layout defined? Or layout for that matter at all? The layout is what the C struct defining the object looks like. These are typically defined in headers in the Include directory (e.g. listobject.h). The definition of compatible layout is defined by the C code that gives the error message when it's incompatible. Sorry, I don't recall exactly where that is, so I recommend that you just look at CPython's source tree. (I know you have a personal desire not to look at CPython, but I can't help you without referring to it anyway.) -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?
On Mon, Apr 28, 2014 at 12:02 PM, Antoine Pitrou solip...@pitrou.netwrote: On Mon, 28 Apr 2014 21:42:02 +0300 Paul Sokolovsky pmis...@gmail.com wrote: Basically, if two classes have compatible layouts, you can inherit from both at once. How is compatible layout defined? Or layout for that matter at all? See Guido's answer. I don't think it's documented anywhere, but you can find the relevant code somewhere in Objects/typeobject.c (it's quite a mouthful, though :-)). (IIRC, layout is determined by tp_basicsize, tp_itemsize, the number of __slots__, and other things perhaps) IIRC the actual inheritance pattern also goes into it. Two structs that each add an identical new field to a common base class's struct should *not* be considered compatible. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Commit-ready patches needing review
On Sun, 27 Apr 2014 12:10:46 -0700 Nikolaus Rath nikol...@rath.org wrote: * http://bugs.python.org/issue21057 (TextIOWrapper does not support reading bytearrays or memoryviews) I've reviewed this one. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7.7. on Windows
On 04/29/2014 05:12 AM, Steve Dower wrote: This would be an incredibly painful change that would surprise and hurt a lot of people. Hi, I think incredibly painful is overstating the case a bit. ;) We're talking about an installer default, a setting that would still be changeable as it always has, that by definition only will affect brand new installs. Yes, it is possible for a non-admin user to install arbitrary packages into a place where an admin user may inadvertently run it, thereby providing escalation of privilege. On the other hand, that applies to a lot of development tools, especially since most users on Windows these days are actually limited administrators - ANYTHING they install could run when they elevate a certain process. None of Microsoft's Dev tools install to C:\, rather to ProgramFiles. The fact that another security issue may exist is not a good justification for creating additional. On the other hand, Python from python.org is a tool that should only be installed by those who are prepared to manage it. On Windows it is easy enough to have a second, secured copy for your admin scripts and an unsecured copy for non-admin tasks. This sounds like the perspective of someone highly technical, forgetting that new users will be installing python as well and vastly outnumber us. Normal people need help to avoid security issues. Microsoft's guidelines on where to install software are clear, and don't make exceptions that tools should be installed to the root of the drive to bypass file system permissions, for convenience. I'm not sure what change you are proposing here... doesn't the installer already have an option to add to PATH? I'm sure I keep disabling it. No, it does not. Unless it got slipped in when I wasn't looking. It should be an option though, I think everyone would agree. python.exe on my PATH because I have 10+ versions installed at any one time. I Remember, python-dev's are not the target users of this package, and are a rather minuscule fraction of the user base. Python installation. At this point, 2.7.6-2.7.7 should be an incredibly safe upgrade, and there's no way to safely change the default installation location This would continue to be the case, as the installer will recognize the previously installed Python 2.7 and use its setting. This should affect only *brand new installs.* or the ACLs on the install directory. No ACLs would be affected or changed or even thought about. Simply installing to the correct folder (on new installs) will accomplish the goal. In short, this design of restricted permissions (read-only) for binaries and PATH conveniences goes back decades under Unix and other OS's. MS Windows has finally caught up in the security department in the last few releases. Please don't keep us back in the bad old days of DOS where everything was installed to the root folder. -- -Mike ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104
On Mon, Apr 28, 2014 at 6:32 PM, Jim J. Jewett jimjjew...@gmail.com wrote: (1) Should fixes to a docstring go in with a patch, even if they aren't related to the changing functionality? [...] It is best if a commit changes one small thing at a time. On the other hand, Nick recently posted that the minimal overhead of a patch commit is about half an hour. It could be much less. As an example, http://bugs.python.org/issue19943 has been closed 9 minutes after the report, it didn't have a patch and the fix had to be applied/grafted/merged on 3 branches. The time passed between the first commit and the push is less than a minute too. Clearly the time increases if you have to run the test suite or if there is a merge conflict/push race, and further decreases if there is a simple typo-fix on default only and a patch already available. Is that overhead enough to override the one-issue-per-patch guideline? That said, if the main fix should go only on default and it has a smaller unrelated fix that should also go on default only, then doing it together is generally OK (for some value of unrelated -- the fix should still be small and near the code you touched for the main fix). If the unrelated fix needs to be ported on all the branches (or in general in branches where the main fix shouldn't go), it's better to have two separate patches. If you commit the smaller fix together with the bigger one, and then decide to backport it, you will have to do a null merge and it gets slightly more complicated. Best Regards, Ezio Melotti ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104
This issue raised too much bikeshedding. To wrap it up, I'll modify the patch with the following: - processes renamed to workers - `workers` defaults to 1 - When `workers` is equal to 0, then `os.cpu_count` will be used - When `workers` 1, multiple processes will be used - When `workers` == 1, run normally (no multiple processes) - Negative values really should raise a ValueError (as multiprocessing.Pool and soon concurrent.futures.Thread/ProcessPoolExecutor) - Will raise NotImplementedError if multiprocessing can't be used (when `workers` equals to 0 or 1) If anyone agrees with the above, then I'll modify the patch. This will be its last iteration, any other bikeshedding should be addressed by the core dev who'll apply it. Thank you. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7.7. on Windows
On Mon, Apr 28, 2014 at 3:07 PM, Mike Miller python-...@mgmiller.net wrote: On 04/29/2014 05:12 AM, Steve Dower wrote: This would be an incredibly painful change that would surprise and hurt a lot of people. Hi, I think incredibly painful is overstating the case a bit. ;) We're talking about an installer default, a setting that would still be changeable as it always has, that by definition only will affect brand new installs. Yes, it is possible for a non-admin user to install arbitrary packages into a place where an admin user may inadvertently run it, thereby providing escalation of privilege. On the other hand, that applies to a lot of development tools, especially since most users on Windows these days are actually limited administrators - ANYTHING they install could run when they elevate a certain process. None of Microsoft's Dev tools install to C:\, rather to ProgramFiles. The fact that another security issue may exist is not a good justification for creating additional. On the other hand, Python from python.org is a tool that should only be installed by those who are prepared to manage it. On Windows it is easy enough to have a second, secured copy for your admin scripts and an unsecured copy for non-admin tasks. This sounds like the perspective of someone highly technical, forgetting that new users will be installing python as well and vastly outnumber us. Normal people need help to avoid security issues. Microsoft's guidelines on where to install software are clear, and don't make exceptions that tools should be installed to the root of the drive to bypass file system permissions, for convenience. I'm not sure what change you are proposing here... doesn't the installer already have an option to add to PATH? I'm sure I keep disabling it. No, it does not. Unless it got slipped in when I wasn't looking. It should be an option though, I think everyone would agree. The option to add the current install to your path was added 3.3. python.exe on my PATH because I have 10+ versions installed at any one time. I Remember, python-dev's are not the target users of this package, and are a rather minuscule fraction of the user base. Knowing which Python you want on your path and that you want it on your path at all is somewhat of an advanced usage. While beginners do want to just open up cmd and type python and have it work, there are better ways to accomplish that which don't involve system-wide path manipulation or installation changes. Several PC manufacturers have been known to use Python for various system utilities, which is how Python sometimes ends up in the path on your grandma's Dell*. Even for a beginner who just wants python to work, we need to be careful to not wreck their entire system by helping them get our fresh Python install to show up. A more reasonable way to help beginners would be to create a shortcut somewhere that starts up cmd with a modified path. They can do whatever they want to do within that context without modifying their entire system. If they learn that they later want their system-wide path manipulated, they can do that within the installer or will known how to do that themselves. * watch Dave Beazley's PyCon 2014 talk for a good story involving one of those manufacturer installed Pythons: https://www.youtube.com/watch?v=RZ4Sn-Y7AP8 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7.7. on Windows
Hi, Those bugs were fixed 9 years ago or so around 2005. I use python in ProgramFiles every day without issue. If another bug has crept in somewhere, it can be fixed. -Mike -- From: Sturla Molden sturla.mol...@gmail.com To: python-dev@python.org Subject: Re: [Python-Dev] Python 2.7.7. on Windows Message-ID: 771463774420395726.352685sturla.molden-gmail@news.gmane.org C:\Program Files\Python27 contains an empty space in the path. If you want to randomly break build tools for C extensions, then go ahead and change it. Sturla ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-Dev Digest, Vol 129, Issue 81
Hi, note the pep, it makes allowances for security enhancements. -Mike Message: 5 Date: Mon, 28 Apr 2014 20:23:12 +0200 From: Antoine Pitrou solip...@pitrou.net To: python-dev@python.org Subject: Re: [Python-Dev] Python 2.7.7. on Windows Message-ID: 20140428202312.6903d62a@fsol Regardless of whether this change this or not, we certainly cannot change it in a bugfix version. So, 3.5 at the earliest it would be. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7.7. on Windows
On 04/29/2014 08:38 AM, Brian Curtin wrote: The option to add the current install to your path was added 3.3. Ok, thanks. So there is some precedent it would be useful. Remember, python-dev's are not the target users of this package, and are a rather minuscule fraction of the user base. Knowing which Python you want on your path and that you want it on your path at all is somewhat of an advanced usage. ... They can do whatever they want to do within that context without modifying their entire system. If they learn that they later want their system-wide path manipulated... Installing python at all is a system-wide change. Why not default to a known safe, ready to use configuration, rather than a tool-box that needs assembly? * watch Dave Beazley's PyCon 2014 talk for a good story involving one of those manufacturer installed Pythons: https://www.youtube.com/watch?v=RZ4Sn-Y7AP8 Thanks, I'm trying to get thru all the talk will watch that shortly. ;) Still, edge cases regarding manufactures should not override the needs of the majority of users. -Mike ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-Dev Digest, Vol 129, Issue 81
On Mon Apr 28 2014 at 4:58:35 PM, Mike Miller python-...@mgmiller.net wrote: Hi, note the pep, it makes allowances for security enhancements. The PEP in question is about fixing fundamentally broken security issues in Python 2.7 (e.g. updating OpenSSL). Tweaking where Python is installed by default on Windows is not fundamentally broken, it's a difference of opinion as to whether the current defaults are the best option or not. -Brett -Mike Message: 5 Date: Mon, 28 Apr 2014 20:23:12 +0200 From: Antoine Pitrou solip...@pitrou.net To: python-dev@python.org Subject: Re: [Python-Dev] Python 2.7.7. on Windows Message-ID: 20140428202312.6903d62a@fsol Regardless of whether this change this or not, we certainly cannot change it in a bugfix version. So, 3.5 at the earliest it would be. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ brett%40python.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104
On 4/28/2014 4:24 PM, Claudiu Popa wrote: This issue raised too much bikeshedding. To wrap it up, I'll modify the patch with the following: - processes renamed to workers - `workers` defaults to 1 - When `workers` is equal to 0, then `os.cpu_count` will be used - When `workers` 1, multiple processes will be used - When `workers` == 1, run normally (no multiple processes) I presume you mean for this to be the default. - Negative values really should raise a ValueError (as multiprocessing.Pool and soon concurrent.futures.Thread/ProcessPoolExecutor) Yay! - Will raise NotImplementedError if multiprocessing can't be used (when `workers` equals to 0 or 1) If anyone agrees with the above, then I'll modify the patch. Having read the thread, though not much of the issue, these look to me like the best choices for a clean, comprehensible API. This will be its last iteration, any other bikeshedding should be addressed by the core dev who'll apply it. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pep8 reasoning
On 4/28/2014 2:12 PM, Chris Barker wrote: I don't think anyone should write code with variable width fonts, The problem is that fixed pitch does not work well for even a half-way complete unicode font and I don't know that there are any available. As far as I know, my Windows 7 only came with one unicode font that Idle finds. An advantage from my viewpoint is that spaces in this font are narrower that the average character, making multiple-of-4 indents and spaces around operators look pretty good to me. Given that PEP8 generally disallows adding spaces to make things line up, as in the following (a style I used to adhere to) a = 134543344 # big number bonge = 2 # short number it seems to me consistent to use the multiple-of-4 hanging indent style and not try to make things line up with an odd number of spaces in this one special case. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python-Dev Digest, Vol 129, Issue 81
On 4/28/2014 5:01 PM, Brett Cannon wrote: On Mon Apr 28 2014 at 4:58:35 PM, Mike Miller python-...@mgmiller.net mailto:python-...@mgmiller.net wrote: Hi, note the pep, it makes allowances for security enhancements. The PEP in question is about fixing fundamentally broken security issues in Python 2.7 (e.g. updating OpenSSL). Tweaking where Python is installed by default on Windows is not fundamentally broken, it's a difference of opinion as to whether the current defaults are the best option or not. Besides which, the PEP was cut down to a specific list of changes. Any more would need a separate dicussion. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104
On Mon, 28 Apr 2014 23:24:16 +0300, Claudiu Popa pcmantic...@gmail.com wrote: - Will raise NotImplementedError if multiprocessing can't be used (when `workers` equals to 0 or 1) I think the most common use case for this ability will be run with the appropriate number of processes for the system I'm on, where 'the appropriate number' is 1 (the main process) if multiprocessing is not available. Otherwise the tool calling compileall would have to figure out how to catch the error (how do you do that when invoking a CLI?) and re-run the script using '1` itself. How you spell this I don't really care, but I think the above is the most common use case. --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pep8 reasoning
On Mon, 28 Apr 2014 18:01:32 -0400, Terry Reedy tjre...@udel.edu wrote: On 4/28/2014 2:12 PM, Chris Barker wrote: I don't think anyone should write code with variable width fonts, The problem is that fixed pitch does not work well for even a half-way complete unicode font and I don't know that there are any available. As far as I know, my Windows 7 only came with one unicode font that Idle finds. An advantage from my viewpoint is that spaces in this font are narrower that the average character, making multiple-of-4 indents and spaces around operators look pretty good to me. My unix fix-width terminal font handles most unicode (even a lot of non-bmp stuff...though I have no idea if it is readable :). I have no idea what font it is, to tell you the truth. --David ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pep8 reasoning
On Mon, Apr 28, 2014 at 3:01 PM, Terry Reedy tjre...@udel.edu wrote: I don't think anyone should write code with variable width fonts, The problem is that fixed pitch does not work well for even a half-way complete unicode font and I don't know that there are any available. ... Given that PEP8 generally disallows adding spaces to make things line up, as in the following (a style I used to adhere to) a = 134543344 # big number bonge = 2 # short number Is ironic that that looks like hell in my variable-width font email client? it seems to me consistent to use the multiple-of-4 hanging indent style and not try to make things line up with an odd number of spaces in this one special case. things aren't going to generally line up with a non-fixed width font no matter how you slice it, so I guess if you need to use one for your code, you're just stuck. But I suppose the line up with the opening bracket style isn't a good idea in that case. All the more reason not to standardize anymore that PEP 8 already does. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pep8 reasoning
On 4/28/2014 7:13 PM, Chris Barker wrote: On Mon, Apr 28, 2014 at 3:01 PM, Terry Reedy tjre...@udel.edu mailto:tjre...@udel.edu wrote: I don't think anyone should write code with variable width fonts, The problem is that fixed pitch does not work well for even a half-way complete unicode font and I don't know that there are any available. ... Given that PEP8 generally disallows adding spaces to make things line up, as in the following (a style I used to adhere to) a = 134543344 # big number bonge = 2 # short number Is ironic that that looks like hell in my variable-width font email client? I believe that has been given as a reason to not do what I used to do. The only things guaranteed to actually line up are the initial graphic characters of lines with the exact same indent. tjr ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7.7. on Windows
On Mon, Apr 28, 2014 at 1:56 PM, Mike Miller python-...@mgmiller.netwrote: * watch Dave Beazley's PyCon 2014 talk for a good story involving one of those manufacturer installed Pythons: https://www.youtube.com/watch?v=RZ4Sn-Y7AP8 Thanks, I'm trying to get thru all the talk will watch that shortly. ;) fun talk -- but start at about 18:00 minutes if you want the relevant bit... By the way, some people (or at least ESRI, with ArcMap) Install and use their own python in: C:\python27 -- if you dont want to interat with that, you need to put it somewhere else! I'd argue that any OEM or third party app that uses the standard location or puts it on the PATH be default is in error, but what can you so? Redhat used to have #!/usr/bin/env python on their admin scripts, which was really painful if you wanted to upgrade anything. Don't know it they've fixed that -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/ORR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?
Hello, On Mon, 28 Apr 2014 12:08:40 -0700 Guido van Rossum gu...@python.org wrote: [] How is compatible layout defined? Or layout for that matter at all? The layout is what the C struct defining the object looks like. These are typically defined in headers in the Include directory (e.g. listobject.h). The definition of compatible layout is defined by the C code that gives the error message when it's incompatible. Sorry, I don't recall exactly where that is, so I recommend that you just look at CPython's source tree. (I know you have a personal desire not to look at CPython, but I can't help you without referring to it anyway.) Well, sure I did, as I mentioned, but as that's first time I see that code (that specific piece is in typeobject.c:extra_ivars()), it would take quite some time be sure I understand all aspects of it. Thanks for confirming that it's governed essentially by CPython implementation details and not some language-level semantics like metaclasses (I mentioned them because error message in Python2 did so, though Python3 doesn't refer to metaclasses). An example would really help me to get a feel of the issue, but I assume lack of them means that there's no well-known idiom where such inheritance is used, and that's good enough on its own. I also tried to figure how it's important to support such multi-base cases, so the code I write didn't require complete rewrite if it hits one day, but everything seems to turn out to be pretty extensible. -- --Guido van Rossum (python.org/~guido) Thanks! -- Best regards, Paul mailto:pmis...@gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7.7. on Windows
Mike Miller wrote: On 04/29/2014 05:12 AM, Steve Dower wrote: This would be an incredibly painful change that would surprise and hurt a lot of people. Hi, I think incredibly painful is overstating the case a bit. ;) We're talking about an installer default, a setting that would still be changeable as it always has, that by definition only will affect brand new installs. Good point about it only affecting new installs, though that still constitutes a lot of installs. Yes, it is possible for a non-admin user to install arbitrary packages into a place where an admin user may inadvertently run it, thereby providing escalation of privilege. On the other hand, that applies to a lot of development tools, especially since most users on Windows these days are actually limited administrators - ANYTHING they install could run when they elevate a certain process. None of Microsoft's Dev tools install to C:\, rather to ProgramFiles. The fact that another security issue may exist is not a good justification for creating additional. The fact that the mitigations are well known by the people who have to worry about them is a good justification for not creating a compatibility issue. It's easy for IT admins to install Python in a way that the files are read-only, the .pyc and .pyo files are already there, and user site packages will be used by default (I think that last one is easy?). The good IT admins even know that they need to do this - perhaps we can help educate the bad admins? (FWIW, if you have admin privileges on your own machine, YOU are the IT admin. Are you a good one or a bad one?) On the other hand, Python from python.org is a tool that should only be installed by those who are prepared to manage it. On Windows it is easy enough to have a second, secured copy for your admin scripts and an unsecured copy for non-admin tasks. This sounds like the perspective of someone highly technical, forgetting that new users will be installing python as well and vastly outnumber us. Normal people need help to avoid security issues. One place where mitigations are added are the distributions (Canopy, Anaconda, etc.). These packages redistribute Python and install to different locations with different permissions (I'm not promising that they all do it properly, but they have the opportunity to do so). The reference implementation of CPython is typically not the best option for normal people, who are much better served by one of these bundles. Microsoft's guidelines on where to install software are clear, and don't make exceptions that tools should be installed to the root of the drive to bypass file system permissions, for convenience. I'm well aware of the guidelines, hence the practicality vs. purity comment. I'm fairly certain that the installation to the root was originally about ease of command-line access rather than bypassing permissions - the permissions probably didn't exist for the first few versions of Python (Python for DOS obviously didn't care... maybe it's always been about backwards compatibility?) At this point, installing into the root is about not breaking everyone who *knows* that Python installs into the root directory and always has. I'm not sure what change you are proposing here... doesn't the installer already have an option to add to PATH? I'm sure I keep disabling it. No, it does not. Unless it got slipped in when I wasn't looking. It should be an option though, I think everyone would agree. Thanks Brett for pointing out that it arrived in Python 3. I'm sure it would be an acceptable addition to 2.7.7, though you'd need to get it in before RC and you'd also need to find someone who is keen and able to keep making 2.7 installers for Windows. Right now, we don't have anyone who is both. python.exe on my PATH because I have 10+ versions installed at any one time. Remember, python-dev's are not the target users of this package, and are a rather minuscule fraction of the user base. My day job revolves around making Python easier to use for non python-devs, especially those who have multiple versions of Python installed simultaneously (as I mentioned, over 50% of our user base). Having PATH or PATHEXT set globally is a source of much confusion. (Though a global PYTHONPATH causes more problems, but that isn't part of this discussion.) Python installation. At this point, 2.7.6-2.7.7 should be an incredibly safe upgrade, and there's no way to safely change the default installation location This would continue to be the case, as the installer will recognize the previously installed Python 2.7 and use its setting. This should affect only *brand new installs.* For example, next time you redeploy your hosted service on a clean VM with the latest security patches? The next time someone does a clean uninstall-reinstall because they've been burned by upgrades before? The next time someone arrives at work and IT
Re: [Python-Dev] Python 2.7.7. on Windows
On Mon, Apr 28, 2014 at 7:04 PM, Steve Dower steve.do...@microsoft.com wrote: Mike Miller wrote: On 04/29/2014 05:12 AM, Steve Dower wrote: This would be an incredibly painful change that would surprise and hurt a lot of people. Hi, I think incredibly painful is overstating the case a bit. ;) We're talking about an installer default, a setting that would still be changeable as it always has, that by definition only will affect brand new installs. Good point about it only affecting new installs, though that still constitutes a lot of installs. Yes, it is possible for a non-admin user to install arbitrary packages into a place where an admin user may inadvertently run it, thereby providing escalation of privilege. On the other hand, that applies to a lot of development tools, especially since most users on Windows these days are actually limited administrators - ANYTHING they install could run when they elevate a certain process. None of Microsoft's Dev tools install to C:\, rather to ProgramFiles. The fact that another security issue may exist is not a good justification for creating additional. The fact that the mitigations are well known by the people who have to worry about them is a good justification for not creating a compatibility issue. It's easy for IT admins to install Python in a way that the files are read-only, the .pyc and .pyo files are already there, and user site packages will be used by default (I think that last one is easy?). The good IT admins even know that they need to do this - perhaps we can help educate the bad admins? (FWIW, if you have admin privileges on your own machine, YOU are the IT admin. Are you a good one or a bad one?) On the other hand, Python from python.org is a tool that should only be installed by those who are prepared to manage it. On Windows it is easy enough to have a second, secured copy for your admin scripts and an unsecured copy for non-admin tasks. This sounds like the perspective of someone highly technical, forgetting that new users will be installing python as well and vastly outnumber us. Normal people need help to avoid security issues. One place where mitigations are added are the distributions (Canopy, Anaconda, etc.). These packages redistribute Python and install to different locations with different permissions (I'm not promising that they all do it properly, but they have the opportunity to do so). The reference implementation of CPython is typically not the best option for normal people, who are much better served by one of these bundles. Microsoft's guidelines on where to install software are clear, and don't make exceptions that tools should be installed to the root of the drive to bypass file system permissions, for convenience. I'm well aware of the guidelines, hence the practicality vs. purity comment. I'm fairly certain that the installation to the root was originally about ease of command-line access rather than bypassing permissions - the permissions probably didn't exist for the first few versions of Python (Python for DOS obviously didn't care... maybe it's always been about backwards compatibility?) At this point, installing into the root is about not breaking everyone who *knows* that Python installs into the root directory and always has. I'm not sure what change you are proposing here... doesn't the installer already have an option to add to PATH? I'm sure I keep disabling it. No, it does not. Unless it got slipped in when I wasn't looking. It should be an option though, I think everyone would agree. Thanks Brett for pointing out that it arrived in Python 3. I'm sure it would be an acceptable addition to 2.7.7, though you'd need to get it in before RC and you'd also need to find someone who is keen and able to keep making 2.7 installers for Windows. Right now, we don't have anyone who is both. If it's an acceptable change to the release manager (Benjamin?), and if there's actually time before the RC (I don't know when it is planned), I am willing to backport my 3.3 change to get this in the 2.7 installer. However, I'm not currently setup to make release installers -- I think I need a signing certificate or something like that. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7.7. on Windows
On Mon, Apr 28, 2014, at 17:14, Brian Curtin wrote: If it's an acceptable change to the release manager (Benjamin?), and if there's actually time before the RC (I don't know when it is planned), I am willing to backport my 3.3 change to get this in the 2.7 installer. That's fine. However, I'm not currently setup to make release installers -- I think I need a signing certificate or something like that. Apparently the current PSF one just expired, so we're in the process of procuring a new one. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pep8 reasoning
R. David Murray writes: My unix fix-width terminal font handles most unicode (even a lot of non-bmp stuff...though I have no idea if it is readable :). Oh, I bet you do. With a true fixed-width Unicode font, it's the *Latin-character* text that's painful, if not unreadable, because the aspect ratio needs to be close to 1.0 to handle ideographs, rather than the 0.5 or less common with Latin fonts. XTerm has an option to treat ideographs as double width. That does the trick (at least, I don't know any scripts that really really want an aspect ratio of 0.7 or 2.0). ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] pep8 reasoning
On Mon, Apr 28, 2014 at 11:12 AM, Chris Barker wrote: not really -- it allows it: # Aligned with opening delimiter. foo = long_function_name(var_one, var_two, var_three, var_four) but all the examples have more than one variable per line...my point is that I think that should be discouraged. i.e. I think the above should be: # Aligned with opening delimiter. foo = long_function_name(var_one, var_two, var_three, var_four) I don't have a problem with discouraging it, but it should not be flagged in pep8 (OK in pep8 --wink-wink-nudge-nudge-say-no-more-eh, though!) I usually do use the style you prefer, especially when the arguments want to be commented, but in many cases the arguments are semantically tuples. Eg, rect1 = paint_rectangle(point[1], point[0],# top, left textheight + 2 * padding, textwidth + 2 * padding, chartreuse) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7.7. on Windows
Mike Miller writes: Microsoft's guidelines on where to install software are clear, and don't make exceptions that tools should be installed to the root of the drive to bypass file system permissions, for convenience. But there's the rub. In this case, Microsoft doesn't have *security*, it has guidelines. They are *still* guidelines, not security, *exactly* because it's convenient for somebody. The fact that taking advantage of that convenience has the side effect of bypassing filesystem permissions is unfortunate (and a bug in Windows IMO). Note that if users actually paid attention to these guidelines, we'd be getting complaints from *them*, not from you. I don't recall ever seeing that. That implies that normal users will install anything anwhere anyway. If it's that unimportant to Microsoft, I see insufficient reason why we should risk confusing those normal users who already have Python 2.7 installed (and as pointed out, they *are* at risk precisely because the proposal changes the default install location). ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] bytes with trailing \0?
Hello,
I was surprised to find the following in bytesobject.c:
,
| [...]
|As always, an extra byte is allocated for a trailing \0 byte (newsize
|does *not* include that), and a trailing \0 byte is stored.
| */
|
| int
| _PyBytes_Resize(PyObject **pv, Py_ssize_t newsize)
| {
| [...]
|
`
Does this mean that bytes objects are internally stored with a trailing
\0? Why is that? Isn't that just wasting a byte, because \0 might also
be in the middle of the byte sequence, and the bytes objects stores its
length explicitly anyway?
Best,
-Nikolaus
--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7.7. on Windows
On Tue, Apr 29, 2014 at 12:07:00PM +0900, Stephen J. Turnbull wrote: Mike Miller writes: Microsoft's guidelines on where to install software are clear, and don't make exceptions that tools should be installed to the root of the drive to bypass file system permissions, for convenience. But there's the rub. In this case, Microsoft doesn't have *security*, it has guidelines. They are *still* guidelines, not security, *exactly* because it's convenient for somebody. The fact that taking advantage of that convenience has the side effect of bypassing filesystem permissions is unfortunate (and a bug in Windows IMO). Note that if users actually paid attention to these guidelines, we'd be getting complaints from *them*, not from you. I don't recall ever seeing that. That implies that normal users will install anything anwhere anyway. I don't think that argument is terribly useful. If people waited for normal users to complain before doing something about Heartbeat, we'd be in a pretty pickle. Normal users don't understand the technology well enough to have a valid threat model or judge the consequences, and they are confused by a mixture of ignorance, misinformation and hype. It's up to technical users to lead, not to follow. If it's that unimportant to Microsoft, I think that's unfair. I'm not a MS fan, not even close. I think their business practices in the past have been reprehensible. But if there is anyone who takes backwards-compatibility even more seriously than Python-Dev, it is them. You should give them the courtesy of assuming that their decision is not based on apathy, but on *exactly* the same reasoning that *you* apply below: I see insufficient reason why we should risk confusing those normal users who already have Python 2.7 installed (and as pointed out, they *are* at risk precisely because the proposal changes the default install location). And thus security vulnerabilities never get fixed :-) I don't have an opinion on the importance or magnitude of this security vulnerability, the risk of confusion, or whether it should be fixed or not. But I wonder why the installer is ignoring the OS's guidelines for where software should be installed? If this were Apple we were talking about, would we ignore their guidelines? Or on Linux, would we blithly install Python in / instead of (say) /usr/local/bin? I don't think so. I would have thought that the mere fact that Microsoft disapproves of installing applications into the root should be good enough reason to not do it. In the absense of an extremely compelling reason not to do so, we should be a good citizen regardless of the OS. -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 2.7.7. on Windows
On 4/28/2014 8:52 PM, Steven D'Aprano wrote: I think that's unfair. I'm not a MS fan, not even close. I think their business practices in the past have been reprehensible. But if there is anyone who takes backwards-compatibility even more seriously than Python-Dev, it is them. I guess there is no one who takes backwards-compatibility even more seriously than Python-Dev. I have perfectly good DOS and Windows programs that no longer run on supported Windows versions. XP-Vista was a bigger break than Python 2-3, from my perspective. And then if you get into the command line programs that have vanished or take different options, breaking batch files, and similar facilities, it gets even worse. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bytes with trailing \0?
It is to improve the experience of passing bytes to a C function that
expects a trailing \0. For example syscalls taking filenames. The wrapper
must still check for embedded \0 but the bytes don't need to be copied.
On Monday, April 28, 2014, Nikolaus Rath nikol...@rath.org wrote:
Hello,
I was surprised to find the following in bytesobject.c:
,
| [...]
|As always, an extra byte is allocated for a trailing \0 byte (newsize
|does *not* include that), and a trailing \0 byte is stored.
| */
|
| int
| _PyBytes_Resize(PyObject **pv, Py_ssize_t newsize)
| {
| [...]
|
`
Does this mean that bytes objects are internally stored with a trailing
\0? Why is that? Isn't that just wasting a byte, because \0 might also
be in the middle of the byte sequence, and the bytes objects stores its
length explicitly anyway?
Best,
-Nikolaus
--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
___
Python-Dev mailing list
Python-Dev@python.org javascript:;
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org
--
--Guido van Rossum (on iPad)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
