Re: [Python-Dev] Misc re.match() complaint
Le Mon, 15 Jul 2013 21:53:42 -0700, Guido van Rossum a écrit : > Hm. I'd still like to change this, but I understand it's debatable... > Is the group() method written in C or Python? Is there a strong enough use case to change it? I can't say the current behaviour seems very useful either, but some people may depend on it. I already find it a bit weird that you're passing a bytearray or memoryview to re.match(), to be honest :-) Regards Antoine. > If it's in C it should > be simple enough to let it just do a little bit of pointer math and > construct a bytes object from the given area of memory -- after all, > it must have a pointer to that memory area in order to do the matching > in the first place (although I realize the code may be separated by a > gulf of abstraction :-). > > --Guido > > On Mon, Jul 15, 2013 at 8:03 PM, Nick Coghlan > wrote: > > On 16 July 2013 12:20, Guido van Rossum wrote: > >> On Mon, Jul 15, 2013 at 7:03 PM, Stephen J. Turnbull > >> wrote: > >>> Or is this something deeper, that a group *is* a new object in > >>> principle? > >> > >> No, I just think of it as returning "a string" and I think it's > >> most useful if that is always an immutable object, even if the > >> target string is some other bytes buffer. > >> > >> FWIW, it feels as if the change in behavior is probably just due to > >> how slices work. > > > > I took a look at the way the 2.7 re code works, and the change does > > indeed appear to be due to the difference in the way slices work for > > buffer and memoryview objects: > > > > Slicing a buffer creates an 8-bit string: > > > buffer(b"abc")[0:1] > > 'a' > > > > Slicing a memoryview creates another memoryview: > > > memoryview(b"abc")[0:1] > > > > > > Unfortunately, memoryview doesn't currently allow subclasses, so it > > isn't easy to create a derivative that coerces to bytes on > > slicing :( > > > > Cheers, > > Nick. > > > > -- > > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > > > ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is it safe to call PyEval_EvalFrameEx() with an exception set?
Le Tue, 16 Jul 2013 02:34:49 +0200, Victor Stinner a écrit : > Hi, > > I'm working on the issue #18408 (fix issues found by my pyfailmalloc > tool). To analyze some bugs, I have to add debug traces in various > functions to find which function returned NULL without setting an > error, or the opposite: returned a valid object, but with an exception > set. > > I would like to add assertions in Python/ceval.c to detect such bugs > earlier. The problem is that some functions rely on the ability to > call PyEval_EvalFrameEx() with an exception set. Is it expected? > Should it be avoided? Well, if your frame is a generator frame, you must be able to throw() into it, i.e. resume it with an exception set. The difficulty here is that there are different meanings for "an exception is set": - PyErr_Occurred() is true - the f_exc_type field and friends are set - possibly another one that I'm forgetting about Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 7/15/2013 11:11 PM, Nick Coghlan wrote: I'll look into adding some stronger wording at the top making it clear that while PEP 8 is a useful starting point and a good default if a project doesn't have a defined style guide of it's own, it is *not* the be-all-and-end-all for Python style guides. Treating it as such as an abuse of the PEP, pure and simple. How about (re)naming it Style Guide for Python Standard Library Code Without 'Standard Library' in the title, I can see how some people might think it meant to apply to all code. I do not think we need to worry about the above being too narrow. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Misc re.match() complaint
On 7/15/2013 7:14 PM, Guido van Rossum wrote: In a discussion about mypy I discovered that the Python 3 version of the re module's Match object behaves subtly different from the Python 2 version when the target string (i.e. the haystack, not the needle) is a buffer object. In Python 2, the type of the return value of group() is always either a Unicode string or an 8-bit string, and the type is determined by looking at the target string -- if the target is unicode, group() returns a unicode string, otherwise, group() returns an 8-bit string. In particular, if the target is a buffer object, group() returns an 8-bit string. I think this is the appropriate behavior: otherwise using regular expression matching to extract a small substring from a large target string would unnecessarily keep the large target string alive as long as the substring is alive. But in Python 3, the behavior of group() has changed so that its return type always matches that of the target string. I think this is bad -- apart from the lifetime concern, it means that if your target happens to be a bytearray, the return value isn't even hashable! Does anyone remember whether this was a conscious decision? Is it too late to fix? In both Python 2 and Python 3, the second sentence of the docs is "Both patterns and strings to be searched can be Unicode strings as well as 8-bit strings." The Python 3 version goes on to say that patterns and targets must match. "However, Unicode strings and 8-bit strings cannot be mixed." I normally consider '8-bit string' to mean 'bytes'. It certainly meant that in Python 2. We use 'buffer object' or 'object satisfying the buffer protocol' to mean 'bytes, byte_arrays, or memoryviews'. I wonder if the change was an artifact of changing the code to prohibit mixing Unicode and bytes. Going on "match.group([group1, ...]) Returns one or more subgroups of the match. If there is a single argument, the result is a single string;" In both 2.x and 3.x docs, I usually understand generic 'string' to mean 'Unicode or bytes'. In any case, The sentence and a half from 'Returns' to 'string' is *exactly the same* as in the 2.x docs. As near as I could tell looking by the, the rest of the entry for match.group is unchanged from 2.x to 3.x. So it is easy to think that the behavior change is an unintended regression. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Misc re.match() complaint
On 7/15/2013 10:20 PM, Guido van Rossum wrote: Or is this something deeper, that a group *is* a new object in principle? No, I just think of it as returning "a string" That is exactly what the doc says it does. See my other post. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is it safe to call PyEval_EvalFrameEx() with an exception set?
2013/7/16 Antoine Pitrou :
> Le Tue, 16 Jul 2013 02:34:49 +0200,
> Victor Stinner a écrit :
>> I would like to add assertions in Python/ceval.c to detect such bugs
>> earlier. The problem is that some functions rely on the ability to
>> call PyEval_EvalFrameEx() with an exception set. Is it expected?
>> Should it be avoided?
>
> Well, if your frame is a generator frame, you must be able to throw()
> into it, i.e. resume it with an exception set.
>
> The difficulty here is that there are different meanings for "an
> exception is set":
> - PyErr_Occurred() is true
> - the f_exc_type field and friends are set
> - possibly another one that I'm forgetting about
Ah yes, a generator can call PyEval_EvalFrameEx() with an exception
set (PyErr_Occurred() is non-NULL), but it sets throwflag to 1.
My question is when an exception is set (PyErr_Occurred() is non-NULL)
at the beginning of the bytecode evaluation loop. Just after the "goto
error;" in this extract of ceval.c:
if (throwflag) /* support for generator.throw() */
goto error;
for (;;) { ... }
Victor
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 16/07/2013 6:44am, Nick Coghlan wrote: Clarifying what constitutes an internal interface in a way that doesn't require renaming anything is a necessary prerequisite for bundling or bootstrapping the pip CLI in Python 3.4 (as pip exposes its internal implemetnation API as "import pip" rather than "import _pip" and renaming it would lead to a lot of pointless code churn). Without that concern, the topic never would have come up. BTW, how does the use of __all__ effect things? Somewhere I got the idea that if a module uses __all__ then anything not listed is internal. I take it that is wrong? -- Richard ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Misc re.match() complaint
On 16 July 2013 19:18, Terry Reedy wrote: > I wonder if the change was an artifact of changing the code to prohibit > mixing Unicode and bytes. I'm pretty sure we the only thing we changed in 3.x is to migrate re to the PEP 3118 buffer API, and the behavioural change Guido is seeing is actually the one between the 2.x buffer (which returns 8-bit strings when sliced) and other types (including memoryview) which return instances of themselves. Getting the old buffer behaviour in 3.x without an extra copy operation should just be a matter of wrapping the input with memoryview (to avoid copying the group elements in the match object) and the output with bytes (to avoid keeping the entire original object alive just to reference a few small pieces of it that were matched by the regex): >>> import re >>> data = bytearray(b"aaabbbcccddd") >>> re.match(b"(a*)b*c*(d*)", data).group(2) bytearray(b'ddd') >>> bytes(re.match(b"(a*)b*c*(d*)", memoryview(data)).group(2)) b'ddd' Given that, I'm inclined to keep the existing behaviour on backwards compatibility grounds. To make the above code work on both 2.x *and* 3.x without making an extra copy, it's possible to keep the bytes call (it should be a no-op on 2.x) and dynamically switch the type used to wrap the input between buffer in 2.x and memoryview in 3.x (unfortunately, the 2.x memoryview doesn't work for this case, as the 2.x re API doesn't accept it as valid input). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 16 July 2013 18:41, Terry Reedy wrote: > On 7/15/2013 11:11 PM, Nick Coghlan wrote: > >> I'll look into adding some stronger wording at the top making it clear >> that while PEP 8 is a useful starting point and a good default if a >> project doesn't have a defined style guide of it's own, it is *not* >> the be-all-and-end-all for Python style guides. Treating it as such as >> an abuse of the PEP, pure and simple. > > > How about (re)naming it > Style Guide for Python Standard Library Code > > Without 'Standard Library' in the title, I can see how some people might > think it meant to apply to all code. I do not think we need to worry about > the above being too narrow. I reread the whole thing today - it turns out we *have* tweaked it over time to account for other people using it for their projects as well. The one genuinely standard library specific element still in it (the prohibition on the use of function annotations) is explicitly qualified as only applying to the standard library and not anything else. So I think we need to continue supporting that use case. I did find it interesting that we *don't* explicitly advise against the use of "import *" for anything other than optional accelerator modules or republishing internal interfaces through a public API, even though we advice against the practice in the tutorial. Perhaps another oversight worth correcting? Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 16 July 2013 20:28, Richard Oudkerk wrote: > On 16/07/2013 6:44am, Nick Coghlan wrote: >> >> Clarifying what constitutes an internal interface in a way that >> doesn't require renaming anything is a necessary prerequisite for >> bundling or bootstrapping the pip CLI in Python 3.4 (as pip exposes >> its internal implemetnation API as "import pip" rather than "import >> _pip" and renaming it would lead to a lot of pointless code churn). >> Without that concern, the topic never would have come up. > > > BTW, how does the use of __all__ effect things? Somewhere I got the idea > that if a module uses __all__ then anything not listed is internal. I take > it that is wrong? Gah, you're right. I did consider that, then forgot to include it in the proposed text. Rather than throwing more versions of the text at the list, I'll create a tracker issue and post a proposed patch for PEP 8. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Why does PEP 8 advise against explicit relative imports?
PEP 8 advises developers to use absolute imports rather than explicit relative imports. Why? Using absolute imports couple the internal implementation of a package to its public name - you can't just change the top level directory name any more, you have to go through and change all the absolute imports as well. You also can't easily vendor a package that uses absolute imports inside another project either, since all the absolute imports will break. What's the *concrete* benefit of using absolute imports that overcomes these significant shortcomings in maintainability and composability? Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why does PEP 8 advise against explicit relative imports?
On Tue, Jul 16, 2013 at 1:40 PM, Nick Coghlan wrote: > PEP 8 advises developers to use absolute imports rather than explicit > relative imports. > > Why? Using absolute imports couple the internal implementation of a > package to its public name - you can't just change the top level > directory name any more, you have to go through and change all the > absolute imports as well. You also can't easily vendor a package that > uses absolute imports inside another project either, since all the > absolute imports will break. > The problem with relative imports (both explicit and implicit) is that it makes it less obvious when you are importing a module under the wrong name. If a package ends up in sys.path directly (for example, by executing something that lives inside it directly) then an explicit relative import directly in the package will fail, but an explicit relative import in a sub-package won't, and you can end up with the subtly confusing mess of multiple imports of the same .py file under different names. > What's the *concrete* benefit of using absolute imports that overcomes > these significant shortcomings in maintainability and composability? > In my experience changing the name of a package is problematic for more reasons than just 'import'. We used to do the put-in-a-package thing for third-party packages at Google, and we quit doing it because it was just too painful: it's hard to find all the places that reference the package by name (think of __import__, or mucking in sys.modules), and it's extra problematic with packages that contain extension modules. You can't rename a package, even for 'vendoring a package', without carefully checking whether it'll work -- unless you don't care if it'll work, of course :) The alternative is to include a package *without* changing its name (by adding the right directory, without __init__.py, to sys.path) and that has worked out a lot better for us. Even so, there's no reason for *the standard library* to use explicit relative imports, and that's what PEP 8 is supposed to cover, right? :) -- Thomas Wouters Hi! I'm an email virus! Think twice before sending your email to help me spread! ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.4 and Windows XP: just 45 days until EOL
Am 12.07.13 01:58, schrieb Christian Heimes: > For Python 3.4 is going to be a very close call. According to PEP 429 > 3.4.0 final is scheduled for February 22, 2014. The extended support > phase of Windows XP ends merely 45 days later on April 8, 2014. Do we > really have to restrict ourselves to an API that is going to become > deprecated 45 days after the estimated release of 3.4.0? I suggest to stick to the words of the PEP, which means that XP needs to be supported in 3.4. For 3.5, according to the PEP, XP support *may* be dropped - it doesn't have to be dropped. Notice that you cannot yet know for sure that XP support ends on April 8 - even though it's likely that it will. Microsoft might (and, in the past did) extend an earlier end-of-support date (although they probably won't do so for XP anymore). The most significant reason for dropping XP support in Python will likely be the desire to switch to new VC versions: Newer CRTs don't support XP anymore. Regards, Martin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 16 July 2013 21:10, Nick Coghlan wrote: > On 16 July 2013 20:28, Richard Oudkerk wrote: >> On 16/07/2013 6:44am, Nick Coghlan wrote: >>> >>> Clarifying what constitutes an internal interface in a way that >>> doesn't require renaming anything is a necessary prerequisite for >>> bundling or bootstrapping the pip CLI in Python 3.4 (as pip exposes >>> its internal implemetnation API as "import pip" rather than "import >>> _pip" and renaming it would lead to a lot of pointless code churn). >>> Without that concern, the topic never would have come up. >> >> >> BTW, how does the use of __all__ effect things? Somewhere I got the idea >> that if a module uses __all__ then anything not listed is internal. I take >> it that is wrong? > > Gah, you're right. I did consider that, then forgot to include it in > the proposed text. > > Rather than throwing more versions of the text at the list, I'll > create a tracker issue and post a proposed patch for PEP 8. Tracker issue: http://bugs.python.org/issue18472 When I went to write this in context, I realised it made more sense to focus on defining what a *public* interface was, and how that differed from an internal interface. WIth that approach, the natural flow of the text was: * the public interface is preferentially defined by the documentation * __all__ is the main programmatic interface * leading underscores are then mostly a helper that makes it easier to remember which is which while actually *working* on the code (but still a good practice) I also spotted a few other oddities as I went through. The obviously wrong ones I just fixed, but there were a couple relating to exceptions that seem unnecessary or ill-advised given PEP 352 and 3151, plus the one about absolute vs explicit relative imports that I started a separate thread for. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why does PEP 8 advise against explicit relative imports?
On 16 July 2013 22:02, Thomas Wouters wrote: > On Tue, Jul 16, 2013 at 1:40 PM, Nick Coghlan wrote: >> >> PEP 8 advises developers to use absolute imports rather than explicit >> relative imports. >> >> Why? Using absolute imports couple the internal implementation of a >> package to its public name - you can't just change the top level >> directory name any more, you have to go through and change all the >> absolute imports as well. You also can't easily vendor a package that >> uses absolute imports inside another project either, since all the >> absolute imports will break. > > > The problem with relative imports (both explicit and implicit) is that it > makes it less obvious when you are importing a module under the wrong name. > If a package ends up in sys.path directly (for example, by executing > something that lives inside it directly) then an explicit relative import > directly in the package will fail, but an explicit relative import in a > sub-package won't, and you can end up with the subtly confusing mess of > multiple imports of the same .py file under different names. > >> >> What's the *concrete* benefit of using absolute imports that overcomes >> these significant shortcomings in maintainability and composability? > > In my experience changing the name of a package is problematic for more > reasons than just 'import'. We used to do the put-in-a-package thing for > third-party packages at Google, and we quit doing it because it was just too > painful: it's hard to find all the places that reference the package by name > (think of __import__, or mucking in sys.modules), and it's extra problematic > with packages that contain extension modules. You can't rename a package, > even for 'vendoring a package', without carefully checking whether it'll > work -- unless you don't care if it'll work, of course :) The alternative is > to include a package *without* changing its name (by adding the right > directory, without __init__.py, to sys.path) and that has worked out a lot > better for us. These are good reasons, but they're not the reasons the PEP currently gives. "Better failure modes" is a good practical benefit :) > Even so, there's no reason for *the standard library* to use explicit > relative imports, and that's what PEP 8 is supposed to cover, right? :) I realised in re-reading the whole thing that we actually admitted defeat on that front back during the last function annotations discussion :) Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why does PEP 8 advise against explicit relative imports?
On 16 July 2013 22:27, Nick Coghlan wrote: > These are good reasons, but they're not the reasons the PEP currently > gives. "Better failure modes" is a good practical benefit :) I have now included an updated rationale for this rule in the patch attached to http://bugs.python.org/issue18472 (the recommendation to use absolute imports remains unchanged). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python 3.4 and Windows XP: just 45 days until EOL
On 12/07/2013 00:58, Christian Heimes wrote: > Hi, > > how do you feel about dropping Windows XP support for Python 3.4? It > would enable us to use some features that are only available on Windows > Vista and newer, for example http://bugs.python.org/issue6926 and > http://bugs.python.org/issue1763 . > > PEP 11 says: > A new feature release X.Y.0 will support all Windows releases > whose extended support phase is not yet expired. > > For Python 3.4 is going to be a very close call. According to PEP 429 > 3.4.0 final is scheduled for February 22, 2014. The extended support > phase of Windows XP ends merely 45 days later on April 8, 2014. Do we > really have to restrict ourselves to an API that is going to become > deprecated 45 days after the estimated release of 3.4.0? I would like to continue support for WinXP. It's still widely, widely used -- MS support notwithstanding. The situation might have been different if Vista had been a viable corporate desktop, but I suspect that many outfits have waited (as we did here) until Win7 before moving forward. Win7 is now our default desktop for new machines, but we're still running a slight majority of WinXP machines. TJG ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 16/07/13 20:28, Richard Oudkerk wrote: On 16/07/2013 6:44am, Nick Coghlan wrote: Clarifying what constitutes an internal interface in a way that doesn't require renaming anything is a necessary prerequisite for bundling or bootstrapping the pip CLI in Python 3.4 (as pip exposes its internal implemetnation API as "import pip" rather than "import _pip" and renaming it would lead to a lot of pointless code churn). Without that concern, the topic never would have come up. BTW, how does the use of __all__ effect things? Somewhere I got the idea that if a module uses __all__ then anything not listed is internal. I take it that is wrong? That is not how I interpret __all__. In the absence of any explicit documentation, I interpret __all__ as nothing more than a list of names which wildcard imports will bring in, without necessarily meaning that other names are private. For example, I might have a module explicitly designed for wildcard imports at the interactive interpreter: from module import * brings in the functions which I expect will be useful interactively, not necessarily the entire public API. For example, pkgutil includes classes with single-underscore methods, which I take as private. It also has a function simplegeneric, which is undocumented and not listed in __all__. In in the absence of even a comment saying "Don't use this", I take it as an oversight, not policy that simplegeneric is private. -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On Tue, 16 Jul 2013 23:19:21 +1000, Steven D'Aprano wrote: > On 16/07/13 20:28, Richard Oudkerk wrote: > > On 16/07/2013 6:44am, Nick Coghlan wrote: > >> Clarifying what constitutes an internal interface in a way that > >> doesn't require renaming anything is a necessary prerequisite for > >> bundling or bootstrapping the pip CLI in Python 3.4 (as pip exposes > >> its internal implemetnation API as "import pip" rather than "import > >> _pip" and renaming it would lead to a lot of pointless code churn). > >> Without that concern, the topic never would have come up. > > > > BTW, how does the use of __all__ effect things? Somewhere I got the idea > > that if a module uses __all__ then anything not listed is internal. I take > > it that is wrong? > > > That is not how I interpret __all__. In the absence of any explicit > documentation, I interpret __all__ as nothing more than a list of names which > wildcard imports will bring in, without necessarily meaning that other names > are private. For example, I might have a module explicitly designed for > wildcard imports at the interactive interpreter: > > from module import * > > brings in the functions which I expect will be useful interactively, not > necessarily the entire public API. > > For example, pkgutil includes classes with single-underscore methods, which I > take as private. It also has a function simplegeneric, which is undocumented > and not listed in __all__. In in the absence of even a comment saying "Don't > use this", I take it as an oversight, not policy that simplegeneric is > private. I think you'd be wrong about that, though. simplegeneric should really be treated as private. I'm speaking here not about the general principle of the thing, but about my understanding of simplegeneric's specific history. --David ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 16 July 2013 23:39, R. David Murray wrote: > On Tue, 16 Jul 2013 23:19:21 +1000, Steven D'Aprano > wrote: >> On 16/07/13 20:28, Richard Oudkerk wrote: >> > On 16/07/2013 6:44am, Nick Coghlan wrote: >> >> Clarifying what constitutes an internal interface in a way that >> >> doesn't require renaming anything is a necessary prerequisite for >> >> bundling or bootstrapping the pip CLI in Python 3.4 (as pip exposes >> >> its internal implemetnation API as "import pip" rather than "import >> >> _pip" and renaming it would lead to a lot of pointless code churn). >> >> Without that concern, the topic never would have come up. >> > >> > BTW, how does the use of __all__ effect things? Somewhere I got the idea >> > that if a module uses __all__ then anything not listed is internal. I >> > take it that is wrong? >> >> >> That is not how I interpret __all__. In the absence of any explicit >> documentation, I interpret __all__ as nothing more than a list of names >> which wildcard imports will bring in, without necessarily meaning that other >> names are private. For example, I might have a module explicitly designed >> for wildcard imports at the interactive interpreter: >> >> from module import * >> >> brings in the functions which I expect will be useful interactively, not >> necessarily the entire public API. >> >> For example, pkgutil includes classes with single-underscore methods, which >> I take as private. It also has a function simplegeneric, which is >> undocumented and not listed in __all__. In in the absence of even a comment >> saying "Don't use this", I take it as an oversight, not policy that >> simplegeneric is private. > > I think you'd be wrong about that, though. simplegeneric should really be > treated as private. I'm speaking here not about the general principle of > the thing, but about my understanding of simplegeneric's specific history. And, indeed, you're correct (the issue that eventually became the functools.singledispatch PEP started life with a title like "move simplegeneric to functools and make it public"). For the general case, the patch I posted to the issue tracker sets the precedence as docs -> __all__ -> leading underscore. If a module has public APIs that aren't in __all__, it should cover them in the docs, otherwise people should assume they're private. It's OK if there are exceptions to that general rule - there's a reason PEP 8 has the great big qualifier pointing out that it isn't universally applicable (even if we might sometimes wish otherwise). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why does PEP 8 advise against explicit relative imports?
On Jul 16, 2013, at 02:02 PM, Thomas Wouters wrote: >In my experience changing the name of a package is problematic for more >reasons than just 'import'. We used to do the put-in-a-package thing for >third-party packages at Google, and we quit doing it because it was just >too painful: it's hard to find all the places that reference the package by >name (think of __import__, or mucking in sys.modules), and it's extra >problematic with packages that contain extension modules. You can't rename >a package, even for 'vendoring a package', without carefully checking >whether it'll work -- unless you don't care if it'll work, of course :) The >alternative is to include a package *without* changing its name (by adding >the right directory, without __init__.py, to sys.path) and that has worked >out a lot better for us. Vendoring also gives no end of headaches to distro packagers because we have to un-vendor that stuff, which can lead to a mess of patches on top of upstream. >Even so, there's no reason for *the standard library* to use explicit >relative imports, and that's what PEP 8 is supposed to cover, right? :) Right! It doesn't bother me at all that this particular recommendation may be followed by third parties too. :) -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On Jul 15, 2013, at 11:02 PM, Chris McDonough wrote: >Given that misunderstanding, is there a way to divorce stdlib-centric >guidelines like the one being discussed now from "PEP8"-ness? I don't >think any amount of marketing effort or reasoned explanation is going to >separate "PEP8" from "correct code" in people's minds at this point. The standard seems to be "if the pep8 program complains, I forward that upstream", so it's probably most effective to discuss with them. OTOH, we've had to clarify PEP 8 in order to get the pep8 developers to adjust their tool, most recently e.g. the closing brace/bracket/paren for multiline collections. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On Jul 16, 2013, at 11:28 AM, Richard Oudkerk wrote: >BTW, how does the use of __all__ effect things? Somewhere I got the idea >that if a module uses __all__ then anything not listed is internal. I take >it that is wrong? Purely technically, __all__ is there to affect how from-import-* works. I personally think it's a good idea to include all your public names, and none of your non-public names, in __all__, but it's not always easy to keep up-to-date. pyflakes has the nice benefit of complaining when something is named in __all__ that doesn't exist in the module, but that's only one part of the keeping-things-up-to-date problem. -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On Jul 16, 2013, at 11:48 PM, Nick Coghlan wrote: >For the general case, the patch I posted to the issue tracker sets the >precedence as docs -> __all__ -> leading underscore. +1 -Barry ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On Jul 16, 2013, at 04:10 PM, Nick Coghlan wrote: >Yeah, we should probably be a bit more vocal in pointing out that PEP >8 is something to be applied judiciously, *not* treated as "sacred >writ, never to be questioned". It's very tempting to just sigh and >ignore it, instead of explicitly pointing out the problems with taking >it too literally for people's personal projects (or, worse, using it >to abuse developers of projects that long predate the recommendations >in more recent versions) :P In my experience, most PEP 8 violation bugs that I see for various upstreams (mine and others) comes from running the pep8 tool over the code. A lot of upstreams do this as part of their normal QA (e.g. you can set up unittests that run pep8, pyflakes, and other linters). Those will generally be fine because they're already committed to conformance to pep8's notion of what PEP 8 says [1]. In other cases, Someone Out There runs pep8 over your code, and files bugs on all the complaints that get produced. That can definitely lead to churn and waste, especially on big, established code bases. -Barry [1] which I do not always agree with anyway, as evidenced by the recent discussion on closing paren indentation for multi-line collections. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Misc re.match() complaint
On Tue, Jul 16, 2013 at 12:55 AM, Antoine Pitrou wrote: > Is there a strong enough use case to change it? I can't say the current > behaviour seems very useful either, but some people may depend on it. This is the crucial question. I personally see the current behavior as an artifact of the (lack of) design process, not as a conscious decision. Given that we also have m.string, m.start(grp) and m.end(grp), those who need something matching the original type (or even something that is known to be a reference into the original object) can use that API; for most use cases, all you care about is is the selected group as a string, and it is more useful if that is always an immutable string (bytes or str). The situation is most egregious if the target string is a bytearray, where there is currently no way to get the result as an immutable bytes object without an extra copy. (There's no API that lets you create a bytes object directly from a slice of a bytearray.) In terms of backwards compatibility, I wouldn't want to do this in a bugfix release, but for a feature release I think it's fine -- the number of applications that could be bitten by this must be extremely small (and the work-around is backward-compatible: just use m.string[m.start() : m.stop()]). > I already find it a bit weird that you're passing a bytearray or > memoryview to re.match(), to be honest :-) Yes, this is somewhat of an odd corner, but actually most built-in APIs taking bytes also take anything else that can be coerced to bytes (io.open() seems to be the exception, and it feels like an accident -- os.open() *does* accept bytearray and friends). This is quite useful for code that interacts with C code or system calls -- often you have a large buffer shared between C and Python code for efficiency, and being able to do pretty much anything to the buffer that you can do to a bytes object (apart from using it as a dict key) helps a lot. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why does PEP 8 advise against explicit relative imports?
On 7/16/2013 7:40 AM, Nick Coghlan wrote: PEP 8 advises developers to use absolute imports rather than explicit relative imports. Why? Using absolute imports couple the internal implementation of a package to its public name - you can't just change the top level directory name any more, you have to go through and change all the absolute imports as well. You also can't easily vendor a package that uses absolute imports inside another project either, since all the absolute imports will break. What's the *concrete* benefit of using absolute imports that overcomes these significant shortcomings in maintainability and composability? From my viewpoint, absolute imports always work, on all python versions, whereas people have reported various problems with relative imports on python-list. At least some of the complaints were valid, at least at the time, but I have not paid too much attention. Even if idlelib files used relative imports, 'idlelib' cannot change and I see no reason to ever change 'idle_test'. I would like to change camelCaseFileNames.py, but if ever done, that would affect both types of imports. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
On 7/16/2013 9:39 AM, R. David Murray wrote: On Tue, 16 Jul 2013 23:19:21 +1000, Steven D'Aprano wrote: For example, pkgutil includes classes with single-underscore methods, which I take as private. It also has a function simplegeneric, which is undocumented and not listed in __all__. In in the absence of even a comment saying "Don't use this", I take it as an oversight, not policy that simplegeneric is private. I think you'd be wrong about that, though. simplegeneric should really be treated as private. I'm speaking here not about the general principle of the thing, but about my understanding of simplegeneric's specific history. I think Steven (valid) point is "Why not, then, say it is internal either in docs or name?"-- which in this case would be in the docs. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #18408: Fix fileio_read() on _PyBytes_Resize() failure
17.07.13 00:09, victor.stinner написав(ла):
http://hg.python.org/cpython/rev/533eb9ab895a
changeset: 84669:533eb9ab895a
user:Victor Stinner
date:Tue Jul 16 21:36:02 2013 +0200
summary:
Issue #18408: Fix fileio_read() on _PyBytes_Resize() failure
bytes is NULL on _PyBytes_Resize() failure
files:
Modules/_io/fileio.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/Modules/_io/fileio.c b/Modules/_io/fileio.c
--- a/Modules/_io/fileio.c
+++ b/Modules/_io/fileio.c
@@ -739,7 +739,7 @@
if (n != size) {
if (_PyBytes_Resize(&bytes, n) < 0) {
-Py_DECREF(bytes);
+Py_CLEAR(bytes);
return NULL;
}
}
Why not Py_DECREF?
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #18408: Fix fileio_read() on _PyBytes_Resize() failure
2013/7/16 Serhiy Storchaka : >> http://hg.python.org/cpython/rev/533eb9ab895a >> summary: >>Issue #18408: Fix fileio_read() on _PyBytes_Resize() failure >> >> bytes is NULL on _PyBytes_Resize() failure > > Why not Py_DECREF? Because Py_DECREF(NULL) does crash. Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #18408: Fix fileio_read() on _PyBytes_Resize() failure
17.07.13 01:03, Victor Stinner написав(ла): 2013/7/16 Serhiy Storchaka : http://hg.python.org/cpython/rev/533eb9ab895a summary: Issue #18408: Fix fileio_read() on _PyBytes_Resize() failure bytes is NULL on _PyBytes_Resize() failure Why not Py_DECREF? Because Py_DECREF(NULL) does crash. Oh, I meaned Py_XDECREF. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #18408: Fix fileio_read() on _PyBytes_Resize() failure
Am 17.07.2013 00:03, schrieb Victor Stinner: > 2013/7/16 Serhiy Storchaka : >>> http://hg.python.org/cpython/rev/533eb9ab895a >>> summary: >>>Issue #18408: Fix fileio_read() on _PyBytes_Resize() failure >>> >>> bytes is NULL on _PyBytes_Resize() failure >> >> Why not Py_DECREF? > > Because Py_DECREF(NULL) does crash. Why not Py_XDECREF() then? ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Issue #18408: Fix fileio_read() on _PyBytes_Resize() failure
2013/7/17 Serhiy Storchaka : > Oh, I meaned Py_XDECREF. Ah ok :-) Well, for this specific code, it can probably be replaced with: if (_PyBytes_Resize(&bytes, n) < 0) return NULL; I'm not sure that _PyBytes_Resize() *always* decref bytes and then set bytes to NULL. I was too lazy to check this. If someone wants to audit the code of _PyBytes_Resize(), there are many other places where Py_XDECREF / Py_CLEAR can be removed on _PyBytes_Resize() failure. And I'm quite sure that there are still places where Py_DECREF() is still used on _PyBytes_Resize() failure ;-) Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Is it safe to call PyEval_EvalFrameEx() with an exception set?
2013/7/16 Victor Stinner : > I would like to add assertions in Python/ceval.c to detect such bugs > earlier. The problem is that some functions rely on the ability to > call PyEval_EvalFrameEx() with an exception set. Is it expected? > Should it be avoided? The current exception can be replaced with a new > exception. Another example. Py_ReprLeave() function can be called while an exception is set. I'm not sure that it is "safe" to call new code while an exception is set. The function can raise a new exception. For example, PyList_SetSlice() can fail because of a MemoryError. I modified Py_ReprLeave() to save and then restore the current exception: http://hg.python.org/cpython/rev/28ff7ac91477 If Py_ReprLeave() raises a new exception, it is clears (replaced by the previous exception context). Victor ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/16/2013 07:09 AM, Nick Coghlan wrote: > I did find it interesting that we *don't* explicitly advise against > the use of "import *" for anything other than optional accelerator > modules or republishing internal interfaces through a public API, > even though we advice against the practice in the tutorial. Perhaps > another oversight worth correcting? +1. 'from foo import *' (from any source other than another module in the same package) is a code stench far worse than anything else PEP8 proscribes. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlHl2GsACgkQ+gerLs4ltQ6+1QCgmu5k6p5otzPxvzGh5mA1Ch7t 2f0AoK2a0/m144bnIwBFLLuY9l2bdWMN =878p -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tweaking PEP 8 guidelines for use of leading underscores
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On Jul 16, 2013, at 07:34 PM, Tres Seaver wrote: >On 07/16/2013 07:09 AM, Nick Coghlan wrote: > >> I did find it interesting that we *don't* explicitly advise against >> the use of "import *" for anything other than optional accelerator >> modules or republishing internal interfaces through a public API, >> even though we advice against the practice in the tutorial. Perhaps >> another oversight worth correcting? > >+1. 'from foo import *' (from any source other than another module in >the same package) is a code stench far worse than anything else PEP8 >proscribes. Maybe we should disable it for anything that isn't at the interactive prompt <0.5 wink>. - -Barry -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.20 (GNU/Linux) iQIcBAEBCAAGBQJR5e5VAAoJEBJutWOnSwa/SK4P/22KvWNRQ7sjDWJ+jebgpXyD mkUeO4r905uqp5Bvedlyj4GoFhdFPQoeQEGgQeNwiMkFxCGJDKiY1XoJAgqj2zOE vUgoyLE5YSAnz8xG/Ib3ZZPXhDJY14gu/czbX/cc83Q+cQJd0+RpnhUt5jQkxAYR k6xXF7hwTHaTYlqzdTDQUtimg21AUb0lgI5XqCMGL4aac4H69JiN+3ydI3BQxqxY MXe4h4rzDxnaBnekHYCouM1jsN6CsLeIHafJR8W4Zgm9vuxZYIha+kAQd+tILFXG d/0hpvLIGP+ubNPIvS1Ay+aYsH7PiL4keCoP0pi+llMTQv+W5wDeCumLpvXjaEFj 46c7HaRaxbBqpactsG98rZOtmBwSgEwzzPSQak+sWHDCX4RFOS3qDh3WhtY0Lw7n NdEkJ0f+6bEt2ot+emDpth/SAxgOEUjNOhZ8/glYtvZcz2wBWRGkCAIEwSJtP1VZ YgRHFsoUUbS6KoI9cZWhwynmIVQf4Wn9a2Zx5/YUbHIM2mY1s60MxcOn3KgPpEU4 Zjp/gyPHsoIORrZU2Q2pwzdWEmSzorZBP/NqRqlhwBTexKdfqyRFztsFMVcnLPDv Cl0t04BGXH8/0fZ+peEOccqnuPsHumr9VqmhMCcLChRPRQyBD9ETSPVwHhprdtka qkLcyWl/M3ijyuAkdYbl =Q+Du -END PGP SIGNATURE- ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why does PEP 8 advise against explicit relative imports?
On 16/07/13 22:02, Thomas Wouters wrote: On Tue, Jul 16, 2013 at 1:40 PM, Nick Coghlan wrote: PEP 8 advises developers to use absolute imports rather than explicit relative imports. Why? Using absolute imports couple the internal implementation of a package to its public name - you can't just change the top level directory name any more, you have to go through and change all the absolute imports as well. You also can't easily vendor a package that uses absolute imports inside another project either, since all the absolute imports will break. The problem with relative imports (both explicit and implicit) is that it makes it less obvious when you are importing a module under the wrong name. If a package ends up in sys.path directly (for example, by executing something that lives inside it directly) then an explicit relative import directly in the package will fail, but an explicit relative import in a sub-package won't, and you can end up with the subtly confusing mess of multiple imports of the same .py file under different names. I don't understand this objection. What do you mean by "importing a module under the wrong name"? As I see it, if relative imports are a bad idea for the std lib, they're a bad idea for everyone. (What am I missing?) So why do we have relative imports? Was it a mistake? I don't think so. It seems to me that you're giving some fairly obscure edge cases, if I've understood you correctly. If adding a package directory to sys.path breaks relative imports, isn't that an argument against adding the package directory to sys.path rather than relative imports? What's the *concrete* benefit of using absolute imports that overcomes these significant shortcomings in maintainability and composability? In my experience changing the name of a package is problematic for more reasons than just 'import'. We used to do the put-in-a-package thing for third-party packages at Google, and we quit doing it because it was just too painful: it's hard to find all the places that reference the package by name (think of __import__, or mucking in sys.modules), Again, these are some fairly odd edge cases. I don't mean to imply that they never occur in practice, obviously they do, but perhaps you're overly sensitive to them. It seems strange to me that PEP 8 should recommend against using relative imports for the 49 packages that would benefit from it just to avoid problems with the 1 that does weird things with __import__ or manually mucks about with sys.modules. Have I missed something? Even so, there's no reason for *the standard library* to use explicit relative imports, and that's what PEP 8 is supposed to cover, right? :) If a third-party package uses relative imports, are you suggesting that those rel imports should be turned into absolute imports as a precondition of it being moved into the std lib? -- Steven ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why does PEP 8 advise against explicit relative imports?
I have to say that a long time ago, when you couldn't tell a relative import from a top-level import, the ban on relative imports made more sense. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why does PEP 8 advise against explicit relative imports?
On 17 July 2013 11:55, Steven D'Aprano wrote: > On 16/07/13 22:02, Thomas Wouters wrote: > As I see it, if relative imports are a bad idea for the std lib, they're a > bad idea for everyone. (What am I missing?) So why do we have relative > imports? Was it a mistake? I don't think so. > > It seems to me that you're giving some fairly obscure edge cases, if I've > understood you correctly. If adding a package directory to sys.path breaks > relative imports, isn't that an argument against adding the package > directory to sys.path rather than relative imports? What Thomas is saying is that, given the current state of the import system, using absolute imports usually give you better behaviour (or at least better error messages) than explicit relative imports do when things go wrong. At present, there are still too many ways to get your import configuration into a bad state (most notably running a module inside a package by filepath rather than the -m switch), and if that happens, using absolute imports is likely to make it a bit easier to find your way out again. I actually agree with that rationale - while I prefer explicit relative imports myself, I'm also more experienced than most when it comes to identifying and resolving import configuration problems (see http://python-notes.curiousefficiency.org/en/latest/python_concepts/import_traps.html) So, for now, avoiding even explicit relative imports is still a good default position. People who feel qualified to reject PEP 8's advice on this topic should also know enough to deal with the potential consequences. I'll be proposing a rewording of this admonition to give a better rationale, but I won't be proposing that the recommendation be changed :) Once we get sys.path initialisation to a better place (amongst other things), then it will be time to revisit the recommendation itself. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why does PEP 8 advise against explicit relative imports?
If sounds like the main problem with relative imports (even explicit) is caused by ignoring the package structure and running modules inside a package as a script, without using -m. Maybe we should update the PEP to make this abundantly clear? I note that an additional problem with informational PEPs like PEP 8 is that they aren't tied to a particular Python version, so even if we solved the above issue perfectly in Python 3.4 there'd still be tons of people applying PEP 8 to older code who would benefit from the warning against relative imports. So maybe we should also add that to the PEP. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Misc re.match() complaint
Terry Reedy writes: > On 7/15/2013 10:20 PM, Guido van Rossum wrote: > > >> Or is this something deeper, that a group *is* a new object in > >> principle? > > > > No, I just think of it as returning "a string" > > That is exactly what the doc says it does. See my other post. The problem is that IIUC '"a string"' is intentionally *not* referring to the usual "str or bytes objects" (at least that's one of the standard uses for scare quotes, to indicate an unusual usage). Either the docstring is using "string" in a similarly ambiguous way, or else it's incorrect under the interpretation that buffer objects are *not* "strings", so they should be inadmissible as targets. Something should be fixed, and I suppose it should be the return type of group(). BTW, I suggest that Terry's usage of "string" (to mean "str or bytes" in 3.x, "unicode or str" in 2.x) be adopted, and Guido's "stringish" be given expanded meaning, including buffer objects. Then we can say informally that in searching and matching a target is a stringish, the pattern is a stringish (?) or compiled re, but the group method returns a string. Steve ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Why does PEP 8 advise against explicit relative imports?
On 16 Jul, 2013, at 14:02, Thomas Wouters wrote: > On Tue, Jul 16, 2013 at 1:40 PM, Nick Coghlan wrote: > >> PEP 8 advises developers to use absolute imports rather than explicit >> relative imports. >> >> Why? Using absolute imports couple the internal implementation of a >> package to its public name - you can't just change the top level >> directory name any more, you have to go through and change all the >> absolute imports as well. You also can't easily vendor a package that >> uses absolute imports inside another project either, since all the >> absolute imports will break. >> > > The problem with relative imports (both explicit and implicit) is that it > makes it less obvious when you are importing a module under the wrong name. > If a package ends up in sys.path directly (for example, by executing > something that lives inside it directly) then an explicit relative import > directly in the package will fail, but an explicit relative import in a > sub-package won't, and you can end up with the subtly confusing mess of > multiple imports of the same .py file under different names. That's only a problem for subpackages (that is, for example when distutils.commands ends up being importable as commands), because explicit relative imports (``from .other import attr``) don't work in toplevel modules or scripts because the leading dot means "the current package". I tend to use explicit relative imports in my code when that increases readability by reducing unnecessary duplication. That is, readability tends to suffer when you have a number of lines like with "from package.module1 import name". Ronald ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
