[Python-Dev] Add calloc-like memory allocators to Python

[Victor Stinner]

Hi,

We made progress on the following issue and the latest patch
(calloc-5.patch) is ready for a review:
http://bugs.python.org/issue21233

This issue should help numpy to reuse Python memory allocators to use
the new tracemalloc module of Python 3.4. The issue is only for Python
3.5.

It was also discussed that numpy might require a memory allocator with
a specified alignment for SIMD instructions. This topic is discussed
in a different issue:
http://bugs.python.org/issue18835


Summary of calloc-5.patch:

- add the following functions:

  * void* PyMem_RawCalloc(size_t nelem, size_t elsize)
  * void* PyMem_Calloc(size_t nelem, size_t elsize)
  * void* PyObject_Calloc(size_t nelem, size_t elsize)
  * PyObject* _PyObject_GC_Calloc(size_t basicsize)

- add "void* calloc(void *ctx, size_t nelem, size_t elsize)" field to
the PyMemAllocator structure
- optimize bytes(n) and bytearray(n) to allocate objects using
calloc() instead of malloc()
- update tracemalloc to trace also calloc()
- document new functions and add unit tests for the calloc "hook" (in _testcapi)

Victor
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Python 2.7.7. on Windows

[Mike Miller]

Greetings,

I've just woken up and noticed Python 2.7.7 is on track to be released, and in a 
rather unique event contains a few security enhancements

in addition to the usual fixes:

http://legacy.python.org/dev/peps/pep-0466/

I thought this might be a good time to make a final plea to fix a
long-standing security issue in the installer on Windows.  By default it
installs Python to the root folder, thereby bypassing filesystem permissions:

http://bugs.python.org/issue1284316

The main rationale given (for not using the standard %ProgramFiles%) has been 
that the full path to python is too long to type, and ease of use is more 
important than the security benefits given by following Windows conventions.


However, adding python to the PATH variable is an alternative solution that
would result in even fewer keystrokes needing to be typed at a console, while
maintaining system security.

As this is an installer setting and not a code change, I gather that the
opportunities for code breakage should be fewer.  Please consider updating
this setting to a more secure and friendly default, for 2.7.7 and 3.5+ as well.

-Mike

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

[Jim J. Jewett]

(1)  Should fixes to a docstring go in with a patch, even if they
aren't related to the changing functionality?

Bytestring compilation has several orthogonal parameters.  Most -- but
not all -- are documented in the docstring.  (Specifically, there is
no explanation of the rx parameter which acts as a filter, and no
mention that symbolic links to directories are ignored.)

It is best if a commit changes one small thing at a time.
On the other hand, Nick recently posted that the minimal overhead of a
patch commit is about half an hour.

Is that overhead enough to override the one-issue-per-patch guideline?

(2)  The patch adds new functionality to use multiple processes in
parallel.  The normal parameter values are integers indicating how
many processes to use.  The parameter also needs two special values --
one to indicate "use os.cpu_count", and the other to indicate "don't
use multiprocessing at all".

(A)  Is there a Best Practices for this situation, with two odd cases?

(B)  Claudiu originally copied the API from a similar APU for
regrtest.  What is the barrier for "do it sensibly" vs "stick with
precedent elsewhere"?  (Apparently regrtest treats any negative number
as a request for the cpu_count calculation;  I suspect that "-5" is
more likely to be an escaping error for "5" than it is to be a real
request for auto-calculation that just happened to choose -5 instead
of -1.)

(C)  How important is it to keep the API consistent between a
top-level CLI command and the internal implementation?  At the the
moment, the request for cpu_count is handled in the CLI wrapper, and
not available to interactive callers.  On the other hand, interactive
callers could just call cpu_count themselves...

(D)  How important is it to maintain consistency with other uses of
the same tool -- multiprocessing has its own was of requesting
auto-calculation.  (So someone used to multiprocessing might assume
that "None" meant to auto-calculate, as opposed to "don't use
multiprocessing at all.")

-jJ
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

[Brett Cannon]

On Mon Apr 28 2014 at 11:32:58 AM, Jim J. Jewett 
wrote:

> (1)  Should fixes to a docstring go in with a patch, even if they
> aren't related to the changing functionality?
>

It should probably be its own commit.


>
> Bytestring compilation has several orthogonal parameters.  Most -- but
> not all -- are documented in the docstring.  (Specifically, there is
> no explanation of the rx parameter which acts as a filter, and no
> mention that symbolic links to directories are ignored.)
>
> It is best if a commit changes one small thing at a time.
> On the other hand, Nick recently posted that the minimal overhead of a
> patch commit is about half an hour.
>
> Is that overhead enough to override the one-issue-per-patch guideline?
>

That's typical for backported patches, not if something is only going into
default. That being said, if the change is truly small and you "sneak" it
in I don't think anyone is going to make you revert the patch to do it as
more atomic commits.


>
> (2)  The patch adds new functionality to use multiple processes in
> parallel.  The normal parameter values are integers indicating how
> many processes to use.  The parameter also needs two special values --
> one to indicate "use os.cpu_count", and the other to indicate "don't
> use multiprocessing at all".
>
> (A)  Is there a Best Practices for this situation, with two odd cases?
>

No. In this situation I would consider 0 or -1 for "use os.cpu_count' and
None for "don't use multi-processing".


>
> (B)  Claudiu originally copied the API from a similar APU for
> regrtest.  What is the barrier for "do it sensibly" vs "stick with
> precedent elsewhere"?  (Apparently regrtest treats any negative number
> as a request for the cpu_count calculation;  I suspect that "-5" is
> more likely to be an escaping error for "5" than it is to be a real
> request for auto-calculation that just happened to choose -5 instead
> of -1.)
>

Regrtest doesn't count as an API to stick to. =)


>
> (C)  How important is it to keep the API consistent between a
> top-level CLI command and the internal implementation?  At the the
> moment, the request for cpu_count is handled in the CLI wrapper, and
> not available to interactive callers.  On the other hand, interactive
> callers could just call cpu_count themselves...
>

It doesn't hurt, but we typically don't promote CLIs for stdlib modules so
there isn't much precedent.


>
> (D)  How important is it to maintain consistency with other uses of
> the same tool -- multiprocessing has its own was of requesting
> auto-calculation.  (So someone used to multiprocessing might assume
> that "None" meant to auto-calculate, as opposed to "don't use
> multiprocessing at all.")
>

That's a judgment call. If there is already no consistency go with the one
that makes the most sense. If consistency exists across the stdlib then
stay consistent.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

[Claudiu Popa]

On Mon, Apr 28, 2014 at 6:32 PM, Jim J. Jewett  wrote:
> (1)  Should fixes to a docstring go in with a patch, even if they
> aren't related to the changing functionality?
>
> Bytestring compilation has several orthogonal parameters.  Most -- but
> not all -- are documented in the docstring.  (Specifically, there is
> no explanation of the rx parameter which acts as a filter, and no
> mention that symbolic links to directories are ignored.)
>
> It is best if a commit changes one small thing at a time.
> On the other hand, Nick recently posted that the minimal overhead of a
> patch commit is about half an hour.
>
> Is that overhead enough to override the one-issue-per-patch guideline?
>
> (2)  The patch adds new functionality to use multiple processes in
> parallel.  The normal parameter values are integers indicating how
> many processes to use.  The parameter also needs two special values --
> one to indicate "use os.cpu_count", and the other to indicate "don't
> use multiprocessing at all".
>
> (A)  Is there a Best Practices for this situation, with two odd cases?
>
> (B)  Claudiu originally copied the API from a similar APU for
> regrtest.  What is the barrier for "do it sensibly" vs "stick with
> precedent elsewhere"?  (Apparently regrtest treats any negative number
> as a request for the cpu_count calculation;  I suspect that "-5" is
> more likely to be an escaping error for "5" than it is to be a real
> request for auto-calculation that just happened to choose -5 instead
> of -1.)
>
> (C)  How important is it to keep the API consistent between a
> top-level CLI command and the internal implementation?  At the the
> moment, the request for cpu_count is handled in the CLI wrapper, and
> not available to interactive callers.  On the other hand, interactive
> callers could just call cpu_count themselves...
>
> (D)  How important is it to maintain consistency with other uses of
> the same tool -- multiprocessing has its own was of requesting
> auto-calculation.  (So someone used to multiprocessing might assume
> that "None" meant to auto-calculate, as opposed to "don't use
> multiprocessing at all.")
>


Thanks for writing this, Jim. Hopefully, the answers to these
questions will solve our disagreements
regarding the API.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7.7. on Windows

[Sturla Molden]

Mike Miller  wrote:

> The main rationale given (for not using the standard %ProgramFiles%) has been 
> that the full path to python is too long to type, and ease of use is more 
> important than the security benefits given by following Windows conventions.

"C:\Program Files\Python27" contains an empty space in the path. If you
want to randomly break build tools for C extensions, then go ahead and
change it. 

Sturla

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

[Charles-François Natali]

>> (2)  The patch adds new functionality to use multiple processes in
>> parallel.  The normal parameter values are integers indicating how
>> many processes to use.  The parameter also needs two special values --
>> one to indicate "use os.cpu_count", and the other to indicate "don't
>> use multiprocessing at all".
>>
>> (A)  Is there a Best Practices for this situation, with two odd cases?
>
>
> No. In this situation I would consider 0 or -1 for "use os.cpu_count' and
> None for "don't use multi-processing".

Why would the user care if multiprocessing is used behind the scene?
It would be strange for processes=1 to fail if multiprocessing is not
available.

If you set a default value of 1, then compileall() will work
regardless of whether multiprocessing is available.

In short:
processes <= 0: use os.cpu_count()
processes == 1 (default): just use normal sequential compiling
processes > 1: use multiprocessing

There's no reason to introduce None. Or am I missing something?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7.7. on Windows

[Steve Dower]

Mike Miller wrote:
> I thought this might be a good time to make a final plea to fix a
> long-standing security issue in the installer on Windows. By default it 
> installs
> Python to the root folder, thereby bypassing filesystem permissions:
>
>http://bugs.python.org/issue1284316

This would be an incredibly painful change that would surprise and hurt a lot of
people.

Mark Hammond's last comment on the linked issue sums up my view as well:
"My take is still that Python is a tool, not an app. People writing an app they
wish to distribute using Python should include Python in their package (ie, not
rely on an installed version) and these apps should conform with the
guidelines."

Yes, it is possible for a non-admin user to install arbitrary packages into a
place where an admin user may inadvertently run it, thereby providing escalation
of privilege. On the other hand, that applies to a lot of development tools,
especially since most users on Windows these days are actually limited
administrators - ANYTHING they install could run when they elevate a certain
process.

I'm all for encouraging apps to redistribute Python (as most do on Windows). In
this case, they can easily apply the correct security settings, typically by
installing pythonxy.dll with their application and ignoring any global settings
(PYTHONPATH, registry keys, etc.).

On the other hand, Python from python.org is a tool that should only be
installed by those who are prepared to manage it. On Windows it is easy enough
to have a second, secured copy for your admin scripts and an unsecured copy for
non-admin tasks.

> The main rationale given (for not using the standard %ProgramFiles%) has been
> that the full path to python is too long to type, and ease of use is more
> important than the security benefits given by following Windows conventions.
>
> However, adding python to the PATH variable is an alternative solution that
> would result in even fewer keystrokes needing to be typed at a console, while
> maintaining system security.
>
> As this is an installer setting and not a code change, I gather that the
> opportunities for code breakage should be fewer. Please consider updating this
> setting to a more secure and friendly default, for 2.7.7 and 3.5+ as well.

I'm not sure what change you are proposing here... doesn't the installer already
have an option to add to PATH? I'm sure I keep disabling it. I don't want
"python.exe" on my PATH because I have 10+ versions installed at any one time. I
have my own set of aliases because even the py.exe launcher can't handle my
setup :) The info we've gotten from our users (Python Tools for Visual Studio)
shows that most have two or more versions of Python installed, so having one or
both of them in PATH or PATHEXT just adds ambiguity.

The other rationales are .pyc generation and package installation. Unlike PATH,
these are not red herrings, but they do only apply to developers and not end
users (where the developer has done his/her job properly).

Speaking as the one who is likely to take over from Martin as the Windows build
manager (the only people arguing so far are my employer's lawyers, but I'm
winning that argument), I don't see any change that can be made to 2.7.7 apart
from adding a link in the installer to a documentation page on how to secure the
Python installation. At this point, 2.7.6->2.7.7 should be an incredibly safe
upgrade, and there's no way to safely change the default installation location
or the ACLs on the install directory.

As for 3.5, I have some ideas that I will raise with python-dev once I've had a
chance to think through all the issues (think proper per-user installs, redist
modules, etc.). More secure installations would certainly be on the table, but
practicality very easily beats purity here IMHO.

Cheers,
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

[Charles-François Natali]

And incidentally, I think that the argument *processes* should be
renamed to *workers*, or *jobs* (like in make), and any mention of
multiprocessing in the documentation should be removed (if any):
multiprocessing is an implementation detail.
When I type:
make -jN

I don't really care that make is using fork() ;-)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

[Jim J. Jewett]

On Mon, Apr 28, 2014 at 12:56 PM, Charles-François Natali
 wrote:
> Why would the user care if multiprocessing is used behind the scene?

Err ... that was another set of questions that I forgot to ask.

(A)  Why bother raising an error if multiprocessing is unavailable?
After all, there is a perfectly fine fallback...

On other hand, errors should not pass silently.  If a user has
explicitly asked for multiprocessing, there should be some notice that
it didn't happen.  And builds are presumably something that a
developer will monitor to respond to the Exception.

(A1)  What sort of Error?  I'm inclined to raise the original
ImportError, but the patch prefers a ValueError.

(B)  Assuming the exception, I suppose your question adds a 3rd
special case of "Whatever the system suggests, and I don't care
whether or not it involves multiprocessing."

> It would be strange for processes=1 to fail if multiprocessing is not
> available.

As Claudiu pointed out, processes=1 should really mean 1 worker
process, which is still different from "do everything in the main
process".  I'm not sure that level of control is really worth the
complexity, but I'm not certain it isn't.

> processes <= 0: use os.cpu_count()

I could understand doing that for 0 or -1; what is the purpose of
doing it for both, let alone for -4?

Are we at the point where the parameter should just take positive
integers or one of a set of specified string values?

-jJ
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] file objects guarantees

[Charles-François Natali]

Hi,

What's meant exactly by a "file object"?

Let me be more specific: for example, pickle.dump() accepts a "file object".

Looking at the code, it doesn't check the return value of its write() method.

So it assumes that write() should always write the whole data (not
partial write).

Same thing for read, it assumes there won't be short reads.

A sample use case would be passing a socket.makefile() to pickle: it
works, because makefile() returns a BufferedReader/Writer which takes
care of short read/write.

But the documentation just says "file object". And if you have a look
the file object definition in the glossary:
https://docs.python.org/3.5/glossary.html#term-file-object

"""
There are actually three categories of file objects: raw binary files,
buffered binary files and text files. Their interfaces are defined in
the io module. The canonical way to create a file object is by using
the open() function.
"""

So someone passing e.g. a raw binary file - which doesn't handle short
reads/writes - would run into trouble.

It's the same thing for e.g. GzipFile, and probably many others.

Would it make sense to add a note somewhere?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?

[Paul Sokolovsky]

Hello,

It's more or less known fact (ref: google) that you can't inherit from
multiple generic builtin (as in "coded in C") types:

class C(dict, list): pass

TypeError: multiple bases have instance lay-out conflict

However, more detailed googling led me to this page of a book:
http://books.google.com.ua/books?id=JnR9hQA3SncC&pg=PA104&lpg=PA104 ,
which suggest that there might be adhoc support for some native types
to serve as multiple bases together, but it doesn't provide an example.
The book is apparently focused on Python2.

I tried to look at typeobject.c:best_base(), which throws the quoted
error above, and the only special rules I could quickly decipher from
it is that it's possible to multi-inherit from a class and its
subclass. Intuitively, it can be understood - it makes no sense to do
that on the same inheritance level, but can happen with recursive
inheritance, and then there's no need for conflict - both classes can be
"merged" into subclass.

Indeed, following works:

import _collections
class Foo(_collections.defaultdict, dict): pass

(opposite order gives MRO conflict)


So, is that it, or disjoint native types are supported as bases
somehow? Also, would someone know if a class-subclass case happens for
example in stdlib?

As the previous question,
https://mail.python.org/pipermail/python-dev/2014-April/134342.html
this one is to help set adequate requirements for implementing multiple
inheritance in MicroPython.


Thanks,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

[Charles-François Natali]

2014-04-28 18:29 GMT+01:00 Jim J. Jewett :
> On Mon, Apr 28, 2014 at 12:56 PM, Charles-François Natali
>  wrote:
>> Why would the user care if multiprocessing is used behind the scene?
>
> Err ... that was another set of questions that I forgot to ask.
>
> (A)  Why bother raising an error if multiprocessing is unavailable?
> After all, there is a perfectly fine fallback...
>
> On other hand, errors should not pass silently.  If a user has
> explicitly asked for multiprocessing, there should be some notice that
> it didn't happen.  And builds are presumably something that a
> developer will monitor to respond to the Exception.

The point I'm making is that he's not asking for multiprocessing, he's
asking for parallel build.
If you pass 1 (or keep the default value), there's no fallback
involved: the code should simply proceed sequentially.

> (A1)  What sort of Error?  I'm inclined to raise the original
> ImportError, but the patch prefers a ValueError.

NotImplementedError would maybe make sense.

> As Claudiu pointed out, processes=1 should really mean 1 worker
> process, which is still different from "do everything in the main
> process".  I'm not sure that level of control is really worth the
> complexity, but I'm not certain it isn't.

I disagree. If you pass job =1 (and not processes = 1), then you don't
care whether multiprocessing is available or not: you just do
everything in your main process. It would be quite wasteful to create
a single child process!

>> processes <= 0: use os.cpu_count()
>
> I could understand doing that for 0 or -1; what is the purpose of
> doing it for both, let alone for -4?
>
> Are we at the point where the parameter should just take positive
> integers or one of a set of specified string values?

Honestly, as long as it accepts 0, I'm happy.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pep8 reasoning

[Chris Barker]

On Sun, Apr 27, 2014 at 1:40 PM, Barry Warsaw  wrote:

> On Apr 27, 2014, at 12:34 PM, Chris Barker wrote:
> >foo = long_function_name(var_one,
> >  var_two,
> >  var_three,
> >  var_four)
>
> Wow, do you really indent those 42 columns?
>

No -- stupid variable-width font!

I don't think anyone should write code with variable width fonts, and I'd
rather not do email that way either, but gmail is making it tough these
days..

Sorry for the added confusion.

This actually is forbidden because you're not using "vertical alignment"
> when
> there is an argument on the first line.  You would have to line up
> `var_two`
> right under `var_one` to be compliant.


yeah -- that was the intent

oh well.

>That is, I find that if the argument list is too long for one line, then
> >splitting it out to only one argument per line is much more readable to
> me.
>
> Sure.  The PEP outlines ways to do that.


not really -- it allows it:

# Aligned with opening delimiter.
foo = long_function_name(var_one, var_two,
 var_three, var_four)

but all the examples have more than one variable per line...my point is
that I think that should be discouraged.

i.e. I think the above should be:

# Aligned with opening delimiter.
foo = long_function_name(var_one,
 var_two,
 var_three,
 var_four)

(done with fixed-width font this time -- but it may not look right in your
mail reader..)

though I doubt there would consensus on requiring that -- but many of us
learn more from examples than the specification, so maybe I'll submit a
patch with an example like that.

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pep8 reasoning

[Chris Barker]

On Sun, Apr 27, 2014 at 6:34 PM, Steven D'Aprano wrote:

> > I would agree with having at least one example done with one arg per
> line.
>
> Is it really necessary? I think that one-arg-per-line is an obvious
> variation of the existing example.
>

not really -- a lot of folks learn more from following examples. If all teh
examples are the same in some aspect, we'll tend to think that that's
intentional, and that is at least the recommended method


> If the only example given was one-arg-per-line, then the reader might
> wrongly assume that *only* one arg was allowed.


indeed -- but maybe less clear but the the fact that none of the examples
show one argument per line, the implication is that that is discouraged.

very had to document a widely variable standard!

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7.7. on Windows

[Antoine Pitrou]

On Mon, 28 Apr 2014 19:52:48 +1200
Mike Miller  wrote:
> 
> I thought this might be a good time to make a final plea to fix a
> long-standing security issue in the installer on Windows.  By default it
> installs Python to the root folder, thereby bypassing filesystem permissions:
> 
> http://bugs.python.org/issue1284316

Regardless of whether this change this or not, we certainly cannot
change it in a bugfix version. So, 3.5 at the earliest it would be.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] file objects guarantees

[Guido van Rossum]

File objects historically have a pretty vague spec. E.g. it's not defined
which methods are supported beyond read() and readline() (for readers) or
write() (for writers).

Short writes shouldn't be allowed (a regular file object's write() doesn't
even return a value for this reason). This means that a raw (unbuffered)

Short reads *should* be expected, since behavior around these varies
widely. If you see code that doesn't expect them please file bugs.

The glossary item doesn't provide much guidance for would-be implementers
of compliant file file objects, and the interface defined in the io module
is too large (e.g. nobody cares to implement readinto()).

I think we should clarify that raw (unbuffered) file objects are not safe.
I don't care about preventing this explicitly though -- when you see "file
object" you should think "duck typing" and program accordingly. (Reading
and writing are already distinct interfaces; ditto for text vs. bytes.)


On Mon, Apr 28, 2014 at 10:54 AM, Charles-François Natali <
cf.nat...@gmail.com> wrote:

> Hi,
>
> What's meant exactly by a "file object"?
>
> Let me be more specific: for example, pickle.dump() accepts a "file
> object".
>
> Looking at the code, it doesn't check the return value of its write()
> method.
>
> So it assumes that write() should always write the whole data (not
> partial write).
>
> Same thing for read, it assumes there won't be short reads.
>
> A sample use case would be passing a socket.makefile() to pickle: it
> works, because makefile() returns a BufferedReader/Writer which takes
> care of short read/write.
>
> But the documentation just says "file object". And if you have a look
> the file object definition in the glossary:
> https://docs.python.org/3.5/glossary.html#term-file-object
>
> """
> There are actually three categories of file objects: raw binary files,
> buffered binary files and text files. Their interfaces are defined in
> the io module. The canonical way to create a file object is by using
> the open() function.
> """
>
> So someone passing e.g. a raw binary file - which doesn't handle short
> reads/writes - would run into trouble.
>
> It's the same thing for e.g. GzipFile, and probably many others.
>
> Would it make sense to add a note somewhere?
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?

[Antoine Pitrou]

On Mon, 28 Apr 2014 20:45:48 +0300
Paul Sokolovsky  wrote:
> 
> So, is that it, or disjoint native types are supported as bases
> somehow? Also, would someone know if a class-subclass case happens for
> example in stdlib?

Well, for instance this trivial example works:

>>> class C(list, object): pass
... 
>>> 

Basically, if two classes have compatible layouts, you can inherit from
both at once.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?

[Guido van Rossum]

Antoine's example works because list inherits from object. The more general
rule "compatible layout" only allows the rarest of cases to work --
basically the two classes involved must have a common base class and one of
the classes must not add any C-level fields to that base class's layout. I
would never count on this except in cases where this possibility is
documented (e.g. when one class is designed to be a mix-in for the other).


On Mon, Apr 28, 2014 at 11:24 AM, Antoine Pitrou wrote:

> On Mon, 28 Apr 2014 20:45:48 +0300
> Paul Sokolovsky  wrote:
> >
> > So, is that it, or disjoint native types are supported as bases
> > somehow? Also, would someone know if a class-subclass case happens for
> > example in stdlib?
>
> Well, for instance this trivial example works:
>
> >>> class C(list, object): pass
> ...
> >>>
>
> Basically, if two classes have compatible layouts, you can inherit from
> both at once.
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

[Ned Deily]

In article 
,
 "Jim J. Jewett"  wrote:
> As Claudiu pointed out, processes=1 should really mean 1 worker
> process, which is still different from "do everything in the main
> process".  I'm not sure that level of control is really worth the
> complexity, but I'm not certain it isn't.

For regrtest, there is an important difference between "do everything in 
the main process" and "do one test at a time in a subprocess", namely 
the inadvertent global side-effects that running some tests have on 
other tests.  The latter option is much safer and gives more 
reproducible test results.  For compileall, I don't think there is a big 
difference between the two cases such that the regrtest semantics need 
to be followed.

-- 
 Ned Deily,
 n...@acm.org

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?

[Paul Sokolovsky]

Hello,

On Mon, 28 Apr 2014 20:24:58 +0200
Antoine Pitrou  wrote:

> On Mon, 28 Apr 2014 20:45:48 +0300
> Paul Sokolovsky  wrote:
> > 
> > So, is that it, or disjoint native types are supported as bases
> > somehow? Also, would someone know if a class-subclass case happens
> > for example in stdlib?
> 
> Well, for instance this trivial example works:
> 
> >>> class C(list, object): pass
> ... 
> >>> 

Well, it's easy to treat "object" class as a special-case, "null"
class. So, let's re-formulate questions above with "where such
native base classes are not 'object'".

> 
> Basically, if two classes have compatible layouts, you can inherit
> from both at once.

How is "compatible layout" defined? Or "layout" for that matter at
all?

> 
> Regards
> 
> Antoine.


-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?

[Antoine Pitrou]

Hi Paul,

On Mon, 28 Apr 2014 21:42:02 +0300
Paul Sokolovsky  wrote:
> > 
> > Basically, if two classes have compatible layouts, you can inherit
> > from both at once.
> 
> How is "compatible layout" defined? Or "layout" for that matter at
> all?

See Guido's answer. I don't think it's documented anywhere, but you can
find the relevant code somewhere in Objects/typeobject.c (it's quite a
mouthful, though :-)).

(IIRC, "layout" is determined by tp_basicsize, tp_itemsize, the
number of __slots__, and other things perhaps)

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pep8 reasoning

[Barry Warsaw]

On Apr 28, 2014, at 11:12 AM, Chris Barker wrote:

>No -- stupid variable-width font!
>
>I don't think anyone should write code with variable width fonts, and I'd
>rather not do email that way either, but gmail is making it tough these
>days..

Ouch.  I'm sure it's gmail being "helpful" the way
 is similarly helpful.

>> Sure.  The PEP outlines ways to do that.
>
>not really -- it allows it:
>
># Aligned with opening delimiter.
>foo = long_function_name(var_one, var_two,
> var_three, var_four)
>
>but all the examples have more than one variable per line...my point is
>that I think that should be discouraged.
>
>i.e. I think the above should be:
>
># Aligned with opening delimiter.
>foo = long_function_name(var_one,
> var_two,
> var_three,
> var_four)
>
>(done with fixed-width font this time -- but it may not look right in your
>mail reader..)

Fortunately, my mail stack is sane. :)

>though I doubt there would consensus on requiring that -- but many of us
>learn more from examples than the specification, so maybe I'll submit a
>patch with an example like that.

I also very much doubt you'd get consensus on that as a requirement, and I
would oppose the PEP taking a stand one way or the other.  I'm not sure that
the PEP needs an example to illustrate its acceptability.

Cheers,
-Barry


signature.asc
Description: PGP signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?

[Guido van Rossum]

On Mon, Apr 28, 2014 at 11:42 AM, Paul Sokolovsky  wrote:

Well, it's easy to treat "object" class as a special-case, "null"
> class.


But the implementation doesn't, at least not for the question you're asking.


> So, let's re-formulate questions above with "where such
> native base classes are not 'object'".
>
> >
> > Basically, if two classes have compatible layouts, you can inherit
> > from both at once.
>
> How is "compatible layout" defined? Or "layout" for that matter at
> all?
>

The layout is what the C struct defining the object looks like. These are
typically defined in headers in the Include directory (e.g. listobject.h).

The definition of compatible layout is defined by the C code that gives the
error message when it's incompatible. Sorry, I don't recall exactly where
that is, so I recommend that you just look at CPython's source tree. (I
know you have a personal desire not to look at CPython, but I can't help
you without referring to it anyway.)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?

[Guido van Rossum]

On Mon, Apr 28, 2014 at 12:02 PM, Antoine Pitrou wrote:

>
> On Mon, 28 Apr 2014 21:42:02 +0300
> Paul Sokolovsky  wrote:
> > >
> > > Basically, if two classes have compatible layouts, you can inherit
> > > from both at once.
> >
> > How is "compatible layout" defined? Or "layout" for that matter at
> > all?
>
> See Guido's answer. I don't think it's documented anywhere, but you can
> find the relevant code somewhere in Objects/typeobject.c (it's quite a
> mouthful, though :-)).
>
> (IIRC, "layout" is determined by tp_basicsize, tp_itemsize, the
> number of __slots__, and other things perhaps)
>

IIRC the actual inheritance pattern also goes into it. Two structs that
each add an identical new field to a common base class's struct should
*not* be considered compatible.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Commit-ready patches needing review

[Antoine Pitrou]

On Sun, 27 Apr 2014 12:10:46 -0700
Nikolaus Rath  wrote:
> 
> * http://bugs.python.org/issue21057 (TextIOWrapper does not support
>   reading bytearrays or memoryviews)

I've reviewed this one.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7.7. on Windows

[Mike Miller]

On 04/29/2014 05:12 AM, Steve Dower wrote:

This would be an incredibly painful change that would surprise and hurt a lot of
people.


Hi, I think "incredibly painful" is overstating the case a bit. ;)  We're 
talking about an installer default, a setting that would still be changeable as 
it always has, that by definition only will affect brand new installs.



Yes, it is possible for a non-admin user to install arbitrary packages into a
place where an admin user may inadvertently run it, thereby providing escalation
of privilege. On the other hand, that applies to a lot of development tools,
especially since most users on Windows these days are actually limited
administrators - ANYTHING they install could run when they elevate a certain
process.


None of Microsoft's Dev tools install to C:\, rather to ProgramFiles.  The fact 
that another security issue may exist is not a good justification for creating 
additional.



On the other hand, Python from python.org is a tool that should only be
installed by those who are prepared to manage it. On Windows it is easy enough
to have a second, secured copy for your admin scripts and an unsecured copy for
non-admin tasks.


This sounds like the perspective of someone highly technical, forgetting that 
new users will be installing python as well and vastly outnumber us.  "Normal 
people" need help to avoid security issues.


Microsoft's guidelines on where to install software are clear, and don't make 
exceptions that "tools" should be installed to the root of the drive to bypass 
file system permissions, for convenience.



I'm not sure what change you are proposing here... doesn't the installer already
have an option to add to PATH? I'm sure I keep disabling it.


No, it does not.  Unless it got slipped in when I wasn't looking.

It should be an option though, I think everyone would agree.


"python.exe" on my PATH because I have 10+ versions installed at any one time. I


Remember, python-dev's are not the target users of this package, and are a 
rather minuscule fraction of the user base.



Python installation. At this point, 2.7.6->2.7.7 should be an incredibly safe
upgrade, and there's no way to safely change the default installation location


This would continue to be the case, as the installer will recognize the 
previously installed Python 2.7 and use its setting.  This should affect only 
*brand new installs.*



or the ACLs on the install directory.


No ACLs would be affected or changed or even thought about.  Simply installing 
to the correct folder (on new installs) will accomplish the goal.


In short, this design of restricted permissions (read-only) for binaries and 
PATH conveniences goes back decades under Unix and other OS's.  MS Windows has 
finally caught up in the security department in the last few releases.  Please 
don't keep us back in the "bad old days" of DOS where everything was installed 
to the root folder.


--
-Mike

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

[Ezio Melotti]

On Mon, Apr 28, 2014 at 6:32 PM, Jim J. Jewett  wrote:
> (1)  Should fixes to a docstring go in with a patch, even if they
> aren't related to the changing functionality?
>
> [...]
>
> It is best if a commit changes one small thing at a time.
> On the other hand, Nick recently posted that the minimal overhead of a
> patch commit is about half an hour.
>

It could be much less.  As an example,
http://bugs.python.org/issue19943 has been closed 9 minutes after the
report, it didn't have a patch and the fix had to be
applied/grafted/merged on 3 branches.  The time passed between the
first commit and the push is less than a minute too.  Clearly the time
increases if you have to run the test suite or if there is a merge
conflict/push race, and further decreases if there is a simple
typo-fix on default only and a patch already available.

> Is that overhead enough to override the one-issue-per-patch guideline?
>

That said, if the main fix should go only on default and it has a
smaller unrelated fix that should also go on default only, then doing
it together is generally OK (for some value of "unrelated" -- the fix
should still be small and near the code you touched for the main fix).
 If the unrelated fix needs to be ported on all the branches (or in
general in branches where the main fix shouldn't go), it's better to
have two separate patches.  If you commit the smaller fix together
with the bigger one, and then decide to backport it, you will have to
do a null merge and it gets slightly more complicated.


Best Regards,
Ezio Melotti
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

[Claudiu Popa]

This issue raised too much bikeshedding. To wrap it up, I'll modify
the patch with the following:

- processes renamed to workers
- `workers` defaults to 1
- When `workers` is equal to 0, then `os.cpu_count` will be used
- When `workers` > 1, multiple processes will be used
- When `workers` == 1, run normally (no multiple processes)
- Negative values really should raise a ValueError (as
multiprocessing.Pool and soon
  concurrent.futures.Thread/ProcessPoolExecutor)
- Will raise NotImplementedError if multiprocessing can't be used
(when `workers` equals to 0 or > 1)

If anyone agrees with the above, then I'll modify the patch. This will
be its last iteration, any other bikeshedding
should be addressed by the core dev who'll apply it.

Thank you.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7.7. on Windows

[Brian Curtin]

On Mon, Apr 28, 2014 at 3:07 PM, Mike Miller  wrote:
>
> On 04/29/2014 05:12 AM, Steve Dower wrote:
>>
>> This would be an incredibly painful change that would surprise and hurt a
>> lot of
>> people.
>
>
> Hi, I think "incredibly painful" is overstating the case a bit. ;)  We're
> talking about an installer default, a setting that would still be changeable
> as it always has, that by definition only will affect brand new installs.
>
>
>> Yes, it is possible for a non-admin user to install arbitrary packages
>> into a
>> place where an admin user may inadvertently run it, thereby providing
>> escalation
>> of privilege. On the other hand, that applies to a lot of development
>> tools,
>> especially since most users on Windows these days are actually limited
>> administrators - ANYTHING they install could run when they elevate a
>> certain
>> process.
>
>
> None of Microsoft's Dev tools install to C:\, rather to ProgramFiles.  The
> fact that another security issue may exist is not a good justification for
> creating additional.
>
>
>> On the other hand, Python from python.org is a tool that should only be
>> installed by those who are prepared to manage it. On Windows it is easy
>> enough
>> to have a second, secured copy for your admin scripts and an unsecured
>> copy for
>> non-admin tasks.
>
>
> This sounds like the perspective of someone highly technical, forgetting
> that new users will be installing python as well and vastly outnumber us.
> "Normal people" need help to avoid security issues.
>
> Microsoft's guidelines on where to install software are clear, and don't
> make exceptions that "tools" should be installed to the root of the drive to
> bypass file system permissions, for convenience.
>
>
>> I'm not sure what change you are proposing here... doesn't the installer
>> already
>> have an option to add to PATH? I'm sure I keep disabling it.
>
>
> No, it does not.  Unless it got slipped in when I wasn't looking.
>
> It should be an option though, I think everyone would agree.

The option to add the current install to your path was added 3.3.

>> "python.exe" on my PATH because I have 10+ versions installed at any one
>> time. I
>
>
> Remember, python-dev's are not the target users of this package, and are a
> rather minuscule fraction of the user base.

Knowing which Python you want on your path and that you want it on
your path at all is somewhat of an advanced usage. While beginners do
want to just open up cmd and type "python" and have it work, there are
better ways to accomplish that which don't involve system-wide path
manipulation or installation changes.

Several PC manufacturers have been known to use Python for various
system utilities, which is how Python sometimes ends up in the path on
your grandma's Dell*. Even for a beginner who just wants "python" to
work, we need to be careful to not wreck their entire system by
helping them get our fresh Python install to show up.

A more reasonable way to help beginners would be to create a shortcut
somewhere that starts up cmd with a modified path. They can do
whatever they want to do within that context without modifying their
entire system. If they learn that they later want their system-wide
path manipulated, they can do that within the installer or will known
how to do that themselves.

* watch Dave Beazley's PyCon 2014 talk for a good story involving one
of those manufacturer installed Pythons:
https://www.youtube.com/watch?v=RZ4Sn-Y7AP8
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7.7. on Windows

[Mike Miller]

Hi,

Those bugs were fixed 9 years ago or so around 2005. I use python in 
ProgramFiles every day without issue.  If another bug has crept in somewhere, it 
can be fixed.


-Mike



--
From: Sturla Molden 
To: python-dev@python.org
Subject: Re: [Python-Dev] Python 2.7.7. on Windows
Message-ID:
<771463774420395726.352685sturla.molden-gmail@news.gmane.org>

"C:\Program Files\Python27" contains an empty space in the path. If you
want to randomly break build tools for C extensions, then go ahead and
change it.

Sturla

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-Dev Digest, Vol 129, Issue 81

[Mike Miller]

Hi, note the pep, it makes allowances for security enhancements.

-Mike

> Message: 5 Date: Mon, 28 Apr 2014 20:23:12 +0200
> From: Antoine Pitrou 
> To: python-dev@python.org Subject:
> Re: [Python-Dev] Python 2.7.7. on Windows
> Message-ID: <20140428202312.6903d62a@fsol>

> Regardless of whether this change this or not, we certainly cannot
> change it in a bugfix version. So, 3.5 at the earliest it would be.

> Regards

> Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7.7. on Windows

[Mike Miller]

On 04/29/2014 08:38 AM, Brian Curtin wrote:

The option to add the current install to your path was added 3.3.


Ok, thanks.  So there is some precedent it would be useful.


Remember, python-dev's are not the target users of this package, and are a
rather minuscule fraction of the user base.


Knowing which Python you want on your path and that you want it on
your path at all is somewhat of an advanced usage.

> ...

They can do
whatever they want to do within that context without modifying their
entire system. If they learn that they later want their system-wide
path manipulated...


Installing python at all is a "system-wide" change.  Why not default to a known 
safe, ready to use configuration, rather than a "tool-box" that needs assembly?



* watch Dave Beazley's PyCon 2014 talk for a good story involving one
of those manufacturer installed Pythons:
https://www.youtube.com/watch?v=RZ4Sn-Y7AP8


Thanks, I'm trying to get thru all the talk will watch that shortly. ;)

Still, edge cases regarding manufactures should not override the needs of the 
majority of users.


-Mike
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-Dev Digest, Vol 129, Issue 81

[Brett Cannon]

On Mon Apr 28 2014 at 4:58:35 PM, Mike Miller 
wrote:

> Hi, note the pep, it makes allowances for security enhancements.
>

The PEP in question is about fixing fundamentally broken security issues in
Python 2.7 (e.g. updating OpenSSL). Tweaking where Python is installed by
default on Windows is not fundamentally broken, it's a difference of
opinion as to whether the current defaults are the best option or not.

-Brett


>
> -Mike
>
>  > Message: 5 Date: Mon, 28 Apr 2014 20:23:12 +0200
>  > From: Antoine Pitrou 
>  > To: python-dev@python.org Subject:
>  > Re: [Python-Dev] Python 2.7.7. on Windows
>  > Message-ID: <20140428202312.6903d62a@fsol>
>
>  > Regardless of whether this change this or not, we certainly cannot
>  > change it in a bugfix version. So, 3.5 at the earliest it would be.
>
>  > Regards
>
>  > Antoine.
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

[Terry Reedy]

On 4/28/2014 4:24 PM, Claudiu Popa wrote:

This issue raised too much bikeshedding. To wrap it up, I'll modify
the patch with the following:

- processes renamed to workers
- `workers` defaults to 1
- When `workers` is equal to 0, then `os.cpu_count` will be used
- When `workers` > 1, multiple processes will be used
- When `workers` == 1, run normally (no multiple processes)


I presume you mean for this to be the default.


- Negative values really should raise a ValueError (as
multiprocessing.Pool and soon
   concurrent.futures.Thread/ProcessPoolExecutor)


Yay!


- Will raise NotImplementedError if multiprocessing can't be used
(when `workers` equals to 0 or > 1)

If anyone agrees with the above, then I'll modify the patch.


Having read the thread, though not much of the issue, these look to me 
like the best choices for a clean, comprehensible API.


> This will

be its last iteration, any other bikeshedding
should be addressed by the core dev who'll apply it.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pep8 reasoning

[Terry Reedy]

On 4/28/2014 2:12 PM, Chris Barker wrote:


I don't think anyone should write code with variable width fonts,


The problem is that fixed pitch does not work well for even a half-way 
complete unicode font and I don't know that there are any available. As 
far as I know, my Windows 7 only came with one unicode font that Idle 
finds. An advantage from my viewpoint is that spaces in this font are 
narrower that the average character, making multiple-of-4 indents and 
spaces around operators look pretty good to me.


Given that PEP8 generally disallows adding spaces to make things line 
up, as in the following (a style I used  to adhere to)


a = 134543344  # big number
bonge = 2  # short number

it seems to me consistent to use the multiple-of-4 hanging indent style 
and not try to make things line up with an odd number of spaces in this 
one special case.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python-Dev Digest, Vol 129, Issue 81

[Terry Reedy]

On 4/28/2014 5:01 PM, Brett Cannon wrote:



On Mon Apr 28 2014 at 4:58:35 PM, Mike Miller mailto:python-...@mgmiller.net>> wrote:

Hi, note the pep, it makes allowances for security enhancements.


The PEP in question is about fixing fundamentally broken security issues
in Python 2.7 (e.g. updating OpenSSL). Tweaking where Python is
installed by default on Windows is not fundamentally broken, it's a
difference of opinion as to whether the current defaults are the best
option or not.


Besides which, the PEP was cut down to a specific list of changes. Any 
more would need a separate dicussion.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

[R. David Murray]

On Mon, 28 Apr 2014 23:24:16 +0300, Claudiu Popa  wrote:
> - Will raise NotImplementedError if multiprocessing can't be used
> (when `workers` equals to 0 or > 1)

I think the most common use case for this ability will be "run with
the appropriate number of processes for the system I'm on", where
'the appropriate number' is 1 (the main process) if multiprocessing
is not available.  Otherwise the tool calling compileall would have to
figure out how to "catch the error" (how do you do that when invoking
a CLI?) and re-run the script using '1` itself.

How you spell this I don't really care, but I think the above is the
most common use case.

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pep8 reasoning

[R. David Murray]

On Mon, 28 Apr 2014 18:01:32 -0400, Terry Reedy  wrote:
> On 4/28/2014 2:12 PM, Chris Barker wrote:
> 
> > I don't think anyone should write code with variable width fonts,
> 
> The problem is that fixed pitch does not work well for even a half-way 
> complete unicode font and I don't know that there are any available. As 
> far as I know, my Windows 7 only came with one unicode font that Idle 
> finds. An advantage from my viewpoint is that spaces in this font are 
> narrower that the average character, making multiple-of-4 indents and 
> spaces around operators look pretty good to me.

My unix fix-width terminal font handles most unicode (even a lot of
non-bmp stuff...though I have no idea if it is readable :).

I have no idea what font it is, to tell you the truth.

--David
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pep8 reasoning

[Chris Barker]

On Mon, Apr 28, 2014 at 3:01 PM, Terry Reedy  wrote:

> I don't think anyone should write code with variable width fonts,
>>
>
> The problem is that fixed pitch does not work well for even a half-way
> complete unicode font and I don't know that there are any available.
>
...

> Given that PEP8 generally disallows adding spaces to make things line up,
> as in the following (a style I used  to adhere to)
>
> a = 134543344  # big number
> bonge = 2  # short number
>

Is ironic that that looks like hell in my variable-width font email client?

it seems to me consistent to use the multiple-of-4 hanging indent style and
> not try to make things line up with an odd number of spaces in this one
> special case.


things aren't going to generally line up with a non-fixed width font no
matter how you slice it, so I guess if you need to use one for your code,
you're just stuck. But I suppose the "line up with the opening bracket"
style isn't a good idea in that case. All the more reason not to
standardize anymore that PEP 8 already does.

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pep8 reasoning

[Terry Reedy]

On 4/28/2014 7:13 PM, Chris Barker wrote:

On Mon, Apr 28, 2014 at 3:01 PM, Terry Reedy mailto:tjre...@udel.edu>> wrote:

I don't think anyone should write code with variable width fonts,

The problem is that fixed pitch does not work well for even a
half-way complete unicode font and I don't know that there are any
available.
...

Given that PEP8 generally disallows adding spaces to make things
line up, as in the following (a style I used  to adhere to)

a = 134543344  # big number
bonge = 2  # short number

Is ironic that that looks like hell in my variable-width font email client?


I believe that has been given as a reason to not do what I used to do. 
The only things guaranteed to actually line up are the initial graphic 
characters of lines with the exact same indent.


tjr


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7.7. on Windows

[Chris Barker]

On Mon, Apr 28, 2014 at 1:56 PM, Mike Miller wrote:

> * watch Dave Beazley's PyCon 2014 talk for a good story involving one
>
>> of those manufacturer installed Pythons:
>> https://www.youtube.com/watch?v=RZ4Sn-Y7AP8
>>
>
> Thanks, I'm trying to get thru all the talk will watch that shortly. ;)
>

fun talk -- but start at about 18:00 minutes if you want the  relevant
bit...

By the way, some people (or at least ESRI, with ArcMap) Install and use
their own python in: C:\python27 -- if you dont want to interat with that,
you need to put it somewhere else!

I'd argue that any OEM or third party app that uses the standard location
or puts it on the PATH be default is in error, but what can you so?

Redhat used to have #!/usr/bin/env python on their admin scripts, which was
really painful if you wanted to upgrade anything. Don't know it they've
fixed that

-Chris


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Multiple inheritance from builtin (C) types [still] supported in Python3?

[Paul Sokolovsky]

Hello,

On Mon, 28 Apr 2014 12:08:40 -0700
Guido van Rossum  wrote:

[]

> > How is "compatible layout" defined? Or "layout" for that matter at
> > all?
> >
> 
> The layout is what the C struct defining the object looks like. These
> are typically defined in headers in the Include directory (e.g.
> listobject.h).
> 
> The definition of compatible layout is defined by the C code that
> gives the error message when it's incompatible. Sorry, I don't recall
> exactly where that is, so I recommend that you just look at CPython's
> source tree. (I know you have a personal desire not to look at
> CPython, but I can't help you without referring to it anyway.)

Well, sure I did, as I mentioned, but as that's first time I see that
code (that specific piece is in typeobject.c:extra_ivars()), it would
take quite some time be sure I understand all aspects of it. Thanks for
confirming that it's governed essentially by CPython implementation
details and not some language-level semantics like metaclasses (I
mentioned them because error message in Python2 did so, though Python3
doesn't refer to metaclasses).

An example would really help me to get a feel of the issue, but I
assume lack of them means that there's no well-known idiom where such
inheritance is used, and that's good enough on its own. I also tried to
figure how it's important to support such multi-base cases, so the code
I write didn't require complete rewrite if it hits one day, but
everything seems to turn out to be pretty extensible.

> 
> -- 
> --Guido van Rossum (python.org/~guido)

Thanks!

-- 
Best regards,
 Paul  mailto:pmis...@gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7.7. on Windows

[Steve Dower]

> Mike Miller wrote:
> On 04/29/2014 05:12 AM, Steve Dower wrote:
>> This would be an incredibly painful change that would surprise and
>> hurt a lot of people.
> 
> Hi, I think "incredibly painful" is overstating the case a bit. ;) We're 
> talking
> about an installer default, a setting that would still be changeable as it
> always has, that by definition only will affect brand new installs.

Good point about it only affecting new installs, though that still constitutes 
a lot of installs.
 
>> Yes, it is possible for a non-admin user to install arbitrary packages
>> into a place where an admin user may inadvertently run it, thereby
>> providing escalation of privilege. On the other hand, that applies to
>> a lot of development tools, especially since most users on Windows
>> these days are actually limited administrators - ANYTHING they install
>> could run when they elevate a certain process.
> 
> None of Microsoft's Dev tools install to C:\, rather to ProgramFiles. The fact
> that another security issue may exist is not a good justification for creating
> additional.

The fact that the mitigations are well known by the people who have to worry 
about them is a good justification for not creating a compatibility issue. It's 
easy for IT admins to install Python in a way that the files are read-only, the 
.pyc and .pyo files are already there, and user site packages will be used by 
default (I think that last one is easy?). The good IT admins even know that 
they need to do this - perhaps we can help educate the bad admins? (FWIW, if 
you have admin privileges on your own machine, YOU are the IT admin. Are you a 
good one or a bad one?)

>> On the other hand, Python from python.org is a tool that should only
>> be installed by those who are prepared to manage it. On Windows it is
>> easy enough to have a second, secured copy for your admin scripts and
>> an unsecured copy for non-admin tasks.
> 
> This sounds like the perspective of someone highly technical, forgetting that
> new users will be installing python as well and vastly outnumber us. "Normal
> people" need help to avoid security issues.

One place where mitigations are added are the distributions (Canopy, Anaconda, 
etc.). These packages redistribute Python and install to different locations 
with different permissions (I'm not promising that they all do it properly, but 
they have the opportunity to do so). The reference implementation of CPython is 
typically not the best option for "normal people", who are much better served 
by one of these bundles.

> Microsoft's guidelines on where to install software are clear, and don't make
> exceptions that "tools" should be installed to the root of the drive to bypass
> file system permissions, for convenience.

I'm well aware of the guidelines, hence the practicality vs. purity comment. 
I'm fairly certain that the installation to the root was originally about ease 
of command-line access rather than bypassing permissions - the permissions 
probably didn't exist for the first few versions of Python (Python for DOS 
obviously didn't care... maybe it's always been about backwards compatibility?)

At this point, installing into the root is about not breaking everyone who 
*knows* that Python installs into the root directory and always has. 

>> I'm not sure what change you are proposing here... doesn't the
>> installer already have an option to add to PATH? I'm sure I keep disabling 
>> it.
> 
> No, it does not. Unless it got slipped in when I wasn't looking.
> 
> It should be an option though, I think everyone would agree.

Thanks Brett for pointing out that it arrived in Python 3. I'm sure it would be 
an acceptable addition to 2.7.7, though you'd need to get it in before RC and 
you'd also need to find someone who is keen and able to keep making 2.7 
installers for Windows. Right now, we don't have anyone who is both.

>> "python.exe" on my PATH because I have 10+ versions installed at any
>> one time.
> 
> Remember, python-dev's are not the target users of this package, and are a
> rather minuscule fraction of the user base.

My day job revolves around making Python easier to use for non "python-dev"s, 
especially those who have multiple versions of Python installed simultaneously 
(as I mentioned, over 50% of our user base). Having PATH or PATHEXT set 
globally is a source of much confusion. (Though a global PYTHONPATH causes more 
problems, but that isn't part of this discussion.)

>> Python installation. At this point, 2.7.6->2.7.7 should be an
>> incredibly safe upgrade, and there's no way to safely change the
>> default installation location
> 
> This would continue to be the case, as the installer will recognize the
> previously installed Python 2.7 and use its setting. This should affect only
> *brand new installs.*

For example, next time you redeploy your hosted service on a clean VM with the 
latest security patches? The next time someone does a clean uninstall-reinstall 
because they've

Re: [Python-Dev] Python 2.7.7. on Windows

[Brian Curtin]

On Mon, Apr 28, 2014 at 7:04 PM, Steve Dower  wrote:
>> Mike Miller wrote:
>> On 04/29/2014 05:12 AM, Steve Dower wrote:
>>> This would be an incredibly painful change that would surprise and
>>> hurt a lot of people.
>>
>> Hi, I think "incredibly painful" is overstating the case a bit. ;) We're 
>> talking
>> about an installer default, a setting that would still be changeable as it
>> always has, that by definition only will affect brand new installs.
>
> Good point about it only affecting new installs, though that still 
> constitutes a lot of installs.
>
>>> Yes, it is possible for a non-admin user to install arbitrary packages
>>> into a place where an admin user may inadvertently run it, thereby
>>> providing escalation of privilege. On the other hand, that applies to
>>> a lot of development tools, especially since most users on Windows
>>> these days are actually limited administrators - ANYTHING they install
>>> could run when they elevate a certain process.
>>
>> None of Microsoft's Dev tools install to C:\, rather to ProgramFiles. The 
>> fact
>> that another security issue may exist is not a good justification for 
>> creating
>> additional.
>
> The fact that the mitigations are well known by the people who have to worry 
> about them is a good justification for not creating a compatibility issue. 
> It's easy for IT admins to install Python in a way that the files are 
> read-only, the .pyc and .pyo files are already there, and user site packages 
> will be used by default (I think that last one is easy?). The good IT admins 
> even know that they need to do this - perhaps we can help educate the bad 
> admins? (FWIW, if you have admin privileges on your own machine, YOU are the 
> IT admin. Are you a good one or a bad one?)
>
>>> On the other hand, Python from python.org is a tool that should only
>>> be installed by those who are prepared to manage it. On Windows it is
>>> easy enough to have a second, secured copy for your admin scripts and
>>> an unsecured copy for non-admin tasks.
>>
>> This sounds like the perspective of someone highly technical, forgetting that
>> new users will be installing python as well and vastly outnumber us. "Normal
>> people" need help to avoid security issues.
>
> One place where mitigations are added are the distributions (Canopy, 
> Anaconda, etc.). These packages redistribute Python and install to different 
> locations with different permissions (I'm not promising that they all do it 
> properly, but they have the opportunity to do so). The reference 
> implementation of CPython is typically not the best option for "normal 
> people", who are much better served by one of these bundles.
>
>> Microsoft's guidelines on where to install software are clear, and don't make
>> exceptions that "tools" should be installed to the root of the drive to 
>> bypass
>> file system permissions, for convenience.
>
> I'm well aware of the guidelines, hence the practicality vs. purity comment. 
> I'm fairly certain that the installation to the root was originally about 
> ease of command-line access rather than bypassing permissions - the 
> permissions probably didn't exist for the first few versions of Python 
> (Python for DOS obviously didn't care... maybe it's always been about 
> backwards compatibility?)
>
> At this point, installing into the root is about not breaking everyone who 
> *knows* that Python installs into the root directory and always has.
>
>>> I'm not sure what change you are proposing here... doesn't the
>>> installer already have an option to add to PATH? I'm sure I keep disabling 
>>> it.
>>
>> No, it does not. Unless it got slipped in when I wasn't looking.
>>
>> It should be an option though, I think everyone would agree.
>
> Thanks Brett for pointing out that it arrived in Python 3. I'm sure it would 
> be an acceptable addition to 2.7.7, though you'd need to get it in before RC 
> and you'd also need to find someone who is keen and able to keep making 2.7 
> installers for Windows. Right now, we don't have anyone who is both.

If it's an acceptable change to the release manager (Benjamin?), and
if there's actually time before the RC (I don't know when it is
planned), I am willing to backport my 3.3 change to get this in the
2.7 installer.

However, I'm not currently setup to make release installers -- I think
I need a signing certificate or something like that.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7.7. on Windows

[Benjamin Peterson]

On Mon, Apr 28, 2014, at 17:14, Brian Curtin wrote:
> If it's an acceptable change to the release manager (Benjamin?), and
> if there's actually time before the RC (I don't know when it is
> planned), I am willing to backport my 3.3 change to get this in the
> 2.7 installer.

That's fine.

> 
> However, I'm not currently setup to make release installers -- I think
> I need a signing certificate or something like that.

Apparently the current PSF one just expired, so we're in the process of
procuring a new one.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pep8 reasoning

[Stephen J. Turnbull]

R. David Murray writes:

 > My unix fix-width terminal font handles most unicode (even a lot of
 > non-bmp stuff...though I have no idea if it is readable :).

Oh, I bet you do.  With a true fixed-width Unicode font, it's the
*Latin-character* text that's painful, if not unreadable, because the
aspect ratio needs to be close to 1.0 to handle ideographs, rather
than the 0.5 or less common with Latin fonts.

XTerm has an option to treat ideographs as double width.  That does
the trick (at least, I don't know any scripts that really really want
an aspect ratio of 0.7 or 2.0).

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] pep8 reasoning

[Stephen J. Turnbull]

 > On Mon, Apr 28, 2014 at 11:12 AM, Chris Barker wrote:

 > not really -- it allows it:
 > 
 > # Aligned with opening delimiter.
 > foo = long_function_name(var_one, var_two,
 >  var_three, var_four)
 > 
 > but all the examples have more than one variable per line...my point is
 > that I think that should be discouraged.
 > 
 > i.e. I think the above should be:
 > 
 > # Aligned with opening delimiter.
 > foo = long_function_name(var_one,
 >  var_two,
 >  var_three,
 >  var_four)

I don't have a problem with discouraging it, but it should not be
flagged in pep8 (OK in pep8 --wink-wink-nudge-nudge-say-no-more-eh,
though!)

I usually do use the style you prefer, especially when the arguments
want to be commented, but in many cases the arguments are semantically
tuples.  Eg,

rect1 = paint_rectangle(point[1], point[0],# top, left
textheight + 2 * padding, textwidth + 2 * padding,
chartreuse)

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7.7. on Windows

[Stephen J. Turnbull]

Mike Miller writes:

 > Microsoft's guidelines on where to install software are clear, and
 > don't make exceptions that "tools" should be installed to the root
 > of the drive to bypass file system permissions, for convenience.

But there's the rub.  In this case, Microsoft doesn't have *security*,
it has "guidelines".  They are *still* guidelines, not security,
*exactly* because it's convenient for somebody.  The fact that taking
advantage of that convenience has the side effect of bypassing
filesystem permissions is unfortunate (and a bug in Windows IMO).

Note that if users actually paid attention to these guidelines, we'd
be getting complaints from *them*, not from you.  I don't recall ever
seeing that.  That implies that "normal users" will install anything
anwhere anyway.

If it's that unimportant to Microsoft, I see insufficient reason why
we should risk confusing those "normal users" who already have Python
2.7 installed (and as pointed out, they *are* at risk precisely
because the proposal changes the default install location).
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] bytes with trailing \0?

[Nikolaus Rath]

Hello,

I was surprised to find the following in bytesobject.c:

,
| [...]
|As always, an extra byte is allocated for a trailing \0 byte (newsize
|does *not* include that), and a trailing \0 byte is stored.
| */
| 
| int
| _PyBytes_Resize(PyObject **pv, Py_ssize_t newsize)
| {
| [...]
| 
`

Does this mean that bytes objects are internally stored with a trailing
\0? Why is that? Isn't that just wasting a byte, because \0 might also
be in the middle of the byte sequence, and the bytes objects stores its
length explicitly anyway?


Best,
-Nikolaus
-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

 »Time flies like an arrow, fruit flies like a Banana.«
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7.7. on Windows

[Steven D'Aprano]

On Tue, Apr 29, 2014 at 12:07:00PM +0900, Stephen J. Turnbull wrote:
> Mike Miller writes:
> 
>  > Microsoft's guidelines on where to install software are clear, and
>  > don't make exceptions that "tools" should be installed to the root
>  > of the drive to bypass file system permissions, for convenience.
> 
> But there's the rub.  In this case, Microsoft doesn't have *security*,
> it has "guidelines".  They are *still* guidelines, not security,
> *exactly* because it's convenient for somebody.  The fact that taking
> advantage of that convenience has the side effect of bypassing
> filesystem permissions is unfortunate (and a bug in Windows IMO).
> 
> Note that if users actually paid attention to these guidelines, we'd
> be getting complaints from *them*, not from you.  I don't recall ever
> seeing that.  That implies that "normal users" will install anything
> anwhere anyway.

I don't think that argument is terribly useful. If people waited for 
"normal users" to complain before doing something about Heartbeat, we'd 
be in a pretty pickle. "Normal users" don't understand the technology 
well enough to have a valid threat model or judge the consequences, and 
they are confused by a mixture of ignorance, misinformation and hype. 
It's up to technical users to lead, not to follow.


> If it's that unimportant to Microsoft, 

I think that's unfair. I'm not a MS fan, not even close. I think their 
business practices in the past have been reprehensible. But if there is 
anyone who takes backwards-compatibility even more seriously than 
Python-Dev, it is them. You should give them the courtesy of assuming 
that their decision is not based on apathy, but on *exactly* the same 
reasoning that *you* apply below:

> I see insufficient reason why
> we should risk confusing those "normal users" who already have Python
> 2.7 installed (and as pointed out, they *are* at risk precisely
> because the proposal changes the default install location).

And thus security vulnerabilities never get fixed :-)

I don't have an opinion on the importance or magnitude of this security 
vulnerability, the risk of confusion, or whether it should be fixed or 
not. But I wonder why the installer is ignoring the OS's guidelines for 
where software should be installed? If this were Apple we were talking 
about, would we ignore their guidelines? Or on Linux, would we blithly 
install Python in / instead of (say) /usr/local/bin? I don't think so. I 
would have thought that the mere fact that Microsoft disapproves of 
installing applications into the root should be good enough reason to 
not do it. In the absense of an extremely compelling reason not to do 
so, we should be a "good citizen" regardless of the OS.



-- 
Steven
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7.7. on Windows

[Glenn Linderman]

On 4/28/2014 8:52 PM, Steven D'Aprano wrote:

I think that's unfair. I'm not a MS fan, not even close. I think their
business practices in the past have been reprehensible. But if there is
anyone who takes backwards-compatibility even more seriously than
Python-Dev, it is them.
I guess there is no one who takes backwards-compatibility even more 
seriously than Python-Dev.


I have perfectly good DOS and Windows programs that no longer run on 
supported Windows versions. XP->Vista was a bigger break than Python 
2->3, from my perspective.


And then if you get into the command line programs that have vanished or 
take different options, breaking batch files, and similar facilities, it 
gets even worse.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] bytes with trailing \0?

[Guido van Rossum]

It is to improve the experience of passing bytes to a C function that
expects a trailing \0. For example syscalls taking filenames. The wrapper
must still check for embedded \0 but the bytes don't need to be copied.

On Monday, April 28, 2014, Nikolaus Rath  wrote:

> Hello,
>
> I was surprised to find the following in bytesobject.c:
>
> ,
> | [...]
> |As always, an extra byte is allocated for a trailing \0 byte (newsize
> |does *not* include that), and a trailing \0 byte is stored.
> | */
> |
> | int
> | _PyBytes_Resize(PyObject **pv, Py_ssize_t newsize)
> | {
> | [...]
> |
> `
>
> Does this mean that bytes objects are internally stored with a trailing
> \0? Why is that? Isn't that just wasting a byte, because \0 might also
> be in the middle of the byte sequence, and the bytes objects stores its
> length explicitly anyway?
>
>
> Best,
> -Nikolaus
> --
> GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
> Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
>
>  »Time flies like an arrow, fruit flies like a Banana.«
> ___
> Python-Dev mailing list
> Python-Dev@python.org 
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (on iPad)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] API and process questions (sparked by Claudiu Popa on 16104

[Nick Coghlan]

On 28 Apr 2014 16:19, "Ezio Melotti"  wrote:
>
> On Mon, Apr 28, 2014 at 6:32 PM, Jim J. Jewett 
wrote:
> > (1)  Should fixes to a docstring go in with a patch, even if they
> > aren't related to the changing functionality?
> >
> > [...]
> >
> > It is best if a commit changes one small thing at a time.
> > On the other hand, Nick recently posted that the minimal overhead of a
> > patch commit is about half an hour.
> >
>
> It could be much less.  As an example,
> http://bugs.python.org/issue19943 has been closed 9 minutes after the
> report, it didn't have a patch and the fix had to be
> applied/grafted/merged on 3 branches.  The time passed between the
> first commit and the push is less than a minute too.  Clearly the time
> increases if you have to run the test suite or if there is a merge
> conflict/push race, and further decreases if there is a simple
> typo-fix on default only and a patch already available.

The 30 minute guesstimate was for doing it right for a patch you didn't
write yourself:
- final recheck that all the required pieces are there
- patch import & merge commands
- run the tests for every affected branch
- coming up with the NEWS entry & commit message

It's generally much, much faster for changes I write myself - another
undesirable incentive since it further encourages us to work on our own
changes over incorporating patches from others.

> > Is that overhead enough to override the one-issue-per-patch guideline?
>
> That said, if the main fix should go only on default and it has a
> smaller unrelated fix that should also go on default only, then doing
> it together is generally OK (for some value of "unrelated" -- the fix
> should still be small and near the code you touched for the main fix).
>  If the unrelated fix needs to be ported on all the branches (or in
> general in branches where the main fix shouldn't go), it's better to
> have two separate patches.  If you commit the smaller fix together
> with the bigger one, and then decide to backport it, you will have to
> do a null merge and it gets slightly more complicated.

What Ezio said :)

I definitely do coarser commits for CPython than I do for beaker-project.org,
specifically because Gerrit provides decent tools for reviewing and merging
a patch series, and we don't currently have anything like that in the
CPython workflow.

Cheers,
Nick.

>
>
> Best Regards,
> Ezio Melotti
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Python 2.7.7. on Windows

[Stephen J. Turnbull]

Steven D'Aprano writes:
 > On Tue, Apr 29, 2014 at 12:07:00PM +0900, Stephen J. Turnbull wrote:

 > > Note that if users actually paid attention to these guidelines, we'd
 > > be getting complaints from *them*, not from you.  I don't recall ever
 > > seeing that.  That implies that "normal users" will install anything
 > > anwhere anyway.
 > 
 > I don't think that argument is terribly useful. If people waited for 
 > "normal users"

I'm not suggesting that we wait for such reports in any case.  My
point is that the "security hole" is apparently wide open and nobody,
not Microsoft and not corporate IT, is doing anything to close it.  If
it were, it would inconvenience users and we'd hear about it.  I infer
they assess the threat to be essentially zero.

 > > If it's that unimportant to Microsoft, 
 > 
 > I think that's unfair.

What's unfair?  Mr. Miller is evidently concerned about users who have
completely left their security up to their OS vendor.  Isn't reference
to the importance of an alleged security hole to that vendor both fair
and relevant to a decision to make a backwards-incompatible change in
a Python bugfix release, even in the installer?

 > You should give them the courtesy of assuming that their decision
 > is not based on apathy,

My point is that Microsoft has *not* made a decision, but left it up
to anybody who has software they hope Windows users will install --
both Python-Dev and crackers.  I infer they do not consider this a
security issue worthy of notice.

 > And thus security vulnerabilities never get fixed :-)

I have no objection *at all* to making the change in the next feature
release.  I think the "good citizenship" argument is more than
sufficient, but of course I'll leave it up to the release manager.  As
for bugfix releases, given the arguments above, I want a stronger
argument than "Microsoft guidelines", that's all.  I don't even ask
for a CVE. :-)

 > I would have thought that the mere fact that Microsoft disapproves
 > of installing applications into the root should be good enough
 > reason to not do it.

I'm not defending the failure to follow the guideline in the Python
2.x.0 releases (IIUC the guideline pre-existed 2.7).  I'm questioning
whether it is a sufficient reason to make a backwards-incompatible
change in a bugfix release.

My take is that Microsoft itself doesn't think it's very important.

Regards,
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com