Re: [Python-Dev] Multiline with statement line continuation
On 12 August 2014 22:15, Steven D'Aprano wrote:
> Compare the natural way of writing this:
>
> with open("spam") as spam, open("eggs", "w") as eggs, frobulate("cheese") as
> cheese:
> # do stuff with spam, eggs, cheese
>
> versus the dynamic way:
>
> with ExitStack() as stack:
> spam, eggs = [stack.enter_context(open(fname), mode) for fname, mode in
> zip(("spam", "eggs"), ("r", "w")]
> cheese = stack.enter_context(frobulate("cheese"))
> # do stuff with spam, eggs, cheese
You wouldn't necessarily switch at three. At only three, you have lots
of options, including multiple nested with statements:
with open("spam") as spam:
with open("eggs", "w") as eggs:
with frobulate("cheese") as cheese:
# do stuff with spam, eggs, cheese
The "multiple context managers in one with statement" form is there
*solely* to save indentation levels, and overuse can often be a sign
that you may have a custom context manager trying to get out:
@contextlib.contextmanager
def dish(spam_file, egg_file, topping):
with open(spam_file), open(egg_file, 'w'), frobulate(topping):
yield
with dish("spam", "eggs", "cheese") as spam, eggs, cheese:
# do stuff with spam, eggs & cheese
ExitStack is mostly useful as a tool for writing flexible custom
context managers, and for dealing with context managers in cases where
lexical scoping doesn't necessarily work, rather than being something
you'd regularly use for inline code.
"Why do I have so many contexts open at once in this function?" is a
question developers should ask themselves in the same way its worth
asking "why do I have so many local variables in this function?"
Regards,
Nick.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reviving restricted mode?
On Mon, 11 Aug 2014, Skip Montanaro wrote: On Mon, Aug 11, 2014 at 12:42 PM, matsjoyce wrote: There maybe some holes in my approach, but I can't find them. There's the rub. Given time, I suspect someone will discover a hole or two. Schneier's Law: Any person can invent a security system so clever that she or he can't think of how to break it. While I would not claim a Python sandbox is utterly impossible, I'm suspicious that the whole "consenting adults" approach in Python is incompatible with a sandbox. The whole idea of a sandbox is to absolutely prevent people from doing things even if they really want to and know what they are doing. Isaac Morland CSCF Web Guru DC 2554C, x36650WWW Software Specialist ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiline with statement line continuation
Nick Coghlan writes:
> On 12 August 2014 22:15, Steven D'Aprano wrote:
>> Compare the natural way of writing this:
>>
>> with open("spam") as spam, open("eggs", "w") as eggs, frobulate("cheese") as
>> cheese:
>> # do stuff with spam, eggs, cheese
>>
>> versus the dynamic way:
>>
>> with ExitStack() as stack:
>> spam, eggs = [stack.enter_context(open(fname), mode) for fname, mode in
>> zip(("spam", "eggs"), ("r", "w")]
>> cheese = stack.enter_context(frobulate("cheese"))
>> # do stuff with spam, eggs, cheese
>
> You wouldn't necessarily switch at three. At only three, you have lots
> of options, including multiple nested with statements:
>
> with open("spam") as spam:
> with open("eggs", "w") as eggs:
> with frobulate("cheese") as cheese:
> # do stuff with spam, eggs, cheese
>
> The "multiple context managers in one with statement" form is there
> *solely* to save indentation levels, and overuse can often be a sign
> that you may have a custom context manager trying to get out:
>
> @contextlib.contextmanager
> def dish(spam_file, egg_file, topping):
> with open(spam_file), open(egg_file, 'w'), frobulate(topping):
> yield
>
> with dish("spam", "eggs", "cheese") as spam, eggs, cheese:
> # do stuff with spam, eggs & cheese
>
> ExitStack is mostly useful as a tool for writing flexible custom
> context managers, and for dealing with context managers in cases where
> lexical scoping doesn't necessarily work, rather than being something
> you'd regularly use for inline code.
>
> "Why do I have so many contexts open at once in this function?" is a
> question developers should ask themselves in the same way its worth
> asking "why do I have so many local variables in this function?"
Multiline with-statement can be useful even with *two* context
managers. Two is not many.
Saving indentations levels along is a worthy goal. It can affect
readability and the perceived complexity of the code.
Here's how I'd like the code to look like:
with (open('input filename') as input_file,
open('output filename', 'w') as output_file):
# code with list comprehensions to transform input file into output file
Even one additional unnecessary indentation level may force to split
list comprehensions into several lines (less readable) and/or use
shorter names (less readable). Or it may force to move the inline code
into a separate named function prematurely, solely to preserve the
indentation level (also may be less readable) i.e.,
with ... as input_file:
with ... as output_file:
... #XXX indentation level is lost for no reason
with ... as infile, ... as outfile: #XXX shorter names
...
with ... as input_file:
with ... as output_file:
transform(input_file, output_file) #XXX unnecessary function
And (nested() can be implemented using ExitStack):
with nested(open(..),
open(..)) as (input_file, output_file):
... #XXX less readable
Here's an example where nested() won't help:
def get_integers(filename):
with (open(filename, 'rb', 0) as file,
mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ) as
mmapped_file):
for match in re.finditer(br'\d+', mmapped_file):
yield int(match.group())
Here's another:
with (open('log'+'some expression that generates filename', 'a') as logfile,
redirect_stdout(logfile)):
...
--
Akira
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reviving restricted mode?
Unless you remove all the things labelled "keep away from children". I wrote this sandbox to allow python to be used as a "mods"/"add-ons" language for a game I'm writing, hence the perhaps too strict nature. About the crashers: as this is for games, its "fine" for the game to crash, as long as the sandbox is not broken while crashing. time and math can probably be allowed, but random imports a lot of undesirable modules. My sandbox doesn't use proxies, due to the introspection and complexity that it involves. Instead it completely isolates the sandboxed globals, and checks all arguments and globals for irregularities before passing control to non- sandboxed functions. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reviving restricted mode?
On Wed, Aug 13, 2014 at 11:11 PM, Isaac Morland wrote: > While I would not claim a Python sandbox is utterly impossible, I'm > suspicious that the whole "consenting adults" approach in Python is > incompatible with a sandbox. The whole idea of a sandbox is to absolutely > prevent people from doing things even if they really want to and know what > they are doing. It's certainly not *fundamentally* impossible to sandbox Python. However, the question becomes one of how much effort you're going to go to and how much you're going to restrict the code. I think I remember reading about something that's like ast.literal_eval, but allows name references; with that, plus some tiny features of assignment, you could make a fairly straight-forward evaluator that lets you work comfortably with numbers, strings, lists, dicts, etc. That could be pretty useful - but it wouldn't so much be "Python in a sandbox" as "an expression evaluator that uses a severely restricted set of Python syntax". If you start with all of Python and then start cutting out the dangerous bits, you're doomed to miss something, and your sandbox is broken. If you start with nothing and then start adding functionality, you're looking at a gigantic job before it becomes anything that you could call an applications language. So while it's theoretically possible (I think - certainly I can't say for sure that it's impossible), it's fairly impractical. I've had my own try at it, and failed quite badly (fortunately noisily and at a sufficiently early stage of development to shift). ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reviving restricted mode?
Unless you remove all the things labelled "keep away from children". I wrote this sandbox to allow python to be used as a "mods"/"add-ons" language for a game I'm writing, hence the perhaps too strict nature. About the crashers: as this is for games, its "fine" for the game to crash, as long as the sandbox is not broken while crashing. time and math can probably be allowed, but random imports a lot of undesirable modules. My sandbox doesn't use proxies, due to the introspection and complexity that it involves. Instead it completely isolates the sandboxed globals, and checks all arguments and globals for irregularities before passing control to non-sandboxed functions. On 13 August 2014 14:11, Isaac Morland wrote: > On Mon, 11 Aug 2014, Skip Montanaro wrote: > > On Mon, Aug 11, 2014 at 12:42 PM, matsjoyce wrote: >> >>> There maybe some holes in my approach, but I can't find them. >>> >> >> There's the rub. Given time, I suspect someone will discover a hole or >> two. >> > > Schneier's Law: > > Any person can invent a security system so clever that she or he > can't > think of how to break it. > > While I would not claim a Python sandbox is utterly impossible, I'm > suspicious that the whole "consenting adults" approach in Python is > incompatible with a sandbox. The whole idea of a sandbox is to absolutely > prevent people from doing things even if they really want to and know what > they are doing. > > Isaac Morland CSCF Web Guru > DC 2554C, x36650WWW Software Specialist > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sum(...) limitation
On 12 Aug 2014, at 10:02, Armin Rigo wrote: > Hi all, > > The core of the matter is that if we repeatedly __add__ strings from a > long list, we get O(n**2) behavior. For one point of view, the > reason is that the additions proceed in left-to-right order. Indeed, > sum() could proceed in a more balanced tree-like order: from [x0, x1, > x2, x3, ...], reduce the list to [x0+x1, x2+x3, ...]; then repeat > until there is only one item in the final list. This order ensures > that sum(list_of_strings) is at worst O(n log n). It might be in > practice close enough from linear to not matter. It also improves a > lot the precision of sum(list_of_floats) (though not reaching the same > precision levels of math.fsum()). I wonder why nobody has mentioned previous year’s discussion of the same issue yet: http://marc.info/?l=python-ideas&m=137359619831497&w=2 Maybe someone can write a PEP about this that can be pointed when the question is discussed again next summer ;-) Ronald ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reviving restricted mode?
On Thu, Aug 14, 2014 at 02:26:29AM +1000, Chris Angelico wrote: > On Wed, Aug 13, 2014 at 11:11 PM, Isaac Morland wrote: > > While I would not claim a Python sandbox is utterly impossible, I'm > > suspicious that the whole "consenting adults" approach in Python is > > incompatible with a sandbox. The whole idea of a sandbox is to absolutely > > prevent people from doing things even if they really want to and know what > > they are doing. The point of a sandbox is that I, the consenting adult writing the application in the first place, may want to allow *untrusted others* to call Python code without giving them control of the entire application. The consenting adults rule applies to me, the application writer, not them, the end-users, even if they happen to be writing Python code. If they want unrestricted access to the Python interpreter, they can run their code on their own machine, not mine. > It's certainly not *fundamentally* impossible to sandbox Python. > However, the question becomes one of how much effort you're going to > go to and how much you're going to restrict the code. I believe that PyPy has an effective sandbox, but to what degree of effectiveness I don't know. http://pypy.readthedocs.org/en/latest/sandbox.html I've had rogue Javascript crash my browser or make my entire computer effectively unusable often enough that I am skeptical about claims that Javascript in the browser is effectively sandboxed, so I'm doubly cautious about Python. -- Steven ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reviving restricted mode?
On Thu, Aug 14, 2014 at 2:58 AM, Steven D'Aprano wrote: >> It's certainly not *fundamentally* impossible to sandbox Python. >> However, the question becomes one of how much effort you're going to >> go to and how much you're going to restrict the code. > > I believe that PyPy has an effective sandbox, but to what degree of > effectiveness I don't know. """ A potential attacker can have arbitrary code run in the subprocess, but cannot actually do any input/output not controlled by the outer process. Additional barriers are put to limit the amount of RAM and CPU time used. Note that this is very different from sandboxing at the Python language level, i.e. placing restrictions on what kind of Python code the attacker is allowed to run (why? read about pysandbox). """ That's quite useful, but isn't the same thing as a Python-in-Python sandbox (or even what I was doing, Python-in-C++). ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiline with statement line continuation
On Aug 13, 2014 7:04 PM, "Akira Li" <4kir4...@gmail.com> wrote:
>
> Nick Coghlan writes:
>
> > On 12 August 2014 22:15, Steven D'Aprano wrote:
> >> Compare the natural way of writing this:
> >>
> >> with open("spam") as spam, open("eggs", "w") as eggs,
frobulate("cheese") as cheese:
> >> # do stuff with spam, eggs, cheese
> >>
> >> versus the dynamic way:
> >>
> >> with ExitStack() as stack:
> >> spam, eggs = [stack.enter_context(open(fname), mode) for fname,
mode in
> >> zip(("spam", "eggs"), ("r", "w")]
> >> cheese = stack.enter_context(frobulate("cheese"))
> >> # do stuff with spam, eggs, cheese
> >
> > You wouldn't necessarily switch at three. At only three, you have lots
> > of options, including multiple nested with statements:
> >
> > with open("spam") as spam:
> > with open("eggs", "w") as eggs:
> > with frobulate("cheese") as cheese:
> > # do stuff with spam, eggs, cheese
> >
> > The "multiple context managers in one with statement" form is there
> > *solely* to save indentation levels, and overuse can often be a sign
> > that you may have a custom context manager trying to get out:
> >
> > @contextlib.contextmanager
> > def dish(spam_file, egg_file, topping):
> > with open(spam_file), open(egg_file, 'w'), frobulate(topping):
> > yield
> >
> > with dish("spam", "eggs", "cheese") as spam, eggs, cheese:
> > # do stuff with spam, eggs & cheese
> >
> > ExitStack is mostly useful as a tool for writing flexible custom
> > context managers, and for dealing with context managers in cases where
> > lexical scoping doesn't necessarily work, rather than being something
> > you'd regularly use for inline code.
> >
> > "Why do I have so many contexts open at once in this function?" is a
> > question developers should ask themselves in the same way its worth
> > asking "why do I have so many local variables in this function?"
>
> Multiline with-statement can be useful even with *two* context
> managers. Two is not many.
>
> Saving indentations levels along is a worthy goal. It can affect
> readability and the perceived complexity of the code.
>
> Here's how I'd like the code to look like:
>
> with (open('input filename') as input_file,
> open('output filename', 'w') as output_file):
> # code with list comprehensions to transform input file into output
file
>
> Even one additional unnecessary indentation level may force to split
> list comprehensions into several lines (less readable) and/or use
> shorter names (less readable). Or it may force to move the inline code
> into a separate named function prematurely, solely to preserve the
> indentation level (also may be less readable) i.e.,
>
> with ... as input_file:
> with ... as output_file:
> ... #XXX indentation level is lost for no reason
>
> with ... as infile, ... as outfile: #XXX shorter names
> ...
>
> with ... as input_file:
> with ... as output_file:
> transform(input_file, output_file) #XXX unnecessary function
>
> And (nested() can be implemented using ExitStack):
>
> with nested(open(..),
> open(..)) as (input_file, output_file):
> ... #XXX less readable
>
> Here's an example where nested() won't help:
>
> def get_integers(filename):
> with (open(filename, 'rb', 0) as file,
> mmap.mmap(file.fileno(), 0, access=mmap.ACCESS_READ) as
mmapped_file):
> for match in re.finditer(br'\d+', mmapped_file):
> yield int(match.group())
>
> Here's another:
>
> with (open('log'+'some expression that generates filename', 'a') as
logfile,
> redirect_stdout(logfile)):
> ...
>
Just a thought, would it bit wierd that:
with (a as b, c as d): "works"
with (a, c): "boom"
with(a as b, c): ?
>
> --
> Akira
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/yoavglazner%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reviving restricted mode?
On Thu, 14 Aug 2014, Steven D'Aprano wrote: On Thu, Aug 14, 2014 at 02:26:29AM +1000, Chris Angelico wrote: On Wed, Aug 13, 2014 at 11:11 PM, Isaac Morland wrote: While I would not claim a Python sandbox is utterly impossible, I'm suspicious that the whole "consenting adults" approach in Python is incompatible with a sandbox. The whole idea of a sandbox is to absolutely prevent people from doing things even if they really want to and know what they are doing. The point of a sandbox is that I, the consenting adult writing the application in the first place, may want to allow *untrusted others* to call Python code without giving them control of the entire application. The consenting adults rule applies to me, the application writer, not them, the end-users, even if they happen to be writing Python code. If they want unrestricted access to the Python interpreter, they can run their code on their own machine, not mine. Yes, absolutely, and I didn't mean to contradict what you are saying. What I am suggesting is that the basic design of Python isn't a good starting point for imposing mandatory restrictions on what code can do. By contrast, take something like Safe Haskell. I'm not absolutely certain that it really is safe as promised, but it's starting from a very different language in which the compiler performs extremely sophisticated type checking and simply won't compile programs that don't work within the type system. This isn't a knock on Python (which I love using, by the way), just being realistic about what the existing language is likely to be able to support. Having said that, I'll be very interested if somebody does come up with a restricted mode Python that is widely accepted as being secure - that would be a real achievement. Isaac Morland CSCF Web Guru DC 2554C, x36650WWW Software Specialist ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Multiline with statement line continuation
On Wed, Aug 13, 2014 at 08:08:51PM +0300, yoav glazner wrote:
[...]
> Just a thought, would it bit wierd that:
> with (a as b, c as d): "works"
> with (a, c): "boom"
> with(a as b, c): ?
If this proposal is accepted, there is no need for the "boom". The
syntax should allow:
# Without parens, limited to a single line.
with a [as name], b [as name], c [as name], ...:
block
# With parens, not limited to a single line.
with (a [as name],
b [as name],
c [as name],
...
):
block
where the "as name" part is always optional. In both these cases,
whether there are parens or not, it will be interpreted as a series of
context managers and never as a single tuple.
Note two things:
(1) this means that even in the unlikely event that tuples become
context managers in the future, you won't be able to use a tuple
literal:
with (1, 2, 3): # won't work as expected
t = (1, 2, 3)
with t: # will work as expected
But I cannot imagine any circumstances where tuples will become context
managers.
(2) Also note that *this is already the case*, since tuples are made by
the commas, not the parentheses. E.g. this succeeds:
# Not a tuple, actually two context managers.
with open("/tmp/foo"), open("/tmp/bar", "w"):
pass
--
Steven
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reviving restricted mode?
On 8/13/2014 12:19 PM, matsjoyce wrote: Unless you remove all the things labelled "keep away from children". I wrote this sandbox to allow python to be used as a "mods"/"add-ons" language for a game I'm writing, hence the perhaps too strict nature. About the crashers: as this is for games, its "fine" for the game to crash, as long as the sandbox is not broken while crashing. time and math can probably be allowed, but random imports a lot of undesirable modules. My sandbox doesn't use proxies, due to the introspection and complexity that it involves. Instead it completely isolates the sandboxed globals, and checks all arguments and globals for irregularities before passing control to non- sandboxed functions. pydev is for mainly for discussion of maintaining current versions and development of the next, and for discussion of PEPs which might apply to the one after next. This discussion should be on python-list or perhaps python-ideas if there is a semi-concrete proposal for a future python. -- Terry Jan Reedy ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Reviving restricted mode?
Hi, I heard that PyPy sandbox cannot be used out of the box. You have to write a policy to allow syscalls. The complexity is moved to this policy which is very hard to write, especially if you only use whitelists. Correct me if I'm wrong. To be honest, I never take a look at this sandbox. Victor Le mercredi 13 août 2014, Steven D'Aprano a écrit : > On Thu, Aug 14, 2014 at 02:26:29AM +1000, Chris Angelico wrote: > > On Wed, Aug 13, 2014 at 11:11 PM, Isaac Morland > wrote: > > > While I would not claim a Python sandbox is utterly impossible, I'm > > > suspicious that the whole "consenting adults" approach in Python is > > > incompatible with a sandbox. The whole idea of a sandbox is to > absolutely > > > prevent people from doing things even if they really want to and know > what > > > they are doing. > > The point of a sandbox is that I, the consenting adult writing the > application in the first place, may want to allow *untrusted others* to > call Python code without giving them control of the entire application. > The consenting adults rule applies to me, the application writer, not > them, the end-users, even if they happen to be writing Python code. If > they want unrestricted access to the Python interpreter, they can run > their code on their own machine, not mine. > > > > It's certainly not *fundamentally* impossible to sandbox Python. > > However, the question becomes one of how much effort you're going to > > go to and how much you're going to restrict the code. > > I believe that PyPy has an effective sandbox, but to what degree of > effectiveness I don't know. > > http://pypy.readthedocs.org/en/latest/sandbox.html > > I've had rogue Javascript crash my browser or make my entire computer > effectively unusable often enough that I am skeptical about claims that > Javascript in the browser is effectively sandboxed, so I'm doubly > cautious about Python. > > > -- > Steven > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] sum(...) limitation
On Tue, Aug 12, 2014 at 11:21 PM, Stephen J. Turnbull wrote: > Redirecting to python-ideas, so trimming less than I might. reasonable enough -- you are introducing some more significant ideas for changes. I've said all I have to say about this -- I don't seem to see anything encouraging form core devs, so I guess that's it. Thanks for the fun bike-shedding... -Chris > Chris Barker writes: > > On Mon, Aug 11, 2014 at 11:07 PM, Stephen J. Turnbull < > step...@xemacs.org> > > wrote: > > > > > I'm referring to removing the unnecessary information that there's a > > > better way to do it, and simply raising an error (as in Python 3.2, > > > say) which is all a RealProgrammer[tm] should ever need! > > > > > > > I can't imagine anyone is suggesting that -- disallow it, but don't tell > > anyone why? > > As I said, it's a regression. That's exactly the behavior in Python 3.2. > > > The only thing that is remotely on the table here is: > > > > 1) remove the special case for strings -- buyer beware -- but consistent > > and less "ugly" > > It's only consistent if you believe that Python has strict rules for > use of various operators. It doesn't, except as far as they are > constrained by precedence. For example, I have an application where I > add bytestrings bytewise modulo N <= 256, and concatenate them. In > fact I use function call syntax, but the obvious operator syntax is > '+' for the bytewise addition, and '*' for the concatenation. > > It's not in the Zen, but I believe in the maxim "If it's worth doing, > it's worth doing well." So for me, 1) is out anyway. > > > 2) add a special case for strings that is fast and efficient -- may be > as > > simple as calling "".join() under the hood --no more code than the > > exception check. > > Sure, but what about all the other immutable containers with __add__ > methods? What about mappings with key-wise __add__ methods whose > values might be immutable but have __add__ methods? Where do you stop > with the special-casing? I consider this far more complex and ugly > than the simple "sum() is for numbers" rule (and even that is way too > complex considering accuracy of summing floats). > > > And I doubt anyone really is pushing for anything but (2) > > I know that, but I think it's the wrong solution to the problem (which > is genuine IMO). The right solution is something generic, possibly a > __sum__ method. The question is whether that leads to too much work > to be worth it (eg, "homogeneous_iterable"). > > > > Because obviously we'd want the attractive nuisance of "if you > > > have __add__, there's a default definition of __sum__" > > > > now I'm confused -- isn't that exactly what we have now? > > Yes and my feeling (backed up by arguments that I admit may persuade > nobody but myself) is that what we have now kinda sucks[tm]. It > seemed like a good idea when I first saw it, but then, my apps don't > scale to where the pain starts in my own usage. > > > > It's possible that Python could provide some kind of feature that > > > would allow an optimized sum function for every type that has > > > __add__, but I think this will take a lot of thinking. > > > > does it need to be every type? As it is the common ones work fine > already > > except for strings -- so if we add an optimized string sum() then we're > > done. > > I didn't say provide an optimized sum(), I said provide a feature > enabling people who want to optimize sum() to do so. So yes, it needs > to be every type (the optional __sum__ method is a proof of concept, > modulo it actually being implementable ;-). > > > > *Somebody* will do it (I don't think anybody is +1 on restricting > > > sum() to a subset of types with __add__). > > > > uhm, that's exactly what we have now > > Exactly. Who's arguing that the sum() we have now is a ticket to > Paradise? I'm just saying that there's probably somebody out there > negative enough on the current situation to come up with an answer > that I think is general enough (and I suspect that python-dev > consensus is that demanding, too). > > > sum() can be used for any type that has an __add__ defined. > > I'd like to see that be mutable types with __iadd__. > > > What I fail to see is why it's better to raise an exception and > > point users to a better way, than to simply provide an optimization > > so that it's a mute issue. > > Because inefficient sum() is an attractive nuisance, easy to overlook, > and likely to bite users other than the author. > > > The only justification offered here is that will teach people that > summing > > strings (and some other objects?) > > Summing tuples works (with appropriate start=tuple()). Haven't > benchmarked, but I bet that's O(N^2). > > > is order(N^2) and a bad idea. But: > > > > a) Python's primary purpose is practical, not pedagogical (not that it > > isn't great for that) > > My argument is that in practical use sum() is a bad idea, period, > until y
[Python-Dev] Documenting enum types
Should new enum types added recently to collect module constants be documented at all? For example AddressFamily is absent in socket.__all__ [1]. [1] http://bugs.python.org/issue20689 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
