[Python-ideas] Re: New clause in FOR and WHILE instead of ELSE

2020-07-11 Thread Rob Cliffe via Python-ideas
On 11/07/2020 06:22, Олег Комлев wrote: ELSE-clause in FOR and WHILE has unclear syntax. I suggest new clause instead: if COND: ... [elif COND: ...] [else: ...] This IF-clause like must be immediately after FOR- or WHILE-cycle (only comment allowed between). It looks like a regular

[Python-ideas] Re: Pickle security improvements

2020-07-11 Thread Greg Ewing
On 12/07/20 1:01 pm, Edwin Zimmerman wrote: As I see it, the unsafe callables (eval, exec, os.system, etc) are generally functions, and safe ones(int, list, dict) are generally classes, though there certainly would be exceptions. Where security is concerned, "there certainly would be

[Python-ideas] Re: Pickle security improvements

2020-07-11 Thread Bruce Leban
The security problem arises from the fact that pickle will call arbitrary functions and that it will unpickle arbitrary classes, not just the ones that you might intend it to. It seems to me that the way to make pickle safe is to limit what it can call. Unpickle can take a list of classes and it

[Python-ideas] Re: Pickle security improvements

2020-07-11 Thread Wes Turner
If there were a configurable allow list of "safe" types, what in the stdlib would and wouldn't be on the list? On Sat, Jul 11, 2020, 9:16 PM Edwin Zimmerman wrote: > As I see it, the unsafe callables (eval, exec, os.system, etc) are > generally functions, and safe ones(int, list, dict) are

[Python-ideas] Re: Pickle security improvements

2020-07-11 Thread Edwin Zimmerman
As I see it, the unsafe callables (eval, exec, os.system, etc) are generally functions, and safe ones(int, list, dict) are generally classes, though there certainly would be exceptions. Would it be too great of a breaking change to block function callables by default?  That might be an

[Python-ideas] Re: Pickle security improvements

2020-07-11 Thread Greg Ewing
On 12/07/20 8:54 am, Wes Turner wrote: Would it be feasible to just NOP callables when safe=True? This would break pickle, because calling constructors is the way many objects are unpickled. And it's not easy to tell which callables are safe to use as constructors and which aren't. -- Greg

[Python-ideas] Re: Pickle security improvements

2020-07-11 Thread Greg Ewing
On 12/07/20 5:31 am, Wes Turner wrote: Is there already a way to load data and not code *with pickle*? As far as I know, pickle has never been able to load code objects. The security problems come from the fact that by default a pickle is able to *call* any module-level callable object that

[Python-ideas] Re: Access (ordered) dict by index; insert slice

2020-07-11 Thread Inada Naoki
On Sun, Jul 12, 2020 at 4:43 AM Christopher Barker wrote: > > > The existing dictionary memory layout doesn't support direct indexing > > (without stepping), so this functionality is not being added as a > > requirement. > > But it does make it much more efficient if the stepping is done inside

[Python-ideas] Re: Pickle security improvements

2020-07-11 Thread Edwin Zimmerman
The bottom line is that pickle should never be used in a security sensitive context.  Several years ago I spent about 5 minutes writing a custom pickle fuzzer.  It ran for about 60 seconds before segfaulting.  Fortunately, the last time I ran my fuzzer (about a year ago), all I could produce

[Python-ideas] Re: Pickle security improvements

2020-07-11 Thread Wes Turner
AFAIU, jsonpickle (and fill, cloud pickle,) will still execute arbitray python (and ctypes) code. Isn't pickle faster than C JSON? Would it be feasible to just NOP callables when safe=True? Or would that be pointless? JSON5 is great but still doesn't handle e.g. complex fractions On Sat, Jul

[Python-ideas] Re: Pickle security improvements

2020-07-11 Thread David Mertz
On Sat, Jul 11, 2020 at 4:24 PM Christopher Barker wrote: > NOTE: I've wanted for ages to make a "PYSON" format / module for when JSON > is not quite enough. e.g. distinction between lists and tuples, dict keys > that aren't strings > https://github.com/jsonpickle/jsonpickle You're not

[Python-ideas] Re: Access (ordered) dict by index; insert slice

2020-07-11 Thread David Mertz
On Sat, Jul 11, 2020 at 4:33 PM David Mertz wrote: > In any case, if "reservoir sampling" is the goal here, we should just add > a function `random.reservoir_sample()` to accommodate using iterators > rather than sequences (https://en.wikipedia.org/wiki/Reservoir_sampling) > Doh! I mean

[Python-ideas] Re: Access (ordered) dict by index; insert slice

2020-07-11 Thread David Mertz
On Sat, Jul 11, 2020 at 3:45 PM Christopher Barker wrote: > random.choice(the_dict.keys()) > > is a little easier than: > > random.choice(list(the_dict.keys()) > Ummm... don't you mean: random.choice(list(the_dict)) If it's keys you care about I've saved you one character over your proposed

[Python-ideas] Re: Pickle security improvements

2020-07-11 Thread Christopher Barker
On Sat, Jul 11, 2020 at 10:33 AM Wes Turner wrote: > Is there already a way to load data and not code *with pickle*? > https://docs.python.org/3/library/pickle.html > I'm not sure if this is what you mean, but there is: ast.literal_eval() which I *think* is safe. NOTE: I've wanted for ages

[Python-ideas] Re: Access (ordered) dict by index; insert slice

2020-07-11 Thread David Mertz
On Sat, Jul 11, 2020 at 3:45 PM Christopher Barker wrote: > On Fri, Jul 10, 2020 at 12:45 PM David Mertz wrote: > > The strongest argument I've seen is: `list(d.items())` adds six > characters. > > 1) Matching our mental model / usability: if I want the nth item (or a > random item) from a

[Python-ideas] Re: Access (ordered) dict by index; insert slice

2020-07-11 Thread Christopher Barker
I had a nice note almost written yesterday, but now there've been a bunch more discussion, so I'm going to try to hit a few points that have been recently made. TL;DR: I personally think it would be a nice feature to add indexing to the dict views. But to be fair, the only real use case I've seen

[Python-ideas] Re: Access (ordered) dict by index; insert slice

2020-07-11 Thread Random832
On Thu, Jul 9, 2020, at 13:26, Stestagg wrote: > Obviously, python is a general-purpose, turing complete language, so > each of these options can be written in other ways. But it would be > nice if the simple, readable versions also worked :D > > The idea that there are future, unspecified

[Python-ideas] Re: Pickle security improvements

2020-07-11 Thread Wes Turner
Would this accomplish something like: pickle.load(safe=True) # or pickle.safe_loads() Is there already a way to load data and not code *with pickle*? https://docs.python.org/3/library/pickle.html On Sat, Jul 11, 2020, 11:01 AM Random832 wrote: > The current practice, by overriding

[Python-ideas] Re: Access (ordered) dict by index; insert slice

2020-07-11 Thread Stephen J. Turnbull
Chris Angelico writes: > I would pick repeatedly from the same dictionary but it might be > mutated in between. So the list would have to be reconstructed > fresh every time. OK, that moves me a couple million Planck lengths away from -1 nm. :-) I guess in that case if I cared about

[Python-ideas] [Suspected Spam]Re: Access (ordered) dict by index; insert slice

2020-07-11 Thread Stephen J. Turnbull
Christopher Barker writes: > > d.keys()[-1] vs list(d.keys())[-1] > > > > Should be compared with `next(reversed(d.keys()))`, or `next(reversed(d))`. > > > > Same point - the idea is to have indexing syntax. I'm a-gonna make you REAALLY MAAD. You can have it. d.popitem()[0]

[Python-ideas] Re: Add builtin function for min(max())

2020-07-11 Thread Random832
On Thu, Jul 9, 2020, at 15:32, Dominik Vilsmeier wrote: > On 09.07.20 21:04, Ethan Furman wrote: > > I'm having a hard time understanding this line: > > > >    if lower == upper is not None: > > > > As near as I can tell, `upper is not None` will be either True or > > False, meaning the

[Python-ideas] Pickle security improvements

2020-07-11 Thread Random832
The current practice, by overriding find_class, is limited to overriding what globals get loaded. This makes it impossible to distinguish globals that will be used as data from globals that will be called as constructors, along with similar concerns with object attributes [especially methods]

[Python-ideas] New clause in FOR and WHILE instead of ELSE

2020-07-11 Thread Олег Комлев
ELSE-clause in FOR and WHILE has unclear syntax. I suggest new clause instead: if COND: ... [elif COND: ...] [else: ...] This IF-clause like must be immediately after FOR- or WHILE-cycle (only comment allowed between). It looks like a regular IF, but COND is special. COND may be "break",