Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Juancarlo Añez]

> So the only thing being ruled out is the dedicated syntax option,
> since it doesn't let us do anything that a new builtin can't do, it's
> harder to find help on (as compared to "help(ntuple)" or searching
> online for "python ntuple"), and it can't be readily backported to
> Python 3.6 as part of a third party library (you can't easily backport
> it any further than that regardless, since you'd be missing the
> order-preservation guarantee for the keyword arguments passed to the
> builtin).
>

If an important revamp of namedtuple will happen (actually, "easy and
friendly immutable structures"), I'd suggest that the new syntax is not
discarded upfront, but rather be left as a final decision, after all the
other forces are resolved.

FWIW, there's another development thread about "easy class declarations
(with typining)". From MHPOV, the threads are different enough to remain
separate.

Cheers!

-- 
Juancarlo *Añez*
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Nick Coghlan]

On 25 July 2017 at 11:57, Nick Coghlan  wrote:
> Having such a builtin implictly create and cache new namedtuple type
> definitions so the end user doesn't need to care about pre-declaring
> them is still fine, and remains the most straightforward way of
> building a capability like this atop the underlying
> `collections.namedtuple` type.

I've updated the example I posted in the other thread with all the
necessary fiddling required for full pickle compatibility with
auto-generated collections.namedtuple type definitions:
https://gist.github.com/ncoghlan/a79e7a1b3f7dac11c6cfbbf59b189621

This shows that given ordered keyword arguments as a building block,
most of the actual implementation complexity now lies in designing an
implicit type cache that plays nicely with the way pickle works:

from collections import namedtuple

class _AutoNamedTupleTypeCache(dict):
"""Pickle compatibility helper for autogenerated
collections.namedtuple type definitions"""
def __new__(cls):
# Ensure that unpickling reuses the existing cache instance
self = globals().get("_AUTO_NTUPLE_TYPE_CACHE")
if self is None:
maybe_self = super().__new__(cls)
self = globals().setdefault("_AUTO_NTUPLE_TYPE_CACHE",
maybe_self)
return self

def __missing__(self, fields):
cls_name = "_ntuple_" + "_".join(fields)
return self._define_new_type(cls_name, fields)

def __getattr__(self, cls_name):
parts = cls_name.split("_")
if not parts[:2] == ["", "ntuple"]:
raise AttributeError(cls_name)
fields = tuple(parts[2:])
return self._define_new_type(cls_name, fields)

def _define_new_type(self, cls_name, fields):
cls = namedtuple(cls_name, fields)
cls.__module__ = __name__
cls.__qualname__ = "_AUTO_NTUPLE_TYPE_CACHE." + cls_name
# Rely on setdefault to handle race conditions between threads
return self.setdefault(fields, cls)

_AUTO_NTUPLE_TYPE_CACHE = _AutoNamedTupleTypeCache()

def auto_ntuple(**items):
cls = _AUTO_NTUPLE_TYPE_CACHE[tuple(items)]
return cls(*items.values())

But given such a cache, you get implicitly defined types that are
automatically shared between instances that want to use the same field
names:

>>> p1 = auto_ntuple(x=1, y=2)
>>> p2 = auto_ntuple(x=4, y=5)
>>> type(p1) is type(p2)
True
>>>
>>> import pickle
>>> p3 = pickle.loads(pickle.dumps(p1))
>>> p1 == p3
True
>>> type(p1) is type(p3)
True
>>>
>>> p1, p2, p3
(_ntuple_x_y(x=1, y=2), _ntuple_x_y(x=4, y=5), _ntuple_x_y(x=1, y=2))
>>> type(p1)


And writing the pickle out to a file and reloading it also works
without needing to explicitly predefine that particular named tuple
variant:

>>> with open("auto_ntuple.pkl", "rb") as f:
... p1 = pickle.load(f)
...
>>> p1
_ntuple_x_y(x=1, y=2)

In effect, implicitly named tuples would be like key-sharing
dictionaries, but sharing at the level of full type objects rather
than key sets.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Nick Coghlan]

On 25 July 2017 at 02:46, Michel Desmoulin  wrote:
> Le 24/07/2017 à 16:12, Nick Coghlan a écrit :
>> On 22 July 2017 at 01:18, Guido van Rossum  wrote:
>>> Honestly I would like to declare the bare (x=1, y=0) proposal dead. Let's
>>> encourage the use of objects rather than tuples (named or otherwise) for
>>> most data exchanges. I know of a large codebase that uses dicts instead of
>>> objects, and it's a mess. I expect the bare ntuple to encourage the same
>>> chaos.
>
> This is the people working on big code base talking.

Dedicated syntax:

(x=1, y=0)

New builtin:

ntuple(x=1, y=0)

So the only thing being ruled out is the dedicated syntax option,
since it doesn't let us do anything that a new builtin can't do, it's
harder to find help on (as compared to "help(ntuple)" or searching
online for "python ntuple"), and it can't be readily backported to
Python 3.6 as part of a third party library (you can't easily backport
it any further than that regardless, since you'd be missing the
order-preservation guarantee for the keyword arguments passed to the
builtin).

Having such a builtin implictly create and cache new namedtuple type
definitions so the end user doesn't need to care about pre-declaring
them is still fine, and remains the most straightforward way of
building a capability like this atop the underlying
`collections.namedtuple` type.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Chris Barker]

On Fri, Jul 21, 2017 at 8:18 AM, Guido van Rossum  wrote:

> Honestly I would like to declare the bare (x=1, y=0) proposal dead. Let's
> encourage the use of objects rather than tuples (named or otherwise) for
> most data exchanges. I know of a large codebase that uses dicts instead of
> objects, and it's a mess. I expect the bare ntuple to encourage the same
> chaos.
>

I've seen the same sort of mess, but I think it's because folks have come
down on the wrong side of "what's code, and what's data?"

Data belongs in dicts (and tuples, and lists, and...) and code belongs in
objects.

With Python's dynamic nature, it is very easy to blur these lines, but the
way I define it:

a_point['x']

is accessing data, and

a_point.x

is running code. It more or less comes down to -- "if you know the names
you need when you are writing the code, then it is probably code. So be
wary if you are using literals for dict keys frequently.

But from this perspective a NamedTuple (with or without a clearly defined
type) is code, as it should be.

In the duck-typing spirit, you should be able to do something like:

p = get_the_point(something)

do_something_with(p.x, p.y)

And not know or care what type p is.

With this perspective, a NamedTuple, with a known type or otherwise, AVOIDS
the chaos of passing dicts around, and thus should be encouraged.

And indeed, making it as easy as possible to create and pass an
object_with_attributes around, rather than a plain tuple or dict would be a
good thing.

I do agree that we have multiple goals on the table, and DON'T want to have
any more similar, but slightly different, lightweight objects with named
attributes.

So it makes sense to:

1) make namedtuple faster

and then, optionally:

2) make it easier to quickly whip out an (anonymous) namedtuple.

Maybe types.SimpleNamespace is the "better" solution to the above, but it
hasn't gained the traction that namedtuple has. And there is a lot to be
said for imutablilty, and the SimpleNamespace docs even say: "...  for a
structured record type use namedtuple()

instead."

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple redesign goals

[Ethan Furman]

On 07/24/2017 09:49 AM, Michel Desmoulin wrote:


You are assuming a namedtuple litteral would mean collections.namedtuple
would lose the type hability. It's not the case. The litterals can be
complement, not a remplacement.

Accelerating namedtuple can be made by rewritting it in C. The litteral
namedtuple is not necessary for that.


Ah, that makes sense.

Personally, though, I'm not excited about another namedtuple variant.

--
~Ethan~

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Paul Moore]

On 24 July 2017 at 17:37, Michel Desmoulin  wrote:
> You are in the wrong thread. This thread is specifically about
> namedtupels literal.

In which case, did you not see Guido's post "Honestly I would like to
declare the bare (x=1, y=0) proposal dead."? The namedtuple literal
proposal that started this thread is no longer an option, so can we
move on? Preferably by dropping the whole idea - no-one has to my mind
offered any sort of "replacement namedtuple" proposal that can't be
implemented as a 3rd party library on PyPI *except* the (x=1, y=0)
syntax proposal, and I see no justification for adding a *fourth*
implementation of this type of object in the stdlib (which means any
proposal would have to include deprecation of at least one of
namedtuple, structseq or types.SimpleNamespace).

The only remaining discussion on the table that I'm aware of is how we
implement a more efficient version of the stdlib namedtuple class (and
there's not much of that to be discussed here - implementation details
can be thrashed out on the tracker issue).

Paul
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[C Anthony Risinger]

On Sun, Jul 23, 2017 at 8:54 PM, Stephen J. Turnbull <
turnbull.stephen...@u.tsukuba.ac.jp> wrote:

> C Anthony Risinger writes:
>
>  > A tuple is a tuple is a tuple. No types. Just convenient accessors.
>
> That's not possible, though.  A *tuple* is an immutable collection
> indexed by the natural numbers, which is useful to define as a single
> type precisely because the natural numbers are the canonical
> abstraction of "sequence".  You can use the venerable idiom
>
> X = 0
> Y = 1
>
> point = (1.0, 1.0)
> x = point[X]
>
> to give the tuple "named attributes", restricting the names to Python
> identifiers.  Of course this lacks the "namespace" aspect of
> namedtuple, where ".x" has the interpretation of "[0]" only in the
> context of a namedtuple with an ".x" attribute.  But this is truly an
> untyped tuple-with-named-attributes.
>
> However, once you attach specific names to particular indexes, you
> have a type.  The same attribute identifiers may be reused to
> correspond to different indexes to represent a different "convenience
> type".  Since we need to be able to pass these objects to functions,
> pickle them, etc, that information has to be kept in the object
> somewhere, either directly (losing the space efficiency of tuples) or
> indirectly in a class-like structure.
>
> I see the convenience of the unnamed-type-typed tuple, but as that
> phrase suggests, I think it's fundamentally incoherent, a high price
> to pay for a small amount of convenience.
>
> Note that this is not an objection to a forgetful syntax that creates
> a namedtuple subtype but doesn't bother to record the type name
> explicitly in the program.  In fact, we already have that:
>
> >>> from collections import namedtuple
> >>> a = namedtuple('_', ['x', 'y'])(0,1)
> >>> b = namedtuple('_', ['x', 'y'])(0,1)
> >>> a == b
> True
> >>> c = namedtuple('_', ['a', 'b'])(0,1)
>
> This even gives you free equality as I suppose you want it:
>
> >>> a == c
> True
> >>> a.x == c.a
> True
> >>> a.a == c.x
> Traceback (most recent call last):
>   File "", line 1, in 
> AttributeError: '_' object has no attribute 'a'
> >>> c.x == a.a
> Traceback (most recent call last):
>   File "", line 1, in 
> AttributeError: '_' object has no attribute 'x'
>
> Bizarre errors are the inevitable price to pay for this kind of abuse,
> of course.
>
> I'm not a fan of syntaxes like "(x=0, y=1)" or "(x:0, y:1)", but I'll
> leave it up to others to decide how to abbreviate the abominably ugly
> notation I used.
>

Sure sure, this all makes sense, and I agree you can't get the accessors
without storing information, somewhere, that links indexes to attributes,
and it makes complete sense it might be implemented as a subtype, just like
namedtuple works today.

I was more commenting on what it conceptually means to have the designation
"literal". It seems surprising to me that one literal has a different type
from another literal with the same construction syntax. If underneath the
hood it's technically a different type stored in some cached and hidden
lookup table, so be it, but on the surface I think most just want a basic
container with simpler named indexes.

Every time I've used namedtuples, I've thought it more of a chore to pick a
name for it, because it's only semi-useful to me in reprs, and I simply
don't care about the type, ever. I only care about the shape for comparison
with other tuples. If I want typed comparisons I can always just use a
class. I'd also be perfectly fine with storing the "type" as a field on the
tuple itself, because it's just a value container, and that's all I'll ever
want from it.

Alas, when I think I want a namedtuple, I usually end up using a dict
subclass that assigns `self.__dict__ = self` within __new__, because this
makes attribute access (and assignment) work automagically, and I care
about that more than order (though it can be made to support both).

At the end of the day, I don't see a way to have both a literal and
something that is externally "named", because the only ways to pass the
name I can imagine would make it look like a value within the container
itself (such as using a literal string for the first item), unless even
more new syntax was added.

-- 

C Anthony
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple redesign goals

[Michel Desmoulin]

Le 24/07/2017 à 13:45, Ethan Furman a écrit :
> [redirecting back to list]
> 
> On 07/24/2017 04:19 AM, Michel Desmoulin wrote:
>> Le 24/07/2017 à 13:02, Ethan Furman a écrit :
>>> On 07/23/2017 10:47 AM, Michel Desmoulin wrote:
> 
 I'm not sure why everybody have such a grip on the type.
>>>
>>> If I understand the goal of "a new namedtuple" correctly, it is not to
>>> come up with yet another namedtuple type -- it is to make the existing
>>> collections.namedtuple a faster experience, and possibly add another way
>>> to create such a thing.
>>>
>>> This means that the "replacement" namedtuple MUST be backwards
>>> compatible with the existing collections.namedtuple, and keeping track
>>> of type is one of the things it does:
>>
>> Is it ? Maybe we should check that, cause we may be arguing around a
>> "nice to have" for nothing.
> 
> Um, yes, it is.  Did you not read the section you snipped? [1]
> 
>> How many people among those intereted by the proposal have a strong
>> need for the type ?
> 
> Whether there is a strong need for it is largely irrelevant; it's there
> now, it needs to stay.  If we were to remove it there would need to be a
> strong need for what we gain and that it outweighs the broken backward
> compatibility commitment that we try very hard to maintain.

You are assuming a namedtuple litteral would mean collections.namedtuple
would lose the type hability. It's not the case. The litterals can be
complement, not a remplacement.

Accelerating namedtuple can be made by rewritting it in C. The litteral
namedtuple is not necessary for that.

> 
> -- 
> ~Ethan~
> 
> [1] My apologies for the first paragraph if this is a language
> translation issue and you were talking about the backwards compatibility
> and not the type tracking.
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Chris Barker]

On Mon, Jul 24, 2017 at 6:31 AM, Steven D'Aprano 
wrote:

> > I'm not sure why everybody have such a grip on the type.
> >
> > When we use regular tuples, noone care, it's all tuples, no matter what.
>
> Some people care.
>
> This is one of the serious disadvantages of ordinary tuples as a
> record/struct type. There's no way to distinguish between (let's say)
> rectangular coordinates (1, 2) and polar coordinates (1, 2), or between
> (name, age) and (movie_title, score). They're all just 2-tuples.
>


sure -- but Python is dynamically typed, and we all like to talk abou tit
as duck typing -- so asking:

Is this a "rect_coord" or a "polar_coord" object isn't only unnecessary,
it's considered non-pythonic.

Bad example, actually, as a rect_coord would likely have names like 'x' and
'y', while a polar_coord would have "r' and 'theta' -- showing why having a
named-tuple-like structure is helpful, even without types.

So back to the example before of "Motorcycle" vs "Car" -- if they have the
same attributes, then who cares which it is? If there is different
functionality tied to each one, then that's what classes and sub-classing
are for.

I think the entire point of this proposed object is that it be as
lightweight as possible -- it's just a data storage object -- if you want
to switch functionality on type, then use subclasses.

As has been said, NameTupule is partly the way it is because it was desired
to be a drop-in replacement for a regular tuple, and need to be reasonably
implemented in pure python.

If we can have an object that is:

immutable
indexable like a tuple
has named attributes
is lightweight and efficient

I think that would be very useful, and would take the place of NamedTuple
for most use-cases, while being both more pythonic and more efficient.

Whether it gets a literal or a simple constructor makes little difference,
though if it got a literal, it would likely end up seeing much wider use
(kind of like the set literal).

I disagree: in my opinion, the whole point is to make namedtuple faster,
> so that Python's startup time isn't affected so badly. Creating new
> syntax for a new type of tuple is scope-creep.
>

I think making it easier to access and use is a worthwhile goal, too. If we
are re-thinking this, a littel scope creep is OK.

Even if we had that new syntax, the problem of namedtuple slowing down
> Python startup would remain. People can't use this new syntax until they
> have dropped support for everything before 3.7, which might take many
> years. But a fast namedtuple will give them benefit immediately their
> users upgrade to 3.7.
>

These aren't mutually exclusive, if 3.7 has collection.NamedTuple wrap the
new object. IIUC, the idea of chached types would mean that objects _would_
be a Type, even if that wasn't usually exposed -- so it could be exposed in
the case where it was constructed from a collections.NamedTuple()

-CHB

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Michel Desmoulin]

Le 24/07/2017 à 16:12, Nick Coghlan a écrit :
> On 22 July 2017 at 01:18, Guido van Rossum  wrote:
>> Honestly I would like to declare the bare (x=1, y=0) proposal dead. Let's
>> encourage the use of objects rather than tuples (named or otherwise) for
>> most data exchanges. I know of a large codebase that uses dicts instead of
>> objects, and it's a mess. I expect the bare ntuple to encourage the same
>> chaos.

This is the people working on big code base talking.

Remember, Python is not just for Google and Dropbox.

We have thousands of user just being sysadmin, mathematicians, bankers,
analysts, that just want a quick way to make a record. They don't want
nor need a class. Dictionaries and collections.namedtuple are verbose
and so they just used regular tuples.

They don't use mypy either so having a type would be moot for them.

In many languages we have the opposite problem: people using classes as
a container for everything. It makes things very complicated with little
value.

Python actually has a good balance here. Yes, Python doesn't have
pattern matching witch makes it harder to check if a nested data
structure match the desired schema but all in all, the
bloat/expressiveness equilibrium is quite nice.

A litteral namedtuple would allow a clearer way to make a quick and
simple record.

> 
> That sounds sensible to me - given ordered keyword arguments, anything
> that bare syntax could do can be done with a new builtin instead, and
> be inherently more self-documenting as a result.
> 
> Cheers,
> Nick.
> 
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Michel Desmoulin]

Le 24/07/2017 à 15:31, Steven D'Aprano a écrit :
> On Sun, Jul 23, 2017 at 07:47:16PM +0200, Michel Desmoulin wrote:
> 
>> I'm not sure why everybody have such a grip on the type.
>>
>> When we use regular tuples, noone care, it's all tuples, no matter what.
> 
> Some people care.
> 
> This is one of the serious disadvantages of ordinary tuples as a 
> record/struct type. There's no way to distinguish between (let's say) 
> rectangular coordinates (1, 2) and polar coordinates (1, 2), or between 
> (name, age) and (movie_title, score). They're all just 2-tuples.


You are just using my figure of speech as a way to counter argument.
It's not a very useful thing to do.

Of course some people care, there are always a few people caring about
anything.

But you just created your manual namedtuple or a namespace and be done
with it.

Rejecting completly the literal syntax just because it doesn't improve
this use case you already had and worked but was a bit verbose is very
radical. Unless you have a very nice counter proposal that makes
everyone happy, accepting the current one doesn't take anything from you.

> 
> 
> [...]
>> The whole point of this is to make it a litteral, simple and quick to
>> use. If you make it more than it is, we already got everything to do
>> this and don't need to modify the language.
> 
> I disagree: in my opinion, the whole point is to make namedtuple faster, 
> so that Python's startup time isn't affected so badly. Creating new 
> syntax for a new type of tuple is scope-creep.

You are in the wrong thread. This thread is specifically about
namedtupels literal. Making namedtuple faster can be done in many other
ways and doesn't require a literal syntax. A literal syntax, while
making things slightly faster by nature, is essentially to make things
faster to read and write.

> 
> Even if we had that new syntax, the problem of namedtuple slowing down 
> Python startup would remain. People can't use this new syntax until they 
> have dropped support for everything before 3.7, which might take many 
> years. But a fast namedtuple will give them benfit immediately their 
> users upgrade to 3.7.

Again you are mixing the 2 things. This is why we have 2 threads: the
debate splitted.

> 
> I agree that there is a strong case to be made for a fast, built-in, 
> easy way to make record/structs without having to pre-declare them. 

Do other languages have such a thing that can be checked against types ?

> But 
> as the Zen of Python says:
> 
> Now is better than never.
> Although never is often better than *right* now.
> 

I agree. I don't thing we need to rush it. I can live without it now.  I
can live without it at all.

> Let's not rush into designing a poor record/struct builtin just because 
> we have a consensus (Raymond dissenting?) that namedtuple is too slow. 

We don't. We can solve the slowness problem without having the
namedtuple. The litteral is a convenience.

> The two issues are, not unrelated, but orthogonal. Record syntax would 
> be still useful even if namedtuple was accelerated, and faster 
> namedtuple would still be necessary even if we have record syntax.

On that we agree.

> 
> I believe that a couple of people (possibly including Guido?) are 
> already thinking about a PEP for that. If that's the case, let's wait 
> and see what they come up with.

Yes but it's about making classes less verbose if I recall. Or at least
use the class syntax. It's nice but not the same thing. Namedtuple
litterals are way more suited for scripting. You really don't want to
write a class in quick scripts, when you do exploratory programming or
data analysis on the fly.

> 
> In the meantime, lets get back to the original question here: how can we 
> make namedtuple faster?

The go to the other thread for that.

> 
> - Guido has ruled out using a metaclass as the implementation, 
>   as that makes it hard to inherit from namedtuple and another
>   class with a different metaclass.
> 
> - Backwards compatibility is a must.
> 
> - *But* maybe we can afford to bend backwards compatibility 
>   a bit. Perhaps we don't need to generate the *entire* class
>   using exec, just __new__.
> 
> - I don't think that the _source attribute itself makes
>   namedtuple slow. That might effect the memory usage of the
>   class object itself, but its just a name binding:
> 
> result._source = class_definition
> 
>   The expensive part is, I'm fairly sure, this:
> 
> exec(class_definition, namespace)
> 
> (Taken from the 3.5 collections/__init__.py.)
> 
> I asked on pythonl...@python.org whether people made us of the _source 
> attribute, and the overwhelming response was that they either didn't 
> know it existed, or if they did know, they didn't use it.
> 
> https://mail.python.org/pipermail/python-list/2017-July/723888.html
> 
> 
> *If* it is accurate to say that nobody uses _source, then perhaps we 
> might be willing to make this minor backwards-incompatible change in 3.7 
> (but not 

Re: [Python-ideas] HTTP compression support for http.server

[Chris Angelico]

On Tue, Jul 25, 2017 at 2:20 AM, Chris Barker  wrote:
> On Thu, Jul 20, 2017 at 12:15 AM, Pierre Quentel 
> wrote:
>> - if so, should it be supported by default ? It is the case in the PR,
>> where a number of content types, eg text/html, are compressed if the user
>> agent accepts the gzip "encoding"
>
>
> I'm pretty wary of compression happening by default -- i.e. someone runs
> exactly the same code with a newer version of Python, and suddenly some
> content is getting compressed.

FWIW I'm quite okay with that. HTTP already has a mechanism for
negotiating compression (Accept-Encoding), designed to be compatible
with servers that don't support it. Any time a server gains support
for something that clients already support, it's going to start
happening as soon as you upgrade.

Obviously this kind of change won't be happening in a bugfix release
of Python, so it would be part of the regular checks when you upgrade
from 3.6 to 3.7 - it'll be in the NEWS file and so on, so you read up
on it before you upgrade.

ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] HTTP compression support for http.server

[Chris Barker]

The opinion of some random guy on the list...

On Thu, Jul 20, 2017 at 12:15 AM, Pierre Quentel 
wrote:

> I have been suggested to require feedback from core devs :
> - should HTTP compression be supported ?
>

Yes. You are quite right, it's pretty standard stuff these days.


> - if so, should it be supported by default ? It is the case in the PR,
> where a number of content types, eg text/html, are compressed if the user
> agent accepts the gzip "encoding"
>

I'm pretty wary of compression happening by default -- i.e. someone runs
exactly the same code with a newer version of Python, and suddenly some
content is getting compressed.

- if not, should the implementation of http.server be adapted so that
> subclasses could implement it ? For the moment the only way to add it is to
> modify method send_head() of SimpleHTTPRequestHandler
>

sure -- though it would be nice for folks to be able to use compression
without going through that process.

The implementation is based on a list of types to compress
> (SimpleHTTPServer.compressed_types) that can be modified at will, eg set
> to the empty list to disable compression.
>

How about having it be an empty list by default and have one or more lists
of common types be pre-populated and available in the SimpleHTTPServer
namespace. that is:

SimpleHTTPServer.compressed_types =
SimpleHTTPServer.standard_compressed_types

Or may be a method to turn on the "standard" set -- though if it really is
simply a list, better to expose that so it's obvious that you can create
your own list or examine and edit the existing one(s).

Thanks for doing this -- nice feature!

-CHB


-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R(206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception

chris.bar...@noaa.gov
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Nick Coghlan]

On 22 July 2017 at 01:18, Guido van Rossum  wrote:
> Honestly I would like to declare the bare (x=1, y=0) proposal dead. Let's
> encourage the use of objects rather than tuples (named or otherwise) for
> most data exchanges. I know of a large codebase that uses dicts instead of
> objects, and it's a mess. I expect the bare ntuple to encourage the same
> chaos.

That sounds sensible to me - given ordered keyword arguments, anything
that bare syntax could do can be done with a new builtin instead, and
be inherently more self-documenting as a result.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple literals [Was: RE a new namedtuple]

[Steven D'Aprano]

On Sun, Jul 23, 2017 at 07:47:16PM +0200, Michel Desmoulin wrote:

> I'm not sure why everybody have such a grip on the type.
> 
> When we use regular tuples, noone care, it's all tuples, no matter what.

Some people care.

This is one of the serious disadvantages of ordinary tuples as a 
record/struct type. There's no way to distinguish between (let's say) 
rectangular coordinates (1, 2) and polar coordinates (1, 2), or between 
(name, age) and (movie_title, score). They're all just 2-tuples.


[...]
> The whole point of this is to make it a litteral, simple and quick to
> use. If you make it more than it is, we already got everything to do
> this and don't need to modify the language.

I disagree: in my opinion, the whole point is to make namedtuple faster, 
so that Python's startup time isn't affected so badly. Creating new 
syntax for a new type of tuple is scope-creep.

Even if we had that new syntax, the problem of namedtuple slowing down 
Python startup would remain. People can't use this new syntax until they 
have dropped support for everything before 3.7, which might take many 
years. But a fast namedtuple will give them benfit immediately their 
users upgrade to 3.7.

I agree that there is a strong case to be made for a fast, built-in, 
easy way to make record/structs without having to pre-declare them. But 
as the Zen of Python says:

Now is better than never.
Although never is often better than *right* now.

Let's not rush into designing a poor record/struct builtin just because 
we have a consensus (Raymond dissenting?) that namedtuple is too slow. 
The two issues are, not unrelated, but orthogonal. Record syntax would 
be still useful even if namedtuple was accelerated, and faster 
namedtuple would still be necessary even if we have record syntax.

I believe that a couple of people (possibly including Guido?) are 
already thinking about a PEP for that. If that's the case, let's wait 
and see what they come up with.

In the meantime, lets get back to the original question here: how can we 
make namedtuple faster?

- Guido has ruled out using a metaclass as the implementation, 
  as that makes it hard to inherit from namedtuple and another
  class with a different metaclass.

- Backwards compatibility is a must.

- *But* maybe we can afford to bend backwards compatibility 
  a bit. Perhaps we don't need to generate the *entire* class
  using exec, just __new__.

- I don't think that the _source attribute itself makes
  namedtuple slow. That might effect the memory usage of the
  class object itself, but its just a name binding:

result._source = class_definition

  The expensive part is, I'm fairly sure, this:

exec(class_definition, namespace)

(Taken from the 3.5 collections/__init__.py.)

I asked on pythonl...@python.org whether people made us of the _source 
attribute, and the overwhelming response was that they either didn't 
know it existed, or if they did know, they didn't use it.

https://mail.python.org/pipermail/python-list/2017-July/723888.html


*If* it is accurate to say that nobody uses _source, then perhaps we 
might be willing to make this minor backwards-incompatible change in 3.7 
(but not in a bug-fix release):

- Only the __new__ method is generated by exec (my rough tests
  suggest that may make namedtuple four times faster);

- _source only gives the source to __new__;

- or perhaps we can save backwards compatibility by making _source
  generate the rest of the template lazily, when needed, even if
  the entire template isn't used by exec.

That risks getting the *actual* source and the *reported* source 
getting out of sync. Maybe its better to just break compatibility rather 
than risk introducing a discrepancy between the two.



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] namedtuple redesign goals

[Ethan Furman]

[redirecting back to list]

On 07/24/2017 04:19 AM, Michel Desmoulin wrote:

Le 24/07/2017 à 13:02, Ethan Furman a écrit :

On 07/23/2017 10:47 AM, Michel Desmoulin wrote:



I'm not sure why everybody have such a grip on the type.


If I understand the goal of "a new namedtuple" correctly, it is not to
come up with yet another namedtuple type -- it is to make the existing
collections.namedtuple a faster experience, and possibly add another way
to create such a thing.

This means that the "replacement" namedtuple MUST be backwards
compatible with the existing collections.namedtuple, and keeping track
of type is one of the things it does:


Is it ? Maybe we should check that, cause we may be arguing around a "nice to 
have" for nothing.


Um, yes, it is.  Did you not read the section you snipped? [1]


How many people among those intereted by the proposal have a strong need for 
the type ?


Whether there is a strong need for it is largely irrelevant; it's there now, it needs to stay.  If we were to remove it 
there would need to be a strong need for what we gain and that it outweighs the broken backward compatibility commitment 
that we try very hard to maintain.


--
~Ethan~

[1] My apologies for the first paragraph if this is a language translation issue and you were talking about the 
backwards compatibility and not the type tracking.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] namedtuple redesign goals

[Ethan Furman]

On 07/23/2017 10:47 AM, Michel Desmoulin wrote:

> I'm not sure why everybody have such a grip on the type.

If I understand the goal of "a new namedtuple" correctly, it is not to come up with yet another namedtuple type -- it is 
to make the existing collections.namedtuple a faster experience, and possibly add another way to create such a thing.


This means that the "replacement" namedtuple MUST be backwards compatible with the existing collections.namedtuple, and 
keeping track of type is one of the things it does:


--> from collections import namedtuple
--> Point = namedtuple('Point', 'x y')
--> p1 = Point(3, 7)
--> p1.x
3
--> p1.y
7
--> isinstance(p1, Point)
True

--
~Ethan~
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/