Re: Dict comprehensions - improvement to docs?

[Paul Rubin]

"Frank Millman"  writes:
> If you did not know about dict comprehensions, there is nothing to tell you 
> that they even exist. I feel that it should be mentioned.

Yeah, the library reference should treat list and dict comprehensions
the same way.  I would have thought neither of them belongs in the
library reference, but if listcomps are there then dictcomps should also
be there.  Probably a case of the manual not being updated when
dictcomps were added.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Marko Rauhamaa]

Steven D'Aprano :

> Marko Rauhamaa wrote:
>> What features do generator iterators provide on top of generic
>> iterators?
>
> The biggest difference is syntactic. Here's an iterator which returns a
> never-ending sequence of squared numbers 1, 4, 9, 16, ...
>
> class Squares:
> def __init__(self):
> self.i = 0
> def __next__(self):
> self.i += 1
> return self.i**2
> def __iter__(self):
> return self
>
>
> Here's the same thing written as a generator:
>
> def squares():
> i = 1
> while True:
> yield i**2
> i += 1

I was actually referring to the offered API. It still seems to me like
all iterators could offer close(), send() and throw(). Why? To make all
iterators drop-in replacements of each other.

Then, you could also stop wondering what to call the thingy returned by
a generator. Why, it would be an iterator. You wouldn't then have any
other use for a generator than the function that returns an iterator.

You could then decide if you want to reserve the name "generator" for
functions that contain a yield statement (syntactic notion) or call any
function returning an iterator a "generator" (semantic notion).


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Chris Angelico]

On Mon, Mar 16, 2015 at 6:12 PM, Marko Rauhamaa  wrote:
>
> I was actually referring to the offered API. It still seems to me like
> all iterators could offer close(), send() and throw(). Why? To make all
> iterators drop-in replacements of each other.
>
> Then, you could also stop wondering what to call the thingy returned by
> a generator. Why, it would be an iterator. You wouldn't then have any
> other use for a generator than the function that returns an iterator.

That just adds unnecessary overhead to every iterator. Also, what
happens when you throw something into iter([1,2,3]) ? Or send it a
value? What happens?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PSF news - BBC launches MicroBit

[Bob Martin]

in 737966 20150315 161218 MRAB  wrote:
>On 2015-03-15 07:26, Dave Farrance wrote:
>> Mark Lawrence  wrote:
>>
>>>http://pyfound.blogspot.co.uk/2015/03/bbc-launches-microbit.html may be
>>>of interest to some of you.
>>
>> "Python is one of the three languages that work with the device."
>>
>> That's cool, and the article says that the Raspberry Pi Foundation is
>> involved in creating learning content for it, which I presume would
>> emphasise Python, given that Python was the RPi Foundation's first choice
>> for controlling the Pi. The RPi has helped to raise the profile of Python,
>> so hopefully this new device will add to that.
>>
>> I wonder what the other two languages are?  I'd guess a C++ subset for
>> one, like the Arduino.  Not sure about the other.  HTML/Javascript is
>> becoming popular in education because it's the easiest way put shapes and
>> animation on the screen, but I don't think it's suited to an embedded
>> device.
>>
>http://www.bbc.co.uk/news/technology-31834927 says:
>
>"""When it launches in September it will be compatible with three
>coding languages - Touch Develop, Python and C++."""

I program my Pis in Rexx (Regina), Pascal (Free) and a bit of Python.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: More or less code in python?

[Dave Angel]

On 03/15/2015 03:09 PM, jonas.thornv...@gmail.com wrote:

Den söndag 15 mars 2015 kl. 20:01:36 UTC+1 skrev Paul Rubin:

jonas.thornv...@gmail.com writes:

I though it would be interesting doing comparissons in timing adding
massive digits in different bases. Especially in Python.


Python has built-in bignums.  Try "print 2**500".


I will try implement the common operators + - * / pow sqrt operators mod floor 
and and a generic parser taking arguments in any base.



Why on earth would you bother?  Use the int(x, base) to convert a string 
in an arbitrary base to an int.  Write a similar function to convert an 
int back into a string of the desired base.  Of course, your strings 
aren't in a standard form, so you'd have to do some work to get those 
instead.  Are you really intending to use a base beyond 36?


Then to add two numbers, use + To multiply, use *, and so on.

Assuming Python 3.x of course.  if you're in Python 2, you'd use "long" 
rather than int.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: More or less code in python?

[Chris Angelico]

On Mon, Mar 16, 2015 at 6:32 PM, Dave Angel  wrote:
> Assuming Python 3.x of course.  if you're in Python 2, you'd use "long"
> rather than int.

Not sure you need to bother. Even in Py2, you can use the int
constructor to get a long.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Ian Kelly]

On Mon, Mar 16, 2015 at 1:12 AM, Marko Rauhamaa  wrote:
> I was actually referring to the offered API. It still seems to me like
> all iterators could offer close(), send() and throw(). Why? To make all
> iterators drop-in replacements of each other.

The purpose of close, send and throw is to implement *coroutines*, not
iterators. The fact that coroutines in Python happen to be implemented
as a type of iterator doesn't mean that all iterators need to have
them. You're free to add these methods to non-generator iterators that
you write in order to create iterators that pretend to be coroutines,
but I'm having a hard time seeing what purpose would be served by
this.

If on the other hand these methods were to be added to the iterator
protocol, it would just create unnecessary implementation overhead for
the 99.99% of non-generator iterators that are under no illusions
about the fact that they aren't coroutines.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Marko Rauhamaa]

Chris Angelico :

> On Mon, Mar 16, 2015 at 6:12 PM, Marko Rauhamaa  wrote:
>>
>> I was actually referring to the offered API. It still seems to me like
>> all iterators could offer close(), send() and throw(). Why? To make all
>> iterators drop-in replacements of each other.
>
> [...]
>
> That just adds unnecessary overhead to every iterator.

Trivial implementations are a normal way to satisfy an API. That's why
/dev/null implements a close() method, for example.

> Also, what happens when you throw something into iter([1,2,3]) ? Or
> send it a value? What happens?

I would expect iter([1, 2, 3]) to behave highly analogously to
(x for x in [1, 2, 3]).

===begin /tmp/test.py===
#!/usr/bin/env python3

def main():
it = (x for x in [1, 2, 3])
print(next(it))
while True:
try:
print(it.send("hello"))
except StopIteration:
break

if __name__ == "__main__":
main()
===end /tmp/test.py=


$ python3 /tmp/test.py
1
2
3


but:

===begin /tmp/test.py===
#!/usr/bin/env python3

def main():
it = iter([1, 2, 3])
print(next(it))
while True:
try:
print(it.send("hello"))
except StopIteration:
break

if __name__ == "__main__":
main()
===end /tmp/test.py=


$ python3 /tmp/test.py
1
Traceback (most recent call last):
  File "/tmp/test.py", line 13, in 
main()
  File "/tmp/test.py", line 8, in main
print(it.send("hello"))
AttributeError: 'list_iterator' object has no attribute 'send'



Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: More or less code in python?

[Dave Angel]

On 03/16/2015 03:36 AM, Chris Angelico wrote:

On Mon, Mar 16, 2015 at 6:32 PM, Dave Angel  wrote:

Assuming Python 3.x of course.  if you're in Python 2, you'd use "long"
rather than int.


Not sure you need to bother. Even in Py2, you can use the int
constructor to get a long.



Actually I was talking about the terminology.  In Python 2.x, you'd be 
talking about longs to make it clear that they were (essentially) 
unlimited in size.  For the code itself, I doubt if you'd want to make 
any distinction at all, as you say.




--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Marko Rauhamaa]

Ian Kelly :

> If on the other hand these methods were to be added to the iterator
> protocol, it would just create unnecessary implementation overhead for
> the 99.99% of non-generator iterators that are under no illusions
> about the fact that they aren't coroutines.

Why should

   (x for x in [1, 2, 3])

be a valid coroutine but

   iter([1, 2, 3])

not?

Anyway, calling close() on an iterator can be a necessary requirement
even in ordinary generator usage. Only now they have to know if the
underlying iterator was sired by a generator or some other constructor.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Steven D'Aprano]

On Monday 16 March 2015 16:39, Chris Angelico wrote:

> On Mon, Mar 16, 2015 at 4:07 PM, Paul Rubin 
> wrote:
>> Python 2 is by now pretty solid and its users don't feel like beta
>> testers any more.  If you're saying using Python 3 by contrast means
>> "being first" and "reporting bugs", that basically translates to "stay
>> away from it except for experimentation".
> 
> Ah but it isn't Py3 that's all about being first - it's the latest
> version of some third-party module. This entire discussion came about
> because of non-stdlib modules - Python itself is quite mature. You
> might be the first to use XYZ with Py3, but you're not the first to
> use XYZ, nor the first to use Py3.

I think that's a distinction that makes no difference. Somebody has to be 
the first to migrate a large project to Python3+XYZ. The fact that others 
have successfully used Python2+XYZ or Python3+UVW or PHP+ABC is no comfort.

Let's not look at this through rose-tinted glasses. Big software projects 
are complicated, complex and even convoluted and major changes to libraries, 
languages or operating systems can break them. John ran into that, and bless 
him for reporting the bugs he found instead of keeping them secret so the 
next guy will run into them too.

It may, or may not, turn out that in hindsight there might have been better 
ways to manage the Python2-3 transaction. Let's be honest, a lot of the 
changes could have been introduced incrementally: with the possibly 
exception of the Unicode string business, I don't think anything *needed* a 
"flag day" cutover, they could have been handled by an incremental 
transition, e.g.:

2.5 add dict.viewkeys
2.6 noisily deprecate dict.keys and dict.iterkeys
2.7 remove dict.keys and dict.iterkeys and alias viewkeys to keys
2.8 deprecate viewkeys
2.9 remove viewkeys alias

Module renames could be handled via stub modules. Even Unicode strings could 
hypothetically have been added via a __future__ import.

BUT... would that actually have improved matters? I doubt it. It would have 
drawn the pain out much longer: every single new release will break 
*something*, and library authors will likely have a much harder job of 
supporting multiple versions.

I would have meant that users would be stuck with long periods of annoying 
deprecation warnings that they can't do anything about, and exponentially 
increased the pressure on the core devs to hold off removing deprecated 
features. What would have happened in practice probably would have been:

2.5 add dict.viewkeys
2.6 noisily deprecate dict.keys and dict.iterkeys
2.7 change deprecation to silent by default

and there it would stay, forever.

I suspect that without the discipline and bloody-mindedness to just say NO 
to requests to delay removal, Python would follow all those other languages 
that have an ever-increasing pile of cruft that can never be removed because 
it will break somebody's code. Design mistakes, obsolete features, 
misspellings, "it's like that for historical reasons", and other nasties.

Let's be frank: the average programmer has the aesthetic sense of a 
hyperactive weasel on LSD and wouldn't know a cleanly designed language if 
it fell from the sky and hit them on the head. Hence the popularity of PHP, 
Perl and C++. And the pointy-haired bosses holding the budget have even 
less, and zero care factor. "What do you mean the Python update broke our 
application *again*?"

With the Python3000 strategy, most user-space application developers can put 
off the pain until they are ready to deal with it, or put it off forever if 
they don't mind lacking an upgrade path beyond a certain point. Sometimes 
applications are just *finished*, there are no more features that need 
adding, no bugs that need fixing, and the only security feature you need is 
to lock the doors to the office when you leave at night. Those people have 
it easy: they can just get off the treadmill and ignore Python 3.x 
(2.7/2.6/2.5/2.4 ...) forever.



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Steven D'Aprano]

On Monday 16 March 2015 17:17, Paul Rubin wrote:

> Chris Angelico  writes:
>> Ah but it isn't Py3 that's all about being first - it's the latest
>> version of some third-party module.
> 
> You know, one of the attractions of Python used to be that it came with
> a powerful enough standard library that you didn't really need third
> party modules very often.

The std lib is *batteries* included. If you need a nuclear reactor, you turn 
to third-party frameworks and libraries like Twisted, Zope, numpy, PLY, etc. 
They would be a bad fit in the standard library. It has been said that the 
stdlib is the place good code goes to die.

It's precisely because the stdlib has much stronger backward compatibility 
requirements and a higher reluctance to break things that fast-changing 
projects (including just bug fixes, not just new features) cannot go into 
the stdlib.


-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Design thinking & Python.

[Kishan Thobhani]

Hello Guys, 

I want to get involved to Python Open source organisation. 

I'm an undergrad student & also a full-time dev. My portfolio lives here, 
http://kishanthobhani.com. 

I would like to get involved in community building or related projects. My 
interests groups ranges from education to human centric design thinking. I have 
strong communication skills with appropriate ettiqutes to be part of healthy 
social communal. As part of my workbench i have linear pro-effiency of 
programming philosophy. 

Please guide me how should i get started & retain my contribution by adding 
value to foundation and its vision. 

Thank you everyone. 

Warm Regards,

Kishan Thobhani ( @kishanio )
Jedi Master at Printajoy | Full Stack Software Engineer at Azoi, Inc.
E: thobhanikis...@gmail.com
W: www.kishanthobhani.com
M: +91-972-594-0782


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Kishan Thobhani]

I believe we push towards accepting new paradigms like python 3. If bugs
were found on libraries used in older context were probably because there
wasn't a need of particular feature. & that is how eventually a software
becomes a legacy. There are times when we might want to re-use this library
or legacy software, then it make sense to solve the bug as well as port it
to new version. Timeline for anything build on top-of-any-language doesn't
necessarily have to cope up with evolving language unless needed.

Warm Regards,

Kishan Thobhani
Jedi Master at Printajoy | Full Stack Engineer at Azoi Inc.
W: www.kishanthobhani.com
E: thobhanikis...@gmail.com
M: +91-972-594-0782

On Mon, Mar 16, 2015 at 4:53 AM, Mario Figueiredo  wrote:

> On Sun, 15 Mar 2015 12:05:21 -0700, John Nagle 
> wrote:
>
> >On 3/14/2015 1:00 AM, Marko Rauhamaa wrote:
> >
> >Some of the bugs I listed are so easy to hit that I suspect those
> >packages aren't used much.  Those bugs should have been found years
> >ago.  Fixed, even.  I shouldn't be discovering them in 2015.
> >
> >I appreciate all the effort put in by developers in fixing these
> >problems.  Python 3 is still a long way from being ready for prime
> >time, though.
> >
>
> What do you mean a long way? Is this a scaremongering tactic? Some
> little FUD to poison the minds of anyone thinking moving to Python 3?
> Or you just chose your words poorly?
>
> Because 3rd-party packages don't define whether the language is ready
> for production or not and the bugs you found on the standard library
> shouldn't be much different from other bugs found onPython 2 during
> Python 2 prime time.
>
> What makes you think your anedoctal bugs constitute any sort of
> evidence this programming language isn't ready to be used by the
> public?
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Dict comprehensions - improvement to docs?

[Terry Reedy]

On 3/16/2015 2:45 AM, Frank Millman wrote:


"Ian Kelly"  wrote in message
news:CALwzidnTUifj_L=DSH_8s+z0L44pxVvdpG1+pfz1Tzm=ect...@mail.gmail.com...

On Sun, Mar 15, 2015 at 11:25 PM, Frank Millman 
wrote:

Hi all

I like dict comprehensions, but I don't use them very often, so when I do
I
need to look up the format.



[...]


Dict comprehensions aren't covered in the library reference because
they're a language feature, not a part of the standard library. The
syntax is defined at:

https://docs.python.org/3/reference/expressions.html#dictionary-displays



This is what the library reference says about lists -

"""
Lists may be constructed in several ways:

Using a pair of square brackets to denote the empty list: []
Using square brackets, separating items with commas: [a], [a, b, c]
Using a list comprehension: [x for x in iterable]
Using the type constructor: list() or list(iterable)
"""

This is what it says about dictionaries -

"""
Dictionaries can be created by placing a comma-separated list of key: value
pairs within braces, for example: {'jack': 4098, 'sjoerd': 4127} or {4098:
'jack', 4127: 'sjoerd'}, or by the dict constructor.

class dict(**kwarg)
class dict(mapping, **kwarg)
class dict(iterable, **kwarg)

Return a new dictionary initialized from an optional positional argument and
a possibly empty set of keyword arguments.
"""

If you did not know about dict comprehensions, there is nothing to tell you
that they even exist.

I feel that it should be mentioned.


Given that list comprehensions are mentioned in the library reference, I 
agree that set and dict comprehensions should be also.  Open an issue on 
the tracker pointing to the list entry.




--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Kishan Thobhani]

I believe we push towards accepting new paradigms like python 3. If bugs were 
found on libraries used in older context were probably because there wasn't a 
need of particular feature. & that is how eventually a software becomes a 
legacy. There are times when we might want to re-use this library or legacy 
software, then it make sense to solve the bug as well as port it to new 
version. Timeline for anything build on top-of-any-language doesn't necessarily 
have to cope up with evolving language unless needed.  


> On 16-Mar-2015, at 4:53 am, Mario Figueiredo  wrote:
> 
> On Sun, 15 Mar 2015 12:05:21 -0700, John Nagle 
> wrote:
> 
>> On 3/14/2015 1:00 AM, Marko Rauhamaa wrote:
>> 
>>   Some of the bugs I listed are so easy to hit that I suspect those
>> packages aren't used much.  Those bugs should have been found years
>> ago.  Fixed, even.  I shouldn't be discovering them in 2015.
>> 
>>   I appreciate all the effort put in by developers in fixing these
>> problems.  Python 3 is still a long way from being ready for prime
>> time, though.
>> 
> 
> What do you mean a long way? Is this a scaremongering tactic? Some
> little FUD to poison the minds of anyone thinking moving to Python 3?
> Or you just chose your words poorly?
> 
> Because 3rd-party packages don't define whether the language is ready
> for production or not and the bugs you found on the standard library
> shouldn't be much different from other bugs found onPython 2 during
> Python 2 prime time.
> 
> What makes you think your anedoctal bugs constitute any sort of
> evidence this programming language isn't ready to be used by the
> public?
> 
> -- 
> https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Dict comprehensions - improvement to docs?

[Frank Millman]

"Paul Rubin"  wrote in message 
news:87fv95fom0@jester.gateway.sonic.net...
> "Frank Millman"  writes:
>> I like dict comprehensions, but I don't use them very often, so when I do 
>> I
>> need to look up the format.
>
> I never felt a need for them.  Do they generate better code than
>
>   d = dict((k,v) for k,v in [('name','paul'),('language','python')])  ?
>

I ran timeit, and dict comps are quite a bit quicker. This is with Python 
3.4.1 -

C:\>python -m timeit -s "x=range(65, 91); y=(chr(z) for z in x)" "dict((a, 
b) for a, b in zip(x, y))"
10 loops, best of 3: 16.1 usec per loop

C:\>python -m timeit -s "x=range(65, 91); y=(chr(z) for z in x)" "{a: b for 
a, b in zip(x, y)}"
10 loops, best of 3: 6.38 usec per loop

Frank



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Paul Rubin]

Steven D'Aprano  writes:
> It may, or may not, turn out that in hindsight there might have been better 
> ways to manage the Python2-3 transaction. Let's be honest, a lot of the 
> changes could have been introduced incrementally...

Why did the changes have to be introduced at all?

> I suspect that without the discipline and bloody-mindedness to just say NO 
> to requests to delay removal, Python would follow all those other languages 
> that have an ever-increasing pile of cruft that can never be removed because 
> it will break somebody's code. Design mistakes, obsolete features, 
> misspellings, "it's like that for historical reasons", and other nasties.

The idea of "maturity" (an advertised feature of Python 2 back in the
day) is that the pile of features is no longer growing so fast.  Right
now I'm messing with some C code written in the 1980s and it still works
fine.  I have a hard time imagining using Python 3 for anything.  Python
2 has its cruft but it's familiar and comfortable, and Python 3 gets
minimal benefit for most of the breakage that it did (possible exception
of the Unicode changes).  It still has most of the same mistakes and
warts.  If I wanted to get away from those, I'd use Go or Ocaml or
Erlang or Haskell or Javascript (possibly through a compiler like
Purescript) or whatever.

I'll see if I can remember what it was but I found some annoying
incompatibility just a week or so ago that I don't think 2to3 would have
caught, making me scared of Python 3 all over again.  I still think
CPython should have stayed with Python 2 forever, and Python 3 should
have been PyPy-based, letting PyPy drive its design, so that it could
have introduced bigger changes with bigger benefits.  As we're seeing,
even small incompatibilities and breakage cause big resistance, so if
you're going to break stuff at all, you might as well go big.  Most old
programs will never be ported either way.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Paul Rubin]

Steven D'Aprano  writes:
> The std lib is *batteries* included. If you need a nuclear reactor, you turn 
> to third-party frameworks and libraries like Twisted, Zope, numpy, PLY, etc. 

I always thought twisted and zope were monstrosities.  I've used threads
instead of twisted and various other possibilities instead of zope.  Not
sure why numpy couldn't go in the stdlib: does it change all that fast?
PLY is an application not a library.  

> It's precisely because the stdlib has much stronger backward compatibility 
> requirements and a higher reluctance to break things that fast-changing 
> projects (including just bug fixes, not just new features) cannot go into 
> the stdlib.

Has that been a problem for PHP?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Steven D'Aprano]

On Monday 16 March 2015 18:12, Marko Rauhamaa wrote:

> Steven D'Aprano :
> 
>> Marko Rauhamaa wrote:
>>> What features do generator iterators provide on top of generic
>>> iterators?
>>
>> The biggest difference is syntactic. Here's an iterator which returns a
>> never-ending sequence of squared numbers 1, 4, 9, 16, ...
>>
>> class Squares:
>> def __init__(self):
>> self.i = 0
>> def __next__(self):
>> self.i += 1
>> return self.i**2
>> def __iter__(self):
>> return self
>>
>>
>> Here's the same thing written as a generator:
>>
>> def squares():
>> i = 1
>> while True:
>> yield i**2
>> i += 1
> 
> I was actually referring to the offered API. It still seems to me like
> all iterators could offer close(), send() and throw(). Why? To make all
> iterators drop-in replacements of each other.

How could these two iterators *possibly* be drop in replacements for each 
other in any real body of code?


def inorder(tree):
for node in inorder(tree.left):
yield node.payload
yield tree.payload
for node in inorder(tree.right):
yield node.payload


def clean_up(collection, condition, cleaner=None):
if cleaner is None:
def cleaner(obj):
try:
obj.cache = {}
obj.ref = None
except Exception:
return False
return True
for i, item in enumerate(collection):
if condition(item):
flag = cleaner(item)
if flag:
yield Warn(MSG % (i, item))


You can't just substitute any random iterator for another, any more than you 
can substitute any random two functions for each other.

Overly pure (in the comp sci theory sense) interfaces can be silly. Just 
because the Soyez rocket and my nephew's tricycle are both instances of 
Vehicle doesn't mean that I should be able to substitute one for the other. 
OOP theory says they should. Reality says No Way.

Why should I, the class designer, be forced to offer send(), throw() and 
close() methods on my iterators if I'm never going to use them as 
coroutines? If I .send() into either of the generators above, its a 
conceptual mistake and I should get an error. The fact that I don't is an 
implementation detail, and one which hopefully will be fixed. I expect that 
the interpreter can tell the difference between 

yield spam

and 

x = yield spam

and only allow send(), close() and throw() on generators which include the 
second form. If it can't, then that's a limitation, not a feature to be 
emulated.


> Then, you could also stop wondering what to call the thingy returned by
> a generator. 

Apart from a few die-hards who are hoping for some sort of Académie 
Française to define the One True meaning of words, most of us stopped 
wondering what to call the thingy returned by a generator function years 
ago. It's a generator, just like introspection of the object tells you.


> Why, it would be an iterator. You wouldn't then have any
> other use for a generator than the function that returns an iterator.

Lots of things are iterators. Doesn't make them generators.


> You could then decide if you want to reserve the name "generator" for
> functions that contain a yield statement (syntactic notion) or call any
> function returning an iterator a "generator" (semantic notion).

Why would you want the second? Do you have special names for {all functions 
which return strings} and {all functions which return bools}?

Functions are normally described by their *form* or *purpose*, not their 
return result, e.g. inner function, public function, callback function, 
reentrant function, filter function.


-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Dict comprehensions - improvement to docs?

[Paul Rubin]

"Frank Millman"  writes:
> dict((a, b) for a, b in zip(x, y))
> 10 loops, best of 3: 16.1 usec per loop
> {a: b for a, b in zip(x, y)}"
> 10 loops, best of 3: 6.38 usec per loop

Hmm, I bet the difference is from the (a,b) consing all those tuples.

Can you try just dict(zip(x,y))  ?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Chris Angelico]

On Mon, Mar 16, 2015 at 6:56 PM, Steven D'Aprano
 wrote:
> [ a whole lot of stuff that I agree with, and then ... ]
> Module renames could be handled via stub modules. Even Unicode strings could
> hypothetically have been added via a __future__ import.

This part I don't agree with. The problem with the Unicode / bytes
distinction is that library functions are written to expect one or the
other. Even in the stdlib, issues have been found, and the future
import can't solve that. You can already create Unicode strings in
Py2, with the u"..." syntax; you can even make it the default with
unicode_literals; but you can't magically make functions in other
modules cope with the other type of string. (Not with a __future__
import, anyway. Maybe "from __fairy_godmother__ import pumpkin"?)
Whenever it's easy, I encourage people to write their Py2 code to be
Py3-compatible - put parens around their print calls, be clear about
integer vs float division, use the "as" syntax for exception catching
- but I don't encourage from __future__ import unicode_literals,
because it tends to just move problems around rather than solving
them.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Dict comprehensions - improvement to docs?

[Frank Millman]

"Terry Reedy"  wrote in message 
news:me644f$bqj$1...@ger.gmane.org...
>
> Given that list comprehensions are mentioned in the library reference, I 
> agree that set and dict comprehensions should be also.  Open an issue on 
> the tracker pointing to the list entry.
>
>

http://bugs.python.org/issue23677

Frank



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Terry Reedy]

On 3/16/2015 1:07 AM, Paul Rubin wrote:


I saved a quote from Hacker News a while back (I don't know who the
author is):

 "You know why I'm not running python 3?

>   Because it doesn't solve a single problem I have.

Quite possibly true.

>It doesn't solve anyone's problems. It solves

 imaginary problems, while creating real problems."


A blatent, selfish lie.


   https://news.ycombinator.com/item?id=7802575

I think the person went a bit overboard,


I call it more than a bit overboard.  But anyway ...


but other than the Unicode revamp I don't know what Python 3

> improvements couldn't have been done

in Python 2 without breaking anything.


Every change potentially breaks something.  (How would you have changed 
1/2 from 0 to .5 without breaking anything?) The issue is cost versus 
gain.  We do maintenance bugfix releases because the gain for bugfixes, 
and the gain of applying them immediately, is *usually* considered 
greater than the cost.  Some bugfixes are delayed to the 'next' version. 
 Non-bugfix changes require deprecation for at least one version before 
making the change.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Chris Angelico]

On Mon, Mar 16, 2015 at 7:36 PM, Steven D'Aprano
 wrote:
> I expect that the interpreter can tell the difference between
>
> yield spam
>
> and
>
> x = yield spam
>
> and only allow send(), close() and throw() on generators which include the
> second form. If it can't, then that's a limitation, not a feature to be
> emulated.
>

Hmm. That would imply that an expression is different if it's used
somewhere. In Python, there's no difference between these two function
calls:

x = func(1,2,3)
func(1,2,3)

except for the actual name binding. If a yield expression enabled
send() if and only if its return value were used, then it would be
extremely surprising:

def use_and_discard():
while True:
_ = (yield 1)

def just_yield():
while True:
yield 1

So... I can send into the first but not into the second?

That said, though, it would be quite reasonable for a *linter* to warn
you about sending into a generator that never uses sent values. With a
good type inference system (not sure if MyPy is sufficient here), it
would be possible to distinguish between those two functions and
declare that the first one has the return type "sendable_generator"
and the second one "nonsendable_generator", and give a warning if you
send into the latter. But I don't think that's the language's job.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Dict comprehensions - improvement to docs?

[Frank Millman]

"Paul Rubin"  wrote in message 
news:87wq2hfibu@jester.gateway.sonic.net...
> "Frank Millman"  writes:
>> dict((a, b) for a, b in zip(x, y))
>> 10 loops, best of 3: 16.1 usec per loop
>> {a: b for a, b in zip(x, y)}"
>> 10 loops, best of 3: 6.38 usec per loop
>
> Hmm, I bet the difference is from the (a,b) consing all those tuples.
>
> Can you try just dict(zip(x,y))  ?

C:\>python -m timeit -s "x = range(65, 91); y = (chr(z) for z in x)" 
"dict(zip(x, y))"
10 loops, best of 3: 11.9 usec per loop

C:\>python -m timeit -s "x = range(65, 91); y = (chr(z) for z in x)" "{a: b 
for a, b in zip(x, y)}"
10 loops, best of 3: 7.24 usec per loop

I ran the dict comp again in case the machine load had changed - it is a bit 
slower, but not much.

Frank



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[INADA Naoki]

google-api-client has ported from PY2 to PY2/3 recently.
https://github.com/google/google-api-python-client

It has 4000 lines of code and 3200 lines of tests.  It is legacy
library. It is indented by 2 spaces
since it was created before Google changes their coding style.

There are two major pull request: automatic change by modernize and
heuristic changes.
https://github.com/google/google-api-python-client/pull/62
https://github.com/google/google-api-python-client/pull/64

I feel this is reasonable effort, not bad.


Another experience is porting Flask application in my company from
Python 2 to Python 3.
It has 26k lines of code and 7.6k lines of tests.

Since we don't need to support both of PY2 and PY3, we used 2to3.
2to3 changes 740 lines. I had to replace google-api-client with
requests+oauthlib since
it had not supported PY3 yet.

After that, we encountered few trouble with untested code. But Porting
effort is surprisingly small.
We're happy now with Python 3.  We can write non-ascii string to log
without fear of UnicodeError.
We can use csv with unicode without hack.


Porting *modern* *application* code to *PY3 only* is easy, while
porting libraries on the edge of
bytes/unicode like google-api-client to PY2/3 is not easy.

I think application developers should use *only* Python 3 from this year.
If we start moving, more library developers will be able to start
writing Python 3 only code from next year.

On Mon, Mar 16, 2015 at 5:25 PM, Paul Rubin  wrote:
> Steven D'Aprano  writes:
>> It may, or may not, turn out that in hindsight there might have been better
>> ways to manage the Python2-3 transaction. Let's be honest, a lot of the
>> changes could have been introduced incrementally...
>
> Why did the changes have to be introduced at all?
>
>> I suspect that without the discipline and bloody-mindedness to just say NO
>> to requests to delay removal, Python would follow all those other languages
>> that have an ever-increasing pile of cruft that can never be removed because
>> it will break somebody's code. Design mistakes, obsolete features,
>> misspellings, "it's like that for historical reasons", and other nasties.
>
> The idea of "maturity" (an advertised feature of Python 2 back in the
> day) is that the pile of features is no longer growing so fast.  Right
> now I'm messing with some C code written in the 1980s and it still works
> fine.  I have a hard time imagining using Python 3 for anything.  Python
> 2 has its cruft but it's familiar and comfortable, and Python 3 gets
> minimal benefit for most of the breakage that it did (possible exception
> of the Unicode changes).  It still has most of the same mistakes and
> warts.  If I wanted to get away from those, I'd use Go or Ocaml or
> Erlang or Haskell or Javascript (possibly through a compiler like
> Purescript) or whatever.
>
> I'll see if I can remember what it was but I found some annoying
> incompatibility just a week or so ago that I don't think 2to3 would have
> caught, making me scared of Python 3 all over again.  I still think
> CPython should have stayed with Python 2 forever, and Python 3 should
> have been PyPy-based, letting PyPy drive its design, so that it could
> have introduced bigger changes with bigger benefits.  As we're seeing,
> even small incompatibilities and breakage cause big resistance, so if
> you're going to break stuff at all, you might as well go big.  Most old
> programs will never be ported either way.
> --
> https://mail.python.org/mailman/listinfo/python-list



-- 
INADA Naoki  
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Paul Rubin]

Terry Reedy  writes:
> Every change potentially breaks something.  (How would you have
> changed 1/2 from 0 to .5 without breaking anything?) 

I would not have changed that.  It was an ill-advised change that broke
things unnecessarily.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: You must register a new account to report a bug (was: Python 2 to 3 conversion - embrace the pain)

[Gene Heskett]

On Monday 16 March 2015 02:17:44 Ben Finney wrote:
> Cameron Simpson  writes:
> > To quote Graham Dumpleton:
> >
> >  For years have seen people make vague grumbles about something not
> > working with mod_wsgi. Not one ever reported bug or described
> > problem.
>
> Hmm. How easy is it for someone who, say, an hour ago had no idea they
> would ever want to contact Graham Dumpleton; and then encounters a
> problem with mod_wsgi?
>
> * Searching for ‘mod_wsgi’ leads to a GitHub page. Want to submit a
>   one-off bug report? Too bad, you have to sign up with yet another
>   service.
>
>   No email address is provided for reporting bugs. The person named is
>   Graham Dumpleton; no email address for him is provided.
>
> * If you're persistent enough to contact Graham Dumpleton directly
>   outside the context of the ‘mod_wsgi’ project, what does that get
> you?
>
>   Searching “Graham Dumpleton” on the web gets a bunch of inconclusive
>   hits.
>
>   He has a Twitter account. Want to send him a single bug report? Too
>   bad, you'd have to sign up for yet another service (Twitter).
>
>   He has a weblog, at Blogger. It gives a specific profile for him.
> With no email address. Want to send a one-off bug report there? Too
> bad, there's no contact information given.
>
> * Are there other ways to report a bug? I can't find them, and I
>   searched for rather longer than someone would be who just wants to
>   describe the problem.
>
> I am unsurprised that people don't end up reporting bugs; there's no
> simple way to do it. No wonder Graham Dumpleton finds a lot of people
> grumbling and not reporting bugs.
>
> >  Sadly becoming the norm. People will just whinge and complain but
> > never actually report issues in Open Source.
>
> Sadly becoming the norm. People will run a software project and just
> assume that users will be willing to go through a registration process
> for every project just to report a bug.

Ben Finney, I hear you, and to me its the most valid lament I have read 
in weeks. I could replay this exact scene for several packages over the 
last 2 or 3 years.  It's BS to have to sign up for yet another stream of 
pure spam from sourceforge or some other equally guilty spammer just to 
report a bug. I object to being more useful as a source to add to  a 
spam list, than as someone who is trying to improve the software by 
reporting the bug.

So guess what?  It doesn't get done. Because of the last 2 bug reports I 
tried to file, I am now getting 5 to 15 spams a day from sourceforge 
that have zilch to do with the package I tried to file a bug against.  
IMNSDHO if the author of a package cannot be bothered to subscribe to 
the mailing list related to his/her package, and accept feedback from 
that mailing list as being the equ of a bug report, perhaps querying the 
user for more details so it is a good bug report, then his buggy package 
deserves to be nibbled to death by ducks.

I am on several lists where the authors do hang out.  And I am the author 
on one of them.  Those lists are the ideal list model all should follow.  
Sadly they are the exception that proves the rule.

I am on the linuxcnc mailing list because I am also a cnc machinist, 
retired hobby grade, although that software certainly is not.  I can 
post about a problem, and how to demonstrate it, and have a new package 
fixing that problem in the repo, to be installed by the package manager, 
typically in under 12 hours.  I have a config problem with a board from 
one of the vendors making and selling the board?  The vendor is there 
and I've had fixes for _my_ missunderstandings that worked perectly in 
15 minutes.

I'd love to see a sig line or hidden header line, with a URL that is that 
projects bug tracker, one that is fully accessible using the lists user 
identity and the same password it takes to log into the server and 
adjust your list preferences.  But that would make it too easy for us to 
disturb some of the prima donna's we seem to have nurtured in the last 
15 years, and we can't possibly be allowed to bother them, can we?

Thanks for bringing up the subject Ben. 
> --
>  \  “[Entrenched media corporations will] maintain the status quo,
> | `\   or die trying. Either is better than actually WORKING for a
> | _o__)  living.” —ringsnake.livejournal.com,
> 2007-11-12 | Ben Finney

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Steven D'Aprano]

Chris Angelico wrote:

> On Mon, Mar 16, 2015 at 7:36 PM, Steven D'Aprano
>  wrote:
>> I expect that the interpreter can tell the difference between
>>
>> yield spam
>>
>> and
>>
>> x = yield spam
>>
>> and only allow send(), close() and throw() on generators which include
>> the second form. If it can't, then that's a limitation, not a feature to
>> be emulated.
>>
> 
> Hmm. That would imply that an expression is different if it's used
> somewhere. In Python, there's no difference between these two function
> calls:
> 
> x = func(1,2,3)
> func(1,2,3)
> 
> except for the actual name binding.

That's a pretty big difference though :-)

Just for the record, it's not just name bindings that make a generator
behave as a coroutine. E.g.:

py> def co():
... print( (yield 1) + 2 )
...
py> a = co()
py> next(a)
1
py> a.send(98)
100
Traceback (most recent call last):
  File "", line 1, in 
StopIteration


Not the most *useful* example, I grant, but it demonstrates the idea.
However, for simplicity let's ignore that and pretend it's only name
binding we care about.


> If a yield expression enabled 
> send() if and only if its return value were used, then it would be
> extremely surprising:

It's surprising to me that you can send() into a non-coroutine and have it
ignored. I'm not even sure that is documented behaviour or just an accident
of implementation.

 
> def use_and_discard():
> while True:
> _ = (yield 1)
> 
> def just_yield():
> while True:
> yield 1
> 
> So... I can send into the first but not into the second?

That's the idea.

The second one is not a coroutine, and cannot be used as a coroutine in any
useful way. I maintain that it is a mistake that you can send() into it at
all, and if somebody does, it is invariably an error. That would be like
send()ing into None, or a str. We know what the Zen says about errors...


> That said, though, it would be quite reasonable for a *linter* to warn
> you about sending into a generator that never uses sent values. With a
> good type inference system (not sure if MyPy is sufficient here), it
> would be possible to distinguish between those two functions and
> declare that the first one has the return type "sendable_generator"
> and the second one "nonsendable_generator", and give a warning if you
> send into the latter. But I don't think that's the language's job.

I do :-)

I think it would be hard for a linter to do this. It can't just do a
type-check, because the types of generator-iterators and
generator-coroutines are the same. It would need to actually analyse the
source code, AST, or byte-code. That's doable, but the compiler already
does that, and compiles different code for the two cases:

py> from dis import dis
py> def g1():
... yield 1
...
py> def g2():
... x = yield 1
...
py> dis(g1)
  2   0 LOAD_CONST   1 (1)
  3 YIELD_VALUE
  4 POP_TOP
  5 LOAD_CONST   0 (None)
  8 RETURN_VALUE
py> dis(g2)
  2   0 LOAD_CONST   1 (1)
  3 YIELD_VALUE
  4 STORE_FAST   0 (x)
  7 LOAD_CONST   0 (None)
 10 RETURN_VALUE


so it should be easy for the compiler to recognise a yield used as if it
were a statement versus one used as an expression, and take appropriate
steps. But maybe there are subtle corner cases I haven't thought of.



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Steven D'Aprano]

INADA Naoki wrote:

> I think application developers should use only Python 3 from this year.
> If we start moving, more library developers will be able to start
> writing Python 3 only code from next year.

Where this is possible, that is an excellent idea.

Alas, too many (Linux) developers insist on using the system Python as their
application language. In most cases they could trivially run "aptitude
install python3" and be done with it, but that's an extra dependency and
therefore Bad.

Red Hat, Debian and Ubuntu are all planning to move to Python 3 Real Soon
Now, and I expect that will give Py3 some more impetus.


-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Steven D'Aprano]

Marko Rauhamaa wrote:

> Chris Angelico :
> 
>> On Mon, Mar 16, 2015 at 6:12 PM, Marko Rauhamaa  wrote:
>>>
>>> I was actually referring to the offered API. It still seems to me like
>>> all iterators could offer close(), send() and throw(). Why? To make all
>>> iterators drop-in replacements of each other.
>>
>> [...]
>>
>> That just adds unnecessary overhead to every iterator.
> 
> Trivial implementations are a normal way to satisfy an API. That's why
> /dev/null implements a close() method, for example.

Absolutely. But you''ll notice it doesn't implement upper() and lower()
methods, because it isn't a string, it's a file.

Iterators shouldn't implement send() etc. methods because they aren't
coroutines.


>> Also, what happens when you throw something into iter([1,2,3]) ? Or
>> send it a value? What happens?
> 
> I would expect iter([1, 2, 3]) to behave highly analogously to
> (x for x in [1, 2, 3]).

I believe that it is an unfortunate mistake that generators which aren't
coroutines support send. At the very least it is an ugly design wart, at
worst it is an outright bug.



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Steven D'Aprano]

Marko Rauhamaa wrote:

> Anyway, calling close() on an iterator can be a necessary requirement
> even in ordinary generator usage. Only now they have to know if the
> underlying iterator was sired by a generator or some other constructor.

Can you give an actual example of that?

(To be honest, I'm not even sure what the use-case for close() on coroutines
is in the first place. If you don't want to send any more items into it,
just don't send any more items into it.)


-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Marko Rauhamaa]

Steven D'Aprano :

> If I .send() into either of the generators above, its a conceptual
> mistake and I should get an error. The fact that I don't is an
> implementation detail, and one which hopefully will be fixed.

So what we have now is:

  (1) plain iterators

  (2) generator iterators

  (3) coroutine generator iterators

At the moment (2) and (3) are unified, which you don't like. (1) and (2)
are not unified, which I don't like.

You and I can pretend (2) didn't have send(), throw() and close() and be
satisfied. Then, generator iterators are just plain iterators from the
point of view of the application programmer.

However, it seems to me close() is intended to be used for (2) as well.
The way I'm reading PEP 342, it seems to encompass the objectives of the
rejected PEP 325  ("Resource-Release Support for Generators"):


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Marko Rauhamaa]

Steven D'Aprano :

> (To be honest, I'm not even sure what the use-case for close() on
> coroutines is in the first place. If you don't want to send any more
> items into it, just don't send any more items into it.)

Without close(), you might leave resources hanging. See PEP 325:

Rejected in favor of PEP 342 which includes substantially all of the
requested behavior in a more refined form.

And PEP 342:

Raymond Hettinger (PEP 288) and Samuele Pedroni (PEP 325) first
formally proposed the ideas of communicating values or exceptions
into generators, and the ability to "close" generators.

One *might* take the stance that resource allocation is outside the
scope of iterators, generator iterators and coroutine generator
iterators. As it stands, it is unclear if the application *must* call
close() on generator iterators that are left hanging.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Jonas Wielicki]

On 16.03.2015 13:02, Steven D'Aprano wrote:
> (To be honest, I'm not even sure what the use-case for close() on coroutines
> is in the first place. If you don't want to send any more items into it,
> just don't send any more items into it.)

Just like with file-likes, it is useful to clean up resources. If you
use a coroutine with the lxml.etree.xmlfile interface [1] for instance
(code quoted from the reference):

def writer(out_stream):
with xmlfile(out_stream) as xf:
 with xf.element('{http://etherx.jabber.org/streams}stream'):
  try:
  while True:
  el = (yield)
  xf.write(el)
  xf.flush()
  except GeneratorExit:
  pass

This allows controlled (i.e. not governed by garbage collection) and
clean shutdown of a coroutine (albeit I’m not sure why they catch the
GeneratorExit here), which will close the opened  tag.

regards,
jwi

   [1]: http://lxml.de/api.html#incremental-xml-generation






signature.asc
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Rustom Mody]

On Monday, March 16, 2015 at 6:02:48 PM UTC+5:30, Marko Rauhamaa wrote:
> So what we have now is:
> 
>   (1) plain iterators
> 
>   (2) generator iterators
> 
>   (3) coroutine generator iterators
> 

> (1) and (2) are not unified, which I don't like.

Can you expand on that a bit?

It may help to read this 15 year old thread starting
https://mail.python.org/pipermail/python-dev/2001-June/015478.html
to see how many python stalwarts dont like the current state of affairs
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Steven D'Aprano]

On Mon, 16 Mar 2015 07:31 pm, Paul Rubin wrote:

> Steven D'Aprano  writes:
>> The std lib is *batteries* included. If you need a nuclear reactor, you
>> turn to third-party frameworks and libraries like Twisted, Zope, numpy,
>> PLY, etc.
> 
> I always thought twisted and zope were monstrosities.  I've used threads
> instead of twisted and various other possibilities instead of zope.  Not
> sure why numpy couldn't go in the stdlib: does it change all that fast?
> PLY is an application not a library.

I don't know whether numpy changes *fast* or not, but its release cycle is
nothing like CPython's release cycle and it would be impractical to
syncronise the two. numpy is also awfully specialised, with a bunch of
dependencies like Fortran compilers and BLAS.

As for PLY, I'm sorry that was a typo I actually was thinking of PIL.


>> It's precisely because the stdlib has much stronger backward
>> compatibility requirements and a higher reluctance to break things that
>> fast-changing projects (including just bug fixes, not just new features)
>> cannot go into the stdlib.
> 
> Has that been a problem for PHP?

You surely aren't looking at PHP for best practices are you? PHP doesn't
just break backwards compatibility on major updates like 4 to 5, but on
minor updates like 5.2 to 5.3:

http://php.net/manual/en/migration53.incompatible.php

Of course, "problem" seems to be relative. PHP culture seems to accept that
code will break when you do a minor upgrade, and it's no big deal to do a
search-and-replace and rename your functions. The same thing would be
considered unacceptable in Python and unthinkable in C.

http://stackoverflow.com/questions/4693575/is-php-5-3-backwards-compatible-with-php-5-2



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Setuptools: no module named 'html.entities'

[Wolfgang Maier]

On 03/16/2015 12:53 AM, Jason Friedman wrote:

Hello,

This is Python 3.3.2 on Linux.
I downloaded Setuptools
(https://pypi.python.org/packages/source/s/setuptools/setuptools-14.3.tar.gz),
exploded the tarball, and I get:

python setup.py build
Traceback (most recent call last):
   File "", line 1521, in
_find_and_load_unlocked
AttributeError: 'module' object has no attribute '__path__'

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
   File "setup.py", line 21, in 
 exec(init_file.read(), command_ns)
   File "", line 11, in 
   File "/home/spjsf/setuptools-14.3/setuptools/__init__.py", line 11,
in 
 from setuptools.extension import Extension
   File "/home/spjsf/setuptools-14.3/setuptools/extension.py", line 8,
in 
 from .dist import _get_unpatched
   File "/home/spjsf/setuptools-14.3/setuptools/dist.py", line 16, in

 from setuptools.depends import Require
   File "/home/spjsf/setuptools-14.3/setuptools/depends.py", line 6, in

 from setuptools import compat
   File "/home/spjsf/setuptools-14.3/setuptools/compat.py", line 44, in

 from html.entities import name2codepoint
ImportError: No module named 'html.entities'; html is not a package




Not sure, but maybe you have a html.py somewhere in your module search 
path taking precedence over the stdlib html package ?
Putting a dummy html.py file into the extracted setuptools folder, at 
least, lets me reproduce your exact error.


What does

python -c "import html; print(html)"

tell you ?

Best,
Wolfgang

--
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Marko Rauhamaa]

Rustom Mody :

> On Monday, March 16, 2015 at 6:02:48 PM UTC+5:30, Marko Rauhamaa wrote:
>> So what we have now is:
>>   (1) plain iterators
>>   (2) generator iterators
>>   (3) coroutine generator iterators
>
>> (1) and (2) are not unified, which I don't like.
>
> Can you expand on that a bit?

(2) contains methods, most notably close(), that (1) does not provide.
If I get an iterator from a black box source, I don't know if I'm
allowed/supposed to call close() on it.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Chris Angelico]

On Mon, Mar 16, 2015 at 10:51 PM, Steven D'Aprano
 wrote:
> Chris Angelico wrote:
> Just for the record, it's not just name bindings that make a generator
> behave as a coroutine. E.g.:
>
> py> def co():
> ... print( (yield 1) + 2 )
> ...
> py> a = co()
> py> next(a)
> 1
> py> a.send(98)
> 100
> Traceback (most recent call last):
>   File "", line 1, in 
> StopIteration

Yep, I agree. It's not the name binding, it's whether the yield
expression's result is instantly discarded vs used in some way.

> It's surprising to me that you can send() into a non-coroutine and have it
> ignored. I'm not even sure that is documented behaviour or just an accident
> of implementation.

I agree conceptually; but there are two different consistencies here,
and they clash, my lords, they clash! One is that the API for a
coroutine includes send() but the API for a non-coroutine generator
doesn't, and that trying to send() into the latter should raise an
error. (Side point: What error? I'd accept AttributeError, but since
it's possible to mix sendable and nonsendable yields, it might be
better to have it raise at the time of the call - that is, when you
send a non-None value into a generator that's just going to discard
the value, it raises on arrival. In that case, possibly ValueError?
GeneratorStyleError?) And on the other hand, we have a very strong
Python convention that any expression can be used as a statement
without altering the effect of that expression in any way. Assigning
something to a name that you never use, or passing it as an argument
to a no-op function, should not suddenly change the behaviour of
anything.

>> That said, though, it would be quite reasonable for a *linter* to warn
>> you about sending into a generator that never uses sent values. With a
>> good type inference system (not sure if MyPy is sufficient here), it
>> would be possible to distinguish between those two functions and
>> declare that the first one has the return type "sendable_generator"
>> and the second one "nonsendable_generator", and give a warning if you
>> send into the latter. But I don't think that's the language's job.
>
> I do :-)
>
> I think it would be hard for a linter to do this. It can't just do a
> type-check, because the types of generator-iterators and
> generator-coroutines are the same. It would need to actually analyse the
> source code, AST, or byte-code. That's doable, but the compiler already
> does that, and compiles different code for the two cases:
>
> py> from dis import dis
> py> def g1():
> ... yield 1
> ...
> py> def g2():
> ... x = yield 1
> ...
> py> dis(g1)
>   2   0 LOAD_CONST   1 (1)
>   3 YIELD_VALUE
>   4 POP_TOP
>   5 LOAD_CONST   0 (None)
>   8 RETURN_VALUE
> py> dis(g2)
>   2   0 LOAD_CONST   1 (1)
>   3 YIELD_VALUE
>   4 STORE_FAST   0 (x)
>   7 LOAD_CONST   0 (None)
>  10 RETURN_VALUE
>
>
> so it should be easy for the compiler to recognise a yield used as if it
> were a statement versus one used as an expression, and take appropriate
> steps. But maybe there are subtle corner cases I haven't thought of.

That code isn't actually all that different, though. What you have is this:

1) Load up a value
2) Yield, which pops the value off, yields it, and pushes the sent value or None
3a) in g1: Discard the value of the expression that we've just finished
3b) in g2: Store the value of that expression into x
4) Load None, and return it.

Steps 1, 3, and 4 would all be identical if you replace the "yield 1"
with, say, "print(1)". It'll pop the 1 off, do something, and leave a
result on the stack. The difference between assignment and
non-assignment comes later.

The risk I see here is that a simple optimization might suddenly make
a semantic change to something. Imagine this function:

def gen():
while True:
x = (yield 1)
if x == 5: break

Okay, so it cares about sent values. Well and good. Then we change it
so it doesn't always care:

def gen(state):
while True:
x = (yield 1)
if state and x == 5: break

Ahh but look! That's a mode-switch parameter. We should break that out
into two functions.

def gen_stoppable():
while True:
x = (yield 1)
if x == 5: break

def gen_unstoppable():
while True:
x = (yield 1)

Now, according to the current rules, you can change that last line to
just "yield 1" without the two functions differing in API. There's no
sent value that will stop the second one, but it will accept and
ignore sent values, just as the first one does. (Imagine that's some
sort of password, and the second generator is for the case where the
account is locked and there IS no password.) But according to your
rules, changing it to no longer assign the yielded value somewhere
would make a semantic and API change to the function. That seems, to
me, at 

Re: Setuptools: no module named 'html.entities'

[Jason Friedman]

On Mon, Mar 16, 2015 at 7:10 AM, Wolfgang Maier <
wolfgang.ma...@biologie.uni-freiburg.de> wrote:

> On 03/16/2015 12:53 AM, Jason Friedman wrote:
>
>> Hello,
>>
>> This is Python 3.3.2 on Linux.
>> I downloaded Setuptools
>> (https://pypi.python.org/packages/source/s/setuptools/
>> setuptools-14.3.tar.gz),
>> exploded the tarball, and I get:
>>
>> python setup.py build
>> Traceback (most recent call last):
>>File "", line 1521, in
>> _find_and_load_unlocked
>> AttributeError: 'module' object has no attribute '__path__'
>>
>> During handling of the above exception, another exception occurred:
>> Traceback (most recent call last):
>>File "setup.py", line 21, in 
>>  exec(init_file.read(), command_ns)
>>File "", line 11, in 
>>File "/home/spjsf/setuptools-14.3/setuptools/__init__.py", line 11,
>> in 
>>  from setuptools.extension import Extension
>>File "/home/spjsf/setuptools-14.3/setuptools/extension.py", line 8,
>> in 
>>  from .dist import _get_unpatched
>>File "/home/spjsf/setuptools-14.3/setuptools/dist.py", line 16, in
>> 
>>  from setuptools.depends import Require
>>File "/home/spjsf/setuptools-14.3/setuptools/depends.py", line 6, in
>> 
>>  from setuptools import compat
>>File "/home/spjsf/setuptools-14.3/setuptools/compat.py", line 44, in
>> 
>>  from html.entities import name2codepoint
>> ImportError: No module named 'html.entities'; html is not a package
>>
>>
>>
> Not sure, but maybe you have a html.py somewhere in your module search
> path taking precedence over the stdlib html package ?
> Putting a dummy html.py file into the extracted setuptools folder, at
> least, lets me reproduce your exact error.
>
> What does
>
> python -c "import html; print(html)"
>
> tell you ?
>
> Bingo!  Much obliged.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Chris Angelico]

On Tue, Mar 17, 2015 at 12:05 AM, Steven D'Aprano
 wrote:
> Of course, "problem" seems to be relative. PHP culture seems to accept that
> code will break when you do a minor upgrade, and it's no big deal to do a
> search-and-replace and rename your functions. The same thing would be
> considered unacceptable in Python and unthinkable in C.

Because, after all, it's no problem to keep a dozen PHP versions
around, and to select which one you want via a shebang at the top of
the file... oh wait. So how do you cope with breakage? Massive code
churn? What about those big PHP applications/engines/frameworks like
MediaWiki, WordPress, etc - do they maintain multiple versions that
are compatible with different PHP versions??

No, don't answer any of those questions. We don't need to discuss this
in detail. *sigh*

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Steven D'Aprano]

On Mon, 16 Mar 2015 11:51 pm, Rustom Mody wrote:

> It may help to read this 15 year old thread starting
> https://mail.python.org/pipermail/python-dev/2001-June/015478.html
> to see how many python stalwarts dont like the current state of affairs

/s/dont/didn't/

That's a fifteen year old thread, written before generators were introduced.
Guido's intuition has been vindicated -- generators have proven to be a
great and powerful feature, and the reuse of "def" for both generator
functions and regular functions has turned out to be no more confusing in
practice than the use of "def" for both functions and methods[1].


The argument is that generator-producers (def-with-yield) are not like
regular functions because calling the def-with-yield doesn't execute the
code inside them. Given three objects:

def f(): return 1

def g(): yield 1

class C: ...

the argument goes that:

* Calling f() executes the code inside f.

* Calling g() *doesn't* execute the code inside f, but runs 
  some initialisation code and returns an instance; it's the
  instance that executes the code inside f, using a special
  method.

* Calling C() also doesn't execute the code inside C, but 
  runs some initialisation code and returns an instance.


Therefore def-with-yield is closer to a class than a function and should not
reuse the same keyword. Or so the argument goes.

But this argument doesn't take into account the way generators are used.
They are used like functions, not like instances[2] like this:

# No
instance = gen()
instance.fe()
instance.fi()
instance.fo()
instance.fum()


but like functions, like this:

# Yes
for x in gen(): ...
sum(gen())
result = map(func, gen())


Here's a pure-Python version of map from Python 2:

def map(func, iterable):
 accum = []
 for item in iterable:
 accum.append(func(item))
 return accum

Here's the same thing for the Python 3 iterator version:

def map(func, iterable):
 for item in iterable:
 yield func(item)


The similarities when writing these are undeniable. Drop the accumulator,
replace any lines that store values into the accumulator with a yield, and
drop the final return. The way we write and use generators is closer to the
way we write and use functions than classes. If we rule out introducing a
new keyword, and insist on picking either "def" or "class", I think it is
obvious that "def" is the better choice and "class" would be completely
inappropriate. And of course, from a comp science theoretic perspective,
generators are a kind of subroutine, not a kind of type.




[1] Methods are, of course, actually just functions under the hood. The
conversion to MethodType is done at method-lookup time, by the descriptor
protocol.

[2] I'm aware that they are instances. You know what I mean.


-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Rustom Mody]

On Monday, March 16, 2015 at 7:10:03 PM UTC+5:30, Steven D'Aprano wrote:
> And of course, from a comp science theoretic perspective,
> generators are a kind of subroutine, not a kind of type.

You just showed Marko a few posts back, that
A generator is a special case of an iterator.
And you wrote the iterator with 'class'.
And from python 2.2(?) classes are types.

So I dont know what "comp science theoretic perspective" you are describing...

From mine:

The generator
def gen():
  yield 1
  yield 2

is much closer to the list (ie data) [1,2]
than to say

def foo():
  print 1
  print 2

The only difference is that the list memoizes the data whereas the generator 
doesn't.

CLU perspective:
The iterator for a collection of complex_numbers can be used interchangeably 
with that for an array of integers
from https://en.wikipedia.org/wiki/CLU_%28programming_language%29
[Note: CLU-iterator ≡ python-generator]

Haskell perspective: 
Lists are by default lazy and memoized.
IOW in Haskell: List ≡ Lazy list ≡ python generator + memoization

Scheme perspective:
Programmer can choose between normal and lazy lists.
Lazy lists are just normal lists + delay/force where
delay x ≡ lambda: x
force x ≡ x()

==
Anyways...

Yes 15 years are past.
I dont expect the def can be revoked now.
[As far as I am concerned its a minor wart]
But the mess around the docs can certainly be cleaned up.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Steven D'Aprano]

On Mon, 16 Mar 2015 07:25 pm, Paul Rubin wrote:

> Steven D'Aprano  writes:
>> It may, or may not, turn out that in hindsight there might have been
>> better ways to manage the Python2-3 transaction. Let's be honest, a lot
>> of the changes could have been introduced incrementally...
> 
> Why did the changes have to be introduced at all?

For the same reason any improvement and functional update is introduced. To
make a better language.

If we were designing Python from scratch today, here are some of the changes
we would certainly make:


- only a single class system, not two independent ones (one of which 
  is buggy);
- Unicode strings by default;
- views and iterators in preference to lists;
- True and False as keywords, like None, not names that can be 
  redefined and shadowed;
- cleaner, more consistent, and more sensible naming conventions 
  for the standard library;
- Python is not C and / should perform true division, not integer
  division;
- some builtins of lesser use should be relegated to modules
- print as a statement was a mistake, it should be a function


etc. Python is a lesser language because of the accumulated cruft and design
mistakes from its 20-year history. If only we could turn back the clock and
redesign it with the benefit of hindsight, without worrying about backwards
compatibility!

Backwards compatibility limits your ability to improve the language and fix
mistakes and obsolete features. Whether you introduce those design fixes
all in one lump or spread out of twenty versions, you still have to break
backwards compatibility, or keep carrying the obsolete code forever.


>> I suspect that without the discipline and bloody-mindedness to just say
>> NO to requests to delay removal, Python would follow all those other
>> languages that have an ever-increasing pile of cruft that can never be
>> removed because it will break somebody's code. Design mistakes, obsolete
>> features, misspellings, "it's like that for historical reasons", and
>> other nasties.
> 
> The idea of "maturity" (an advertised feature of Python 2 back in the
> day) is that the pile of features is no longer growing so fast.  Right
> now I'm messing with some C code written in the 1980s and it still works
> fine.

Of course, not *all* C code written in the 1980s still works fine, but your
point is taken. C, like Fortran and Java, have an even stricter view of
backward compatibility than Python.

The downside of that is that learning and using C means you have to know and
learn a whole lot of ugly, dirty cruft, if only so you know what it means
when you see it in somebody else's code. Long after feature X has been made
obsolete, people still need to deal with it, *even if nobody uses it*.

And they will. We both know that no matter how bad and foolish a feature is,
no matter how loudly you tell people not to use this deprecated, insecure,
unsafe feature, until it is actually gone people will continue to write new
code using it.

You can't please everybody. Some people want Python to change even faster,
some want Python to stop changing so fast...


> I have a hard time imagining using Python 3 for anything.  Python 
> 2 has its cruft but it's familiar and comfortable, 

And that, I think, is the crux of the matter. Programmers, for all their
attraction to shiny new tech, are like cats in that they have a deep
distrust for anything too different. "Give me something completely new,
only exactly the same as the old one."


> and Python 3 gets 
> minimal benefit for most of the breakage that it did (possible exception
> of the Unicode changes).  It still has most of the same mistakes and
> warts.  If I wanted to get away from those, I'd use Go or Ocaml or
> Erlang or Haskell or Javascript (possibly through a compiler like
> Purescript) or whatever.

I find it hard to imagine anyone seriously preferring Javascript to Python
because Python has too many language warts. That's like somebody sitting
down to an entire Death By Chocolate cake for desert instead of a slice of
apple pie because apple pie has too many calories.

Nobody is stopping you from migrating all your code to Ocaml if you like.
(If you think migrating it from Python 2 to 3 is too hard, just try
migrating your code to another language.) And of course people *should*
learn new languages, that is a good thing.


> I'll see if I can remember what it was but I found some annoying
> incompatibility just a week or so ago that I don't think 2to3 would have
> caught, making me scared of Python 3 all over again.

If you remember, do tell.


-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Steven D'Aprano]

On Tue, 17 Mar 2015 12:13 am, Marko Rauhamaa wrote:

> If I get an iterator from a black box source, I don't know if I'm
> allowed/supposed to call close() on it.

In no particular order:

#1
if hasattr(some_iterator, 'close'):
some_iterator.close()


#2
close = getattr(some_iterator, 'close', None)
if close is not None:
close()


#3
try:
close = some_iterator.close
except AttributeError:
pass
else:
close()


#4 (The most pythonic solution of all.) 
If you want to call close(), for some reason, *make it part of your API* and
insist that whatever object is passed to you supports close(), or else it
is an error.




-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Mark Lawrence]

On 16/03/2015 14:19, Rustom Mody wrote:

On Monday, March 16, 2015 at 7:10:03 PM UTC+5:30, Steven D'Aprano wrote:

And of course, from a comp science theoretic perspective,
generators are a kind of subroutine, not a kind of type.


You just showed Marko a few posts back, that
A generator is a special case of an iterator.
And you wrote the iterator with 'class'.
And from python 2.2(?) classes are types.

So I dont know what "comp science theoretic perspective" you are describing...

 From mine:

The generator
def gen():
   yield 1
   yield 2

is much closer to the list (ie data) [1,2]
than to say

def foo():
   print 1
   print 2

The only difference is that the list memoizes the data whereas the generator 
doesn't.

CLU perspective:
The iterator for a collection of complex_numbers can be used interchangeably 
with that for an array of integers
from https://en.wikipedia.org/wiki/CLU_%28programming_language%29
[Note: CLU-iterator ≡ python-generator]

Haskell perspective:
Lists are by default lazy and memoized.
IOW in Haskell: List ≡ Lazy list ≡ python generator + memoization

Scheme perspective:
Programmer can choose between normal and lazy lists.
Lazy lists are just normal lists + delay/force where
delay x ≡ lambda: x
force x ≡ x()

==
Anyways...

Yes 15 years are past.
I dont expect the def can be revoked now.
[As far as I am concerned its a minor wart]
But the mess around the docs can certainly be cleaned up.



So write the patches to correct the docs, then raise the issue on the 
bug tracker to get the patches accepted.  IIRC for doc patches you don't 
even have to provide diff files against the rst files, plain text will 
do, the core devs will do the rest for you.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Steven D'Aprano]

On Tue, 17 Mar 2015 01:19 am, Rustom Mody wrote:

> On Monday, March 16, 2015 at 7:10:03 PM UTC+5:30, Steven D'Aprano wrote:
>> And of course, from a comp science theoretic perspective,
>> generators are a kind of subroutine, not a kind of type.
> 
> You just showed Marko a few posts back, that
> A generator is a special case of an iterator.
> And you wrote the iterator with 'class'.
> And from python 2.2(?) classes are types.

Yes, but iterators aren't *classes*, they are instances.


> So I dont know what "comp science theoretic perspective" you are
> describing...

http://en.wikipedia.org/wiki/Generator_(computer_programming)

"A generator is very similar to a function that returns an array, in that a
generator has parameters, can be called, and generates a sequence of
values. However, instead of building an array containing all the values and
returning them all at once, a generator yields the values one at a time,
which requires less memory and allows the caller to get started processing
the first few values immediately. In short, a generator looks like a
function but behaves like an iterator."





-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Steven D'Aprano]

On Tue, 17 Mar 2015 01:35 am, Steven D'Aprano wrote:

> On Tue, 17 Mar 2015 01:19 am, Rustom Mody wrote:
> 
>> On Monday, March 16, 2015 at 7:10:03 PM UTC+5:30, Steven D'Aprano wrote:
>>> And of course, from a comp science theoretic perspective,
>>> generators are a kind of subroutine, not a kind of type.
>> 
>> You just showed Marko a few posts back, that
>> A generator is a special case of an iterator.
>> And you wrote the iterator with 'class'.
>> And from python 2.2(?) classes are types.
> 
> Yes, but iterators aren't *classes*, they are instances.
> 
> 
>> So I dont know what "comp science theoretic perspective" you are
>> describing...
> 
> http://en.wikipedia.org/wiki/Generator_(computer_programming)
> 
> "A generator is very similar to a function that returns an array, in that
> a generator has parameters, can be called, and generates a sequence of
> values. However, instead of building an array containing all the values
> and returning them all at once, a generator yields the values one at a
> time, which requires less memory and allows the caller to get started
> processing the first few values immediately. In short, a generator looks
> like a function but behaves like an iterator."

Oops, itchy trigger finger... the next paragraph is even more important:

"Generators can be implemented in terms of more expressive control flow
constructs, such as coroutines or first-class continuations.[2] Generators,
also known as semicoroutines,[3] are a special case of (and weaker than)
coroutines, in that they always yield control back to the caller (when
passing a value back), rather than specifying a coroutine to jump to; see
comparison of coroutines with generators."



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Rustom Mody]

On Monday, March 16, 2015 at 7:57:08 PM UTC+5:30, Mark Lawrence wrote:
> On 16/03/2015 14:19, Rustom Mody wrote:
> > ==
> > Anyways...
> >
> > Yes 15 years are past.
> > I dont expect the def can be revoked now.
> > [As far as I am concerned its a minor wart]
> > But the mess around the docs can certainly be cleaned up.
> >
> 
> So write the patches to correct the docs, then raise the issue on the 
> bug tracker to get the patches accepted.  IIRC for doc patches you don't 
> even have to provide diff files against the rst files, plain text will 
> do, the core devs will do the rest for you.

I would gladly do that if it was a minor correction here and there.
But the problem is a bit deeper even though it can be kept mostly¹ in the docs
and not modify any syntax/semantics of python.

In particular for:

def potato(x):
  yield x+1

tomato = potato(3)

what shall we call potato and tomato.
I believe this thread clearly shows that the docs are confused and inconsistent.
Yet I dont see any consensus on what/how to classify tomato/potato.

Function -- trouble on one side
Generator -- trouble on another
Iterator -- trouble on third 
etc

¹ Grey areas excepted eg output of help(); inspect module etc
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Ian Kelly]

On Mon, Mar 16, 2015 at 8:32 AM, Steven D'Aprano
 wrote:
> On Tue, 17 Mar 2015 12:13 am, Marko Rauhamaa wrote:
>
>> If I get an iterator from a black box source, I don't know if I'm
>> allowed/supposed to call close() on it.
>
> In no particular order:
>
> #1
> if hasattr(some_iterator, 'close'):
> some_iterator.close()
>
>
> #2
> close = getattr(some_iterator, 'close', None)
> if close is not None:
> close()
>
>
> #3
> try:
> close = some_iterator.close
> except AttributeError:
> pass
> else:
> close()

Note that these fail if some_iterator is a file-like object. Those
have close methods, but as part of the file api, not the iterator api,
and so the owner of the object is probably not expecting you to close
it for them.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Ian Kelly]

On Mon, Mar 16, 2015 at 7:39 AM, Steven D'Aprano
 wrote:
> On Mon, 16 Mar 2015 11:51 pm, Rustom Mody wrote:
>
>> It may help to read this 15 year old thread starting
>> https://mail.python.org/pipermail/python-dev/2001-June/015478.html
>> to see how many python stalwarts dont like the current state of affairs
>
> /s/dont/didn't/
>
> That's a fifteen year old thread, written before generators were introduced.
> Guido's intuition has been vindicated -- generators have proven to be a
> great and powerful feature, and the reuse of "def" for both generator
> functions and regular functions has turned out to be no more confusing in
> practice than the use of "def" for both functions and methods[1].
>
>
> The argument is that generator-producers (def-with-yield) are not like
> regular functions because calling the def-with-yield doesn't execute the
> code inside them. Given three objects:

There is another argument, which is that for functions and classes,
the descriptive keyword ("def" or "class") is the first thing you see when
reading the code. For generators, the descriptive keyword ("yield") could
be buried *anywhere* in that block. One can glance at a generator
function and fail to notice that it is a generator function. This is
the one that really bugs me about reuse of "def", although you are
correct that this is a case where practicality has won over purity.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Rustom Mody]

On Monday, March 16, 2015 at 8:07:22 PM UTC+5:30, Rustom Mody wrote:
> I would gladly do that if it was a minor correction here and there.
> But the problem is a bit deeper even though it can be kept mostly¹ in the docs
> and not modify any syntax/semantics of python.
> 
> In particular for:
> 
> def potato(x):
>   yield x+1
> 
> tomato = potato(3)
> 
> what shall we call potato and tomato.
> I believe this thread clearly shows that the docs are confused and 
> inconsistent.
> Yet I dont see any consensus on what/how to classify tomato/potato.
> 
> Function -- trouble on one side
> Generator -- trouble on another
> Iterator -- trouble on third 
> etc

And portmanteaus like generator-function are really the worst issue of all

I gave the example of 'pineapple'.

Steven gave another dozen examples that according to him are all ok.
Combine them with his earlier:
> Hopefully it is clear from context what we actually mean. When in doubt, we
> should be explicit. 

So if there is no ambiguity
pineapple can be apple (or is it pine)?
butterfly can be butter
And of course mush is much better than mushroom!

If anything the last 15 years have shown that portmanteaus will be shortformed;
they will be shortformed randomly and inconsistently;
even in the official docs and implementation -- not to mention all over the net.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Marko Rauhamaa]

Ian Kelly :

> For generators, the descriptive keyword ("yield") could be buried
> *anywhere* in that block. One can glance at a generator function and
> fail to notice that it is a generator function. This is the one that
> really bugs me about reuse of "def", although you are correct that
> this is a case where practicality has won over purity.

I don't think that's all that big of a deal even though I will readily
admit having stumbled on it early on. I removed the last "yield"
statement from a generator expecting it to keep on being a generator.
Thus, you have to do things like:

   def nothing():
   if False:
   yield None

The "pass" and "global" statements make me think it might be more
Pythonic to have separate syntax for the special case:

   def nothing():
   yield not


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Ian Kelly]

On Mon, Mar 16, 2015 at 9:09 AM, Marko Rauhamaa  wrote:
> Ian Kelly :
>
>> For generators, the descriptive keyword ("yield") could be buried
>> *anywhere* in that block. One can glance at a generator function and
>> fail to notice that it is a generator function. This is the one that
>> really bugs me about reuse of "def", although you are correct that
>> this is a case where practicality has won over purity.
>
> I don't think that's all that big of a deal even though I will readily
> admit having stumbled on it early on. I removed the last "yield"
> statement from a generator expecting it to keep on being a generator.
> Thus, you have to do things like:
>
>def nothing():
>if False:
>yield None
>
> The "pass" and "global" statements make me think it might be more
> Pythonic to have separate syntax for the special case:
>
>def nothing():
>yield not

My (limited) experience with asyncio is that it also makes this a bit
worse. I write a function intended to be a coroutine invoked from
coroutines using "yield from", and then I realize that it's not a
coroutine because it never uses "yield from" itself. Or inversely I
write a normal utility function that is called from coroutines, then
later add a "yield from" to it, and now I have to go back and revise
every place where it's called to make those use "yield from" as well.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Dict comprehensions - improvement to docs?

[Ian Kelly]

On Mon, Mar 16, 2015 at 3:01 AM, Frank Millman  wrote:
> C:\>python -m timeit -s "x = range(65, 91); y = (chr(z) for z in x)"
> "dict(zip(x, y))"
> 10 loops, best of 3: 11.9 usec per loop
>
> C:\>python -m timeit -s "x = range(65, 91); y = (chr(z) for z in x)" "{a: b
> for a, b in zip(x, y)}"
> 10 loops, best of 3: 7.24 usec per loop

Since the setup code is only run once, the generator expression used
for y is only iterated over once. On every subsequent loop, zip is
producing an empty result. So this measurement is really just
capturing the overhead of the dict construction. Compare:

$ python3 -m timeit -s "x = range(65, 91); y = (chr(z) for z in x)"
"dict(zip(x,y))"
100 loops, best of 3: 0.9 usec per loop
$ python3 -m timeit -s "x = range(65, 91); y = [chr(z) for z in x]"
"dict(zip(x,y))"
10 loops, best of 3: 2.69 usec per loop
$ python3 -m timeit -s "x = range(65, 91); y = (chr(z) for z in x)"
"{a:b for a,b in zip(x,y)}"
100 loops, best of 3: 0.837 usec per loop
$ python3 -m timeit -s "x = range(65, 91); y = [chr(z) for z in x]"
"{a:b for a,b in zip(x,y)}"
10 loops, best of 3: 2.67 usec per loop
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Marko Rauhamaa]

Ian Kelly :

> Or inversely I write a normal utility function that is called from
> coroutines, then later add a "yield from" to it, and now I have to go
> back and revise every place where it's called to make those use "yield
> from" as well.

That is actually quite awkward. Smooth interlocking of coroutines
suggests you should be able to introduce a blocking state anywhere with
a "yield from" statement. To avoid the problem you mention, you'd then
have to give up direct function call altogether and yield from
everything under the sun:

 # Use the supercomputer over RPC -- or not!
 root = yield from my_math.sqrt(x)

It could be done, but it won't look like Python code anymore. It
probably won't look like a computer program anymore.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Python+Glade+Gtk tutorial?

[Dave Farrance]

In the past, I've used Visual-C++ for creating dialog-based interfaces for
controlling equipment and displaying data, and I'm looking for something
of similar ease-of-use in Python since that's now my preferred language.
A web-search told me that Glade seems to be most peoples choice (over
QT-Designer) for a GUI builder.  So even though I use a KDE desktop
(Kubuntu 14.10), I decided to try Glade.

So I web-searched for a Glade "hello world" example to get me started. The
top Google hit for that gave me a tutorial that only took a short time to
work through, and did convince me of Glade's ease-of-use, but the Python
code crashed.  Searching on the error-message told me that I needed to
select Glade's "libglade" output format instead of its "gtkbuilder" output
format to work with that tutorial.  Glade doesn't offer that option any
more -- you just get XML for "gtkbuilder".

So... I searched on "gtkbuilder hello world", which gave me the "Python
GTK+3 Tutorial" at readthedocs.org, and the Python code examples there
gave a somewhat different syntax, from which I was able to figure out how
to fix the tutorial, and then the code worked fine.

So am I understanding this correctly:  If I use this include line:

"from gi.repository import Gtk, Gdk, GObject, Pango" etc...

... I get, in effect, the libraries used in Gnome-3 even with python2?
Whatever "gi.repository" is?  It's a bit hard to figure this out from the
complexity of differing versions that I've turned up from Google searches.

Am I on the right track now?  Glade is a good choice for GUI building? And
even though I'm using Python2, I should be ignoring all examples turned up
by searching for "PyGTK" because they all seem to be GTK+2 and obsolete?
Is that "Python GTK+3 Tutorial" as good as any for me to work through?

https://python-gtk-3-tutorial.readthedocs.org/en/latest/
-- 
https://mail.python.org/mailman/listinfo/python-list

Auto-completion: why not module name?

[candide]

Python 3.4 provides auto-completion facility within a Python console embedded 
in a command line terminal. 


But apparently this facility doesn't allow the user to complete with standard 
module name. For instance, editing the following :

>>> import ma

and hitting the TAB key doesn't generate 

math 

as I was expecting but the following strings :

map( max(

[tested on Ubuntu 12.10]

Is it the expected behaviour ?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Auto-completion: why not module name?

[Chris Angelico]

On Tue, Mar 17, 2015 at 3:23 AM, candide  wrote:
> Python 3.4 provides auto-completion facility within a Python console embedded 
> in a command line terminal.
>
>
> But apparently this facility doesn't allow the user to complete with standard 
> module name. For instance, editing the following :
>
 import ma
>
> and hitting the TAB key doesn't generate
>
> math
>
> as I was expecting but the following strings :
>
> map( max(
>
> [tested on Ubuntu 12.10]
>
> Is it the expected behaviour ?

Looks to me like it's just doing the normal tab-completion of globals,
rather than having a special case for the 'import' statement. So what
you're talking about here is a feature request:

When the input begins "import ", please can tab-completion enumerate
available modules?

This is a very plausible feature request, but be aware that it will
involve a very costly disk search. Figuring out what modules could be
imported means going through the entire Python module search path,
enumerating .py (and other) files, and that could be a lot slower than
you think.

It might even already be available, but disabled by default (because
of that cost). But someone other than me will be better placed to
answer that.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Auto-completion: why not module name?

[Jonas Wielicki]

On 16.03.2015 17:40, Chris Angelico wrote:
> This is a very plausible feature request, but be aware that it will
> involve a very costly disk search. Figuring out what modules could be
> imported means going through the entire Python module search path,
> enumerating .py (and other) files, and that could be a lot slower than
> you think.

With possible side-effects, like help("modules") has. I don’t think that
this is a trivial task (and if it is, why is help("modules") not
implemented in that trivial way?)

regards,
jwi



signature.asc
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Auto-completion: why not module name?

[Ian Kelly]

On Mon, Mar 16, 2015 at 10:40 AM, Chris Angelico  wrote:
> Looks to me like it's just doing the normal tab-completion of globals,
> rather than having a special case for the 'import' statement. So what
> you're talking about here is a feature request:
>
> When the input begins "import ", please can tab-completion enumerate
> available modules?
>
> This is a very plausible feature request, but be aware that it will
> involve a very costly disk search. Figuring out what modules could be
> imported means going through the entire Python module search path,
> enumerating .py (and other) files, and that could be a lot slower than
> you think.

Completeness would also be an issue. There is no general requirement
that module finders use the file system at all (one could write a
finder and loader to import modules from the network, for instance),
nor is there any API for enumerating the modules available.

Another difficulty would be packages. You can't generally know whether
foo.bar is a module until you've already imported foo.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Michael Torrie]

On 03/16/2015 03:13 AM, INADA Naoki wrote:
> I think application developers should use *only* Python 3 from this year.
> If we start moving, more library developers will be able to start
> writing Python 3 only code from next year.

An admirable sentiment, but I'm currently running the latest RHEL
release (v7) and Python3 is not part of the standard install.  I can get
it via Software Collections, but that installs to /opt and, by default,
does not integrate into the system environment (there are good reasons
for this of course).  So Python3 apps will never be integrated fully on
his major distribution.  And it is a major server platform.

RHEL 8 will turn this around completely, as Python 3 will be the default
system python, but that won't be out for several years.

A bit off topic here, but all of this highlights major weaknesses in the
Linux software distribution model. While we Linux nerds like to poke fun
at Windows for not even having a proper package manager until Windows
10, in fact the package manager is not always the best way to go.  Works
well for core system parts, and for distro maintainers.  But it sucks
miserably for developers, and to a lesser degree, end users.  I should
be able to have a stable core distro like RHEL 7 (or any distro), but
develop and distribute apps for Python 3 easily.  Say what you want
about Red Hat's Poettering, but what he says about this problem makes a
lot of sense:
http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.html.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Terry Reedy]

On 3/16/2015 5:13 AM, INADA Naoki wrote:


Another experience is porting Flask application in my company from
Python 2 to Python 3.
It has 26k lines of code and 7.6k lines of tests.

Since we don't need to support both of PY2 and PY3, we used 2to3.
2to3 changes 740 lines.


That is less than 3% of the lines.  Were any changes incorrect?  How 
many lines *not* flagged by 2to3 needed change?



I had to replace google-api-client with
requests+oauthlib since
it had not supported PY3 yet.


Other than those needed for this change, which 2to3 could not anticipate 
or handle?



After that, we encountered few trouble with untested code. But Porting
effort is surprisingly small.
We're happy now with Python 3.  We can write non-ascii string to log
without fear of UnicodeError.
We can use csv with unicode without hack.


People who use ascii only or perhaps one encoding everywhere severely 
underestimate the benefit of unicode strings (and utf-8) everywhere.



Porting *modern* *application* code to *PY3 only* is easy, while
porting libraries on the edge of
bytes/unicode like google-api-client to PY2/3 is not easy.

I think application developers should use *only* Python 3 from this year.
If we start moving, more library developers will be able to start
writing Python 3 only code from next year.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Terry Reedy]

On 3/16/2015 4:31 AM, Paul Rubin wrote:


sure why numpy couldn't go in the stdlib: does it change all that fast?


First there was Numerical Python, the first killer app (though a 
library) for Python.  Then there was was NumArray by a competing group, 
with some not-quite forward compatible changes.  This somewhat split 
scientific Python computing.  This was not good.  Then some some people 
(successfully) reunified things with numpy, which drew on both 
numberical_python and numarray.  I guess things are pretty stable now, 
but not static.


In any case, the developer groups are separate, and both codebases are 
pretty large.  Having to compile both to work on either would be a nuisance.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Mark Lawrence]

On 16/03/2015 17:47, Terry Reedy wrote:

On 3/16/2015 5:13 AM, INADA Naoki wrote:


Another experience is porting Flask application in my company from
Python 2 to Python 3.
It has 26k lines of code and 7.6k lines of tests.

Since we don't need to support both of PY2 and PY3, we used 2to3.
2to3 changes 740 lines.


That is less than 3% of the lines.  Were any changes incorrect?  How
many lines *not* flagged by 2to3 needed change?


I had to replace google-api-client with
requests+oauthlib since
it had not supported PY3 yet.


Other than those needed for this change, which 2to3 could not anticipate
or handle?



Any of the 47 open issues on the bug tracker to start with :(

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Mark Lawrence]

On 16/03/2015 14:37, Rustom Mody wrote:

On Monday, March 16, 2015 at 7:57:08 PM UTC+5:30, Mark Lawrence wrote:

On 16/03/2015 14:19, Rustom Mody wrote:

==
Anyways...

Yes 15 years are past.
I dont expect the def can be revoked now.
[As far as I am concerned its a minor wart]
But the mess around the docs can certainly be cleaned up.



So write the patches to correct the docs, then raise the issue on the
bug tracker to get the patches accepted.  IIRC for doc patches you don't
even have to provide diff files against the rst files, plain text will
do, the core devs will do the rest for you.


I would gladly do that if it was a minor correction here and there.
But the problem is a bit deeper even though it can be kept mostly¹ in the docs
and not modify any syntax/semantics of python.

In particular for:

def potato(x):
   yield x+1

tomato = potato(3)

what shall we call potato and tomato.
I believe this thread clearly shows that the docs are confused and inconsistent.
Yet I dont see any consensus on what/how to classify tomato/potato.

Function -- trouble on one side
Generator -- trouble on another
Iterator -- trouble on third
etc

¹ Grey areas excepted eg output of help(); inspect module etc



So the docs are confused and inconsistent but in an open source 
community it's not *MY* responsibility to deal with it, somebody else can.


Making mountains out of mole hills is all I see in this entire thread.

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[INADA Naoki]

On Tue, Mar 17, 2015 at 2:47 AM, Terry Reedy  wrote:
> On 3/16/2015 5:13 AM, INADA Naoki wrote:
>
>> Another experience is porting Flask application in my company from
>> Python 2 to Python 3.
>> It has 26k lines of code and 7.6k lines of tests.
>>
>> Since we don't need to support both of PY2 and PY3, we used 2to3.
>> 2to3 changes 740 lines.
>
>
> That is less than 3% of the lines.  Were any changes incorrect?  How many
> lines *not* flagged by 2to3 needed change?

All changes are OK. Flask (and Werkzeug) handles most part of pain.
Application using Flask uses unicode most everywhere on Python 2 already.

Few changes 2to3 can't handle is like this:

-reader = DictReader(open(file_path, 'r'), delimiter='\t')
+reader = DictReader(open(file_path, 'r', encoding='utf-8'), delimiter='\t')

Since csv module in Python 2 doesn't support unicode, we had to parse
csv as bytestring.
And our server doesn't have utf-8 locale, we should specify encoding
explicitly on PY3.
There were few (less than 10, maybe) easy trouble like this.


>
>> I had to replace google-api-client with
>> requests+oauthlib since
>> it had not supported PY3 yet.
>
>
> Other than those needed for this change, which 2to3 could not anticipate or
> handle?
>
>> After that, we encountered few trouble with untested code. But Porting
>> effort is surprisingly small.
>> We're happy now with Python 3.  We can write non-ascii string to log
>> without fear of UnicodeError.
>> We can use csv with unicode without hack.
>
>
> People who use ascii only or perhaps one encoding everywhere severely
> underestimate the benefit of unicode strings (and utf-8) everywhere.


I agree. We may lost log easily on Python 2. It makes investigating bug hard.

>>> import logging
>>> logging.error("%s %s", u'こんにちは', 'こんにちは')
Traceback (most recent call last):
...
  File 
"/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py",
line 335, in getMessage
msg = msg % self.args
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe3 in
position 0: ordinal not in range(128)
Logged from file , line 1


And log including unicode is hard to read.

>>> logging.error("%s", [u'こんにちは'])
ERROR:root:[u'\u3053\u3093\u306b\u3061\u306f']


Python 3 makes our development faster and easier.

Since old Python programmers knows how to avoid pitfalls in Python 2,
writing Python 2 is not a pain.
But when teaching Python to PHP programmer, teaching tons of pitfalls is pain.
This is why I think new applications should start with Python 3.

>
>> Porting *modern* *application* code to *PY3 only* is easy, while
>> porting libraries on the edge of
>> bytes/unicode like google-api-client to PY2/3 is not easy.
>>
>> I think application developers should use *only* Python 3 from this year.
>> If we start moving, more library developers will be able to start
>> writing Python 3 only code from next year.
>
>
> --
> Terry Jan Reedy
>
> --
> https://mail.python.org/mailman/listinfo/python-list



--
INADA Naoki  
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Steven D'Aprano]

On Tue, 17 Mar 2015 04:20 am, Michael Torrie wrote:

> On 03/16/2015 03:13 AM, INADA Naoki wrote:
>> I think application developers should use *only* Python 3 from this year.
>> If we start moving, more library developers will be able to start
>> writing Python 3 only code from next year.
> 
> An admirable sentiment, but I'm currently running the latest RHEL
> release (v7) and Python3 is not part of the standard install.  I can get
> it via Software Collections, but that installs to /opt and, by default,
> does not integrate into the system environment (there are good reasons
> for this of course). So Python3 apps will never be integrated fully on 
> his major distribution.  And it is a major server platform.

I'm sorry, that makes no sense to me. What does it matter whether Python3 is
installed to /opt or /usr or /bin or /who/the/feck/cares, so long as your
application runs when you run it? It's just another dependency, and no more
than one call to yum away.

I must admit that the Linux filesystem layout strikes me as awfully pedantic
and fussy. We're happy to use our home directory as an undifferentiated bag
with no structure, dumping binaries, scripts, documents, data files, config
files and everything else into $HOME, but for applications we have:

/bin
/sbin
/usr/bin
/usr/sbin
/usr/local/bin
/usr/local/sbin
/opt 

and probably more, to say nothing of

/lib
/usr/lib
/usr/local/lib



Yesterday I wrote:

Alas, too many (Linux) developers insist on using the system Python
as their application language. In most cases they could trivially
run "aptitude install python3" and be done with it, but that's an 
extra dependency and therefore Bad.

I never imagined that anyone would argue that they can't install and use
Python3 because of the location where it is installed. As an application
developer, apart from ensuring that the PATH is setup correctly and maybe
having to adjust a few hash-bang lines, how does the location of the Python
binary affect you?



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Chris Angelico]

On Tue, Mar 17, 2015 at 4:20 AM, Michael Torrie  wrote:
> A bit off topic here, but all of this highlights major weaknesses in the
> Linux software distribution model. While we Linux nerds like to poke fun
> at Windows for not even having a proper package manager until Windows
> 10, in fact the package manager is not always the best way to go.  Works
> well for core system parts, and for distro maintainers.  But it sucks
> miserably for developers, and to a lesser degree, end users.  I should
> be able to have a stable core distro like RHEL 7 (or any distro), but
> develop and distribute apps for Python 3 easily.  Say what you want
> about Red Hat's Poettering, but what he says about this problem makes a
> lot of sense:
> http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.html.

It most assuredly does NOT suck for end users. Apart from issues of
naming (grab "avconv" or "ffmpeg"?), it's easy - if someone needs to
do audio manipulation, I can tell him/her to "sudo apt-get install
sox" and that'll get the necessary program on any Debian-based distro,
and likewise one command for any Red Hat distro. I'm not sure what you
mean by "for developers" - do you mean that it's hard to package your
software for each distro? Because the package manager benefits you
even if you don't package your own program. Imagine you need a
PostgreSQL database for your Python application - which also means you
need psycopg2, of course. How do you go about writing installation
instructions?

* WINDOWS *
1) Install the latest Python 3 from https://www.python.org/downloads/windows/
2) Install the appropriate version of psycopg2 from
http://www.stickpeople.com/projects/python/win-psycopg/
3) Install the latest PostgreSQL from
http://www.postgresql.org/download/windows/
4) Install my program from blah blah blah

* LINUX: Debian-based *
1) As root, type: apt-get install postgresql python3-psycopg2
2) Install my program from blah blah blah

* LINUX: Red Hat-based *
1) As root, type: yum install postgresql python3-psycopg2
2) Install my program from blah blah blah

(I don't have a Red Hat system handy to check, so the above examples
might need to be tweaked. But you get the idea.)

Without actually going to any effort to build your own packages, you
can still take advantage of one-command installation of all your
dependencies. Without a package manager, you have to assemble them
from all over the internet.

I call that a benefit :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Paul Rubin]

Steven D'Aprano  writes:
>> Why did the changes have to be introduced at all?
> For the same reason any improvement and functional update is
> introduced. To make a better language.

There comes a point when you decide that maintaining existing code is
important enough that you have to stop breaking things.  That point is
called "maturity", and it's fine if you decide that your language
doesn't have it.  Haskell for a long time had a motto "avoid success at
all costs", i.e. it had an explicit goal of being a research testbed for
language geeks and its designers wanted to be able to change stuff
without alienating a big user community.  Python claimed

> If we were designing Python from scratch today, here are some of the
> changes we would certainly make: [mostly good changes]

I agree with most of those changes and I'd add some of my own that are
even more radical.  But, they shouldn't be made in dribs and drabs in
the production releases: it's better to make a fork of the language that
does all the changes at the same time.  Think of C++ as a fork of C in
that manner, although it suffered a lot from trying to be a superset.

> If only we could turn back the clock and redesign it with the benefit
> of hindsight, without worrying about backwards compatibility!

We are currently getting the disadvantages of both approaches.  We break
stuff enough to inflict pain on users, without bringing enough benefits
to make them want to accept the pain.

> design fixes all in one lump or spread out of twenty versions, you
> still have to break backwards compatibility, or keep carrying the
> obsolete code forever.

If people are still using it then it's not obsolete.  Code is only
obsolete when the last user is dead.

> Of course, not *all* C code written in the 1980s still works fine,

Can you give an example of some way C changed that broke old code that
was written in accordance with the language manuals?  Of course tons of
C code then and now used implementation-dependent hacks that went
outside the documented behavior, but that doesn't count.  

> but your point is taken. C, like Fortran and Java, have an even
> stricter view of backward compatibility than Python.

Right, they take their deployed base seriously, while Python has been
more like an experimental language by comparison.

> The downside of that is that learning and using C means you have to
> know and learn a whole lot of ugly, dirty cruft, if only so you know
> what it means when you see it in somebody else's code. Long after
> feature X has been made obsolete, people still need to deal with it,
> *even if nobody uses it*.

Can you give an example?  I wouldn't count things like gets, which
aren't as much changes in the language, as recognition that using it was
buggy from the start.

> And they will. We both know that no matter how bad and foolish a
> feature is, no matter how loudly you tell people not to use this
> deprecated, insecure, unsafe feature, until it is actually gone people
> will continue to write new code using it.

I haven't noticed that much in C.  Maybe it's more of an issue with C++.

> You can't please everybody. Some people want Python to change even
> faster, some want Python to stop changing so fast...

I don't notice anyone wanting Python to change faster, in the sense of
breaking existing code more often.  Maybe there are some specific
proposals that I missed.

> Programmers... are like cats in that they have a deep distrust for
> anything too different. "Give me something completely new, only
> exactly the same as the old one."

It's more like "don't break the old thing unless the new thing is so
much better that I want to change over anyway".  Python to Erlang or
Ocaml or Go or Haskell are far more drastic changes than Python 2 to
Python 3, but at the same time possibly more attractive.

> I find it hard to imagine anyone seriously preferring Javascript to
> Python because Python has too many language warts. 

There are only a few things about raw JS that are really warty and hard
to avoid, and Python has different but comparable warts.  There's
admittedly a bunch of crap in JS that shouldn't be used, but that's
relatively harmless as long as you ignore it.  Current JS
implementations are far superior to Python in performance and there's a
ton of JS library code out there, both for browsers and on the server
side, that might or might not have Python counterparts.  JS is also a
compilation target for other languages like Purescript, Ocaml, or even C
(emscripten).  I haven't tried Purescript yet but it looks cool, and it
might be the nerd answer to JS cruft.  It's Haskell-like but its data
and evaluation models translate more directly to JS than Haskell does.

> Nobody is stopping you from migrating all your code to Ocaml if you
> like.

We were talking about new projects, not migration.

Do you know if any of the big Python shops (Google maybe?) are using
Python 3 these days?  Have they migrated Python 2 codebases or are they

Re: Dict comprehensions - improvement to docs?

[Paul Rubin]

Ian Kelly  writes:
> Since the setup code is only run once, the generator expression used
> for y is only iterated over once. 

Ah, thanks, I'd been wondering what was going on with Frank's example
but hadn't gotten around to trying to analyze it.
-- 
https://mail.python.org/mailman/listinfo/python-list

anaconda bug?

[George Trojan]

I am not sure it is just me or there is a bug in anaconda. I installed 
miniconda from http://conda.pydata.org/miniconda.html, then several 
python3.4.3 packages. Then created virtual environment. When I switch to 
that environment I can not create tk root window. Here is the traceback:


(venv-3.4.3) $ python
Python 3.4.3 |Continuum Analytics, Inc.| (default, Mar  6 2015, 12:03:53)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tkinter
>>> tkinter.Tk()
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/gtrojan/miniconda3/lib/python3.4/tkinter/__init__.py", 
line 1851, in __init__
self.tk = _tkinter.create(screenName, baseName, className, 
interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: Can't find a usable init.tcl in the following 
directories:
/opt/anaconda1anaconda2anaconda3/lib/tcl8.5 
/home/gtrojan/venv-3.4.3/lib/tcl8.5 /home/gtrojan/lib/tcl8.5 
/home/gtrojan/venv-3.4.3/library /home/gtrojan/library 
/home/gtrojan/tcl8.5.18/library /home/tcl8.5.18/library


It looks like the search path for tcl/tk libraries is messed up. The 
same command works fine when I exit virtual environment though. I do not 
have a problem with python 3.4.2 built from source on the same system.


For a workaround, I set TCL_LIBRARY and TK_LIBRARY in activate:

# bug in anaconda?
_OLD_TCL_LIBRARY="$TCL_LIBRARY"
TCL_LIBRARY="/home/gtrojan/miniconda3/lib/tcl8.5"
export TCL_LIBRARY
_OLD_TK_LIBRARY="$TK_LIBRARY"
TK_LIBRARY="/home/gtrojan/miniconda3/lib/tk8.5"
export TK_LIBRARY

I have found somewhat similar bug report: 
https://github.com/conda/conda/issues/348.


George



--
https://mail.python.org/mailman/listinfo/python-list

Re: anaconda bug?

[memilanuk]

Might be just you...

monte@machin-shin:~$ python
Python 3.4.3 |Continuum Analytics, Inc.| (default, Mar  6 2015, 12:03:53)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tkinter
>>> tkinter.Tk()

>>>


Just for the heck of it I created a new venv (using conda create -n 
test) and tried it again.  Same thing.


How are you creating your venv?

Monte


--
Shiny!  Let's be bad guys.

Reach me @ memilanuk (at) gmail dot com

--
https://mail.python.org/mailman/listinfo/python-list

[OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain

[Michael Torrie]

Since Python 3's adoption is directly impacted by package managers and
curated repos (or lack thereof), I feel justified in continuing this
thread just a bit farther.

On 03/16/2015 04:02 PM, Chris Angelico wrote:
> It most assuredly does NOT suck for end users. Apart from issues of
> naming (grab "avconv" or "ffmpeg"?), it's easy - if someone needs to
> do audio manipulation, I can tell him/her to "sudo apt-get install
> sox" and that'll get the necessary program on any Debian-based distro,
> and likewise one command for any Red Hat distro. I'm not sure what you
> mean by "for developers" - do you mean that it's hard to package your
> software for each distro? Because the package manager benefits you
> even if you don't package your own program. Imagine you need a
> PostgreSQL database for your Python application - which also means you
> need psycopg2, of course. How do you go about writing installation
> instructions?

Except that it does suck in lots of ways.  I didn't say the Windows
method was better.  Certainly Poettering was not advocating for the
complete laissez faire crap shoot that is Windows software installation.
 Far from it.

But after 20 years, the package manager idea certain has revealed many
shortcomings (in short, it sucks in many ways).  Package managers work
great for setting up the core distro, and also if the packages you need
are in the repos.  Distros like RHEL and Debian tend to have long-term
stability at the expense of not having recent versions of programs and
libraries.

Although I will note that one major up and coming Linux distribution
eschews package managers completely.  That is ChromeOS, and it is
compelling enough that the CoreOS server container OS is based on it.
They threw out package managers entirely and use image snapshots and
diffs. (I'm not advocating that solution, but it is something people
have tried to get around package manager problems).

For developers things are even more grim.  Package managers certainly
don't work so well for third-party apps like VirtualBox, LibreOffice,
Firefox, etc.  Part of the issue is the multiple moving targets distros
present in terms of what's available in the system.  It's so bad in fact
that major projects that offer binary packages on their web sites end up
bundling copies of libraries they use, such as GTK, SSL, etc.   This is
how VirtualBox, Firefox, and LibreOffice all do it.  It works but it's
wasteful and they still have to target at least three to five different
distro/package manager combinations.

Ideally you should get your package in a mainline repo where ostensibly
it's updated and maintained and just works with your distro of choice.
Except when it doesn't.  As the prime example, take the OwnCloud, which
is, or was, in the official Ubuntu repos.  Do not use it! It's broken
and insecure, and no one is maintaining the package in the official
repos.  In fact OwnCloud.org asked Ubuntu to remove it entirely.  The
same thing has happened several times with Oracle Java and Ubuntu.
So we can't deny the curated package model does have issues and flaws.
PPAs address this for a fairly narrow audience (Ubuntu and derivatives
only).

This isn't saying that package managers don't work and don't have a
place.  They just have serious shortcomings that I believe are part of
what prevents major commercial players from even bothering to target
Linux (though the economics of software on Linux is probably the primary
reason).

> * LINUX: Red Hat-based *
> 1) As root, type: yum install postgresql python3-psycopg2
> 2) Install my program from blah blah blah

Yes that definitely works on Fedora more or less (if you're willing to
upgrade Fedora every year at least ensuring you have the latest and
greatest).  Step 2 is still a problem though.  Have to step outside the
package manager again.  Not ideal.

> (I don't have a Red Hat system handy to check, so the above examples
> might need to be tweaked. But you get the idea.)

Not merely tweaked.  It simply does not work on RHEL, any version.
Python3 can be installed from Software Collections (and that is somewhat
reasonable), but it won't integrate by default, so you can't use
#!/usr/bin/python3 in your apps by default without altering the system
paths.  Alternatively compile from source, but in many server rooms
that's strictly forbidden as it's not maintainable.

> Without actually going to any effort to build your own packages, you
> can still take advantage of one-command installation of all your
> dependencies. Without a package manager, you have to assemble them
> from all over the internet.

And again the ideas in the link I posted also enable all this and more,
and do it in a maintainable, logical way.  A natural extension of the
package manager.  I didn't expect anyone to actually read the post, as
interesting as it is, and thus I'm not disappointed that you dont' seem
to have read it.

> I call that a benefit :)

Yes, it is a benefit, but it also has serious drawbacks, with few 

Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain

[Chris Angelico]

On Tue, Mar 17, 2015 at 12:46 PM, Michael Torrie  wrote:
> But after 20 years, the package manager idea certain has revealed many
> shortcomings (in short, it sucks in many ways).  Package managers work
> great for setting up the core distro, and also if the packages you need
> are in the repos.  Distros like RHEL and Debian tend to have long-term
> stability at the expense of not having recent versions of programs and
> libraries.
>

AIUI, your main beef with the packaging system is, as you rightly
note, an inherent conflict between stability and currency. Very true.
RHEL especially, Debian to a somewhat lesser extent, and every OS that
has a curated package system, will run into that problem.

But the solution isn't necessarily to throw out the packaging system.
All you need is to expand it. I don't know how you do it with yum, but
with apt, you simply add something to /etc/apt/sources.list (or the .d
directory), grab your index files, and install. That's how I install
PostgreSQL on Debian Wheezy; the Debian repos ship Postgres 9.1, but
by simply adding the apt.postgresql.org repo, I can grab 9.4 using the
exact same system. Ubuntu's PPA system achieves the same thing, as
mentioned, but even without PPAs, you can still have multiple
repositories.

The hardest part is managing library versions, and that's always going
to be a problem. Sometimes the latest version of an application
demands a newer version of a library than you have, and if you upgrade
that library, you might need to upgrade a whole lot else, too, so you
may as well upgrade everything and call it a new version of the
distro.

The versioning problem is just as much an issue no matter how you try
to cope with it. Package managers can't magically solve everything,
but they can make a lot of jobs easier, so on that basis, I say
they're beneficial. We don't need a 100% solution to be able to make
use of a 90% solution.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain

[Michael Torrie]

On 03/16/2015 07:57 PM, Chris Angelico wrote:
> But the solution isn't necessarily to throw out the packaging system.
> All you need is to expand it. 

Yes.  And of course that's exactly what Poettering is talking about in
his paper.  Despite what many think of him, he's a deep thinker and it's
worth reading what he says about this.

> I don't know how you do it with yum, but
> with apt, you simply add something to /etc/apt/sources.list (or the .d
> directory), grab your index files, and install. That's how I install
> PostgreSQL on Debian Wheezy; the Debian repos ship Postgres 9.1, but
> by simply adding the apt.postgresql.org repo, I can grab 9.4 using the
> exact same system. Ubuntu's PPA system achieves the same thing, as
> mentioned, but even without PPAs, you can still have multiple
> repositories.

PPAs are problematic from a trust point of view.  They aren't a whole
lot better than installing random installers on windows.

> The hardest part is managing library versions, and that's always going
> to be a problem. Sometimes the latest version of an application
> demands a newer version of a library than you have, and if you upgrade
> that library, you might need to upgrade a whole lot else, too, so you
> may as well upgrade everything and call it a new version of the
> distro.

Again this is the core issue that the Poettering crew at RH is working
on.  A cross between images and OS X's framework system.

> The versioning problem is just as much an issue no matter how you try
> to cope with it. Package managers can't magically solve everything,
> but they can make a lot of jobs easier, so on that basis, I say
> they're beneficial. We don't need a 100% solution to be able to make
> use of a 90% solution.

I agree.

Though if I was a developer trying to ship a large package like
LibreOffice, I would be very frustrated though.  Certainly if I was a
commercial app developer, this is a huge stumbling block.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Michael Torrie]

On 03/16/2015 02:36 PM, Steven D'Aprano wrote:
> I'm sorry, that makes no sense to me. What does it matter whether Python3 is
> installed to /opt or /usr or /bin or /who/the/feck/cares, so long as your
> application runs when you run it? It's just another dependency, and no more
> than one call to yum away.

Of course it matters.  How else will you refer to it in the #!
invocation?  If Python3 is in the system path then you can use a wrapper
script like Calibre does.  But in the case of RHEL with Software
Collections (which does use yum), it does not place it in the system path.

> Yesterday I wrote:
> 
> Alas, too many (Linux) developers insist on using the system Python
> as their application language. In most cases they could trivially
> run "aptitude install python3" and be done with it, but that's an 
> extra dependency and therefore Bad.
> 
> I never imagined that anyone would argue that they can't install and use
> Python3 because of the location where it is installed. As an application
> developer, apart from ensuring that the PATH is setup correctly and maybe
> having to adjust a few hash-bang lines, how does the location of the Python
> binary affect you?

Well the thing is that it's *not* in the PATH by default.  So it does
affect me. The official way to use Python 3 in RHEL Software Collection
is to invoke with with this command line:

scl enable python33 bash

This gives you a shell where python3 is in the path.  You can put
Python3 in the path permanently by modifying /etc/profile.  There are
good reasons for SCL doing things this way, and it's not the only way.
Just pointing out that distro life isn't always rosy.



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Mark Lawrence]

On 16/03/2015 22:02, Chris Angelico wrote:

On Tue, Mar 17, 2015 at 4:20 AM, Michael Torrie  wrote:

A bit off topic here, but all of this highlights major weaknesses in the
Linux software distribution model. While we Linux nerds like to poke fun
at Windows for not even having a proper package manager until Windows
10, in fact the package manager is not always the best way to go.  Works
well for core system parts, and for distro maintainers.  But it sucks
miserably for developers, and to a lesser degree, end users.  I should
be able to have a stable core distro like RHEL 7 (or any distro), but
develop and distribute apps for Python 3 easily.  Say what you want
about Red Hat's Poettering, but what he says about this problem makes a
lot of sense:
http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.html.


It most assuredly does NOT suck for end users. Apart from issues of
naming (grab "avconv" or "ffmpeg"?), it's easy - if someone needs to
do audio manipulation, I can tell him/her to "sudo apt-get install
sox" and that'll get the necessary program on any Debian-based distro,
and likewise one command for any Red Hat distro. I'm not sure what you
mean by "for developers" - do you mean that it's hard to package your
software for each distro? Because the package manager benefits you
even if you don't package your own program. Imagine you need a
PostgreSQL database for your Python application - which also means you
need psycopg2, of course. How do you go about writing installation
instructions?

* WINDOWS *
1) Install the latest Python 3 from https://www.python.org/downloads/windows/
2) Install the appropriate version of psycopg2 from
http://www.stickpeople.com/projects/python/win-psycopg/
3) Install the latest PostgreSQL from
http://www.postgresql.org/download/windows/
4) Install my program from blah blah blah



pip install xyz fits *MY* needs, so blow you Jack, I'm all right. 
Hopefully to be part of IDLE as well, that should save the teachers or 
trainers from changing a few nappies.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Terry Reedy]

On 3/16/2015 6:36 PM, Paul Rubin wrote:


Do you know if any of the big Python shops (Google maybe?) are using
Python 3 these days?


LibreOffice uses Python3.3 (or later, don't know) both for internal 
scripting and the Python bridge.  The FSR unicode that works everywhere 
for all codepoints had something to do with that.



--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain

[Ben Finney]

Thanks for discussing this, Michael.

Michael Torrie  writes:

> For developers things are even more grim. Package managers certainly
> don't work so well for third-party apps like VirtualBox, LibreOffice,
> Firefox, etc. Part of the issue is the multiple moving targets distros
> present in terms of what's available in the system. It's so bad in
> fact that major projects that offer binary packages on their web sites
> end up bundling copies of libraries they use, such as GTK, SSL, etc.

In my experience it's far more extensive than that. The trend seems to
be to bundle every third-party library with one's own work, and dump it
all in the end-user's lap.

> This is how VirtualBox, Firefox, and LibreOffice all do it. It works

It “works fine” only if you ignore:

* The third-party library will have bug fixes.

  Upgrading the OS package for libGTK will get those bug fixes, but
  won't affect all the bundled ones in other packages. Who is
  responsible for fixing those?

* The third-party library will have security vulnerabilities. This is a
  special, but fairly common, case of needing a bug fix.

  The operating systems will make it a priority to upgrade the OS
  package for the library, and the package manager makes sure that's as
  easy as can be feasible so there's some chance it will be fixed.

  Who is responsible for fixing all the bundled versions of the library
  with bug fixes?

* The third-party library willl behave differently from the OS packaged
  version.

  Often, for the developer, that is exactly the point: the developer
  doesn't want the OS packaged version because they want a version with
  different behaviour. So they bundle a specific version, to heck with
  what the OS provides.

  From the point of view of the user, though, deviations from how they
  expect the OS to behave are a bug. If libGTK makes some widget behave
  in a particular way all across the operating system, except in program
  FooBar and BuzQuux, they're going to consider those two programs
  buggy. In my opinion they're correct: those programs should work with
  the rest of the OS, not be rogue.

> but it's wasteful and they still have to target at least three to five
> different distro/package manager combinations.

I think “wasteful” is not much of an issue (and is less of an issue as
storage becomes plentiful). Far more problematic are the divergences
between OS packaged libraries and bundled libraries, as I detailed
above.

> Ideally you should get your package in a mainline repo where
> ostensibly it's updated and maintained and just works with your distro
> of choice. Except when it doesn't.

Thanks again. This is an important and difficult problem, with competing
forces at play, and I am not at all satisfied with the current state of
packaging.

-- 
 \   “He who wonders discovers that this in itself is wonder.” |
  `\  —Maurits Cornelis Escher |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain

[Paul Rubin]

Chris Angelico  writes:
> On Tue, Mar 17, 2015 at 12:46 PM, Michael Torrie  wrote:
>> But after 20 years, the package manager idea certain has revealed many
>> shortcomings (in short, it sucks in many ways). ...
> The hardest part is managing library versions, and that's always going
> to be a problem. 

Have either of you tried Nixos or Guix yet?  I've been wanting to.  The
ideas sound intriguing from what I've seen so far.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain

[Michael Torrie]

On 03/16/2015 08:40 PM, Ben Finney wrote:
> Thanks again. This is an important and difficult problem, with competing
> forces at play, and I am not at all satisfied with the current state of
> packaging.

I agree.  Though I like the concept of package managers and curated
repos, compared to the dismal world of Windows, I also am not very
satisfied with the current state of things.

That's why I think the ideas coming out of this Red Hat think tank are
interesting and worth discussion.  They put forth a possible solution
that keeps the existing things we love about package managers and our
distros of choice, while addressing these important issues in a
maintainable, flexible, and still secure way.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Rustom Mody]

On Monday, March 16, 2015 at 11:50:33 PM UTC+5:30, Mark Lawrence wrote:
> On 16/03/2015 14:37, Rustom Mody wrote:
> > On Monday, March 16, 2015 at 7:57:08 PM UTC+5:30, Mark Lawrence wrote:
> >> On 16/03/2015 14:19, Rustom Mody wrote:
> >>> ==
> >>> Anyways...
> >>>
> >>> Yes 15 years are past.
> >>> I dont expect the def can be revoked now.
> >>> [As far as I am concerned its a minor wart]
> >>> But the mess around the docs can certainly be cleaned up.
> >>>
> >>
> >> So write the patches to correct the docs, then raise the issue on the
> >> bug tracker to get the patches accepted.  IIRC for doc patches you don't
> >> even have to provide diff files against the rst files, plain text will
> >> do, the core devs will do the rest for you.
> >
> > I would gladly do that if it was a minor correction here and there.
> > But the problem is a bit deeper even though it can be kept mostly¹ in the 
> > docs
> > and not modify any syntax/semantics of python.
> >
> > In particular for:
> >
> > def potato(x):
> >yield x+1
> >
> > tomato = potato(3)
> >
> > what shall we call potato and tomato.
> > I believe this thread clearly shows that the docs are confused and 
> > inconsistent.
> > Yet I dont see any consensus on what/how to classify tomato/potato.
> >
> > Function -- trouble on one side
> > Generator -- trouble on another
> > Iterator -- trouble on third
> > etc
> >
> > ¹ Grey areas excepted eg output of help(); inspect module etc
> >
> 
> So the docs are confused and inconsistent but in an open source 
> community it's not *MY* responsibility to deal with it, somebody else can.
> 
> Making mountains out of mole hills is all I see in this entire thread.

Ok... Lets see... What if any agreement is there on this.

To start with basic terminology. Given

> def potato(x):
>yield x+1
>
> tomato = potato(3)
>
> What shall we call potato and tomato?

Steven suggested that potato can be called 'factory'
Not ideal, but way better than 'generator-function'.
Oscar does not like it.
No better/other suggestions (that we see here).

What next?

Ok Let me throw out a suggestion:
 - potato is a generator
 - tomato is a cursor.
Acceptable?

So much for terminological questions.
When we go from 'simple-generators' to coroutine-generators there seem to be
bigger conceptual problems and implementation-gaffes.
Quite frankly I hardly understand this part.
Here are some questions.

1. Is the name generator appropriate for coroutine-generator?
The wikipedia-link that Steven posted suggests the other-way-round 'is-a'
relation: "a generator is a (semi)coroutine¹"
2. Can we say
   yield-statement ≡ generator
   yield-expression ≡ coroutine
   ?
   What of x = yield y
   x = yield x

  My (vaguest) impression is that 'yield' stops being the meaningful word
  for coroutine -- see footnote from Frederik Lundh in the 2001 thread

> suspend instead of yield, but that's me

¹ Non-trivial terminological headache: Is this 'generator' in the old sense or 
new
sense?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain

[Mark Lawrence]

On 17/03/2015 02:51, Michael Torrie wrote:

On 03/16/2015 08:40 PM, Ben Finney wrote:

Thanks again. This is an important and difficult problem, with competing
forces at play, and I am not at all satisfied with the current state of
packaging.


I agree.  Though I like the concept of package managers and curated
repos, compared to the dismal world of Windows, I also am not very
satisfied with the current state of things.

That's why I think the ideas coming out of this Red Hat think tank are
interesting and worth discussion.  They put forth a possible solution
that keeps the existing things we love about package managers and our
distros of choice, while addressing these important issues in a
maintainable, flexible, and still secure way.



Reading this makes me realise how lucky I am not having to worry about 
such issues.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Mario Figueiredo]

On Tue, 17 Mar 2015 09:02:38 +1100, Chris Angelico 
wrote:
>
>Imagine you need a
>PostgreSQL database for your Python application - which also means you
>need psycopg2, of course. How do you go about writing installation
>instructions?
>
>* WINDOWS *
>1) Install the latest Python 3 from https://www.python.org/downloads/windows/
>2) Install the appropriate version of psycopg2 from
>http://www.stickpeople.com/projects/python/win-psycopg/
>3) Install the latest PostgreSQL from
>http://www.postgresql.org/download/windows/
>4) Install my program from blah blah blah
>

Are you saying this is a problem for any developer? Especially
considering this is a one-time operation...

Or maybe you mean lazy developers. But lazy developers are an edge
case not worth being catered for.

>
>Without actually going to any effort to build your own packages, you
>can still take advantage of one-command installation of all your
>dependencies. Without a package manager, you have to assemble them
>from all over the internet.

With the advantage of storing them on a portable pen to install
somewhere else offline... But hey, that isn't good, right?

Anyways, on the specific case of Python packages, you are of course
wrong. First it was easy_install, now you have pip.

You also have chocolatey, nuget, ninite, npackd, etc. It's not that
you don't have package manager-like options in windows. It's just
that, like with linux unofficial, unsupported, edge, etc repos, you
rarely want to put your faith on a anonymous package developer or on a
bleeding edge package.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain

[Michael Torrie]

On 03/16/2015 09:01 PM, Mark Lawrence wrote:
> Reading this makes me realise how lucky I am not having to worry about 
> such issues.

How so?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain

[Michael Torrie]

On 03/16/2015 08:45 PM, Paul Rubin wrote:
> Chris Angelico  writes:
>> On Tue, Mar 17, 2015 at 12:46 PM, Michael Torrie  wrote:
>>> But after 20 years, the package manager idea certain has revealed many
>>> shortcomings (in short, it sucks in many ways). ...
>> The hardest part is managing library versions, and that's always going
>> to be a problem. 
> 
> Have either of you tried Nixos or Guix yet?  I've been wanting to.  The
> ideas sound intriguing from what I've seen so far.

Haven't but I will take a look..

-- 
https://mail.python.org/mailman/listinfo/python-list

Package manager cooperation? (was Weaknesses of distro package managers)

[Rustom Mody]

On Tuesday, March 17, 2015 at 8:10:16 AM UTC+5:30, Ben Finney wrote:
> Thanks for discussing this, Michael.
> 
> Michael Torrie  writes:
> 
> > For developers things are even more grim. Package managers certainly
> > don't work so well for third-party apps like VirtualBox, LibreOffice,
> > Firefox, etc. Part of the issue is the multiple moving targets distros
> > present in terms of what's available in the system. It's so bad in
> > fact that major projects that offer binary packages on their web sites
> > end up bundling copies of libraries they use, such as GTK, SSL, etc.
> 
> In my experience it's far more extensive than that. The trend seems to
> be to bundle every third-party library with one's own work, and dump it
> all in the end-user's lap.
> 
> > This is how VirtualBox, Firefox, and LibreOffice all do it. It works
> 
> It "works fine" only if you ignore:

I believe the problem is that package-managers will multiply and proliferate.
Will they insist on warring or can they cooperate?

eg Lets take it that apt is the most mature, stable etc package manager.
But its also the most backward in the sense of being downstream.

OTOH many large-scale systems have sprouted their own packaging-systems

eg the full texlive system is some 2GB download! and has its own tlmgr

It would be good for things like apt to make a public-API and thereafter
For things like tlmgr, firefox-plugins, and of course
python-pip
ruby-gems
haskell-cabal
etc etc

to try to be at least quasi-auto interoperable with apt
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Mark Lawrence]

On 17/03/2015 02:52, Rustom Mody wrote:

On Monday, March 16, 2015 at 11:50:33 PM UTC+5:30, Mark Lawrence wrote:

On 16/03/2015 14:37, Rustom Mody wrote:

On Monday, March 16, 2015 at 7:57:08 PM UTC+5:30, Mark Lawrence wrote:

On 16/03/2015 14:19, Rustom Mody wrote:

==
Anyways...

Yes 15 years are past.
I dont expect the def can be revoked now.
[As far as I am concerned its a minor wart]
But the mess around the docs can certainly be cleaned up.



So write the patches to correct the docs, then raise the issue on the
bug tracker to get the patches accepted.  IIRC for doc patches you don't
even have to provide diff files against the rst files, plain text will
do, the core devs will do the rest for you.


I would gladly do that if it was a minor correction here and there.
But the problem is a bit deeper even though it can be kept mostly¹ in the docs
and not modify any syntax/semantics of python.

In particular for:

def potato(x):
yield x+1

tomato = potato(3)

what shall we call potato and tomato.
I believe this thread clearly shows that the docs are confused and inconsistent.
Yet I dont see any consensus on what/how to classify tomato/potato.

Function -- trouble on one side
Generator -- trouble on another
Iterator -- trouble on third
etc

¹ Grey areas excepted eg output of help(); inspect module etc



So the docs are confused and inconsistent but in an open source
community it's not *MY* responsibility to deal with it, somebody else can.

Making mountains out of mole hills is all I see in this entire thread.


Ok... Lets see... What if any agreement is there on this.

To start with basic terminology. Given


def potato(x):
yield x+1

tomato = potato(3)

What shall we call potato and tomato?


Steven suggested that potato can be called 'factory'
Not ideal, but way better than 'generator-function'.
Oscar does not like it.
No better/other suggestions (that we see here).

What next?

Ok Let me throw out a suggestion:
  - potato is a generator
  - tomato is a cursor.
Acceptable?



No.  In Python potato is a generator function, tomato is a generator. 
Why complicate something that is so simple?  I couldn't care less what 
they are called in any other language.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain

[Mario Figueiredo]

On Mon, 16 Mar 2015 21:05:03 -0600, Michael Torrie 
wrote:

>On 03/16/2015 09:01 PM, Mark Lawrence wrote:
>> Reading this makes me realise how lucky I am not having to worry about 
>> such issues.
>
>How so?

Speaking for myself (I know you didn't ask me) what you call the
"dismal world of windows", I call install exactly what I want, exactly
when I want.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: [OT] Weaknesses of distro package managers - was Re: Python 2 to 3 conversion - embrace the pain

[Mark Lawrence]

On 17/03/2015 03:05, Michael Torrie wrote:

On 03/16/2015 09:01 PM, Mark Lawrence wrote:

Reading this makes me realise how lucky I am not having to worry about
such issues.


How so?



I don't work due to ill health.  I use Python at home on Windows purely 
for my own needs which pip now covers, particularly given the relatively 
recent introduction of wheel files.  For anything else download the msi 
or exe file and click to install.  Seems like utopia compared to the 
real world that the rest of you have to live and work in.  Sure I have 
to do some research first and maybe make mistakes, but I do have the 
luxury of time, no boss and no stupid deadlines set by some dumbo sales 
person, sorry, account manager :)


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Rustom Mody]

On Tuesday, March 17, 2015 at 8:37:25 AM UTC+5:30, Mark Lawrence wrote:
> >
> > Ok Let me throw out a suggestion:
> >   - potato is a generator
> >   - tomato is a cursor.
> > Acceptable?
> >
> 
> No.  In Python potato is a generator function, tomato is a generator. 
> Why complicate something that is so simple?  I couldn't care less what 
> they are called in any other language.

Ok so lets see...
https://docs.python.org/3.4/tutorial/classes.html#generators
https://docs.python.org/3.4/glossary.html#term-generator

Are these consistent with (your notion of) python?
Maybe they are "any other language"?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Mario Figueiredo]

On Mon, 16 Mar 2015 19:52:59 -0700 (PDT), Rustom Mody
 wrote:

>
>When we go from 'simple-generators' to coroutine-generators there seem to be
>bigger conceptual problems and implementation-gaffes.
>Quite frankly I hardly understand this part.

There's a line after which terminology just becomes pedantic. And if
you cross it once more, you get into the realm of obfuscation.
Congratulations, I suppose! You managed to have reached that edge.

Seriously, you started this thread well. I even nodded to my screen
when many posts back you mentioned correct terminology was an
important aspect of any official spec.

But you just couldn't stop, could you? What you aren't realizing is
that your quest for the perfect set of terms is exactly what is
leading you into a confusing state, producing the opposite effect of
making things harder to understand, instead of easier.

Continue this thread if you must, but please never write a spec in
your life.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Mark Lawrence]

On 17/03/2015 03:18, Rustom Mody wrote:

On Tuesday, March 17, 2015 at 8:37:25 AM UTC+5:30, Mark Lawrence wrote:


Ok Let me throw out a suggestion:
   - potato is a generator
   - tomato is a cursor.
Acceptable?



No.  In Python potato is a generator function, tomato is a generator.
Why complicate something that is so simple?  I couldn't care less what
they are called in any other language.


Ok so lets see...
https://docs.python.org/3.4/tutorial/classes.html#generators
https://docs.python.org/3.4/glossary.html#term-generator

Are these consistent with (your notion of) python?
Maybe they are "any other language"?



I'll already suggested you write the patches and put them on the bug 
tracker.  If you can't be bothered please have the courtesy to stop 
bleating about it.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: Package manager cooperation? (was Weaknesses of distro package managers)

[Michael Torrie]

On 03/16/2015 09:09 PM, Rustom Mody wrote:
> OTOH many large-scale systems have sprouted their own packaging-systems

And indeed PIP and CPAN are both forms of package managers to fit the
special needs of those languages' developers. Sometimes that works well
with the OS package manager, sometimes it's at odds.

> eg the full texlive system is some 2GB download! and has its own tlmgr
> 
> It would be good for things like apt to make a public-API and thereafter
> For things like tlmgr, firefox-plugins, and of course
> python-pip
> ruby-gems
> haskell-cabal
>
> to try to be at least quasi-auto interoperable with apt

A meta package manager could, in theory, interact with all these systems
in a unified way.  That's certainly one approach.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Michael Torrie]

On 03/16/2015 09:04 PM, Mario Figueiredo wrote:
> Are you saying this is a problem for any developer? Especially
> considering this is a one-time operation...
> 
> Or maybe you mean lazy developers. But lazy developers are an edge
> case not worth being catered for.

I guess I'm saying the package managers are a problem for developers who
wish to distribute their products or projects to end users.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: generator/coroutine terminology

[Rustom Mody]

On Tuesday, March 17, 2015 at 8:55:27 AM UTC+5:30, Mark Lawrence wrote:
> On 17/03/2015 03:18, Rustom Mody wrote:
> > On Tuesday, March 17, 2015 at 8:37:25 AM UTC+5:30, Mark Lawrence wrote:
> >>>
> >>> Ok Let me throw out a suggestion:
> >>>- potato is a generator
> >>>- tomato is a cursor.
> >>> Acceptable?
> >>>
> >>
> >> No.  In Python potato is a generator function, tomato is a generator.
> >> Why complicate something that is so simple?  I couldn't care less what
> >> they are called in any other language.
> >
> > Ok so lets see...
> > https://docs.python.org/3.4/tutorial/classes.html#generators
> > https://docs.python.org/3.4/glossary.html#term-generator
> >
> > Are these consistent with (your notion of) python?
> > Maybe they are "any other language"?
> >
> 
> I'll already suggested you write the patches and put them on the bug 
> tracker.  If you can't be bothered please have the courtesy to stop 
> bleating about it.

Here are two of your posts (within a couple of hours)

1. 
> So the docs are confused and inconsistent but in an open source
> community it's not *MY* responsibility to deal with it, somebody else can.
> Making mountains out of mole hills is all I see in this entire thread. 

2. 
> No.  In Python potato is a generator function, tomato is a generator.
> Why complicate something that is so simple?  I couldn't care less what
> they are called in any other language. 

I understand the first as saying: "Since something is wrong correct it; Dont 
bleat!"

The second is saying "Nothing is wrong!"

Please decide which side you belong
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python 2 to 3 conversion - embrace the pain

[Ben Finney]

Mario Figueiredo  writes:

> On Tue, 17 Mar 2015 09:02:38 +1100, Chris Angelico 
> wrote:
> >
> >Imagine you need a PostgreSQL database for your Python application -
> >which also means you need psycopg2, of course. How do you go about
> >writing installation instructions?
> >
> >* WINDOWS *
> >1) Install the latest Python 3 from https://www.python.org/downloads/windows/
> >2) Install the appropriate version of psycopg2 from
> >http://www.stickpeople.com/projects/python/win-psycopg/
> >3) Install the latest PostgreSQL from
> >http://www.postgresql.org/download/windows/
> >4) Install my program from blah blah blah
> >
>
> Are you saying this is a problem for any developer?

Yes. They need to write installation instructions, and the more complex
those instructions are the fewer prospective users will bother.

The more complexities in the installation process, the more
opportunities there are for the recipient to make a mistake. Each
mistake is demoralising and will cause some proportion of people to give
up.

So if the installation process *necessitates* complex instructions,
that's a problem for the developer.

You can blame “lazy users” if you want to. That gets you no new users of
your program though.

-- 
 \  “I tell you the truth: some standing here will not taste death |
  `\before they see the Son of Man coming in his kingdom.” —Jesus, |
_o__) c. 30 CE, as quoted in Matthew 16:28 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list