Re: Survey: improving the Python std lib docs

[rzed]

A portion of this thread seems to be focusing on what key word args parameters 
actually mean, in the Python sense. There is documentation for that, and a 
modicum of experience with Python makes this a relatively simple question and 
answer. However, when docs for a specific function or method specify keyword 
arguments without any indication of what the *expected* arguments might be, or 
what *useful* arguments might be, then it's not really helpful. If I have to 
look at the source code to figure out how the args are used, I don't need the 
docs at all. Which is why I suggest a line or two with sample invocations to 
provide at least a starting point to understand the parameters. 

A doc signature that says something like "do_something(p1, d)" tells me very 
little. Even knowing that p1 is a "Point" is not all that valuable, except to 
give me another item to look up (the expected Point implementation). However, 
an example like 'do_something((141,380),"January, 2017")' tells me a lot more - 
that the Point is actually a tuple with (probably) x-y coordinates, and 'd' is 
a description associated with that position. That's a lot of useful information 
at a cost of a couple of dozen characters. So why not add a few examples? 
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: Survey: improving the Python std lib docs

[Deborah Swanson]

> -Original Message-
> From: Python-list 
> [mailto:python-list-bounces+python=deborahswanson.net@python.o
> rg] On Behalf Of Gregory Ewing
> Sent: Thursday, May 18, 2017 5:00 PM
> To: python-list@python.org
> Subject: Re: Survey: improving the Python std lib docs
> 
> 
> Deborah Swanson wrote:
> 
> > somenamedtuple._replace(kwargs)
> > 
> > Return a new instance of the named tuple replacing specified 
> > fields with new values: (Examples
> > box)---|
> > |>>>
> > |
> > |
> > |
> > |>>> p = Point(x=11, y=22)
> > |
> > |>>> p._replace(x=33)
> > |
> > |Point(x=33, y=22)
> 
> To someone who has some basic Python knowledge about what 
> keyword arguments are, that's actually pretty clear. What 
> it's saying is that any of the namedtuple's attribute names 
> can be used as a keyword argument to the _replace method.
> 
> Namedtuple is something of a special case, because it doesn't 
> have a fixed set of acceptable keyword args. Most things do, 
> and there should be something somewhere in the docs 
> indicating what they are. Note that you may need to consult 
> docs for base classes to get the full picture, since e.g. 
> constructors often pass any keyword args they don't 
> understand on to their base class constructor.

I'm getting a clearer picture, and even though it might not be as
cleanly straightforward as using recordsclass, I think it would be a
good learning experience to use "somenamedtuple._replace(kwargs)" in a
context of my own devising, rather than just copying out code given for
a different problem from someone else.

Besides, I need to compile memoryslots.c to make a usable recordsclass
module, and I haven't successfully compiled a C program outside of
Visual Studio yet(and that was a couple decades ago). So I'm going to
see if I can really understand kwargs in a namedtuple context first.  

It will be good to have a bag of tricks to tackle all my Excel
spreadsheet conversion-to-Python and new-from-scratch projects with. I'm
saying goodbye to Microsoft Excel, but I want to take all my work and
ideas with me. And who knows, maybe I'll have some templates and
tutorials to contribute for others who'd like to do the same.

> > Now, I understood what he meant from the context of the problem he
was 
> > helping me with, but, how in the world would I have gotten that 
> > possible use of somenamedtuple._replace(kwargs) from the blurb and 
> > examples in the docs?
> 
>  From knowledge about keyword arguments in general, 
> exrapolation from the example, and following logic.
> 
> I know that's probably not a very helpful answer, but the 
> reason it seems so obscure to you now is because you don't 
> have all of the basic Python knowledge that the docs assume 
> the reader has. If you come back to the same docs later when 
> you do have that knowledge, you will see it all more clearly.

Oh, I think I see what you mean now. But I'm sure glad I've had this
list to talk to, or I'd never have figured it out from scratch on my
own.

> > If there's a tutorial somewhere on kwargs in general, what specific 
> > types of kwargs there are, what special operators can be specifed
for 
> > them and how the kwargs are used, it would be very helpful to have 
> > that tutorial referenced in a footnote-link next to kwargs whenever 
> > kwargs is part of a function specification.
> 
> This is all general knowlege about the Python language that 
> you can't expect the reference documentation to point out in 
> relation to every function and method.

No, perhaps not. I was frustrated by not being able to see and
understand what I needed to know from the docs, but maybe more in depth
tutorials for beginners, and some kind of pointers to them, would fill
that gap more effectively.
 
> > I have an item on my todo list to google "python kwargs", but it
keeps 
> > getting pushed deeper and deeper down my list because I have so
little 
> > hope that it will turn up anything useful.
> 
> I strongly encourage you to do it sooner rather than later, 
> because you'll gain some important foundational knowledge 
> that's vital for understanding many things you'll see in 
> Python code and Python docs.
> 
> -- 
> Greg

I may do that Google search when I get to it, but I've learned a lot
right here. Maybe enough, but Ill have to work with what I now know to
see that.

Thanks very much,

Deborah

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[Gregory Ewing]

Deborah Swanson wrote:


somenamedtuple._replace(kwargs)

Return a new instance of the named tuple replacing specified fields
with new values:
(Examples
box)---|
|>>>
|
|
|
|>>> p = Point(x=11, y=22)
|
|>>> p._replace(x=33)
|
|Point(x=33, y=22)


To someone who has some basic Python knowledge about what keyword
arguments are, that's actually pretty clear. What it's saying is
that any of the namedtuple's attribute names can be used as a
keyword argument to the _replace method.

Namedtuple is something of a special case, because it doesn't have a
fixed set of acceptable keyword args. Most things do, and there should
be something somewhere in the docs indicating what they are. Note
that you may need to consult docs for base classes to get the full
picture, since e.g. constructors often pass any keyword args they
don't understand on to their base class constructor.


Now, I understood what he meant from the context of the problem he was
helping me with, but, how in the world would I have gotten that possible
use of somenamedtuple._replace(kwargs) from the blurb and examples in
the docs?


From knowledge about keyword arguments in general, exrapolation
from the example, and following logic.

I know that's probably not a very helpful answer, but the reason
it seems so obscure to you now is because you don't have all of
the basic Python knowledge that the docs assume the reader has.
If you come back to the same docs later when you do have that
knowledge, you will see it all more clearly.


If there's a tutorial somewhere on kwargs in general, what specific
types of kwargs there are, what special operators can be specifed for
them and how the kwargs are used, it would be very helpful to have that
tutorial referenced in a footnote-link next to kwargs whenever kwargs is
part of a function specification.


This is all general knowlege about the Python language that you
can't expect the reference documentation to point out in relation
to every function and method.


I have an item on my todo list to google "python kwargs", but it keeps
getting pushed deeper and deeper down my list because I have so little
hope that it will turn up anything useful.


I strongly encourage you to do it sooner rather than later, because
you'll gain some important foundational knowledge that's vital for
understanding many things you'll see in Python code and Python docs.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

RE: Survey: improving the Python std lib docs

[Deborah Swanson]

justin walters wrote, on Thursday, May 18, 2017 8:09 AM
> To: python-list@python.org
> Subject: Re: Survey: improving the Python std lib docs
> 
> So, args can be treated as a simple (named)? tuple or a 
> simple dictionary. `*` unpacks a list or tuple and `**` 
> unpacks a dictionary. I'm sure it's a lot more nuanced than 
> this, but for practical reasons, the above explanation has 
> never failed me.
> 
> I hope I didn't miss the entire point of the thread. Someone 
> needed help with `**kwargs` right?

I think I was mainly the one who wasn't sure what "**kwargs" are, and
this, plus your examples I snipped, and bits and pieces I already knew,
neatly pulls it all together.

Think I have a good working idea of it now, so thanks!

Deborah

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[justin walters]

On Thu, May 18, 2017 at 8:08 AM, justin walters 
wrote:

>
>
> On Thu, May 18, 2017 at 12:09 AM, Deborah Swanson <
> pyt...@deborahswanson.net> wrote:
>
>> Michael Torrie wrote, on Wednesday, May 17, 2017 3:11 PM
>> >
>> > On 05/17/2017 02:31 PM, Ned Batchelder wrote:
>> > > Can you give an example of such a method? Often, that signature is
>> > > used because there is no pre-conception of what the arguments might
>> > > be.
>> >
>> > I'm not sure if this afflicts the standard library, but in my
>> > own code, since Python doesn't support constructors with
>> > different signatures, you pretty much have to rely on kwargs
>> > with __init__() to handle different permutations of
>> > construction arguments.  Not that we don't know what the
>> > arguments might be, we just don't know which of them we'll
>> > have to deal with.  Maybe the standard library is better
>> > designed than my own code, but I find I have to implement
>> > __init__(self, **kwargs) all the time.
>>
>>
>> While I can respect and appreciate your specialized interest in specific
>> kwargs, and I read in other's posts that help()'s display of kwargs is
>> conditional on several things, I really want to reiterate and clarify my
>> beginner's request.
>>
>> I know of several functions I've used without any *args or **kwargs
>> because I had no clue from the docs what they might be. And, I know of
>> one function now, from posts in this list, which takes kwargs, but I
>> only know one application of one possible kwarg in this function because
>> one member of this list, Peter Otten, used it in a suggestion he gave
>> me.
>>
>> This is all there is in the docs for that function:
>>
>>
>> somenamedtuple._replace(kwargs)
>>
>> Return a new instance of the named tuple replacing specified fields
>> with new values:
>> (Examples
>> box)---|
>> |>>>
>> |
>> |
>> |
>> |>>> p = Point(x=11, y=22)
>> |
>> |>>> p._replace(x=33)
>> |
>> |Point(x=33, y=22)
>> |
>> |
>> |
>> |>>> for partnum, record in inventory.items():
>> |
>> |... inventory[partnum] =
>> record._replace(price=newprices[partnum], |
>> |  timestamp=time.now())
>> |
>> |---
>> |
>>
>> From this doc entry's example, I (sort of) get that x is a keyword arg
>> that is to be equal to 33 for any use of p._replace() in the code
>> following, until
>> p._replace(kwarg) is respecified, or the code ends. So the response from
>> Python shows me that p has transformed to the Point(x=33, y=22). A
>> trivial example that doesn't begin to hint at the poential powers of
>> this kwarg.
>>
>> The inventory[partnum] example is also quite trivial compared to the
>> kwarg use that Peter showed me.
>>
>> When I asked him for an explanation, this is what I and he he said:
>>
>> Deborah Swanson wrote:
>>
>> > I know it's your "ugly" answer, but can I ask what the '**' in
>> >
>> > fix = {label: max(values, key=len)}
>> > group[:] = [record._replace(**fix) for record in group]
>> >
>> > means?
>>
>> (Peter wrote)
>> d = {"a": 1, "b": 2}
>> f(**d)
>>
>> is equivalent to
>>
>> f(a=1, b=2)
>>
>> so ** is a means to call a function with keyword arguments when you want
>> to
>> decide about the *names* at runtime. Example:
>>
>> >>> def f(a=1, b=2):
>> ... print("a =", a)
>> ... print("b =", b)
>> ... print()
>> ...
>> >>> for d in [{"a": 10}, {"b": 42}, {"a": 100, "b": 200}]:
>> ... f(**d)
>> ...
>> a = 10
>> b = 2
>>
>> a = 1
>> b = 42
>>
>> a = 100
>> b = 200
>>
>> Starting from a namedtuple `record`
>>
>> record._replace(Location="elswhere")
>>
>> creates a new namedtuple with the Location attribute changed to
>> "elsewhere",
>> and the slice [:] on the left causes all items in the `groups` list to
>> be
>> replaced with new namedtuples,
>>
>> group[:] = [record._replace(Location="elsewhere") for record in group]
>>
>> is basically the same as
>>
>> tmp = group.copy()
>> group.clear()
>> for record in tmp:
>> group.append(record_replace(Location="elsewhere"))
>>
>> To support not just Location, but also Kind and Notes we need the double
>>
>> asterisk.
>>
>>
>>
>> Now, I understood what he meant from the context of the problem he was
>> helping me with, but, how in the world would I have gotten that possible
>> use of somenamedtuple._replace(kwargs) from the blurb and examples in
>> the docs? Or any other possible kwarg and how they're used, including
>> special operators like "**" ?
>>
>> If there's a tutorial somewhere on kwargs in general, what specific
>> types of kwargs there are, what special operators can be specifed for
>> them and how the kwargs are used, it would be very helpful to have that
>> tutorial referenced in a footnote-link next to kwargs whenever kwargs is
>> part of a function specification.
>>
>> Or, if no such tutorial exists, the footnote-link could point to any
>> more detailed 

Re: Survey: improving the Python std lib docs

[justin walters]

On Thu, May 18, 2017 at 12:09 AM, Deborah Swanson  wrote:

> Michael Torrie wrote, on Wednesday, May 17, 2017 3:11 PM
> >
> > On 05/17/2017 02:31 PM, Ned Batchelder wrote:
> > > Can you give an example of such a method? Often, that signature is
> > > used because there is no pre-conception of what the arguments might
> > > be.
> >
> > I'm not sure if this afflicts the standard library, but in my
> > own code, since Python doesn't support constructors with
> > different signatures, you pretty much have to rely on kwargs
> > with __init__() to handle different permutations of
> > construction arguments.  Not that we don't know what the
> > arguments might be, we just don't know which of them we'll
> > have to deal with.  Maybe the standard library is better
> > designed than my own code, but I find I have to implement
> > __init__(self, **kwargs) all the time.
>
>
> While I can respect and appreciate your specialized interest in specific
> kwargs, and I read in other's posts that help()'s display of kwargs is
> conditional on several things, I really want to reiterate and clarify my
> beginner's request.
>
> I know of several functions I've used without any *args or **kwargs
> because I had no clue from the docs what they might be. And, I know of
> one function now, from posts in this list, which takes kwargs, but I
> only know one application of one possible kwarg in this function because
> one member of this list, Peter Otten, used it in a suggestion he gave
> me.
>
> This is all there is in the docs for that function:
>
>
> somenamedtuple._replace(kwargs)
>
> Return a new instance of the named tuple replacing specified fields
> with new values:
> (Examples
> box)---|
> |>>>
> |
> |
> |
> |>>> p = Point(x=11, y=22)
> |
> |>>> p._replace(x=33)
> |
> |Point(x=33, y=22)
> |
> |
> |
> |>>> for partnum, record in inventory.items():
> |
> |... inventory[partnum] =
> record._replace(price=newprices[partnum], |
> |  timestamp=time.now())
> |
> |---
> |
>
> From this doc entry's example, I (sort of) get that x is a keyword arg
> that is to be equal to 33 for any use of p._replace() in the code
> following, until
> p._replace(kwarg) is respecified, or the code ends. So the response from
> Python shows me that p has transformed to the Point(x=33, y=22). A
> trivial example that doesn't begin to hint at the poential powers of
> this kwarg.
>
> The inventory[partnum] example is also quite trivial compared to the
> kwarg use that Peter showed me.
>
> When I asked him for an explanation, this is what I and he he said:
>
> Deborah Swanson wrote:
>
> > I know it's your "ugly" answer, but can I ask what the '**' in
> >
> > fix = {label: max(values, key=len)}
> > group[:] = [record._replace(**fix) for record in group]
> >
> > means?
>
> (Peter wrote)
> d = {"a": 1, "b": 2}
> f(**d)
>
> is equivalent to
>
> f(a=1, b=2)
>
> so ** is a means to call a function with keyword arguments when you want
> to
> decide about the *names* at runtime. Example:
>
> >>> def f(a=1, b=2):
> ... print("a =", a)
> ... print("b =", b)
> ... print()
> ...
> >>> for d in [{"a": 10}, {"b": 42}, {"a": 100, "b": 200}]:
> ... f(**d)
> ...
> a = 10
> b = 2
>
> a = 1
> b = 42
>
> a = 100
> b = 200
>
> Starting from a namedtuple `record`
>
> record._replace(Location="elswhere")
>
> creates a new namedtuple with the Location attribute changed to
> "elsewhere",
> and the slice [:] on the left causes all items in the `groups` list to
> be
> replaced with new namedtuples,
>
> group[:] = [record._replace(Location="elsewhere") for record in group]
>
> is basically the same as
>
> tmp = group.copy()
> group.clear()
> for record in tmp:
> group.append(record_replace(Location="elsewhere"))
>
> To support not just Location, but also Kind and Notes we need the double
>
> asterisk.
>
>
>
> Now, I understood what he meant from the context of the problem he was
> helping me with, but, how in the world would I have gotten that possible
> use of somenamedtuple._replace(kwargs) from the blurb and examples in
> the docs? Or any other possible kwarg and how they're used, including
> special operators like "**" ?
>
> If there's a tutorial somewhere on kwargs in general, what specific
> types of kwargs there are, what special operators can be specifed for
> them and how the kwargs are used, it would be very helpful to have that
> tutorial referenced in a footnote-link next to kwargs whenever kwargs is
> part of a function specification.
>
> Or, if no such tutorial exists, the footnote-link could point to any
> more detailed information that might explain what this creature "kwargs"
> might be.
>
> I have an item on my todo list to google "python kwargs", but it keeps
> getting pushed deeper and deeper down my list because I have so little
> hope that it will turn up 

RE: Survey: improving the Python std lib docs

[Deborah Swanson]

Michael Torrie wrote, on Wednesday, May 17, 2017 3:11 PM
> 
> On 05/17/2017 02:31 PM, Ned Batchelder wrote:
> > Can you give an example of such a method? Often, that signature is 
> > used because there is no pre-conception of what the arguments might 
> > be.
> 
> I'm not sure if this afflicts the standard library, but in my 
> own code, since Python doesn't support constructors with 
> different signatures, you pretty much have to rely on kwargs 
> with __init__() to handle different permutations of 
> construction arguments.  Not that we don't know what the 
> arguments might be, we just don't know which of them we'll 
> have to deal with.  Maybe the standard library is better 
> designed than my own code, but I find I have to implement 
> __init__(self, **kwargs) all the time.


While I can respect and appreciate your specialized interest in specific
kwargs, and I read in other's posts that help()'s display of kwargs is
conditional on several things, I really want to reiterate and clarify my
beginner's request.

I know of several functions I've used without any *args or **kwargs
because I had no clue from the docs what they might be. And, I know of
one function now, from posts in this list, which takes kwargs, but I
only know one application of one possible kwarg in this function because
one member of this list, Peter Otten, used it in a suggestion he gave
me.

This is all there is in the docs for that function:


somenamedtuple._replace(kwargs)

Return a new instance of the named tuple replacing specified fields
with new values:
(Examples
box)---|
|>>>
|
|
|
|>>> p = Point(x=11, y=22)
|
|>>> p._replace(x=33)
|
|Point(x=33, y=22)
|
|
|
|>>> for partnum, record in inventory.items():
|
|... inventory[partnum] =
record._replace(price=newprices[partnum], |
|  timestamp=time.now())
|
|---
|

>From this doc entry's example, I (sort of) get that x is a keyword arg
that is to be equal to 33 for any use of p._replace() in the code
following, until 
p._replace(kwarg) is respecified, or the code ends. So the response from
Python shows me that p has transformed to the Point(x=33, y=22). A
trivial example that doesn't begin to hint at the poential powers of
this kwarg.

The inventory[partnum] example is also quite trivial compared to the
kwarg use that Peter showed me.

When I asked him for an explanation, this is what I and he he said:

Deborah Swanson wrote:

> I know it's your "ugly" answer, but can I ask what the '**' in
> 
> fix = {label: max(values, key=len)}
> group[:] = [record._replace(**fix) for record in group]
> 
> means?

(Peter wrote)
d = {"a": 1, "b": 2}
f(**d)

is equivalent to

f(a=1, b=2)

so ** is a means to call a function with keyword arguments when you want
to 
decide about the *names* at runtime. Example:

>>> def f(a=1, b=2):
... print("a =", a)
... print("b =", b)
... print()
... 
>>> for d in [{"a": 10}, {"b": 42}, {"a": 100, "b": 200}]:
... f(**d)
... 
a = 10
b = 2

a = 1
b = 42

a = 100
b = 200

Starting from a namedtuple `record`

record._replace(Location="elswhere")

creates a new namedtuple with the Location attribute changed to
"elsewhere", 
and the slice [:] on the left causes all items in the `groups` list to
be 
replaced with new namedtuples,

group[:] = [record._replace(Location="elsewhere") for record in group]

is basically the same as

tmp = group.copy()
group.clear()
for record in tmp:
group.append(record_replace(Location="elsewhere"))

To support not just Location, but also Kind and Notes we need the double

asterisk.



Now, I understood what he meant from the context of the problem he was
helping me with, but, how in the world would I have gotten that possible
use of somenamedtuple._replace(kwargs) from the blurb and examples in
the docs? Or any other possible kwarg and how they're used, including
special operators like "**" ?

If there's a tutorial somewhere on kwargs in general, what specific
types of kwargs there are, what special operators can be specifed for
them and how the kwargs are used, it would be very helpful to have that
tutorial referenced in a footnote-link next to kwargs whenever kwargs is
part of a function specification.

Or, if no such tutorial exists, the footnote-link could point to any
more detailed information that might explain what this creature "kwargs"
might be.

I have an item on my todo list to google "python kwargs", but it keeps
getting pushed deeper and deeper down my list because I have so little
hope that it will turn up anything useful.

Deborah

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[Chris Angelico]

On Thu, May 18, 2017 at 8:11 AM, Michael Torrie  wrote:
> On 05/17/2017 02:31 PM, Ned Batchelder wrote:
>> Can you give an example of such a method? Often, that signature is used
>> because there is no pre-conception of what the arguments might be.
>
> I'm not sure if this afflicts the standard library, but in my own code,
> since Python doesn't support constructors with different signatures, you
> pretty much have to rely on kwargs with __init__() to handle different
> permutations of construction arguments.  Not that we don't know what the
> arguments might be, we just don't know which of them we'll have to deal
> with.  Maybe the standard library is better designed than my own code,
> but I find I have to implement __init__(self, **kwargs) all the time.

Also worth noting is that decorators sometimes have to go *a,**kw on
their functions. Until a couple of versions ago, help() would always
just show *a,**kw, but now there's a protocol between
functools.wraps() and help() that carries the original signature
through. Depending on exactly where you're viewing the info, eg
whether it uses inspect.getsignature or not, you might see the carried
original, or you might see *a,**kw.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[Michael Torrie]

On 05/17/2017 02:31 PM, Ned Batchelder wrote:
> Can you give an example of such a method? Often, that signature is used
> because there is no pre-conception of what the arguments might be.

I'm not sure if this afflicts the standard library, but in my own code,
since Python doesn't support constructors with different signatures, you
pretty much have to rely on kwargs with __init__() to handle different
permutations of construction arguments.  Not that we don't know what the
arguments might be, we just don't know which of them we'll have to deal
with.  Maybe the standard library is better designed than my own code,
but I find I have to implement __init__(self, **kwargs) all the time.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[Ned Batchelder]

On Wednesday, May 17, 2017 at 5:48:30 AM UTC-4, Cem Karan wrote:
> On May 16, 2017, at 12:36 PM, rzed  wrote:
> 
> > On Friday, May 12, 2017 at 6:02:58 AM UTC-4, Steve D'Aprano wrote:
> >> One of the more controversial aspects of the Python ecosystem is the Python
> >> docs. Some people love them, and some people hate them and describe them as
> >> horrible.
> >> 
> > [...]
> > 
> > One thing I would love to see in any function or class docs is a few 
> > example invocations, preferably non-trivial. If I need to see more, I can 
> > read the entire doc, but most times I just want a refresher on how the 
> > function is called. Does it use keywords? Are there required nameless 
> > parameters? In what order? A line or two would immediately clarify that 
> > most of the time. 
> > 
> > Apart from that, links to docs for uncommon functions (or to the docs of 
> > the module, if there are many) would be at least somewhat useful.
> 
> I'd like to see complete signatures in the docstrings, so when I use help() 
> on something that has *args or **kwargs I can see what the arguments actually 
> are.


Can you give an example of such a method? Often, that signature is used
because there is no pre-conception of what the arguments might be.

--Ned.
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: Survey: improving the Python std lib docs

[Deborah Swanson]

Cem Karan wrote, on Wednesday, May 17, 2017 2:48 AM
> 
> On May 16, 2017, at 12:36 PM, rzed  wrote:
> 
> > On Friday, May 12, 2017 at 6:02:58 AM UTC-4, Steve D'Aprano wrote:
> >> One of the more controversial aspects of the Python 
> ecosystem is the 
> >> Python docs. Some people love them, and some people hate them and 
> >> describe them as horrible.
> >> 
> > [...]
> > 
> > One thing I would love to see in any function or class docs is a few

> > example invocations, preferably non-trivial. If I need to 
> see more, I can read the entire doc, but most times I just 
> want a refresher on how the function is called. Does it use 
> keywords? Are there required nameless parameters? In what 
> order? A line or two would immediately clarify that most of the time.
> > 
> > Apart from that, links to docs for uncommon functions (or 
> to the docs 
> > of the module, if there are many) would be at least somewhat useful.
> 
> I'd like to see complete signatures in the docstrings, so 
> when I use help() on something that has *args or **kwargs I 
> can see what the arguments actually are.
> 
> Thanks,
> Cem Karan

I'd also like to see all of the above, especially what the possible
*args or **kwargs are.

Sure hope someone who might do these things is reading this.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[Cem Karan]

On May 16, 2017, at 12:36 PM, rzed  wrote:

> On Friday, May 12, 2017 at 6:02:58 AM UTC-4, Steve D'Aprano wrote:
>> One of the more controversial aspects of the Python ecosystem is the Python
>> docs. Some people love them, and some people hate them and describe them as
>> horrible.
>> 
> [...]
> 
> One thing I would love to see in any function or class docs is a few example 
> invocations, preferably non-trivial. If I need to see more, I can read the 
> entire doc, but most times I just want a refresher on how the function is 
> called. Does it use keywords? Are there required nameless parameters? In what 
> order? A line or two would immediately clarify that most of the time. 
> 
> Apart from that, links to docs for uncommon functions (or to the docs of the 
> module, if there are many) would be at least somewhat useful.

I'd like to see complete signatures in the docstrings, so when I use help() on 
something that has *args or **kwargs I can see what the arguments actually are.

Thanks,
Cem Karan
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[rzed]

On Friday, May 12, 2017 at 6:02:58 AM UTC-4, Steve D'Aprano wrote:
> One of the more controversial aspects of the Python ecosystem is the Python
> docs. Some people love them, and some people hate them and describe them as
> horrible.
> 
[...]

One thing I would love to see in any function or class docs is a few example 
invocations, preferably non-trivial. If I need to see more, I can read the 
entire doc, but most times I just want a refresher on how the function is 
called. Does it use keywords? Are there required nameless parameters? In what 
order? A line or two would immediately clarify that most of the time. 

Apart from that, links to docs for uncommon functions (or to the docs of the 
module, if there are many) would be at least somewhat useful. 

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[Marco Buttu]

On 15/05/2017 13:44, Ned Batchelder wrote:


As it is, if I make a suggestion about the itertools docs (why do we need
20-line "equivalent to" Python code, and why don't we have any usage
examples?), then I have to debate it with the developer of itertools,
who has a different aesthetic and style than the developer of logging,
or email, or re, and so on.


True story


If we had one person who had the authority to make doc-wide decisions,
then we might be able to move towards coherent guidelines for the docs
to be more uniform.


That could be a solution :-)

--
Marco Buttu

INAF-Osservatorio Astronomico di Cagliari
Via della Scienza n. 5, 09047 Selargius (CA)
Phone: 070 711 80 217
Email: mbu...@oa-cagliari.inaf.it

--
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[Ned Batchelder]

On Friday, May 12, 2017 at 6:02:58 AM UTC-4, Steve D'Aprano wrote:
> One of the more controversial aspects of the Python ecosystem is the Python
> docs. Some people love them, and some people hate them and describe them as
> horrible.
> 

I have a number of ideas for improving the docs, but I think there is a
larger issue that needs to be addressed first: there is no BDFL for the
docs. They are written and maintained piecemeal, by the core dev that
wrote the code. If one documentation-focused person had decision-making
power over all the docs, then we might be able to get some consistency
throughout.

As it is, if I make a suggestion about the itertools docs (why do we need
20-line "equivalent to" Python code, and why don't we have any usage
examples?), then I have to debate it with the developer of itertools,
who has a different aesthetic and style than the developer of logging,
or email, or re, and so on.

If we had one person who had the authority to make doc-wide decisions,
then we might be able to move towards coherent guidelines for the docs
to be more uniform.

--Ned.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[jeanbigboute]

On Saturday, May 13, 2017 at 3:39:52 PM UTC-7, Terry Reedy wrote:
> On 5/13/2017 1:23 PM, jeanbigbo...@gmail.com wrote:
> 
> > Thank you for bringing up this important topic.  As an occasional Python 
> > user, I find that Python documentation is all over the usability map - some 
> > great, some difficult.  The Python docs have been at best a starting point. 
> >  I usually need to go to other sites like this group, StackOverflow, and a 
> > blog or two to understand how to do something.  After I learn to do that 
> > one thing, I am not any more independent, self-reliant, or able to 
> > contribute back to the community.  Although Matlab gets a lot of grief in 
> > the open source community, its documentation is concise, complete, and 
> > self-contained.
> 
> Thank you for writing a response focused on your experience with the docs.

... [ trimmed ] ...

I appreciate your prompt reply.  I would like to 'aggregate' my thoughts along 
a couple of items you mentioned.

1) Old documentation (e.g. Classes referencing Modula-3) and opening trackers.

The Classes and Pkgutil docs are representative of a lot of the Python docs 
that fall into the hard-to-use category.  I would have to file many trackers to 
cover problems I have had.  That's not very grateful in context of an open 
source tool.  Discussing this at a top-level as we are doing here I think is 
fair game.  This is something that needs to be addressed comprehensively or 
possibly intentionally left alone.

If the original doc suite was written 20 years ago, it may not be possible or 
practical to get it caught up when the language evolves so quickly.  
The right answer might be to "leave it be" and accept that newer methods of Q 
have to take over.  This is true even of some paid packages - to get help with 
Microsoft, avoid the documentation wherever possible.  This approach has its 
problems especially for corporate users.  They're often behind firewalls and 
can't participate.

2) The blog post:  I think the post author's attempt was a good, honest try and 
an example of what the documentation suite might become.  I agree it is hard to 
come up with good, tested examples across multiple platforms and that's where 
some paid packages have an advantage.  I think the Python community could go a 
little easier on non-free tools.  Trying things and seeing what works is good 
in many cases but there isn't the time for that in a lot of workplaces, 
especially those that bill services by the hour.

--- JBB
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[Terry Reedy]

On 5/13/2017 1:23 PM, jeanbigbo...@gmail.com wrote:


Thank you for bringing up this important topic.  As an occasional Python user, 
I find that Python documentation is all over the usability map - some great, 
some difficult.  The Python docs have been at best a starting point.  I usually 
need to go to other sites like this group, StackOverflow, and a blog or two to 
understand how to do something.  After I learn to do that one thing, I am not 
any more independent, self-reliant, or able to contribute back to the 
community.  Although Matlab gets a lot of grief in the open source community, 
its documentation is concise, complete, and self-contained.


Thank you for writing a response focused on your experience with the docs.


A table of functions/classes at the start of a Python doc would be very 
helpful.  I'd also like to see a tree-like overview of a package so I can get 
an idea of what's available and how deep the dot-indexed hierarchy goes.  I've 
tried to write code to extract/present this for myself and have failed.  IDEs 
and Notebooks allow descending a branch one dot at a time - very time consuming.

Many of the docs go deep into CS terminology from the start which can be 
daunting. Here's the first paragraph from the 'Classes' documentation:
https://docs.python.org/2/tutorial/classes.html


The tutorial is meant to be for 'beginners'.


"Compared with other programming languages, Python’s class mechanism adds classes 
with a minimum of new syntax and semantics. It is a mixture of the class mechanisms found 
in C++ and Modula-3. Python classes provide all the standard features of Object Oriented 
Programming: the class inheritance mechanism allows multiple base classes, a derived 
class can override any methods of its base class or classes, and a method can call the 
method of a base class with the same name."

I am sure it is all correct.  I just don't know what it means.


When that was likely written, more that two decades ago, python 
beginners were experienced programmers who would have known much of the 
above.  Python beginners today are much different (and even experienced 
programmers are unlikely to know anything about Modula-x.)


If you would like it changed, open an issue on the tracker.


I saw pkgutil referenced in answers to some StackOverflow questions and I 
looked into it:

https://docs.python.org/2/library/pkgutil.html

"This module provides utilities for the import system, in particular package 
support."

Then it gets very specialized.

I am well aware of the distinction between documentation and tutorials. I think 
that the Python docs of today are too close to research articles and the 
specialized knowledge that implies.



This blog post is "in-the-face" [remainder of sentence moved below]
http://cryto.net/~joepie91/blog/2013/02/19/the-python-documentation-is-bad-and-you-should-feel-bad/


This blog post has several sins that you avoided, and did not help to 
improve the docs.  But anyway, to stay on topic...


> [moved] but has an example at the end of how docs might be rewritten.

The example is a separate page written after the original rant.
http://cryto.net/%7Ejoepie91/walk.html

This example expands 7 lines 6 times to about 42 and in the process 
manages to omit the following important info, which is about 1/5 of the 
original text.


"The visit function may modify names to influence the set of directories 
visited below dirname, e.g. to avoid visiting certain parts of the tree. 
(The object referred to by names must be modified in place, using del or 
slice assignment.)"


I am not sure where it would go in the template.  Templates can be 
straightjackets ;-).


The big issue is examples.  The problem with examples is that they are a 
maintenance burden.  Really.  Without testing, they are not dependable. 
And testing doc examples is not trivial.  So they need to really add value.


There are no examples in the os.path doc, and the particular issue for 
os.path is that results are idiosyncratic to a particular system. For 
some things, like the os.path functions, it is trivial for a user to try 
things out for themselves on their system.  The following took just a 
few minutes.


Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 18:41:36) [MSC v.1900 64 bit 
(AMD64)] on win32

Type "copyright", "credits" or "license()" for more information.
>>> import sys
>>> x = sys.executable
>>> import os.path as op
>>> op.isfile(x)
True
>>> op.isdir(x)
False
>>> x
'C:\\Programs\\Python36\\pythonw.exe'
>>> op.isabs(x)
True
>>> op.islink(x)
False
>>> op.getsize(x)
98968
>>> op.isdir(op.split(x)[0])
True
>>> op.splitdrive(x)
('C:', '\\Programs\\Python36\\pythonw.exe')
>>> op.join(x, 'Lib')
'C:\\Programs\\Python36\\pythonw.exe\\Lib'
>>> op.isdir(op.join(x, 'Lib'))
False
>>> op.getmtime(x)
1490136264.0

In any case, besides the tutorial and some of the reference chapters, 
python comes with some 'How-to's with examples.  There are also multiple 
3rd party sites that expand on the 

Re: Survey: improving the Python std lib docs

[Terry Reedy]

On 5/12/2017 6:02 AM, Steve D'Aprano wrote:


Here are a couple of suggestions for improving(?) the docs. What do you
think?
(They're not my ideas, the originated on Reddit.)



(1) Table of functions/classes at the start of each module doc


The only thing possibly 'new' here is 'each' versus 'selected'.  'Each' 
could be seen as an aspiration.  It could also be seen as a diversion 
from making concrete improvements, one chapter at a time, by a 
self-volunteer who will write and *commit to maintaining* such a table 
for a particular module.



The docs for builtins starts with a table of built-in functions:

https://docs.python.org/3/library/functions.html


Table has names only, alphabetically.  Maintenance will be collective.

https://docs.python.org/3/library/itertools.html#module-itertools

Categorized names, alphabetical within categories, with columns for 
arguments, results, and examples.  Raymond Hettinger maintains it.


> The statistics module shows something similar:
> https://docs.python.org/3/library/statistics.html

Categorized names with explanation, with a logical order within 
categories that is copied in the details section.  Steve will presumably 
make sure the index is updated if anyone adds new functions.


Steve, did you get any opposition to including that table?  Does the 
abstract idea of 'more tables' need discussion?



Docs for other modules should do similar, e.g. for the string module there
should be a table showing:

ascii_letters
ascii_lowercase
ascii_uppercase
capwords
digits
Formatter
hexdigits
octdigits
printable
punctuation
Template
whitespace



which link to the detailed documentation for that object.
https://docs.python.org/3/library/string.html


I disagree.  The alphabetical list mixes together 9 string constants, 
defining subsets of ascii characters, and 3 callables (1 function and 2 
classes).  The doc *could* start with a short table


Contents
| string constants | define subsets of ascii characters   |
| capwords | Capitalize each word in a string |
| Formatter| Custom formatting extending builtin format() |
| Template | Simple $-based string substitution   |

But this listing of mostly unused *objects* misses the 99% meat of the 
chapter, the definition format strings and format specs and the format() 
examples.  While this chapter originally documented the string module, 
it now mostly documents format() strings.  The doc for the string 
functions which are now strings methods was moved elsewhere.  (The 
capwords function is a vestigial remnant that was not made a string 
method.)  There is already a sidebar Table of Contents, which is 
generated automatically.


To me, the biggest deficiency of this chapter is the lack of any 
Formatter examples.  From a cursory reading, I have no idea how to use 
it.  I also question the placement of Formatter before the format() doc 
it depends on.


A table I would like is one that replaces the wordy contents of the 
'String constants' section, which should perhaps be renamed 'Character 
categories'.  I may have proposed this a decade ago and had the idea 
rejected.


| digits  | '0123456789'
...
| ascii_uppercase | 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
...
| printable   | digits + ascii_letters + punctuation + whitespace |

There should then be an explanation or reference to how at access 
similar unicode categories.

--

Conclusion: specific chapters need specific thought as to whether and 
what type of table might be useful.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[jeanbigboute]

On Friday, May 12, 2017 at 3:02:58 AM UTC-7, Steve D'Aprano wrote:
> One of the more controversial aspects of the Python ecosystem is the Python
> docs. Some people love them, and some people hate them and describe them as
> horrible.
> 
> Here are a couple of suggestions for improving(?) the docs. What do you
> think?
>

.[ deletia ].

Thank you for bringing up this important topic.  As an occasional Python user, 
I find that Python documentation is all over the usability map - some great, 
some difficult.  The Python docs have been at best a starting point.  I usually 
need to go to other sites like this group, StackOverflow, and a blog or two to 
understand how to do something.  After I learn to do that one thing, I am not 
any more independent, self-reliant, or able to contribute back to the 
community.  Although Matlab gets a lot of grief in the open source community, 
its documentation is concise, complete, and self-contained.  

A table of functions/classes at the start of a Python doc would be very 
helpful.  I'd also like to see a tree-like overview of a package so I can get 
an idea of what's available and how deep the dot-indexed hierarchy goes.  I've 
tried to write code to extract/present this for myself and have failed.  IDEs 
and Notebooks allow descending a branch one dot at a time - very time 
consuming.  

Many of the docs go deep into CS terminology from the start which can be 
daunting. Here's the first paragraph from the 'Classes' documentation:
https://docs.python.org/2/tutorial/classes.html

"Compared with other programming languages, Python’s class mechanism adds 
classes with a minimum of new syntax and semantics. It is a mixture of the 
class mechanisms found in C++ and Modula-3. Python classes provide all the 
standard features of Object Oriented Programming: the class inheritance 
mechanism allows multiple base classes, a derived class can override any 
methods of its base class or classes, and a method can call the method of a 
base class with the same name."

I am sure it is all correct.  I just don't know what it means.  

I saw pkgutil referenced in answers to some StackOverflow questions and I 
looked into it:

https://docs.python.org/2/library/pkgutil.html

"This module provides utilities for the import system, in particular package 
support."

Then it gets very specialized.  

I am well aware of the distinction between documentation and tutorials. I think 
that the Python docs of today are too close to research articles and the 
specialized knowledge that implies.

This blog post is "in-the-face" but has an example at the end of how docs might 
be rewritten.

http://cryto.net/~joepie91/blog/2013/02/19/the-python-documentation-is-bad-and-you-should-feel-bad/

--- JBB
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[dieter]

Steve D'Aprano  writes:
> One of the more controversial aspects of the Python ecosystem is the Python
> docs. Some people love them, and some people hate them and describe them as
> horrible.
>
> Here are a couple of suggestions for improving(?) the docs. What do you
> think?
>
> (They're not my ideas, the originated on Reddit.)
>
>
> (1) Table of functions/classes at the start of each module doc
>
> The docs for builtins starts with a table of built-in functions:
>
> https://docs.python.org/3/library/functions.html

>From my point of view, a good (manually maintained) documentation should not
contain information that can easily be obtained automatically.

Ideally, we should separate between overview information (explaining the
essential concepts and their relations) and detail information (list
of classes, functions, ...) with links between them. The detail information
is likely generated automatically from the source.

> ...
> (2) The PHP documentation allows you to search for a term by typing it into
> the URL after the domain, e.g. to search for "split", go to:
>
> http://php.net/split
>
>
> If you try the same thing with the Python docs:
>
> http://python.org/split
>
> you get a 404. Suggestion: 404s should redirect and search the docs.

Generate an "index" page from the complete documentation with links to
the term definitions. Then people who miss such a functionality can bookmark
that page.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[Chris Angelico]

On Sat, May 13, 2017 at 4:05 AM,   wrote:
> On Friday, May 12, 2017 at 3:02:58 AM UTC-7, Steve D'Aprano wrote:
>
>> (1) Table of functions/classes at the start of each module doc
>>
>> The docs for builtins starts with a table of built-in functions:
>>
>> https://docs.python.org/3/library/functions.html
>>
>>
>> Docs for other modules should do similar...
>
> I agree with this suggestion.  I usually know what I want out of a module, 
> but I don't know the exact name(s) of the relevant function(s).  Frequently, 
> I find myself looking through the docs... and in parallel, I start a Python 
> interpreter, type "import foo", and then "dir(foo)" to see everything that 
> the module foo contains.
>

While I don't disagree with the docs suggestion, it's worth noting
that dir(foo) is a powerful feature, and part of what makes Python so
awesome. So don't be afraid to use it :)

TBH I'm +0.5 on the table; if it includes absolutely everything the
module offers, it'll be unworkably large on some modules, but if it
doesn't, how do you pick which are the "important" ones? Possibly this
is the right solution for most modules, but there'll be a few special
cases (eg argparse) that prefix it with "hey, check out this quick
start guide first".

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[Ethan Furman]

On 05/12/2017 03:02 AM, Steve D'Aprano wrote:


Here are a couple of suggestions for improving(?) the docs. What do you
think?



(1) Table of functions/classes at the start of each module doc


I like this idea.  Even if I don't know the exact thing I am looking for I can 
usually get close from the names.

--
~Ethan~

--
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[jladasky]

On Friday, May 12, 2017 at 3:02:58 AM UTC-7, Steve D'Aprano wrote:

> (1) Table of functions/classes at the start of each module doc
> 
> The docs for builtins starts with a table of built-in functions:
> 
> https://docs.python.org/3/library/functions.html
> 
> 
> Docs for other modules should do similar...

I agree with this suggestion.  I usually know what I want out of a module, but 
I don't know the exact name(s) of the relevant function(s).  Frequently, I find 
myself looking through the docs... and in parallel, I start a Python 
interpreter, type "import foo", and then "dir(foo)" to see everything that the 
module foo contains.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[Dan Sommers]

On Fri, 12 May 2017 21:14:01 +1000, Chris Angelico wrote:

> On Fri, May 12, 2017 at 8:02 PM, Steve D'Aprano
>  wrote:
>> (2) The PHP documentation allows you to search for a term by typing it into
>> the URL after the domain, e.g. to search for "split", go to:
>>
>> http://php.net/split
>>
>>
>> If you try the same thing with the Python docs:
>>
>> http://python.org/split
>>
>> you get a 404. Suggestion: 404s should redirect and search the docs.
> 
> So long as we're talking about the existing search functionality, I'm
> a big NO on this one. It's way too slow to be used as a general 404.

I agree with ChrisA.  That said, if part of the 404 page were a link to
such a search, that may be convenient.

python.org/split does not exist.

Would you like to _search the python documentation for split_?

(where the underscore-bracketed text would be a link to
https://docs.python.org/3/search.html?q=split).

Dan
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Survey: improving the Python std lib docs

[Chris Angelico]

On Fri, May 12, 2017 at 8:02 PM, Steve D'Aprano
 wrote:
> (2) The PHP documentation allows you to search for a term by typing it into
> the URL after the domain, e.g. to search for "split", go to:
>
> http://php.net/split
>
>
> If you try the same thing with the Python docs:
>
> http://python.org/split
>
> you get a 404. Suggestion: 404s should redirect and search the docs.

So long as we're talking about the existing search functionality, I'm
a big NO on this one. It's way too slow to be used as a general 404.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list