Re: [Python-Dev] Escaping docs markup in NEWS entries?

[Wes Turner]

On Tue, May 2, 2017 at 1:58 PM, David Goodger  wrote:

> On Mon, May 1, 2017 at 5:03 AM, Wes Turner  wrote:
> > Where would be a good place for test cases for an rst_escape() function?
> > Docutils?
> >
> > https://github.com/westurner/dotfiles/blob/develop/scripts/
> git-changelog.py
>
> I'd say that the test cases for any function should reside in the same
> project/codebase as the function itself. Where do you intend the
> function to reside?
>
> I'm not certain what rst_escape() is supposed to do; what is its
> purpose? The docstrings of rst_escape() as well as the
> git-changelog.py module lack such explanations.
>

def rst_escape(_str: str) -> str:

Args:
_str: string to safely escape for literal inclusion in a
ReStructuredText document
Returns:
str: safely-escaped ReStructuredText



>
> I am also uncertain if this is necessary, based on the remainder of
> the thread. Perhaps it would be better to fix this "make suspicious"
> so that it doesn't produce spurious results? (Note: I am unclear on
> exactly what "make suspicious" is supposed to do, or why.)
>

An rst_escape(text: ) function is necessary in order to prevent
ReStructuredText injection (and e.g. * unintentional partial italicization).

There are other directives for including text in a ReStructuredText
document, e.g.:

- ``.. include::``
- ``.. raw::``

And there are extensions for changelogs and sphinx builds:

- ``.. git_changelog::`` -- https://pypi.org/project/sphinx-git/
- ``.. changelog::``, ``.. change::`` -- https://pypi.org/project/changelog/


But, for a README.rst in a git repository, for example, it's far easier to
diff the document with the data included than it is to have to build from a
source .rst to a dest .rst.rst and then diff the .rst.rst files.

There are CWE URLs for describing the type of vulnerability that a
community-reviewed rst_escape() function would be designed to prevent:

- CWE-116: Improper Encoding or Escaping of Output
  https://cwe.mitre.org/data/definitions/116.html
  - CWE-74: Improper Neutralization of Special Elements in Output Used by a
Downstream Component ('Injection')
https://cwe.mitre.org/data/definitions/74.html#Relationships

- CWE-89: SQL Injection
  https://cwe.mitre.org/data/definitions/89.htm

  - 2011 Top25 #1 issue
https://cwe.mitre.org/top25/#CWE-89

- [ ] CWE-???: RST Injection


Not a bug in docutils; a PEBKAM implementation "BUG,SEC" issue


>
> I apologize for the azure hue of my answers, to questions that came
> from out of the blue.
> ;-)
>
> David Goodger
> 
>
>
> > - rst_escape # YMMV
> > - $ git-changelog.py -r "release/0.3.14" --hdr= "+"`
> >
> > On Monday, May 1, 2017, Nick Coghlan  wrote:
> >>
> >> On 1 May 2017 at 17:13, Martin Panter  wrote:
> >> > On 1 May 2017 at 06:37, Nick Coghlan  wrote:
> >> >> Hi folks,
> >> >>
> >> >> I'm trying to write a NEWS entry that explains that the
> >> >> ":func:`bytes`" cross-references have changed to refer to the type
> >> >> descriptions by default (matching other builtin container types), so
> >> >> you now need to use ``:ref:`func-bytes`" to refer to the old target
> in
> >> >> the list of builtin functions (if you really want that for some
> >> >> reason).
> >> >>
> >> >> Unfortunately, my first two attempts both cause warnings in "make
> >> >> suspicious" with the following output:
> >> >
> >> > What is the full output? Usually it includes instructions to add false
> >> > positives to Doc/tools/susp-ignored.csv; maybe that is all you have to
> >> > do?
> >>
> >> You're right, that would be likely be the way to go if I decided to
> >> keep the escaped markup.
> >>
> >> However...
> >>
> >> >> My fallback plan is to just include the unescaped markup in the NEWS
> >> >> entry (rather than trying to make it readable even in rendered form),
> >> >> but I figured I'd ask for advice here first.
> >> >
> >> > I thought the NEWS file was mainly regarded as plain text, so it would
> >> > be important to avoid ugly RST markup like backslashes if possible.
> >>
> >> ... I think you're right on this point, so it makes more sense to skip
> >> the escaping entirely,
> >> and just use the correct link markup in the NEWS entry.
> >
> > How convenient.
> >>
> >>
> >> Cheers,
> >> Nick.
> >>
> >> --
> >> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> >> ___
> >> Python-Dev mailing list
> >> Python-Dev@python.org
> >> https://mail.python.org/mailman/listinfo/python-dev
> >> Unsubscribe:
> >> https://mail.python.org/mailman/options/python-dev/
> wes.turner%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 

Re: [Python-Dev] Escaping docs markup in NEWS entries?

[David Goodger]

On Mon, May 1, 2017 at 5:03 AM, Wes Turner  wrote:
> Where would be a good place for test cases for an rst_escape() function?
> Docutils?
>
> https://github.com/westurner/dotfiles/blob/develop/scripts/git-changelog.py

I'd say that the test cases for any function should reside in the same
project/codebase as the function itself. Where do you intend the
function to reside?

I'm not certain what rst_escape() is supposed to do; what is its
purpose? The docstrings of rst_escape() as well as the
git-changelog.py module lack such explanations.

I am also uncertain if this is necessary, based on the remainder of
the thread. Perhaps it would be better to fix this "make suspicious"
so that it doesn't produce spurious results? (Note: I am unclear on
exactly what "make suspicious" is supposed to do, or why.)

I apologize for the azure hue of my answers, to questions that came
from out of the blue.
;-)

David Goodger



> - rst_escape # YMMV
> - $ git-changelog.py -r "release/0.3.14" --hdr= "+"`
>
> On Monday, May 1, 2017, Nick Coghlan  wrote:
>>
>> On 1 May 2017 at 17:13, Martin Panter  wrote:
>> > On 1 May 2017 at 06:37, Nick Coghlan  wrote:
>> >> Hi folks,
>> >>
>> >> I'm trying to write a NEWS entry that explains that the
>> >> ":func:`bytes`" cross-references have changed to refer to the type
>> >> descriptions by default (matching other builtin container types), so
>> >> you now need to use ``:ref:`func-bytes`" to refer to the old target in
>> >> the list of builtin functions (if you really want that for some
>> >> reason).
>> >>
>> >> Unfortunately, my first two attempts both cause warnings in "make
>> >> suspicious" with the following output:
>> >
>> > What is the full output? Usually it includes instructions to add false
>> > positives to Doc/tools/susp-ignored.csv; maybe that is all you have to
>> > do?
>>
>> You're right, that would be likely be the way to go if I decided to
>> keep the escaped markup.
>>
>> However...
>>
>> >> My fallback plan is to just include the unescaped markup in the NEWS
>> >> entry (rather than trying to make it readable even in rendered form),
>> >> but I figured I'd ask for advice here first.
>> >
>> > I thought the NEWS file was mainly regarded as plain text, so it would
>> > be important to avoid ugly RST markup like backslashes if possible.
>>
>> ... I think you're right on this point, so it makes more sense to skip
>> the escaping entirely,
>> and just use the correct link markup in the NEWS entry.
>
> How convenient.
>>
>>
>> Cheers,
>> Nick.
>>
>> --
>> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] __getattribute__'s error is not available in __getattr__

[Ethan Furman]

On 05/01/2017 08:47 PM, Jason Maldonis wrote:


If this should be asked in learn python I apologize -- please just tell me 
without answering.


If you could re-ask this question over on Python List I'd love to discuss which errors you are seeing from 
__getattribute__ (beside AttributeError, of course, as that's the only one I would expect to see).


--
~Ethan~

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Possible bug in class-init, lookin for mentors

[Mariatta Wijaya]

Justus, welcome.

Consider joining the core-mentorship mailing list. If you have specific
questions on how to contribute or how to get started, we can help you
there. https://mail.python.org/mailman/listinfo/core-mentorship

Thanks, Erik. Yes, CPython has moved to GitHub https://github.com/python/
cpython but we will continue using bugs.python.org as the issue tracker.
We prefer that discussions happen on the issue tracker, and GitHub is for
code reviewing only.

The Dev Guide has been updated with our new GitHub workflow, specifically:
http://cpython-devguide.readthedocs.io/pullrequest.html
http://cpython-devguide.readthedocs.io/gitbootcamp.html

Thanks :)


Mariatta Wijaya

On Tue, May 2, 2017 at 9:01 AM, Erik Bray  wrote:

> On Fri, Apr 21, 2017 at 12:09 PM, Justus Schwabedal
>  wrote:
> > Hi everyone,
> >
> > I possibly found a bug in class initialization and would like to fix it.
> > Because it's my first journey to core-dev, I would really appreciate the
> > help of a mentor that I may ask a few questions to get me up to speed.
> >
> > To my person, I have previously worked on larger projects in python, c,
> and
> > c++ if that information helps, and I'm really curious to learn more about
> > the interiors of the greatest interpreter known to wo-/men.
> >
> > Here comes the bug-producing example:
> >
> > `class Foo:
> > def __init__(self, bar=[]):
> > self.list = bar
> >
> > spam_1 = Foo()
> > spam_2 = Foo()
> >
> > spam_1.list.append(42)
> > print(spam_2.list)`
> >
> > At least I think it's a bug.  Maybe it's a feature..
>
> Sorry to resurrect an old-ish thread; I haven't looked at Python-dev
> in several weeks.  I just wanted to point out that while everyone on
> this thread pointed out how this isn't a bug (clear to anyone who's
> spent enough time with Python), we have here an experienced C/C++
> developer who is interested in helping out on Python core devel, and
> no one took him up on that offer.
>
> Jus, for what it's worth, there are a slew of *actual* Python bugs to
> be worked on--many languishing for years--due to lack of available
> developer time.  You can have a good look at the (daunting) list of
> bugs at the old bugs.python.org [1].  IIUC Python development is
> slowly moving over to GitHub, but the issue list hasn't migrated yet
> so that would still be the place to start.
>
> If you find a bug that looks worth your time to try to fix, you should
> probably follow up on the issue for that bug itself.  I can't make any
> promises anyone will have time for mentorship, but I'd be willing to
> point you in the right direction.  I'm not a core developer but I know
> Python internals reasonably well and might be able to help *depending*
> on the issue.
>
> Best,
> Erik
>
> [1] http://bugs.python.org/
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> mariatta.wijaya%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Guarantee the success of some object creation C API functions

[Brett Cannon]

On Mon, 1 May 2017 at 21:19 Nick Coghlan  wrote:

> On 2 May 2017 at 07:52, Chris Angelico  wrote:
> > On Tue, May 2, 2017 at 6:52 AM, Terry Reedy  wrote:
> >> The promise makes it clear that breaking the property is a bug to be
> fixed.
> >> It only decreases the probability for someone who has read the promise.
> >> Unfortunately, 'never fail' is hard to test ;-).
> >>
> >
> > Aside from straight-up bugs, how can one of these functions fail? Is
> > memory allocation failure the only way? If so, the proposed
> > implementation (private references to pre-created singletons) ought to
> > guarantee that, to the exact extent that anything else can be
> > guaranteed.
> >
> > (Or is that your point - that "never fail" is always "modulo bugs"?)
> >
> > Incidentally, this guarantee, if implemented the obvious way, will
> > also mean that (), "", 0, etc are singletons. People talk casually
> > about the "empty tuple singleton", but I don't think it's actually
> > guaranteed anywhere.
>
> I don't think it is either, and I don't think it's a guarantee we want
> to make - even with Serhiy's proposed change, it's still
> straightforward to produce non-singleton instances of these values
> using the low level C APIs, while the true singletons (True, False,
> None, Ellipsis) go out of their way to make it difficult to create new
> instances of the corresponding types, even when mucking about at the C
> layer.
>
> The assertion Serhiy is proposing to make is much weaker, and would be
> a matter of adding something like the following to the C API
> reference:
>
> "Similar to the singleton values True, False, None, and Ellipsis,
> process global instances of the builtin values (), '', b'', 0, and 1
> are pre-allocated at interpreter startup, so APIs returning these
> values should never fail, even in low memory conditions. However,
> additional empty instances of these types may still be created through
> the C API, so they should always be compared by value rather than by
> identity."
>
> The specific "never fails as it returns a pointer to a pre-allocated
> instance" cases can then be documented on the affected public API
> functions.
>
> So +1 from me for making pre-allocation a CPython C API guarantee, -1
> for making it a language level singleton commitment.


I agree with Nick's votes.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Possible bug in class-init, lookin for mentors

[Erik Bray]

On Fri, Apr 21, 2017 at 12:09 PM, Justus Schwabedal
 wrote:
> Hi everyone,
>
> I possibly found a bug in class initialization and would like to fix it.
> Because it's my first journey to core-dev, I would really appreciate the
> help of a mentor that I may ask a few questions to get me up to speed.
>
> To my person, I have previously worked on larger projects in python, c, and
> c++ if that information helps, and I'm really curious to learn more about
> the interiors of the greatest interpreter known to wo-/men.
>
> Here comes the bug-producing example:
>
> `class Foo:
> def __init__(self, bar=[]):
> self.list = bar
>
> spam_1 = Foo()
> spam_2 = Foo()
>
> spam_1.list.append(42)
> print(spam_2.list)`
>
> At least I think it's a bug.  Maybe it's a feature..

Sorry to resurrect an old-ish thread; I haven't looked at Python-dev
in several weeks.  I just wanted to point out that while everyone on
this thread pointed out how this isn't a bug (clear to anyone who's
spent enough time with Python), we have here an experienced C/C++
developer who is interested in helping out on Python core devel, and
no one took him up on that offer.

Jus, for what it's worth, there are a slew of *actual* Python bugs to
be worked on--many languishing for years--due to lack of available
developer time.  You can have a good look at the (daunting) list of
bugs at the old bugs.python.org [1].  IIUC Python development is
slowly moving over to GitHub, but the issue list hasn't migrated yet
so that would still be the place to start.

If you find a bug that looks worth your time to try to fix, you should
probably follow up on the issue for that bug itself.  I can't make any
promises anyone will have time for mentorship, but I'd be willing to
point you in the right direction.  I'm not a core developer but I know
Python internals reasonably well and might be able to help *depending*
on the issue.

Best,
Erik

[1] http://bugs.python.org/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com