Re: Canonical list of Python security vulnerabilities
On Sat, Jul 15, 2023 at 1:02 PM Dieter Maurer wrote: > > I am active in the `Zope` community (a web application server > based on Python). This community has a security mailing list > for security related reports > and issues public CVE (= "Commun Vulnerabilities and Exposures") reports > (via a "GitHUB" service) as soon as a security risk has been resolved. > > I expect that security risks for Python itself are handled in > a similar way (as, Python too, maintains its code on "GitHUB"). Yes the Python community does have a security mailing list, but as I noted earlier, it appears to be moribund. And yes, the cpython GitHub repository does have a security tab, but it reports "There aren’t any published security advisories." > ... > For details about CVE, read > "https://en.wikipedia.org/wiki/Common_Vulnerabilities_and_Exposures";. Thanks for the link, Dieter. I found the NIST search interface to be buggy, and there doesn't seem to be a way to search the Mitre site effectively to get vulnerabilities just for the Python language and standard libraries. I've downloaded the entire corpus of JSON CVEs and I'm digging into what would be involved in querying it myself. Cheers, Bob -- https://mail.python.org/mailman/listinfo/python-list
Re: Canonical list of Python security vulnerabilities
On Fri, Jul 14, 2023 at 3:02 PM Barry wrote: > Where do you get your python from? Directly from python.org. > You may find that the organisation that packages python that you use has such > a list. That's my hope. Just haven't found it yet. :-} -- https://mail.python.org/mailman/listinfo/python-list
Re: Canonical list of Python security vulnerabilities
On Fri, Jul 14, 2023 at 1:35 PM Bob Kline wrote: > Can someone point me to the official catalog of security vulnerabilities > in Python I did try entering "python security vulnerabilities" in the search box of the python.org web site, but what I got back was "No results found." -- https://mail.python.org/mailman/listinfo/python-list
Canonical list of Python security vulnerabilities
Can someone point me to the official catalog of security vulnerabilities in Python (by which I mean cpython and the standard libraries)? I found https://www.cvedetails.com/vulnerability-list/vendor_id-10210/product_id-18230/Python-Python.html but that isn't maintained by python.org. I also found security-annou...@python.org, but there hasn't been anything posted there in over a year as far as I can tell, and even before that it's pretty thin. If there's a better place to ask, please advise. Thanks. -- Bob Kline https://www.rksystems.com mailto:bkl...@rksystems.com -- https://mail.python.org/mailman/listinfo/python-list
The argparse docs don't say who's responsible for closing FileType objects
The subject line pretty much says it all. Should the programmer close the file? If the programmer does that, and the user has asked that the file object be hooked up to standard in (or standard out) what will happen? If the programmer doesn't close it, does it get closed cleanly in the face of an exception? Thanks! -- https://mail.python.org/mailman/listinfo/python-list
Re: Part of RFC 822 ignored by email module
On 1/20/2011 5:34 PM, Martin Gregorie wrote: On Thu, 20 Jan 2011 16:25:52 -0500, Bob Kline wrote: On 1/20/2011 3:48 PM, Martin Gregorie wrote: That's only a problem if your code cares about the composition of the whitespace and this, IMO is incorrect behaviour. When the separator between syntactic elements in a header is 'whitespace' it should not matter what combination of newlines, tabs and spaces make up the whitespace element. That would be true for what the RFC calls "structured" fields, but not for the others (such as the Subject header). Subject text comparisons should work correctly if you were to split the subject text using the 'whitespace' definition and then reassemble it using a single space in place of each whitespace separator. Its either that or assuming that all MUAs use the same line length and all use a line split of "CRLF " - the whitespace that's needed to align the continuation with the test on the first subject line. Many MUAs will do that, but its unlikely that all will. Thanks. I'm not sure everyone would agree that it's OK to collapse multiple consecutive spaces into one, but I'm beginning to suspect that those more concerned with preserving as much as possible of the original message are in the minority. It sounds like my take-home distillation from this thread is "yes, the module ignores what the spec says about unfolding, but it doesn't matter." I guess I can live with that. -- Bob Kline http://www.rksystems.com mailto:bkl...@rksystems.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Part of RFC 822 ignored by email module
On 1/20/2011 3:48 PM, Martin Gregorie wrote: That's only a problem if your code cares about the composition of the whitespace and this, IMO is incorrect behaviour. When the separator between syntactic elements in a header is 'whitespace' it should not matter what combination of newlines, tabs and spaces make up the whitespace element. That would be true for what the RFC calls "structured" fields, but not for the others (such as the Subject header). -- Bob Kline http://www.rksystems.com mailto:bkl...@rksystems.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Part of RFC 822 ignored by email module
On 1/20/2011 12:58 PM, Dennis Lee Bieber wrote:
I'd first be concerned about the absence of the line ending sequence
specified by the RFC: carriage return (CR; \r) followed by line feed
(LF; \n).
I realized after I posted the original message that this might distract
from the issue I'm asking about. The module is a little sloppy about
its own generation of CRLF tokens when it serializes messages (it leaves
the insertion of the CR to processing further downstream), and I was
mimicking that behavior in what I fed to the method in my example. I
should have avoided that problem and included the \r. Let's pretend
that I did: the behavior is unaffected by the absence or presence of the
CR character.
...
However, the module does appear to trim leading whitespace that
occurs between the : and text (and the line end is considered for that
trimming, but not any whitespace after it).
Exactly. I'm focusing on the word "equivalent" in the RFC. Either the
module is going to strip leading white space from the value or it's
not. Returning different values for the unwrapped header, depending on
whether wrapping was present, is failing to treat the two sequences as
equivalent. Logically, the processing sequence should be:
1. Unwrap the header ("Subject:\r\n foo" becomes "Subject: foo")
2. Trim leading white space (" foo" becomes "foo")
Ideally, the behavior of trimming the leading white space would be
reflected documentation (but it isn't).
--
Bob Kline
http://www.rksystems.com
mailto:bkl...@rksystems.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: Part of RFC 822 ignored by email module
On 1/20/2011 12:23 PM, Carl Banks wrote:
On Jan 20, 7:08 am, Bob Kline wrote:
I just noticed that the following passage in RFC 822:
The process of moving from this folded multiple-line
representation of a header field to its single line represen-
tation is called "unfolding". Unfolding is accomplished by
regarding CRLF immediately followed by a LWSP-char as
equivalent to the LWSP-char.
is not being honored by the email module. The following two invocations
of message_from_string() should return the same value, but that's not
what happens:
>>> import email
>>> email.message_from_string("Subject: blah").get('SUBJECT')
'blah'
>>> email.message_from_string("Subject:\n blah").get('SUBJECT')
' blah'
Note the space in front of the second value returned, but missing from
the first. Can someone convince me that this is not a bug?
That's correct, according to my reading of RFC 822 (I doubt it's
changed so I didn't bother to look up what the latest RFC on that
subject is.)
The RFC says that in a folded line the whitespace on the following
line is considered a part of the line.
Thanks for responding. I think your interpretation of the RFC is the
same is mine. What I'm saying is that by not returning the same value
in the two cases above the module is not "regarding CRLF immediately
followed by a LWSP-char as equivalent to the LWSP-char."
--
Bob Kline
http://www.rksystems.com
mailto:bkl...@rksystems.com
--
http://mail.python.org/mailman/listinfo/python-list
Part of RFC 822 ignored by email module
I just noticed that the following passage in RFC 822:
The process of moving from this folded multiple-line
representation of a header field to its single line represen-
tation is called "unfolding". Unfolding is accomplished by
regarding CRLF immediately followed by a LWSP-char as
equivalent to the LWSP-char.
is not being honored by the email module. The following two invocations
of message_from_string() should return the same value, but that's not
what happens:
>>> import email
>>> email.message_from_string("Subject: blah").get('SUBJECT')
'blah'
>>> email.message_from_string("Subject:\n blah").get('SUBJECT')
' blah'
Note the space in front of the second value returned, but missing from
the first. Can someone convince me that this is not a bug?
--
Bob Kline
http://www.rksystems.com
mailto:bkl...@rksystems.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: Change in cgi handling of POST requests
Aahz wrote: Interesting. Nobody has responded, so I suggest first filing a report using bugs.python.org and then asking on python-dev (with reference to your report). http://bugs.python.org/issue5340 Cheers, Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: Change in cgi module's handling of POST requests
Joshua Kugler wrote:
We just upgraded Python to 2.6 on some of our servers and a number of our
CGI scripts broke because the cgi module has changed the way it handles
POST requests. When the 'action' attribute was not present in the form
element on an HTML page the module behaved as if the value of the
attribute was the URL which brought the user to the page with the form,
but without the query (?x=y...) part.
This does not make sense. Can you give an example?
Sure. Here's a tiny repro script:
#!/usr/bin/python
import cgi, xml.sax.saxutils
def quote(me): return me and xml.sax.saxutils.quoteattr(str(me)) or ''
print """\
Content-type: text/html
""" % quote(cgi.FieldStorage().getvalue('x'))
end of repro script
Try it out on this pre-2.6 Python page:
http://www.rksystems.com/cgi-bin/cgi-repro.py?x=y
When the page comes up, click Submit. Click it several times. No
change in the content of the text field, which is populated when the
page first comes up from the GET request's URL, and then subsequently
from the POST request's parameters.
For comparison, here's the equivalent Perl page, which behaves the same way:
http://www.rksystems.com/cgi-bin/cgi-repro.pl?x=y
Or PHP; again, same behavior, no matter how many times you click the
Submit button:
http://www.rksystems.com/cgi-repro.php?x=y
Now try the Python script above from a server where Python has been
upgraded to version 2.6:
http://mahler.nci.nih.gov/cgi-bin/cgi-repro.py?x=y
Notice that when you click on the Submit button, the field is populated
with the string representation of the list which FieldStorage.getvalue()
returns. Each time you click the submit button you'll see the effect
recursively snowballing. This is exactly the same script as the one
behind the first URL above, byte for byte.
Now FieldStorage.getvalue () is
giving the script a list of two copies of the value for some of the
parameters (folding in the parameters from the previous request) instead
of the single string it used to return for each.
There is a function call to get only one value. I think it's get_first() or
some such.
That's true, but risky. I have no guarantee that the value entered by
the user on the form will be first on the list. I might instead get the
initial value carried over from the URL which brought up the form to
begin with. We're working around the problem by modifying the broken
scripts to explicitly set the action attributes.
Well, the CGI module hasn't had many changes. There was this bug fix a few
months back:
http://svn.python.org/view?rev=64447&view=rev
Looks like that was where it happened.
It is possible that by fixing a bug, they brought the behavior in line with
what it *should* be.
That's certainly possible. I'm not contending that Perl and PHP and the
previous versions of Python all got it right and the new Python is
wrong. It could very well be the other way around.[1] But my
expectation, based on what I've seen happen over the years with other
proposed changes to the language and the libraries, was that there would
have been some (possibly extended) discussion of the risks of breaking
existing code, and the best way to phase in the change with as little
sudden breakage as possible. I haven't been able to find that
discussion, and I was hoping some kind soul would point me in the right
direction.
Or maybe the browser behavior changed?
Clearly not, as you will see by using the same browser to try out the
URLs above. If you look at the HTML source when the page first comes up
for each of the scripts, you'll see it's the same. It's the behavior on
the server (that is, in the Python library module) which changes.
The server
does not care about an "action" attribute. That only tells the browser
where to send the data.
Well that's a pretty good formulation of the conclusion you would come
to based on the behavior of all of Perl, PHP, and (pre-2.6) Python. And
intuitively, that's how one (or at least I) would expect things to
work. The parameters in the original URL are appropriately used to seed
initial values in the form when the form is invoked with a GET request,
but after that point it's hard to see them as anything but history. But
that's not how the new version of the cgi module is behaving. It's
folding in the parameters it finds in the original URL, which it gets
from the environment's QUERY_STRING variable, in with the fields it
parses from the POST request's body.
It is possible the browser did not properly format
a request when there was no "action" attribute.
When the 'action' attribute is not present in the form element, the
browser implicitly assigns it the value of the original URL which first
brought up the page with the form. This browser behavior has not
changed. It's doing the same thing no matter which version of which
language and libraries are used to implement the
Change in cgi module's handling of POST requests
[Didn't realize the mirror didn't work both ways] We just upgraded Python to 2.6 on some of our servers and a number of our CGI scripts broke because the cgi module has changed the way it handles POST requests. When the 'action' attribute was not present in the form element on an HTML page the module behaved as if the value of the attribute was the URL which brought the user to the page with the form, but without the query (?x=y...) part. Now FieldStorage.getvalue () is giving the script a list of two copies of the value for some of the parameters (folding in the parameters from the previous request) instead of the single string it used to return for each. I searched this newsgroup looking for a discussion of the proposal to impose this change of behavior, and perhaps I wasn't using the right phrases in my search, but I didn't find anything. I see that Perl still behaves the way pre-2.6 Python used to (not that I view that as a reason for anything). We'll work around the breakage by explicitly setting the 'action' attri bute everywhere, of course, but I usually see some discussion (often heated) of the impact of behavior changes on existing software when something like this is in the works. Did I miss it? I also noted that the module is making some deprecated use of the BaseException class. Cheers. -- http://mail.python.org/mailman/listinfo/python-list
Re: Un(der)documented bits of cgi.py
Aahz wrote: > What I suggest doing is submitting a doc patch to > http://bugs.python.org/ Done. Thanks for the suggestion. Bob -- http://mail.python.org/mailman/listinfo/python-list
Un(der)documented bits of cgi.py
I'm trying to detect and intelligently deal with problems created when a user of a Python CGI page uploads a file and then gets impatient and clicks on some other button or the browser's cancel button (or even closes the page). If the file is large enough, and the user is impatient enough, this can result in the FieldStorage.file object getting some, but not all of the bytes from the user's file, with no documented means of detecting that this has happened. Poking around in the code for cgi.py in the standard library, it appears that there is an undocumented 'done' attribute which might be useful here (it seems as if it might get set to -1 when this condition is encountered), but of course one is reluctant to leave behind software which relies on undocumented behavior of packages, since those bits have a way of disappearing in the middle of the night, suddenly breaking one's software. Can anyone think of a good reason why it would not be desirable to expose a publicly documented means of detecting the condition described above? Cheers and thanks! Bob Kline -- http://mail.python.org/mailman/listinfo/python-list
Re: getting database column names from query
Jason Nordwick wrote: > I'm using MySQLdb and can connect and issue queries that return result > sets, but I how do I get the column names for those result sets? [d[0] for d in k.description] Cheers, Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: Optimization of __len__() in cgi.py
Georg Brandl wrote: > Post a RFE to the Python Tracker at > http://sourceforge.net/tracker/?group_id=5470&atid=355470 > > If you want, assign it to me (gbrandl). Done, thanks. Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: Optimization of __len__() in cgi.py
Sybren Stuvel wrote: > FieldStorage.__nonzero__ tried first if it exists. You might want to > use that for more optimization. Excellent suggestion! It would be nice if this were adopted to supplement the original optimization, rather than replace it. Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: Optimization of __len__() in cgi.py
Marc 'BlackJack' Rintsch wrote:
>> def keys(self):
>> return {}.fromkeys([i.name for i in self.list]).keys()
>
> This does not maintain the order of `self.list`. Don't know if there's
> code relying on this.
Such code would be flying in the face of an implication that the order
of the keys is *not* preserved, as the keys() method is documented as
"Dictionary style keys() method" (and the documentation for dictionary
keys() says "... listed in an arbitrary order which is non-random ...").
If it were decided that the behavior of keys() should be preserved even
to the extent of preserving the order of the first occurrence of each
field name, then the optimization could just be moved to __len__():
def __len__(self):
return len({}.fromkeys([i.name for i in self.list]))
> But maybe it's even faster to change `FieldStorage.__len__()` to return
> the length of `self.list` directly without the need to create an
> intermediate list that's thrown away immediately.
This approach also risks breaking code that assumes it's getting the
number of unique field names (an assumption which in this case would be
justified, given the documentation of the keys() method quoted above).
Bob
--
http://mail.python.org/mailman/listinfo/python-list
Optimization of __len__() in cgi.py
I have a suggestion for speeding up the performance of code like this:
fields = cgi.FieldStorage()
if fields: ...
which, as it turns out, invokes FieldStorage.__len__(), which in turn
calls FieldStorage.keys(), which builds a list of keys by hand, taking
several minutes for large forms. This implementation of keys() reduces
the amount of time taken by several orders of magnitude:
def keys(self):
return {}.fromkeys([i.name for i in self.list]).keys()
Is there a better place for submitting suggestions like this?
Bob Kline
--
http://mail.python.org/mailman/listinfo/python-list
Re: getting data with proper encoding to the finish
On Wed Mar 16 22:21:42 CET 2005 John Machin wrote: > Apparently pyXLWriter doesn't handle Unicode at all. Although > Unicode came in with Excel 1997 (BIFF8 format file), pyXLWriter > appears to support only Excel 5(?) (BIFF5 format file). As > Serge suggested, appeal to the porter to appeal to the author > of the Perl module it's ported from; but don't hold your breath > in the meantime. I don't think it's necessary to go that far back up the chain. The Perl module from which pyXLWriter is ported supports Unicode strings (and writes BIFF8 files), at least for the current version. About a week ago I wrote to Evgeny Filatov, who did the port (I'm having a difficult time mentally reconciling the name "Evgeny" with "ASCII guy" :-) ), but haven't heard anything from him. Does anyone have a better address than the one I found in the README ([EMAIL PROTECTED])? Has anyone done any work to pick up where he left off? -- Bob Kline http://www.rksystems.com mailto:[EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
