[issue28971] nntplib is broken when responses are longer than _MAXLINE

Matej Cepl added the comment: I understood, and I was saying that if you kick nntplib out of the standard library, than I will just embed it into my program and I won't bother to maintain it publicly. -- ___ Python tracker

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Christian Heimes added the comment: I mean fork, as in maintain your own fork on PyPI and outside of Python core. -- ___ Python tracker ___ ___

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Matej Cepl added the comment: If that was the price of keeping nntplib inside of the Python standard library, yes. -- ___ Python tracker ___ _

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Christian Heimes added the comment: Matej, would you be interested to fork nntplib and take over maintenance and responsibility? -- ___ Python tracker ___ ___

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Christian Heimes added the comment: No, the module is still supported until EOL of Python 3.9. I expect 3.9 to go into security fix-only mode in 2024. -- versions: +Python 3.8, Python 3.9 -Python 3.5, Python 3.6 ___ Python tracker

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Matej Cepl added the comment: @mbussonn That's exactly the point: I completely disagree with removal of nntplib from the standard library, so I went through all bugs here related to it. -- ___ Python tracker _

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Matthias Bussonnier added the comment: Would that be anyway closed by pep 594 (https://www.python.org/dev/peps/pep-0594/#nntplib) which suggest the removal nntp ? -- nosy: +mbussonn ___ Python tracker

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Xavier de Gaye added the comment: What do you mean ? What is "this" ? -- ___ Python tracker ___ ___ Python-bugs-list mailing list U

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Matej Cepl added the comment: Could @xdegaye make a PR for this? -- nosy: +mcepl ___ Python tracker ___ ___ Python-bugs-list mailin

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Changes by Martin Panter : Added file: http://bugs.python.org/file46020/max_over_line.patch ___ Python tracker ___ ___ Python-bugs-list mailin

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Changes by Martin Panter : Removed file: http://bugs.python.org/file46019/max_over_line.patch ___ Python tracker ___ ___ Python-bugs-list mail

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Martin Panter added the comment: Max_over_line.patch is my attempt: Keep the original _MAXLINES = 2048 code, but override it with _MAX_OVER_LINE = 64000 when reading OVER response lines. I also added a test case. -- Added file: http://bugs.python.org/file46019/max_over_line.patch

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Martin Panter added the comment: I will try to come up with something in a few days -- ___ Python tracker ___ ___ Python-bugs-list mai

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Xavier de Gaye added the comment: Martin in response to your last review, I still hold to my opinion stated in msg283294 but this should not prevent this high priority issue to progress. Can you propose another patch. -- ___ Python tracker

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Martin Panter added the comment: The first offending message I found is number 183465: >>> s.group("comp.lang.python") ('211 4329 179178 183507 comp.lang.python', 4329, 179178, 183507, 'comp.lang.python') >>> s._putcmd("OVER 183465") >>> s._getresp() '224 Overview information for 183465 follows

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Serhiy Storchaka added the comment: On 32-bit platform the size of empty bytes object is 17 bytes not counting padding and GC links. Plus 4 bytes for a pointer in a list. On 64 bit platform numbers are about twice larger. Therefore add at least 20-40 bytes per line. Is not 100 MiB too large fo

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Xavier de Gaye added the comment: > If sender sends a lot of empty lines and file is not None, LF or CRLF is > stripped from lines Oh, I missed that. Maybe give a weight of 4 to 8 bytes or even more to a line, this value being added to the bytes count whether the line is empty or not. Do you h

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Xavier de Gaye added the comment: I responded to your last review Serhiy, but the psf mail system reports a delivery failure, so you may have missed the notification as well as the other reviewers. -- ___ Python tracker

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Serhiy Storchaka added the comment: If sender sends a lot of empty lines and file is not None, LF or CRLF is stripped from lines, and len(line) is 0. Every empty line increases the size of the lines list by 4 or 8 bytes. Since count is not changed, the loop is not bounded. Every LF byte sent b

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Xavier de Gaye added the comment: This patch defines a _MAXBYTES limit of 100 Mb. The limit is checked upon reading one line and also upon reading the multi-lines of the response to a user command. -- Added file: http://bugs.python.org/file45939/nntplib_maxbytes.patch

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Roundup Robot added the comment: New changeset 1386795d266e by Xavier de Gaye in branch '3.5': Issue #28971: Temporarily skip test_over until a permanent solution is found https://hg.python.org/cpython/rev/1386795d266e New changeset a33047e08711 by Xavier de Gaye in branch '3.6': Issue #28971: M

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Xavier de Gaye added the comment: > I'd be in favor of temporarily skipping the affected tests with a message > that points back to here until we have a permanent solution. I am working on it. -- ___ Python tracker

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Christian Heimes added the comment: A larger limit is totally fine. The check protects against DoS with hundreds of MB. I'm currently travelling and won't be available much until next week. Am 16. Dezember 2016 19:37:03 MEZ, schrieb Xiang Zhang : > >Xiang Zhang added the comment: > >Xavier's p

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Xiang Zhang added the comment: Xavier's plan sounds good. We could increase the line length limitation to 64K and add another limitation of the maximum lines a multi-line block could contain. Any limitation is violated the connection is refused. This situation seems quite similar to http.clien

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Zachary Ware added the comment: This is causing all buildbots to fail again; I'd be in favor of temporarily skipping the affected tests with a message that points back to here until we have a permanent solution. -- keywords: +buildbot priority: normal -> high _

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Serhiy Storchaka added the comment: The limit to readline() was added to prevent consuming an excessive amount of memory. But this doesn't help in case of long multiline responses, since all lines are accumulated in a list in memory. A malicious server could cause a client consuming an excessi

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Xavier de Gaye added the comment: It seems that the comment placed above the definition of _MAXLINE in the nntplib module is not correct: "RFC 3977 limits NNTP line length to 512 characters, including CRLF. We have selected 2048 just to be on the safe side." The 512 characters limit in RFC

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Serhiy Storchaka added the comment: It seems to me there are two issues: 1) The limit of line length is not large enough. 2) After raising an error on too long line the NNTP object is left in broken state. The first issue can be solved by increasing the default limit or by patching the nntp m

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Xavier de Gaye added the comment: The patch: 1) Increases _MAXLINE to 4096. 2) Reverts issue 16040 and that is not correct, please ignore that part. The changes made in issue 16040 limit the amount of data read by readline() and does not close the nntp session when the server sends a message wh

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Changes by Serhiy Storchaka : -- nosy: +christian.heimes, serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailing list Un

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Martin Panter added the comment: Just a quick note for the moment: It may not be wise to drop the limit to readline(). That is undoing Issue 16040. Maybe we need a better test if this change doesn't fail the test suite. -- ___ Python tracker

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Xavier de Gaye added the comment: > I found that there were three other mails raising this same > NNTPDataError('line too long') in the 40 last mails. Cannot reproduce it. These three exceptions must have been NNTPProtocolError instead, caused by the initial NNTPDataError. -- __

[issue28971] nntplib is broken when responses are longer than _MAXLINE

Xavier de Gaye added the comment: When the server sends a line longer than _MAXLINE, nntplib reads only _MAXLINE + 1 bytes leaving the remaining bytes left to be processed by the next command that will interpret those bytes as a protocol error. Hence the failing tests that follow the first NNT