[issue35922] robotparser crawl_delay and request_rate do not work with no matching entry

Tal Einat added the comment: Rémi, thanks for the great work writing the PR and quickly going through several iterations of reviews and revisions! -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker

[issue35922] robotparser crawl_delay and request_rate do not work with no matching entry

miss-islington added the comment: New changeset 45d6547acfb9ae1639adbe03dd14f38cd0642ca2 by Miss Islington (bot) in branch '3.7': bpo-35922: Fix RobotFileParser when robots.txt has no relevant crawl delay or request rate (GH-11791) https://github.com/python/cpython/commit/45d6547acfb9ae1639a

[issue35922] robotparser crawl_delay and request_rate do not work with no matching entry

miss-islington added the comment: New changeset 58a1a76baefc92d9e2392a5dbf65e39e44fb8f55 by Miss Islington (bot) in branch '3.8': bpo-35922: Fix RobotFileParser when robots.txt has no relevant crawl delay or request rate (GH-11791) https://github.com/python/cpython/commit/58a1a76baefc92d9e23

[issue35922] robotparser crawl_delay and request_rate do not work with no matching entry

Change by miss-islington : -- pull_requests: +13971 pull_request: https://github.com/python/cpython/pull/14122 ___ Python tracker ___ __

[issue35922] robotparser crawl_delay and request_rate do not work with no matching entry

Change by miss-islington : -- pull_requests: +13970 pull_request: https://github.com/python/cpython/pull/14121 ___ Python tracker ___ __

[issue35922] robotparser crawl_delay and request_rate do not work with no matching entry

Tal Einat added the comment: New changeset 8047e0e1c620f69cc21f9ca48b24bf2cdd5c3668 by Tal Einat (Rémi Lapeyre) in branch 'master': bpo-35922: Fix RobotFileParser when robots.txt has no relevant crawl delay or request rate (GH-11791) https://github.com/python/cpython/commit/8047e0e1c620f69cc2

[issue35922] robotparser crawl_delay and request_rate do not work with no matching entry

Guido van Rossum added the comment: Yes, this looks like a bugfix. Who wants an AttributeError? :-) -- nosy: +gvanrossum ___ Python tracker ___ ___

[issue35922] robotparser crawl_delay and request_rate do not work with no matching entry

Tal Einat added the comment: The PR is looking good, I'll likely merge it soon. I'm quite sure this should go into 3.8, but should it be backported to 3.7? This is certainly a bugfix, but still a slight change of behavior, so perhaps we should avoid changing this in 3.7? -- nosy: +

[issue35922] robotparser crawl_delay and request_rate do not work with no matching entry

Change by Tal Einat : -- versions: +Python 3.9 -Python 3.6 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: ht

[issue35922] robotparser crawl_delay and request_rate do not work with no matching entry

Rémi Lapeyre added the comment: Thanks for your report Joseph, I opened a new PR to fix this. -- nosy: +orsenthil ___ Python tracker ___ ___

[issue35922] robotparser crawl_delay and request_rate do not work with no matching entry

Change by Rémi Lapeyre : -- keywords: +patch pull_requests: +11796 stage: -> patch review ___ Python tracker ___ ___ Python-bugs-lis

[issue35922] robotparser crawl_delay and request_rate do not work with no matching entry

Change by Rémi Lapeyre : -- nosy: +remi.lapeyre ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.p

[issue35922] robotparser crawl_delay and request_rate do not work with no matching entry

New submission from Joseph Myers : RobotFileParser.crawl_delay and RobotFileParser.request_rate raise AttributeError for a robots.txt with no matching entry for the given user-agent, including no default entry, rather than returning None which would be correct according to the documentation.