[issue39187] urllib.robotparser does not respect the longest match for the rule

matele secretaire added the comment: Thank you -- ___ Python tracker <https://bugs.python.org/issue39187> ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue39187] urllib.robotparser does not respect the longest match for the rule

matele secretaire added the comment: I can't find a documentation about it, but all of the robots.txt checkers I find behave like this. You can test on this site: https://www.st-info.fr/robots.txt, I believe that this is how it's implemented now in most parsers ? -- nosy