Glenn Linderman <v+pyt...@g.nevcal.com> added the comment: [offlist] Paul, thanks for the enumeration and response. Some folks have more experience, but the rest of us need to learn. Having the proposal in the ticket, with an explanation of its deficiencies is not all bad, however, others can learn, perhaps. On the other hand, I'm willing to learn more, if you are willing to address my concerns below.
I had read the whole thread and issue, but it still seemed like a leap of faith to conclude that the only, or at least best, solution is changing the hash. Yet, changing the hash still doesn't seem like a sufficient solution, due to long-lived processes. On 1/7/2012 6:40 PM, Paul McMillan wrote: > Paul McMillan<p...@mcmillan.ws> added the comment: > >> Alex, I agree the issue has to do with the origin of the data, but the >> modules listed are the ones that deal with the data supplied by this >> particular attack. > They deal directly with the data. Do any of them pass the data > further, or does the data stop with them? For web forms and requests, which is the claimed vulnerability, I would expect that most of them do not pass the data further, without validation or selection, and it is unlikely that the form is actually expecting data with colliding strings, so it seems very unlikely that they would be passed on. At least that is how I code my web apps: just select the data I expect from my form. At present I do not reject data I do not expect, but I'll have to consider either using SafeDict (which I can start using ASAP, not waiting for a new release of Perl to be installed on my Web Server (currently running Perl 2.4), or rejecting data I do not expect prior to putting it in a dict. That might require tweaking urllib.parse a bit, or cgi, or both. > A short and very incomplete > list of vulnerable standard lib modules includes: every single parsing > library (json, xml, html, plus all the third party libraries that do > that), all of numpy (because it processes data which probably came > from a user [yes, integers can trigger the vulnerability]), difflib, > the math module, most database adaptors, anything that parses metadata > (including commonly used third party libs like PIL), the tarfile lib > along with other compressed format handlers, the csv module, > robotparser, plistlib, argparse, pretty much everything under the > heading of "18. Internet Data Handling" (email, mailbox, mimetypes, > etc.), "19. Structured Markup Processing Tools", "20. Internet > Protocols and Support", "21. Multimedia Services", "22. > Internationalization", TKinter, and all the os calls that handle > filenames. The list is impossibly large, even if we completely ignore > user code. This MUST be fixed at a language level. > > I challenge you to find me 15 standard lib components that are certain > to never handle user-controlled input. I do appreciate your enumeration, but I'll decline the challenge. While all of them can be interesting exploits of naïve applications (written by programmers who may be quite experienced in some things, but can naïvely overlook other things), most of them probably do not apply to the documented vulnerability. Many I had thought of, but rejected for this context; some I had not. So while there are many possible situations where happily stuffing things into a dict may be an easy solution, there are many possible cases where it should be prechecked on the way in. And there is another restriction: if the user-controlled input enters a user-run program, it is unlikely to be attacked in the same manner than web servers are attacked. A user, for example, is unlikely to contrive colliding file names for the purpose of making his file listing program run slow. So it is really system services and web services that need to be particularly careful. Randomizing the hash seed might reduce the problem from any system/web services to only long-running system/web services, but doesn't really solve the complete problem, as far as I can tell... only proper care in writing the application (and the stdlib code) will solve the complete problem. Sadly, beefing up the stdlib code will probably reduce performance for things that will not be exploited to be careful enough in the cases that could be exploited. >> Note that changing the hash algorithm for a persistent process, even though >> each process may have a different seed or randomized source, allows attacks >> for the life of that process, if an attack vector can be created during its >> lifetime. This is not a problem for systems where each request is handled by >> a different process, but is a problem for systems where processes are >> long-running and handle many requests. > This point has been made many times now. I urge you to read the entire > thread on the mailing list. Your implementation is impractical because > your "safe" implementation completely ignores all hash caching (each > entry must be re-hashed for that dict). Your implementation is still > vulnerable in exactly the way you mentioned if you ever have any kind > of long-lived dict in your program thread. I had read the entire thread, or at least the part that made it to my inbox. It is a very interesting discussion. I have to admit I was aware that hashes got cached, but the fact that strings keep getting recreated causes the cache to be sidestepped did slip past me. On the other hand, the only strings that will get recreated in a CGI situation are a few FORM parameters per request... the expected ones. And only if the application keeps referring to the CGI dict to find them, instead of binding the values locally. For JSON data structures, it seems more likely that they would be iterated, rather than point lookups being done. But it does depend on the application whether missing the hash caching would be severe. A long-lived dict is only vulnerable if it continues to accept updates during its lifetime. For web requests, the vulnerability of concern here, that is generally not the case... web requests are generally discarded at completion. >> You have entered the class of people that claim lots of vulnerabilities, >> without enumeration. > I have enumerated. Stop making this argument. I appreciate the enumeration, even though you were not the person to whom that statement was addressed. And, I see that a randomized hash per process helps reduce a large class of potential vulnerabilities for short-lived processes, but I still have concerns that simply randomizing the hash per process adds startup overhead, gratuitous incompatibility, and doesn't fully solve the problem, as collisions are still possible, and attackers are clever. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue13703> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com