Re: [PHP] mysql_real_escape_string() question
On Fri, 29 Sep 2006 11:41:34 -0500 (CDT), Richard Lynch wrote: Consider that the user could provide *ANY* string, of any size, of any composition, for their Subject Maybe they POST a worm in Subject, and it has no newlines, but still manages to propogate through Outlook. Or maybe it's just a nice subject in Japanese. I know nada about Unicode, uuencode, and all that crap. Or, maybe, it's not even a VALID subject for SMTP, for whatever the arcana rules of SMTP-ness are. My contention is that the lowly application developer (me) should not need a degree in i18n nor SMTP just to pass on a valid SMTP subject in an email. I've been meaning to look into this, so I might as well do it now. The obvious assumption would be that the mail() function would: a) escape all its arguments to make them suitable for use in an email This should, at a minimum, run something like addslashes() to escape newlines. Ideally, it should also encode the arguments as quoted-printable if necessary, but the mail() function would then need to know what character encoding the strings are in. So, the burden of escaping appears to be on the user, rather than on the mail() function. What then is it that has to be done? The 'Subject:' header is fairly simple. It contents is '*text' in RFC 822 terms, where problematic 'text' parts can be replaced by 'encoded-word' in RFC 2047 terms. In other words, it can be QP-encoded directly. The 'To:' header is more problematic. Is it an (RFC 822) 'address', 'mailbox', 'addr-spec' or something else? The escape mechanism would be different (I think), and the PHP documentation doesn't provide any information on this. b) escape all parameters sent to the sendmail program through the shell. It would be interesting to run a sendmail dummy that dumps its arguments to a file to see what's really going on. I don't seem to be able to test this on my old Windows clunker at home. For *any* data that PHP has to pass back and forth in its glue there are potentials for the kind of problems we've seen with spam, site defacing, viruses, etc. What I'm suggesting is that in addition to mysql_escape[_real]_string, maybe there needs to be more escape string functions. So with all these potential issues, I'm wondering if there isn't a more systemic approach to this. Wouldn't it be nice if strings were associated with a (hidden) character encoding field, so that mail() and other functions could just do their thing and not bother us users with the detaily bits? Wait a minute... Isn't that what multi-byte string are for? The mb_send_mail() function looks like a strong contender here. And no, I haven't tried any funky mb_ stuff so far... Plus, for the functions that we *DO* have, a grid of from and to and the appropriate converter function seems like it would be a Good Idea. It's all to easy to find a problem like ' where addslashes seems like the right answer but, in reality, what I do not know is that ~ is also a special character to the [mumble] extension/protocol/whatever and I'm using the wrong escape function. There are 2 reasons why I'm not using the right escape function. #1. The right one just plain doesn't exist. #2. The docs, wonderful as they are, don't really lay out something as fundamental as the right escape function for situation X, because you need a degree in CS just to know that X is really a Y so the right function is Z. Alas, there are many parts of PHP that is seriously underspecified. --nfe -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] mysql_real_escape_string() question
On Fri, September 29, 2006 8:34 pm, Chris Shiflett wrote: I'm looking for a guide, a chart, a grid, an organized systemic documentation of what data should be escaped how as it travels through the glue that is PHP... That's a great idea. Want to write it? :-) I'd be happy to help. Okay. We have a team of 2 so far... :-) Is there any nifty excel/grid-like UI on-line where we can just start shoving in the from and to axis and filling in the obvious ones? Any good Wikis support that? Unless we wanna pass ASCII art back-n-forth... | MySQL | -+-+ raw text |mysql_real_escape_string | |mysql_escape_string | |Use real_ for charset- | |specific escaping| -+-+ :-) -- Some people have a gift link here. Know what I want? I want you to buy a CD from some starving artist. http://cdbaby.com/browse/from/lynch Yeah, I get a buck. So? -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] mysql_real_escape_string() question
On Thu, September 28, 2006 2:06 pm, tedd wrote: I realize that you are not asking for an answer, but for a guide -- however -- isn't the real problem here simply one of injection? Just stop the user from injecting stuff in the subject and that would fix it right? Or, am I underestimating the problem? Underestimating. Stopping header injection is only one step of a potential world of problems. Consider that the user could provide *ANY* string, of any size, of any composition, for their Subject Maybe they POST a worm in Subject, and it has no newlines, but still manages to propogate through Outlook. Or maybe it's just a nice subject in Japanese. I know nada about Unicode, uuencode, and all that crap. Or, maybe, it's not even a VALID subject for SMTP, for whatever the arcana rules of SMTP-ness are. My contention is that the lowly application developer (me) should not need a degree in i18n nor SMTP just to pass on a valid SMTP subject in an email. For *any* data that PHP has to pass back and forth in its glue there are potentials for the kind of problems we've seen with spam, site defacing, viruses, etc. What I'm suggesting is that in addition to mysql_escape[_real]_string, maybe there needs to be more escape string functions. I believe JSON is one such in the pipeline, for Javascript string escaping? Or am I mis-remembering? It just seems to me that if we manage to lock down email and MySQL, the Bad Guys are just gonna turn to the next biggest (most-used) extension and look for exploits there. So with all these potential issues, I'm wondering if there isn't a more systemic approach to this. Plus, for the functions that we *DO* have, a grid of from and to and the appropriate converter function seems like it would be a Good Idea. It's all to easy to find a problem like ' where addslashes seems like the right answer but, in reality, what I do not know is that ~ is also a special character to the [mumble] extension/protocol/whatever and I'm using the wrong escape function. There are 2 reasons why I'm not using the right escape function. #1. The right one just plain doesn't exist. #2. The docs, wonderful as they are, don't really lay out something as fundamental as the right escape function for situation X, because you need a degree in CS just to know that X is really a Y so the right function is Z. -- Like Music? http://l-i-e.com/artists.htm -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] mysql_real_escape_string() question
At 11:41 AM -0500 9/29/06, Richard Lynch wrote: On Thu, September 28, 2006 2:06 pm, tedd wrote: I realize that you are not asking for an answer, but for a guide -- however -- isn't the real problem here simply one of injection? Just stop the user from injecting stuff in the subject and that would fix it right? Or, am I underestimating the problem? Underestimating. Stopping header injection is only one step of a potential world of problems. Consider that the user could provide *ANY* string, of any size, of any composition, for their Subject Maybe they POST a worm in Subject, and it has no newlines, but still manages to propogate through Outlook. Then limiting the number of characters allowed would provide a degree of security. Worms take some amount of space and reducing that allotment makes it harder to create one. I know nada about Unicode, uuencode, and all that crap. Unicode is nothing more that an expanded ASCII (I'll get flamed for that statement). But, Unicode is simply extending the 7-bit ASCII problem to 8-bit so that more code-points (characters) can be added for global communications. If you understand ASCII, then you have the basics for Unicode. So with all these potential issues, I'm wondering if there isn't a more systemic approach to this. Identify the problem. One of the axioms in security programming is something like Don't program for things that might be, but rather for things that are known. I think Shiflett said something to that affect in his book. If you can show me the minimum size for a worm, then setting the character limit in a subject line would protect from that -- but -- are worms, or other evil code, transmitted by subject lines? #2. The docs, wonderful as they are, don't really lay out something as fundamental as the right escape function for situation X, because you need a degree in CS just to know that X is really a Y so the right function is Z. Degrees are overrated -- I have plenty of them and I'm still asking questions. Just give me someone who knows WTF their doing, and that's fine with me. IMO, technology is moving too fast for colleges to keep up. It's the people on the bleeding edge that are innovation, not the ones in the classroom. tedd -- --- http://sperling.com http://ancientstones.com http://earthstones.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] mysql_real_escape_string() question
Richard Lynch wrote: Though I confess, I'm sometimes at a loss how to properly escape certain data for certain situations... Here's an example: Take the Subject of an email. Sure, I've sanitized it to be sure there are no newlines for header injection. But now how do I properly escape it to be sure it's a kosher email subject? Where's the PHP function smtp_escape()? Some contexts don't require/support escaping. It's a bummer in the sense that it places more responsibility on your filtering, but that's about it. In several cases, ctype_print() can be used as a nice defense in depth mechanism to make sure there are no newlines or carriage returns (or anything else that isn't printable). I'm looking for a guide, a chart, a grid, an organized systemic documentation of what data should be escaped how as it travels through the glue that is PHP... That's a great idea. Want to write it? :-) I'd be happy to help. Chris -- Chris Shiflett http://shiflett.org/ -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] mysql_real_escape_string() question
On Thu, September 28, 2006 10:06 am, tedd wrote: In one of my snip-its, namely: http://xn--ovg.com/pdf I was generating a pdf document after the user filled in a form. I had been cleaning the user input by using -- $name = mysql_real_escape_string($name); -- even though I wasn't using MySQL (the code was a cut-paste from some other code I had). But you *WERE* using MySQL! However, everything worked! But, a couple of days ago it suddenly stopped working. Now, I get the following error: Warning: mysql_real_escape_string(): Access denied for user 'nobody'@'localhost' (using password: NO) in ... One of two things happened. Some auto-connect script is no longer running, or the 'nobody' user in MySQL got nuked. Cuz you used to be connected to MySQL, and it was using MySQL database information to do the escaping. When I comment-out the offending statement, it runs. I replaced the statement, but wonder what happened -- when did using mysql_real_escape_string() require a password? mysql_real_escape_string talks back to MySQL to ask it what character encoding you are using, so it knows how to correctly escape multi-byte/unicode/funky characters for MySQL usage. Take out the _real bit, and it's doing a fake version that ignores multibyte/unicode/funky characters. So, short term, just delete '_real' from your function call, and it will act exactly like before, except with the caveat that any unicode/multibyte/funky characters may not be escaped the same way as they were. What's up with that? Any ideas as to what happened? One also has to ask WHY you would use MySQL's escaping for data that's not going into MySQL. That's almost certainly wrong Though I confess, I'm sometimes at a loss how to properly escape certain data for certain situations... Here's an example: Take the Subject of an email. Sure, I've sanitized it to be sure there are no newlines for header injection. But now how do I properly escape it to be sure it's a kosher email subject? Where's the PHP function smtp_escape()? I'm just passing it on from one user to another. I don't want to munge it, nor make any assumptions about its format. It's just data to me. But to SMTP, there are bound to be all kinds of rules about it that I have no desire, much less time, to research, code, and test in as thorough a fashion as I should to be Professional. And every developer who sends an email with PHP needs this, right? So of the myriad PHP functions available, which one is the right one to escape an email Subject. I'm *NOT* asking for an answer to this specific question about email Subjects! I'm looking for a guide, a chart, a grid, an organized systemic documentation of what data should be escaped how as it travels through the glue that is PHP... -- Like Music? http://l-i-e.com/artists.htm -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] mysql_real_escape_string() question
At 11:06 AM -0500 9/28/06, Richard Lynch wrote: What's up with that? Any ideas as to what happened? One also has to ask WHY you would use MySQL's escaping for data that's not going into MySQL. That's almost certainly wrong Richard: Yes, according to: http://us3.php.net/mysql_real_escape_string The use of the statement looks for a MySQL connection and if there isn't one, it tries to connect and thus producing the error I got when it failed. The answer to WHY I did it is that I cut/pasted the code from another script. I was only concerned with cleaning the input and not with using MySQL -- it was just one statement in several to scrub various inputs. However, I would have thought that if the code was going to fail, it would have done so immediately. But it instead, it ran for weeks without failure until just a few days ago. I checked the script almost daily to see if it ran. While it seems very unlikely to me, I must have also tested code that opened a MySQL connection before testing that script. Strange circumstances. Thanks for the clarification. tedd -- --- http://sperling.com http://ancientstones.com http://earthstones.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP] mysql_real_escape_string() question
At 11:06 AM -0500 9/28/06, Richard Lynch wrote: Though I confess, I'm sometimes at a loss how to properly escape certain data for certain situations... Here's an example: Take the Subject of an email. Sure, I've sanitized it to be sure there are no newlines for header injection. But now how do I properly escape it to be sure it's a kosher email subject? Where's the PHP function smtp_escape()? I'm just passing it on from one user to another. I don't want to munge it, nor make any assumptions about its format. It's just data to me. But to SMTP, there are bound to be all kinds of rules about it that I have no desire, much less time, to research, code, and test in as thorough a fashion as I should to be Professional. And every developer who sends an email with PHP needs this, right? So of the myriad PHP functions available, which one is the right one to escape an email Subject. I'm *NOT* asking for an answer to this specific question about email Subjects! I'm looking for a guide, a chart, a grid, an organized systemic documentation of what data should be escaped how as it travels through the glue that is PHP... Richard: I realize that you are not asking for an answer, but for a guide -- however -- isn't the real problem here simply one of injection? Just stop the user from injecting stuff in the subject and that would fix it right? Or, am I underestimating the problem? tedd -- --- http://sperling.com http://ancientstones.com http://earthstones.com -- PHP General Mailing List (http://www.php.net/) To unsubscribe, visit: http://www.php.net/unsub.php
