ID: 35286 Updated by: [EMAIL PROTECTED] Reported By: [EMAIL PROTECTED] Status: Open Bug Type: Unknown/Other Function Operating System: n/a PHP Version: 5.0.5 -Assigned To: +Assigned To: helly New Comment:
Fixed in head Previous Comments: ------------------------------------------------------------------------ [2005-11-19 06:53:42] [EMAIL PROTECTED] better patch - this one does not increment CG(zend_lineno) Index: zend_language_scanner.l =================================================================== RCS file: /repository/ZendEngine2/zend_language_scanner.l,v retrieving revision 1.131.2.3 diff -u -r1.131.2.3 zend_language_scanner.l --- zend_language_scanner.l 15 Nov 2005 13:29:28 -0000 1.131.2.3 +++ zend_language_scanner.l 19 Nov 2005 05:52:01 -0000 @@ -1465,6 +1465,12 @@ yymore(); } +<ST_ONE_LINE_COMMENT><<EOF>> { + zendlval->value.str.val = yytext; /* no copying - intentional */ + zendlval->value.str.len = yyleng; + zendlval->type = IS_STRING; + return T_COMMENT; +} <ST_ONE_LINE_COMMENT>{NEWLINE} { zendlval->value.str.val = yytext; /* no copying - intentional */ zendlval->value.str.len = yyleng; ------------------------------------------------------------------------ [2005-11-19 04:11:52] [EMAIL PROTECTED] This patch fixes the issue with no negative side effects Index: zend_language_scanner.l =================================================================== RCS file: /repository/ZendEngine2/zend_language_scanner.l,v retrieving revision 1.131.2.3 diff -u -r1.131.2.3 zend_language_scanner.l --- zend_language_scanner.l 15 Nov 2005 13:29:28 -0000 1.131.2.3 +++ zend_language_scanner.l 19 Nov 2005 03:11:22 -0000 @@ -1465,6 +1465,7 @@ yymore(); } +<ST_ONE_LINE_COMMENT><<EOF>> | <ST_ONE_LINE_COMMENT>{NEWLINE} { zendlval->value.str.val = yytext; /* no copying - intentional */ zendlval->value.str.len = yyleng; ------------------------------------------------------------------------ [2005-11-19 02:10:19] [EMAIL PROTECTED] the original reproduce script has no closing newline - the newline does change the behavior ------------------------------------------------------------------------ [2005-11-19 02:04:20] [EMAIL PROTECTED] Doesn't drop anything here: The code: <?php $arr = token_get_all("<?php print 'foo'; # you'll see it print 'bar'; # but not this one "); foreach ($arr as $token) { if (is_array($token)) var_dump($token[1]); } ?> The output: string(6) "<?php " string(1) " " string(5) "print" string(1) " " string(5) "'foo'" string(2) " " string(16) "# you'll see it " string(5) "print" string(1) " " string(5) "'bar'" string(2) " " string(19) "# but not this one " ------------------------------------------------------------------------ [2005-11-19 01:53:39] [EMAIL PROTECTED] Description: ------------ The tokenizer extension is ignoring a final comment when tokenizing a script that does not contain a closing ?> Reproduce code: --------------- var_dump(token_get_all("<?php print 'foo'; # you'll see it print 'bar'; # but not this one")); Expected result: ---------------- array(13) { [0]=> array(2) { [0]=> int(366) [1]=> string(6) "<?php " } [1]=> array(2) { [0]=> int(266) [1]=> string(5) "print" } [2]=> array(2) { [0]=> int(369) [1]=> string(1) " " } [3]=> array(2) { [0]=> int(315) [1]=> string(5) "'foo'" } [4]=> string(1) ";" [5]=> array(2) { [0]=> int(369) [1]=> string(1) " " } [6]=> array(2) { [0]=> int(364) [1]=> string(16) "# you'll see it " } [7]=> array(2) { [0]=> int(369) [1]=> string(1) " " } [8]=> array(2) { [0]=> int(266) [1]=> string(5) "print" } [9]=> array(2) { [0]=> int(369) [1]=> string(1) " " } [10]=> array(2) { [0]=> int(315) [1]=> string(5) "'bar'" } [11]=> string(1) ";" [12]=> array(2) { [0]=> int(369) [1]=> string(1) " " } [13]=> array(2) { [0]=> int(364) [1]=> string(18) "# but not this one" } Actual result: -------------- array(13) { [0]=> array(2) { [0]=> int(366) [1]=> string(6) "<?php " } [1]=> array(2) { [0]=> int(266) [1]=> string(5) "print" } [2]=> array(2) { [0]=> int(369) [1]=> string(1) " " } [3]=> array(2) { [0]=> int(315) [1]=> string(5) "'foo'" } [4]=> string(1) ";" [5]=> array(2) { [0]=> int(369) [1]=> string(1) " " } [6]=> array(2) { [0]=> int(364) [1]=> string(16) "# you'll see it " } [7]=> array(2) { [0]=> int(369) [1]=> string(1) " " } [8]=> array(2) { [0]=> int(266) [1]=> string(5) "print" } [9]=> array(2) { [0]=> int(369) [1]=> string(1) " " } [10]=> array(2) { [0]=> int(315) [1]=> string(5) "'bar'" } [11]=> string(1) ";" [12]=> array(2) { [0]=> int(369) [1]=> string(1) " " } } ------------------------------------------------------------------------ -- Edit this bug report at http://bugs.php.net/?id=35286&edit=1