#29761 [Com]: token_get_all()

[guth at fiifo dot u-psud dot fr]

 ID:               29761
 Comment by:       guth at fiifo dot u-psud dot fr
 Reported By:      steemann at globalpark dot de
 Status:           Open
 Bug Type:         Scripting Engine problem
 Operating System: Linux 2.4.22
 PHP Version:      4.3.8
 New Comment:

Same problem with PHP 5.0.*, with any PHP script.


Previous Comments:
------------------------------------------------------------------------

[2004-08-19 17:48:22] steemann at globalpark dot de

Description:
------------
Don't know if it's a tokenizer issue or something else: 
 
I just wanted to check out the token_get_all() function. 
I tried the test code from the manual page (see code 
below). I am using token_get_all() with a constant input. 
 
Unfortunately, on my box, the result for the test code is 
varying, i. e. tokening a constant string has two 
different results. I found this out it by simple hitting 
reload in the browser multiple times. I've put both 
results I've gotten in the "Actual result" textarea. The 
possibility for each result seems to be about 50%. 
 
 
Environment: 
Configure Command =>  './configure' 
'--with-apxs=/opt/pdp/apache/bin/apxs' 
'--prefix=/opt/pdp/php' '--with-mysql=/opt/pdp/mysql/' 
'--with-config-file-path=/opt/pdp/' 
'--with-iconv=/opt/pdp/iconv/' 
'--with-imagick=/opt/pdp/ImageMagick/' '--with-zlib' 
'--with-gd' '--with-freetype-dir=/usr' 
'--enable-gd-native-ttf' '--enable-gd-imgstrttf' 
'--with-jpeg-dir=/usr' '--with-png-dir=/usr' '--with-dom' 
'--with-dom-xslt' '--with-dom-exslt' '--enable-sysvsem' 
'--enable-sysvshm' '--disable-debug' '--enable-sockets' 
PHP API => 20020918 
PHP Extension => 20020429 
Zend Extension => 20021010 
Debug Build => no 
Thread Safety => disabled 
 
 
Maybe my box is f**ked up?? 

Reproduce code:
---------------
$data="<?php ob_start(); ?>";
foreach(token_get_all($data) as $c)
{
  if(is_array($c))
  {
    print(token_name($c[0]) . ": '".htmlentities($c[1])."'\n");
  }
  else
  {
    print("$c\n");
  }
}


Expected result:
----------------
T_OPEN_TAG: '&lt;?php ' 
T_STRING: 'ob_start' 
( 
) 
; 
T_WHITESPACE: ' ' 
T_CLOSE_TAG: '?&gt;' 

Actual result:
--------------
In about 50% I get: 
---------------------------- 
< 
? 
T_STRING: 'php' 
T_WHITESPACE: ' ' 
T_STRING: 'ob_start' 
( 
) 
; 
T_WHITESPACE: ' ' 
T_CLOSE_TAG: '?&gt;' 
 
 
 
 
the other 50% I get: 
---------------------------- 
T_OPEN_TAG: '&lt;?php ' 
T_STRING: 'ob_start' 
( 
) 
; 
T_WHITESPACE: ' ' 
T_CLOSE_TAG: '?&gt;' 
 


------------------------------------------------------------------------


-- 
Edit this bug report at http://bugs.php.net/?id=29761&edit=1