Re: [PHP-DEV] Re: [PHP-QA] Parallel run-tests

[zoe slattery]

Nuno - just a PS to the last note. It is (mainly) the task allocation 
across processors which means that running tests in parallel on a 4-way 
machine is not 4 times as fast as running them in sequence.


Here are some results from a run on my 2-way Mac - 
http://static.inky.ws/image/3257/image.jpg. The blocks of colour are 
just representations of the time it takes a group to run - and Open 
office is allocating the colors randomly so they don't have any 
significance. I've annotated the chart to show which groups are taking a 
long time


The net is that P0 runs its half of the tasks and then just hangs about 
waiting for P1 to finish :-/.


I have added a 'debug' flag to the code which will print information 
about how tasks are allocated if anyone wants to try on a 4 or 8 way. 
Given that there are not many groups that take a long time to run the 
simplest thing seems to be to map these to specific processors - that's 
easy enough and requires no difficult code. There are more elegant 
solutions of course.


Zoe

Hi Nuno


Hi,

Here you have a dump of a run of PHP_HEAD in the gcov machine (almost 
13k tests) without valgrind:

http://gcov.php.net/~nlopess/dump_PHP_HEAD_z4.txt

It was run with -z 4.  However, the reported CPU usage is only 213% 
(instead of ~400%).
I spent a lot of time this weekend trying to figure this out, not 
completely successfully. The main problem is the way that groups of 
tests are allocated between processors.  The allocation is done 
randomly from a list of groups (directories of tests), so for example, 
on my two way machine, if  the "array" and the "file" tests get 
allocated to the same processor then the whole run can take as long as 
long with two processors as with one. The second processor just 
finishes its task list and hangs around waiting for the first one to 
complete.


Ideally all the processors would do something like pop tasks from a 
list, so they'd all be busy all the time. However, I think this might 
be difficult (for me) to implement correctly (?). A simpler solution 
might be to have an allocation map either hard coded or read from a 
configuration file - the student who implemented the parallel code did 
allow for this. Having observed the issue with 'array' and 'file' I 
was going to try this next. Any thoughts on the right way to do this 
would be helpful.


As you can see in the dump, there are a few BORK'ed tests.
BORK'ed tests don't worry me too much. This version BORKs on tests 
that have 'unrecognised sections', the current implementation just 
ignores them so there are a few tests (for example those which contain 
SERVER test sections) which run-tests just silently does nothing with, 
I don't really like that behaviour and think that BORKs are an 
improvement.


I'll have a look through your results and make sure that it's nothing 
more than that.


BTW, sometimes I get this error when launching run-tests.php 
(non-deterministic):
Fatal error: Call to undefined method rtGroupResults::run() in 
src/taskScheduler/rtTaskSchedulerFile.php on line 225
That is probably a bug - sorry. I added that class at the weekend, 
I'll look into it.


Nuno


- Original Message -
Hi Nuno - did you ever get a chance to look at the parallel version 
of run-tests? I had some free time to work on it last week so 
REDIRECT is now implemented, I have tested it using ext/pdo_mysql 
and ext/pdo_sqlite.


I'm testing using about 8000 phpt tests and getting the same results 
as run-tests. There is still work to do on performance but I think 
it might at this stage be worth trying a bigger sample.


Zoe

Alright, thanks for the reply!
I'll try to have a look at the code and give it a try next weekend.

Nuno


-Original Message- From: zoe slattery Sent: Monday, May 21, 
2012 8:26 AM

Subject: Re: [PHP-QA] Parallel run-tests
On 21/05/2012 06:45, Nuno Lopes wrote:

Hi Zoe,

Thanks for undertaking this project!

I just have a few concerns about this:
- The speedup seems a bit low to me. Maybe with a higher number of 
extensions enabled it will improve?..
I don't think it will improve with a higher number of extensions. 
It _may_ improve as a result of someone doing some work on the 
tests are scheduled - but I wouldn't count on it. Looking at 
improving performance is important and I wanted to get some 
confirmation that it is faster, I think we have that level of 
confirmation now. However, if I'm the only person working on this, 
further performance work will come after complete implementation + 
debugging.
- Is there any developer documentation available?  If I wanted to 
do a change or a bug fix today, how could I do it?
Lots - https://wiki.php.net/qa/runtests. I have just updated the 
"Development" section which has instructions on how to build and 
test the code. The 'Documentation' section

[PHP-DEV] Re: [PHP-QA] Parallel run-tests

[zoe slattery]

Hi Nuno


Hi,

Here you have a dump of a run of PHP_HEAD in the gcov machine (almost 
13k tests) without valgrind:

http://gcov.php.net/~nlopess/dump_PHP_HEAD_z4.txt

It was run with -z 4.  However, the reported CPU usage is only 213% 
(instead of ~400%).
I spent a lot of time this weekend trying to figure this out, not 
completely successfully. The main problem is the way that groups of 
tests are allocated between processors.  The allocation is done randomly 
from a list of groups (directories of tests), so for example, on my two 
way machine, if  the "array" and the "file" tests get allocated to the 
same processor then the whole run can take as long as long with two 
processors as with one. The second processor just finishes its task list 
and hangs around waiting for the first one to complete.


Ideally all the processors would do something like pop tasks from a 
list, so they'd all be busy all the time. However, I think this might be 
difficult (for me) to implement correctly (?). A simpler solution might 
be to have an allocation map either hard coded or read from a 
configuration file - the student who implemented the parallel code did 
allow for this. Having observed the issue with 'array' and 'file' I was 
going to try this next. Any thoughts on the right way to do this would 
be helpful.


As you can see in the dump, there are a few BORK'ed tests.
BORK'ed tests don't worry me too much. This version BORKs on tests that 
have 'unrecognised sections', the current implementation just ignores 
them so there are a few tests (for example those which contain SERVER 
test sections) which run-tests just silently does nothing with, I don't 
really like that behaviour and think that BORKs are an improvement.


I'll have a look through your results and make sure that it's nothing 
more than that.


BTW, sometimes I get this error when launching run-tests.php 
(non-deterministic):
Fatal error: Call to undefined method rtGroupResults::run() in 
src/taskScheduler/rtTaskSchedulerFile.php on line 225
That is probably a bug - sorry. I added that class at the weekend, I'll 
look into it.


Nuno


- Original Message -
Hi Nuno - did you ever get a chance to look at the parallel version 
of run-tests? I had some free time to work on it last week so 
REDIRECT is now implemented, I have tested it using ext/pdo_mysql and 
ext/pdo_sqlite.


I'm testing using about 8000 phpt tests and getting the same results 
as run-tests. There is still work to do on performance but I think it 
might at this stage be worth trying a bigger sample.


Zoe

Alright, thanks for the reply!
I'll try to have a look at the code and give it a try next weekend.

Nuno


-Original Message- From: zoe slattery Sent: Monday, May 21, 
2012 8:26 AM

Subject: Re: [PHP-QA] Parallel run-tests
On 21/05/2012 06:45, Nuno Lopes wrote:

Hi Zoe,

Thanks for undertaking this project!

I just have a few concerns about this:
- The speedup seems a bit low to me. Maybe with a higher number of 
extensions enabled it will improve?..
I don't think it will improve with a higher number of extensions. It 
_may_ improve as a result of someone doing some work on the tests 
are scheduled - but I wouldn't count on it. Looking at improving 
performance is important and I wanted to get some confirmation that 
it is faster, I think we have that level of confirmation now. 
However, if I'm the only person working on this, further performance 
work will  come after complete implementation + debugging.
- Is there any developer documentation available?  If I wanted to 
do a change or a bug fix today, how could I do it?
Lots - https://wiki.php.net/qa/runtests. I have just updated the 
"Development" section which has instructions on how to build and 
test the code. The 'Documentation' section has not been updated 
recently but I think it is still valid.


- Can it be packaged as a single drop-in file replacement for 
run-tests.php? The deployment is very important to me.  IMHO, the 
optimal solution would be to have a drop-in replacement for the 
current script, so that most developers wouldn't even notice the 
difference.
Yes - totally agree. I think I experimented with packaging it as a 
.phar (so it would be called run-tests.phar, not run-tests.php), but 
it's so long ago that I can't remember. I have added it to the to-do 
list (https://wiki.php.net/qa/runtests/todos).
- From previous emails exchanged in this thread, it seems that this 
new version requires a few extensions to run (gzip, soap??). This 
is not acceptable. The script must be able to run with 
--disable-all. Of course in that case the parallel version will fail.

No, it doesn't. That was a stupid bug and I fixed it last night :-)

Zoe


Nuno


-Original Message- From: zoe slattery
Sent: Thursday, May 17, 2012 1:00 PM
To: internals@lists.php.

[PHP-DEV] Re: [PHP-QA] Parallel run-tests

[zoe slattery]

Hi Nuno - did you ever get a chance to look at the parallel version of 
run-tests? I had some free time to work on it last week so REDIRECT is 
now implemented, I have tested it using ext/pdo_mysql and ext/pdo_sqlite.


I'm testing using about 8000 phpt tests and getting the same results as 
run-tests. There is still work to do on performance but I think it might 
at this stage be worth trying a bigger sample.


Zoe

Alright, thanks for the reply!
I'll try to have a look at the code and give it a try next weekend.

Nuno


-Original Message----- From: zoe slattery Sent: Monday, May 21, 
2012 8:26 AM

Subject: Re: [PHP-QA] Parallel run-tests
On 21/05/2012 06:45, Nuno Lopes wrote:

Hi Zoe,

Thanks for undertaking this project!

I just have a few concerns about this:
- The speedup seems a bit low to me. Maybe with a higher number of 
extensions enabled it will improve?..
I don't think it will improve with a higher number of extensions. It 
_may_ improve as a result of someone doing some work on the tests are 
scheduled - but I wouldn't count on it. Looking at improving 
performance is important and I wanted to get some confirmation that it 
is faster, I think we have that level of confirmation now. However, if 
I'm the only person working on this, further performance work will  
come after complete implementation + debugging.
- Is there any developer documentation available?  If I wanted to do 
a change or a bug fix today, how could I do it?
Lots - https://wiki.php.net/qa/runtests. I have just updated the 
"Development" section which has instructions on how to build and test 
the code. The 'Documentation' section has not been updated recently 
but I think it is still valid.


- Can it be packaged as a single drop-in file replacement for 
run-tests.php? The deployment is very important to me.  IMHO, the 
optimal solution would be to have a drop-in replacement for the 
current script, so that most developers wouldn't even notice the 
difference.
Yes - totally agree. I think I experimented with packaging it as a 
.phar (so it would be called run-tests.phar, not run-tests.php), but 
it's so long ago that I can't remember. I have added it to the to-do 
list (https://wiki.php.net/qa/runtests/todos).
- From previous emails exchanged in this thread, it seems that this 
new version requires a few extensions to run (gzip, soap??). This is 
not acceptable. The script must be able to run with --disable-all. Of 
course in that case the parallel version will fail.

No, it doesn't. That was a stupid bug and I fixed it last night :-)

Zoe


Nuno


-Original Message- From: zoe slattery
Sent: Thursday, May 17, 2012 1:00 PM
To: internals@lists.php.net ; PHP QA
Subject: [PHP-QA] Parallel run-tests

Hi

Over the past couple of weeks I have updated the parallel run-tests
(fixed a couple of minor bugs in the PHP code and the build.xml), it's
now almost at the point where I could go ahead and implement the last
pieces.  Here is a summary and a few questions:

1. In rebasing the code the the dev't stream I found a number of tests
with non-standard sections. My code checks test case structure and
objects to anything non-standard, the current run-tests.php mainly
ignores this kind of thing. I fixed up about 15 of these tests (see
#62022) already - I'll fix the rest if there are no objections - I will
open another bug report first.

2. If there is agreement to use this code it would make sense to
replace  the existing run-tests code with it, or rather,  it would make
no sense to try and maintain both versions. The new code is OO PHP, it's
in http://git.php.net/repository/phpruntests.git, is there any problem
with it staying there long term and maybe copying a run-tests.phar into
the PHP source directory? I have no idea what the right answer is,
suggestions welcome.

3. I ran a couple of small tests on my dual core Mac yesterday. For a
standard set of tests, the parallel code ran in 207 seconds, sequential
in 293 seconds and the standard run-tests.php took 298 seconds. This is
an improvement but I suspect we could still do better by looking at the
scheduling algorithm.
At the moment it's very simple, we just assemble a list of directories
with tests in and hand them out to processors till everything is done.
Being able to handle tests that must be run in sequence (mysql, mysqli)
will mean making some changes to this. So, perhaps we give an explicit
list to p1 and let the scheduler distribute the rest of the tests? Or
maybe we should have a 'process map' for all tests for extensions that
are build by default? Again, suggestions welcome.

4. REDIRECTTEST still needs to be implemented, I understand how it works
and this isn't (afaict) a major issue.

5. Testing. I'm able to do basic testing on Mac OSX  and Linux. I really
need access to an 8 way Linux system, or someone who has access and
would be interested in looking 

Re: [PHP-DEV] Re: [PHP-QA] Parallel run-tests

[zoe slattery]

On 21/05/2012 08:59, Devis Lucato wrote:

Hi,

nice work, out of curiosity I tried on a VM with 24 virtual cores:

Sequential new phpruntests: 29.6 seconds
Parallel -z 4: 12.1 - 12.3 secs
Parallel -z 8: 10.6 - 10.9 secs
Parallel -z 16: 10.6 - 11.3 secs
Parallel -z 24: 10.9 - 11.7 secs
Cool - thanks! I'll add those points to the performance chart. At a 
quick look that fits pretty much with the other data. Out of curiosity, 
how many 'test groups' was it?



Btw I'm seeing this error:

PHP Warning:  Invalid argument supplied for foreach() in 
phpruntests/src/testrun/rtPhpTestRun.php on line 107
PHP Notice:  Undefined offset: 1 in 
phpruntests/src/testrun/rtPhpTestRun.php on line 107
My bad. I started to implement REDIRECTTEST and checked in some initial 
changes, I can see what I've done wrong and will fix later today.


Zoe



Devis

[PHP-DEV] Re: [PHP-QA] Parallel run-tests

[zoe slattery]

On 21/05/2012 06:45, Nuno Lopes wrote:

Hi Zoe,

Thanks for undertaking this project!

I just have a few concerns about this:
- The speedup seems a bit low to me. Maybe with a higher number of 
extensions enabled it will improve?..
I don't think it will improve with a higher number of extensions. It 
_may_ improve as a result of someone doing some work on the tests are 
scheduled - but I wouldn't count on it. Looking at improving performance 
is important and I wanted to get some confirmation that it is faster, I 
think we have that level of confirmation now. However, if I'm the only 
person working on this, further performance work will  come after 
complete implementation + debugging.
- Is there any developer documentation available?  If I wanted to do a 
change or a bug fix today, how could I do it?
Lots - https://wiki.php.net/qa/runtests. I have just updated the 
"Development" section which has instructions on how to build and test 
the code. The 'Documentation' section has not been updated recently but 
I think it is still valid.


- Can it be packaged as a single drop-in file replacement for 
run-tests.php? The deployment is very important to me.  IMHO, the 
optimal solution would be to have a drop-in replacement for the 
current script, so that most developers wouldn't even notice the 
difference.
Yes - totally agree. I think I experimented with packaging it as a .phar 
(so it would be called run-tests.phar, not run-tests.php), but it's so 
long ago that I can't remember. I have added it to the to-do list 
(https://wiki.php.net/qa/runtests/todos).
- From previous emails exchanged in this thread, it seems that this 
new version requires a few extensions to run (gzip, soap??). This is 
not acceptable. The script must be able to run with --disable-all. Of 
course in that case the parallel version will fail.

No, it doesn't. That was a stupid bug and I fixed it last night :-)

Zoe


Nuno


-Original Message- From: zoe slattery
Sent: Thursday, May 17, 2012 1:00 PM
To: internals@lists.php.net ; PHP QA
Subject: [PHP-QA] Parallel run-tests

Hi

Over the past couple of weeks I have updated the parallel run-tests
(fixed a couple of minor bugs in the PHP code and the build.xml), it's
now almost at the point where I could go ahead and implement the last
pieces.  Here is a summary and a few questions:

1. In rebasing the code the the dev't stream I found a number of tests
with non-standard sections. My code checks test case structure and
objects to anything non-standard, the current run-tests.php mainly
ignores this kind of thing. I fixed up about 15 of these tests (see
#62022) already - I'll fix the rest if there are no objections - I will
open another bug report first.

2. If there is agreement to use this code it would make sense to
replace  the existing run-tests code with it, or rather,  it would make
no sense to try and maintain both versions. The new code is OO PHP, it's
in http://git.php.net/repository/phpruntests.git, is there any problem
with it staying there long term and maybe copying a run-tests.phar into
the PHP source directory? I have no idea what the right answer is,
suggestions welcome.

3. I ran a couple of small tests on my dual core Mac yesterday. For a
standard set of tests, the parallel code ran in 207 seconds, sequential
in 293 seconds and the standard run-tests.php took 298 seconds. This is
an improvement but I suspect we could still do better by looking at the
scheduling algorithm.
At the moment it's very simple, we just assemble a list of directories
with tests in and hand them out to processors till everything is done.
Being able to handle tests that must be run in sequence (mysql, mysqli)
will mean making some changes to this. So, perhaps we give an explicit
list to p1 and let the scheduler distribute the rest of the tests? Or
maybe we should have a 'process map' for all tests for extensions that
are build by default? Again, suggestions welcome.

4. REDIRECTTEST still needs to be implemented, I understand how it works
and this isn't (afaict) a major issue.

5. Testing. I'm able to do basic testing on Mac OSX  and Linux. I really
need access to an 8 way Linux system, or someone who has access and
would be interested in looking at performance? Any volunteers? This is
probably the most interesting part of the project :-)

6. Windows. I'm not in a position to do anything much with Windows
except some very basic checks to make sure that the sequential version
runs. The parallel code won't work because we used pcntl(), however I
know that Stefan and George were keen to design the code so that a
Windows solution could be implemented if anyone thought of one. If
anyone wants to pick up this aspect I'd be happy to get them started.

Zoe



--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Re: [PHP-QA] Parallel run-tests

[zoe slattery]

HI Felipe - a couple of updates.


1) Is your ~/dev/php5_4/sapi/cli/php  built --with-zlib?
2) This test uses the php-cgi, are you setting it with an env 
variable? Or relying on run-tests.php to find it (it should do that)


Even if it is missing zlib function that is causing the initial error, 
this should be caught by a SKIPIF section. There is only one test (as 
far as I know) that uses GZIP_POST, this is 
ext/soap/tests/server019.phpt.
I rebuilt PHP with soap and without zlib. The skipif section in this 
test seems to be working fine for me.


Would you be able to run that test on its own and send me the output? 
I think this should do it:


~/dev/php5_4/sapi/cli/php run-tests.php -p ~/dev/php5_4/sapi/cli/php 
~/dev/php5_4/ -vvv ~/dev/php5_4/ext/soap/tests/server019.php

I'd still like to see this.

Do you see the same issue in a sequential run? (ie without -z). If this 
is specific to the number of processes that are being run I'd need to 
put some extra debug in I think.


Thanks, Zoe



Thanks, Zoe



Notice: Undefined offset: 0 in
/home/felipe/dev/phpruntests/src/testcase/rtTestDifference.php on line
91
Fatal error: Allowed memory size of 134217728 bytes exhausted at
/home/felipe/dev/php5_4/Zend/zend_hash.c:330 (tried to allocate 81
bytes) in /home/felipe/dev/phpruntests/src/testcase/rtPhpTestFile.php
on line 108
/home/felipe/dev/php5_4/Zend/zend_hash.c(551) : ht=0x7f96e6356a50 is
inconsistent
[...]
Fatal error: Allowed memory size of 134217728 bytes exhausted at
ext/standard/var_unserializer.re:290 (tried to allocate 32 bytes) in
/home/felipe/dev/phpruntests/src/taskScheduler/rtTaskSchedulerFile.php
on line 183



2012/5/17 zoe slattery:

Hi

Over the past couple of weeks I have updated the parallel run-tests 
(fixed a
couple of minor bugs in the PHP code and the build.xml), it's now 
almost at
the point where I could go ahead and implement the last pieces.  
Here is a

summary and a few questions:

1. In rebasing the code the the dev't stream I found a number of 
tests with
non-standard sections. My code checks test case structure and 
objects to
anything non-standard, the current run-tests.php mainly ignores this 
kind of
thing. I fixed up about 15 of these tests (see #62022) already - 
I'll fix
the rest if there are no objections - I will open another bug report 
first.


2. If there is agreement to use this code it would make sense to 
replace
  the existing run-tests code with it, or rather,  it would make no 
sense to

try and maintain both versions. The new code is OO PHP, it's in
http://git.php.net/repository/phpruntests.git, is there any problem 
with it

staying there long term and maybe copying a run-tests.phar into the PHP
source directory? I have no idea what the right answer is, suggestions
welcome.

3. I ran a couple of small tests on my dual core Mac yesterday. For a
standard set of tests, the parallel code ran in 207 seconds, 
sequential in

293 seconds and the standard run-tests.php took 298 seconds. This is an
improvement but I suspect we could still do better by looking at the
scheduling algorithm.
At the moment it's very simple, we just assemble a list of 
directories with
tests in and hand them out to processors till everything is done. 
Being able

to handle tests that must be run in sequence (mysql, mysqli) will mean
making some changes to this. So, perhaps we give an explicit list to 
p1 and
let the scheduler distribute the rest of the tests? Or maybe we 
should have

a 'process map' for all tests for extensions that are build by default?
Again, suggestions welcome.

4. REDIRECTTEST still needs to be implemented, I understand how it 
works and

this isn't (afaict) a major issue.

5. Testing. I'm able to do basic testing on Mac OSX  and Linux. I 
really
need access to an 8 way Linux system, or someone who has access and 
would be
interested in looking at performance? Any volunteers? This is 
probably the

most interesting part of the project :-)

6. Windows. I'm not in a position to do anything much with Windows 
except
some very basic checks to make sure that the sequential version 
runs. The
parallel code won't work because we used pcntl(), however I know 
that Stefan
and George were keen to design the code so that a Windows solution 
could be
implemented if anyone thought of one. If anyone wants to pick up 
this aspect

I'd be happy to get them started.

Zoe

--
PHP Quality Assurance Mailing List<http://www.php.net/>
To unsubscribe, visit: http://www.php.net/unsub.php










--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Re: [PHP-QA] Parallel run-tests

[zoe slattery]

On 19/05/2012 22:33, Felipe Pena wrote:

Hi Zoe,

great work on this new run-tests! :D

I was testing it and noticed a problem when running using the following command:
$ ~/dev/php5_4/sapi/cli/php run-tests.php  -z 8 -p
~/dev/php5_4/sapi/cli/php ~/dev/php5_4/

It shows the PHP messages:
[...]
Fatal error: Call to undefined function gzencode() in
/home/felipe/dev/phpruntests/src/testcase/sections/configurationsections/rtGzipPostSection.php
on line 23
[...]

Thanks for that - couple of questions:

1) Is your ~/dev/php5_4/sapi/cli/php  built --with-zlib?
2) This test uses the php-cgi, are you setting it with an env variable? 
Or relying on run-tests.php to find it (it should do that)


Even if it is missing zlib function that is causing the initial error, 
this should be caught by a SKIPIF section. There is only one test (as 
far as I know) that uses GZIP_POST, this is ext/soap/tests/server019.phpt.


Would you be able to run that test on its own and send me the output? I 
think this should do it:


~/dev/php5_4/sapi/cli/php run-tests.php -p ~/dev/php5_4/sapi/cli/php 
~/dev/php5_4/ -vvv ~/dev/php5_4/ext/soap/tests/server019.php

 
Thanks, Zoe




Notice: Undefined offset: 0 in
/home/felipe/dev/phpruntests/src/testcase/rtTestDifference.php on line
91
Fatal error: Allowed memory size of 134217728 bytes exhausted at
/home/felipe/dev/php5_4/Zend/zend_hash.c:330 (tried to allocate 81
bytes) in /home/felipe/dev/phpruntests/src/testcase/rtPhpTestFile.php
on line 108
/home/felipe/dev/php5_4/Zend/zend_hash.c(551) : ht=0x7f96e6356a50 is
inconsistent
[...]
Fatal error: Allowed memory size of 134217728 bytes exhausted at
ext/standard/var_unserializer.re:290 (tried to allocate 32 bytes) in
/home/felipe/dev/phpruntests/src/taskScheduler/rtTaskSchedulerFile.php
on line 183



2012/5/17 zoe slattery:

Hi

Over the past couple of weeks I have updated the parallel run-tests (fixed a
couple of minor bugs in the PHP code and the build.xml), it's now almost at
the point where I could go ahead and implement the last pieces.  Here is a
summary and a few questions:

1. In rebasing the code the the dev't stream I found a number of tests with
non-standard sections. My code checks test case structure and objects to
anything non-standard, the current run-tests.php mainly ignores this kind of
thing. I fixed up about 15 of these tests (see #62022) already - I'll fix
the rest if there are no objections - I will open another bug report first.

2. If there is agreement to use this code it would make sense to replace
  the existing run-tests code with it, or rather,  it would make no sense to
try and maintain both versions. The new code is OO PHP, it's in
http://git.php.net/repository/phpruntests.git, is there any problem with it
staying there long term and maybe copying a run-tests.phar into the PHP
source directory? I have no idea what the right answer is, suggestions
welcome.

3. I ran a couple of small tests on my dual core Mac yesterday. For a
standard set of tests, the parallel code ran in 207 seconds, sequential in
293 seconds and the standard run-tests.php took 298 seconds. This is an
improvement but I suspect we could still do better by looking at the
scheduling algorithm.
At the moment it's very simple, we just assemble a list of directories with
tests in and hand them out to processors till everything is done. Being able
to handle tests that must be run in sequence (mysql, mysqli) will mean
making some changes to this. So, perhaps we give an explicit list to p1 and
let the scheduler distribute the rest of the tests? Or maybe we should have
a 'process map' for all tests for extensions that are build by default?
Again, suggestions welcome.

4. REDIRECTTEST still needs to be implemented, I understand how it works and
this isn't (afaict) a major issue.

5. Testing. I'm able to do basic testing on Mac OSX  and Linux. I really
need access to an 8 way Linux system, or someone who has access and would be
interested in looking at performance? Any volunteers? This is probably the
most interesting part of the project :-)

6. Windows. I'm not in a position to do anything much with Windows except
some very basic checks to make sure that the sequential version runs. The
parallel code won't work because we used pcntl(), however I know that Stefan
and George were keen to design the code so that a Windows solution could be
implemented if anyone thought of one. If anyone wants to pick up this aspect
I'd be happy to get them started.

Zoe

--
PHP Quality Assurance Mailing List<http://www.php.net/>
To unsubscribe, visit: http://www.php.net/unsub.php







--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Parallel test runs - thanks!

[zoe slattery]

Hi

A few people were able to use the new run-tests.php code to run tests in 
parallel on different platforms. I've plotted the results here: 
http://zoah.co.uk/update-parallel-run-tests-php.


Thanks very much for doing that, thanks also for the bugs you found in 
doing so (blush) -  I will fix.


Interesting data, suggests that 4 processes is optimal and that the best 
we can hope for is being able to run in about 1/3rd of the time they run 
in at the moment. At some stage I will look at where it's spending its 
time and see if we can improve on these results. For now I think 
completing the implementation and fixing bugs has a higher priority for me.


Zoe

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Parallel run-tests

[zoe slattery]

Hi Miah -


Any chance you could write up the steps you took to test this - or point
me in the right direction so I can try myself?

Re 8 way, is that 8 physical or 8 threads? If I can get the tests
running I can give you a data point for a 4 core/8 thread i7 under
linux.
that would be brilliant! I have a the steps written down in various 
places - a bit scattered around :-/
I'll assemble a list of instructions and put it on the PHP wiki later 
tonight, then I'll send out a link.


Thanks so much for the offer!

Zoe





--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Re: [PHP-QA] Parallel run-tests

[zoe slattery]

Olivier - brilliant thanks!

So, we have two points now: http://zoah.co.uk/update-parallel-run-tests-php

Anyone do an 8 way?

Zoe

5. Testing. I'm able to do basic testing on Mac OSX  and Linux

Hi Zoe,

Here is a quick test on my 4-cores Linux server :

tested on PHP 5.3.9
412 tests executed
Sequential : 115,3 seconds
Parallel (4 threads) : 43,8 seconds
Original run-tests.php : 115,4 seconds

Olivier



--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Parallel run-tests

[zoe slattery]

Hi

Over the past couple of weeks I have updated the parallel run-tests 
(fixed a couple of minor bugs in the PHP code and the build.xml), it's 
now almost at the point where I could go ahead and implement the last 
pieces.  Here is a summary and a few questions:


1. In rebasing the code the the dev't stream I found a number of tests 
with non-standard sections. My code checks test case structure and 
objects to anything non-standard, the current run-tests.php mainly 
ignores this kind of thing. I fixed up about 15 of these tests (see 
#62022) already - I'll fix the rest if there are no objections - I will 
open another bug report first.


2. If there is agreement to use this code it would make sense to 
replace  the existing run-tests code with it, or rather,  it would make 
no sense to try and maintain both versions. The new code is OO PHP, it's 
in http://git.php.net/repository/phpruntests.git, is there any problem 
with it staying there long term and maybe copying a run-tests.phar into 
the PHP source directory? I have no idea what the right answer is, 
suggestions welcome.


3. I ran a couple of small tests on my dual core Mac yesterday. For a 
standard set of tests, the parallel code ran in 207 seconds, sequential 
in 293 seconds and the standard run-tests.php took 298 seconds. This is 
an improvement but I suspect we could still do better by looking at the 
scheduling algorithm.
At the moment it's very simple, we just assemble a list of directories 
with tests in and hand them out to processors till everything is done. 
Being able to handle tests that must be run in sequence (mysql, mysqli) 
will mean making some changes to this. So, perhaps we give an explicit 
list to p1 and let the scheduler distribute the rest of the tests? Or 
maybe we should have a 'process map' for all tests for extensions that 
are build by default? Again, suggestions welcome.


4. REDIRECTTEST still needs to be implemented, I understand how it works 
and this isn't (afaict) a major issue.


5. Testing. I'm able to do basic testing on Mac OSX  and Linux. I really 
need access to an 8 way Linux system, or someone who has access and 
would be interested in looking at performance? Any volunteers? This is 
probably the most interesting part of the project :-)


6. Windows. I'm not in a position to do anything much with Windows 
except some very basic checks to make sure that the sequential version 
runs. The parallel code won't work because we used pcntl(), however I 
know that Stefan and George were keen to design the code so that a 
Windows solution could be implemented if anyone thought of one. If 
anyone wants to pick up this aspect I'd be happy to get them started.


Zoe

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] run-tests.php - GZIP_POST and DEFLATE_POST question

[zoe slattery]

Hi

When we implemented these sections in the parallel version of 
run-tests.php we seem to have decided to change the way that they work.


https://wiki.php.net/qa/runtests/decisions/notimplemented

I can't find any discussion of this either on internals or on QA.

In the current version of run-tests.php there is a --GZIP_POST-- section 
which just has a "1" in it which tells run-tests.php to gzip the 
contents of a --POST-- section. See here: 
http://qa.php.net/phpt_details.php#gzip_post_section


I'm not sure why it was done this way - it seems to me to make more 
sense to just have a single --GZIP_POST-- section. Maybe I'm missing 
something?


Anyway - it  affects two tests (afaict) and I was wondering if there 
would be any objection to my making a small change to the way these two 
sections are implemented in run-tests.php? And change the tests of course.


Zoe





--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Re: [PHP-QA] running tests in parallel?

[zoe slattery]

Hi

Thanks for the replies to this thread - I'll get on with it then :-)

Zoe

It's very important! Take a look at http://gcov.php.net
A single build takes almost 2 days with most extensions enabled plus 
valgrind testing.


So, yes, parallel testing would be highly appreciated (but do not 
forget my initial requirements, such as e.g. mysql and mysqli tests 
cannot run at the same time).


Nuno

-Original Message----- From: zoe slattery
Sent: Thursday, May 03, 2012 11:54 AM
Subject: [PHP-QA] running tests in parallel?

Hi

Some years ago (in 2009) Stefan Priebsch, myself and a GSOC student
worked on a version of run-tests.php which would run tests in parallel -
I wrote about it here:
http://zoomsplatter.blogspot.co.uk/2009/07/on-no-not-more-tests.html.

The code was complete apart from finding some way to handle REDIRECT and
does run faster on multi core systems but for various reasons we all ran
out of time to work on it.  I recently checked out the code and it
still  works, there are a few little things to be fixed, and of course
REDIRECT still needs to be implemented. I have some free time now and
would be happy to do this, however, I've been fairly out of touch with
PHP since 2009, so before I get on and make this work:

(a) Would it still be helpful if the tests could be run faster?
(b) Has anyone come up with a better solution in the years since I
looked at this?

Thanks,
Zoe



--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] running tests in parallel?

[zoe slattery]

Hi

Some years ago (in 2009) Stefan Priebsch, myself and a GSOC student 
worked on a version of run-tests.php which would run tests in parallel - 
I wrote about it here: 
http://zoomsplatter.blogspot.co.uk/2009/07/on-no-not-more-tests.html.


The code was complete apart from finding some way to handle REDIRECT and 
does run faster on multi core systems but for various reasons we all ran 
out of time to work on it.  I recently checked out the code and it 
still  works, there are a few little things to be fixed, and of course 
REDIRECT still needs to be implemented. I have some free time now and 
would be happy to do this, however, I've been fairly out of touch with 
PHP since 2009, so before I get on and make this work:


(a) Would it still be helpful if the tests could be run faster?
(b) Has anyone come up with a better solution in the years since I 
looked at this?


Thanks,
Zoe

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] Do you need different OS/platform to test on? Let "me" know

[zoe]

Hi Scott


From there it can run tests and do whatever we wish with the results, 
like grab all test failures and upload it to your test result viewer. 
If a build fails we can have it use twitter / irc to inform people 
that a slave failed to build PHP for whatever reason.
I wonder if we can use the new run-tests code as part of this? Georg has 
just integrated a first try at parallel running. We are working on 
having something that it's easy to grab all the test results from.


Olivier sent a note earlier this week as an attempt to map out 
requirements  what should happen with the ad-hoc test reports, however 
I'm wondering if we should just scrap the ad-hoc test reports and put 
the effort into getting results from a defined set of platforms? Aside 
from the fact that they go to a mailing list which is hard to view, 
would anyone make use of the ad-hoc reports if we could get test results 
from  a representative set of platforms in a more controlled way?


I'd like to be able to see test results across several platforms 
(various *ix, windows, mac), 64 bit, 32 bit. And when a test fails 
to be able to get to the relevant files. This is similar to the gcov 
pages (but more) with the ability to have different views of the 
results, I like Mark's summary page but would want to be able have a few 
different views, the test fest results page has a few of the attributes 
that I'd like to see (not the color scheme of course). Is the sort of 
thing we would be able to do?


It's also got various scheduling parts built in, so we can do 
continuous regular builds to make sure its still compiling and daily 
memory builds to check for new memory leaks.


Other cool features are notifying people when it broke via IRC, 
twitter or mailing list. And optionally packaging the freshly built 
tests and uploading them. We could hypothetically get our snapshots 
built this way so we never release a broken snapshot?

That would be good :-)


The project URL is http://buildbot.net/trac and the vast documentation 
is at http://djmitche.github.com/buildbot/docs/0.7.10/


I might be able to sit down and give this a go on the plane next week.

Zoe


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Call for a doc push

[zoe]

Hey Gwynne


Zoe, have a look at <http://news.php.net/php.doc.cvs/4180> :). I 
don't have anything that can read that ODP file you linked (at least, 
not in any useful sense), but the opcodes.html file is all there. 
With any luck, someone else will come along and clean it up a bit, 
and/or expand on it.



Brilliant! Thanks.

http://enquirysolve.co.uk/fisheye/view_image.php?image_id=3790
Is the Opcode paper as a PDF, but it's simply an OpenOffice 
presentation so doesn't need anything 'expensive' to read and edit it.


Lester - this helps. The reason that I made the .odp available was so 
that people would be able to update it - this kind of information tends 
to get out of date and I think Andy did that work almost two years ago :-/.


I can't think of a better way than .opd or .pdf right now either. I 
guess it really needs converting into a set of pictures that could be 
embedded in documentation as part of the manual, I have a feeling this 
might be a lot of work.


Zoe

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Call for a doc push

[zoe]

Hi

Guys and gals, in the old days we had a very close tie between the code
and the documentation.  As the project has grown the two have drifted
apart.  I think this is mostly because the phpdoc team has done an
amazing job keeping up with the code changes and writing awesome
documentation.  This has made us a bit lazy and complacent.

I would like to encourage everyone on this list to spend a little bit of
time looking at the parts of the documentation that cover things you are
familiar with.  Or even just going through some of the doc bugs and
helping out in general.
  
This reminded me that a while ago I made some opcode documentation 
available in the form of a set of charts written by Andy Wharmby 
(http://www.zapt.info/PHPOpcodes_Sep2008.odp) and some opcode samples 
(http://www.zapt.info/opcodes.html) from the IBM Japan team. I don't 
think either of these ever made it the PHP docs and as they are  still 
on my personal website and they are somewhat at risk :-/ 

I heard recently that people were using them 
(http://tr.im/phpcompilerinternals) so I'd like to put them somewhere 
more permanent. My current plan is to move them both to sp1.php.net. 
Does anyone object? Or better still would anyone be willing to integrate 
them with the PHP docs?


Zoe



--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] detecting bad --CLEAN-- sections

[zoe]

Greg Beaver wrote:

hi,

I've noticed that it is very difficult to tell if your --CLEAN--
sections are actually working, so I wonder if it might be possible to
brainstorm ways of fixing this.
  
Yes, I found the same thing and fixed 40 tests under ext/standard/tests 
last week - in PHP53. Tedious work.

A couple of options come to mind that could be considered

- append this code to the end of the .clean.php file: ?> and capture the output, then check to make sure
that we see "--CLEAN-- works" in the output.  This would detect fatal
errors (which is the most common problem I find, especially when a
--CLEAN-- section includes a helper file).
- add a --REMOVEFILES--/--REMOVEDIRS-- section where one can specify
relative paths to files or whole directories that should be nixed, and
have run-tests do the removal
  
In the new run-tests code (http://cvs.php.net/viewvc.cgi/phpruntests/) 
we just check and warn if there is *any* output from executing the CLEAN 
section. This works in the sense that it's how I found all the messed up 
tests.


If you want to talk about other requirements for that code I think 
php-qa is the right place to do it. By the way, - CLEAN-- sections 
aren't the aren't the only problems. I've found a few sections called 
--SKIP--, which of course do nothing :-).


I  started a requirements page for the new code, here 
wiki.php.net/qa/runtests, but I think it's better to discuss on the list 
before adding there.


Zoe


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Test Fest mentors

[zoe]

Hi all

The 2009 TestFest is due to run between April and June, we have 12 user 
groups around the world signed up already (wiki.php.net/qa/testfest) and 
one or two individuals who would like to participate.


We really need a few more experienced internals people to act as 
mentors. Being a mentor means helping answer user groups questions about 
test writing and reviewing and committing their tests.


Please would you add your name and other  details to the table here 
wiki.php.net/qa/testfest if you can help with this?


Thanks, Zoe



--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Re: cvs: php-src(PHP_5_3) /ext/standard/tests/general_functions phpinfo.phpt

[zoe]

Hi Zoe,

Is there documentation on this change?  
Hi Greg - you mean the change I made to the phpinfo.phpt test? If so I 
changed it because it was failing and should not have been. Or do you 
mean changes to run-tests.php options?

I'm finding it a little bit
arbitrary that run-tests keeps changing basic stuff so often, it's
making me split tests simply to have a different EXPECTF for php 5.2 vs.
5.3.  First it was %s, then %a, and now %A to match multi-line?
  
Sure - all the --EXPECTF-- options are documented here 
http://qa.php.net/expectf_details.php, and, as far as I'm aware all the 
EXPECTF options should be common to PHP 52, PHP 53 and PHP 6. If they 
aren't then it's an oversight and I'll be happy to fix. I don't think 
any of them should require changes to existing tests.


Zoe



--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Re: Bug #46701

[zoe]

Moriyoshi Koizumi wrote:

Please don't even think of backporting. This will definitely break a
lot of things, and this kind of thing must not be done in a minor
release.

  

--snip--


I guess the patch relies on the 5.3's DVAL_TO_LVAL behavior that was
changed by the fix for bug #42868, right?
If so, this patch shouldn't be MFH'ed as the #42868 patch was not
merged although I didn't remember any discussion on this.

See also: http://marc.info/?l=php-internals&m=120799720922202&w=2



Hey all

I'm sorry - I should have replied to this before since I was responsible 
for raising #42868.  I didn't do a good job at explaining what the issue 
was in that bug, mainly because I didn't know what it was when I 
started. The central problem is that PHP's behaviour on casting double 
to int defaults to whatever the underlying C environment does. On 
Windows and Linux (all of the versions that I looked at) this turns out 
to be a simple truncation of the last 32 bits. Unfortunately the C 
behaviour is 'undefined' (Kernigan and Ritchie, page 197, A6.3). The 
issue that I found in #42868 was that on the Mac the casting behaviour 
is completely different so many of the PHP tests failed. I believe that 
PHP should behave in a platform independent way - that is what the fix 
to #42868 achieves. It is also fair to say that any applications that 
depend on the overflow behaviour in PHP 5.2 cannot be guaranteed to run 
on any platform.


Zoe

Regards,
Moriyoshi



--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

  

Ilia Alshanetsky








  



--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] PHP TestFest 2009

[zoe]

Hi all

I apologise for spamming both lists with this and also having blogged 
first, I intended then all to go at the same time :-). We (Lukas, Scott, 
Pierre and others ) have been talking about this a little on IRC and I 
updated the QA web pages a while ago. We thought it was time to be a bit 
more public. I have appended a copy of my  my blog entry below.


As far as mentors go, I've said 'let me know on the IRC channels if you 
can help' - what I probably meant was watch out because we'll be 
pestering you for help on those channels :-). We'll also be putting up a 
list of things to do on the TestFest wiki 
(http://wiki.php.net/qa/testfest), help with any of them will earn 
undying gratitude and your choice of beverage when I next see you.



*PHP TestFest 2009*

It's time to start getting the 2009 PHP TestFest underway. The TestFest 
is a worldwide event in which PHP user groups and individuals contribute 
to PHP by writing tests for PHP. It's a great way to contribute to one 
of the most successful open source projects there has ever been, it's 
also pretty cool to see your name in the the source distribution for 
code that's running on over 20 million web domains.


So how does it work?


User groups can register by sending a mail to php...@lists.php.net. In 
the mail we'd like you to pick an date for your event (somewhere between 
April 1st and June 30th), it would be great if you could let us have the 
name and e-mail for the primary contact too. As these arrive on the QA 
list, someone (probably me) will transfer them to the TestFest wiki 
. We will help you to work out what 
sort of event to run and how to organise it. Individuals can register 
for the TestFest too, just send a mail to the php...@lists.php.net.



User groups (or individuals) can pick areas in which they'd like to 
write tests, this might be something you know quite well already, or 
maybe something new you'd like to know about. For example, the London 
PHP group picked the dom extension last year. I didn't know much about 
that when we started but I did when we finished!



What do we need?

A bit of infrastructure! Scott MacVicar is putting together an SVN 
repository for tests so that contributors will be able to commit tests 
directly. He's still working on how access control will workmore on 
this later.


Sponsors! We have one offer of ElePHPants. We'll do the same as we did 
last year and hold a draw at the end of the TestFest. Anyone who 
contributes a test will have a chance to win one. Of course, the more 
you contribute the higher your chances will be!


Mentors, mentors and more mentors! We need internals developers who know 
how to write tests, have karma in cvs.php.net and who can review tests 
and commit them. Again - if you can help with this send a note to the 
php...@lists.php.net or let me know on IRC (efnet #php.pecl or freenode 
#phptestfest, #phpwomen).


For more information on PHP's TestFest have a look here 
 and here 



--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Function return values in PHP5.3

[zoe]

Hi Lukas, Johannes

Please could I ask for a PHP5.3 release manager's point of view on this? 
I raised this ext/imap bug yesterday 
http://bugs.php.net/bug.php?id=46902 after a discussion with Ilia 
because I noticed that all of the function return values which result 
from a failure of parameter parsing had changed from 'false' in PHP5.2 
to NULL in PHP5.3.


So, questions:

1) Is the accepted practice that functions will return NULL when 
zend_parse_parameters() fails in PHP5.3?


2) Is it acceptable to change the behaviour of functions between 5.2 and 
5.3?


3) Does this apply to *all*  PHP functions?

4) Assuming that the answer (3) is no, is there any documentation about 
which functions have changed? (If not would you like some?)



Zoë



--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] PHP opcode documentation

[zoe]

Hi

I have some PHP opcode documentation which might be useful, an html 
version is linked here: http://www.zapt.info/opcodes.html.


The hard work (writing the samples) was all done by Mich and Toyo. All 
I've done is written some PHP and used a slightly modified VLD to be 
able to automatically generate the web page from the sample code. Sara 
said this could also be done with parsekit, but by that time I'd already 
fixed VLD (if Derick forgets to apply my VLD patch I'll swap my script 
to use parsekit :-) ).


There are a few codes that don't have samples yet, but these can be 
added easily by just dropping code into the right directory and then 
re-running the PHP.



I'd like to put the samples and scripts somewhere, but where?  It would 
be good to have them somewhere that people could add the missing samples 
if they have them.


Zoë



--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Re: Subversion migration

[zoe]

David Soria Parra wrote:


As far as I know the actual conversion is done, but a lot of the 
CVSROOT/ scripts are not yet rewritten to fit the subversion hook 
system. Also Marcus proposed and I guess it was somehow accepted, that 
the new scripts are done in Python and not in Perl.


  
Is there a list of what needs to be written? I have some spare time and 
could help, although I confess I'd prefer to write in PHP.


Zoe


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] testfest

[zoe]

Hi Lig - I think I may have got half way through replying to this and 
then forgotten...




1) Who is willing to be a mentor?  What areas do you want to mentor? 
What areas are you willing to mentor?  Where are you physically 
located if the local user group would like to work with you - what 
time frames would you be available?  (feel free to respond off list to 
me with your info)
Please add my name to the the people who will mentor DOM? I'm also happy 
to do general mentoring on style of test writing ...


Also - Stas had already agreed to mentor but I don't see his name on the 
Wiki...



Zoe

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] QA reports

[zoe]

Hi - I always assumed that it was intentional that the test results were 
sent to [EMAIL PROTECTED] (and I have a mail filter set up to deal with 
them).


After some discussion on IRC it appears that they should be going to 
[EMAIL PROTECTED] This would be much better as they currently 
tend to drown and discussion on [EMAIL PROTECTED]


I think fixing this is a one line change to 
qaweb/buildtest-process.php,  I'm happy to make that if there is 
agreement. Please let me know if it isn't as simple as this.


Thanks,

Zoe

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Math functions (new parameter parsing)

[zoe]

Felipe Pena wrote:

Hi all,

I made a patch for 5_3 where all functions in ext/standard/math.c use
the new parameter parsing. (As was done in HEAD)
If anyone think it good, let me know, and then i'll commit it.

Several tests break with the patch. Hence, i'll fix them also, if
agreed. :)

  
Felipe - Great work!! I'll wait for the patch to go in and then re-run 
the tests that I'm still trying to debug to see if they pass.


Zoe

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Updates to the QA pages

[zoe]

Philip Olson wrote:


A few moons ago the following [rough] wiki page was started with the 
aim to answer questions not yet covered within the QA site, questions 
like "installing lcov", "how to test if a test covers what I want", 
and such. Please feel free to steal information from it for qaweb:


http://doc.php.net/wiki/writing-tests
For now, I have put a link in the QA web pages that points to this. I 
think the QA pages could probably do with a bit of restructuring to make 
them easier to navigate and at that stage this information should go in, 
it's very useful. I have restructuring on my long term to-do list - so 
I'd be more than happy if someone else got there first :-)


Regarding your proposed changes, could you provide a unified diff? 
Looks good, but it's difficult to see what's been updated. Or, just 
commit... :)

If there isn't any further feedback by this time tomorrow I'll commit.

Thanks for your help,
Zoe

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Re: Updates to the QA pages

[zoe]

Edward Z. Yang wrote:

Some minor grammar picks:


Thanks - all fixed.

Out of curiosity, I'm wondering why run-tests.php itself can't run a
test file by itself.

  
It can - in fact that's exactly what it does, run .phpt files. However, 
sometimes easier and quicker when you are writing the test to run it as 
a PHP script and look at the output on the screen.


Zoe

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Updates to the QA pages

[zoe]

Hi all

Rob Nicholson and I have made a set updates the QA pages, mainly based 
on things that we've learned over the past
year of test case writing - we had a lot of input from Raghu and Marcus 
on this too of course.


The updated pages are here for review: http://zapt.info/phpt/write-test.php.

Feedback welcome!

Zoe Slattery

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Writing PHPT tests

[zoe]

Rob Nicholson wrote:


Hi Zoe,

I notice also that the testcase generator and the testcases we have 
been committing differ slightly from the conventions listed at: 
http://qa.php.net/write-test.php . This was based on feedback we 
received so I think that http://qa.php.net/write-test.php should be 
updated. Specifically the  convention of writing multiple small tests 
to cover basic, variation  and error cases separately differs from the 
naming convention suggested on the QA pages.
Rob - you are right - I think the most important thing we learned is 
that size matters - the smaller the better.
 We have also been following a convention w.r.t comments in the 
testcase which I think is helpful so I'd suggest we cover this too. 
Since the testcase generator adds these comments its  worth 
documenting them.


I'll produce a patch to the documentation for review if you like.

Please do :-) I admit I have been meaning to do this for ages and it's 
bad that I haven't got around to it.


Rob Nicholson


From:   zoe <[EMAIL PROTECTED]>
To: internals@lists.php.net
Date:   02/02/2008 11:21
Subject:[PHP-DEV] Writing PHPT tests






Hi - for any of you that are writing PHPT tests for existing extensions
- I put a PHP script called generate_phpt.php into PHP 5.3 yesterday
which might help a little.

It's quite a simple command line script (Raghu and I wrote it last
year), it works by looking at the {{{proto line for a function in PHP
source code and construct a test case frame from it. It can be used to
construct very simple tests cases - or to turn an existing PHP file into
PHPT format. It doesn't try and guess what the results of a test should
be :-). I will document it properly on qa.php.net later. In the meantime:

php generate_phpt.php --help

tells you what it's supposed to do.

Zoe

PS - It doesn't work for PHP6 right now because the {{{proto line is
different.

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php







/
/

/Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with 
number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 
3AU/









--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Writing PHPT tests

[zoe]

Jani Taskinen wrote:
You simply ignored my email or never got it? The correct place for 
such scripts is not in the "root" but in scripts/dev/


--Jani

Jani - of course I wouldn't ignore a mail from you :-) - I didn't see 
it. I talked a bit to Johannes on IRC about where to put it and in the 
end said I'd put it in the top level and move or remove it if anyone 
disagreed. I'll move it.


Thanks, Zoe


zoe kirjoitti:
Hi - for any of you that are writing PHPT tests for existing 
extensions - I put a PHP script called generate_phpt.php into PHP 5.3 
yesterday which might help a little.


It's quite a simple command line script (Raghu and I wrote it last 
year), it works by looking at the {{{proto line for a function in PHP 
source code and construct a test case frame from it. It can be used 
to construct very simple tests cases - or to turn an existing PHP 
file into PHPT format. It doesn't try and guess what the results of a 
test should be :-). I will document it properly on qa.php.net later. 
In the meantime:


php generate_phpt.php --help

tells you what it's supposed to do.

Zoe

PS - It doesn't work for PHP6 right now because the {{{proto line is 
different.







--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Writing PHPT tests

[zoe]

Hi - for any of you that are writing PHPT tests for existing extensions 
- I put a PHP script called generate_phpt.php into PHP 5.3 yesterday 
which might help a little.


It's quite a simple command line script (Raghu and I wrote it last 
year), it works by looking at the {{{proto line for a function in PHP 
source code and construct a test case frame from it. It can be used to 
construct very simple tests cases - or to turn an existing PHP file into 
PHPT format. It doesn't try and guess what the results of a test should 
be :-). I will document it properly on qa.php.net later. In the meantime:


php generate_phpt.php --help

tells you what it's supposed to do.

Zoe

PS - It doesn't work for PHP6 right now because the {{{proto line is 
different.


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] php 5.2.4 test results reference

[Zoe Slattery]

Derick Rethans wrote:

On Wed, 17 Oct 2007, jean jayet wrote:

  

i have the following naive question : how can get a reference of test
results

smth to be able to find :

such set of test case id MUST pass
such set test case id MUST fail
such set test case id skip if such extension missing etc,...



Well, they should all pass - skipped tests depends totally on the 
configuration - I don't think we've any information on that at all.


  
Jean - if the tests don't pass on your system it would be helpful if you 
could raise defects (http://bugs.php.net/) checking first to make sure 
that someone hasn't already done so, and  make the .diff and .out files 
available. In some cases when we write tests it isn't possible to test 
on ALL platforms, so sometimes the tests may fail because they really 
should not pass on a specific platform (if so we will fix by adding a 
--SKIPIFF-- to the test case), sometimes it's because of an 
implementation difference - we can't tell till we see the output from 
the test.


As Derick says, they should all pass :-)

Zoe





regards,
Derick

  


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Re: cvs: php-src(PHP_5_2) / run-tests.php /ext/bz2/tests 005.phpt /ext/standard/tests/general_functions phpcredits.phpt phpinfo.phpt /ext/standard/tests/strings sha1.phpt /ext/zlib/t

[Zoe Slattery]

Michael Wallner wrote:

Nuno Lopes wrote:
  

nlopess Fri Sep 14 15:28:04 2007 UTC



  

  Log:
  changes to run-tests.php:
  - change %s to %a
  - make %s = [^\r\n]+
  - fix tests accordingly



I think this is a very bad change.  While tests of bundled extensions
can be updated accordingly, pecl extension tests have no way to be
version agnostic in this regard.


  
Mike - that's a fair point. When we looked through the PHP tests we 
found that the vast majority (all except 4) relied on the %s behaviour 
as is is currently implemented - that is *not* matching over a line end. 
Only 4 tests were actually intentionally using %s to match over line 
endings. From what you say it sounds as though many more of the PECL 
tests behave this way?


Zoe Slattery

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] [patch] Late static bindings (LSB)

[Zoe Slattery]

Stanislav Malyshev wrote:

So we have at least three different views on concept.
It seems that patch will be delayed again. :(


I think we better spend time on figuring out the concept and then do 
the patch than first do the patch and then discover we don't know how 
it was supposed to work :)

How about writing the test cases first - and then the patch?

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Re: [PHP-QA] run-tests.php

[Zoe Slattery]

Zoe Slattery wrote:
 
It is a nice patch - but I'm seeing some errors that I don't 
understand yet when I use it - like this:


Warning: preg_match(): Compilation failed: missing terminating ] for 
character class at offset 5 in /home/zoe/BUILDS/php52/run-tests.php on 
line 1609


I'm not having the best afternoon :-( - so this is probably something 
I'm doing wrong - does anyone else see it?
To answer my own question, these warnings are coming from the tests that 
haven't been modified to use %a instead of %s. So they may get generated 
if people write tests and use %s where they should (as soon as the fix 
is committed) be using %a.


Zoe
 


  





--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Re: [PHP-QA] run-tests.php

[Zoe Slattery]

Marcus Boerger wrote:

Hello Nuno,

Thursday, September 13, 2007, 9:20:51 PM, you wrote:

  

  now i get the issue. It grabbs more than one line! This means
run-tests.php has to be fixed. The whole thing is designed under
the assumption that %s catches no new lines and hence only one
line.

That would be the best answer but I think it's a difficult fix and 
whatever it was might break other tests. I think it would mean replacing 
%s with something other than .+?. I'll have a look tomorrow and see if I 
can find a regex that would do - one of the things that had me confused 
for a while was that I thought the ? (non-greedy qualifier) should 
guarentee the minimum match but it doesn't work quite like that.
  



  
OK, so basically it is impossible to fix the current regex to match what we 
want (I had a quick chat with the PCRE author and he thinks the same).
I propose the following patch: 
http://web.ist.utl.pt/nuno.lopes/php_run_tests_%s.txt



  

The main changes are:
'%s' => '[^\r\n]+'



Nice patch! The only thing i wonder about is why you disallow /r. Shouldn't
we allow it to prevent problems? Or is it enough to take care of it only at
line endings, and if so where are we doing that?

best regards
marcus

  
It is a nice patch - but I'm seeing some errors that I don't understand 
yet when I use it - like this:


Warning: preg_match(): Compilation failed: missing terminating ] for 
character class at offset 5 in /home/zoe/BUILDS/php52/run-tests.php on 
line 1609


I'm not having the best afternoon :-( - so this is probably something 
I'm doing wrong - does anyone else see it?


Zoe

'%a' => '.+'



  
This means that some tests have to be fixed, but the majority passes. Most 
important is that with this patch I found some tests that had the expected 
output wrong.



  
So unless someone rejects this patch, I'll commit it (and update the 
http://qa.php.net/write-test.php page too).
Anyway, I don't think that a function name should be replaced by %s. The 
expected output should be as stricter as possible.




  
Nuno 





Best regards,
 Marcus


  


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Re: [PHP-QA] run-tests.php

[Zoe Slattery]

Hi again

On second thoughts - cancel my last two notes - yours is a better fix. :-)

Zoe


Johannes Schlüter wrote:

Hi Nuno,

On Thu, 2007-09-13 at 20:20 +0100, Nuno Lopes wrote:
  

  now i get the issue. It grabbs more than one line! This means
run-tests.php has to be fixed. The whole thing is designed under
the assumption that %s catches no new lines and hence only one
line.

That would be the best answer but I think it's a difficult fix and 
whatever it was might break other tests. I think it would mean replacing 
%s with something other than .+?. I'll have a look tomorrow and see if I 
can find a regex that would do - one of the things that had me confused 
for a while was that I thought the ? (non-greedy qualifier) should 
guarentee the minimum match but it doesn't work quite like that.
  
OK, so basically it is impossible to fix the current regex to match what we 
want (I had a quick chat with the PCRE author and he thinks the same).
I propose the following patch: 
http://web.ist.utl.pt/nuno.lopes/php_run_tests_%s.txt



Should be http://web.ist.utl.pt/nuno.lopes/php_run_tests_%25s.txt I
guess ;-)

  

The main changes are:
'%s' => '[^\r\n]+'
'%a' => '.+'

This means that some tests have to be fixed, but the majority passes. Most 
important is that with this patch I found some tests that had the expected 
output wrong.


So unless someone rejects this patch, I'll commit it (and update the 
http://qa.php.net/write-test.php page too).
Anyway, I don't think that a function name should be replaced by %s. The 
expected output should be as stricter as possible.



from a short review of the patch I think it's fine and a good idea. So
please go on and commit.

johannes




  


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Re: [PHP-QA] run-tests.php

[Zoe Slattery]

..PS - here is a slightly longer explanation of how I got to the idea 
that to removing  the "s" modifier would be a better fix.


I did a little more work to understand how the regex matching for %s was 
working in run-tests.php.


In the current version %s maps to .+?  This *should* match one or more 
characters but the non-greedy qualifyier should guarentee a minumum 
match. Also, the "." would not normally match an end of line unless it 
was used with a /s modifier.


I was confused buy the fact that in run-tests.php  .+? appears from my 
previous example to be acting in a greedy way and matching end of line 
characters.


More staring at the run-tests.php code revealed the explanation in this 
line of code:


if (preg_match("/^$wanted_re$/s", $output))

Here the $ at the end of wanted_re has the effect of making the match 
greedy - and the /s modifier is making "." match eol characters.
The /s modifier has been in the code ever since the --EXPECTF-- section 
was introduced and the $ was added between v1.73 and 1.74 by Derick, so 
it's been there since 2002.



Johannes Schlüter wrote:

Hi Nuno,

On Thu, 2007-09-13 at 20:20 +0100, Nuno Lopes wrote:
  

  now i get the issue. It grabbs more than one line! This means
run-tests.php has to be fixed. The whole thing is designed under
the assumption that %s catches no new lines and hence only one
line.

That would be the best answer but I think it's a difficult fix and 
whatever it was might break other tests. I think it would mean replacing 
%s with something other than .+?. I'll have a look tomorrow and see if I 
can find a regex that would do - one of the things that had me confused 
for a while was that I thought the ? (non-greedy qualifier) should 
guarentee the minimum match but it doesn't work quite like that.
  
OK, so basically it is impossible to fix the current regex to match what we 
want (I had a quick chat with the PCRE author and he thinks the same).
I propose the following patch: 
http://web.ist.utl.pt/nuno.lopes/php_run_tests_%s.txt



Should be http://web.ist.utl.pt/nuno.lopes/php_run_tests_%25s.txt I
guess ;-)

  

The main changes are:
'%s' => '[^\r\n]+'
'%a' => '.+'

This means that some tests have to be fixed, but the majority passes. Most 
important is that with this patch I found some tests that had the expected 
output wrong.


So unless someone rejects this patch, I'll commit it (and update the 
http://qa.php.net/write-test.php page too).
Anyway, I don't think that a function name should be replaced by %s. The 
expected output should be as stricter as possible.



from a short review of the patch I think it's fine and a good idea. So
please go on and commit.

johannes




  


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Re: [PHP-QA] run-tests.php

[Zoe Slattery]

Hi guys - There is an alternative  patch if you have't already committed 
this. The line:


if (preg_match("/^$wanted_re$/s", $output))

can just be changed to remove the "s" modifier, I was working on trying 
to fix up the tests and make sure that they all work. The problematic 
ones are the php_info and php_credits which rely on the %s matching over 
line endings.


Zoe



  

The main changes are:
'%s' => '[^\r\n]+'
'%a' => '.+'

This means that some tests have to be fixed, but the majority passes. Most 
important is that with this patch I found some tests that had the expected 
output wrong.


So unless someone rejects this patch, I'll commit it (and update the 
http://qa.php.net/write-test.php page too).
Anyway, I don't think that a function name should be replaced by %s. The 
expected output should be as stricter as possible.



from a short review of the patch I think it's fine and a good idea. So
please go on and commit.

johannes




  


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Re: [PHP-QA] Re: [PHP-DEV] Re: [PHP-QA] run-tests.php

[Zoe Slattery]

Marcus Boerger wrote:


I meant %s = [^\n]+?
  
Ah - I should have guessed that. Yes, it solves the problem, but then 
running make test gives an extra 7 failing tests - so I need to look at 
why they are failing and work out how to fix them.

and continuing our private discussion earlier we might want to either strip
of terminating \n prior to matching or adding \n to the expression, that'd
be:

%s = [^\n]+?\n?



However - do all OS reliably end a line with \n? (Linux is \n, Windows \r\n, 
Mac?). If the don't having a
platform-specific run-tests.php would be a fairly bad idea
  


For that reason I think it would be better to strip of the line ending prior
to do the matching.
  

Yes - I think so.

Zoe

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Re: [PHP-QA] run-tests.php

[Zoe Slattery]

Marcus Boerger wrote:

Hello Zoe,

  %s = [\n]+?

I'm not sure what this would match? I think we need something that matches any 
character but doesn't run over a line end? If so it would be more like [.\\n]+? 
- although this isn't right either (as in, I've tried both and they don't work 
:-( ). I'll see if I can find the right format

However - do all OS reliably end a line with \n? (Linux is \n, Windows \r\n, 
Mac?). If the don't having a platform-specific run-tests.php would be a fairly 
bad idea

Zoe


If that affects any test we have to fix those :-)

marcus
  



Best regards,
 Marcus


  


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Re: [PHP-QA] run-tests.php

[Zoe Slattery]

Marcus Boerger wrote:

Hello Zoe,

  now i get the issue. It grabbs more than one line! This means
run-tests.php has to be fixed. The whole thing is designed under
the assumption that %s catches no new lines and hence only one
line.


  


That would be the best answer but I think it's a difficult fix and 
whatever it was might break other tests. I think it would mean replacing 
%s with something other than .+?. I'll have a look tomorrow and see if I 
can find a regex that would do - one of the things that had me confused 
for a while was that I thought the ? (non-greedy qualifier) should 
guarentee the minimum match but it doesn't work quite like that.


Zoe

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Testing math functions

[Zoe Slattery]

Pierre wrote:

On 8/23/07, Zoe Slattery <[EMAIL PROTECTED]> wrote:
  

Hi - I've writing a few tests for the math extension and have a question
about floating point precision.

Here's a small example:

--TEST--
Test return type and value for expected input sin()
--INI--
precision = 14
--FILE--

--EXPECT--
sin 360 = float(-2.4492127076448E-16)

Is it right to test for an exact number in this way? I was slightly
suprised that I got the same number from Windows and Linux (maybe I
shouldn't be).

If not, I could write the test above to check that sin 360 is zero
plus/minus some small number - but how small?



It really depends on the architecture. Using a precision of 10 should
put you on the safe side, at least for the common architecture (intel,
amd and ppc).
  
Yes - you are right thanks. I'm currently testing that the answer is 
within +/- 1.0E-10 of what I expect - that works on the processor 
architectures that I have access to. I guess I'll find out what happens 
on the others wheh I check the test cases in :-).

--Pierre

  


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Testing math functions

[Zoe Slattery]

Richard Lynch wrote:

On Thu, August 23, 2007 10:28 am, Zoe Slattery wrote:
  

Hi - I've writing a few tests for the math extension and have a
question
about floating point precision.

Here's a small example:

--TEST--
Test return type and value for expected input sin()
--INI--
precision = 14
--FILE--

--EXPECT--
sin 360 = float(-2.4492127076448E-16)

Is it right to test for an exact number in this way? I was slightly
suprised that I got the same number from Windows and Linux (maybe I
shouldn't be).

If not, I could write the test above to check that sin 360 is zero
plus/minus some small number - but how small?



Seems to me that if 'precision' setting is supposed to affect the
output of your calculations, then you should, in theory, be able to
rely on 14 decimal places, no?...
  
Actually that isn't what "precision=14" does. Precision=14 will just 
print out the first 14 digits of the answer ignoring leading zeros.
So when I change the ini setting to precision=3, the result I get for 
sin(360) is -2.45E-16, this doesn't help if the processor rounding error 
is > 1.0 E-16 say.

I realize that's an over-simplistic answer, perhaps, and I thought the
precision only applied to BC_MATH and/or GMP calculations.
  

I'm sorry - I don't know enough about BC_MATH or GMP to comment.

I'd be surprised if it was ALWAYS right for every OS, and I strongly
suspect it's going to fail on 64-bit hardware big-time.
  
As Piere says - and I just verified by using a few different processors 
- the answer is processor dependent.

I guess the first question I have is "What precisely are you testing?"

The 'precision' setting?
  

Nope :-)

Or just that sin(2 * M_PI) is kinda sorta close to 0?
  
Yes - the most simple test of sin() or cos() functions is that they 
should give the expected result for known values.

If you just want "close to 0" then double the answer you are getting
now, and call that "close enough" :-)
  
Well - maybe. After trying a few different processors a reasonable thing 
to do seems to be to check that the answer of a floating point 
calculation is within  "plus or minus  1.0E-10"  of the expected result. 
This feels a bit arbitrary - but there doesn't seems to be a better 
solution (except maybe a lot of research into allowable rounding errors 
from each sort of processor technology and I don't think this particular 
case warrants that)

If you think 'precision' is supposed to affect it and guarantee 14
decimal places of precision, then 1.0E-14 to 1.0E14 should be your
range, no?

  

See above

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Testing math functions

[Zoe Slattery]

Hi - I've writing a few tests for the math extension and have a question 
about floating point precision.


Here's a small example:

--TEST--
Test return type and value for expected input sin()
--INI--
precision = 14
--FILE--

--EXPECT--
sin 360 = float(-2.4492127076448E-16)

Is it right to test for an exact number in this way? I was slightly 
suprised that I got the same number from Windows and Linux (maybe I 
shouldn't be).


If not, I could write the test above to check that sin 360 is zero 
plus/minus some small number - but how small?


Zoe Slattery
IBM

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Building on Windows

[Zoe Slattery]

Hi Richard - finally done this, see here: 
http://www-03.ibm.com/developerworks/blogs/page/phpblog


Please comment on the blog if you have suggestions for improvement .

Thanks

Zoe

Richard Quadling wrote:

I would be REALLY grateful if you could do the whole process with no
assumed knowledge about the compiler or special tools.

Ideally, including the dependent elements too (SDK, .NET, etc).

Regards and thanks up front!

Richard.

On 15/06/07, Zoe Slattery <[EMAIL PROTECTED]> wrote:

Rob, Pierre, Stanislav - thank you!!!

I have PHP6 building with a free development environment. Elizabeth's
tutorial + the manual covers most of it. I still have a few spurious
compiler warnings which I think I can deal with but I have a working
php.exe.

It's been such a pain getting here that I'm going to rip it all out and
do it again :-) I'll write up the steps and make them available
somewhere then I 'll probably need to recuperate by working exclusively
on Linux for an extended period.


Zoe Slattery
IBM

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php







--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Array broken in 5_2?

[Zoe Slattery]

Pierre wrote:

Hi,

If you run 'make test' using 5.2 (maybe head too, did not test), I get
a lot of errors like:

Notice: Undefined offset:  534 in
/home/pierre/cvs/php52_dev/run-tests.php on line 1682

and obviously many tests fail.

As far as I can tell, something have been broken recently. The tests
have worked ealier this week or last weekend. I'm sorry, but I have no
time to investigate further.

--Pierre

H Pierre - I've just checked out and can't reproduce on Linux (Ubuntu 
6.10), so I guess it must be fixed? Or platform specific?


Zoe Slattery
IBM

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Zend tests

[Zoe Slattery]

Hi

I started to look at the test coverage of the Zend engine this week. I 
tried to commit  a couple of small tests (see PHP6 versions below) but 
looks like my CVS ID doesn't let me commit to Zend/tests.


Would it be possible to upgrade me so that I can commit tests to 
Zend/tests as well as ext/standard/tests?


The tests I wanted to add were to check when class constants get 
initialised, I think this behaviour changed recently in PHP5 (the second 
test used to have output 'string(10) "const in B"'). The current 
behaviour looks correct and I couldn't see a test that already covers this.


selfParent_001.phpt
==

--TEST--
Test when constants are initialised. See also selfParent_002.phpt.
--FILE--
 const myDynConst = self::myConst;   


 public static function test() {
   var_dump(self::myDynConst);
 }
}

class B extends A {
 const myConst = "const in B";

 public static function test() {
   var_dump(parent::myDynConst);
 }
}

A::test();
B::test();
?>
--EXPECT--
string(10) "const in A"
string(10) "const in A"
--UEXPECT--
unicode(10) "const in A"
unicode(10) "const in A"

selfParent_002.phpt
==

--TEST--
Test when constants are initialised. See also selfParent_001.phpt.
--FILE--
 const myDynConst = self::myConst;   


 public static function test() {
   var_dump(self::myDynConst);
 }
}
class B extends A {
 const myConst = "const in B";

 public static function test() {
   var_dump(parent::myDynConst);
 }
}
B::test();
A::test();
?>
--EXPECT--
string(10) "const in A"
string(10) "const in A"
--UEXPECT--
unicode(10) "const in A"
unicode(10) "const in A"

Zoe Slattery
IBM

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Building on Windows

[Zoe Slattery]

Rob, Pierre, Stanislav - thank you!!!

I have PHP6 building with a free development environment. Elizabeth's  
tutorial + the manual covers most of it. I still have a few spurious 
compiler warnings which I think I can deal with but I have a working 
php.exe.


It's been such a pain getting here that I'm going to rip it all out and 
do it again :-) I'll write up the steps and make them available 
somewhere then I 'll probably need to recuperate by working exclusively 
on Linux for an extended period.



Zoe Slattery
IBM

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Building on Windows

[Zoe Slattery]

Hi Rob
Other than a slew of warnings, it built fine for me the other day with 
VS 2005. What version of the SDK are you using as I wonder if that 
might play a part in it?
Was your VS 2005 the Express Edition (ie the free one)? I downloaded the 
platform SDK yesterday at the same time as I got the compiler so I hoped 
:-)  that they'd work together. The Platform SDK is 2003 SP1


Zoe Slattery
IBM

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Building on Windows

[Zoe Slattery]

Hi

Has anyone built PHP6 using Visual C++ Express Edition V8? The symptoms 
I get are pretty well described in http://bugs.php.net/bug.php?id=39130 
and I'm just wondering if this was fixed for PHP5 and not PHP6?



Zoe Slattery
IBM UK

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Bugzilla category for test case bugs?

[Zoe Slattery]

Pierre wrote:

On 6/5/07, Zoe Slattery <[EMAIL PROTECTED]> wrote:


Hi all
Would it be possible to have a Bugzilla category for test case bugs?
The only alternative seems to be to raise a bug against the
implementation - which doesn't seem quite right.


Why not simply put them in the related category? For example GD for a
problem in a GD test. With an extra category, I will simply not see
any bugs related to my extension tests, even when I'm fetching the
'GD' category bugs.
That's reasonable. The reason that I asked was that all I'm doing is 
writing functional tests - so, for example - if I make a mistake in a 
functional test for "Filesystem" I don't necessarily see why the 
developer working on Filesystem should have to look at it, or spend 
their time fixing it. By the way, this came up because a colleague found 
a mistake in a test and couldn't work out the right way to raise a defect.


I also noticed that there are separate categories for Documentation 
defects - so it seemed to me to be consistent to have one for tests.


I'm happy to ask people to  with raise  them against the implementation 
- I just didn't want to cause unecessary work for the core developers. 
Of course I'm *hoping* that there won't be enough defects in the test 
cases to cause anyone major headaches :-)


What would rock is a bug "type" field. A bug can be GD and Testing
related, or Ming and Documentation.
--Pierre



--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Bugzilla category for test case bugs?

[Zoe Slattery]

Hi all
Would it be possible to have a Bugzilla category for test case bugs?  
The only alternative seems to be to raise a bug against the 
implementation - which doesn't seem quite right.


Zoe Slattery
IBM

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Re: Help with the snaps site

[Zoe Slattery]

John Mertic wrote:

Below is a link to my take on the redesign:

http://files.edin.dk/php/installer/snaps-html/index.html
(Warning - none of the links on the page go to anywhere)


+1 - looks better and makes getting the latest snap easier

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Tests - and a question

[Zoe Slattery]

Stanislav Malyshev wrote:
I think it does. If one needs to see which tests there are or are 
there any tests that test particular thing or inspect certain aspects 
of a functionality, navigating a maze of func_01234.phpt, all alike, 
is a nightmare. Of course, I could open all of them one by one and 
remember which one does what - that is if I were a being with 
unlimited lifespan, infallible memory and never felt bored - but for 
regular human it's a painful experience. So when writing a new tests 
it would be very good to give them names that make sense, it doesn't 
cost anything and helps people. Renaming old ones might be somewhat 
boring task, so not sure anybody ever would do that...
I'm happy to rename as we work through - if we can see an obvious and 
more descriptive name. Also, what about making the test directory 
structure match the functional reference for core functions? Some of it 
does at the moment (strings, array) but it would be good to put all the 
variable handling tests in ext/standard/tests/variable_handling for 
example. It's another thing we could do as we go along...


Zoe Slattery
IBM UK

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Committing failing tests?

[Zoe Slattery]

Derick Rethans wrote:

On Tue, 15 May 2007, Zoe Slattery wrote:

  

I have some updates that I'd like to make to
ext/tests/standard/array/range.phpt, the updates work fine with
unicode.semantics=off but not with unicode.semantics=on. The problem seems to
be that the warning Notices that are generated are different with unicode on.
This doesn't look like intentional behaviour but I'm guessing it might well be
something that isn't implemented yet.

So, my question is - is it reasonable/helpful to add a test in CVS that I know
will fail if it's in an area that is currently under development?



I would say that it's a good idea to commit those if you are certain 
that the "unicode" behavior is wrong. In other cases I suggest to write 
a mail to internals about it.
  


I've looked at this a bit more and have absolutely no idea whether the 
behaviour that I get is expected or not, it is certainly different.  So, 
here is a very cutdown test case that shows the difference. Note that 
it's not intended to do anything sensible - I've just pulled it out of a 
larger test caes that executes range() with all sorts of input.


This passes fine on PHP6 with unicode.semantics=off.

--TEST--
PHP6 unicode Notices different
--FILE--
echo "\n-- creating an array with low = '$low_arr' and high = 
'$high_arr' --\n";

var_dump( range( $low_arr, $high_arr ) );

--EXPECTF--

-- creating an array with low = 'ABCD' and high = 'Array' --

Notice: Array to string conversion in %s on line %d
array(1) {
 [0]=>
 string(1) "A"
}
--UEXPECTF--

-- creating an array with low = 'ABCD' and high = 'Array' --

Notice: Array to string conversion in %s on line %d
array(1) {
 [0]=>
 unicode(1) "A"
}

Here is the actual output on PHP6

Notice: Array to string conversion in /home/zoe/TESTS/range/ztest.php on 
line 5


-- creating an array with low = 'ABCD' and high = 'Array' --

Notice: Array to string conversion in /home/zoe/TESTS/range/ztest.php on 
line 6

array(1) {
 [0]=>
 unicode(1) "A"
}

So. with unicode.semantics=on an additional Notice is generated which 
appears to be caused by the echo statment. Any ideas?



regards,
Derick

  


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Committing failing tests?

[Zoe Slattery]

Hi

I have some updates that I'd like to make to 
ext/tests/standard/array/range.phpt, the updates work fine with 
unicode.semantics=off but not with unicode.semantics=on. The problem 
seems to be that the warning Notices that are generated are different 
with unicode on. This doesn't look like intentional behaviour but I'm 
guessing it might well be something that isn't implemented yet.


So, my question is - is it reasonable/helpful to add a test in CVS that 
I know will fail if it's in an area that is currently under development?


Zoe Slattery
IBM UK

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Tests - and a question

[Zoe Slattery]

Zoe Slattery wrote:

Antony Dovgal wrote:

On 05/15/2007 12:44 PM, Zoe Slattery wrote:
We're on it :-) Will probably have a few more questions on Unicode 
testing later, noticed that you are putting UEXPECT section in 
tests- but need to understand the implemenentation plan first...


We have two operation modes in PHP6 - "native" and Unicode. In these 
modes expected output of the same test may differ very much.
That's why in some cases we have to use two expected output sections 
- EXPECT(F) for native and UEXPECT(F) for Unicode mode.
Thanks - I've just been looking at this and would like to confirm that 
I've understood it. We can have both --EXEPECT(F)-- and --UEXPECT(F)-- 
in a PHPT file. In PHP5 run-tests.php will just ignore the --U.*-- 
sections. In PHP6 run-tests.php will use the --U.*-- sections if 
unicode_semantics=on and ignore the corresponding --EXPECT(F)-- 
sections. If unicode_semantics=off PHP6 will behave the same way as PHP5.


Correct? Or is it more complicated that that?
Hmm - this isn't quite right. I get the test to behave differently with 
an --INI-- section which uses unicode_semantics=on/off, but not when I 
use a php.ini file...




* There is also UEXPECTREGEX - similar to EXPECTREGEX, but used in 
Unicode mode.







--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Tests - and a question

[Zoe Slattery]

Antony Dovgal wrote:

On 05/15/2007 12:44 PM, Zoe Slattery wrote:
We're on it :-) Will probably have a few more questions on Unicode 
testing later, noticed that you are putting UEXPECT section in tests- 
but need to understand the implemenentation plan first...


We have two operation modes in PHP6 - "native" and Unicode. In these 
modes expected output of the same test may differ very much.
That's why in some cases we have to use two expected output sections - 
EXPECT(F) for native and UEXPECT(F) for Unicode mode.
Thanks - I've just been looking at this and would like to confirm that 
I've understood it. We can have both --EXEPECT(F)-- and --UEXPECT(F)-- 
in a PHPT file. In PHP5 run-tests.php will just ignore the --U.*-- 
sections. In PHP6 run-tests.php will use the --U.*-- sections if 
unicode_semantics=on and ignore the corresponding --EXPECT(F)-- 
sections. If unicode_semantics=off PHP6 will behave the same way as PHP5.


Correct? Or is it more complicated that that?


* There is also UEXPECTREGEX - similar to EXPECTREGEX, but used in 
Unicode mode.




--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Tests - and a question

[Zoe Slattery]

Antony Dovgal wrote:

On 05/15/2007 11:52 AM, SoftVirus wrote:

Hello Zoe, Marcus &  Tony,

Files names having _(basic|variation|error|object).phpt looks much 
better.

File names _[0-9]*.phpt does not give much detail about what are the
functions being testing by this testcase until one looks at the --TEST--
section of the file. It would be nice to have the name of the testcase
accomodate name of the functions that are being tested.  For example if
009.phpt is written to test basic functionality of current() and next()
functions, the better file name could be current_next_basic.phpt.


Well, aesthetically - yes, "current_next_basic" might look better to 
those who don't like numbers.
But you still have to open the test and read its --TEST-- section 
contents to learn what exactly it's testing.


That said, I'm happy with both func_b.phpt and func_basic.phpt (and 
even 009.phpt) =)

We just need more tests, doesn't matter what their names look like.

We're on it :-) Will probably have a few more questions on Unicode 
testing later, noticed that you are putting UEXPECT section in tests- 
but need to understand the implemenentation plan first...


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] Tests - and a question

[Zoe Slattery]

Marcus Boerger wrote:

Hi Marcus & Tony

Hello Zoe,

  that's pretty fine. Instead of _[bveo][0-9]?.phpt you could do
_(basic|variation|error|object)[0-9]{3}.phpt. Which is a tad bit better.
However those are just filenames helping you to find the correct test
file while developing. For that reason we didn not do so much about
generating those files. We Only made it briefly and suggested a three
digit numbering scheme. After all th ename in --TEST.. is the important
thing.
  
Thanks for the responses - you have both made the same point that it's 
the --TEST-- section that should really give a descriptive title. Marcus 
- I like your suggestion of  basic|variation etc  better than our 
current b|v etc scheme. Raghu - are you OK to switch to longer test case 
names?


I think the test names are important to some degree - if you are trying 
to look for a testcase for a particular function it is easier to look at 
file names than to have to grep for the function name in a test title. 
Is there a mapping between the functions listed in the functional ref 
and their test cases? This kind of thing would be useful to us so I was 
thinking I'd write one. It's probably lazy of me :-)  but I'd just like 
to click on a function name and go straight to a list of existing test 
cases.


You also both made the same point that shorter tests are better and I 
think we can improve here - although we have been making an effort to 
make the output readable.



Thanks for the effort into test writing!!!

best regards
marcus
  
Monday, May 14, 2007, 6:43:51 PM, you wrote:


  

Hi all



  
We added a few tests for array, string and variable handling functions 
at the weekend. As always - any feedback will be welcome.



  
One thing we had some difficulty with was keeping the test output small 
enough to be easily read. We chose to divide up tests like this:



  

   test_name_b.phptb=basic function
   test_name_v.phptv=variation
   test_name_e.phpte=error cases
   test_name_o.phpto=object specific



  
This may not be the right scheme and I'm happy to change it if anyone 
has better suggestions, although it is useful to seperate out the error 
case tests because quite a lot of the differences between php5 and php6 
seem to be in the exact form of the error message. Using this doesn't 
completely solve the problem anyway - I think some of the "v" tests will 
will end up being test_name_v1, test_name_v2 etc.




  

Thanks, Zoe
IBM UK






Best regards,
 Marcus


  


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] Tests - and a question

[Zoe Slattery]

Hi all

We added a few tests for array, string and variable handling functions 
at the weekend. As always - any feedback will be welcome.


One thing we had some difficulty with was keeping the test output small 
enough to be easily read. We chose to divide up tests like this:


  test_name_b.phptb=basic function
  test_name_v.phptv=variation
  test_name_e.phpte=error cases
  test_name_o.phpto=object specific

This may not be the right scheme and I'm happy to change it if anyone 
has better suggestions, although it is useful to seperate out the error 
case tests because quite a lot of the differences between php5 and php6 
seem to be in the exact form of the error message. Using this doesn't 
completely solve the problem anyway - I think some of the "v" tests will 
will end up being test_name_v1, test_name_v2 etc.



Thanks, Zoe
IBM UK

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] php.ini and PHPT

[Zoe Slattery]

Hi Marcus

Marcus Boerger wrote:

Then I saw you changed a hell lot of expectations from "(-%d)" to "(%i)".
While this pretty much works it drops an important detail of the expectation.
The old version was aware of the sign of the result while the new one is not.
That means that now you might fix a bug in those functions that lead to an
erroneous change in the sign and not being able to detect that. Though a lot
os code is relying on the sign (e.g. any sorting algorithm).
  
Eek - then I misunderstood what %i does, as you saw,  I had originally 
used %d - it's easy enough to change back :-)

Do b) and be as precise as possible. That means we usually even test for the
name of the testfile in the message excluding the directory and any
(back)slash.
  


Will do - thanks for the feedback, it's really helpful
  


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] GSoC

[Zoe Slattery]

Hey Ilia

This is a fantastic idea! And as for not being complex - actually I 
think it is quite a challenge. I (and a couple of colleagues) have spent 
a bit of time over the past two months trying to work out how best to do 
this,  you need to get a pretty good idea of how PHP works in order to 
be able to write the tests. This might not seem to hard to you who are 
already so familiar with the internals but - for a student  that wanted 
to get a good understanding of how PHP works - this would be a great 
project. And of course - gaining this kind of understanding in PHP means 
that you get to understand general principles that apply to other languages.


Zoe
IBM

Ilia Alshanetsky wrote:
Ca  we have a project designed to improve code coverage of PHP by 
creating .phpt tests? I may not seem like a complex project, but in 
terms of improving PHP as a language it would be exceptionally useful. 
Probably more so then any one extension that can come out of GSoC.



On 15-Mar-07, at 10:50 AM, Marcus Boerger wrote:


Hello guys,

  PHP has been chosen as a summer of code project. I suggets we put the
following on php.net. Any comments?

---
The PHP team is proud to once again participate in the Google Summer 
of Code.
We are still looking for project ideas of interested students. In 
case you
want to spend your summer with your favorite open source project, 
PHP, and get
some money for adding an interesting project to it, you should 
contact us at
[EMAIL PROTECTED] The deadline for submitting ideas is the 24th of 
March 2007.
The current list of ideas can be seen here http://php.net/ideas. If 
you have

not heard of the Google Summer of Code, then have a look here
http://code.google.com/soc.
---

Best regards,
 Marcus

--PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php



Ilia Alshanetsky

--PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php




--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] php.ini and PHPT

[Zoe Slattery]

Hi Marcus

Thanks - that answers the question ...and test cases are on their way:-)

I have just committed 3 tests in the PHP 6 stream (strcasecmp, strcmp, 
strlen) and would very much appreciate any feedback on them. We are 
aiming for both line coverage and functional coverage, it would be good 
to know if we are missing obvious things.


For the future I think we will probably split out the test cases into 
good-path tests and error-path tests, putting both into one test case 
has resulted in the test cases being quite big.


I have one specific question though, I notice that the string that 
follows Warning: is changing in PHP6, for example in PHP5 one would expect:


Warning: Wrong parameter count for strcasecmp() in blah.php on line 3

In PHP6:

Warning: strcasecmp() expects exactly 2 parameters, 0 given in blah.php 
on line 3


The PHP6 message is clearly an improvement and the test cases that I 
have just put in PHP6 check the Warning: like this:


--EXPECTF--
Warning: strcasecmp() expects exactly 2 parameters, 0 given in %s on line %d

But of course, this will only pass on PHP6.  We can deal with this in 
two ways:


(a) Use "Warning: %s" in the EXPECTF section. This will pass test on 
PHP5 and PHP6 but it won't check that PHP is giving the right error 
mesage - ie "exactly 2 parameters" as opposed to "exactly 42 parameters"


(b) We can have different test cases for PHP5 and PHP6.

I don't particularly like either of these, but have a slight preference 
for (b) because  it's a more satisfactory test of the implementation.


What's your view? Or anyone else's?


Marcus Boerger wrote:

Hello Zoe,

  we decided to go the --INI-- route to test as many configurations as
possible. And only for specific tests change specific ini settings as
necessary. If one usees phpt as a test framework for his on web stuff
then it might be a good idea to do as you suggested. You would simply
do "php run-tests.php -c ". Hope this satisfies you :-)

best regards
marcus

p.s.: Looking forward for new tests from your side

Wednesday, March 14, 2007, 4:39:43 PM, you wrote:

  

Hi - I wonder if anyone could help me with a testing question?



  
We have written a few additional test cases for string functions which I 
was about to check-in, then I noticed that the test cases pass OK if I 
use the php.ini-dist file and fail if I don't specify an file (ie use 
all default settings).



  
The difference turns out to the be the use of precision =12 in the 
php.ini-dist, rather than the the default value of 14.



  
It's easy enough to work around this and ensure that test cases pass in 
both environments with an --INI-- section in the test case, however - I 
wondered if there would be any value in having a php-ini file that was 
explicitly used for tests?



  
We had been using the php.ini-dist file because it's slightly easier to 
be precise about the test environment if you can say what you are using 
rather than saying "I'm using the default values". I also find it rather 
too easy to get the wrong php.ini file accidentally :-( so specifying 
one is easier, even if it only contains default values.



  

Zoe
IBM






Best regards,
 Marcus


  


--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] php.ini and PHPT

[Zoe Slattery]

Hi - I wonder if anyone could help me with a testing question?

We have written a few additional test cases for string functions which I 
was about to check-in, then I noticed that the test cases pass OK if I 
use the php.ini-dist file and fail if I don't specify an file (ie use 
all default settings).


The difference turns out to the be the use of precision =12 in the 
php.ini-dist, rather than the the default value of 14.


It's easy enough to work around this and ensure that test cases pass in 
both environments with an --INI-- section in the test case, however - I 
wondered if there would be any value in having a php-ini file that was 
explicitly used for tests?


We had been using the php.ini-dist file because it's slightly easier to 
be precise about the test environment if you can say what you are using 
rather than saying "I'm using the default values". I also find it rather 
too easy to get the wrong php.ini file accidentally :-( so specifying 
one is easier, even if it only contains default values.


Zoe
IBM

--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

[PHP-DEV] CVS Account Request: zoe

[Zoe Slattery]

I've been looking at the php tests and would like to be able to contribute more 
tests for basic PHP functions to help increase the test coverage and prevent 
regressions when code is changed. I don't know how granular the CVS access 
controls are so am not sure what is possible. I've been working from 
gcov.phpt.net and looking for gaps in coverage - so far just 
/ext/standard/levenshtein.c and ext/standard/assert.c. It would also be helpful 
if I could maintain update the documentation if there is agreement that 
something is working as designed but not as documented.This was discussed 
briefly with Andi G and Zeev who suggested applying.

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php