> -----Original Message----- > From: Benjamin Eberlei [mailto:kont...@beberlei.de] > Sent: Friday, February 27, 2015 1:19 PM > To: Dmitry Stogov > Cc: Zeev Suraski; PHP internals > Subject: Re: [PHP-DEV] Coercive STH - some real world tests and updated > RFC > > > > On Fri, Feb 27, 2015 at 9:44 AM, Dmitry Stogov <dmi...@zend.com> wrote: > > > I've added the link to the patch > > https://github.com/php/php-src/pull/1125/files > > > Thanks! > > First, the necessary PHPUnit changes (dev-master) to avoid errors: > > https://gist.github.com/beberlei/8a33ae940829f1186da2 > > > - Doctrine DBAL testsuite: 8 failures > - Doctrine ORM: Crashes unrelated after half the tests, but has about > 30-50% > failures like Symfony2 > - Symfony2 Testsuite: 6215 failures > - Twig: Tests: 1108, Assertions: 1616, Errors: 125.
Thanks for the tests! We'll look into those. > Now probably many of the failures are related to few code paths that need > fixing, however I have to find out that and will be a lot of work. But > this is the > good PHP code! Judging by the fact WordPress, Magento and Drupal appear a lot cleaner - you have to wonder..? :) But on a more serious note, I don't think we should assume just about anything from the test suite as reflecting on real world app behavior. I'm not familiar with the Symfony test suite (yet), but I do know that the Symfony skeleton app ran cleanly without a single warning. Is it possible it's testing a lot of things with garbage input, as you would in unit tests, as the our own PHP test suite does? Those account for most of the failures. Also take into account that a project with that many tests (and specifically, failures) is probably not going to have just one or two people responsible for it, but more. It's not that you personally would have to fix Symfony, there are at least a dozen if not dozen people that would help out. > For untested or older PHP code (and yes there is alot out there) i now > have to > collect the errors in production to find all the occurances. This is > nothing i > can just do whenever I want, I need a process to collect and fix this over > time. Now every company needs this process for every project they have out > there. I wish that were true, but in reality, migration is painfully slow and judging by the numbers, doesn't happen - more often than happening. The 'good' companies have ~5yrs after PHP 7 comes out and before PHP 7.x will no longer be supported (and that's according to my hope for a fast PHP 8 timeline; it may be longer as you suggested). Would all projects migrate by that timeline? Let's make a simpler question - would all projects migrate to PHP 7 by 2017, the year 5.6 theoretically stops being supported? We all know the answer to that. The good companies, the security conscious ones that do keep up with new versions, will likely be willing to invest the effort of fixing the code (or hiring someone to do it for them), especially as it'll help them make their apps more robust/secure. And especially as - at least so far - it seems that real world apps have not even a handful of issues per page. All that said, I think we must understand the huge gap between the very high degree of test failures you're seeing in test suites, and the almost none we're seeing in real world apps. If it's similar to the PHP test suite, it's a non-issue, and even if it would take some work to fix the unit tests, it's well worth it. If it's legit, but as you're guessing, may just be several common paths of code that will do away with most of the failures - it's still worth it. Investing several days over the course of several years is great ROI for the value it brings. If, however, the test suite does find a multitude of many different issues - it may bring me to reconsider the RFC, as I told Anthony a few days ago - especially if the signal to noise ratio would be bad. After the results we've seen with real world apps I find it hard to believe, but it's certainly a possible scenario. At the end of the day, it boils down to how much real work do we think will be required to update real world apps out there. Let's also not set unreasonable bars for this change compared to other compatibility breakages we've done (and are stil doing) over the years - again, most of which didn't bring any tangible value to developers (which this change does, a lot). > And the typical agency has hundrets/thousands of drupal, typo3, > wordpress installations out there Correct, but would you have to patch each one separately? You'd have to install updated core for Drupal, Typo3, WP - which you commonly have to do anyway if you care about security. Then there'd be some work to get your common custom modules that you developed update - between the two of those, that should account for most of the code. It's not as if you're going to start with each deployment from scratch, is it? Anyway, thanks again for running the tests - we'll definitely need to understand that better. Thanks, Zeev -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
