Re: [PHP-DEV] Re: [APC-DEV] Re: [PHP-DEV] [RFC] APC/PHP Lazy Loading
Johannes Schlüter wrote: On Fri, 2009-03-13 at 18:27 +0100, Kalle Sommer Nielsen wrote: I must admit that I think it would be bad to delay 5.3 more and have another beta because of this, Well, according to Dmitry (didn't test it myself) this doesn't bring real improvements so I don't think it's needed. I want to emphasize that this does provide real performance increases, it's just going to depend heavily on the code structure. Some projects may see gains, others may not. Per my original post I'm interested in getting more testing done on various projects to see how general the gains are and what I can do to improve them. To also re-iterate my previous emails I do agree that this probably isn't the best time to include this in 5.3, but would of course work to help get it in if the majority wanted to see that happen. Considering the merge errors that Dmitry found I think it would be preferred to get this tested more before it's included in a major release, as I had originally intended. I appreciate the code changes Dmitry has contributed and his initial testing. however if possible I would be all for having APC in RC1 next week :) This was asked on IRC multiple times and always said something along the lines "talk to the current APC maintainers and if they think it's worth propose to internals" as the current APC maintainers can judge stability and risks better and have a better knowledge about missing 5.3 support. But nobody seemed t be that interested. By looking at the bug tracker I see 63 open APC bugs including missing support for 5.3 features (http://pecl.php.net/bugs/bug.php?id=15901) and some crashes from that I think APC benefits from an independent release schedule. Additionally APC doesn't offer core functionality and will need configuration to be useful (temp path, shm method, segment sizes, ...) so I don't think the additional installation step is that much of additional trouble. I'm trying to get the majority of the APC bugs resolved, but I think this will take some time. There's also some new and undocumented functionality added to APC as well as some major changes I plan to create in an APC-4.0 branch. So I don't think this would be the best time to integrate it into core, especially considering the late notice. -shire -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: [APC-DEV] Re: [PHP-DEV] [RFC] APC/PHP Lazy Loading
On Fri, 2009-03-13 at 18:27 +0100, Kalle Sommer Nielsen wrote: > I must admit that I think it would be bad to delay 5.3 more and have > another beta because of this, Well, according to Dmitry (didn't test it myself) this doesn't bring real improvements so I don't think it's needed. > however if possible I would be all for > having APC in RC1 next week :) This was asked on IRC multiple times and always said something along the lines "talk to the current APC maintainers and if they think it's worth propose to internals" as the current APC maintainers can judge stability and risks better and have a better knowledge about missing 5.3 support. But nobody seemed t be that interested. By looking at the bug tracker I see 63 open APC bugs including missing support for 5.3 features (http://pecl.php.net/bugs/bug.php?id=15901) and some crashes from that I think APC benefits from an independent release schedule. Additionally APC doesn't offer core functionality and will need configuration to be useful (temp path, shm method, segment sizes, ...) so I don't think the additional installation step is that much of additional trouble. johannes -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: [APC-DEV] Re: [PHP-DEV] [RFC] APC/PHP Lazy Loading
Hi 2009/3/13 Pierre Joye : > hi, > > Then I would rather have a beta2 and not a RC. This change while being > great may require more tweaks, and as Marcus suggested, we could push > APC in too while being at it. I'm all for it (both :). I must admit that I think it would be bad to delay 5.3 more and have another beta because of this, however if possible I would be all for having APC in RC1 next week :) > > Cheers, > -- Kalle Sommer Nielsen ka...@php.net -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: [APC-DEV] Re: [PHP-DEV] [RFC] APC/PHP Lazy Loading
hi, Then I would rather have a beta2 and not a RC. This change while being great may require more tweaks, and as Marcus suggested, we could push APC in too while being at it. I'm all for it (both :). Cheers, On Wed, Mar 11, 2009 at 5:13 PM, Lukas Kahwe Smith wrote: > > On 11.03.2009, at 17:10, Rasmus Lerdorf wrote: >>> >>> Anyway, it's very good job and 20-30% speedup on some real-life >>> applications makes sense to include it into 5.3 (from my point of view). >> >> Makes sense to me as well, especially since I don't see any drawbacks >> for non-accelerated scripts. >> >> I also think some of those applications that didn't show any gains could >> be trivially modified to make use of it. Moving from conditionally >> loaded stuff to lazy-loaded is likely to speed things up. Depending of >> course on how granular the conditional loading is. > > > Can we get this patch to release quality by this weekend? > So that people can test it on Monday/Tuesday ahead of RC1? > > regards, > Lukas Kahwe Smith > m...@pooteeweet.org > > > > > -- > PHP Internals - PHP Runtime Development Mailing List > To unsubscribe, visit: http://www.php.net/unsub.php > > -- Pierre http://blog.thepimp.net | http://www.libgd.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
[PHP-DEV] Re: [APC-DEV] Re: [PHP-DEV] [RFC] APC/PHP Lazy Loading
Hi,
I've fixed all the issues I found in the original patch (attached).
However then I realized that speed-up was caused by bugs that leaded to
render pages in improper way. After fix the speed-up disappear. :(
So now I don't see any reason to include it into 5.3.
Thanks. Dmitry.
Lukas Kahwe Smith wrote:
On 11.03.2009, at 17:10, Rasmus Lerdorf wrote:
Anyway, it's very good job and 20-30% speedup on some real-life
applications makes sense to include it into 5.3 (from my point of view).
Makes sense to me as well, especially since I don't see any drawbacks
for non-accelerated scripts.
I also think some of those applications that didn't show any gains could
be trivially modified to make use of it. Moving from conditionally
loaded stuff to lazy-loaded is likely to speed things up. Depending of
course on how granular the conditional loading is.
Can we get this patch to release quality by this weekend?
So that people can test it on Monday/Tuesday ahead of RC1?
regards,
Lukas Kahwe Smith
m...@pooteeweet.org
Index: Zend/zend.c
===
RCS file: /repository/ZendEngine2/zend.c,v
retrieving revision 1.308.2.12.2.35.2.29
diff -u -p -d -r1.308.2.12.2.35.2.29 zend.c
--- Zend/zend.c 18 Feb 2009 10:55:08 - 1.308.2.12.2.35.2.29
+++ Zend/zend.c 13 Mar 2009 13:46:07 -
@@ -550,6 +550,10 @@ static void executor_globals_ctor(zend_e
EG(current_module) = NULL;
EG(exit_status) = 0;
EG(active) = 0;
+ EG(lookup_function_hook) = NULL;
+ EG(lookup_class_hook) = NULL;
+ EG(defined_function_hook) = NULL;
+ EG(declared_class_hook) = NULL;
}
/* }}} */
Index: Zend/zend_API.c
===
RCS file: /repository/ZendEngine2/zend_API.c,v
retrieving revision 1.296.2.27.2.34.2.60
diff -u -p -d -r1.296.2.27.2.34.2.60 zend_API.c
--- Zend/zend_API.c 14 Jan 2009 11:56:08 - 1.296.2.27.2.34.2.60
+++ Zend/zend_API.c 13 Mar 2009 13:46:07 -
@@ -2180,7 +2180,7 @@ ZEND_API zend_class_entry *zend_register
if (!parent_ce && parent_name) {
zend_class_entry **pce;
- if (zend_hash_find(CG(class_table), parent_name,
strlen(parent_name)+1, (void **) &pce)==FAILURE) {
+ if (zend_find_class(CG(class_table), parent_name,
strlen(parent_name)+1, &pce TSRMLS_CC)==FAILURE) {
return NULL;
} else {
parent_ce = *pce;
@@ -2228,14 +2228,20 @@ ZEND_API zend_class_entry *zend_register
ZEND_API int zend_register_class_alias_ex(const char *name, int name_len,
zend_class_entry *ce TSRMLS_DC) /* {{{ */
{
char *lcname = zend_str_tolower_dup(name, name_len);
- int ret;
+ ulong hash = zend_inline_hash_func(lcname, name_len+1);
+ zend_class_entry **pce;
- ret = zend_hash_add(CG(class_table), lcname, name_len+1, &ce,
sizeof(zend_class_entry *), NULL);
- efree(lcname);
- if (ret == SUCCESS) {
- ce->refcount++;
+ if((EG(lookup_class_hook) &&
+ /* Check if we have a same-name class already exsiting in our
hook, then fail as we would normally below.
+* This could happen if we have two classes with the same name,
and one of them is being bound at run-time */
+ EG(lookup_class_hook)(lcname, name_len+1, hash, &pce) == SUCCESS) ||
+ zend_hash_quick_add(CG(class_table), lcname, name_len+1, hash, &ce,
sizeof(zend_class_entry *), NULL) == FAILURE) {
+ efree(lcname);
+ return FAILURE;
}
- return ret;
+ efree(lcname);
+ ce->refcount++;
+ return SUCCESS;
}
/* }}} */
@@ -2412,7 +2418,7 @@ static int zend_is_callable_check_func(i
}
/* Check if function with given name exists.
* This may be a compound name that includes namespace name */
- if (zend_hash_find(EG(function_table), lmname, mlen+1,
(void**)&fcc->function_handler) == SUCCESS) {
+ if (zend_find_function(EG(function_table), lmname, mlen+1,
&fcc->function_handler TSRMLS_CC) == SUCCESS) {
efree(lmname);
return 1;
}
@@ -3552,6 +3558,86 @@ ZEND_API void zend_restore_error_handlin
}
/* }}} */
+ZEND_API int zend_quick_find_function(HashTable *function_table, const char
*name, int len, zend_uint h, zend_function **fe TSRMLS_DC) /* {{{ */
+{
+ if (zend_hash_quick_find(function_table, name, len, h, (void**)fe) ==
SUCCESS) {
+ return SUCCESS;
+ } else if (EG(lookup_function_hook)) {
+ return EG(lookup_function_hook)(name, len, h, fe);
+ }
+ return FAILURE;
+}
+/* }}} */
+
+ZEND_API int zend_find_function(HashTable *function_table, const char *name,
int len, zend_function **fe TSRMLS_DC) /* {{{ */
+{
+ ulong h = zend_inline_hash_fu
Re: [PHP-DEV] Re: [APC-DEV] Re: [PHP-DEV] [RFC] APC/PHP Lazy Loading
On 11.03.2009, at 20:58, shire wrote: Lukas Kahwe Smith wrote: Can we get this patch to release quality by this weekend? So that people can test it on Monday/Tuesday ahead of RC1? I don't see this being a problem, I do have a few items I'd like to point out for feedback/suggestions: 1) Currently it doesn't support method level lazy loading, it's probably advisable to wait and include this later, but if everyone thinks it's significant enough we could probably add support for that. I'm not sure on all the details this involves, perhaps someone familiar with method calls could suggest difficulty and/or optimized ways to do the lookups. 2) Currently I've inserted these hook calls into everywhere we call/ lookup classes. This has the downside that any extension wanting to mess with the function table, lookup entries, etc will need to be aware of the callback hooks. I think a better architecture is to create an API for function table and class table operations. Perhaps this could be done later if we want this likely more stable version in 5.3, and perhaps I'm overstating the significance of other functions doing these sort of things and not being aware of this new feature. (I believe dmitry mentions several extenions that need changes to support this). On the upside not creating an API means existing code will still work if they don't use lazy loading. ;-) 3) The largest portion of time currently is simply adding dummy entries to the lazy hash tables so we can detect name collisions and availability, my next goal is to improve the performance of this by either creating a faster copy of the entries or determine some better method of performing lookups/marking functions as available (something like lookups directly into the APC shared memory segment). Just putting this out there in case anyone has some interesting suggestions. I think all the above 3 items don't necessarily need to be done to have this work in 5.3 (and #3 is pretty much APC/opcode cache centric) and might cause unecessary complication for an initial support of this, but what does everyone else think? Hmm, thanks for your open assessment here. What you are saying does make me wonder if we really should push this into 5.3. Even if any changes we need to do can eventually be handled by APC, I do think that there will be other extension authors that would also suffer from us messing around with these internals all too much in every release. So maybe we should wait until the next bigger step before introducing this. Other people can apply the patch themselves if they really need the performance in the mean time .. regards, Lukas Kahwe Smith m...@pooteeweet.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
[PHP-DEV] Re: [APC-DEV] Re: [PHP-DEV] Re: [APC-DEV] Re: [PHP-DEV] [RFC] APC/PHP Lazy Loading
Hi! there is no such thing. Let's either do it now or go for 5.4. Can we please stop trying new big features each time we approach RC? 5.3 is not last PHP version ever, and it's long overdue. Can't we just release it without putting more and more potentially unstable changes into it, and then discuss other stuff for 5.4, 6.0 and whatever comes in orderly manner? -- Stanislav Malyshev, Zend Software Architect s...@zend.com http://www.zend.com/ (408)253-8829 MSN: s...@zend.com -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
[PHP-DEV] Re: [APC-DEV] Re: [PHP-DEV] Re: [APC-DEV] Re: [PHP-DEV] [RFC] APC/PHP Lazy Loading
Hello Lukas, Wednesday, March 11, 2009, 11:15:23 PM, you wrote: > On 11.03.2009, at 19:55, Marcus Boerger wrote: >> Last but not least, Lukas, what happened, to putting APC into core? > That was planned for PHP 6.0. there is no such thing. Let's either do it now or go for 5.4. > regards, > Lukas Kahwe Smith > m...@pooteeweet.org Best regards, Marcus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: [APC-DEV] Re: [PHP-DEV] [RFC] APC/PHP Lazy Loading
On 11.03.2009, at 19:55, Marcus Boerger wrote: Last but not least, Lukas, what happened, to putting APC into core? That was planned for PHP 6.0. regards, Lukas Kahwe Smith m...@pooteeweet.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
[PHP-DEV] Re: [APC-DEV] Re: [PHP-DEV] [RFC] APC/PHP Lazy Loading
Lukas Kahwe Smith wrote: Can we get this patch to release quality by this weekend? So that people can test it on Monday/Tuesday ahead of RC1? I don't see this being a problem, I do have a few items I'd like to point out for feedback/suggestions: 1) Currently it doesn't support method level lazy loading, it's probably advisable to wait and include this later, but if everyone thinks it's significant enough we could probably add support for that. I'm not sure on all the details this involves, perhaps someone familiar with method calls could suggest difficulty and/or optimized ways to do the lookups. 2) Currently I've inserted these hook calls into everywhere we call/lookup classes. This has the downside that any extension wanting to mess with the function table, lookup entries, etc will need to be aware of the callback hooks. I think a better architecture is to create an API for function table and class table operations. Perhaps this could be done later if we want this likely more stable version in 5.3, and perhaps I'm overstating the significance of other functions doing these sort of things and not being aware of this new feature. (I believe dmitry mentions several extenions that need changes to support this). On the upside not creating an API means existing code will still work if they don't use lazy loading. ;-) 3) The largest portion of time currently is simply adding dummy entries to the lazy hash tables so we can detect name collisions and availability, my next goal is to improve the performance of this by either creating a faster copy of the entries or determine some better method of performing lookups/marking functions as available (something like lookups directly into the APC shared memory segment). Just putting this out there in case anyone has some interesting suggestions. I think all the above 3 items don't necessarily need to be done to have this work in 5.3 (and #3 is pretty much APC/opcode cache centric) and might cause unecessary complication for an initial support of this, but what does everyone else think? Regarding a couple of Dmitry's suggestions: 1) I would prefer to add additional hash_value argument into lookup_function_hook() and lookup_class_hook to prevent multiple calculation. I'm upset I didn't do that in the first place ;-) 2) function_exists() should use lookup_function_hook(). 3) interface_exists() and class_alias() should use lookup_class_hook(). I'm pretty sure I already have support for this, it may have been lost while porting it over, I'll double check. I'll follow up with Dmitry on getting these and any other items taken care of, thanks! -shire -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
Re: [PHP-DEV] Re: [APC-DEV] Re: [PHP-DEV] [RFC] APC/PHP Lazy Loading
Hello Lukas, Wednesday, March 11, 2009, 5:10:57 PM, you wrote: > Dmitry Stogov wrote: >> Hi Shire, >> >> I run patched APC on a number of real-life applications and got more >> than 30% speedup on XOOPS (99 req/sec instead of 60%) and 20% on >> ZendFramework (41 req/sec instead of 32), however most applications >> (drupal, qdig, typo3, wordpress) didn't show significant speed >> difference. As was expected the patch doesn't affects PHP without APC or >> with APC and lazy loading disabled. >> >> I also got APC warning with Zend Framewoek based blog application, but I >> didn't try to look deeper. >> >> [Wed Mar 11 17:53:02 2009] [apc-warning] [Wed Mar 11 17:53:02 2009] >> [apc-warning] apc_lookup_class_hook: could not install blogrow in >> /var/www/html/bench/fw/ZendFramework-1.5.0RC3/library/Zend/Loader.php on >> line 86 >> >> I didn't look careful into APC code, just into PHP patch and I see the >> following issues: >> >> 1) I would prefer to add additional hash_value argument into >> lookup_function_hook() and lookup_class_hook to prevent multiple >> calculation. >> >> 2) function_exists() should use lookup_function_hook(). >> >> 3) interface_exists() and class_alias() should use lookup_class_hook(). >> >> 4) ext/soap, ext/reflection, ext/wddx and ext/spl autoload support >> >> Anyway, it's very good job and 20-30% speedup on some real-life >> applications makes sense to include it into 5.3 (from my point of view). > Makes sense to me as well, especially since I don't see any drawbacks > for non-accelerated scripts. > I also think some of those applications that didn't show any gains could > be trivially modified to make use of it. Moving from conditionally > loaded stuff to lazy-loaded is likely to speed things up. Depending of > course on how granular the conditional loading is. Right, basically the patch adresses what we say at conferences. Autload can do an amazing job but also harm you. The patch on the other hand somehow combines the advantages of both worlds. You can be explicit because you know it better and for that you do not get any punishment from Autoload execution time. And then again, you don't get punished by including too much. Becuase after all the compiler knows it even better. And of course, given that some people gain over 20%, I don't see why we even need to discuss putting this in. Last but not least, Lukas, what happened, to putting APC into core? marcus Best regards, Marcus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
[PHP-DEV] Re: [APC-DEV] Re: [PHP-DEV] [RFC] APC/PHP Lazy Loading
Lukas Kahwe Smith wrote: On 11.03.2009, at 17:10, Rasmus Lerdorf wrote: Anyway, it's very good job and 20-30% speedup on some real-life applications makes sense to include it into 5.3 (from my point of view). Makes sense to me as well, especially since I don't see any drawbacks for non-accelerated scripts. I also think some of those applications that didn't show any gains could be trivially modified to make use of it. Moving from conditionally loaded stuff to lazy-loaded is likely to speed things up. Depending of course on how granular the conditional loading is. Can we get this patch to release quality by this weekend? I think it's possible. I could help with it. Thanks. Dmitry. So that people can test it on Monday/Tuesday ahead of RC1? regards, Lukas Kahwe Smith m...@pooteeweet.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
[PHP-DEV] Re: [APC-DEV] Re: [PHP-DEV] [RFC] APC/PHP Lazy Loading
On 11.03.2009, at 17:10, Rasmus Lerdorf wrote: Anyway, it's very good job and 20-30% speedup on some real-life applications makes sense to include it into 5.3 (from my point of view). Makes sense to me as well, especially since I don't see any drawbacks for non-accelerated scripts. I also think some of those applications that didn't show any gains could be trivially modified to make use of it. Moving from conditionally loaded stuff to lazy-loaded is likely to speed things up. Depending of course on how granular the conditional loading is. Can we get this patch to release quality by this weekend? So that people can test it on Monday/Tuesday ahead of RC1? regards, Lukas Kahwe Smith m...@pooteeweet.org -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
[PHP-DEV] Re: [APC-DEV] Re: [PHP-DEV] [RFC] APC/PHP Lazy Loading
Dmitry Stogov wrote: > Hi Shire, > > I run patched APC on a number of real-life applications and got more > than 30% speedup on XOOPS (99 req/sec instead of 60%) and 20% on > ZendFramework (41 req/sec instead of 32), however most applications > (drupal, qdig, typo3, wordpress) didn't show significant speed > difference. As was expected the patch doesn't affects PHP without APC or > with APC and lazy loading disabled. > > I also got APC warning with Zend Framewoek based blog application, but I > didn't try to look deeper. > > [Wed Mar 11 17:53:02 2009] [apc-warning] [Wed Mar 11 17:53:02 2009] > [apc-warning] apc_lookup_class_hook: could not install blogrow in > /var/www/html/bench/fw/ZendFramework-1.5.0RC3/library/Zend/Loader.php on > line 86 > > I didn't look careful into APC code, just into PHP patch and I see the > following issues: > > 1) I would prefer to add additional hash_value argument into > lookup_function_hook() and lookup_class_hook to prevent multiple > calculation. > > 2) function_exists() should use lookup_function_hook(). > > 3) interface_exists() and class_alias() should use lookup_class_hook(). > > 4) ext/soap, ext/reflection, ext/wddx and ext/spl autoload support > > Anyway, it's very good job and 20-30% speedup on some real-life > applications makes sense to include it into 5.3 (from my point of view). Makes sense to me as well, especially since I don't see any drawbacks for non-accelerated scripts. I also think some of those applications that didn't show any gains could be trivially modified to make use of it. Moving from conditionally loaded stuff to lazy-loaded is likely to speed things up. Depending of course on how granular the conditional loading is. -Rasmus -- PHP Internals - PHP Runtime Development Mailing List To unsubscribe, visit: http://www.php.net/unsub.php
