Re: [PHP-DEV] [Request for Comment] Loading dynamic libraries with cross dependencies.

[Sara Golemon]

On Mon, Mar 6, 2017 at 3:50 AM, Joe Watkins  wrote:
> FWIW, I wouldn't mind improvements in this area (apcu and the apc bc layer
> suffer from this).
>
apcu_bc is exactly what precipitated this discussion for me. :)

> Anything from a better error message, all the way to removing the necessary
> load order limitation would be nice.
>
I've gotten enough encouragement to merit writing up a formal RFC and
work on a prototype.  Minimum goals are going to be: 1) Better error
message when ordering/dep/conflict is wrong. 2) Programmatic interface
to get dep order without having to first know dep order.  and as a
stetch goal: 3) make dep resolution for php.ini listed exts
automatically shuffle loads as needed.

-Sara

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] [Request for Comment] Loading dynamic libraries with cross dependencies.

[Joe Watkins]

Morning all,

FWIW, I wouldn't mind improvements in this area (apcu and the apc bc layer
suffer from this).

Anything from a better error message, all the way to removing the necessary
load order limitation would be nice.

Cheers
Joe

On Sat, Mar 4, 2017 at 10:23 AM, Derick Rethans  wrote:

> On Fri, 3 Mar 2017, Sara Golemon wrote:
>
> > On Fri, Mar 3, 2017 at 5:47 AM, Derick Rethans  wrote:
> > >> The average PHP extension is dependent only on the PHP runtime it was
> > >> built for and probably one or more 3rd party libraries.  However, a
> > >> special class of extensions (such as PDO drivers and much of the XML
> > >> family) have dependencies on other PHP extensions which may (or may
> > >> not) be preset at the time they are loaded in via ini settings and/or
> > >> dl().
> > >
> > > See also:
> > > https://derickrethans.nl/undefined-symbol.html
> > >
> > Exactly!  The same underlying issue at work.
> >
> > Would I be correct in assuming there's a desire to resolve this issue?
> > Or are we considering "know your dependencies and configure for them"
> > to be enough?
>
> Getting this addressed would certainly be great. For the MongoDB
> extension we've worked around it so that we can at least show a
> reasonable error message though.
>
> cheers,
> Derick
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>
>

Re: [PHP-DEV] [Request for Comment] Loading dynamic libraries with cross dependencies.

[Derick Rethans]

On Fri, 3 Mar 2017, Sara Golemon wrote:

> On Fri, Mar 3, 2017 at 5:47 AM, Derick Rethans  wrote:
> >> The average PHP extension is dependent only on the PHP runtime it was
> >> built for and probably one or more 3rd party libraries.  However, a
> >> special class of extensions (such as PDO drivers and much of the XML
> >> family) have dependencies on other PHP extensions which may (or may
> >> not) be preset at the time they are loaded in via ini settings and/or
> >> dl().
> >
> > See also:
> > https://derickrethans.nl/undefined-symbol.html
> >
> Exactly!  The same underlying issue at work.
> 
> Would I be correct in assuming there's a desire to resolve this issue?
> Or are we considering "know your dependencies and configure for them"
> to be enough?

Getting this addressed would certainly be great. For the MongoDB 
extension we've worked around it so that we can at least show a 
reasonable error message though.

cheers,
Derick

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] [Request for Comment] Loading dynamic libraries with cross dependencies.

[Sara Golemon]

>> Would I be correct in assuming there's a desire to resolve this issue?
>> Or are we considering "know your dependencies and configure for them"
>> to be enough?
>>
>
> Configuration should be dead simple, having this load order requirements
> is not dead simple. It is actually very complicated because it is
> sometimes not trivial to find out what extension need to be loaded in
> which order. Especially (never encountered it with PHP extensions but
> other stuff) if you have many dependencies that need to be loaded in the
> right order.
>
> Definitely should be addressed!
>
I certainly agree, but I didn't want to jump in too deep given that
this seems to be a relatively rare occurrence and sometimes a little
extra documentation can be a lot more valuable than code.

-Sara

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] [Request for Comment] Loading dynamic libraries with cross dependencies.

[Fleshgrinder]

On 3/3/2017 8:14 PM, Sara Golemon wrote:
> On Fri, Mar 3, 2017 at 5:47 AM, Derick Rethans  wrote:
>> See also:
>> https://derickrethans.nl/undefined-symbol.html
>>
> Exactly!  The same underlying issue at work.
> 
> Would I be correct in assuming there's a desire to resolve this issue?
> Or are we considering "know your dependencies and configure for them"
> to be enough?
> 
> -Sara
> 

Configuration should be dead simple, having this load order requirements
is not dead simple. It is actually very complicated because it is
sometimes not trivial to find out what extension need to be loaded in
which order. Especially (never encountered it with PHP extensions but
other stuff) if you have many dependencies that need to be loaded in the
right order.

Definitely should be addressed!

-- 
Richard "Fleshgrinder" Fussenegger

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] [Request for Comment] Loading dynamic libraries with cross dependencies.

[Sara Golemon]

On Fri, Mar 3, 2017 at 5:47 AM, Derick Rethans  wrote:
>> The average PHP extension is dependent only on the PHP runtime it was
>> built for and probably one or more 3rd party libraries.  However, a
>> special class of extensions (such as PDO drivers and much of the XML
>> family) have dependencies on other PHP extensions which may (or may
>> not) be preset at the time they are loaded in via ini settings and/or
>> dl().
>
> See also:
> https://derickrethans.nl/undefined-symbol.html
>
Exactly!  The same underlying issue at work.

Would I be correct in assuming there's a desire to resolve this issue?
Or are we considering "know your dependencies and configure for them"
to be enough?

-Sara

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] [Request for Comment] Loading dynamic libraries with cross dependencies.

[Derick Rethans]

On Tue, 28 Feb 2017, Sara Golemon wrote:

> The average PHP extension is dependent only on the PHP runtime it was
> built for and probably one or more 3rd party libraries.  However, a
> special class of extensions (such as PDO drivers and much of the XML
> family) have dependencies on other PHP extensions which may (or may
> not) be preset at the time they are loaded in via ini settings and/or
> dl().

See also:
https://derickrethans.nl/undefined-symbol.html

cheers,
Derick

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php

Re: [PHP-DEV] [Request for Comment] Loading dynamic libraries with cross dependencies.

[Michael Wallner]

On 28/02/17 20:13, Sara Golemon wrote:
> The average PHP extension is dependent only on the PHP runtime it was
> built for and probably one or more 3rd party libraries.  However, a
> special class of extensions (such as PDO drivers and much of the XML
> family) have dependencies on other PHP extensions which may (or may
> not) be preset at the time they are loaded in via ini settings and/or
> dl().
> 
> This situation was partially addressed by Wez back in PHP 5.2 with the
> addition of zend_module_dep in the module entry header.  Combined with
> RTLD_LAZY during DL_LOAD() (a portability macro for dlopen()), this
> allows modules to be inspected for dependencies and conflicts prior to
> being registered with the runtime.

I, for a long time, thought that the module deps facility was broken,
until I discovered, that it was just not working for me, because I used
a function pointer:
https://github.com/m6w6/ext-http/commit/7493ad081ec06081cb8ca0f804d98a263f93ffbc

> 
> Unfortunately RTLD_LAZY has some limitations.  From `man dlopen`:
> """Lazy binding is performed only for function references; references
> to variables are always immediately bound when the shared object is
> loaded."""  Given PHP's heavy use of XG() macros to access "globals",
> the odds of a missing symbol during module load is therefore fairly
> high resulting in an inability to inspect zend_module_dep structures
> and provide a meaningful error message to end users.

In my book, using XG() should be avoided and must not be used outside of
the owning module. If really needed, there should be an appropriate
public API. IMO.


-- 
Regards,
Mike



signature.asc
Description: OpenPGP digital signature

[PHP-DEV] [Request for Comment] Loading dynamic libraries with cross dependencies.

[Sara Golemon]

The average PHP extension is dependent only on the PHP runtime it was
built for and probably one or more 3rd party libraries.  However, a
special class of extensions (such as PDO drivers and much of the XML
family) have dependencies on other PHP extensions which may (or may
not) be preset at the time they are loaded in via ini settings and/or
dl().

This situation was partially addressed by Wez back in PHP 5.2 with the
addition of zend_module_dep in the module entry header.  Combined with
RTLD_LAZY during DL_LOAD() (a portability macro for dlopen()), this
allows modules to be inspected for dependencies and conflicts prior to
being registered with the runtime.

Unfortunately RTLD_LAZY has some limitations.  From `man dlopen`:
"""Lazy binding is performed only for function references; references
to variables are always immediately bound when the shared object is
loaded."""  Given PHP's heavy use of XG() macros to access "globals",
the odds of a missing symbol during module load is therefore fairly
high resulting in an inability to inspect zend_module_dep structures
and provide a meaningful error message to end users.

Why do I care what the error message looks like? Full disclosure: The
company I work for wants to know reliably what order to list
extensions in php.ini files that are partially autogenerated.  atm
this comes in the form of an explicitly curated relationships table,
but that's not the most scalable approach.
Additionally, resolving this will likely help other users when
presented with what are currently somewhat cryptic load errors. (e.g.
"Unable to load 'foo': unresolved symbol _libf2_blarg")


I'd like to open a discussion on how we might address this in future
releases of PHP.  Some back-of-the-napkin brainstorming has yielded
the following, but I'd love to hear others' thoughts on the subject:

Make use of the already extant PHP_ADD_EXTENSION_DEP config.m4 macro
to produce a look-aside data structure which can be accessed without
fully loading the extension.  This look-aside could be stored in an
arbitrary .text section in the DSO or as a separate file. (I prefer
bundling with the DSO for what I hope are obvious reasons).

-Sara

-- 
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php