Send kea-dev mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.isc.org/mailman/listinfo/kea-dev
or, via email, send a message with subject or body 'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is more specific
than "Re: Contents of kea-dev digest..."
Today's Topics:
1. Re: Strange behaviour with Option 53 - what am I missing?
(Francis Dupont)
2. Re: Strange behaviour with Option 53 - what am I missing?
(Dave Cole)
----------------------------------------------------------------------
Message: 1
Date: Wed, 31 Jan 2018 14:14:20 +0000
From: Francis Dupont <[email protected]>
To: Dave Cole <[email protected]>
Cc: Francis Dupont <[email protected]>, "[email protected]"
<[email protected]>
Subject: Re: [kea-dev] Strange behaviour with Option 53 - what am I
missing?
Message-ID: <[email protected]>
Content-Type: text/plain; charset="us-ascii"
Dave Cole writes:
> I was hoping to be able to minimise the amount of extra work done by the bi=
> nding, so eliminating unnecessary object allocation / destruction would be =
> a good thing. I recently had a play with exposing the shared_ptr use_count=
> to confirm a behaviour I was seeking. It turns out that the shared_ptr wo=
> rks very nicely with Python wrapping.
=> from the time (more than 30 years ago) I was a CS researcher working
on garbage collection I got a bad opinion about reference counting:
- expensive in space (need to store the reference count in each object)
- expensive in time (need to increment and decrement the reference count
with an atomic operation at each reference change)
- unbound delay when releasing the last reference
- too easy to break with cycles
- fragment memory (vs compact memory)
So it is clearly the worst garbage collection system. But both shared pointers
and C Python are based on reference counting so they work together without
problems...
BTW I didn't forget your concern about hook support in Kea using dlopen()
with RTLD_LOCAL (vs RTLD_GLOBAL) flag. I'll try to do something about this
in 1.4-final or 1.5 milestone...
Thanks
Francis Dupont <[email protected]>
------------------------------
Message: 2
Date: Wed, 31 Jan 2018 22:18:32 +0000
From: Dave Cole <[email protected]>
To: Francis Dupont <[email protected]>
Cc: "[email protected]" <[email protected]>
Subject: Re: [kea-dev] Strange behaviour with Option 53 - what am I
missing?
Message-ID:
<syxpr01mb17277e795dfda2f615d5354af2...@syxpr01mb1727.ausprd01.prod.outlook.com>
Content-Type: text/plain; charset="iso-8859-1"
It is pretty easy to work around in the Python case. Here is a bit of my code:
- - %< - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
static int
findSymbol(void *lib, void **sym, const char *name) {
*sym = dlsym(lib, name);
if (*sym == NULL) {
LOG_INFO(logger, PYTHON_LOAD_FAILED).arg(string("could not dlsym ") +
name);
return (1);
}
return (0);
}
static void (*dl_Py_Initialize)(void);
static void * (*dl_PyCapsule_Import)(const char *name, int no_block);
int
loadPythonLibrary(const string &filename) {
void *lib = dlopen(filename.c_str(), RTLD_NOW | RTLD_GLOBAL);
if (lib == NULL) {
LOG_INFO(logger, PYTHON_LOAD_FAILED).arg("could not dlopen " +
filename);
return (1);
}
if (findSymbol(lib, (void **) &dl_PyCapsule_Import, "PyCapsule_Import")
|| findSymbol(lib, (void **) &dl_Py_Initialize, "Py_Initialize")) {
return (1);
}
return (0);
}
static bool kea_loaded = false;
int
load(LibraryHandle& handle) {
// Load the embedded libpython interpreter.
ConstElementPtr libpython_elem = handle.getParameter("libpython");
if (!libpython_elem || libpython_elem->getType() != Element::string) {
LOG_ERROR(logger, PYTHON_LOAD_FAILED).arg("expected libpython
configuration parameter");
return (1);
}
if (loadPythonLibrary(libpython_elem->stringValue())) {
return (1);
}
// Get the name of the Python module implementing the hook.
ConstElementPtr module_elem = handle.getParameter("module");
if (!module_elem || module_elem->getType() != Element::string) {
LOG_ERROR(logger, PYTHON_LOAD_FAILED).arg("expected module
configuration parameter");
return (1);
}
const string &module = module_elem->stringValue();
// Initialise the Python interpreter.
dl_Py_Initialize();
// Load the kea module C API.
PyKea_API = (void **) dl_PyCapsule_Import("kea._C_API", 0);
if (PyKea_API == NULL) {
LOG_ERROR(logger, PYTHON_LOAD_FAILED).arg("could not import the kea
module _C_API");
return (1);
}
// Bootstrap the kea module.
PyKea_SetLogger(logger, PYTHON_LOG);
if (PyKea_Load(handle, module)) {
return (1);
}
kea_loaded = true;
return (0);
}
- - %< - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
The PyKea_ stuff is simply an implementation of a Python Capsule:
https://docs.python.org/2/extending/extending.html#using-capsules
All of the actual Kea binding code is inside a separate library that is a
vanilla Python extension module. This means that the problems caused by
RTLD_LOCAL are minimised to just the code above.
The idea is that while running inside Kea the kea module uses the capabilities
inside Kea for logging, and uses Kea supplied instances of LibraryHandle and
CalloutHandle and logging. For unit test code I have simple wrapper of
CalloutManager so I can construct a CalloutHandle. Then in my unittests I
subclass the kea.LibraryHandle to replace getParameter(), and so on...
From: Francis Dupont <[email protected]>
Sent: Thursday, 1 February 2018 1:14:20 AM
To: Dave Cole
Cc: Francis Dupont; [email protected]
Subject: Re: [kea-dev] Strange behaviour with Option 53 - what am I missing?
?
Dave Cole writes:
> I was hoping to be able to minimise the amount of extra work done by the bi=
> nding, so eliminating unnecessary object allocation / destruction would be =
> a good thing.? I recently had a play with exposing the shared_ptr use_count=
>? to confirm a behaviour I was seeking.? It turns out that the shared_ptr wo=
> rks very nicely with Python wrapping.
=> from the time (more than 30 years ago) I was a CS researcher working
on garbage collection I got a bad opinion about reference counting:
?- expensive in space (need to store the reference count in each object)
?- expensive in time (need to increment and decrement the reference count
? with an atomic operation at each reference change)
?- unbound delay when releasing the last reference
?- too easy to break with cycles
?- fragment memory (vs compact memory)
So it is clearly the worst garbage collection system. But both shared pointers
and C Python are based on reference counting so they work together without
problems...
BTW I didn't forget your concern about hook support in Kea using dlopen()
with RTLD_LOCAL (vs RTLD_GLOBAL) flag. I'll try to do something about this
in 1.4-final or 1.5 milestone...
Thanks
Francis Dupont <[email protected]>
------------------------------
Subject: Digest Footer
_______________________________________________
kea-dev mailing list
[email protected]
https://lists.isc.org/mailman/listinfo/kea-dev
------------------------------
End of kea-dev Digest, Vol 47, Issue 1
**************************************
