Re: cvs commit: squid3 configure.in
On mån, 2007-09-24 at 09:49 -0600, Alex Rousskov wrote: The implications range from the Module is a part of the GPL program and, hence, must be GPLed! FSF point of view to there are no implications except wasting time on discussing FSF FUD! point of view. My personal view on that is that a isolated, strict and well defined module API is an application boundary, pretty much like any other ICP like mechanism. But requires a isolated, strict and well defined API specified under an open non-copyleft license (i.e. modifed BSD or similar). An module API requiring knowledge of the GPL licensed internals is not. That's all I have to say in that matter now as the discussion is quite pointless without at least one target module application. Regards Henrik signature.asc Description: This is a digitally signed message part
Re: cvs commit: squid3 configure.in
On Wed, 2007-09-19 at 15:41 +1200, Amos Jeffries wrote: On Fri, 2007-09-07 at 09:43 +0800, Adrian Chadd wrote: That said, doing dynamic loading has GPL implications that I'm not currently comfortable with. What would those be? The implications range from the Module is a part of the GPL program and, hence, must be GPLed! FSF point of view to there are no implications except wasting time on discussing FSF FUD! point of view. Let's not waste our time talking about this here: We would not reach a conclusion, and even if we do, it would not matter. There is an OSI mailing list for such discussions... Alex.
Re: cvs commit: squid3 configure.in
On Wed, 2007-09-19 at 15:41 +1200, Amos Jeffries wrote: On Fri, 2007-09-07 at 09:43 +0800, Adrian Chadd wrote: That said, doing dynamic loading has GPL implications that I'm not currently comfortable with. What would those be? The implications range from the Module is a part of the GPL program and, hence, must be GPLed! FSF point of view to there are no implications except wasting time on discussing FSF FUD! point of view. Oh. I'd heard bits about that before. Never worked on proper modular apps in open source before so it was never as issue for me. Kind of assumed it was resolved during the long hoohah over new version Open Source copyright contracts last year (or ten...). Let's not waste our time talking about this here: We would not reach a conclusion, and even if we do, it would not matter. There is an OSI mailing list for such discussions... Far too true. Thanks for the reminder Alex. Amos
Re: cvs commit: squid3 configure.in
On Fri, 2007-09-07 at 09:43 +0800, Adrian Chadd wrote: On Thu, Sep 06, 2007, Henrik Nordstrom wrote: Maybe. I'll leave that to you to judge. Don't have a clear opinion either way. But why not? HTCP and SNMP is very non-intrusive, just enabling code which is never used unless configured in squid.conf, with no significant change in other code. Maybe one of the goals for squid-3.1 should be runtime loadable dyanmic modules for stuff. (Although I'm not sure how well that'd work with C++ and its symbol munging..) It works fine and easily. In fact a number of the current modules are runtime enabled, though statically linked. That said, doing dynamic loading has GPL implications that I'm not currently comfortable with. -Rob -- GPG key available at: http://www.robertcollins.net/keys.txt. signature.asc Description: This is a digitally signed message part
Re: cvs commit: squid3 configure.in
On Fri, 2007-09-07 at 09:43 +0800, Adrian Chadd wrote: On Thu, Sep 06, 2007, Henrik Nordstrom wrote: Maybe. I'll leave that to you to judge. Don't have a clear opinion either way. But why not? HTCP and SNMP is very non-intrusive, just enabling code which is never used unless configured in squid.conf, with no significant change in other code. Maybe one of the goals for squid-3.1 should be runtime loadable dyanmic modules for stuff. (Although I'm not sure how well that'd work with C++ and its symbol munging..) It works fine and easily. In fact a number of the current modules are runtime enabled, though statically linked. That said, doing dynamic loading has GPL implications that I'm not currently comfortable with. What would those be? Amos
Re: cvs commit: squid3 configure.in
On lör, 2007-09-08 at 07:53 +0200, Pierangelo Masarati wrote: I think you don't need to burn your brains too much on this point. Figuring the perfect API out of the blue could mean spend a lot of energies on something that would eventually freeze anyway, and if one really needs a feature that was introduced few hours before, he won't complain if minimal adjustments are required for a while (he'll probably have more issues within his own plugin). Having said this, your point definitely makes sense, and should drive the design, but without excessive concern. Yes, and is why I say reasonably in shape. but for the goal of spliting Squid in loadable objects all under our control then it's not so important and just 'b' suffices to get it done. Well, I believe it's a pity to hide this feature. Modularity and extensibility often represent a winning point in software, as someone with special needs will always stump in. Yes, but modularity without good boundaries just gets messy. Regards Henrik signature.asc Description: This is a digitally signed message part
Re: cvs commit: squid3 configure.in
Adrian Chadd wrote: Ok, how's that work for exposing C++ interfaces in Squid to external modules? Things could get quite ugly if you're going C++ - C - C++.. Perhaps I was unclear. There's no C++ - C - C++; the only C step is module_init(), which is called when the module is loaded and registers a bunch of C++ functional objects in their containers (e.g., in Squid, probably an execution stack). The rest is C++. Only, if you seek portability, and thus do not rely on -rdynamic working fine with g++ only on Linux (and most Unices), you need to move everything your modules may need to use, include the functional objects storage (the map in my example) into a (dynamic) library the modules can be linked against. Note that, AFAIR, I could make those modules work also on Cygwin; don't remember if either by -rdynamic or it's not needed at all. I know it is on Linux. Cheers, p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: [EMAIL PROTECTED] ---
Re: cvs commit: squid3 configure.in
Kinkie wrote: Watch out for Windows.. AFAIK in that OS it's not allowed to export symbols from an executable image onto a DLL.. this is one of the reasons why most complex applications in Windows are shipped as one huge DLL and a stub executable. That's why you'd need to move all stuff possibly needed by modules into a dll. I've recently stumped into a nice piece of software, EDLL http://edll.sourceforge.net/ (untested by me so far) that resolves reverse dependencies load-time in a platform-independent manner (provided some backend for that platform is available, of course). It also supports windows' dlls. There's also a FlexDLL tool http://alain.frisch.fr/flexdll.html that supposedly does something simlar. I realize it's yet another piece of code to be put in between, and something no one will probably build and distribute widely enough. Again, my 2c (or so). p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: [EMAIL PROTECTED] ---
Re: cvs commit: squid3 configure.in
On Fri, 2007-09-07 at 07:54 +0200, Pierangelo Masarati wrote: The interface is minimal and it's pure C. Of course, it needs reverse linking, so, for example, the function RegisterSetMyData() and any other function the module needs to see must be either exported by the main to the module (the undocumented -rdynamic option to gcc), or placed in yet another library that is linked both by the modules and by the main object. Watch out for Windows.. AFAIK in that OS it's not allowed to export symbols from an executable image onto a DLL.. this is one of the reasons why most complex applications in Windows are shipped as one huge DLL and a stub executable. Kinkie
Re: cvs commit: squid3 configure.in
On Fri, 2007-09-07 at 09:43 +0800, Adrian Chadd wrote: On Thu, Sep 06, 2007, Henrik Nordstrom wrote: Maybe. I'll leave that to you to judge. Don't have a clear opinion either way. But why not? HTCP and SNMP is very non-intrusive, just enabling code which is never used unless configured in squid.conf, with no significant change in other code. Maybe one of the goals for squid-3.1 should be runtime loadable dyanmic modules for stuff. I hope to do it for eCAP (embedded ICAP) in Squid 3.1 if we get enough interest in that feature. Our Traffic Spicer (written in C++) works with dynamic C++ adapter modules so I know it is doable. Alex.
Re: cvs commit: squid3 configure.in
Loadable dynamics modules maybe are not so important for existing features of squid3 (HTCP,ICAP or SNMP), but they are important to allow people/companies to extend squid. The eCAP interface sounds really a good idea. Also maybe there are others parts of squid which can extended in such way (eg ACLs ...) Just some personal opinions: I am watching closely squid at least the last 2 years and I had see how difficult is for squid to get funds for its developments. I think it is easier for the squid project to get money from people/companies which creating/sell solutions based on squid than from the end users (Adrian try hard every day in squid-users mailing list but ...). Regards, Christos Alex Rousskov wrote: On Fri, 2007-09-07 at 09:43 +0800, Adrian Chadd wrote: On Thu, Sep 06, 2007, Henrik Nordstrom wrote: Maybe. I'll leave that to you to judge. Don't have a clear opinion either way. But why not? HTCP and SNMP is very non-intrusive, just enabling code which is never used unless configured in squid.conf, with no significant change in other code. Maybe one of the goals for squid-3.1 should be runtime loadable dyanmic modules for stuff. I hope to do it for eCAP (embedded ICAP) in Squid 3.1 if we get enough interest in that feature. Our Traffic Spicer (written in C++) works with dynamic C++ adapter modules so I know it is doable. Alex.
Re: cvs commit: squid3 configure.in
On Fri, Sep 07, 2007, Alex Rousskov wrote: Maybe one of the goals for squid-3.1 should be runtime loadable dyanmic modules for stuff. I hope to do it for eCAP (embedded ICAP) in Squid 3.1 if we get enough interest in that feature. Our Traffic Spicer (written in C++) works with dynamic C++ adapter modules so I know it is doable. Cool. Don't suppose I could take a peek at how you did it? I'd like to write a few things up as loadable modules - the logging stuff would be a place to start, but I'd like to expose the whole storage API as a clean interface (even given mostly the current semantics) to allow entirely different storage modules to be selected at runtime. Adrian
Re: cvs commit: squid3 configure.in
On lör, 2007-09-08 at 01:48 +0300, Tsantilas Christos wrote: I am watching closely squid at least the last 2 years and I had see how difficult is for squid to get funds for its developments. I think it is easier for the squid project to get money from people/companies which creating/sell solutions based on squid than from the end users (Adrian try hard every day in squid-users mailing list but ...). I'd like to adjust that picture a little. It's not that there has been zero funding after the NLANR grants. Over the years there has been quite noticeable directed funding for developing new features. This generally comes from just that group of people you mention above: People selling solutions where Squid is one component, or people selling an online service where they use Squid in the delivery of that service. But what the project is missing is funding for the day to day activities of fixing bugs etc. And secondly to enable significant steps in the development, but I suspect that's at large a marketing problem. We know pretty well where we want to take Squid and what it requires, but very poor at marketing those projects or how interested parties can contribute with money, time or knowledge. More over being a such loosely connected group of people that we are the answer is likely to differ significantly depending on who in the developer group gets the question adding to the confusion.. Squid-2 has worked reasonably well over the years with a reasonable amount of inflow in directed projects, both in funding and contributed code. This despite the paralyzing caused by the way overdue Squid-3 for which there for a long time was not even a project plan but which still froze all other development. Luckily this has changed with Squid-3 getting some of the same attention, and we now have a reasonably clear path forward even if it's likely to be somewhat of a bumpy road for the next year or so while the transition is being made.. Regards Henrik signature.asc Description: This is a digitally signed message part
Re: cvs commit: squid3 configure.in
On lör, 2007-09-08 at 08:53 +0800, Adrian Chadd wrote: Don't suppose I could take a peek at how you did it? I'd like to write a few things up as loadable modules - the logging stuff would be a place to start, but I'd like to expose the whole storage API as a clean interface (even given mostly the current semantics) to allow entirely different storage modules to be selected at runtime. There is two steps you need to do in order to prepare for loadable module: a) Define the APIs the module may be used. Same as for any other code linked into Squid. b) Have a runtime registry where the module can register. Such registry allows for a simple dlopen reference to call a function in the loaded module which then registers with the main code. This is already in place for a number of component types in Squid, even if not yet used for runtime loading of modules (just runtime registering of modules, the part taking place after the shared object has been brought into memory by dlopen). In C++ this is implemented using virtual base classes defining the API for calling the module allowing it to register and get instanciated when needed, and just standard C++ for the API the module may call into the rest of the code.. The ESI code is an example where this has been done, and where the ESI parsers easily could be turned into loadable modules. The module simply calls RegisterESIParser(name, moduleobject); at load time to register itself (currently when the binary loads, but may be just as well from a dynamic loaded module). There is not a single reference from the main code to the individual parsers. This is rightfyllt ignoring the Windows DLL export issue, which is just a platform oddity and not hard to get around should one want to. So in the end getting Squid ready for dynamically loaded modules means a) Getting the APIs the modules needs to use to perform their actions reasonably in shape. b) Adding suitable component registries, allowing modules to register at runtime and avoiding direct knowledge of the implementation in other parts of the code. 'a' is quite important if we are to open for third-party modules as there is then a need for a reasonably stable API the modules can use. It's not going to be seen well if key aspects of those APIs changes every month.. but for the goal of spliting Squid in loadable objects all under our control then it's not so important and just 'b' suffices to get it done. Regards Henrik signature.asc Description: This is a digitally signed message part
Re: cvs commit: squid3 configure.in
Henrik Nordstrom wrote: (snip) a) Getting the APIs the modules needs to use to perform their actions reasonably in shape. b) Adding suitable component registries, allowing modules to register at runtime and avoiding direct knowledge of the implementation in other parts of the code. 'a' is quite important if we are to open for third-party modules as there is then a need for a reasonably stable API the modules can use. It's not going to be seen well if key aspects of those APIs changes every month.. I think you don't need to burn your brains too much on this point. Figuring the perfect API out of the blue could mean spend a lot of energies on something that would eventually freeze anyway, and if one really needs a feature that was introduced few hours before, he won't complain if minimal adjustments are required for a while (he'll probably have more issues within his own plugin). Having said this, your point definitely makes sense, and should drive the design, but without excessive concern. but for the goal of spliting Squid in loadable objects all under our control then it's not so important and just 'b' suffices to get it done. Well, I believe it's a pity to hide this feature. Modularity and extensibility often represent a winning point in software, as someone with special needs will always stump in. Cheers, p. Ing. Pierangelo Masarati OpenLDAP Core Team SysNet s.r.l. via Dossi, 8 - 27100 Pavia - ITALIA http://www.sys-net.it --- Office: +39 02 23998309 Mobile: +39 333 4963172 Email: [EMAIL PROTECTED] ---
Re: cvs commit: squid3 configure.in
On Mon, 2007-08-27 at 07:37 -0600, Henrik Nordstrom wrote: hno 2007/08/27 07:37:02 MDT Log: Enable HTCP and SNMP support by default in the binary, but default off in squid.conf Should we enable ICAP in the binary by default as well? It is probably nearly as widely used as HTCP? Thanks, Alex.
Re: cvs commit: squid3 configure.in
On tor, 2007-09-06 at 10:00 -0600, Alex Rousskov wrote: On Mon, 2007-08-27 at 07:37 -0600, Henrik Nordstrom wrote: hno 2007/08/27 07:37:02 MDT Log: Enable HTCP and SNMP support by default in the binary, but default off in squid.conf Should we enable ICAP in the binary by default as well? It is probably nearly as widely used as HTCP? Maybe. I'll leave that to you to judge. Don't have a clear opinion either way. But why not? HTCP and SNMP is very non-intrusive, just enabling code which is never used unless configured in squid.conf, with no significant change in other code. Regards Henirk signature.asc Description: This is a digitally signed message part
Re: cvs commit: squid3 configure.in
On Thu, Sep 06, 2007, Henrik Nordstrom wrote: Maybe. I'll leave that to you to judge. Don't have a clear opinion either way. But why not? HTCP and SNMP is very non-intrusive, just enabling code which is never used unless configured in squid.conf, with no significant change in other code. Maybe one of the goals for squid-3.1 should be runtime loadable dyanmic modules for stuff. (Although I'm not sure how well that'd work with C++ and its symbol munging..) Adrian
Re: cvs commit: squid3 configure.in
Adrian Chadd wrote:
Maybe one of the goals for squid-3.1 should be runtime loadable dyanmic
modules for stuff.
(Although I'm not sure how well that'd work with C++ and its symbol
munging..)
I've some experience in this sense, http://www.aero.polimi.it/~mbdyn/.
Initially, I was setting a structure containing calls to functions (a la
Apache), and lt_dlsym() a pointer to this structure which was cast to
extern C void * to avoid mangling.
Recently, I completely reworked things the other way 'round (a la
OpenLDAP): the module contains an 'extern C' function that is resolved
and called by the main program at load time. This function can be
passed any C argument, and in the API I defined I also pass pointers to
C++ arguments as void *, which eventually need to be cast to their C++
counterparts (purists may dislike casting void *, but it's for the sake
of avoiding mangling odds). However, this way modules should need
minimal interaction with C++ data structures, since their main purpose
would be to register functional objects which later will deal with real
data. Something like
// functional object that sets a handler; modules will inherit from this
class SetMyData {
public:
// ...
virtual void Set();
};
// container for registered objects; the key is a string
typedef mapstring, SetMyData MyDataMap;
MyDataMap mydatamap;
// helper that registers a new functional object
bool RegisterSetMyData(const string s, SetMyData* smd){
return mydatamap.insert(MyDataMap::value_type(s, smd).second;
}
// the module:
class ModuleMyData {
// ...
};
class ModuleSetMyData {
// ...
ModuleSetMyData(int argc, char *const argv[]);
virtual void Set();
};
// this is the only symbol the module needs to export; call any helper
// required to register the functional objects the module wants the main
// to use; functional objects may make use of parameters passed like
// command-line arguments
extern C int
module_init(int argc, char *const argv[])
{
SetMyData* smd = new ModuleSetMyData(argc, argv);
RegisterSetMyData(mydataname, smd);
}
// end of the module
The interface is minimal and it's pure C. Of course, it needs reverse
linking, so, for example, the function RegisterSetMyData() and any other
function the module needs to see must be either exported by the main to
the module (the undocumented -rdynamic option to gcc), or placed in
yet another library that is linked both by the modules and by the main
object.
Cheers, p.
Ing. Pierangelo Masarati
OpenLDAP Core Team
SysNet s.r.l.
via Dossi, 8 - 27100 Pavia - ITALIA
http://www.sys-net.it
---
Office: +39 02 23998309
Mobile: +39 333 4963172
Email: [EMAIL PROTECTED]
---
