Re: [fpc-devel] Dynamically Loading Libraries
On Wed, 2009-11-11 at 11:25 +0100, Marco van de Voort wrote: In our previous episode, Jeppe Johansen said: -Ivo Steinmann http://www.dict.cc/englisch-deutsch/criticism.html What about saying that procedure proc; external 'libname' name 'proc'; denotes a function that's dynamically loaded implicitly, while procedure proc; external name 'proc'; denotes a static linked function. This syntax is not possible. The external 'xxx' name 'proc' is there for a reason, for systems with multiple linker namespaces. Moreover there is already an enormous codebase that uses this. In my opinion that seems like the only logical solution. Explicit dynamic loading will ofcourse still be possible, but you wouldn't need tools or lots of ifdefs You still need to define which functions must be there to start. I'm not aware of how well it would work on Mac platforms, but I'm mostly certain most GNU/Linux and BSD's, and Windows has support for implicit dynamic linking in the executable formats This is ofcourse a solution that would require modification of the compiler, but I think in the long run that it'll make alot of things more comprehensible I don't. I think this dynamically loading of headers has already gone to far. It is a good workaround for a few headers that are versionwise a disaster, like MySQL, but universally, it doesn't solve problems. Offcourse it does. Only with more stable headers you never notice because you never has to change them. Well, that is, after the first convertion to pascal. But as you do this first conversion, this is really usefull. Now, on the contrary, there are 10 different ways to mimic this behaviour, every library uses it's own way. Which is not usefull for the programmer. Besides, it is an optimisation. Oracle for example exports more then 500 functions headers. Those are all initialised at start, maybe 20 are used. Those can be eliminated. Joost. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
On Wed, 2009-11-11 at 13:21 +0100, Jonas Maebe wrote:
On 10 Nov 2009, at 23:24, Jeppe Johansen wrote:
What about saying that
procedure proc; external 'libname' name 'proc';
denotes a function that's dynamically loaded implicitly, while
procedure proc; external name 'proc';
denotes a static linked function.
In my opinion that seems like the only logical solution. Explicit
dynamic loading will ofcourse still be possible, but you wouldn't
need tools or lots of ifdefs
I'm not aware of how well it would work on Mac platforms, but I'm
mostly certain most GNU/Linux and BSD's, and Windows has support for
implicit dynamic linking in the executable formats
What do you mean by implicit dynamic linking?
This is ofcourse a solution that would require modification of the
compiler, but I think in the long run that it'll make alot of things
more comprehensible
It would also require modifications to several existing header
translations, and depending on what implicit dynamic linking means,
it would also make it impossible to use the same unit for static and
for dynamic linking, while this is currently possible. It would also
break backwards compatibility.
Offcourse not. For existing headers nothing changes. For new ones one
can add the 'lazydynamic' keyword or something like that to the
definition.
It it now also impossible to use the same unit for dynamic ans static
loading. This is all done using ifdefs and include files generating two
units sharing the same code. This all stays the same. Or maybe we can
add a flag {$disablelazydynamic} - that way we only need one ifdef.
It has no influence on backwards compatibility.
Joost.
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
On Thu, 2009-11-12 at 11:15 +0100, Jonas Maebe wrote: Joost van der Sluis wrote on do, 12 nov 2009: On Wed, 2009-11-11 at 13:21 +0100, Jonas Maebe wrote: On 10 Nov 2009, at 23:24, Jeppe Johansen wrote: What about saying that procedure proc; external 'libname' name 'proc'; denotes a function that's dynamically loaded implicitly, while procedure proc; external name 'proc'; denotes a static linked function. In my opinion that seems like the only logical solution. Explicit dynamic loading will ofcourse still be possible, but you wouldn't need tools or lots of ifdefs I'm not aware of how well it would work on Mac platforms, but I'm mostly certain most GNU/Linux and BSD's, and Windows has support for implicit dynamic linking in the executable formats What do you mean by implicit dynamic linking? This is ofcourse a solution that would require modification of the compiler, but I think in the long run that it'll make alot of things more comprehensible It would also require modifications to several existing header translations, and depending on what implicit dynamic linking means, it would also make it impossible to use the same unit for static and for dynamic linking, while this is currently possible. It would also break backwards compatibility. Offcourse not. For existing headers nothing changes. For new ones one can add the 'lazydynamic' keyword or something like that to the definition. The mail I was reacting to, as quoted above, said: *** What about saying thatprocedure proc; external 'libname' name 'proc'; denotes a function that's dynamically loaded implicitly, while procedure proc; external name 'proc'; denotes a static linked function. In my opinion that seems like the only logical solution. Explicit dynamic loading will ofcourse still be possible, but you wouldn't need tools or lots of ifdefs *** That would break backwards compatibility (it would no longer be possible to use a unit with the first syntax for using a statically linked library). You appear to be talking about some different proposal. You're right. I missed that, sorry for that. Joost. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Joost van der Sluis wrote on do, 12 nov 2009: On Wed, 2009-11-11 at 13:21 +0100, Jonas Maebe wrote: On 10 Nov 2009, at 23:24, Jeppe Johansen wrote: What about saying that procedure proc; external 'libname' name 'proc'; denotes a function that's dynamically loaded implicitly, while procedure proc; external name 'proc'; denotes a static linked function. In my opinion that seems like the only logical solution. Explicit dynamic loading will ofcourse still be possible, but you wouldn't need tools or lots of ifdefs I'm not aware of how well it would work on Mac platforms, but I'm mostly certain most GNU/Linux and BSD's, and Windows has support for implicit dynamic linking in the executable formats What do you mean by implicit dynamic linking? This is ofcourse a solution that would require modification of the compiler, but I think in the long run that it'll make alot of things more comprehensible It would also require modifications to several existing header translations, and depending on what implicit dynamic linking means, it would also make it impossible to use the same unit for static and for dynamic linking, while this is currently possible. It would also break backwards compatibility. Offcourse not. For existing headers nothing changes. For new ones one can add the 'lazydynamic' keyword or something like that to the definition. The mail I was reacting to, as quoted above, said: *** What about saying thatprocedure proc; external 'libname' name 'proc'; denotes a function that's dynamically loaded implicitly, while procedure proc; external name 'proc'; denotes a static linked function. In my opinion that seems like the only logical solution. Explicit dynamic loading will ofcourse still be possible, but you wouldn't need tools or lots of ifdefs *** That would break backwards compatibility (it would no longer be possible to use a unit with the first syntax for using a statically linked library). You appear to be talking about some different proposal. Jonas This message was sent using IMP, the Internet Messaging Program. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Marco van de Voort wrote: In our previous episode, Jeppe Johansen said: -Ivo Steinmann http://www.dict.cc/englisch-deutsch/criticism.html What about saying that procedure proc; external 'libname' name 'proc'; denotes a function that's dynamically loaded implicitly, while procedure proc; external name 'proc'; denotes a static linked function. This syntax is not possible. The external 'xxx' name 'proc' is there for a reason, for systems with multiple linker namespaces. Moreover there is already an enormous codebase that uses this. I don't know if I'm making a fool of myself, but what systems? Skimming the sources, I couldn't find any mention of it in the ELF writers. An external symbol will simply be linked to the first occurrence of a matching symbol in the first shared or static object(unless there's some name mangling going on that I didn't spot). The COFF writer already does it the way I suggested. I'm not aware of how well it would work on Mac platforms, but I'm mostly certain most GNU/Linux and BSD's, and Windows has support for implicit dynamic linking in the executable formats This is ofcourse a solution that would require modification of the compiler, but I think in the long run that it'll make alot of things more comprehensible I don't. I think this dynamically loading of headers has already gone to far. It is a good workaround for a few headers that are versionwise a disaster, like MySQL, but universally, it doesn't solve problems. In principle, doing it this way would make cross compiling to an ELF platform with dynamic linking easier, once the internal ELF linker is made. Since the dynamic linked symbols and their locations will be known ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
In our previous episode, Jeppe Johansen said: procedure proc; external name 'proc'; denotes a static linked function. This syntax is not possible. The external 'xxx' name 'proc' is there for a reason, for systems with multiple linker namespaces. Moreover there is already an enormous codebase that uses this. I don't know if I'm making a fool of myself, but what systems? Skimming the sources, I couldn't find any mention of it in the ELF writers. An external symbol will simply be linked to the first occurrence of a matching symbol in the first shared or static object(unless there's some name mangling going on that I didn't spot). The COFF writer already does it the way I suggested. Windows, Darwin, and afaik Solaris. The ELF writers are probably not updated for Solaris yet. This is ofcourse a solution that would require modification of the compiler, but I think in the long run that it'll make alot of things more comprehensible I don't. I think this dynamically loading of headers has already gone to far. It is a good workaround for a few headers that are versionwise a disaster, like MySQL, but universally, it doesn't solve problems. In principle, doing it this way would make cross compiling to an ELF platform with dynamic linking easier, once the internal ELF linker is made. While partially true, this is a brute force approach. Having a few simple map files (that can be generated on target from the libs) over which files contain which symbols might help too. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Regarding a similar subject, I found an article by Hallvards Vassbotn showing a method to delay load dlls in Delphi, its usage is pretty similar to static linking, but the library is loaded on demand (without explicitly calling LoadLibrary). This is a short version: http://hallvards.blogspot.com/2008/03/tdm8-delayloading-of-dlls.html The long PDF version: http://vassbotn.googlepages.com/DelayLoadingOfDLLs.pdf Leonardo M. Ramé http://leonardorame.blogspot.com --- On Wed, 11/11/09, Marco van de Voort mar...@stack.nl wrote: From: Marco van de Voort mar...@stack.nl Subject: Re: [fpc-devel] Dynamically Loading Libraries To: FPC developers' list fpc-devel@lists.freepascal.org Date: Wednesday, November 11, 2009, 8:54 AM In our previous episode, Jeppe Johansen said: procedure proc; external name 'proc'; denotes a static linked function. This syntax is not possible. The external 'xxx' name 'proc' is there for a reason, for systems with multiple linker namespaces. Moreover there is already an enormous codebase that uses this. I don't know if I'm making a fool of myself, but what systems? Skimming the sources, I couldn't find any mention of it in the ELF writers. An external symbol will simply be linked to the first occurrence of a matching symbol in the first shared or static object(unless there's some name mangling going on that I didn't spot). The COFF writer already does it the way I suggested. Windows, Darwin, and afaik Solaris. The ELF writers are probably not updated for Solaris yet. This is ofcourse a solution that would require modification of the compiler, but I think in the long run that it'll make alot of things more comprehensible I don't. I think this dynamically loading of headers has already gone to far. It is a good workaround for a few headers that are versionwise a disaster, like MySQL, but universally, it doesn't solve problems. In principle, doing it this way would make cross compiling to an ELF platform with dynamic linking easier, once the internal ELF linker is made. While partially true, this is a brute force approach. Having a few simple map files (that can be generated on target from the libs) over which files contain which symbols might help too. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
On 11 Nov 2009, at 11:54, Marco van de Voort wrote: In our previous episode, Jeppe Johansen said: This syntax is not possible. The external 'xxx' name 'proc' is there for a reason, for systems with multiple linker namespaces. Moreover there is already an enormous codebase that uses this. I don't know if I'm making a fool of myself, but what systems? Skimming the sources, I couldn't find any mention of it in the ELF writers. An external symbol will simply be linked to the first occurrence of a matching symbol in the first shared or static object(unless there's some name mangling going on that I didn't spot). The COFF writer already does it the way I suggested. Windows, Darwin, and afaik Solaris. The library name is not used on Darwin. You also can't tell the static linker that a name should only be looked up in a particular library. It simply goes through all libraries/frameworks. Once it finds a library/framework that contains the symbol, I think it does record in the final binary from which library/framework the symbol comes though (so that if at run time multiple libraries export the symbol, the same one will always be chosen). Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
On 10 Nov 2009, at 23:24, Jeppe Johansen wrote: What about saying that procedure proc; external 'libname' name 'proc'; denotes a function that's dynamically loaded implicitly, while procedure proc; external name 'proc'; denotes a static linked function. In my opinion that seems like the only logical solution. Explicit dynamic loading will ofcourse still be possible, but you wouldn't need tools or lots of ifdefs I'm not aware of how well it would work on Mac platforms, but I'm mostly certain most GNU/Linux and BSD's, and Windows has support for implicit dynamic linking in the executable formats What do you mean by implicit dynamic linking? This is ofcourse a solution that would require modification of the compiler, but I think in the long run that it'll make alot of things more comprehensible It would also require modifications to several existing header translations, and depending on what implicit dynamic linking means, it would also make it impossible to use the same unit for static and for dynamic linking, while this is currently possible. It would also break backwards compatibility. And like Marco I'm not sure what the big problem is that is solved with this. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Jonas Maebe wrote:
procedure proc; external 'libname' name 'proc';
denotes a function that's dynamically loaded implicitly, while
procedure proc; external name 'proc';
denotes a static linked function.
In my opinion that seems like the only logical solution. Explicit
dynamic loading will ofcourse still be possible, but you wouldn't
need tools or lots of ifdefs
I'm not aware of how well it would work on Mac platforms, but I'm
mostly certain most GNU/Linux and BSD's, and Windows has support for
implicit dynamic linking in the executable formats
What about saying that
What do you mean by implicit dynamic linking?
Using a runtime linker. On windows implicit dynamic loaded functions are
written in the .idata section. And using dynamic runtime linking on
GNU/Linux, with dl
This is ofcourse a solution that would require modification of the
compiler, but I think in the long run that it'll make alot of things
more comprehensible
It would also require modifications to several existing header
translations, and depending on what implicit dynamic linking means,
it would also make it impossible to use the same unit for static and
for dynamic linking, while this is currently possible. It would also
break backwards compatibility. And like Marco I'm not sure what the
big problem is that is solved with this.
If this method was implemented we could make both methods possible like this
{$ifdef LINKSTATIC}
const
libname := '';
{$ifdef LINUX}
{$l library}
{$endif}
{$ifdef WIN32}
{$l library}
{$endif}
{$else}
const
{$ifdef LINUX}
libname := 'library.so';
{$endif}
{$ifdef WIN32}
libname := 'library.dll';
{$endif}
{$endif}
procedure ImportedFunction; external libname name 'ImportedFunction';
procedure ImportedFunction2; external libname name 'ImportedFunction2';
I hope I get the point across :)
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Ivo Steinmann wrote: Hello all I started some wiki for dynamic loading libraries support discussion. http://wiki.freepascal.org/Dynamically_loading_headers I already added an experimental implementation. If you have got some ideas, whises and criticism, don't hesitate to write. -Ivo Steinmann http://www.dict.cc/englisch-deutsch/criticism.html What about saying that procedure proc; external 'libname' name 'proc'; denotes a function that's dynamically loaded implicitly, while procedure proc; external name 'proc'; denotes a static linked function. In my opinion that seems like the only logical solution. Explicit dynamic loading will ofcourse still be possible, but you wouldn't need tools or lots of ifdefs I'm not aware of how well it would work on Mac platforms, but I'm mostly certain most GNU/Linux and BSD's, and Windows has support for implicit dynamic linking in the executable formats This is ofcourse a solution that would require modification of the compiler, but I think in the long run that it'll make alot of things more comprehensible Regards, Jeppe ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
On Mon, 2009-11-09 at 09:14 +0100, Florian Klaempfl wrote: Ivo Steinmann schrieb: Delphi have got also a solution: http://docwiki.embarcadero.com/RADStudio/en/Libraries_and_Packages#Delayed_Loading Soon or later people will request this anyways I fear ... MeMeMe! ;) This time a feature I like, while I do not like things like 'for..in' (Because it's not an orthogonal feature, another word I just learned in respect to programming) It's a nightmare to maintain external library-headers when all exported functions have to be declared twice. Once for static loading, and one for dynamic loading. And the fact that all bindings are really done even when they are not used is also very annoying. To solve this adding wrappers for each function. Means that you have to declare each exported function three times... I don't see the problem when the exported function is defined 'static' - so a link-error happens when it is not available at compile time, and a error occurs when the program is started while the library is not available. Or weak. Which means that a library is loaded when the first call to a function is made. Then the error of a missing library/exported function can be given on the moment of the first function call. Nothing special here? (In fact the compiler generates the necessary wrapper-function: Function AExternalfunction(param1: type1): type2; begin if not loaded(ExternalLibrary) then LoadDynamicLibrary(ExternalLibraryName) // if fails exception if not assigned(TempFunctionVariable) then @tempFunctionVariable:=GerProcedureAddress(...) // fail-exception Result := tempFunctionVariable(param1); ) Joost ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Joost van der Sluis schrieb: On Mon, 2009-11-09 at 09:14 +0100, Florian Klaempfl wrote: Ivo Steinmann schrieb: Delphi have got also a solution: http://docwiki.embarcadero.com/RADStudio/en/Libraries_and_Packages#Delayed_Loading Soon or later people will request this anyways I fear ... MeMeMe! ;) This time a feature I like, while I do not like things like 'for..in' (Because it's not an orthogonal feature, another word I just learned in respect to programming) It's a nightmare to maintain external library-headers when all exported functions have to be declared twice. Once for static loading, and one for dynamic loading. And the fact that all bindings are really done even when they are not used is also very annoying. To solve this adding wrappers for each function. Means that you have to declare each exported function three times... I don't see the problem when the exported function is defined 'static' - so a link-error happens when it is not available at compile time, and a error occurs when the program is started while the library is not available. Or weak. Which means that a library is loaded when the first call to a function is made. Then the error of a missing library/exported function can be given on the moment of the first function call. Nothing special here? (In fact the compiler generates the necessary wrapper-function: Function AExternalfunction(param1: type1): type2; begin if not loaded(ExternalLibrary) then LoadDynamicLibrary(ExternalLibraryName) // if fails exception if not assigned(TempFunctionVariable) then @tempFunctionVariable:=GerProcedureAddress(...) // fail-exception Result := tempFunctionVariable(param1); ) Joost ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel or the compiler even generates that: var Externalfunction: Function(param1: type1): type2 := @NotInitialized; Function NotInitialized(param1: type1): type2; begin if not loaded(ExternalLibrary) then LoadDynamicLibrary(ExternalLibraryName) // if fails exception Externalfunction := GerProcedureAddress(...) // fail-exception Result := Externalfunction(param1); end; The first call to a function is routed through an initialization function. All later calls are done directly. This avoids unnecessary singleton checks. And it would be also nice that it's possible to initialize the library by the user. So that the user is not forced to check at each library call, if it fails or not because the library is not available. So the compiler should also generate a public function function InitializeLibrary[Name]: Boolean; or even function InitializeLibrary[Name](const LibraryName: String): Boolean; -Ivo Steinmann ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
On Tue, 2009-11-03 at 17:28 -0300, Luiz Americo Pereira Camara wrote: Joost van der Sluis escreveu: On Mon, 2009-11-02 at 17:13 -0300, Luiz Americo Pereira Camara wrote: I can write such tool. I have made such a tool, which I used for the database-bindings. I don't know if I still have it tough, and it was very rough. But it's easy doable, indeed. Should we wait to your tool or can i start another one? Better start a new one. Joost ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Delphi have got also a solution: http://docwiki.embarcadero.com/RADStudio/en/Libraries_and_Packages#Delayed_Loading ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
In our previous episode, Ivo Steinmann said: Delphi have got also a solution: http://docwiki.embarcadero.com/RADStudio/en/Libraries_and_Packages#Delayed_Loading It doesn't really say how it does it?!? ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Ivo Steinmann escreveu: are you also on irc in #fpc @freenode? I'm often online there with nick Aison I don't use irc often, but should do if necessary. I will try to do the tool this weekend and keep you informed even if is not used. Luiz ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Luiz Americo Pereira Camara schrieb: Ivo Steinmann escreveu: are you also on irc in #fpc @freenode? I'm often online there with nick Aison I don't use irc often, but should do if necessary. I will try to do the tool this weekend and keep you informed even if is not used. Luiz ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel I think we need it ;) ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
On 04 Nov 2009, at 02:37, Ivo Steinmann wrote: The more important question is: should we do it this way or should I also write a 2nd implementation by modifying the compiler (branched)? Then we don't need an external tool at all. I prefer the compiler not to be modified for something like this. The required functionality is easily implementable without doing so, and external linking issues are already hard enough to debug without adding another abstraction layer on top that hides in the source code how a function is actually being imported. Jonas ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Jonas Maebe schrieb:
On 04 Nov 2009, at 02:37, Ivo Steinmann wrote:
The more important question is: should we do it this way or should I
also write a 2nd implementation by modifying the compiler (branched)?
Then we don't need an external tool at all.
I prefer the compiler not to be modified for something like this. The
required functionality is easily implementable without doing so, and
external linking issues are already hard enough to debug without
adding another abstraction layer on top that hides in the source code
how a function is actually being imported.
Jonas
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
you are right of course ;)
but one thing would be nice to have. A compiler switch and/or built in
function to check wheter a symbol is used or not. With this information
is easy to implement smart loading.
something like
{$SYMTRACING OFF}
// used symbols in this section aren't marked as used.
{$SYMTRACING ON}
and then
{$IF symused(XYZ)}{$IFEND}
or even
a builtin function symused(XYZ): Boolean;
this way it would be possible to create a structure that contains all
symbols including the hint if a symbol is used or not.
const
{$SYMRACING OFF}
zorba_symbols: array[0..6] of TLibSymbol = (
(pvar:@create_simple_store; name:'create_simple_store';
inuse:symused(create_simple_store)),
(pvar:@shutdown_simple_store; name:'shutdown_simple_store';
inuse:symused(shutdown_simple_store)),
(pvar:@zorba_implementation; name:'zorba_implementation';
inuse:symused(zorba_implementation)),
(pvar:@Zorba_CompilerHints_default;
name:'Zorba_CompilerHints_default';
inuse:symused(Zorba_CompilerHints_default)),
(pvar:@Zorba_SerializerOptions_default;
name:'Zorba_SerializerOptions_default';
inuse:symused(Zorba_SerializerOptions_default)),
(pvar:@Zorba_SerializerOptions_free;
name:'Zorba_SerializerOptions_free';
inuse:symused(Zorba_SerializerOptions_free)),
(pvar:@Zorba_SerializerOptions_set;
name:'Zorba_SerializerOptions_set';
inuse:symused(Zorba_SerializerOptions_set)),
);
{$SYMTRACING ON}
Then you can do
LoadLibrarySymbols(libhandle, zorba_symbols);
if one of the used symbols is not present, an exception is raised
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Ivo Steinmann wrote: but one thing would be nice to have. A compiler switch and/or built in function to check wheter a symbol is used or not. With this information is easy to implement smart loading. The problem is that when compiling a unit, you never know which symbols will be used in the program and which will be not. It only becomes known at the linking stage. Sergei ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
On Mon, 2009-11-02 at 17:13 -0300, Luiz Americo Pereira Camara wrote: Ivo Steinmann escreveu: Vincent Snijders schrieb: Maybe it is better to generate a foobar_dyn.inc based on the foobar.inc or generate both foobar.inc and foobar_dyn.inc from a common file format (maybe even the original header file) [..] Most libraries are translated by a tool like h2pas from the original headers and then rehashed manually. The best solution would be, if there's a tool that generates the var procedures directly from all external procedures. I think that creating a tool to translate a static header to a dyn header should be easy to create (i'm not talking to convert c header directly ;-)). Should do the following right? Correct me if i'm wrong: Static function: function foobar_dosomething(a:cint; b:cdouble): cint; extdecl;external foobarlib; Becomes a dyn function: var foobar_dosomething: function(a:cint; b:cdouble): cint; extdecl; I can write such tool. I have made such a tool, which I used for the database-bindings. I don't know if I still have it tough, and it was very rough. But it's easy doable, indeed. Joost ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Joost van der Sluis escreveu: On Mon, 2009-11-02 at 17:13 -0300, Luiz Americo Pereira Camara wrote: I can write such tool. I have made such a tool, which I used for the database-bindings. I don't know if I still have it tough, and it was very rough. But it's easy doable, indeed. Should we wait to your tool or can i start another one? Luiz ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
The more important question is: should we do it this way or should I also write a 2nd implementation by modifying the compiler (branched)? Then we don't need an external tool at all. -Ivo ;) Luiz Americo Pereira Camara schrieb: Joost van der Sluis escreveu: On Mon, 2009-11-02 at 17:13 -0300, Luiz Americo Pereira Camara wrote: I can write such tool. I have made such a tool, which I used for the database-bindings. I don't know if I still have it tough, and it was very rough. But it's easy doable, indeed. Should we wait to your tool or can i start another one? Luiz ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Ivo Steinmann schreef:
Hello all
I started some wiki for dynamic loading libraries support discussion.
http://wiki.freepascal.org/Dynamically_loading_headers
I already added an experimental implementation. If you have got some
ideas, whises and criticism, don't hesitate to write.
I don't like the looks of lines like:
{$IFDEF S}function{$ELSE}var{$ENDIF}foobar_dosomething{$IFDEF D}:
function{$ENDIF}(a:cint; b:cdouble): cint; extdecl;{$IFDEF S}external
foobarlib;{$ENDIF}
They are hard to read. I understand that this way makes sure that both
static and dynamical declaration are in sync, but the loss of
readability is too much.
Maybe it is better to generate a foobar_dyn.inc based on the foobar.inc
or generate both foobar.inc and foobar_dyn.inc from a common file format
(maybe even the original header file).
Vincent
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Ivo Steinmann schreef: Most libraries are translated by a tool like h2pas from the original headers and then rehashed manually. The best solution would be, if there's a tool that generates the var procedures directly from all external procedures. What do you exactly mean? A tool which takes for example: http://svn.freepascal.org/trunk/packages/libpng/src/png.pp and outputs a png_dyn.pp. But else it's much simpler to maintain a construct as the one above than allways starting from the orignal h file or updating allways both declarations. I know what i'm talking about, since I translated about 15 headers in /fpc/packages ^^ But of course you are right, it's not the nice solution ;) The best solution would be to have got some support from the compiler itself ;) What kind of help? Vincent ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Ivo Steinmann escreveu: Vincent Snijders schrieb: Maybe it is better to generate a foobar_dyn.inc based on the foobar.inc or generate both foobar.inc and foobar_dyn.inc from a common file format (maybe even the original header file) [..] Most libraries are translated by a tool like h2pas from the original headers and then rehashed manually. The best solution would be, if there's a tool that generates the var procedures directly from all external procedures. I think that creating a tool to translate a static header to a dyn header should be easy to create (i'm not talking to convert c header directly ;-)). Should do the following right? Correct me if i'm wrong: Static function: function foobar_dosomething(a:cint; b:cdouble): cint; extdecl;external foobarlib; Becomes a dyn function: var foobar_dosomething: function(a:cint; b:cdouble): cint; extdecl; I can write such tool. Luiz ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Luiz Americo Pereira Camara schreef: Ivo Steinmann escreveu: Vincent Snijders schrieb: Maybe it is better to generate a foobar_dyn.inc based on the foobar.inc or generate both foobar.inc and foobar_dyn.inc from a common file format (maybe even the original header file) [..] Most libraries are translated by a tool like h2pas from the original headers and then rehashed manually. The best solution would be, if there's a tool that generates the var procedures directly from all external procedures. I think that creating a tool to translate a static header to a dyn header should be easy to create (i'm not talking to convert c header directly ;-)). Should do the following right? Correct me if i'm wrong: Static function: function foobar_dosomething(a:cint; b:cdouble): cint; extdecl;external foobarlib; Becomes a dyn function: var foobar_dosomething: function(a:cint; b:cdouble): cint; extdecl; And a statement to initialize the variable. Vincent ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Vincent Snijders wrote: Should do the following right? Correct me if i'm wrong: Static function: function foobar_dosomething(a:cint; b:cdouble): cint; extdecl;external foobarlib; Becomes a dyn function: var foobar_dosomething: function(a:cint; b:cdouble): cint; extdecl; And a statement to initialize the variable. I'd also note that the current approach (loading all functions at once) actually defeats the nature of dynamic linking, and smartlinking as well. If some function is absent in the library, error will occur on loading regardless of whether this function is actually used by the program. With static linking, if the problematic function isn't used, there will be no loading error. For this reason, I use the following approach in my projects (in pseudocode): interface function foo(args): integer; implementation type tfoo = function(args): integer; const bad_ptr = pointer(1); var _foo: pointer = bad_ptr; function foo(args): integer; var p: PPointer; begin p := @_foo; Result := E_NOTIMPL; // other values may be used, or an exception raised if p^ = bad_ptr then p^ := GetProcAddress(libHandle, 'foo_name'); if Assigned(P^) then Result := tfoo(p^)(args); end; Such approach, of course, introduces somewhat bigger overhead, but allows the compiler to optimize unused parts away. Sergei ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Vincent Snijders escreveu:
Luiz Americo Pereira Camara schreef:
I think that creating a tool to translate a static header to a dyn
header should be easy to create (i'm not talking to convert c header
directly ;-)).
Should do the following right? Correct me if i'm wrong:
Static function:
function foobar_dosomething(a:cint; b:cdouble): cint;
extdecl;external foobarlib;
Becomes a dyn function:
var foobar_dosomething: function(a:cint; b:cdouble): cint; extdecl;
And a statement to initialize the variable.
Do you mean this?
sqlite3_symbols: array[0..152{$IFDEF SQLITE_OBSOLETE}+6{$ENDIF}] of
TLibSymbol = (
(pvar:@sqlite3_libversion; name:'sqlite3_libversion'; weak:false), etc..
Luiz
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Luiz Americo Pereira Camara schreef:
Vincent Snijders escreveu:
Luiz Americo Pereira Camara schreef:
I think that creating a tool to translate a static header to a dyn
header should be easy to create (i'm not talking to convert c header
directly ;-)).
Should do the following right? Correct me if i'm wrong:
Static function:
function foobar_dosomething(a:cint; b:cdouble): cint;
extdecl;external foobarlib;
Becomes a dyn function:
var foobar_dosomething: function(a:cint; b:cdouble): cint; extdecl;
And a statement to initialize the variable.
Do you mean this?
sqlite3_symbols: array[0..152{$IFDEF SQLITE_OBSOLETE}+6{$ENDIF}] of
TLibSymbol = (
(pvar:@sqlite3_libversion; name:'sqlite3_libversion'; weak:false), etc..
That is one solution, filling the symbols array.
Another solution is simply:
@sqlite3_libversion := GetProcAddress(libHandle, 'sqlite3_libversion');
I was pointing at a missing link.
Vincent
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Luiz Americo Pereira Camara schrieb: Ivo Steinmann escreveu: Vincent Snijders schrieb: Maybe it is better to generate a foobar_dyn.inc based on the foobar.inc or generate both foobar.inc and foobar_dyn.inc from a common file format (maybe even the original header file) [..] Most libraries are translated by a tool like h2pas from the original headers and then rehashed manually. The best solution would be, if there's a tool that generates the var procedures directly from all external procedures. I think that creating a tool to translate a static header to a dyn header should be easy to create (i'm not talking to convert c header directly ;-)). Should do the following right? Correct me if i'm wrong: Static function: function foobar_dosomething(a:cint; b:cdouble): cint; extdecl;external foobarlib; Becomes a dyn function: var foobar_dosomething: function(a:cint; b:cdouble): cint; extdecl; I can write such tool. Luiz ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel exactly Including the list of symbols I suggested. ___ fpc-devel maillist - fpc-devel@lists.freepascal.org http://lists.freepascal.org/mailman/listinfo/fpc-devel
Re: [fpc-devel] Dynamically Loading Libraries
Vincent Snijders schrieb:
Ivo Steinmann schreef:
Most libraries are translated by a tool like h2pas from the original
headers and then rehashed manually. The best solution would be, if
there's a tool that generates the var procedures directly from all
external procedures.
What do you exactly mean? A tool which takes for example:
http://svn.freepascal.org/trunk/packages/libpng/src/png.pp
and outputs a png_dyn.pp.
But else it's much simpler to maintain a construct as the one above than
allways starting from the orignal h file or updating allways both
declarations. I know what i'm talking about, since I translated about 15
headers in /fpc/packages ^^
But of course you are right, it's not the nice solution ;) The best
solution would be to have got some support from the compiler itself ;)
What kind of help?
Vincent
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
well, there are a lot of possibilities, but that's one:
unit foobar;
{$DYNLOADING ON}
function foobar_dosomething(a:cint; b:cdouble): cint; extdecl;external
foobarlib;
Now when compiler flag {$DYNLOADING ON} is set, the compiler don't do
smart/static linking.
Internally it creates an array of all USED!! symbols (similar to the one
I recommended) and also generates functions like
InitializeFoobar;
TryInitializeFoobar; // silent
FinalizeFoobar;
Then the user have to call this functions before he can use one of the
external symbols. If one of the used symbols is not found in the
library, we get an error similar to when do smart linking, but at runtime.
This way we could use the same headers as dynamic and static, just by
switching {$DYNLOADING ON} or OFF
This mechanism would be also interesting for packages in lazarus.
-Ivo
___
fpc-devel maillist - fpc-devel@lists.freepascal.org
http://lists.freepascal.org/mailman/listinfo/fpc-devel
