[Mono-list] P/Invoke, Mono, .NET and Windows XP funny platform
Hi, We have observed a somewhat curious behaviour that we don't know if it is a bug or the windows XP linkers behaves this way. Let me explain it. We are developing a distribuited application based on an Client interface written in C#, using Mono, which runs over the af-arch platform (http://fact.aspl.es). That platform is mainly written using C so we are P/Invoking from the client interface to the platform so we have access to its services. Under linux we have no problems running the application with any mono version. But under windows XP platform (allways improving the programmer experience) we are running into some troubles. After compiling all libraries from the af-arch platform to produce native dll we have found the application hang ups randomly while running it. We are using the mingw tools to produce native libraries which doesn't rely on cygwin1.dll. After some time debuging the application we didn't found any thing which may lead to those funny random hang ups. Then, we started to use dependency walker to figure out if we were loading something like cygwin1.dll that would make the application break. But we found the mono runtime was loading the same libraries the af-arch is liked to but not the ones we used to compile the af-arch but the ones the mono installer comes with. The af-arch mainly depends on glib-2.0 and libxml-2.0. Now, if we compile the af-arch using the libraries that Mono installs the application run perfectly. If we do so using other libraries we have downloaded the hangs up. So here are the question: * Any application which p/invokes libraries that are also provided by mono must compile its native dll versions against the mono dll or it is posible to compile these ones against, for example, a glib not provided by the mono installer? * What happens if the mono runtime detects a P/Invoke over a library A.dll which depends on B.dll and then another P/Invoke over the library C.dll which depends on B'.dll knowing that B.dll and B'.dll are the same library but not the same file? Many thanks, cheers. ___ Mono-list maillist - Mono-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-list
Re: [Mono-list] P/Invoke, Mono, .NET and Windows XP funny platform
Your fundamental problem is that you're targeting Windows XP. Ha ha only serious. (A colloquialism for "that's funny, but I'm serious too...") The slightly longer explanation is here: http://www.mono-project.com/Interop_with_Native_Libraries#Windows_DLL_Search_Path The real explanation is that Win32 is broken when it comes to supporting multiple different versions of the "same" library. Absolutely, fundamentally, *broken*. More to the point, it doesn't support this AT ALL -- you can only have ONE version of any given library in use at any point in time. Win32 has three issues you're running into: 1. No built-in versioning support for libraries, as noted above. Which is to say, if you load C:\Windows\System32\OLE32.dll, you can't say that you want version 1 or version 2 of that library, you get whichever version is at C:\Windows\System32\OLE32.dll. 2. This doesn't mean you can't have multiple different versions installed. It just means that each different version needs to reside in a different directory, which leaves you the option of (a) always specifying a full (or relative) directory for your library, or (b) accepting the default path and placing your library into an appropriate location. So if you didn't like Microsoft's OLE32.DLL, you could place a copy into your application's directory, and *that* one would get loaded. Alas, this means you can have the scenario you're describing, where both mono and your app bundle glib. This is unfortunate, because of issue (3). 3. Win32 doesn't keep track of the full path name to a library. It only remembers the basename of the library. Translation: If you LoadLibrary() C:\mono\glib.dll, Win32 will know that it's loaded "glib.dll". If you then LoadLibrary() C:\yourapp\glib.dll, Win32 will hand you back a handle to C:\mono\glib.dll, because they both share the same basename (glib.dll) -- C:\yourapp\glib.dll is NOT loaded. Win32 doesn't operate on full paths. This is a feature (in certain contexts, anyway -- it's what allows you to provide your own version of OLE32.dll, or any other system library, and have it be used). So, to answer your questions... On Thu, 2005-06-09 at 19:23 +0200, Francis Brosnan Blázquez wrote: > * Any application which p/invokes libraries that are also provided by > mono must compile its native dll versions against the mono dll or it > is posible to compile these ones against, for example, a glib not > provided by the mono installer? Is it possible? Yes. But to use your version of glib and not mono's version, you'd need to alter the library search order for mono to ensure it loads your version of glib and not mono's. This probably is bad, because it will effect every application started with mono, and your glib might not be compatible with mono's glib, which would (likely) kill mono instantly. So it's possible, but it's not advisable. :-) Plus, it'll use extra disk space, so it would be nicer to use mono's glib anyway... Alternatively, name your glib.dll with a different basename, e.g. af- arch-glib.dll, and rebuild all your libraries against this basename. > * What happens if the mono runtime detects a P/Invoke over a library > A.dll which depends on B.dll and then another P/Invoke over the > library C.dll which depends on B'.dll knowing that B.dll and B'.dll > are the same library but not the same file? Mono isn't in control of library loading, Win32 is. (Mono uses GLib's g_module API, which in turn uses LoadLibrary().) So, what would Win32 do? It would load A.dll (as found through the normal DLL search path), would look for and load B.dll (ditto), and when C.dll was loaded it would see that B.dll was already loaded, and not load a new one, so C.dll would get the *first* B.dll that was ever loaded. In short, don't do that. :-) (I'm not entirely sure why this works for you on Linux either, unless you depend on a different version of glib than mono does, in which case you'd bring in a different .so-name. However, given that Linux/ELF doesn't require a symbol to come from a given library, and instead matches a symbol reference to ANY MATCHING SYMBOL within the address space, I fail to see how, if B.so.2 is already loaded, when C.so.1 is loaded and brings in B.so.3, you could ensure that C.so.1 actually uses the functions in B.so.3 and not B.so.2, since the dynamic linker should find the B.so.2 symbols first... Madness, I say. :-) - Jon ___ Mono-list maillist - Mono-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-list
RE: [Mono-list] P/Invoke, Mono, .NET and Windows XP funny platform
Jonathan Pryor wrote: > Your fundamental problem is that you're targeting Windows XP. > > The real explanation is that Win32 is broken when it comes to > supporting multiple different versions of the "same" library. > Absolutely, fundamentally, *broken*. More to the point, it > doesn't support this AT ALL -- you can only have ONE version > of any given library in use at any point in time. I don't know if this is relevant to the discussion but, AFAIK, since WinXP there is support for versioning libraries. See http://msdn.microsoft.com/library/default.asp?url=/library/en-us/sbscs/setup/isolated_applications_and_side_by_side_assemblies_start_page.asp Regards, Jeroen ___ Mono-list maillist - Mono-list@lists.ximian.com http://lists.ximian.com/mailman/listinfo/mono-list
Re: [Mono-list] P/Invoke, Mono, .NET and Windows XP funny platform
First of all, thanks for your reply. It have been really productive.
Some issues about your reply:
Your fundamental problem is that you're targeting Windows XP.
Ha ha only serious. (A colloquialism for "that's funny, but I'm serious
too...")
The slightly longer explanation is here:
http://www.mono-project.com/Interop_with_Native_Libraries#Windows_DLL_Search_Path
The real explanation is that Win32 is broken when it comes to supporting
multiple different versions of the "same" library. Absolutely,
fundamentally, *broken*. More to the point, it doesn't support this AT
ALL -- you can only have ONE version of any given library in use at any
point in time.
Win32 has three issues you're running into:
1. No built-in versioning support for libraries, as noted above.
Which is to say, if you load C:\Windows\System32\OLE32.dll, you
can't say that you want version 1 or version 2 of that library,
you get whichever version is at C:\Windows\System32\OLE32.dll.
2. This doesn't mean you can't have multiple different versions
installed. It just means that each different version needs to
reside in a different directory, which leaves you the option of
(a) always specifying a full (or relative) directory for your
library, or (b) accepting the default path and placing your library
into an appropriate location.
So if you didn't like Microsoft's OLE32.DLL, you could place a copy
into your application's directory, and *that* one would get loaded.
Alas, this means you can have the scenario you're describing, where
both mono and your app bundle glib. This is unfortunate, because of
issue (3).
3. Win32 doesn't keep track of the full path name to a library. It
only remembers the basename of the library.
Translation: If you LoadLibrary() C:\mono\glib.dll, Win32 will know
that it's loaded "glib.dll". If you then LoadLibrary()
C:\yourapp\glib.dll, Win32 will hand you back a handle to
C:\mono\glib.dll, because they both share the same basename
(glib.dll) -- C:\yourapp\glib.dll is NOT loaded.
Win32 doesn't operate on full paths. This is a feature (in certain
contexts, anyway -- it's what allows you to provide your own version
of OLE32.dll, or any other system library, and have it be used).
So, to answer your questions...
On Thu, 2005-06-09 at 19:23 +0200, Francis Brosnan Blázquez wrote:
* Any application which p/invokes libraries that are also provided by
mono must compile its native dll versions against the mono dll or it
is posible to compile these ones against, for example, a glib not
provided by the mono installer?
Is it possible? Yes. But to use your version of glib and not mono's
version, you'd need to alter the library search order for mono to ensure
it loads your version of glib and not mono's.
This probably is bad, because it will effect every application started
with mono, and your glib might not be compatible with mono's glib, which
would (likely) kill mono instantly.
So it's possible, but it's not advisable. :-)
Plus, it'll use extra disk space, so it would be nicer to use mono's
glib anyway...
The space impact that may have including several libraries using
different names don't
worry us. But setting a dependency against the mono libraries, such as
libglib-2.0, force
us to run over Mono and only over it. As you pointed following, the
library renaming
could be a solution to be able to run on top of Mono and microsoft .NET
runtime.
Alternatively, name your glib.dll with a different basename, e.g. af-
arch-glib.dll, and rebuild all your libraries against this basename.
About the library renaming issue.
Renaming library basename could help us to find a solution to build
application that can run
on top of mono and .NET runtime ensuring the libraries the af-arch
framework relies on will
allways be loaded using the dll version we have used (appending the
"af-arch-")
on develop stage.
My question is: Could this confuse the windows dinamic library loading
to load a library
called glib.dll and another one called af-arch-glib.dll both exporting
the same symbols?
In other words, while running applications using .NET and doing P/Invoke
over the
af-arch-glib.dll the LoadLibrary will find it with no problem as well the
Mono enviroment will do but, in the case of Mono, it have the glib.dll
already loaded
exporting the same (or mostly) symbols leading to have loaded into
memory two
version for the same entry point.
Using as follow [DllImport("af-arch-glib")] will ensure the .NET runtime
as well as Mono to
load the af-arch-glib.dll library. But will this also guide both
runtimes to differenciate the symbols
exported from glib.dll and those ones from af-arch-glib.dll?
* What happens if the mono runtime detects a P/Invoke over a library
A.dll which depends on B.dll and then another P/Invoke over the
library C.dll which depends on B'.dll knowing that B.dll and B'.dll
are the same li
Re: [Mono-list] P/Invoke, Mono, .NET and Windows XP funny platform
On Mon, 2005-06-13 at 16:30 +0200, Francis Brosnan Blázquez wrote:
> About the library renaming issue.
>
> Renaming library basename could help us to find a solution to build
> application that can run on top of mono and .NET runtime ensuring the
> libraries the af-arch framework relies on will allways be loaded using
> the dll version we have used (appending the "af-arch-") on develop
> stage.
>
> My question is: Could this confuse the windows dinamic library loading
> to load a library called glib.dll and another one called af-arch-
> glib.dll both exporting the same symbols?
No. Win32 is less flexible than ELF (which has its pros and cons). ELF
will link an exported symbol from *any* library to any symbol import
that exists, which is rather useful -- it allows you to use LD_PRELOAD
to replace the GTK+ file selector without recompiling any apps (as was
occasionally done before GTK+ 2.4).
Win32, and "bi-level" Mach-O libraries (I forget the correct term, but
they're the recommendation for Mac OS X >= 10.2) don't have this
flexibility; instead, they require that a symbol come from a
specifically-named library. In practice this is similar to C++/C#
namespaces or Java packages: you can have several different Foo classes,
but as long as they all reside in a different namespace/package, there
are no problems -- a class loads whatever it was compiled against.
This reduces any external flexibility (LD_PRELOAD has no equivalent
under Win32), but increases sanity (if you have a plugin that brings in
a DLL with the same symbols as some dependency of yours, there is no
conflict as long as the DLLs have different names; consider also any
versioning issues -- you don't need to worry about adding a symbol which
might be present in a different library, since the actual symbol name is
"tied" to the library name). This is why there can be N different C
runtime libraries without conflict (msvcrt.dll, cygwin1.dll, whatever
Borland's compiler needs...).
> In other words, while running applications using .NET and doing
> P/Invoke over the af-arch-glib.dll the LoadLibrary will find it with
> no problem as well the Mono enviroment will do but, in the case of
> Mono, it have the glib.dll already loaded exporting the same (or
> mostly) symbols leading to have loaded into memory two version for the
> same entry point.
>
> Using as follow [DllImport("af-arch-glib")] will ensure the .NET
> runtime as well as Mono toload the af-arch-glib.dll library. But will
> this also guide both runtimes to differenciate the symbols exported
> from glib.dll and those ones from af-arch-glib.dll?
Yes, for the reasons stated above.
> >(I'm not entirely sure why this works for you on Linux either, unless
> >you depend on a different version of glib than mono does, in which case
> >you'd bring in a different .so-name. However, given that Linux/ELF
> >doesn't require a symbol to come from a given library, and instead
> >matches a symbol reference to ANY MATCHING SYMBOL within the address
> >space, I fail to see how, if B.so.2 is already loaded, when C.so.1 is
> >loaded and brings in B.so.3, you could ensure that C.so.1 actually uses
> >the functions in B.so.3 and not B.so.2, since the dynamic linker should
> >find the B.so.2 symbols first... Madness, I say. :-)
>
> Well, what is happening is that we use Debian ;-). As other Linux
> distributions, the libraries dependecy is handled by the package
> system avoid you lot of problems. Libraries dependencies are set to
> the same for both mono and af-arch, but, libglib and libxml are not
> bundle as part of the mono package as opposed to windows platform.
In other words, your app uses the same dependencies as mono, not a
different version, which makes for a different (and simpler!) scenario
than your Win32 build. :-)
- Jon
___
Mono-list maillist - Mono-list@lists.ximian.com
http://lists.ximian.com/mailman/listinfo/mono-list
