Re: Can't compile GHC 6.8.2
Barney Stratford wrote: I have a workaround for the linker bugs that keep popping up under Mac OS X, and now have a working GHC 6.8. I found that saying -fasm didn't help matters. Instead, on the assumption that the linker gets it wrong and there is in fact nothing wrong with the object file, I applied these two patches: diff -u a/compiler/Makefile b/compiler/Makefile --- a/compiler/Makefile Mon Dec 10 18:11:32 2007 +++ b/compiler/Makefile Wed Nov 26 09:35:12 2008 @@ -753,7 +753,7 @@ $(SED) -e [EMAIL PROTECTED]@$(GHC_PATH)@g -e [EMAIL PROTECTED]@$(FPTOOLS_TOP_ABS)@g $ $@ $(INPLACE_PROG): $(INPLACE_SRC) - $(HC) -cpp $(INPLACE_EXTRA_FLAGS) $ -o $@ + $(HC) -cpp $(INPLACE_EXTRA_FLAGS) $ -o $@ -optl -Wl,-read_only_relocs,warning all :: $(INPLACE_PROG) diff -u a/mk/target.mk b/mk/target.mk --- a/mk/target.mk Mon Dec 10 18:11:31 2007 +++ b/mk/target.mk Thu Nov 27 12:14:02 2008 @@ -231,7 +231,7 @@ ifneq $(BootingFromHc) YES $(HS_PROG) :: $(OBJS) - $(HC) -o $@ $(HC_OPTS) $(LD_OPTS) $(OBJS) + $(HC) -o $@ $(HC_OPTS) $(LD_OPTS) $(OBJS) -optl -Wl,-read_only_relocs,warning else # see bootstrap.mk $(HS_PROG) :: $(OBJS) Another way to fix these problems would be to mumble something about -segprot __TEXT rwx rwx, but this is definitely a hack. I also had to patch Cabal so it ranlibs the libraries it creates (which is required under Mac OS), and fixed the .hi-boot files in compiler/typecheck. (Shouldn't these be updated automatically?) Without those changes, the typechecker won't typecheck. Oh, the irony! Perhaps someone that knows more about how the build system works could figure out the right way to do these things. Presumably you'll get the same error when using your new GHC to link any Haskell program? So in fact GHC itself should be adding this option when linking the RTS on this platform. The right place to add this would be in rts/package.conf.in, in extra-ld-opts, with suitable platform-specific #ifdefs. If you can whip up a patch and test it, I'll commit. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: ghci linker and circular dependencies: bug or feature?
Andrea Rossato wrote:
Hello,
this is a followup of this:
http://article.gmane.org/gmane.comp.lang.haskell.cafe/48644
which didn't have replays. In order to make the review a bit easier I
prepared a minimal test case which, I believe, could prove there's a
bug in the ghci linker. I'm not submitting a bug report because I'm
not familiar with these problems and, in order not to waste other
people's time, I'd rather have some preliminary review.
All the code examples can be found here:
http://gorgias.mine.nu/ffi-test/
Here a tarball of everything:
http://gorgias.mine.nu/ffi-test.tar.gz
Take a simple C library like this:
#include stdio.h
extern char my_var[];
void my_fun()
{
fprintf (stderr, %s\n, my_var);
}
Since my_var is defined as external, the dynamic library compiled
and linked with:
gcc -Wall -fPIC -c -o mylib.o mylib.c
gcc -shared -Wl,-soname,libmylib.so.1 -o libmylib.so.1.0 mylib.o
would have my_var as undefined.
If I write some Haskell bindings (Test.hsc) like this:
module Test where
import Foreign.C
import Foreign
#include mylib.h
foreign import ccall unsafe my_fun my_cfun :: IO ()
my_fun :: IO ()
my_fun = my_cfun
I'll need to include a stub.c file to initialize my_var:
char my_var[] = Hello World!!;
And now come my problems:
1. First I have a Cabal problem. If I set:
extra-libraries: mylib
this will be used also when compiling dist/build/Test_hsc_make, and,
since libmylib.so.1.0 has an undefined reference to my_var, which will
be initialized only later, by stub.c, the compilation of the bindings
will fail with:
Configuring mylib-0.1...
Preprocessing library mylib-0.1...
/tmp/ffi-test/libmylib.so: undefined reference to `my_var'
collect2: ld returned 1 exit status
linking dist/build/Test_hsc_make.o failed
command was: /usr/bin/gcc -L/tmp/ffi-test -lmylib
-L/usr/lib/ghc-6.10.1/base-4.0.0.0 [...]
I can find a work-around by not setting cabal extra-libraries and
instead setting
ghc-options: -lmylib
But when compiling a test file:
import Test
main = my_fun
I'll need to pass -lmylib to ghc --make.
2. I also have ghci problem. If I try with:
ghci test_lib.hs
GHCi, version 6.10.1: http://www.haskell.org/ghc/ :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer ... linking ... done.
Loading package base ... linking ... done.
Ok, modules loaded: Main.
Prelude Main main
Loading package mylib-0.1 ... linking ... interactive:
/home/andrea/.cabal/lib/mylib-0.1/ghc-6.10.1/HSmylib-0.1.o: unknown symbol `my_fun'
ghc: unable to load package `mylib-0.1'
Now, I have to remember to pass -lmylib:
ghci -lmylib test_lib.hs
GHCi, version 6.10.1: http://www.haskell.org/ghc/ :? for help
Loading object (dynamic) mylib ... failed.
command line: user specified .o/.so/.DLL could not be loaded
(/tmp/ffi-test/libmylib.so: undefined symbol: my_var)
Whilst trying to load: (dynamic) mylib
Additional directories searched: (none)
Now, the my_var symbol is included in:
/home/andrea/.cabal/lib/mylib-0.1/ghc-6.10.1/HSmylib-0.1.o
and indeed:
nm --print-arma /home/andrea/.cabal/lib/mylib-0.1/ghc-6.10.1/HSmylib-0.1.o
| grep my_var
001c D my_var
So I desperately try:
ghci /home/andrea/.cabal/lib/mylib-0.1/ghc-6.10.1/HSmylib-0.1.o -lmylib /tmp/ffi-test/bindings/test_lib.hs
GHCi, version 6.10.1: http://www.haskell.org/ghc/ :? for help
Loading object (static)
/home/andrea/.cabal/lib/mylib-0.1/ghc-6.10.1/HSmylib-0.1.o ... done
Loading object (dynamic) mylib ... failed.
command line: user specified .o/.so/.DLL could not be loaded
(/tmp/ffi-test/libmylib.so: undefined symbol: my_var)
Whilst trying to load: (dynamic) mylib
Additional directories searched: (none)
Is this a bug? An intended behaviour? Is there a way out? What am I
missing?
It's not a bug - or rather, it's a consequence of the fact that GHC does
its own runtime linking. The dynamic library libmylib.so is being linked
by the system linker, which has its own symbol table and knows nothing
about GHC's linker and its symbol table. The file HSmylib-0.1.o is loaded
by GHC's linker, so the my_var symbol is in GHC's symbol table, and hence
can't be found by the system linker when loading libmylib.so.
Some good news is that we should be able to make this work when we start
using shared libraries, because HSmylib will be a shared library and it
will be linked by the system linker (what I'm not sure about is how you
load mutually recursive shared objects at runtime, but presumably there's a
way to do that).
Perhaps you could work around the Cabal/hsc2hs problem by making a dummy
library containing my_var, and passing it to Cabal using
--hsc2hs-option=-ldummy.
Cheers,
Simon
Obviously the problem goes away if I link mylib.o with stub.o into
libmylib.so.1.0.
Re: Re[2]: ghci and ghc -threaded broken with pipes forking
Had you deprecated the non-threaded RTS, we would probably have no problems described in ticket #2848 :-/ I think you'll have to deprecate it anyway, because it will be more and more difficult to maintain two versions of code, we may conduct small survey on amount of usage of old RTS (i mean ask this in haskell-cafe) For the only application I tried, using the threaded RTS imposes a 100% performance penalty - i.e. computation time doubles, compared to the non-threaded RTS. This was with ghc-6.8.2, and maybe the overhead has improved since then? Regards, Malcolm ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: ghc 6.10.1 on freebsd 7 amd64 - ghci problems
Markus Barenhoff wrote: On Fri 28.11 09:42, Simon Marlow wrote: Markus Barenhoff wrote: On Thu 27.11 09:49, Simon Marlow wrote: Hi! I checked out and translated the head version of ghc today from darcs. It compiled fine. When I now try to start the ghci I get the following: snip GHCi, version 6.11.20081126: http://www.haskell.org/ghc/ :? for help ghc: internal error: loadObj: failed to mmap() memory below 2Gb; asked for 626688 bytes at 0x4000, got 0x801635000. Try specifying an address with +RTS -xmaddr -RTS (GHC version 6.11.20081126 for x86_64_unknown_freebsd) Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug Abort (core dumped) snip If it helps somehow, you can find the core dump here: http://www.alios.org/~alios/ghc.core.bz2 That's odd, because 6.8.3 is using 0x4000 on FreeBSD and is working fine (or is it?). yes, it worked, but I haven't installed anymore - so I'am not sure, where it does maps its stuff into memory. The memory map shows that there's nothing mapped in at the place we're asking for memory, but still FreeBSD decides to give us memory somewhere else. I checked back in 6.8.3 and it looks like we were using MAP_FIXED - now MAP_FIXED is dangerous, because it will overwrite existing mappings (like the binary itself!), which is why we're not doing that now. I've committed a patch that uses MAP_FIXED on FreeBSD if other attempts to get memory below 2Gb fail. I'd be grateful if you could test it out, the patch is this one, pushed to HEAD today: Wed Dec 10 11:57:51 GMT 2008 Simon Marlow [EMAIL PROTECTED] * On FreeBSD, try MAP_FIXED if ordinary mmap() fails to give us suitable memory This appears to be necessary on FreeBSD. It might be necessary on other OSs too, but I'm being cautious because using MAP_FIXED can lead to crashes by overwriting existing mappings, and we have no (easy) way to prevent that. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Parallel forkOS does not work on ubuntu
Hoang Truong wrote: Hi Simon, I tried with forkIO and added another dowork functions but the result is the same: only one core is used, three other cores are idle. Do you have any other suggestions? Is there anything I should take care when installing GHC? I also did try the Wombat.hs from the tutorial, but only one core is used and the times are almost the same. seq sum: 119201850 seq time: 20.959932 seconds. par sum: 119201850 par time: 20.959547 seconds. Your program is suffering from microbenchmarkitis, I'm afraid. There's only one spark, which tickles a bug in the scheduler in 6.10.1 and earlier (but sometimes doesn't happen due to random scheduling behaviour). Even with that fixed, the program uses fib which tickles another bug: when optimised, fib doesn't do any allocation, and GHC's scheduler relies on allocation happening at regular enough intervals. In 6.10.1 we never get to do load-balancing in this example, because fib doesn't ever yield control to the scheduler. In HEAD, where we have work-stealing and don't rely on the scheduler for load-balancing, the load-balancing problem goes away but reveals another problem: the second thread wants to GC, but in order to GC it has to synchronise with the other running threads, but the other thread is running fib and never yields. We can fix this by allowing CPUs to GC independently (which we plan to do), but even then you could still run into the same problem because eventually a global GC will be required. If you really want to see the program running in parallel, turn off -O. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re[2]: Parallel forkOS does not work on ubuntu
Hello Simon, Wednesday, December 10, 2008, 5:24:28 PM, you wrote: good explanation of various shortanges on the way to multi-threading. may be it worth a link from GHC Concurrency pages? Hoang Truong wrote: Hi Simon, I tried with forkIO and added another dowork functions but the result is the same: only one core is used, three other cores are idle. Do you have any other suggestions? Is there anything I should take care when installing GHC? I also did try the Wombat.hs from the tutorial, but only one core is used and the times are almost the same. seq sum: 119201850 seq time: 20.959932 seconds. par sum: 119201850 par time: 20.959547 seconds. Your program is suffering from microbenchmarkitis, I'm afraid. There's only one spark, which tickles a bug in the scheduler in 6.10.1 and earlier (but sometimes doesn't happen due to random scheduling behaviour). Even with that fixed, the program uses fib which tickles another bug: when optimised, fib doesn't do any allocation, and GHC's scheduler relies on allocation happening at regular enough intervals. In 6.10.1 we never get to do load-balancing in this example, because fib doesn't ever yield control to the scheduler. In HEAD, where we have work-stealing and don't rely on the scheduler for load-balancing, the load-balancing problem goes away but reveals another problem: the second thread wants to GC, but in order to GC it has to synchronise with the other running threads, but the other thread is running fib and never yields. We can fix this by allowing CPUs to GC independently (which we plan to do), but even then you could still run into the same problem because eventually a global GC will be required. If you really want to see the program running in parallel, turn off -O. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Parallel forkOS does not work on ubuntu
Thanks Simon for your detail explanation. It does help me as I am new to Haskell. Btw, I did not use -O option. Hoang On Wed, Dec 10, 2008 at 9:24 PM, Simon Marlow [EMAIL PROTECTED] wrote: Hoang Truong wrote: Hi Simon, I tried with forkIO and added another dowork functions but the result is the same: only one core is used, three other cores are idle. Do you have any other suggestions? Is there anything I should take care when installing GHC? I also did try the Wombat.hs from the tutorial, but only one core is used and the times are almost the same. seq sum: 119201850 seq time: 20.959932 seconds. par sum: 119201850 par time: 20.959547 seconds. Your program is suffering from microbenchmarkitis, I'm afraid. There's only one spark, which tickles a bug in the scheduler in 6.10.1 and earlier (but sometimes doesn't happen due to random scheduling behaviour). Even with that fixed, the program uses fib which tickles another bug: when optimised, fib doesn't do any allocation, and GHC's scheduler relies on allocation happening at regular enough intervals. In 6.10.1 we never get to do load-balancing in this example, because fib doesn't ever yield control to the scheduler. In HEAD, where we have work-stealing and don't rely on the scheduler for load-balancing, the load-balancing problem goes away but reveals another problem: the second thread wants to GC, but in order to GC it has to synchronise with the other running threads, but the other thread is running fib and never yields. We can fix this by allowing CPUs to GC independently (which we plan to do), but even then you could still run into the same problem because eventually a global GC will be required. If you really want to see the program running in parallel, turn off -O. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
IMCROSS and ghc
I was wondering if anyone has gotten ghc to work with IMCROSS[1]. I'm Cross is a way to build native windows and mac osx apps on linux. quite convinient for making distribution packages for those other platforms or if you want to use linux's toolset during your build. basically, IMCROSS installs a couple new gccs with names like /usr/local/bin/i386-mingw-gcc and so forth, is there some way to get ghc to use said non-native compiler as its back end? It seems that ghc will also need to build the libraries with different #define's as well to reflect the target windows or mac environment. [1] http://www.sandroid.org/imcross/ John -- John Meacham - ⑆repetae.net⑆john⑈ ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: IMCROSS and ghc
On Wed, 2008-12-10 at 11:19 -0800, John Meacham wrote: I was wondering if anyone has gotten ghc to work with IMCROSS[1]. I'm Cross is a way to build native windows and mac osx apps on linux. quite convinient for making distribution packages for those other platforms or if you want to use linux's toolset during your build. basically, IMCROSS installs a couple new gccs with names like /usr/local/bin/i386-mingw-gcc and so forth, is there some way to get ghc to use said non-native compiler as its back end? It seems that ghc will also need to build the libraries with different #define's as well to reflect the target windows or mac environment. It sounds like you would need to set ghc up as a cross-compiler. I'm told that this there is a fair bit of work required to get ghc to support cross-compilation. I expect Ian or Simon could provide more details on what would need doing if you're interested in pursuing it. Duncan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
