Re: [Haskell-cafe] External Interfaces
If the function that you are calling doesn't modify the value you are passing, and only returns a value, without any additional side-effects, such as printing to output, or modifying another memory value visible to Haskell code, then yes, I think you can skip IO layer. In other words, any mutations that are carried out by C function must not be visible to the calling code. That is the criteria I use when deciding whether to use IO wrapper or not, for C FFI functions. For example, FFI signature for sin function in GNU C math.h can be declared without IO wrapper. The sin function has a hidden internal state but it is not visible to the calling code. It may acquire its own memory, do something with it, and release it after it is done, but as long as it doesn't touch the region of memory owned by Haskell code, or deterministically communicate with it in anyway other than the return value, its state can be considered hidden. Printing to output is a form of communication. On Sat, Mar 3, 2012 at 1:24 PM, Victor Miller wrote: > I'd like to write reasonable Haskell interfaces to a few external > libraries. To be specific -- all of them do limited calculations (no > I/O) but each has an internal state. I know that I can have results > dump everything into the IO monad, but I'd like to avoid the "sin bin" > (I think it was Phil Wadler who called it that). To be specific, I > have in mind the library pari ( http://en.wikipedia.org/wiki/PARI/GP ) > which does symbolic algebra calculations. The way that the pari state > works is that state of works is that we can add definitions of new > quantities, and bind them to a calculated value. As part of the state > there might be certain cached calculations. If I avoid deleting bound > objects, it seems to me that it would be perfectly valid to make the > interface into a monad have a complicated state which is being managed > by the pari library. As long as any queries from pari give the same > value (which should be the case if I never delete anything) then it > should be valid without dumping things into the IO monad. Does anyone > see anything wrong with my reasoning? > > Victor > > ___ > Haskell-Cafe mailing list > Haskell-Cafe@haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe > ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] FFI: Overhead of foreign unsafe imports
On Sun, Feb 26, 2012 at 1:36 PM, Yves Parès wrote:
> Hello,
> When I was using C code from Python, the overhead put on calling C code by
> Python was significant.
> To simplify, say I have on C-side two procedures f and g, I do all the
> stuff to call them in a row from Python, well I'm better off factorizing:
> adding on C side a wrapper h procedure that calls them both, and then call
> h from Python, because then I will have half as much overhead:
>
> Instead of SwitchToC -> call f -> SwitchToPython -> SwitchToC -> call g ->
> SwitchToPython,
> the factorization leads to SwitchToC -> call f -> call g -> SwitchToPython,
> which gives the same result yet is different performance-wise because each
> switching has a cost.
>
> This is painful, because if another time I have to call f and j (another
> function), then I have to make another wrapper.
>
> In Haskell world, now, given that my functions f and g would have been
> imported using *unsafe*:
>
> foreign import unsafe "f" f :: Thing -> Foo -> IO ()
> foreign import unsafe "g" g :: Stuff -> Bar -> IO ()
> foreign import unsafe "h" h :: Thing -> Foo -> Stuff -> Bar -> IO ()
>
> Are
> doStuff = f x y >> g z w
> and
> doStuff = h x y z w
> equivalent, or is there an overhead (e.g. due to IO monad, or due to the
> way the FFI does the calls) when compiled (if need be with optimizations)
> with GHC?
>
Each unsafe call to FFI should be pretty fast - I have measured it to be
about 15ns on my computer (~30-50 clock cycles). Assuming C implementation
of (f;g) and h take about the same time in C, first version of doStuff
would likely be a bit slower than second version because of one additional
FFI call - I would expect it to take ~15ns more on my computer. From what I
have seen in my code, GHC optimizes away IO monad when compiled with -O2
flag. So, impact of IO monad on performance should usually be
negligible/close to zero.
Keep in mind that unsafe call to FFI will block the OS capability used up
for that FFI thread until that FFI call returns (more here:
http://blog.melding-monads.com/category/haskell/). So, if it takes long
time to execute, you might want to use the safe version instead. I have
seen safe version take about ~150ns in my tests.
Like Jason said, you could create a micro-benchmark to see the difference.
For example, like below:
Haskell code:
import Control.Monad (forM_)import Foreign.C.Types (CInt)import
Data.Time.Clock (diffUTCTime, getCurrentTime)import Foreign.C
foreign import ccall safe "print"
printsafe :: CInt -> IO ()foreign import ccall unsafe "print"
printunsafe :: CInt -> IO ()
main = do
let l = 5
a <- getCurrentTime
forM_ [1..l] $ \x -> printsafe x
b <- getCurrentTime
forM_ [1..l] $ \x -> printunsafe x
c <- getCurrentTime
print $ "safe call average overhead " ++ show ((diffUTCTime b
a)/fromIntegral l)
print $ "unsafe call average overhead " ++ show ((diffUTCTime c
b)/fromIntegral l)
C code:
#include
void print(int x){
return;}
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Experiments in Haskell Packaging
Hi Chris, I have created a ticket request for user-space install of GHC and haskell platform. For some of us, development environment is very similar to production environment (minus the dev tools) to keep the deployment procedures consistent, and simple. So, we don't have root privileges in dev environment too. But, this is not really a big deal as all of the development tools that I am familiar with, can be installed in user-land in user-defined locations. Thanks, Sanket On Thu, Jan 5, 2012 at 5:15 AM, Chris Dornan wrote: > Hi Sanket, > > ** ** > > If you like you can put a ticket into the issue > tracker<https://github.com/haskell-hub/hub-gen/issues>(otherwise I will do it > myself). > > > ** ** > > I am sorry that you have had to go to such lengths to get Haskell working > on your systems. > > ** ** > > Unfortunately I don’t think it is going to be straightforward to make the > packages relocatable. > > ** ** > > Could you say a little more about why you need to do this? Do you need a > user-space install? > > ** ** > > I had targeted this distribution primarily at development systems and > build servers where I would expect populating /usr/hs with read-only data > and code wouldn’t be a problem. > > ** ** > > It sounds like you might be immediately better off working directly with > the source code. I can help with this obviously as I have had to solve > these problems. (The REHEL5-supplied gcc and binutils tools are too to > build recent GHC releases, so I use more recent tools, which I also package > with the distribution.) > > ** ** > > In the medium/long term It may be easier to address these needs with a > separate ‘lite’ distribution focused on field-deployment with > relocatablepackages, which may as well be user-space deployable too. > > > ** ** > > This will need to be researched and could take some time. > > ** ** > > If anybody else is interested in this then please get in contact. > > ** ** > > Also, if anybody has any other gripes or ideas about how Haskell > distributions generally it would be good to hear. > > ** ** > > Chris > > ** ** > > *From:* Sanket Agrawal [mailto:sanket.agra...@gmail.com] > *Sent:* 05 January 2012 01:36 > *To:* Chris Dornan > *Cc:* haskell-cafe@haskell.org > *Subject:* Re: [Haskell-cafe] Experiments in Haskell Packaging > > ** ** > > Hi Chris, > > ** ** > > Is it possible to also make the GHC and haskell platform distributions > available as relocatable rpms, instead of HUB packaging? The reason is that > for some of us, when we work in complex production environment which has > its own deployment procedures, we have to deploy it differently. A > hard-coded path like /usr/hs won't work, and a packaging environment that > depends on that hard-coded path won't work either in that case. > > ** ** > > I actually downloaded RHEL5 GHC binary rpm from justhub.org, and unpacked > it using rpm2cpio. Then, I ended up doing grep and sed to replace all > occurrences of "/usr/hs" with the path I installed it in. It worked well > except for some caveats - some of the binary/data files such as *.hi seem > to have the hard-coded path in them. That hasn't caused any issues so far, > except for haddock which fails if gcc is called since it is coded to > /usr/hs/bin/gcc (though haddock bash shell has right gcc path coded in it - > I guess it ignores it in favor of the path in some binary file). > > ** ** > > It will be very nice to have what I did above, as relocatable rpms, and > without HUB dependency. That will help with, I suspect, reducing the > duplication of efforts when some of us, who want to use haskell at work, > end up compiling our own binaries for RHEL platform (since glibc on RHEL > 5/6 is older than what ghc-pwd demands). I tried to compile GHC 7.4 on RHEL > 5, but it failed to link because of some GHC bug. That is how I ended up > using your compiled binary with my grep/sed hack. It was a great > time-saver. Thank you for doing this work. > > ** ** > > -Sanket > > ** ** > > On Tue, Jan 3, 2012 at 7:49 AM, Chris Dornan > wrote: > > 'Antoine Latter' : > > > All of this is in my head, but assuming I already had some sort of > > Linux build-server set up, it would be nice to combine it with your > > work to make it easier to have the build-server run tests against > > multiple versions of GHC/HP. That's all I was getting at. > > Working within the Hub framework this is straightforward. If you are using > public releases rather than special builds of the
Re: [Haskell-cafe] Experiments in Haskell Packaging
Hi Chris,
Is it possible to also make the GHC and haskell platform distributions
available as relocatable rpms, instead of HUB packaging? The reason is that
for some of us, when we work in complex production environment which has
its own deployment procedures, we have to deploy it differently. A
hard-coded path like /usr/hs won't work, and a packaging environment that
depends on that hard-coded path won't work either in that case.
I actually downloaded RHEL5 GHC binary rpm from justhub.org, and unpacked
it using rpm2cpio. Then, I ended up doing grep and sed to replace all
occurrences of "/usr/hs" with the path I installed it in. It worked well
except for some caveats - some of the binary/data files such as *.hi seem
to have the hard-coded path in them. That hasn't caused any issues so far,
except for haddock which fails if gcc is called since it is coded to
/usr/hs/bin/gcc (though haddock bash shell has right gcc path coded in it -
I guess it ignores it in favor of the path in some binary file).
It will be very nice to have what I did above, as relocatable rpms, and
without HUB dependency. That will help with, I suspect, reducing the
duplication of efforts when some of us, who want to use haskell at work,
end up compiling our own binaries for RHEL platform (since glibc on RHEL
5/6 is older than what ghc-pwd demands). I tried to compile GHC 7.4 on RHEL
5, but it failed to link because of some GHC bug. That is how I ended up
using your compiled binary with my grep/sed hack. It was a great
time-saver. Thank you for doing this work.
-Sanket
On Tue, Jan 3, 2012 at 7:49 AM, Chris Dornan wrote:
> 'Antoine Latter' :
>
> > All of this is in my head, but assuming I already had some sort of
> > Linux build-server set up, it would be nice to combine it with your
> > work to make it easier to have the build-server run tests against
> > multiple versions of GHC/HP. That's all I was getting at.
>
> Working within the Hub framework this is straightforward. If you are using
> public releases rather than special builds of the tools then it should be
> quite easy.
>
> Supposing you needed to test with Haskell Platform 2011.2.0.1 and
> hexpat-0.19.7:
>
># install the platform (probably done already)
>
>sudo yum install haskell-platform-2011.2.0.1-hub
>
> # create new hub 'test-2011.2.0.1' based this HP
>
> hub init 2011.2.0.1 test-2011.2.0.1
>
> # install hexpat-0.19.7 into it
>
> hub install-into test-2011.2.0.1 hexpat-2011.2.0.1
>
> Repeat as necessary to create your array of test environments.
>
> Now you just have to arrange select the appropriate hub for each test.
> Suppose the environment variable $testhubs contains all of your test hubs
> ('test-2011.2.0.1', etc.), then the following shell command would prime the
> tools to successively use each of the test environments on each invocation
> of make.
>
>for h in $testhubs; do HUB=$h make; done
>
> Note that the justhub distribution isn't necessary for this -- you can do
> it manually.
>
> But it does make it easier.
>
> Chris
>
>
> ___
> Haskell-Cafe mailing list
> Haskell-Cafe@haskell.org
> http://www.haskell.org/mailman/listinfo/haskell-cafe
>
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] GHC compilation issue: Working around bug #4374?
My apologies for posting it here. Glasgow-haskell-users mailing list seems
more relevant for this question. So, I am going to send it there instead,
with a note about this cross-posting.
On Tue, Jan 3, 2012 at 4:26 PM, Sanket Agrawal wrote:
>
>
> I am getting the compilation error below when building GHC 7.0.4 on RHEL5
> (x86_64) – I configured make to point to ncurses and libgmp. I will
> appreciated pointers on how to work around the issue below. I don’t have
> root privileges. My apologies if this has already been discussed in this
> forum. I searched for the issue but could find only the GHC bug ticket
> below.
>
> GHC bug ticket: http://hackage.haskell.org/trac/ghc/ticket/4374 - it
> doesn’t have description on how to work around the bug.
>
> Compilation error:
> ---
> $
> C_INCLUDE_PATH=/efs/dist/fsf/ncurses/5.6/common/include/ncurses:/home/saagrawa/scripts/misc/libgmp/include
> LIBRARY_PATH=/efs/dist/fsf/ncurses/5.6/exec/lib:/home/saagrawa/scripts/misc/libgmp/lib
> make
> ===--- updating makefiles phase 0
> make -r --no-print-directory -f ghc.mk phase=0 just-makefiles
> ===--- updating makefiles phase 1
> make -r --no-print-directory -f ghc.mk phase=1 just-makefiles
>
> ===--- updating makefiles phase 2
> make -r --no-print-directory -f ghc.mk phase=2 just-makefiles
> ===--- updating makefiles phase 3
> make -r --no-print-directory -f ghc.mk phase=3 just-makefiles
> ===--- finished updating makefiles
> make -r --no-print-directory -f ghc.mk all
> "inplace/bin/ghc-stage1"
> libraries/integer-gmp/dist-install/build/GHC/Integer.dyn_o
> libraries/integer-gmp/dist-install/build/GHC/Integer/GMP/Internals.dyn_o
> libraries/integer-gmp/dist-install/build/GHC/Integer/Type.dyn_o
> libraries/integer-gmp/dist-install/build/cbits/gmp-wrappers.dyn_o
> libraries/integer-gmp/dist-install/build/cbits/cbits.dyn_o
> libraries/integer-gmp/gmp/objs/*.o `/usr/bin/find
> libraries/integer-gmp/dist-install/build -name "*_stub.dyn_o" -print`
> -shared -dynamic -dynload deploy -dylib-install-name
> /home/saagrawa/scripts/misc/ghc/7.0.4/lib/ghc-7.0.4/`basename
> "libraries/integer-gmp/dist-install/build/
> libHSinteger-gmp-0.2.0.3-ghc7.0.4.so" | sed
> 's/^libHS//;s/[-]ghc.*//'`/`basename
> "libraries/integer-gmp/dist-install/build/
> libHSinteger-gmp-0.2.0.3-ghc7.0.4.so"` -no-auto-link-packages -package
> ghc-prim-0.2.0.0 -o libraries/integer-gmp/dist-install/build/
> libHSinteger-gmp-0.2.0.3-ghc7.0.4.so
> /efs/dist/fsf/gcc/4.4.3-build004/.exec/x86-64.rhel.5/libexec/gcc/x86_64-unknown-linux-gnu/4.4.3/ld:
> libraries/integer-gmp/gmp/objs/aors.o: relocation R_X86_64_32 against
> `__gmpz_sub' can not be used when making a shared object; recompile with
> -fPIC
> libraries/integer-gmp/gmp/objs/aors.o: could not read symbols: Bad value
> collect2: ld returned 1 exit status
> make[1]: *** [libraries/integer-gmp/dist-install/build/
> libHSinteger-gmp-0.2.0.3-ghc7.0.4.so] Error 1
> make: *** [all] Error 2
>
> $ ll /home/saagrawa/scripts/misc/libgmp/lib|awk '{print $9,$10,$11}' ###
> the libgmp lib folder has dynamic libraries
> **
> **
> libgmp.so.3.5.2*
> libgmp.so.3 -> libgmp.so.3.5.2*
> libgmp.so -> libgmp.so.3.5.2*
> libgmp.la*
> libgmp.a
>
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] GHC compilation issue: Working around bug #4374?
I am getting the compilation error below when building GHC 7.0.4 on RHEL5
(x86_64) – I configured make to point to ncurses and libgmp. I will appreciated
pointers on how to work around the issue below. I don’t have root privileges.
My apologies if this has already been discussed in this forum. I searched for
the issue but could find only the GHC bug ticket below.
GHC bug ticket: http://hackage.haskell.org/trac/ghc/ticket/4374 - it doesn’t
have description on how to work around the bug.
Compilation error:
---
$
C_INCLUDE_PATH=/efs/dist/fsf/ncurses/5.6/common/include/ncurses:/home/saagrawa/scripts/misc/libgmp/include
LIBRARY_PATH=/efs/dist/fsf/ncurses/5.6/exec/lib:/home/saagrawa/scripts/misc/libgmp/lib
make
===--- updating makefiles phase 0
make -r --no-print-directory -f ghc.mk phase=0 just-makefiles
===--- updating makefiles phase 1
make -r --no-print-directory -f ghc.mk phase=1 just-makefiles
===--- updating makefiles phase 2
make -r --no-print-directory -f ghc.mk phase=2 just-makefiles
===--- updating makefiles phase 3
make -r --no-print-directory -f ghc.mk phase=3 just-makefiles
===--- finished updating makefiles
make -r --no-print-directory -f ghc.mk all
"inplace/bin/ghc-stage1"
libraries/integer-gmp/dist-install/build/GHC/Integer.dyn_o
libraries/integer-gmp/dist-install/build/GHC/Integer/GMP/Internals.dyn_o
libraries/integer-gmp/dist-install/build/GHC/Integer/Type.dyn_o
libraries/integer-gmp/dist-install/build/cbits/gmp-wrappers.dyn_o
libraries/integer-gmp/dist-install/build/cbits/cbits.dyn_o
libraries/integer-gmp/gmp/objs/*.o `/usr/bin/find
libraries/integer-gmp/dist-install/build -name "*_stub.dyn_o" -print` -shared
-dynamic -dynload deploy -dylib-install-name
/home/saagrawa/scripts/misc/ghc/7.0.4/lib/ghc-7.0.4/`basename
"libraries/integer-gmp/dist-install/build/libHSinteger-gmp-0.2.0.3-ghc7.0.4.so"
| sed 's/^libHS//;s/[-]ghc.*//'`/`basename
"libraries/integer-gmp/dist-install/build/libHSinteger-gmp-0.2.0.3-ghc7.0.4.so"`
-no-auto-link-packages -package ghc-prim-0.2.0.0 -o
libraries/integer-gmp/dist-install/build/libHSinteger-gmp-0.2.0.3-ghc7.0.4.so
/efs/dist/fsf/gcc/4.4.3-build004/.exec/x86-64.rhel.5/libexec/gcc/x86_64-unknown-linux-gnu/4.4.3/ld:
libraries/integer-gmp/gmp/objs/aors.o: relocation R_X86_64_32 against
`__gmpz_sub' can not be used when making a shared object; recompile with -fPIC
libraries/integer-gmp/gmp/objs/aors.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
make[1]: ***
[libraries/integer-gmp/dist-install/build/libHSinteger-gmp-0.2.0.3-ghc7.0.4.so]
Error 1
make: *** [all] Error 2
$ ll /home/saagrawa/scripts/misc/libgmp/lib|awk '{print $9,$10,$11}' ### the
libgmp lib folder has dynamic libraries
libgmp.so.3.5.2*
libgmp.so.3 -> libgmp.so.3.5.2*
libgmp.so -> libgmp.so.3.5.2*
libgmp.la*
libgmp.a___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] How to define Storable instance for data constructor with storable vectors
I am trying to write a union data structure which can store vectors - a
simplified definition of data structure is below:
-
-- V is Data.Vector.Storable
data Elems = I {-# UNPACK #-} !GHC.Int.Int32
| S {-# UNPACK #-} !GHC.Int.Int32 {-# UNPACK #-} !(Ptr CChar)
| T {-# UNPACK #-} !(V.Vector Elems)
-
What I can't figure out is how to store Vector elements when defining
Storable instance for Elems type - the main stumbling block is that my
approach requires defining a Storable instance of ForeignPtr. Snippets of
code below:
-
instance Storable Elems where
sizeOf _ = ...
alignment _ = ...
{-# INLINE poke #-}
poke p x = case x of
...
T x -> do
poke (castPtr p :: Ptr Word8) 3 -- store a tag for data
constructor T. We will check it when doing peek
let (fp,_,n) = V.unsafeToForeignPtr x
poke (castPtr p1) n
poke (castPtr (p1 `plusPtr` 8)) fp
where p1 = (castPtr p :: Ptr Word8) `plusPtr` 1 -- get pointer to
start of the element. First byte is type of element
{-# INLINE peek #-}
peek p = do
let p1 = (castPtr p::Ptr Word8) `plusPtr` 1 -- get pointer to
start of the element. First byte is type of element
t <- peek (castPtr p::Ptr Word8)
case t of
-- handle all data constructors except T here
_ -> do
x <- peek (castPtr p1 :: Ptr Int)
y <- peek (castPtr (p1 `plusPtr` 8) :: Ptr (ForeignPtr
Elems))
return (T (V.unsafeFromForeignPtr y 0 x)) -- return
vector elements
-
If I use above approach, I will need to define Storable instance of
ForeignPtr for Elems data type. Is it possible to define peek/poke
operations for storing vectors, using something other than ForeignPtr? I
prefer using ForeignPtr because it has finalizer associated with it. I
guess it might be possible to store Ptr through withForeignPtr, but then
how do you convert back from the Ptr to Vector in peek?
If storing the ForeignPtr is a cleaner approach, how would Storable
definition for ForeignPtr look like? GHC code for ForeignPtr seems to
export only ForeignPtr constructor.
I am aware above code is not portable (e.g., assumption that Int is 8
bytes). That is not really an issue because I am writing code for a very
specific platform. If there is a library defined somewhere which has
already dealt with similar issue, I will very much appreciate pointers.
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
