Re: [Haskell-cafe] Strange results when trying to create large Bool arrays.
Ketil Malde wrote: I'd really like to have errors on overflow, at least as an option, even if it is costly in terms of performance. Is there a Trac ticket or something for this? Not that I know of. I filed a Trac ticket against ByteString's readInt function before I noticed that read has the same problem, and it was closed because read does the same thing. I've been reluctant to pop my head over the parapet since. CPUs generally don't trap on integer overflow, so generating the additional tests and jumps necessary to handle this would be a bit involved, and would certainly cost a few percent in performance. There's also overflow in truncation and sign conversions to worry about, e.g. Word32 -> Word16, Word32 -> Int (on 32-bit systems), etc. On the other hand, integer overflows have long been a popular attack vector for getting programs to misbehave in the exploit community. If you Google for "ia32 integer overflow" or "i386 integer overflow", the first several *pages* of results in each case consist entirely of security advisories. Haskell's FFI makes it as vulnerable as the libraries it interfaces to. Here's a cute-looking paper entitled "Efficient and accurate detection of integer-based attacks". http://www.cs.cmu.edu/~dbrumley/pubs/integer-ndss-07.pdf http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Strange results when trying to create large Bool arrays.
On Wed, 2007-07-11 at 10:55 -0700, Bryan O'Sullivan wrote: > In a similar vein, I was initially perplexed when I > found that an expression like this produces garbage instead of an error: > >read "111" :: Int > > I have not seen a lot of interest expressed in fixing this sort of > misbehaviour, which jars a little with the usual emphasis on stringency > and testing. I'd really like to have errors on overflow, at least as an option, even if it is costly in terms of performance. Is there a Trac ticket or something for this? -k ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Strange results when trying to create large Bool arrays.
On Jul 11, 2007, at 19:11 , Thomas Conway wrote:
Is there a compelling reason (hysterical raisins is not a compelling
reason) why Data.*.{length,size,take,drop,etc} use Int and not Num n
=> or similar?
Efficiency, but many of them have generic equivalents (e.g.
genericLength, genericDrop, etc.) at least in Data.List.
--
brandon s. allbery [solaris,freebsd,perl,pugs,haskell] [EMAIL PROTECTED]
system administrator [openafs,heimdal,too many hats] [EMAIL PROTECTED]
electrical and computer engineering, carnegie mellon universityKF8NH
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Strange results when trying to create large Bool arrays.
On 7/12/07, Stefan O'Rear <[EMAIL PROTECTED]> wrote:
Indeed. I beleive that Int should be removed from the Prelude.
metoo.
Actually, one of the really annoying things that I am finding in my
code which is a mixture of ByteString, Word16, Word32 and Word64, is
that all the standard libraries use Int everywhere, so I end up having
to get out the fromIntegral spray-gun, which uglifies the code and
makes it much harder to read.
Is there a compelling reason (hysterical raisins is not a compelling
reason) why Data.*.{length,size,take,drop,etc} use Int and not Num n
=> or similar?
If the answer is efficiency, then cannot they use Int# or similar
internally, and have an optimizable fromIntegral *inside* them?
T.
--
Dr Thomas Conway
[EMAIL PROTECTED]
Silence is the perfectest herald of joy:
I were but little happy, if I could say how much.
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Strange results when trying to create large Bool arrays.
On Wed, Jul 11, 2007 at 08:16:50PM +0100, Andrew Coppin wrote: > Bryan O'Sullivan wrote: >> Richard Kelsall wrote: >> >>> I guess there must be a switch to make it produce a nice >>> error message rather than overflowing without warning. >> >> Actually, there isn't. > > I for one sometimes wish there was... > > Of course, sometimes you purposely write code which you know is going to > overflow and wrap round in a specific way. But frequently you *don't* want > this behaviour - and I wish there were some pragma or something to make > this be checked. AFAIK, most CPU types give you an efficient way to testing > for such conditions... Indeed. I beleive that Int should be removed from the Prelude. People who need the algebraic properties of rings modulo 2^(2^n) can use the sized integral types from Data.Int and Data.Word; people who want speed and can satisfy the proof obligations can use Int and Word from the same modules. Everyone else can use Integer, which should be made shorter than Int for obvious psychological reasons. Stefan signature.asc Description: Digital signature ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Strange results when trying to create large Bool arrays.
Bryan O'Sullivan wrote: Richard Kelsall wrote: I guess there must be a switch to make it produce a nice error message rather than overflowing without warning. Actually, there isn't. I for one sometimes wish there was... Of course, sometimes you purposely write code which you know is going to overflow and wrap round in a specific way. But frequently you *don't* want this behaviour - and I wish there were some pragma or something to make this be checked. AFAIK, most CPU types give you an efficient way to testing for such conditions... ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Strange results when trying to create large Bool arrays.
Albert Y. C. Lai wrote: I'm just being picky here: where the underlying machine's behaviour is 2's complement binary, it (Int, +, *) is actually a tidy, well-behaved mathematical ring, isomorphic to Z / 2^n Z. Yes, naturally it wasn't until a few moments after I had sent the message that I noticed my error. http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Strange results when trying to create large Bool arrays.
Bryan O'Sullivan wrote: Int is a bit of an odd fish that way; it's a window onto the underlying machine's behaviour, not a tidy, well-behaved mathematical ring. I'm just being picky here: where the underlying machine's behaviour is 2's complement binary, it (Int, +, *) is actually a tidy, well-behaved mathematical ring, isomorphic to Z / 2^n Z. Furthermore, if there were overflow errors, the result would not be a ring, but an error monad over a ring or a CPO lifting a ring. (Not to say that they are not tidy, well-behaved, mathematical.) ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Strange results when trying to create large Bool arrays.
On Wed, Jul 11, 2007 at 10:55:28AM -0700, Bryan O'Sullivan wrote: > Richard Kelsall wrote: >> It seems to randomly claim to have successfully created huge sizes >> of array. > > This may be outside of Haskell's control, as you're not actually touching > the memory you allocate. I wouldn't be surprised if the underlying page > allocation is succeeding by virtue of the OS being willing to overcommit > resources that may not actually be used. (This would be normal behaviour > on Linux, for example.) In such a case, the Haskell runtime may not > receive an error until you try to actually touch the data. Not a sufficient explanation - you can only allocate addrssable memory, even on Linux I can't allocate more than 3GB of storage. > You can get GHC to fix an upper limit on the heap size it will try to use, > by passing "+RTS -M768m -RTS" to your compiled program on the command line. > That should cause your program to crash more reliably. That still won't work because GHC trims the array size to an Int interally, and trims the byte count to an Int again before passing it to the OS. [EMAIL PROTECTED]:~$ ghci Loading package base ... linking ... done. ___ ___ _ / _ \ /\ /\/ __(_) / /_\// /_/ / / | |GHC Interactive, version 6.7.20070612, for Haskell 98. / /_\\/ __ / /___| |http://www.haskell.org/ghc/ \/\/ /_/\/|_|Type :? for help. Prelude> :m + Data.Array.Unboxed Prelude Data.Array.Unboxed> array (0,maxBound :: Int) [(-2, 42::Int)] :: UArray Int Int array Segmentation fault [EMAIL PROTECTED]:~$ http://hackage.haskell.org/trac/ghc/ticket/229 (yes, that's a 3 digit bug number) Stefan signature.asc Description: Digital signature ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Strange results when trying to create large Bool arrays.
Richard Kelsall wrote: main = do n <- getArgs >>= readIO . head :: IO Int a <- newArray (1,n) True :: IO (IOUArray Int Bool) printf "Created array 1 .. %8d \n" (n::Int) :: IO () It appears to work up to quite large numbers, but then gets strange. When I give it an array size of 1,000,000,000,000 it returns this Created array 1 .. -727379968 Presumably the Int has overflowed without warning when read from the argument. Yes, that's right. I guess there must be a switch to make it produce a nice error message rather than overflowing without warning. Actually, there isn't. Int is a bit of an odd fish that way; it's a window onto the underlying machine's behaviour, not a tidy, well-behaved mathematical ring. In a similar vein, I was initially perplexed when I found that an expression like this produces garbage instead of an error: read "111" :: Int I have not seen a lot of interest expressed in fixing this sort of misbehaviour, which jars a little with the usual emphasis on stringency and testing. It seems to randomly claim to have successfully created huge sizes of array. This may be outside of Haskell's control, as you're not actually touching the memory you allocate. I wouldn't be surprised if the underlying page allocation is succeeding by virtue of the OS being willing to overcommit resources that may not actually be used. (This would be normal behaviour on Linux, for example.) In such a case, the Haskell runtime may not receive an error until you try to actually touch the data. You can get GHC to fix an upper limit on the heap size it will try to use, by passing "+RTS -M768m -RTS" to your compiled program on the command line. That should cause your program to crash more reliably. http://www.haskell.org/mailman/listinfo/haskell-cafe
