[Haskell-cafe] Slow Text.JSON parser
Dear list members,
I tried Text.JSON from hackage and did an initial test to see how well
it performs. I created a single JSON file of roughly 6 MB containing a
single JSON array with 30906 JSON objects and used the following code
to parse it:
module Main where
import System.IO
import Data.Time.Clock
import System.Environment
import Text.Printf
import Text.JSON
parse s = do
start <- getCurrentTime
let !len = decode s
end <- getCurrentTime
print len
printf "Elapsed time = %s\n" (show $ diffUTCTime end start)
where
decode s = case decodeStrict s of
Ok (JSArray a) -> length a
_ -> -1
main = do
file <- getArgs >>= return . head
withFile file ReadMode (\h -> hGetContents h >>= parse)
The outcome was something like:
30906
Elapsed time = 2.902755s
on my 2GHz core 2 duo.
Another Java-based JSON parser (Jackson:
http://www.cowtowncoder.com/hatchery/jackson/index.html) gives me:
30906
Elapsed time = 480 ms
Now I wonder why Text.JSON is so slow in comparison and what can be
done about it. Any ideas? Or is the test case invalid?
Thanks,
Levi
---
The Java code for the Jackson test is:
import org.codehaus.jackson.JsonParser;
import org.codehaus.jackson.JsonFactory;
import org.codehaus.jackson.map.JsonTypeMapper;
import org.codehaus.jackson.map.JsonNode;
import java.io.File;
class Test {
public static void main(String[] args) throws Exception {
final long start = System.currentTimeMillis();
final JsonTypeMapper mapper = new JsonTypeMapper();
final JsonParser parser = new
JsonFactory().createJsonParser(new File(args[0]));
final JsonNode root = mapper.read(parser);
final long end = System.currentTimeMillis();
System.out.println(root.size());
System.out.println(String.format("Elapsed time = %d ms", end - start));
}
}
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Slow Text.JSON parser
"Levi Greenspan" writes: > Now I wonder why Text.JSON is so slow in comparison and what can be > done about it. Any ideas? Or is the test case invalid? I haven't used JSON, but at first glance, I'd blame String IO. Can't you decode from ByteString? -k -- If I haven't seen further, it is by standing in the footprints of giants ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Slow Text.JSON parser
ketil: > "Levi Greenspan" writes: > > > Now I wonder why Text.JSON is so slow in comparison and what can be > > done about it. Any ideas? Or is the test case invalid? > > I haven't used JSON, but at first glance, I'd blame String IO. Can't > you decode from ByteString? > Text.JSON was never optimised for performance. It was designed for small JSON objects. For things above 1M I'd suggest using Data.Binary (or a quick JSON encoding over bytestrings). Shouldn't be too hard to prepare. -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Slow Text.JSON parser
JSON is a UNICODE format, like any modern format is today. ByteStrings are not going to work. If everybody starts yelling "ByteString" every time String performance is an issue, I don't see how Haskell is ever going to be a "real world programming language". On Jan 13, 2009, at 4:00 PM, Don Stewart wrote: ketil: "Levi Greenspan" writes: Now I wonder why Text.JSON is so slow in comparison and what can be done about it. Any ideas? Or is the test case invalid? I haven't used JSON, but at first glance, I'd blame String IO. Can't you decode from ByteString? Text.JSON was never optimised for performance. It was designed for small JSON objects. For things above 1M I'd suggest using Data.Binary (or a quick JSON encoding over bytestrings). Shouldn't be too hard to prepare. -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- Sjoerd Visscher sjo...@w3future.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Slow Text.JSON parser
utf8-string allows one to decode utf8 from bytestrings. It was built so that we could decode utf8 strings at work from bytestrings :) http://hackage.haskell.org/packages/archive/utf8-string/0.3.3/doc/html/Data-ByteString-UTF8.html Enjoy! Libraries win every day of the week. -- Don sjoerd: > JSON is a UNICODE format, like any modern format is today. ByteStrings > are not going to work. > > If everybody starts yelling "ByteString" every time String performance > is an issue, I don't see how Haskell is ever going to be a "real world > programming language". > > On Jan 13, 2009, at 4:00 PM, Don Stewart wrote: > > >ketil: > >>"Levi Greenspan" writes: > >> > >>>Now I wonder why Text.JSON is so slow in comparison and what can be > >>>done about it. Any ideas? Or is the test case invalid? > >> > >>I haven't used JSON, but at first glance, I'd blame String IO. Can't > >>you decode from ByteString? > >> > > > >Text.JSON was never optimised for performance. It was designed for > >small > >JSON objects. For things above 1M I'd suggest using Data.Binary (or a > >quick JSON encoding over bytestrings). Shouldn't be too hard to > >prepare. > > > >-- Don > >___ > >Haskell-Cafe mailing list > >Haskell-Cafe@haskell.org > >http://www.haskell.org/mailman/listinfo/haskell-cafe > > -- > Sjoerd Visscher > sjo...@w3future.com > > > > ___ > Haskell-Cafe mailing list > Haskell-Cafe@haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Slow Text.JSON parser
On Tue, Jan 13, 2009 at 4:39 PM, Sjoerd Visscher wrote: > JSON is a UNICODE format, like any modern format is today. ByteStrings are > not going to work. I don't understand this statement. Why can one not make a parser from ByteStrings that can decode UTF-8? Luke > > > If everybody starts yelling "ByteString" every time String performance is > an issue, I don't see how Haskell is ever going to be a "real world > programming language". > > > On Jan 13, 2009, at 4:00 PM, Don Stewart wrote: > > ketil: >> >>> "Levi Greenspan" writes: >>> >>> Now I wonder why Text.JSON is so slow in comparison and what can be done about it. Any ideas? Or is the test case invalid? >>> >>> I haven't used JSON, but at first glance, I'd blame String IO. Can't >>> you decode from ByteString? >>> >>> >> Text.JSON was never optimised for performance. It was designed for small >> JSON objects. For things above 1M I'd suggest using Data.Binary (or a >> quick JSON encoding over bytestrings). Shouldn't be too hard to prepare. >> >> -- Don >> ___ >> Haskell-Cafe mailing list >> Haskell-Cafe@haskell.org >> http://www.haskell.org/mailman/listinfo/haskell-cafe >> > > -- > Sjoerd Visscher > sjo...@w3future.com > > > > > ___ > Haskell-Cafe mailing list > Haskell-Cafe@haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe > ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Slow Text.JSON parser
It is not impossible, but a lot of work. And if you want to do it correctly you would have to support UTF-16 (BE of LE) and UTF-32 (BE of LE) as well. You can't expect someone to start writing utf encoders and decoders every time he needs a fast parser. Sjoerd On Jan 14, 2009, at 12:42 AM, Luke Palmer wrote: On Tue, Jan 13, 2009 at 4:39 PM, Sjoerd Visscher wrote: JSON is a UNICODE format, like any modern format is today. ByteStrings are not going to work. I don't understand this statement. Why can one not make a parser from ByteStrings that can decode UTF-8? Luke If everybody starts yelling "ByteString" every time String performance is an issue, I don't see how Haskell is ever going to be a "real world programming language". On Jan 13, 2009, at 4:00 PM, Don Stewart wrote: ketil: "Levi Greenspan" writes: Now I wonder why Text.JSON is so slow in comparison and what can be done about it. Any ideas? Or is the test case invalid? I haven't used JSON, but at first glance, I'd blame String IO. Can't you decode from ByteString? Text.JSON was never optimised for performance. It was designed for small JSON objects. For things above 1M I'd suggest using Data.Binary (or a quick JSON encoding over bytestrings). Shouldn't be too hard to prepare. -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- Sjoerd Visscher sjo...@w3future.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- Sjoerd Visscher sjo...@w3future.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Slow Text.JSON parser
On 2009 Jan 13, at 18:54, Sjoerd Visscher wrote: It is not impossible, but a lot of work. And if you want to do it correctly you would have to support UTF-16 (BE of LE) and UTF-32 (BE of LE) as well. You can't expect someone to start writing utf encoders and decoders every time he needs a fast parser. ...whereas making a linked list of Word32 run quickly is trivial? -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allb...@kf8nh.com system administrator [openafs,heimdal,too many hats] allb...@ece.cmu.edu electrical and computer engineering, carnegie mellon universityKF8NH ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Slow Text.JSON parser
On 2009 Jan 13, at 22:43, Brandon S. Allbery KF8NH wrote: On 2009 Jan 13, at 18:54, Sjoerd Visscher wrote: It is not impossible, but a lot of work. And if you want to do it correctly you would have to support UTF-16 (BE of LE) and UTF-32 (BE of LE) as well. You can't expect someone to start writing utf encoders and decoders every time he needs a fast parser. ...whereas making a linked list of Word32 run quickly is trivial? Correction: a linked list of *indirect* Word32s. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] allb...@kf8nh.com system administrator [openafs,heimdal,too many hats] allb...@ece.cmu.edu electrical and computer engineering, carnegie mellon universityKF8NH ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Slow Text.JSON parser
Sjoerd Visscher writes: > JSON is a UNICODE format, like any modern format is today. ByteStrings > are not going to work. Well, neither is String as used in the code I responded to. I'm not intimately familiar with JSON, but I believe ByteStrings would work on UTF-8 input, and both ByteString and String would fail on UTF-16 and UTF-32. > If everybody starts yelling "ByteString" every time String performance > is an issue, I don't see how Haskell is ever going to be a "real world > programming language". Insisting on linked lists of 32-bit characters isn't going to help, either. I'm also looking forward to a fast, robust, and complete UniCode support, but the OP asked about performance. -k -- If I haven't seen further, it is by standing in the footprints of giants ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Slow Text.JSON parser
Hi,
Somebody told me about Parsec 3, which uses a Stream type class so it
can parse any data type. This sounded like the right way to do
encoding independent parsing, so I decided to see how it would work to
parse UTF8 JSON.
Sadly I could not use Text.JSON.Parsec directly, because it uses the
old Parsec CharParser type. So I copied to code, and also replaced
p_number with the "floating" parser from Text.Parsec.Token, because
Text.JSON.Parsec uses readFloat (a dirty hack imho) which works only
on String.
If Text.JSON.Parsec was written for Parsec 3, the only thing to write
to get UTF8 JSON parsing would be:
instance (Monad m, U.UTF8Bytes string index) => Stream (U.UTF8 string)
m Char where
uncons = return . U.uncons
I did not do any performance measuring yet, I was glad I got it
working. Any comments on the code is appreciated!
greetings,
Sjoerd Visscher
{-# LANGUAGE FlexibleInstances, MultiParamTypeClasses,
UndecidableInstances #-}
import qualified Data.String.UTF8 as U
import qualified Data.ByteString as B
import Text.Parsec hiding (many, optional, (<|>))
import Control.Applicative
import Text.JSON.Types
import Control.Monad
import Data.Char
import Numeric
instance (Monad m, U.UTF8Bytes string index) => Stream (U.UTF8 string)
m Char where
uncons = return . U.uncons
type CharParser st = Parsec (U.UTF8 B.ByteString) st
parseFile :: FilePath -> IO (Either ParseError JSValue)
parseFile fileName = do
bs <- B.readFile fileName
return $ runParser json () fileName (U.fromRep bs)
parseString :: String -> Either ParseError JSValue
parseString s = runParser json () "(unknown)" (U.fromString s)
json :: CharParser () JSValue
json = spaces *> p_value
tok :: CharParser () a -> CharParser () a
tok p = p <* spaces
p_value :: CharParser () JSValue
p_value = (JSNull <$ p_null)
<|> (JSBool <$> p_boolean)
<|> (JSArray <$> p_array)
<|> (JSString<$> p_js_string)
<|> (JSObject<$> p_js_object)
<|> (JSRational False <$> p_number)
"JSON value"
p_null :: CharParser () ()
p_null= tok (string "null") >> return ()
p_boolean:: CharParser () Bool
p_boolean = tok
( (True <$ string "true")
<|> (False <$ string "false")
)
p_array :: CharParser () [JSValue]
p_array = between (tok (char '[')) (tok (char ']'))
$ p_value `sepBy` tok (char ',')
p_string :: CharParser () String
p_string = between (tok (char '"')) (char '"') (many p_char)
where p_char= (char '\\' >> p_esc)
<|> (satisfy (\x -> x /= '"' && x /= '\\'))
p_esc = ('"' <$ char '"')
<|> ('\\' <$ char '\\')
<|> ('/' <$ char '/')
<|> ('\b' <$ char 'b')
<|> ('\f' <$ char 'f')
<|> ('\n' <$ char 'n')
<|> ('\r' <$ char 'r')
<|> ('\t' <$ char 't')
<|> (char 'u' *> p_uni)
"escape character"
p_uni = check =<< count 4 (satisfy isHexDigit)
where check x | code <= max_char = pure (toEnum code)
| otherwise = empty
where code = fst $ head $ readHex x
max_char = fromEnum (maxBound :: Char)
p_object :: CharParser () [(String,JSValue)]
p_object = between (tok (char '{')) (tok (char '}'))
$ p_field `sepBy` tok (char ',')
where p_field = (,) <$> (p_string <* tok (char ':')) <*> p_value
p_number :: CharParser () Rational
p_number = tok floating where
floating :: CharParser () Rational
floating= do{ n <- decimal
; fract <- option 0 fraction
; expo <- option 1 exponent'
; return ((fromInteger n + fract)*expo)
}
fraction= do{ char '.'
; digits <- many1 digit "fraction"
; return (foldr op 0 digits)
}
"fraction"
where
op d f= (f + fromIntegral (digitToInt d))/10
exponent' = do{ oneOf "eE"
; f <- sign
; e <- decimal "exponent"
; return (power (f e))
}
"exponent"
where
power e | e < 0 = 1/power(-e)
| otherwise = fromInteger (10^e)
sign= (char '-' >> return negate)
<|> (char '+' >> return id)
<|> return
Re: [Haskell-cafe] Slow Text.JSON parser
It occurs to me you could also use attoparsec, which is specifically
optimised for bytestring processing.
sjoerd:
> Hi,
>
> Somebody told me about Parsec 3, which uses a Stream type class so it
> can parse any data type. This sounded like the right way to do
> encoding independent parsing, so I decided to see how it would work to
> parse UTF8 JSON.
>
> Sadly I could not use Text.JSON.Parsec directly, because it uses the
> old Parsec CharParser type. So I copied to code, and also replaced
> p_number with the "floating" parser from Text.Parsec.Token, because
> Text.JSON.Parsec uses readFloat (a dirty hack imho) which works only
> on String.
>
> If Text.JSON.Parsec was written for Parsec 3, the only thing to write
> to get UTF8 JSON parsing would be:
>
> instance (Monad m, U.UTF8Bytes string index) => Stream (U.UTF8 string)
> m Char where
> uncons = return . U.uncons
>
> I did not do any performance measuring yet, I was glad I got it
> working. Any comments on the code is appreciated!
>
> greetings,
> Sjoerd Visscher
>
> {-# LANGUAGE FlexibleInstances, MultiParamTypeClasses,
> UndecidableInstances #-}
> import qualified Data.String.UTF8 as U
> import qualified Data.ByteString as B
>
> import Text.Parsec hiding (many, optional, (<|>))
> import Control.Applicative
>
> import Text.JSON.Types
> import Control.Monad
> import Data.Char
> import Numeric
>
> instance (Monad m, U.UTF8Bytes string index) => Stream (U.UTF8 string)
> m Char where
> uncons = return . U.uncons
>
> type CharParser st = Parsec (U.UTF8 B.ByteString) st
>
> parseFile :: FilePath -> IO (Either ParseError JSValue)
> parseFile fileName = do
> bs <- B.readFile fileName
> return $ runParser json () fileName (U.fromRep bs)
>
> parseString :: String -> Either ParseError JSValue
> parseString s = runParser json () "(unknown)" (U.fromString s)
>
> json :: CharParser () JSValue
> json = spaces *> p_value
>
> tok :: CharParser () a -> CharParser () a
> tok p = p <* spaces
>
> p_value :: CharParser () JSValue
> p_value = (JSNull <$ p_null)
> <|> (JSBool <$> p_boolean)
> <|> (JSArray <$> p_array)
> <|> (JSString<$> p_js_string)
> <|> (JSObject<$> p_js_object)
> <|> (JSRational False <$> p_number)
> "JSON value"
>
> p_null :: CharParser () ()
> p_null= tok (string "null") >> return ()
>
> p_boolean:: CharParser () Bool
> p_boolean = tok
> ( (True <$ string "true")
> <|> (False <$ string "false")
> )
>
> p_array :: CharParser () [JSValue]
> p_array = between (tok (char '[')) (tok (char ']'))
> $ p_value `sepBy` tok (char ',')
>
> p_string :: CharParser () String
> p_string = between (tok (char '"')) (char '"') (many p_char)
> where p_char= (char '\\' >> p_esc)
> <|> (satisfy (\x -> x /= '"' && x /= '\\'))
>
> p_esc = ('"' <$ char '"')
> <|> ('\\' <$ char '\\')
> <|> ('/' <$ char '/')
> <|> ('\b' <$ char 'b')
> <|> ('\f' <$ char 'f')
> <|> ('\n' <$ char 'n')
> <|> ('\r' <$ char 'r')
> <|> ('\t' <$ char 't')
> <|> (char 'u' *> p_uni)
> "escape character"
>
> p_uni = check =<< count 4 (satisfy isHexDigit)
> where check x | code <= max_char = pure (toEnum code)
> | otherwise = empty
> where code = fst $ head $ readHex x
> max_char = fromEnum (maxBound :: Char)
>
> p_object :: CharParser () [(String,JSValue)]
> p_object = between (tok (char '{')) (tok (char '}'))
> $ p_field `sepBy` tok (char ',')
> where p_field = (,) <$> (p_string <* tok (char ':')) <*> p_value
>
> p_number :: CharParser () Rational
> p_number = tok floating where
>
> floating :: CharParser () Rational
> floating= do{ n <- decimal
> ; fract <- option 0 fraction
> ; expo <- option 1 exponent'
> ; return ((fromInteger n + fract)*expo)
> }
>
> fraction= do{ char '.'
> ; digits <- many1 digit "fraction"
> ; return (foldr op 0 digits)
> }
>"fraction"
> where
> op d f= (f + fromIntegral (digitToInt d))/10
>
> exponent' = do{ oneOf "eE"
> ; f <- sign
> ; e <- decimal "exponent"
>
Re: [Haskell-cafe] Slow Text.JSON parser
Attoparsec does not have something like the Stream class, so I do not
see how I could do UTF8 parsing easily.
On Jan 17, 2009, at 11:50 PM, Don Stewart wrote:
It occurs to me you could also use attoparsec, which is specifically
optimised for bytestring processing.
sjoerd:
Hi,
Somebody told me about Parsec 3, which uses a Stream type class so it
can parse any data type. This sounded like the right way to do
encoding independent parsing, so I decided to see how it would work
to
parse UTF8 JSON.
Sadly I could not use Text.JSON.Parsec directly, because it uses the
old Parsec CharParser type. So I copied to code, and also replaced
p_number with the "floating" parser from Text.Parsec.Token, because
Text.JSON.Parsec uses readFloat (a dirty hack imho) which works only
on String.
If Text.JSON.Parsec was written for Parsec 3, the only thing to write
to get UTF8 JSON parsing would be:
instance (Monad m, U.UTF8Bytes string index) => Stream (U.UTF8
string)
m Char where
uncons = return . U.uncons
I did not do any performance measuring yet, I was glad I got it
working. Any comments on the code is appreciated!
greetings,
Sjoerd Visscher
{-# LANGUAGE FlexibleInstances, MultiParamTypeClasses,
UndecidableInstances #-}
import qualified Data.String.UTF8 as U
import qualified Data.ByteString as B
import Text.Parsec hiding (many, optional, (<|>))
import Control.Applicative
import Text.JSON.Types
import Control.Monad
import Data.Char
import Numeric
instance (Monad m, U.UTF8Bytes string index) => Stream (U.UTF8
string)
m Char where
uncons = return . U.uncons
type CharParser st = Parsec (U.UTF8 B.ByteString) st
parseFile :: FilePath -> IO (Either ParseError JSValue)
parseFile fileName = do
bs <- B.readFile fileName
return $ runParser json () fileName (U.fromRep bs)
parseString :: String -> Either ParseError JSValue
parseString s = runParser json () "(unknown)" (U.fromString s)
json :: CharParser () JSValue
json = spaces *> p_value
tok :: CharParser () a -> CharParser () a
tok p = p <* spaces
p_value :: CharParser () JSValue
p_value = (JSNull <$ p_null)
<|> (JSBool <$> p_boolean)
<|> (JSArray <$> p_array)
<|> (JSString<$> p_js_string)
<|> (JSObject<$> p_js_object)
<|> (JSRational False <$> p_number)
"JSON value"
p_null :: CharParser () ()
p_null= tok (string "null") >> return ()
p_boolean:: CharParser () Bool
p_boolean = tok
( (True <$ string "true")
<|> (False <$ string "false")
)
p_array :: CharParser () [JSValue]
p_array = between (tok (char '[')) (tok (char ']'))
$ p_value `sepBy` tok (char ',')
p_string :: CharParser () String
p_string = between (tok (char '"')) (char '"') (many p_char)
where p_char= (char '\\' >> p_esc)
<|> (satisfy (\x -> x /= '"' && x /= '\\'))
p_esc = ('"' <$ char '"')
<|> ('\\' <$ char '\\')
<|> ('/' <$ char '/')
<|> ('\b' <$ char 'b')
<|> ('\f' <$ char 'f')
<|> ('\n' <$ char 'n')
<|> ('\r' <$ char 'r')
<|> ('\t' <$ char 't')
<|> (char 'u' *> p_uni)
"escape character"
p_uni = check =<< count 4 (satisfy isHexDigit)
where check x | code <= max_char = pure (toEnum code)
| otherwise = empty
where code = fst $ head $ readHex x
max_char = fromEnum (maxBound :: Char)
p_object :: CharParser () [(String,JSValue)]
p_object = between (tok (char '{')) (tok (char '}'))
$ p_field `sepBy` tok (char ',')
where p_field = (,) <$> (p_string <* tok (char ':')) <*> p_value
p_number :: CharParser () Rational
p_number = tok floating where
floating :: CharParser () Rational
floating= do{ n <- decimal
; fract <- option 0 fraction
; expo <- option 1 exponent'
; return ((fromInteger n + fract)*expo)
}
fraction= do{ char '.'
; digits <- many1 digit "fraction"
; return (foldr op 0 digits)
}
"fraction"
where
op d f= (f + fromIntegral (digitToInt d))/10
exponent' = do{ oneOf "eE"
; f <- sign
; e <- decimal "exponent"
; return (power (f e))
}
"exponent"
where
power e | e < 0 = 1/pow
Re: [Haskell-cafe] Slow Text.JSON parser
Maybe. Handling the common cases reasonably well is probably worth doing first (+profiling) before opting for a heart&lung transplant.. To wit, I've trivially improved the handling of string and integer lits in version 0.4.3 (just released.) It cuts down the running times by a factor of 2-3 on larger inputs -- http://hackage.haskell.org/cgi-bin/hackage-scripts/package/json Not saying that there aren't additional wins to be had :) hth --sigbjorn On 1/17/2009 14:50, Don Stewart wrote: It occurs to me you could also use attoparsec, which is specifically optimised for bytestring processing. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Slow Text.JSON parser
Ketil Malde wrote: Sjoerd Visscher writes: JSON is a UNICODE format, like any modern format is today. ByteStrings are not going to work. Well, neither is String as used in the code I responded to. I'm not intimately familiar with JSON, but I believe ByteStrings would work on UTF-8 input, and both ByteString and String would fail on UTF-16 and UTF-32. ByteStrings can handle Unicode just fine, provided the right (de)serialization tools: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/utf8-light http://hackage.haskell.org/cgi-bin/hackage-scripts/package/utf8-string -- Live well, ~wren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Slow Text.JSON parser
On Sun, Jan 18, 2009 at 6:07 AM, Sigbjorn Finne wrote: > > Maybe. Handling the common cases reasonably well is > probably worth doing first (+profiling) before opting for > a heart&lung transplant.. > > To wit, I've trivially improved the handling of string and > integer lits in version 0.4.3 (just released.) It cuts down > the running times by a factor of 2-3 on larger inputs -- Indeed, I have just tried version 0.4.3 and my previous test which took about 3 seconds to run is now running in about one second. Very nice improvement. Thanks for all your work Sigbjorn. Cheers, Levi ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
