[Haskell-cafe] Parsec line number off-by-one
Recently I've been playing around with Parsec for a simple parsing project. While I was able to quickly construct my grammar (simplified version attached), getting it working has been a bit tricky. In particular, I am now stuck trying to figure out why Parsec is mis-reporting line numbers. Parsec seems convinced that line 12 of my input (also attached) has a "%" character, $ runghc Test.hs Left "(unknown)" (line 12, column 1): unexpected "%" expecting space or atom name while my file clearly disagrees, 10 %FLAG ATOM_NAME 11 %FORMAT(20a4) 12 C1 H1 C2 H2 C3 H3 C4 H4 C5 C6 C7 C8 N1 C9 H9 C10 H10 C11 H11 C12 13 H12 C13 H13 C14 C15 N2 C16 C17 C29 H18 C19 H19 C20 H20 C21 H21 C22 H221H222H223 ... 18 %FLAG CHARGE 19 %FORMAT(5E16.8) The task here is to identify the block of data lines (lines 12-17), ending at the beginning of the next block (starting with "%"). It seems likely that my problem stems from the fact that I use "try" to accomplish this but this is as far as I can reason. Any ideas what might cause this sort of off-by-one? Does anyone see a better (i.e. working) way to formulate my grammar? Any and all help would be greatly appreciated. Thanks. Cheers, - Ben module Main(main) where import Debug.Trace import Data.Maybe import Text.ParserCombinators.Parsec import Text.Parsec.Prim (ParsecT) import qualified Text.ParserCombinators.Parsec.Token as P import Text.ParserCombinators.Parsec.Language (emptyDef) data PrmTopBlock = AtomNames [String] deriving (Show, Eq) countBetween m n p = do xs <- count m p ys <- count (n - m) $ option Nothing $ do y <- p return (Just y) return (xs ++ catMaybes ys) restLine = do a <- many (noneOf "\n") eol return a eol = do skipMany $ char ' ' char '\n' natural = P.integer $ P.makeTokenParser emptyDef float = do sign <- option 1 (do s <- oneOf "+- " return $ if s == '-' then -1 else 1) x <- P.float $ P.makeTokenParser emptyDef return $ sign * x flagDecl x = do string $ "%FLAG " ++ x eol formatDecl = do string "%FORMAT(" count <- many1 digit format <- letter length <- many1 digit char ')' eol return (count, format, length) -- |Multiple lines of lists of a given item linesOf item = do ls <- many1 $ try (do lookAhead (noneOf "%") l <- many1 item eol return $ trace (show l) l) return $ concat ls atomNameBlock = do flagDecl "ATOM_NAME" formatDecl atomNames <- linesOf atomName return $ AtomNames atomNames where atomName = do spaces name <- countBetween 1 4 (alphaNum <|> oneOf "\'+-") "atom name" return name ignoredBlock = do string "%FLAG" "ignored block flag" restLine formatDecl skipMany (noneOf "%" >> restLine) hello = do ignoredBlock ignoredBlock atomNameBlock parsePrmTopFile input = parse hello "(unknown)" input test = do a <- readFile "test.prmtop" print $ parsePrmTopFile a main = test test.prmtop Description: Binary data ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Parsec line number off-by-one
Hi Ben, This is indeed a bug in parsec. I have written a patch that fixes this. Currently Antoine Latter (current parsec's maintainer) and I are working on getting these patches into the next parsec release. As a workaround until then, you can apply the attached patch manually. darcs get http://code.haskell.org/parsec3 cd parsec3 darcs apply parsec.dpatch cabal install With this patch, the error message is: Left "(unknown)" (line 18, column 1): expecting space or atom name * Ben Gamari [2011-09-20 23:32:34-0400] > Recently I've been playing around with Parsec for a simple parsing > project. While I was able to quickly construct my grammar (simplified > version attached), getting it working has been a bit tricky. In > particular, I am now stuck trying to figure out why Parsec is > mis-reporting line numbers. Parsec seems convinced that line 12 of my > input (also attached) has a "%" character, > > $ runghc Test.hs > Left "(unknown)" (line 12, column 1): > unexpected "%" > expecting space or atom name > > while my file clearly disagrees, > > 10 %FLAG ATOM_NAME > > 11 %FORMAT(20a4) > > 12 C1 H1 C2 H2 C3 H3 C4 H4 C5 C6 C7 C8 N1 C9 H9 C10 H10 C11 > H11 C12 > 13 H12 C13 H13 C14 C15 N2 C16 C17 C29 H18 C19 H19 C20 H20 C21 H21 C22 > H221H222H223 > ... > 18 %FLAG CHARGE > 19 %FORMAT(5E16.8) > > > The task here is to identify the block of data lines (lines 12-17), > ending at the beginning of the next block (starting with "%"). It seems > likely that my problem stems from the fact that I use "try" to > accomplish this but this is as far as I can reason. > > Any ideas what might cause this sort of off-by-one? Does anyone see a > better (i.e. working) way to formulate my grammar? Any and all help > would be greatly appreciated. Thanks. > > Cheers, > > - Ben > > > ___ > Haskell-Cafe mailing list > Haskell-Cafe@haskell.org > http://www.haskell.org/mailman/listinfo/haskell-cafe -- Roman I. Cheplyaka :: http://ro-che.info/ 4 patches for repository http://code.haskell.org/parsec3: Sun Feb 20 18:24:22 EET 2011 Roman Cheplyaka * Choose the longest match when merging error messages Sun Feb 20 18:24:49 EET 2011 Roman Cheplyaka * try: do not reset the error position Sun Feb 20 18:29:20 EET 2011 Roman Cheplyaka * lookAhead: do not consume input on success; update documentation Sun Feb 20 19:30:26 EET 2011 Roman Cheplyaka * Improve New patches: [Choose the longest match when merging error messages Roman Cheplyaka **20110220162422 Ignore-this: 54e2733159a1574abb229e09ff6935c1 ] hunk ./Text/Parsec/Error.hs 137 = ParseError pos (msg : filter (msg /=) msgs) mergeError :: ParseError -> ParseError -> ParseError -mergeError (ParseError pos msgs1) (ParseError _ msgs2) -= ParseError pos (msgs1 ++ msgs2) +mergeError (ParseError pos1 msgs1) (ParseError pos2 msgs2) += case pos1 `compare` pos2 of +-- select the longest match +EQ -> ParseError pos1 (msgs1 ++ msgs2) +GT -> ParseError pos1 msgs1 +LT -> ParseError pos2 msgs2 instance Show ParseError where show err [try: do not reset the error position Roman Cheplyaka **20110220162449 Ignore-this: 8508bc41fc6dcd9b7c06aac762f12c71 ] hunk ./Text/Parsec/Prim.hs 435 try :: ParsecT s u m a -> ParsecT s u m a try p = -ParsecT $ \s@(State _ pos _) cok _ eok eerr -> -let pcerr parseError = eerr $ setErrorPos pos parseError -in unParser p s cok pcerr eok eerr +ParsecT $ \s cok _ eok eerr -> +unParser p s cok eerr eok eerr -- | The parser @tokenPrim showTok posFromTok testTok@ accepts a token @t@ -- with result @x@ when the function @testTok t@ returns @'Just' x@. The [lookAhead: do not consume input on success; update documentation Roman Cheplyaka **20110220162920 Ignore-this: e884771490209b93e9fec044543a18ef ] { hunk ./Text/Parsec/Combinator.hs 279 <|> do{ x <- p; xs <- scan; return (x:xs) } --- | @lookAhead p@ parses @p@ without consuming any input. - -lookAhead :: (Stream s m t) => ParsecT s u m a -> ParsecT s u m a -lookAhead p = do{ state <- getParserState -; x <- p -; setParserState state -; return x -} - hunk ./Text/Parsec/Prim.hs 40 , (<|>) , label , labels +, lookAhead , Stream(..) , tokens , try hunk ./Text/Parsec/Prim.hs 439 ParsecT $ \s cok _ eok eerr -> unParser p s cok eerr eok eerr +-- | @lookAhead p@ parses @p@ without consuming any input. +-- +-- If @p@ fails and consumes some input, so does @lookAhead@. Combine with 'try' +-- if this is undesirable. +
Re: [Haskell-cafe] Parsec line number off-by-one
Hi, 1. your "lookAhead" is unnecessary, because your items (atomNames) never start with "%". 2. your "try" fails in (line 12, column 1), because the last item (aka atomName) starts consuming "\n", before your eol parser is called. So rather than calling spaces before every real atom, I would call it after every real atom and after your formatDecl (so before your linesOf parser). atomNameBlock = do flagDecl "ATOM_NAME" formatDecl spaces atomNames <- many1 atomName return $ AtomNames atomNames where atomName = do name <- countBetween 1 4 (alphaNum <|> oneOf "\'+-") "atom name" spaces return name Since spaces also consume "\n", linesOf can just be "many1"! HTH Christian Am 21.09.2011 05:32, schrieb Ben Gamari: Recently I've been playing around with Parsec for a simple parsing project. While I was able to quickly construct my grammar (simplified version attached), getting it working has been a bit tricky. In particular, I am now stuck trying to figure out why Parsec is mis-reporting line numbers. Parsec seems convinced that line 12 of my input (also attached) has a "%" character, $ runghc Test.hs Left "(unknown)" (line 12, column 1): unexpected "%" expecting space or atom name while my file clearly disagrees, 10 %FLAG ATOM_NAME 11 %FORMAT(20a4) 12 C1 H1 C2 H2 C3 H3 C4 H4 C5 C6 C7 C8 N1 C9 H9 C10 H10 C11 H11 C12 13 H12 C13 H13 C14 C15 N2 C16 C17 C29 H18 C19 H19 C20 H20 C21 H21 C22 H221H222H223 ... 18 %FLAG CHARGE 19 %FORMAT(5E16.8) The task here is to identify the block of data lines (lines 12-17), ending at the beginning of the next block (starting with "%"). It seems likely that my problem stems from the fact that I use "try" to accomplish this but this is as far as I can reason. Any ideas what might cause this sort of off-by-one? Does anyone see a better (i.e. working) way to formulate my grammar? Any and all help would be greatly appreciated. Thanks. Cheers, - Ben ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Parsec line number off-by-one
On Wed, 21 Sep 2011 11:27:31 +0200, Christian Maeder wrote: > Hi, > > 1. your "lookAhead" is unnecessary, because your items (atomNames) never > start with "%". > I see. > 2. your "try" fails in (line 12, column 1), because the last item (aka > atomName) starts consuming "\n", before your eol parser is called. > Ahh, this is a good point. I for some reason seeded the thought in my mind that spaces takes the ' ' character, not '\n'. > So rather than calling spaces before every real atom, I would call it > after every real atom and after your formatDecl (so before your linesOf > parser). > Excellent solution. I appreciate your help. That would have taken me quite a bit of head-banging to find. Cheers, - Ben ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Parsec line number off-by-one
On Wed, 21 Sep 2011 09:37:40 +0300, Roman Cheplyaka wrote: > Hi Ben, > > This is indeed a bug in parsec. > Ahh good. I'm glad I'm not crazy. Given that it seems the lookahead is actually unnecessary, looks like I can skip the patch too. Thanks for your reply! Cheers, - Ben ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe