Hi, I was just thinking. if it would be a line based parser then the regexpr could be applied only to the current read line and it wouldn't be a problem. If there was a readline mode then it could work with ply.
Cheers Guillaume On May 14, 11:31 am, "A.T.Hofkamp" <[email protected]> wrote: > guillaume wrote: > > Hi, > > > I've been using PLY to parse my little DSL and so force. > > I now need to parse an hybrid legacy language that can be a request > > message but also host a data message (and then it is can be big for a > > message around 20 MB). > > As I am familiar with the PLY lexer, I would like to use it to > > tokenize this message. > > However ply.lexer does only take a string as input and I would like to > > avoid load 20 MB in memory. > > Is there a way to have the PLY lexer reading from a stream or is there > > a way to make PLY believe that it is a string and behind the scene in > > reality it is an IO stream. > > No. > > PLY (like most parsing packages) uses RE for its tokenizing, and that package > needs the whole string. > > To fix, please adapt RE. It would make a lot of people happy :) > Alternatively, you can write a custom lexer that reads from file, and connect > that to the ply.yacc parser. > > Albert --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "ply-hack" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/ply-hack?hl=en -~----------~----~----~----~------~----~------~--~---
