Re: How to make Bison disregard tokens?

2007-03-02 Thread Edsko de Vries
Hi Tim, > But if you already have comments handled like this, why don't you simply > treat a blank line like a full-line comment (which it is, to some > extent)? I've just implemented this. One minute's worth of work, and it works like a charm. You are a genius :) Bedankt :) Edsko ___

Re: How to make Bison disregard tokens?

2007-03-02 Thread Edsko de Vries
Hey, > I would expect a } to be a token, so the comment would be associated > with that }, never with the f() call. > As long as you don't discard significant punctuation like this, I doubt > there is any need for NOPs. Alas, I cannot do that, because we don't represent the } in the abstract synt

Re: How to make Bison disregard tokens?

2007-03-02 Thread Tim Van Holder
Edsko de Vries wrote: > if(...) > { > blah > // TODO: must do something else here > } > > f(); > > the comment is not associated with the call to f, but with a NOP > instruction which is the last instruction in the body of the if. That I would expect a } to be a token, so the comment

Re: How to make Bison disregard tokens?

2007-03-02 Thread henrik . sorensen
On Friday 02 March 2007 15:23, Edsko de Vries wrote: > > do you have a link or can you send me the scanner file alone ? > Sure. phc is hosted by Google code; the scanner is at > http://phc.googlecode.com/svn/phc/trunk/src/generated_src/php_scanner.lex but when you compare with http://pl1gcc.cvs.s

Re: How to make Bison disregard tokens?

2007-03-02 Thread Edsko de Vries
On Fri, Mar 02, 2007 at 03:18:59PM +0100, [EMAIL PROTECTED] wrote: > On Friday 02 March 2007 15:15, Edsko de Vries wrote: > > On Fri, Mar 02, 2007 at 03:02:43PM +0100, [EMAIL PROTECTED] wrote: > > > On Friday 02 March 2007 14:54, Edsko de Vries wrote: > > > > If you have a > > > > good solution for

Re: How to make Bison disregard tokens?

2007-03-02 Thread henrik . sorensen
On Friday 02 March 2007 15:15, Edsko de Vries wrote: > On Fri, Mar 02, 2007 at 03:02:43PM +0100, [EMAIL PROTECTED] wrote: > > On Friday 02 March 2007 14:54, Edsko de Vries wrote: > > > If you have a > > > good solution for that, I'd be very interested to hear it! > > > > look at pl1gcc.sourceforge.

Re: How to make Bison disregard tokens?

2007-03-02 Thread Edsko de Vries
On Fri, Mar 02, 2007 at 03:02:43PM +0100, [EMAIL PROTECTED] wrote: > On Friday 02 March 2007 14:54, Edsko de Vries wrote: > > If you have a > > good solution for that, I'd be very interested to hear it! > look at pl1gcc.sourceforge.net > > in the scanner you 'just' have to do all the bean counting

Re: How to make Bison disregard tokens?

2007-03-02 Thread henrik . sorensen
On Friday 02 March 2007 15:04, Edsko de Vries wrote: > If we insert a NOP instead for blank lines, then the comment gets > associated with the NOP, and we get exactly what the user wrote. > > Incidentally, the same problem also arises with Henrik's location > tracking solution. no as pointed out e

Re: How to make Bison disregard tokens?

2007-03-02 Thread Edsko de Vries
> To me this seems like the same problem as preserving comments in a > parse-process-generate system (which is a pain in the ass). That's exactly what it is :) > You could have the lexer tally empty lines and set that count as a > property of the semantic value of "real" tokens. > > {snip} Yes

Re: How to make Bison disregard tokens?

2007-03-02 Thread henrik . sorensen
On Friday 02 March 2007 14:54, Edsko de Vries wrote: > If you have a > good solution for that, I'd be very interested to hear it! look at pl1gcc.sourceforge.net in the scanner you 'just' have to do all the bean counting yourself, and you have to come up with a prober structure to keep the locatio

Re: How to make Bison disregard tokens?

2007-03-02 Thread henrik . sorensen
On Friday 02 March 2007 14:50, Tim Van Holder wrote: > To me this seems like the same problem as preserving comments in a > parse-process-generate system (which is a pain in the ass). not really a problem. If using the location tracking feature, it would be trivial to add some fields to keep trac

Re: How to make Bison disregard tokens?

2007-03-02 Thread Edsko de Vries
On Fri, Mar 02, 2007 at 02:43:19PM +0100, [EMAIL PROTECTED] wrote: > On Friday 02 March 2007 13:39, Edsko de Vries wrote: > > On Fri, Mar 02, 2007 at 01:17:04PM +0100, [EMAIL PROTECTED] wrote: > > > On Friday 02 March 2007 13:01, Edsko de Vries wrote: > > > > Well, no, that's the point. Blank lines

Re: How to make Bison disregard tokens?

2007-03-02 Thread Tim Van Holder
Edsko de Vries wrote: > On Fri, Mar 02, 2007 at 12:55:29PM +0100, [EMAIL PROTECTED] wrote: >> On Friday 02 March 2007 12:30, Edsko de Vries wrote: >>> Hi, >>> >>> Is there any way I can make Bison ignore tokens when it can't deal with >>> them? I'll try to explain. For every completely blank line i

Re: How to make Bison disregard tokens?

2007-03-02 Thread henrik . sorensen
On Friday 02 March 2007 13:39, Edsko de Vries wrote: > On Fri, Mar 02, 2007 at 01:17:04PM +0100, [EMAIL PROTECTED] wrote: > > On Friday 02 March 2007 13:01, Edsko de Vries wrote: > > > Well, no, that's the point. Blank lines should be recorded as NOPs > > > where possible, so that we know where the

Re: How to make Bison disregard tokens?

2007-03-02 Thread Edsko de Vries
On Fri, Mar 02, 2007 at 01:17:04PM +0100, [EMAIL PROTECTED] wrote: > On Friday 02 March 2007 13:01, Edsko de Vries wrote: > > > Well, no, that's the point. Blank lines should be recorded as NOPs where > > possible, so that we know where they are and unparse them. > Can you reveal some details reg

Re: How to make Bison disregard tokens?

2007-03-02 Thread henrik . sorensen
On Friday 02 March 2007 13:01, Edsko de Vries wrote: > Well, no, that's the point. Blank lines should be recorded as NOPs where > possible, so that we know where they are and unparse them. Can you reveal some details regarding what you are trying to do ? Henrik

Re: How to make Bison disregard tokens?

2007-03-02 Thread Edsko de Vries
On Fri, Mar 02, 2007 at 12:55:29PM +0100, [EMAIL PROTECTED] wrote: > On Friday 02 March 2007 12:30, Edsko de Vries wrote: > > Hi, > > > > Is there any way I can make Bison ignore tokens when it can't deal with > > them? I'll try to explain. For every completely blank line in the input, > > my leexe

Re: How to make Bison disregard tokens?

2007-03-02 Thread henrik . sorensen
On Friday 02 March 2007 12:30, Edsko de Vries wrote: > Hi, > > Is there any way I can make Bison ignore tokens when it can't deal with > them? I'll try to explain. For every completely blank line in the input, > my leexer generates a NOP token. In some situations the parser can deal > with this NOP

How to make Bison disregard tokens?

2007-03-02 Thread Edsko de Vries
Hi, Is there any way I can make Bison ignore tokens when it can't deal with them? I'll try to explain. For every completely blank line in the input, my leexer generates a NOP token. In some situations the parser can deal with this NOP token; in others it can't. Specifically, a NOP is a statement: