Re: Dynamic token kinds

2018-12-23 Thread Frank Heckenbach
Akim Demaille wrote: > [gnulib] > > I never liked that > > design, and apparently it's now causing me actual problems by > > breaking the simple download and vastly increasing the clone size > > (AFAICS, most of it is downloading the gnulib history which I don't > > care a tiny bit about, especial

Re: Dynamic token kinds

2018-12-23 Thread Akim Demaille
> Le 23 déc. 2018 à 12:46, Frank Heckenbach a écrit : > > Akim Demaille wrote: > >>> but >>> unfortunately git is quite data hungry and I'm a bit short on data >>> volume (no broadband in my new appartment yet, and mobile data is >>> paid in gold here in Germany), >> >> Ouch. We're lucky in

Re: Dynamic token kinds

2018-12-23 Thread Frank Heckenbach
Akim Demaille wrote: > > but > > unfortunately git is quite data hungry and I'm a bit short on data > > volume (no broadband in my new appartment yet, and mobile data is > > paid in gold here in Germany), > > Ouch. We're lucky in France, that's very cheap. I'm lucky to have a relatively fast

Re: Dynamic token kinds

2018-12-23 Thread Akim Demaille
Hi Frank! > Le 22 déc. 2018 à 21:12, Frank Heckenbach a écrit : > > Sorry, now I seem to be the one to have trouble using your patch: > > - I guess the right thing is to clone the repository, Yes, it is. > but > unfortunately git is quite data hungry and I'm a bit short on data > volume (no

Re: Dynamic token kinds

2018-12-22 Thread Frank Heckenbach
Akim Demaille wrote: > > To actually allow this, you could have the typed constructors all > > accept the typeless tokens as well, but I don't consider that really > > necessary. Unless you want to support that for backward (bugward?) > > compatibility, I'll just change my code to make two separat

Re: Dynamic token kinds

2018-12-22 Thread Akim Demaille
Hi Frank! > Le 22 déc. 2018 à 01:14, Frank Heckenbach a écrit : > > Akim Demaille wrote: > >> I like this idea. I have a draft for it in my repo, as "make-symbol". >> Please, try it and report about it. > > Again, sorry for the delay (still busy), but now I tried it > (removing the "b4_parse_

Re: Dynamic token kinds

2018-12-21 Thread Frank Heckenbach
Akim Demaille wrote: > Hi Frank, Hi all, > > > Le 16 déc. 2018 à 10:02, Frank Heckenbach a écrit > > : > > > > So to make it safe, we might need something like this: > > > > static inline > > symbol_type > > make_symbol (token_type type, b4_locations_if([const location_type& l, > > ])T&&

Re: Dynamic token kinds

2018-12-20 Thread Akim Demaille
Any comment from someone? > Le 19 déc. 2018 à 09:13, Akim Demaille a écrit : > > > >> Le 19 déc. 2018 à 07:34, Akim Demaille a écrit : >> >> Hi Frank, Hi all, >> >>> Le 16 déc. 2018 à 10:02, Frank Heckenbach a écrit >>> : >>> >>> So to make it safe, we might need something like this: >>>

Re: Dynamic token kinds

2018-12-19 Thread Akim Demaille
> Le 19 déc. 2018 à 07:34, Akim Demaille a écrit : > > Hi Frank, Hi all, > >> Le 16 déc. 2018 à 10:02, Frank Heckenbach a écrit : >> >> So to make it safe, we might need something like this: >> >> static inline >> symbol_type >> make_symbol (token_type type, b4_locations_if([const location

Re: Dynamic token kinds

2018-12-18 Thread Akim Demaille
Hi Frank, Hi all, > Le 16 déc. 2018 à 10:02, Frank Heckenbach a écrit : > > So to make it safe, we might need something like this: > > static inline > symbol_type > make_symbol (token_type type, b4_locations_if([const location_type& l, ])T&& > v); > > auto-generated for each semantic type

Re: Dynamic token kinds

2018-12-17 Thread Hans Åberg
> On 17 Dec 2018, at 18:37, Frank Heckenbach wrote: > > Do you actually use Bison's variant? Otherwise, what you do is > irrelevant here (sorry), as this is a proposal specifically about > Bison's variant. As I said, I do not use it now, but I wanted to know whether I could use it before actu

Re: Dynamic token kinds

2018-12-17 Thread Frank Heckenbach
Hans Åberg wrote: > > What I suggest to add (without changing the above), is: > > > > symbol_type make_symbol (token_type type); > > // checks at runtime that type is V_FOO or V_BAR > > > > symbol_type make_symbol (token_type type, int &&); > > // checks at runtime that type is I_BAZ or I_QU

Re: Dynamic token kinds

2018-12-17 Thread Hans Åberg
> On 17 Dec 2018, at 11:17, Frank Heckenbach wrote: > > Hans Åberg wrote: > >>> On 17 Dec 2018, at 10:48, Frank Heckenbach wrote: >> >> Might Bison generate a function with a switch statement, generate the right >> return for the lexer to use? > > Different semantic types need separate fun

Re: Dynamic token kinds

2018-12-17 Thread Frank Heckenbach
Hans Åberg wrote: > > On 17 Dec 2018, at 10:48, Frank Heckenbach wrote: > > > > I think we agree here, and that was actually my concern when I > > started this thread. I don't want to have to write a separate case > > for each token kind in my lexer. Of course, we need a separate case > > for ea

Re: Dynamic token kinds

2018-12-17 Thread Hans Åberg
> On 17 Dec 2018, at 10:48, Frank Heckenbach wrote: > > Hans Åberg wrote: > >>> On 16 Dec 2018, at 15:48, Frank Heckenbach wrote: >>> >>> Hans Åberg wrote: >>> The idea would be that rather than returning the token value, something that does not need translation. I don't know if t

Re: Dynamic token kinds

2018-12-17 Thread Frank Heckenbach
Hans Åberg wrote: > > On 16 Dec 2018, at 15:48, Frank Heckenbach wrote: > > > > Hans Åberg wrote: > > > >> The idea would be that rather than returning the token value, > >> something that does not need translation. I don't know if that > >> helps up the static approach, though. > > > > Not su

Re: Dynamic token kinds

2018-12-16 Thread Hans Åberg
> On 16 Dec 2018, at 15:48, Frank Heckenbach wrote: > > Hans Åberg wrote: > >> The idea would be that rather than returning the token value, >> something that does not need translation. I don't know if that >> helps up the static approach, though. > > Not sure what you mean here. Please elabo

Re: Dynamic token kinds

2018-12-16 Thread Frank Heckenbach
Hans Åberg wrote: > > On 16 Dec 2018, at 11:13, Frank Heckenbach wrote: > > > > Hans Åberg wrote: > > > >> Perhaps an entirely static approach can be achieved by the type > >> being a part of the token_type. On the other hand, the use is for > >> dynamic token lookup, so it may be too much of a

Re: Dynamic token kinds

2018-12-16 Thread Hans Åberg
> On 16 Dec 2018, at 11:13, Frank Heckenbach wrote: > > Hans Åberg wrote: > >> Perhaps an entirely static approach can be achieved by the type >> being a part of the token_type. On the other hand, the use is for >> dynamic token lookup, so it may be too much of an effort for that. > > Not sur

Re: Dynamic token kinds

2018-12-16 Thread Frank Heckenbach
Hans Åberg wrote: > Perhaps an entirely static approach can be achieved by the type > being a part of the token_type. On the other hand, the use is for > dynamic token lookup, so it may be too much of an effort for that. Not sure what you mean with "part of", but with Bison's variant the semantic

Re: Dynamic token kinds

2018-12-16 Thread Hans Åberg
> On 16 Dec 2018, at 10:02, Frank Heckenbach wrote: > > Hans Åberg wrote: > >>> On 30 Nov 2018, at 00:40, Frank Heckenbach wrote: >>> >>> Hans Åberg wrote: >>> It seems pretty standard to have lookup tokens with different syntactic behavior, for example when they are declared of

Re: Dynamic token kinds

2018-12-16 Thread Frank Heckenbach
Hans Åberg wrote: > > On 30 Nov 2018, at 00:40, Frank Heckenbach wrote: > > > > Hans Åberg wrote: > > > >>> Sure, though for my taste "unsafe" sounds a bit harsh, perhaps > >>> "unchecked"? If you put in the next release, I'll change my code to > >>> use it. > >> > >> It seems pretty standard

Re: Dynamic token kinds

2018-12-15 Thread Hans Åberg
> On 30 Nov 2018, at 00:40, Frank Heckenbach wrote: > > Hans Åberg wrote: > >>> Sure, though for my taste "unsafe" sounds a bit harsh, perhaps >>> "unchecked"? If you put in the next release, I'll change my code to >>> use it. >> >> It seems pretty standard to have lookup tokens with differen

Re: Dynamic token kinds

2018-11-29 Thread Frank Heckenbach
Hans Åberg wrote: > > Sure, though for my taste "unsafe" sounds a bit harsh, perhaps > > "unchecked"? If you put in the next release, I'll change my code to > > use it. > > It seems pretty standard to have lookup tokens with different > syntactic behavior, for example when they are declared of >

Re: Dynamic token kinds

2018-11-29 Thread Hans Åberg
> On 29 Nov 2018, at 01:12, Frank Heckenbach wrote: > > Akim Demaille wrote: > >> Wrt to the symbol constructor, you are right to be worried: I don't >> consider it (so far?) to be part of the public API. I do understand >> something like it is needed, but I don't like that it looks safe >> t

Re: Dynamic token kinds

2018-11-28 Thread Frank Heckenbach
Akim Demaille wrote: > Wrt to the symbol constructor, you are right to be worried: I don't > consider it (so far?) to be part of the public API. I do understand > something like it is needed, but I don't like that it looks safe > to use. > > Would you be ok with parser::unsafe_make_symbol, or so

Re: Dynamic token kinds

2018-11-27 Thread Akim Demaille
Hi Frank! > Le 25 nov. 2018 à 18:17, Frank Heckenbach a écrit : > > [...] Then my yylex function does this (slightly simplified from my actual > code; GetToken, TokenText and Loc are functions of my RE-based > lexer): > > switch (GetToken ()) > { >// ... >case lt_Identifier: > if