Commit 8c87a623 says:
> c++: get rid of symbol_type::token ()
> It is not used. And its implementation was wrong when api.token.raw
> was defined, as it was still mapping to the external token numbers,
> instead of the internal ones. Besides it was provided only when
> api.token.constructor is defined, yet always declared.
>
> * data/skeletons/c++.m4 (by_type::token): Remove, useless.
Well, I disagree. It's not useless, I'm using it, and I think my use
case is legitimate.
Low priority, I have an easy workaround.
My parser uses the C++ parser template with
%define api.token.constructor
%define api.value.type variant
I've got a hand-written lexer which returns symbol_type.
And because I'm a good boy, I have unit tests for my lexer. And in my
unit tests, I call my lexer on small bits of input, and then I need to
inspect the resulting symbol_type and make sure the right kind of token
is returned:
So I have things like:
Lexer lex("12345");
symbol_type t = lex.nextToken();
assert(t.token() == token::INTLIT);
assert(t.value.as<int>() == 12345);
... which suddenly stopped working.
Fortunately, I can instead do...
assert(t.type_get() == by_type(token::INTLIT).type_get());
... but that's a bit ugly.
In general, I think symbol_type should always provide enough public API
in order to allow inspecting it. In my case, I only need it for
implementing tests, but it general it might be useful when writing the
lexer, as well.
So I request to reinstate symbol_type::token().
Cheers,
Wolfgang
P.S.: Keep up the good work, bison is my favorite way of parsing things
using C++....
