Thanks! Is it possible to do the same to tok::cxx_exceptspec_end? (I also note we don't currently use an end-of-delayed-tokens marker when delay-parsing function bodies. It probably takes a bit more work to construct a testcase that goes awry, but I suspect that case would benefit from using a synthetic tok::eof too.)
On Sun, Jan 11, 2015 at 9:17 PM, David Majnemer <[email protected]> wrote: > Author: majnemer > Date: Sun Jan 11 23:17:40 2015 > New Revision: 225619 > > URL: http://llvm.org/viewvc/llvm-project?rev=225619&view=rev > Log: > Parse: Get rid of tok::cxx_defaultarg_end, use EOF instead > > I added setEofData/getEofData to solve this sort of problem back in > r224505. Use the Param's decl to tell us if this is *our* EOF token. > > Modified: > cfe/trunk/include/clang/Basic/TokenKinds.def > cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp > cfe/trunk/lib/Parse/ParseDecl.cpp > cfe/trunk/lib/Parse/Parser.cpp > > Modified: cfe/trunk/include/clang/Basic/TokenKinds.def > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/TokenKinds.def?rev=225619&r1=225618&r2=225619&view=diff > > ============================================================================== > --- cfe/trunk/include/clang/Basic/TokenKinds.def (original) > +++ cfe/trunk/include/clang/Basic/TokenKinds.def Sun Jan 11 23:17:40 2015 > @@ -116,7 +116,6 @@ TOK(eof) // End of file. > TOK(eod) // End of preprocessing directive (end of line > inside a > // directive). > TOK(code_completion) // Code completion marker > -TOK(cxx_defaultarg_end) // C++ default argument end marker > TOK(cxx_exceptspec_end) // C++ exception-specification end marker > > // C99 6.4.9: Comments. > > Modified: cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp?rev=225619&r1=225618&r2=225619&view=diff > > ============================================================================== > --- cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp (original) > +++ cfe/trunk/lib/Parse/ParseCXXInlineMethods.cpp Sun Jan 11 23:17:40 2015 > @@ -342,7 +342,9 @@ void Parser::ParseLexedMethodDeclaration > Actions.ActOnParamDefaultArgumentError(LM.DefaultArgs[I].Param, > EqualLoc); > else { > - if (!TryConsumeToken(tok::cxx_defaultarg_end)) { > + if (Tok.is(tok::eof) && Tok.getEofData() == > LM.DefaultArgs[I].Param) { > + ConsumeAnyToken(); > + } else { > // The last two tokens are the terminator and the saved value of > // Tok; the last token in the default argument is the one before > // those. > @@ -360,8 +362,11 @@ void Parser::ParseLexedMethodDeclaration > "ParseAssignmentExpression went over the default arg > tokens!"); > // There could be leftover tokens (e.g. because of an error). > // Skip through until we reach the original token position. > - while (Tok.getLocation() != origLoc && Tok.isNot(tok::eof)) > + while (Tok.getLocation() != origLoc) { > + if (Tok.is(tok::eof) && Tok.getEofData() != > LM.DefaultArgs[I].Param) > + break; > ConsumeAnyToken(); > + } > > delete Toks; > LM.DefaultArgs[I].Toks = nullptr; > @@ -652,7 +657,6 @@ bool Parser::ConsumeAndStoreUntil(tok::T > > switch (Tok.getKind()) { > case tok::eof: > - case tok::cxx_defaultarg_end: > case tok::annot_module_begin: > case tok::annot_module_end: > case tok::annot_module_include: > > Modified: cfe/trunk/lib/Parse/ParseDecl.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/ParseDecl.cpp?rev=225619&r1=225618&r2=225619&view=diff > > ============================================================================== > --- cfe/trunk/lib/Parse/ParseDecl.cpp (original) > +++ cfe/trunk/lib/Parse/ParseDecl.cpp Sun Jan 11 23:17:40 2015 > @@ -5631,8 +5631,9 @@ void Parser::ParseParameterDeclarationCl > // stop when we parse it later on. > Token DefArgEnd; > DefArgEnd.startToken(); > - DefArgEnd.setKind(tok::cxx_defaultarg_end); > + DefArgEnd.setKind(tok::eof); > DefArgEnd.setLocation(Tok.getLocation()); > + DefArgEnd.setEofData(Param); > DefArgToks->push_back(DefArgEnd); > Actions.ActOnParamUnparsedDefaultArgument(Param, EqualLoc, > > (*DefArgToks)[1].getLocation()); > > Modified: cfe/trunk/lib/Parse/Parser.cpp > URL: > http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Parse/Parser.cpp?rev=225619&r1=225618&r2=225619&view=diff > > ============================================================================== > --- cfe/trunk/lib/Parse/Parser.cpp (original) > +++ cfe/trunk/lib/Parse/Parser.cpp Sun Jan 11 23:17:40 2015 > @@ -262,10 +262,6 @@ bool Parser::SkipUntil(ArrayRef<tok::Tok > // Ran out of tokens. > return false; > > - case tok::cxx_defaultarg_end: > - // It's never desirable to consume the 'end-of-default-argument' > token. > - return false; > - > case tok::annot_pragma_openmp_end: > // Stop before an OpenMP pragma boundary. > case tok::annot_module_begin: > > > _______________________________________________ > cfe-commits mailing list > [email protected] > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits >
_______________________________________________ cfe-commits mailing list [email protected] http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
