sammccall added inline comments.
================
Comment at: clang/lib/Tooling/Syntax/Tokens.cpp:335
clang::Token T;
- while (!L.LexFromRawLexer(T))
+ while (!L.LexFromRawLexer(T) && L.getCurrentBufferOffset() < FR.endOffset())
AddToken(T);
----------------
Discussed offline, this loop includes an extra token if the truncation is at
whitespace between tokens. (Please test this case)
Also the eof comment is confusing.
I think the loop should be rewritten.
================
Comment at: clang/lib/Tooling/Syntax/Tokens.cpp:346
+ const LangOptions &LO) {
+ return tokenize(syntax::FileRange(SM, SM.getLocForStartOfFile(FID),
+ SM.getLocForEndOfFile(FID)),
----------------
nit: FileRange(FID, 0, SM.getFileIDSize(FID)) is a lot more direct :-)
Repository:
rG LLVM Github Monorepo
CHANGES SINCE LAST ACTION
https://reviews.llvm.org/D74962/new/
https://reviews.llvm.org/D74962
_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
