On 6/30/23 18:59, Lewis Hyatt wrote:
In order to support processing #pragma in preprocess-only mode (-E or -save-temps for gcc/g++), we need a way to obtain the #pragma tokens from libcpp. In full compilation modes, this is accomplished by calling pragma_lex (), which is a symbol that must be exported by the frontend, and which is currently implemented for C and C++. Neither of those frontends initializes its parser machinery in preprocess-only mode, and consequently pragma_lex () does not work in this case.Address that by adding a new function c_init_preprocess () for the frontends to implement, which arranges for pragma_lex () to work in preprocess-only mode, and adjusting pragma_lex () accordingly. In preprocess-only mode, the preprocessor is accustomed to controlling the interaction with libcpp, and it only knows about tokens that it has called into libcpp itself to obtain. Since it still needs to see the tokens obtained by pragma_lex () so that they can be streamed to the output, also add a new libcpp callback, on_token_lex (), that ensures the preprocessor sees these tokens too. Currently, there is one place where we are already supporting #pragma in preprocess-only mode, namely the handling of `#pragma GCC diagnostic'. That was done by directly interfacing with libcpp, rather than making use of pragma_lex (). Now that pragma_lex () works, that code is no longer necessary; remove it. gcc/c-family/ChangeLog: * c-common.h (c_init_preprocess): Declare new function. * c-opts.cc (c_common_init): Call it. * c-pragma.cc (pragma_diagnostic_lex_normal): Rename to... (pragma_diagnostic_lex): ...this. (pragma_diagnostic_lex_pp): Remove. (handle_pragma_diagnostic_impl): Call pragma_diagnostic_lex () in all modes. (c_pp_invoke_early_pragma_handler): Adapt to support pragma_lex () usage. * c-pragma.h (pragma_lex_discard_to_eol): Declare new function. gcc/c/ChangeLog: * c-parser.cc (pragma_lex): Support preprocess-only mode. (pragma_lex_discard_to_eol): New function. (c_init_preprocess): New function. gcc/cp/ChangeLog: * parser.cc (c_init_preprocess): New function. (maybe_read_tokens_for_pragma_lex): New function. (pragma_lex): Support preprocess-only mode. (pragma_lex_discard_to_eol): New funtion. libcpp/ChangeLog: * include/cpplib.h (struct cpp_callbacks): Add new callback on_token_lex. * macro.cc (cpp_get_token_1): Support new callback. --- Notes: Hello-In r13-1544, I added support for processing `#pragma GCC diagnostic' inpreprocess-only mode. Because pragma_lex () doesn't work in that mode, in that patch I called into libcpp directly to obtain the tokens needed to process the pragma. As part of the review, Jason noted that it would probably be better to make pragma_lex () usable in preprocess-only mode, and we decided just to add a comment about that for the time being, and to go ahead and implement that in the future, if it became necessary to support other pragmas during preprocessing.I think now is a good time to proceed with that plan, because I would liketo fix PR87299, which is about another pragma (#pragma GCC target) not working in preprocess-only mode. This patch makes the necessary changes for pragma_lex () to work in preprocess-only mode.I have also added a new callback, on_token_lex (), to libcpp. This is so thepreprocessor can see and stream out all the tokens that pragma_lex () gets from libcpp, since it won't otherwise see them. This seemed the simplest approach to me. Another possibility would be to add a wrapper function in c-family/c-lex.cc, which would call cpp_get_token_with_location(), and then also stream the token in preprocess-only mode, and then change all calls into libcpp in that file to use the wrapper function. The libcpp callback seemed cleaner to me FWIW.
I think the other way sounds better to me; there are only three calls to cpp_get_... in c_lex_with_flags.
The rest of the patch looks good. Jason
