[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #35 from Max TenEyck Woodbury  ---
True, none of the specifically listed maintainers have commented on
this version of the patch.  There are three people listed and a
general reference to all C and C++ front end maintainers.  It is
possible that they are not even aware of this discussion.

You are -not- listed specifically as a libcpp maintainer, only as a
general C front end maintainer specializing in soft-fp, i18n,
documentation and option handling.  On the other hand your name does
appear in the libcpp ChangeLog and I presume jsm28 is your committer
ID.  Your code looks good in general.

I do appreciate your helping me clarify my arguments.  I have indeed
been sloppy in my phrasing on more than one occasion.  On the other
hand, you have -not- addressed the main reason for adding this change:
this alternate implementation of #line __LINE__ is useful while the
current implementation is not.  As noted in an earlier comment, it
could -help- address at least two other issues currently being
discussed in other reports.

On 10/26/17, joseph at codesourcery dot com  wrote:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687
>
> --- Comment #34 from joseph at codesourcery dot com  dot com> ---
> None of the other preprocessor maintainers have commented on this bug in
> the past four years to disagree with my view of the natural identification
> of the current line for this __LINE__ token.  Unless any comment soon I
> think we can take there to be consensus on my interpretation, i.e. that
> this is INVALID and the existing interpretation is the most appropriate
> one.
>
> --
> You are receiving this mail because:
> You reported the bug.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[joseph at codesourcery dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #34 from joseph at codesourcery dot com  ---
None of the other preprocessor maintainers have commented on this bug in 
the past four years to disagree with my view of the natural identification 
of the current line for this __LINE__ token.  Unless any comment soon I 
think we can take there to be consensus on my interpretation, i.e. that 
this is INVALID and the existing interpretation is the most appropriate 
one.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #33 from Max TenEyck Woodbury  ---
The way it is currently implemented by GCC makes #line __LINE__ a
useless construct.  The form where subsequent line numbers remain
unchanged is very useful.  From the literature, that was the way it
was expected to work.  In THAT sense it is more portable.

I have had occasion to write C like pre-processors more than once.
When coded to collect the entire directive and then dispatch on the
directive keyword, the natural outcome for #line __LINE__ was to leave
the subsequent line numbers unchanged.
That falls out naturally if you code the specifiction for identifying
directives following the standards as a separate step.  It is only
when you merge the directive collection, tokenization  and
interpretation steps that you get an interpretation of #line __LINE__
that changes subsequent line numbers.

On 10/25/17, joseph at codesourcery dot com  wrote:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687
>
> --- Comment #32 from joseph at codesourcery dot com  dot com> ---
> The evidence from the DR discussion is that it's the *less* portable
> interpretation - that none of the implementations tested behaved as you
> suggest.
>
> --
> You are receiving this mail because:
> You reported the bug.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[joseph at codesourcery dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #32 from joseph at codesourcery dot com  ---
The evidence from the DR discussion is that it's the *less* portable 
interpretation - that none of the implementations tested behaved as you 
suggest.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #31 from Max TenEyck Woodbury  ---
The request in 82176 would remove all file components except the
filename itself.  It also puts control of the option on the comand
line which would usually mandate its use for all modules in a project.
It is a strong departure from the standard's requirement.  The request
in 70268 seems to be fairly similar if less draconian and a bit more
complex in its implementation.  Both could be resolved by consistantly
including #line ... "" directives near the
begining of the modules.  That is a good practice since it also
provides a natural internal source code identification of the modules.
The main objection to doing that would be the problem of maintaining
the value of '...'.

And that brings us to this request.  This is not that much of a
strain on the interpretation of what counts as the line number.  The
interpretation of __LINE__ is actually unchanged in the more recent
patch.  This simply changes -when- the substitution takes effect.  The
updated standard enumerates two times the expansion can be done.  This
change specifies that the more portable interpretation be used.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[joseph at codesourcery dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #30 from joseph at codesourcery dot com  ---
An option to use just the file's basename in __FILE__ is bug 82176.  I 
think that's a much more reasonable feature than straining the 
interpretation of what counts as the line number for the purposes of 
__LINE__ in a particular context.  See also bug 70268, and the ongoing 
discussion of BUILD_PATH_PREFIX_MAP (where Jeff objected to the use of an 
environment variable, but a command-line option version should still be 
reasonable).

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #29 from Max TenEyck Woodbury  ---
While #line is indeed most commonly used by code generators, it can be used in
other contexts.  The most common other use is to remove sensitive and useless
file name components from the file specification.  Since there are bad actors
who might be able to abuse that sensitive information, that use is becoming
more critical.  While it is possible to hide that information without using
#line __LINE__, doing so introduces potential code maintenance issues. 

If you look at the old programming literature, you can find places where the
use of #line __LINE__ is assumed to not change subsequent line numbers.  I
think allowing #line __LINE__ to change subsequent line numbers makes this
construction unusable for normal purposes.  If #line __LINE__ preserves
subsequent line numbering, I find it is much easier to write safe portable
code.  If you look at the question from a portability point of view, the
natural choice is to have #line __LINE__ leave subsequent line numbers
unchanged.

When I said 'not robust' I meant that it was fairly easy to frustrate the check
as implemented.  This might be considered a good thing if you want the
alternate behavior; you can still get the alternate behavior by hiding the use
of __LINE__ behind another macro, but then you would be back in the unspecified
behavior realm.  While it is possible to change the implementation to detect
hidden __LINE__ references, I found that that requires adding a communication
path between the macro.c module and the directives.c module.  I played around
and found that to be unnecessarily complicated.  It is almost always better to
leave the coder a choice of behaviors, so being able to hide the use of
__LINE__ if the user wants to should be considered a 'good thing'.

Joseph: Do you know of any existing cases where this change would cause user's
problems?  The existing behavior has caused me enough trouble that I now use a
patched version of GCC in place of any of the usual binary distributions.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[joseph at codesourcery dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #28 from joseph at codesourcery dot com  ---
Well, I also think the existing choice is the more natural choice for the 
value of __LINE__ in this case (#line is mainly for use by code 
generators, which can always count lines and specify the desired number 
when changing the filename).  And the note on the patch "Note that this 
solution is not robust; hiding the __LINE__ reference inside another macro 
will bypass this special case test." indicates changing the choice is not 
so simple after all (having a choice of line number that depends on 
whether __LINE__ is inside another macro seems even less natural).

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

Max TenEyck Woodbury  changed:

   What|Removed |Added

 Status|RESOLVED|REOPENED
 Resolution|INVALID |---

--- Comment #27 from Max TenEyck Woodbury  ---
The existence of examples in older programming literature indicates that the
proposed interpretation where #line __LINE__ did not change subsequent line
numbers was considered useful.  I have found this very useful myself.

The conclusion that this is not a "BUG" is correct, but it it is a useful
feature and it would make GCC easier to use.  GCC specifies the unspecified in
many places and it could do so here.  It is actually fairly easy to implement
as the patch I submitted to patches-gcc shows.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[jsm28 at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

Joseph S. Myers  changed:

   What|Removed |Added

 Status|SUSPENDED   |RESOLVED
 Resolution|--- |INVALID

--- Comment #26 from Joseph S. Myers  ---
DR#464 
concluded the existing choice is valid (and noted that apparently no
implementations follow the other choice, although the resolution says that
choice is valid as well), so there is no bug here.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #25 from Max TenEyck Woodbury  ---
(In reply to jos...@codesourcery.com from comment #24)
> ...
> 
> I don't believe the standard makes any such attempt to accommodate such a 
> (marginal) need;

GAG! Without this, a number of problems associated with externally developed 
development environments are very difficult, if not impossible, to manage.  
It is ultimately a portability problem; there must be a way to identify the 
source for the code that is independent of the particulars of the build 
environment and at the same time not change the line sequencing without forcing 
the code maintainers to maintain a count of code lines.  That puts this problem 
squarely in the middle of the domain of problems that language standards were 
developed to address!

>  the wording is simply the most natural way to describe 
> the possible forms of #line directive within the style used by the 
> standard.  Note there are no examples in the standard to indicate any 
> particular intent regarding corner cases of #line or __LINE__.

The 'natural' way to say this would be much closer to the way it was expressed 
in K&R.  Someone went to a great deal of trouble to rephrase it for some 
reason and, at least in my opinion, that reason included the interaction of 
__LINE__ and directives.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[joseph at codesourcery dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #24 from joseph at codesourcery dot com  ---
On Fri, 29 Nov 2013, mtewoodbury at gmail dot com wrote:

> > The question is not when it takes place, it's what the "current token" is 
> > when it takes place, because "line number" is defined in terms of the 
> > current token rather than the time of expansion.
> 
> If that is what they meant, they would have said something like "the line 
> number when the __LINE__ token is seen', but they say 'current line' and 
> define 'current line' elsewhere.
> 
> > I think the current token is clearly __LINE__ on the #line directive.  
> 
> But the standard says 'current line'.  It says nothing about 'current 
> token' in this context.

The only uses of the phrase "current line" in C11 are in subclause 5.2.2 
Character display semantics; I suppose you mean "current source line".  
What 6.10.4 says is "The line number of the current source line is one 
greater than the number of new-line characters read or introduced in 
translation phase 1 (5.1.1.2) while processing the source file to the 
current token.", and 6.10.8.1 says __LINE__ expands to "The presumed line 
number (within the current source file) of the current source line (an 
integer constant).".  "line number of the current source line" is defined 
as a whole in terms of the "current token"; "current source line" is not 
something that's defined or meaningful by itself.  Not processing until 
expansion occurs, not processing until the form of a syntax construct is 
identified, but processing to the current token, which is surely __LINE__ 
in this case.

> You are arguing about macro expansion again.  Macro expansion is an 
> immediate process and, as you pointed out, somewhat ambiguous.  The case 
> being argued here is the expansion of  in directives where their 
> expansion is delayed until after the form of the directive has been 
> established.

Expansion of those tokens is just one instance of macro expansion.  
Exactly the same ambiguity occurs in the case of

#line macro \
( \
__LINE__ \
)

(is "macro", "__LINE__" or ")" the current token?) and does not occur in 
the case of plain

#line __LINE__

> So everyone else is wrong?

Implementations not treating #line __LINE__ as giving the next line the 
same line number as the #line directive are either using a very strained 
interpretation of the standard, or haven't considered the issue and have 
achieved their results by accident.

> Realty, you should consider what the users of the language need to be able 
> to do.  There is need for an easy way to change __FILE__ without messing 
> up the line number sequence.  '#line __LINE__ ...' is fairly obviously that 
> mechanism and the extra verbiage in the standard is a fairly obvious 
> attempt to accommodate that need.

I don't believe the standard makes any such attempt to accommodate such a 
(marginal) need; the wording is simply the most natural way to describe 
the possible forms of #line directive within the style used by the 
standard.  Note there are no examples in the standard to indicate any 
particular intent regarding corner cases of #line or __LINE__.

If you disagree with an implementor (or conformance testsuite vendor, 
etc.) about the interpretation of the standard, the next step is to file a 
DR with WG14.  You can send proposed DRs directly to the convenor - most 
go that way - you don't need to go via your National Body (although WG14 
has no obligation to accept a document as a DR unless the formal NB 
process is followed).

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #23 from Max TenEyck Woodbury  ---
(In reply to jos...@codesourcery.com from comment #22)
> On Fri, 29 Nov 2013, mtewoodbury at gmail dot com wrote:
> 
> > The elaborate description of the different forms of the '#line' (and other) 
> > directives makes it clear that  expansion is not to take place 
> > until after the  for the directive has been seen.  
> 
> The question is not when it takes place, it's what the "current token" is 
> when it takes place, because "line number" is defined in terms of the 
> current token rather than the time of expansion.

If that is what they meant, they would have said something like "the line 
number when the __LINE__ token is seen', but they say 'current line' and 
define 'current line' elsewhere.

> I think the current token is clearly __LINE__ on the #line directive.  

But the standard says 'current line'.  It says nothing about 'current 
token' in this context.

> Consider the case where 
> there are no more tokens after that directive, just the newline at the end 
> of the directive (you can easily construct cases where the value of 
> __LINE__ in such a context affects the validity of the program, by 
> concatenating it with something else to form what might or might not be a 
> macro name) - it's not possible the the current token to be anything on a 
> subsequent line, because there aren't any subsequent tokens.  So the 
> current token has got to be __LINE__ (or maybe, in the more complicated 
> cases I referred to, something else on the #line line).  That's different 
> from
> 
> macro
> (
> __LINE__
> )
> 
> where all of the tokens from "macro" to ")" are involved in expansion at 
> the same time and you can argue for any of "macro", "__LINE__" and ")" as 
> being the "current token".  (It's also different from the cases where 
> backslash-newline appears in the middle of __LINE__ so the question is 
> whether "to the current token" means to the start of end of that token.)
> 
You are arguing about macro expansion again.  Macro expansion is an 
immediate process and, as you pointed out, somewhat ambiguous.  The case 
being argued here is the expansion of  in directives where their 
expansion is delayed until after the form of the directive has been 
established.

> > Accepted usage is for '#line __LINE__' to leave the line numbering 
> > unchanged.  
> 
> It's quite possible there's a common bug among multiple other 
> implementations where it does leave the line number unchanged, but I'm now 
> tending to that actually being a bug in those implementations rather than 
> an ambiguity in the standard.  That is, there is no way in the standard to 
> leave the line number unchanged, and no defect, and it might be better to 
> wait until a revision process for a new major C standard version is 
> underway them submit a proposal for e.g. #line with just a filename and no 
> line number (or a #file directive - either would be a reasonable way of 
> achieving that goal).

So everyone else is wrong?

Realty, you should consider what the users of the language need to be able 
to do.  There is need for an easy way to change __FILE__ without messing 
up the line number sequence.  '#line __LINE__ ...' is fairly obviously that 
mechanism and the extra verbiage in the standard is a fairly obvious 
attempt to accommodate that need.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[joseph at codesourcery dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #22 from joseph at codesourcery dot com  ---
On Fri, 29 Nov 2013, mtewoodbury at gmail dot com wrote:

> The elaborate description of the different forms of the '#line' (and other) 
> directives makes it clear that  expansion is not to take place 
> until 
> after the  for the directive has been seen.  

The question is not when it takes place, it's what the "current token" is 
when it takes place, because "line number" is defined in terms of the 
current token rather than the time of expansion.  I think the current 
token is clearly __LINE__ on the #line directive.  Consider the case where 
there are no more tokens after that directive, just the newline at the end 
of the directive (you can easily construct cases where the value of 
__LINE__ in such a context affects the validity of the program, by 
concatenating it with something else to form what might or might not be a 
macro name) - it's not possible the the current token to be anything on a 
subsequent line, because there aren't any subsequent tokens.  So the 
current token has got to be __LINE__ (or maybe, in the more complicated 
cases I referred to, something else on the #line line).  That's different 
from

macro
(
__LINE__
)

where all of the tokens from "macro" to ")" are involved in expansion at 
the same time and you can argue for any of "macro", "__LINE__" and ")" as 
being the "current token".  (It's also different from the cases where 
backslash-newline appears in the middle of __LINE__ so the question is 
whether "to the current token" means to the start of end of that token.)

> Accepted usage is for '#line __LINE__' to leave the line numbering unchanged. 
>  

It's quite possible there's a common bug among multiple other 
implementations where it does leave the line number unchanged, but I'm now 
tending to that actually being a bug in those implementations rather than 
an ambiguity in the standard.  That is, there is no way in the standard to 
leave the line number unchanged, and no defect, and it might be better to 
wait until a revision process for a new major C standard version is 
underway them submit a proposal for e.g. #line with just a filename and no 
line number (or a #file directive - either would be a reasonable way of 
achieving that goal).

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #21 from Max TenEyck Woodbury  ---
(In reply to Joseph S. Myers from comment #20)
> Suspending pending a DR since I think the present code is correct and while
> the standard is ambiguous I think the interpretation here strains the
> wording of the standard.  I'd proposed to close this if a DR (or other WG14
> document) isn't under consideration at the Parma meeting of WG14.

A 'DR' is not necessary and is unlikely to be submitted.  The ambiguity
referred 
to is in the expansion of macros, not in the processing of directives.  

The elaborate description of the different forms of the '#line' (and other) 
directives makes it clear that  expansion is not to take place until 
after the  for the directive has been seen.  

Accepted usage is for '#line __LINE__' to leave the line numbering unchanged.  
Any other interpretation would require some form of expression evaluation in 
order to leave line numbering unchanged; a possibility that has been rejected.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[jsm28 at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

Joseph S. Myers  changed:

   What|Removed |Added

 Status|UNCONFIRMED |SUSPENDED
   Last reconfirmed||2013-11-28
 Ever confirmed|0   |1

--- Comment #20 from Joseph S. Myers  ---
Suspending pending a DR since I think the present code is correct and while the
standard is ambiguous I think the interpretation here strains the wording of
the standard.  I'd proposed to close this if a DR (or other WG14 document)
isn't under consideration at the Parma meeting of WG14.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[joseph at codesourcery dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #19 from joseph at codesourcery dot com  ---
On Tue, 19 Nov 2013, mtewoodbury at gmail dot com wrote:

> it's been a couple weeks.  Any progress?

I don't recall seeing this on gcc-patches (any patch attached to Bugzilla 
is taken as work-in-progress, not a submission for review) - please give 
the URL in the list archives of the patch submission.

(I don't see your assignment in fencepost:/gd/gnuorg/copyright.list, but 
the FSF does not always record paperwork in a particularly timely fashion; 
if it's been at least a week since you sent in paperwork, I advise chasing 
up ass...@gnu.org.  It would require more careful consideration to see if 
this patch is actually large enough to need paperwork.)

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #18 from Max TenEyck Woodbury  ---
(In reply to jos...@codesourcery.com from comment #16)
> On Mon, 4 Nov 2013, mtewoodbury at gmail dot com wrote:
> 
> > Could you all give me some idea on how soon this might be applied?
> 
> At some time after seeing this on gcc-patches, I or another preprocessor 
> reviewer will review the patch (after checking that the FSF copyright.list 
> file reflects receipt of the assignment, if the patch seems big enough to 
> need it).

it's been a couple weeks.  Any progress?

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #17 from Max TenEyck Woodbury  ---
It might be better to put the CUR__LINE__ definition in 'internal.h' instead of
in 'cpplib.h'.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[joseph at codesourcery dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #16 from joseph at codesourcery dot com  ---
On Mon, 4 Nov 2013, mtewoodbury at gmail dot com wrote:

> Could you all give me some idea on how soon this might be applied?

At some time after seeing this on gcc-patches, I or another preprocessor 
reviewer will review the patch (after checking that the FSF copyright.list 
file reflects receipt of the assignment, if the patch seems big enough to 
need it).

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #15 from Max TenEyck Woodbury  ---
Could you all give me some idea on how soon this might be applied?

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #14 from Max TenEyck Woodbury  ---
Created attachment 31125
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=31125&action=edit
Patch to postpone __LINE__ evaluation to the end of a # line directive.

Patch includes changes to 2 source code files, a header and the line4.c test
case.

make bootstrap OK.

make -k check of patched and un-patched code matches except for time stamps.

Formal release of changes copyright to FSF granted.

Max TenEyck Woodbury

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #13 from Max TenEyck Woodbury  ---
(In reply to jos...@codesourcery.com from comment #12)
> On Wed, 30 Oct 2013, mtewoodbury at gmail dot com wrote:
> 
>> Thank you, I will look info all of that.  My own resources have limits; when
>> it comes to testing generated code on many architectures, I will definitely
>> need such resources.
> 
> A fix for this particular bug should only need testing on one 
> architecture.  It's not like changes to the register allocator that are 
> likely to have architecture-specific issues.

Yes.  'make -w -k check' on the base configuration is still in progress. 
There's enough stuff wrong with the base that I need a log to check against. 
Using the compile farm here would add a queuing delay, slowing the process
further.

Oh, and could you mark this bug as either 'confirmed' or 'in-progress'?

I do have another bug (58884) that will need those resources if I am the one
that tackles fixing it...

Later...

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[joseph at codesourcery dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #12 from joseph at codesourcery dot com  ---
On Wed, 30 Oct 2013, mtewoodbury at gmail dot com wrote:

> Thank you, I will look info all of that.  My own resources have limits; when 
> it
> comes to testing generated code on many architectures, I will definitely need
> such resources.

A fix for this particular bug should only need testing on one 
architecture.  It's not like changes to the register allocator that are 
likely to have architecture-specific issues.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #11 from Max TenEyck Woodbury  ---
(In reply to Manuel López-Ibáñez from comment #10)
> 
> If you are planning to do sporadic GCC development, it may be worthwhile to
> ask for an account in the Compile Farm http://gcc.gnu.org/wiki/CompileFarm
> 
> (I could have never done any GCC development without it.)
> 
> For building GCC and testing patches there are several scripts in contrib/.
> I also have my own:
> http://gcc.gnu.org/wiki/
> ManuelL%C3%B3pezIb%C3%A1%C3%B1ez?action=AttachFile&do=view&target=gccfarming

Thank you, I will look info all of that.  My own resources have limits; when it
comes to testing generated code on many architectures, I will definitely need
such resources.

I will also look at the scripts after I have a bit better handle on exactly
what kinds of problems testing raise.  I was mildly surprised that changing
line numbers in headers produced differences in what is obviously generated
code.  Without having seen that buried deep in the build directory tree, I am
sure I would not appreciate any scripts that interact with that aspect of the
build process.

I am at the base of the learning curve here.  I have some climbing to do before
I will be able to make significant contributions...

'make check' on the un-patched sources in progress.  Later...

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[manu at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

Manuel López-Ibáñez  changed:

   What|Removed |Added

 CC||manu at gcc dot gnu.org

--- Comment #10 from Manuel López-Ibáñez  ---
(In reply to Max TenEyck Woodbury from comment #9)
> temporary file names, but I think that I can get past that.  It just takes
> hours for the builds to complete...

If you are planning to do sporadic GCC development, it may be worthwhile to ask
for an account in the Compile Farm http://gcc.gnu.org/wiki/CompileFarm

(I could have never done any GCC development without it.)

For building GCC and testing patches there are several scripts in contrib/. I
also have my own:
http://gcc.gnu.org/wiki/ManuelL%C3%B3pezIb%C3%A1%C3%B1ez?action=AttachFile&do=view&target=gccfarming

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #9 from Max TenEyck Woodbury  ---
(In reply to jos...@codesourcery.com from comment #8)
> Thanks for working on this bug.  http://gcc.gnu.org/contribute.html 
> describes how to submit changes (including testcases etc.).

Thank you for the pointer.

As I said, I have a patch.  The patch includes an addition to the line4.c test
case.

What is delaying this is my testing of the change.  Specifically, I am
configuring, building bootstrap and building check for both the patched and
un-patched code then comparing the results.  The comparison was throwing up
many spurious differences due entirely to the slightly different directory
names I was using in the configuration process.  I think I now have those
details worked out, but there are still spurious differences due to temporary
file names, but I think that I can get past that.  It just takes hours for the
builds to complete...

Argh! -- someone put in a patch that I didn't track properly.  More details to
account for and another round of delays.  It'll be done when it's properly
tested.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[joseph at codesourcery dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #8 from joseph at codesourcery dot com  ---
Thanks for working on this bug.  http://gcc.gnu.org/contribute.html 
describes how to submit changes (including testcases etc.).

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #7 from Max TenEyck Woodbury  ---
I have a patch written and I am testing it now.

What steps (other than posting it here when the tests are done) need to be done
to get it applied?

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #6 from Max TenEyck Woodbury  ---
(In reply to Andrew Pinski from comment #5)
> > Simply to make identification host independent.  The fact that my
> >projects are stored on '/VOL10' on one of my machines and '/DATA0.2'
> 
> This sounds like a bug in how you are compiling the sources.  Also there are
> options inside GDB to transpose paths to the path on your machine.

It is precisely because the identification varies with the way to file name is
passed to the compiler that makes setting __FILE__ desirable.  The compiler
invocation details are not always under the developer's control.  Tools like
'make', 'autoconf' and 'automake' dictate the forms.  So, I have fairly strong
objections to calling it an external procedural BUG.  The issue REALLY is that
__LINE__ gets messed up; leave discussions of how the compiler is invoked out
of it.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[pinskia at gcc dot gnu.org]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

Andrew Pinski  changed:

   What|Removed |Added

   Severity|major   |normal

--- Comment #5 from Andrew Pinski  ---
> Simply to make identification host independent.  The fact that my
>projects are stored on '/VOL10' on one of my machines and '/DATA0.2'

This sounds like a bug in how you are compiling the sources.  Also there are
options inside GDB to transpose paths to the path on your machine.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

Max TenEyck Woodbury  changed:

   What|Removed |Added

   Severity|normal  |major

--- Comment #4 from Max TenEyck Woodbury  ---
Why is this important?

1)  There must be a way to modify the '__FILE__' value without messing
up the '__LINE__' value.  A 'Google' shows that the expected way to
do this is:

# line __LINE__ "new-__FILE__-value"

but with this bug, the __LINE__ value gets messed up and all 
subsequent diagnostics are off by at least one.  It also makes
debugging more difficult if not impossible.

Now, why change the '__FILE__' value?

Simply to make identification host independent.  The fact that my
projects are stored on '/VOL10' on one of my machines and '/DATA0.2'
on another only confuses any trouble reports.  What is important is
that the file is 'myproject/configure.c' (or whatever).

2)  Similarly, adding multi-line comments to the '# line __LINE__ ...'
directive should NOT mess with the line numbering.

3)  Why not use a specific value in place of '__LINE__'?

Because that embeds a maintenance problem that should not be there
into the source.  If something gets added before the '# line  ...'
directive, then  has to be updated.  If the source is 'generated'
by some tool, this is reasonable, but it is NOT reasonable for base
level source files.  Similarly, there should be a freedom to edit
the comment without having to play with the line number.

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #3 from Max TenEyck Woodbury  ---
Turn off the special case immediatly after getting the number to
eliminate side effects...

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

Max TenEyck Woodbury  changed:

   What|Removed |Added

 CC||mtewoodbury at gmail dot com

--- Comment #2 from Max TenEyck Woodbury  ---
Grumble!

Modified testsuite/gcc.dg/cpp/line4.c to test __LINE__ use.  I think I
saw a stronger set of tests while 'googling' the problem.  I was a bit
short on understanding when I saw it, so I didn't mark it for study...

I've looked at the code that handles this.  It's in the libcpp tree.

Thw indentation is messed up.  '\t' used in many places WITHOUT consistant
tab widths.  I see spots with '\t' stops at 4 and at 8.  Indent width is
mostly 2.  Style is a bit mixed.  FOR MY OWN USE, I've cleaned this up.
The result is much more readable.

I tried a simple look-ahead hack, but no go.

Going the mild KLUGE route:  Use a special value to indicate the bad
value and emit it under special circumstances.

- define the special value CUR__LINE__ in include/libcpp.h (-1)
- modify state.in_directive to have a special value (2) when in the #line
  directive (in do_line).
- check for the CUR__LINE__ value at the end of do_line after flushing the
  tokens and use replace the special value with the current line number
  before doing the state change.
- in macro.c, __LINE__ processing, look for the special stat.in_directive
  value and return CUR__LINE__ if it has the special value.

Potential problem if someone uses __LINE__ to construct the file name.

Intrest?

[Bug preprocessor/58687] "#line __LINE__ ..." changes subsequent line numbers

[mtewoodbury at gmail dot com]

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58687

--- Comment #1 from Max TenEyck Woodbury  ---
Created attachment 31080
  --> http://gcc.gnu.org/bugzilla/attachment.cgi?id=31080&action=edit
test case

Comment out the '#line' directives and it compiles without error...