Re: YYEOF shouldn't be defined with bison -y

[Akim Demaille]

Hi!

> Le 18 août 2018 à 15:46, Dimitrios Apostolou  a écrit :
> 
> On Sat, 18 Aug 2018 06:39:36 +0200,
> Akim Demaille wrote:
>> 
>> There’s something which is not fully clear here: unless _you_ used YYEOF
>> somewhere in your code, there’s no reason for the parser not to be
>> compilable with byacc.
> 
> Yes, YYEOF was used in the project that I maintained, but I was not the one
> that introduced it. After years of patches from various people, it was
> incidentally discovered that the parser fails to compile with byacc, because
> at some point usage of YYEOF was introduced, and it went undetected, despite
> having always generated the parser using "bison -y".
> 
>> So I expect that you did use YYEOF.  In which case, the question is rather:
>> where was it used, to what purpose, and how would you have expected to
>> write it independently of YYEOF?
> 
> I'm not sure of the details at the moment, but YYEOF usage got removed
> eventually. What I did myself to ensure that this does not happen again, I
> changed the Travis build environment to uninstall bison and install byacc. So
> now every commit is verified that is yacc-compatible before being
> merged. Still I believe that « bison -y" ought to catch such errors.

I’m still trying to figure out how to handle this, and I
see nothing satisfying.  YYEOF is defined to 0, and I certainly
do not want to replace symbolic values by actual numbers.  And
Bison itself has to check in many places whether a token is YYEOF
or not.  Using 0 instead of YYEOF is wrong.  And bison will not
start looking at the user’s action to see if she’s using YYEOF
(what if she took care of defining it if it is not).  Since it’s
a macro, it’s straightforward for the user to provide a definition
if it is undefined.

Besides, I see that byacc defines YYEOF, since April 2014, and
does not provide an option -y that would allow to control this.

So I don’t see any approach that would improve the global situation,
sorry :(

Akim

Re: YYEOF shouldn't be defined with bison -y

[Dimitrios Apostolou]

On Sat, 18 Aug 2018 06:39:36 +0200,
Akim Demaille wrote:
> 
> Hi!
> 
> I’m coming long after the discussion, sorry.
> 
> > Le 5 déc. 2017 à 02:32, Dimitrios Apostolou  a écrit :
> > 
> > On Mon, 04 Dec 2017 21:24:40 +0100,
> > Kaz Kylheku wrote:
> >> 
> >> On 03.12.2017 18:34, Dimitrios Apostolou wrote:
> >>> Hello list,
> >>> 
> >>> I am configuring the build using AC_PROG_YACC, which invokes bison with 
> >>> "-y"
> >>> option.  But bison-generated parser compiled successfully, and YYEOF
> >>> extension
> >>> in the parser remained undetected.
> >>> 
> >>> Generating the parser with "byacc" on the other hand creates a file that
> >>> does
> >>> not compile because of undefined symbol YYEOF.
> >> 
> >> What is your error report?
> >> 
> >> Are you saying that you cannot boostrap a build of Bison using Byacc?
> > 
> > My error report has nothing to do with bootstrapping Bison, sorry for not
> > being clear.
> > 
> > I am generating a parser as part of a different project, and that project is
> > configured with AC_PROG_YACC. As such, it invokes Bison with "-y" option, 
> > and
> > I would expect that Bison tries hard to fail when the parser uses 
> > non-portable
> > features, such as YYEOF.
> > 
> > This is not the case however, and the project was released, and this was
> > discovered after longtime that somebody built it with byacc instead of
> > Bison. With that version of byacc, the generated parser C code failed to 
> > build
> > with "undefined symbol YYEOF".
> > 
> > I would expect that "bison -y" undefines YYEOF, so that the software that 
> > uses
> > YYEOF fails to build.
> 
> There’s something which is not fully clear here: unless _you_ used YYEOF
> somewhere in your code, there’s no reason for the parser not to be
> compilable with byacc.

Yes, YYEOF was used in the project that I maintained, but I was not the one
that introduced it. After years of patches from various people, it was
incidentally discovered that the parser fails to compile with byacc, because
at some point usage of YYEOF was introduced, and it went undetected, despite
having always generated the parser using "bison -y".

> So I expect that you did use YYEOF.  In which case, the question is rather:
> where was it used, to what purpose, and how would you have expected to
> write it independently of YYEOF?

I'm not sure of the details at the moment, but YYEOF usage got removed
eventually. What I did myself to ensure that this does not happen again, I
changed the Travis build environment to uninstall bison and install byacc. So
now every commit is verified that is yacc-compatible before being
merged. Still I believe that "bison -y" ought to catch such errors.


Thanks for revisiting this.

Dimitris

Re: YYEOF shouldn't be defined with bison -y

[Akim Demaille]

Hi!

I’m coming long after the discussion, sorry.

> Le 5 déc. 2017 à 02:32, Dimitrios Apostolou  a écrit :
> 
> On Mon, 04 Dec 2017 21:24:40 +0100,
> Kaz Kylheku wrote:
>> 
>> On 03.12.2017 18:34, Dimitrios Apostolou wrote:
>>> Hello list,
>>> 
>>> I am configuring the build using AC_PROG_YACC, which invokes bison with "-y"
>>> option.  But bison-generated parser compiled successfully, and YYEOF
>>> extension
>>> in the parser remained undetected.
>>> 
>>> Generating the parser with "byacc" on the other hand creates a file that
>>> does
>>> not compile because of undefined symbol YYEOF.
>> 
>> What is your error report?
>> 
>> Are you saying that you cannot boostrap a build of Bison using Byacc?
> 
> My error report has nothing to do with bootstrapping Bison, sorry for not
> being clear.
> 
> I am generating a parser as part of a different project, and that project is
> configured with AC_PROG_YACC. As such, it invokes Bison with "-y" option, and
> I would expect that Bison tries hard to fail when the parser uses non-portable
> features, such as YYEOF.
> 
> This is not the case however, and the project was released, and this was
> discovered after longtime that somebody built it with byacc instead of
> Bison. With that version of byacc, the generated parser C code failed to build
> with "undefined symbol YYEOF".
> 
> I would expect that "bison -y" undefines YYEOF, so that the software that uses
> YYEOF fails to build.

There’s something which is not fully clear here: unless _you_ used YYEOF
somewhere in your code, there’s no reason for the parser not to be
compilable with byacc.

So I expect that you did use YYEOF.  In which case, the question is rather:
where was it used, to what purpose, and how would you have expected to
write it independently of YYEOF?

Re: YYEOF shouldn't be defined with bison -y

[Hans Åberg]

> On 5 Dec 2017, at 02:32, Dimitrios Apostolou  wrote:
> 
> I am generating a parser as part of a different project, and that project is
> configured with AC_PROG_YACC. As such, it invokes Bison with "-y" option, and
> I would expect that Bison tries hard to fail when the parser uses non-portable
> features, such as YYEOF.
> 
> This is not the case however, and the project was released, and this was
> discovered after longtime that somebody built it with byacc instead of
> Bison. With that version of byacc, the generated parser C code failed to build
> with "undefined symbol YYEOF".
> 
> I would expect that "bison -y" undefines YYEOF, so that the software that uses
> YYEOF fails to build.

The generated parser  .c file is intended to run on all platforms, so by 
setting the makefile to only compile if the grammar .y file is altered, such 
problems might be avoided. Automake AC_PROG_YACC is only designed for Yacc 
compatibility, so otherwise, for example if you want the program to be Bison, 
another symbol should be used.

According to POSIX [1], the names starting with YY are for the implementation 
of the Yacc program, so if such are used in the grammar, it is not portable 
(like names starting with _ in C). From a Flex .l file, one can return EOF.


1. http://pubs.opengroup.org/onlinepubs/009604599/utilities/yacc.html

Re: YYEOF shouldn't be defined with bison -y

[Kaz Kylheku]

On 04.12.2017 17:32, Dimitrios Apostolou wrote:

Alright, so maybe it's not a bug. But if that's the case, what's the
recommended way to ensure that the parser is portable to most yacc 
flavours?


Install the set which represents "most flavors" in your opinion, make 
your
build system flexible to switch among them, build the program with each 
one

in turn and run your regression test suite.

That's the main thing here: Yacc parsers are not platforms; this is not
the kind of portability like to a new operating system, chip, or
system board (where you have to plan for the future, because you don't
know what those things will be five years from now). The Yacc
implementations are there, you can get all the ones you care about
onto the development system.

Re: YYEOF shouldn't be defined with bison -y

[Dimitrios Apostolou]

On Mon, 04 Dec 2017 21:24:40 +0100,
Kaz Kylheku wrote:
> 
> On 03.12.2017 18:34, Dimitrios Apostolou wrote:
> > Hello list,
> > 
> > I am configuring the build using AC_PROG_YACC, which invokes bison with "-y"
> > option.  But bison-generated parser compiled successfully, and YYEOF
> > extension
> > in the parser remained undetected.
> > 
> > Generating the parser with "byacc" on the other hand creates a file that
> > does
> > not compile because of undefined symbol YYEOF.
> 
> What is your error report?
> 
> Are you saying that you cannot boostrap a build of Bison using Byacc?

My error report has nothing to do with bootstrapping Bison, sorry for not
being clear.

I am generating a parser as part of a different project, and that project is
configured with AC_PROG_YACC. As such, it invokes Bison with "-y" option, and
I would expect that Bison tries hard to fail when the parser uses non-portable
features, such as YYEOF.

This is not the case however, and the project was released, and this was
discovered after longtime that somebody built it with byacc instead of
Bison. With that version of byacc, the generated parser C code failed to build
with "undefined symbol YYEOF".

I would expect that "bison -y" undefines YYEOF, so that the software that uses
YYEOF fails to build.

> 
> The Bison source tree contains the generated parse-gram.c file, so I would
> expect that you don't need an installation of any kind of Yacc to be
> able to build it. (So if it is intended to work with Byacc but doesn't,
> it's a problem, but shouldn't be a blocker.)
> 
> It is not wrong for Yacc implementations to introduce additional YY*
> identifiers; they are in the "Yacc namespace"; sort of like POSIX (or anyone
> else) can introduce errno macros beginning with E in the  header.

Alright, so maybe it's not a bug. But if that's the case, what's the
recommended way to ensure that the parser is portable to most yacc flavours?


Thanks,
Dimitris

Re: YYEOF shouldn't be defined with bison -y

[Kaz Kylheku]

On 03.12.2017 18:34, Dimitrios Apostolou wrote:

Hello list,

I am configuring the build using AC_PROG_YACC, which invokes bison with 
"-y"
option.  But bison-generated parser compiled successfully, and YYEOF 
extension

in the parser remained undetected.

Generating the parser with "byacc" on the other hand creates a file 
that does

not compile because of undefined symbol YYEOF.


What is your error report?

Are you saying that you cannot boostrap a build of Bison using Byacc?

The Bison source tree contains the generated parse-gram.c file, so I 
would

expect that you don't need an installation of any kind of Yacc to be
able to build it. (So if it is intended to work with Byacc but doesn't,
it's a problem, but shouldn't be a blocker.)

It is not wrong for Yacc implementations to introduce additional YY*
identifiers; they are in the "Yacc namespace"; sort of like POSIX (or 
anyone
else) can introduce errno macros beginning with E in the  
header.

YYEOF shouldn't be defined with bison -y

[Dimitrios Apostolou]

Hello list,

I am configuring the build using AC_PROG_YACC, which invokes bison with "-y"
option.  But bison-generated parser compiled successfully, and YYEOF extension
in the parser remained undetected.

Generating the parser with "byacc" on the other hand creates a file that does
not compile because of undefined symbol YYEOF.


Thanks,
Dimitris