Re: Counterexamples timeout

[Akim Demaille]

I will have to make another tarball, I had left a debugging trace.  Sorry about 
that.


commit 25b3d0e1a3f97a33615099e4b211f3953990c203
Author: Akim Demaille 
Date:   Tue Sep 20 08:19:02 2022 +0200

cex: check -Dcex.timeout

* src/counterexample.c (counterexample_init): Remove stray debug
trace.
Complain about invalid values.
* tests/input.at (-Dcex.timeout): New.

diff --git a/src/counterexample.c b/src/counterexample.c
index 8a4f8b2c..82ee43da 100644
--- a/src/counterexample.c
+++ b/src/counterexample.c
@@ -30,6 +30,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -1209,15 +1210,23 @@ static xtime_t cumulative_time;
 void
 counterexample_init (void)
 {
+  // Check cex.timeout.
   {
-char *cp = muscle_percent_define_get ("cex.timeout");
+const char *variable = "cex.timeout";
+char *cp = muscle_percent_define_get (variable);
 if (*cp != '\0')
   {
 char *end = NULL;
 double v = c_strtod (cp, &end);
-if (*end == '\0' && errno == 0)
+if (*end == '\0' && errno == 0 && v >= 0)
   time_limit = v;
-fprintf (stderr, "lim: %f from %s\n", time_limit, cp);
+else
+  {
+location loc = muscle_percent_define_get_loc (variable);
+complain (&loc, complaint,
+  _("invalid value for %%define variable %s: %s"),
+  quote (variable), quote_n (1, cp));
+  }
   }
 free (cp);
   }
diff --git a/tests/input.at b/tests/input.at
index 5cacb066..04b31f99 100644
--- a/tests/input.at
+++ b/tests/input.at
@@ -3283,3 +3283,39 @@ m4_pushdef([AT_TEST],
 m4_popdef([AT_TEST])
 
 AT_CLEANUP
+
+
+## --- ##
+## -Dcex.timeout.  ##
+## --- ##
+
+AT_SETUP([[-Dcex.timeout]])
+
+AT_KEYWORDS([cex])
+
+# AT_TEST(OPTIONS)
+# 
+m4_pushdef([AT_TEST],
+[AT_DATA([[input.y]],
+[[
+%%
+exp: %empty;
+]])
+AT_BISON_CHECK([[$1 input.y]], [[$2]], [[]], [m4_ifvaln([$3], [$3])])
+])
+
+# It's ok to define the timeout even if we don't run -Wcex.
+AT_TEST([-Dcex.timeout=1], [0])
+AT_TEST([-Dcex.timeout=1 -Wcex], [0])
+AT_TEST([-Dcex.timeout=1.5 -Wcex], [0])
+# No time at all.  Not really wrong.
+AT_TEST([-Dcex.timeout=0 -Wcex], [0])
+
+AT_TEST([-Dcex.timeout=foo -Wcex], [1],
+[[:3: error: invalid value for %define variable 
'cex.timeout': 'foo']])
+AT_TEST([-Dcex.timeout=-1.5 -Wcex], [1],
+[[:3: error: invalid value for %define variable 
'cex.timeout': '-1.5']])
+
+m4_popdef([AT_TEST])
+
+AT_CLEANUP

Re: Counterexamples timeout

[Akim Demaille]

Frank,

> Le 16 sept. 2022 à 11:12, Frank Heckenbach  a écrit :
> 
> Thanks. It looks good; I can't easily test it now, but after the
> next release (but don't bother making one just for this).

I recently made a tarball available:

- https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.2.44-de30.tar.gz  5.6M
- https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.2.44-de30.tar.lz  3.1M
- https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.2.44-de30.tar.xz  3.1M

Cheers!

Re: Counterexamples timeout

[Akim Demaille]

Frank,

> Le 16 sept. 2022 à 11:12, Frank Heckenbach  a écrit :
> 
> Akim Demaille wrote:
> 
>>> Le 15 sept. 2022 à 19:02, Jacob L. Mandelson  a 
>>> écrit :
>>> 
>>> I think the English text needs a few minor edits.[...]
>> 
>> Thanks a lot for this!  I installed the following commit, with your fixes.
> 
> Thanks. It looks good; I can't easily test it now, but after the
> next release (but don't bother making one just for this).

I'd like to make a (bug-fix) release soon anyway.  The latest release is
quite old now, and several issues have been addressed.

>> +@deffn Directive {%define cex.timeout} @var{duration}
> 
> Does this mean this can be set in the grammar file? I wonder if
> that's useful -- it seems more of an environmental setting to me.
> But if it would be more work to make it not settable this way, don't
> worry.

Yes, it does mean you can define it in the grammar.  I also agree
it does not make a lot of sense.

But %define and -D are two interfaces to the same store.  It would
indeed require added complexity to have one but not the other.

Cheers.

Re: Counterexamples timeout

[Akim Demaille]

Hi Jacob,

> Le 15 sept. 2022 à 19:02, Jacob L. Mandelson  a écrit 
> :
> 
> I think the English text needs a few minor edits.[...]

Thanks a lot for this!  I installed the following commit, with your fixes.

Cheers.

commit c0258915474496874318f4767ba6ebcc1e9471f0
Author: Akim Demaille 
Date:   Tue Sep 13 08:25:59 2022 +0200

cex: provide the user with a means to change the timeout

Reported by Frank Heckenbach.
https://lists.gnu.org/r/bug-bison/2022-07/msg00011.html

* bootstrap.conf: Use c_strtod, so that even in French locales "1.5"
is accepted, instead of "1,5".
* src/counterexample.c, src/state-item.c: Use xtime_t instead of
time_t, so that accuracy goes from seconds to nanoseconds.
( counterexample_init): Depend on cex.timeout rather than
$TIME_LIMIT.
* doc/bison.texi (%define Summary): Document cex.timeout.

diff --git a/NEWS b/NEWS
index f480cb95..40199a7a 100644
--- a/NEWS
+++ b/NEWS
@@ -8,6 +8,10 @@ GNU Bison NEWS
 
   The C++ skeletons now expose copy and move operators for symbols.
 
+  The `cex.timeout` %define variable allows to control when we give up
+  finding a unifying counterexample.  For instance `bison -Wcex
+  -Dcex.timeout=.5 gram.y` to limit to 1/2s.
+
 * Noteworthy changes in release 3.8.2 (2021-09-25) [stable]
 
   Fixed portability issues of bison on Cygwin.
diff --git a/bootstrap.conf b/bootstrap.conf
index 9a6070ba..fbe89422 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -19,7 +19,7 @@
 gnulib_modules='
   argmatch array-list assert assure attribute
   bitsetv
-  calloc-posix close closeout config-h c-strcase
+  calloc-posix close closeout config-h c-strcase c-strtod
   configmake
   dirname
   error execute extensions
@@ -44,7 +44,7 @@ gnulib_modules='
   realloc-posix
   relocatable-prog relocatable-script
   rename
-  spawn-pipe stdbool stpcpy stpncpy strdup-posix strerror strtod strverscmp
+  spawn-pipe stdbool stpcpy stpncpy strdup-posix strerror strverscmp
   sys_ioctl
   termios
   timevar
diff --git a/doc/bison.texi b/doc/bison.texi
index b2eddcf8..5ffe21d0 100644
--- a/doc/bison.texi
+++ b/doc/bison.texi
@@ -6894,6 +6894,23 @@ @node %define Summary
 @c api.value.union.name
 
 
+@c == cex.timeout
+
+@deffn Directive {%define cex.timeout} @var{duration}
+
+@itemize @bullet
+@item Language(s): all
+
+@item Purpose:
+Define the time limit for finding unifying counterexamples.
+
+@item Accepted Values: duration in seconds, e.g., @samp{1}, @samp{0.5}.
+
+@item Default Value: 5
+@end itemize
+@end deffn
+
+
 @c == lr.default-reduction
 
 @deffn Directive {%define lr.default-reduction} @var{when}
diff --git a/lib/.gitignore b/lib/.gitignore
index 03d09956..9f15868f 100644
--- a/lib/.gitignore
+++ b/lib/.gitignore
@@ -40,6 +40,8 @@
 /c-strcasecmp.c
 /c-strcaseeq.h
 /c-strncasecmp.c
+/c-strtod.c
+/c-strtod.h
 /calloc.c
 /canonicalize-lgpl.c
 /canonicalize.c
diff --git a/m4/.gitignore b/m4/.gitignore
index c3a50c44..4748c6e0 100644
--- a/m4/.gitignore
+++ b/m4/.gitignore
@@ -7,6 +7,7 @@
 /asm-underscore.m4
 /assert.m4
 /builtin-expect.m4
+/c-strtod.m4
 /calloc.m4
 /canonicalize.m4
 /chdir-long.m4
@@ -83,6 +84,7 @@
 /include_next.m4
 /inline.m4
 /intdiv0.m4
+/intl-thread-locale.m4
 /intl.m4
 /intldir.m4
 /intlmacosx.m4
diff --git a/src/counterexample.c b/src/counterexample.c
index 71436fd6..8a4f8b2c 100644
--- a/src/counterexample.c
+++ b/src/counterexample.c
@@ -23,7 +23,9 @@
 
 #include "system.h"
 
+#include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -39,6 +41,7 @@
 #include "gram.h"
 #include "lalr.h"
 #include "lssi.h"
+#include "muscle-tab.h"
 #include "nullable.h"
 #include "parse-simulation.h"
 
@@ -1115,7 +1118,7 @@ unifying_example (state_item_number itm1,
  (Hash_comparator) visited_comparator,
  (Hash_data_freer) search_state_free);
   ssb_append (initial);
-  time_t start = time (NULL);
+  xtime_t start = gethrxtime ();
   bool assurance_printed = false;
   search_state *stage3result = NULL;
   counterexample *cex = NULL;
@@ -1160,7 +1163,7 @@ unifying_example (state_item_number itm1,
 }
   if (TIME_LIMIT_ENFORCED)
 {
-  double time_passed = difftime (time (NULL), start);
+  double time_passed = (gethrxtime () - start) / 1e9;
   if (!assurance_printed && time_passed > ASSURANCE_LIMIT
   && stage3result)
 {
@@ -1201,25 +1204,24 @@ cex_search_end:;
   return cex;
 }
 
-static time_t cumulative_time;
+static xtime_t cumulative_time;
 
 void
 counterexample_init (void)
 {
-  /* Recognize $TIME_LIMIT.  Not a public feature, just to help
- debugging.  If we need something public, a %define/-D/-F variable
- would be more appropriate. *

Re: Counterexamples timeout

[Akim Demaille]

Hi Frank,

> Le 29 juil. 2022 à 14:09, Frank Heckenbach  a écrit :
> 
> Hi,
> 
> I don't know if it's something peculiar about my grammars, but it
> seems when using "-Wcounterexamples", Bison either finds
> counterexamples almost immediately or runs into the timeout.
> (Or maybe 6 seconds (see below) are just so eternally long to a
> modern CPU that there's not much between that and infinity. ;)
> [...]

I agree.  Let's change that.  What do you think about this proposal?

commit 6a15951030ade757de13159e545394af3cd40205
Author: Akim Demaille 
Date:   Tue Sep 13 08:25:59 2022 +0200

cex: provide the user with a means to change the timeout

Reported by Frank Heckenbach.
https://lists.gnu.org/r/bug-bison/2022-07/msg00011.html

* bootstrap.conf: Use c_strtod, so that even in French locales "1.5"
is accepted, instead of "1,5".
* src/counterexample.c, src/state-item.c: Use xtime_t instead of
time_t, so that accuracy goes from seconds to nanoseconds.
( counterexample_init): Depend on cex.timeout rather than
$TIME_LIMIT.
* doc/bison.texi (%define Summary): Document cex.timeout.

diff --git a/NEWS b/NEWS
index f480cb95..32e7361a 100644
--- a/NEWS
+++ b/NEWS
@@ -8,6 +8,10 @@ GNU Bison NEWS
 
   The C++ skeletons now expose copy and move operators for symbols.
 
+  The `cex.timeout` %define variable allows to control when we give up
+  finding a unifying counterexample.  For instance `bison -Wcex
+  -Dcex.timeout=.5 gram.y` to give 1/2s of credit.
+
 * Noteworthy changes in release 3.8.2 (2021-09-25) [stable]
 
   Fixed portability issues of bison on Cygwin.
diff --git a/bootstrap.conf b/bootstrap.conf
index 9a6070ba..fbe89422 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -19,7 +19,7 @@
 gnulib_modules='
   argmatch array-list assert assure attribute
   bitsetv
-  calloc-posix close closeout config-h c-strcase
+  calloc-posix close closeout config-h c-strcase c-strtod
   configmake
   dirname
   error execute extensions
@@ -44,7 +44,7 @@ gnulib_modules='
   realloc-posix
   relocatable-prog relocatable-script
   rename
-  spawn-pipe stdbool stpcpy stpncpy strdup-posix strerror strtod strverscmp
+  spawn-pipe stdbool stpcpy stpncpy strdup-posix strerror strverscmp
   sys_ioctl
   termios
   timevar
diff --git a/doc/bison.texi b/doc/bison.texi
index b2eddcf8..d0074765 100644
--- a/doc/bison.texi
+++ b/doc/bison.texi
@@ -6894,6 +6894,23 @@ @node %define Summary
 @c api.value.union.name
 
 
+@c == cex.timeout
+
+@deffn Directive {%define cex.timeout} @var{duration}
+
+@itemize @bullet
+@item Language(s): all
+
+@item Purpose:
+Define the time credit allocated to finding unifying counterexamples.
+
+@item Accepted Values: duration in seconds, e.g., @samp{1}, @samp{0.5}.
+
+@item Default Value: 5
+@end itemize
+@end deffn
+
+
 @c == lr.default-reduction
 
 @deffn Directive {%define lr.default-reduction} @var{when}
diff --git a/lib/.gitignore b/lib/.gitignore
index 03d09956..9f15868f 100644
--- a/lib/.gitignore
+++ b/lib/.gitignore
@@ -40,6 +40,8 @@
 /c-strcasecmp.c
 /c-strcaseeq.h
 /c-strncasecmp.c
+/c-strtod.c
+/c-strtod.h
 /calloc.c
 /canonicalize-lgpl.c
 /canonicalize.c
diff --git a/m4/.gitignore b/m4/.gitignore
index c3a50c44..4748c6e0 100644
--- a/m4/.gitignore
+++ b/m4/.gitignore
@@ -7,6 +7,7 @@
 /asm-underscore.m4
 /assert.m4
 /builtin-expect.m4
+/c-strtod.m4
 /calloc.m4
 /canonicalize.m4
 /chdir-long.m4
@@ -83,6 +84,7 @@
 /include_next.m4
 /inline.m4
 /intdiv0.m4
+/intl-thread-locale.m4
 /intl.m4
 /intldir.m4
 /intlmacosx.m4
diff --git a/src/counterexample.c b/src/counterexample.c
index 71436fd6..8a4f8b2c 100644
--- a/src/counterexample.c
+++ b/src/counterexample.c
@@ -23,7 +23,9 @@
 
 #include "system.h"
 
+#include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -39,6 +41,7 @@
 #include "gram.h"
 #include "lalr.h"
 #include "lssi.h"
+#include "muscle-tab.h"
 #include "nullable.h"
 #include "parse-simulation.h"
 
@@ -1115,7 +1118,7 @@ unifying_example (state_item_number itm1,
  (Hash_comparator) visited_comparator,
  (Hash_data_freer) search_state_free);
   ssb_append (initial);
-  time_t start = time (NULL);
+  xtime_t start = gethrxtime ();
   bool assurance_printed = false;
   search_state *stage3result = NULL;
   counterexample *cex = NULL;
@@ -1160,7 +1163,7 @@ unifying_example (state_item_number itm1,
 }
   if (TIME_LIMIT_ENFORCED)
 {
-  double time_passed = difftime (time (NULL), start);
+  double time_passed = (gethrxtime () - start) / 1e9;
   if (!assurance_printed && time_passed > ASSURANCE_LIMIT
   && stage3result)
 {
@

Re: Minor documentation and message issues

[Akim Demaille]

Hi Jacob and Frank,

> Le 25 juil. 2022 à 17:15, Jacob L. Mandelson  a écrit 
> :
> 
> On Mon, Jul 25, 2022 at 03:44:50PM +0200, Frank Heckenbach wrote:
>> [...]
>> Another thing, when running with "-Wcounterexamples", I sometimes
>> get messages like this:
>> 
>>  Productions leading up to the conflict state found.  Still finding a 
>> possible unifying counterexample...time limit exceeded: 6,00
>> 
>> Maybe it's because I'm not a native English speaker, but I've always
>> found the usage of "find" in the meaning of "search" confusing, but
>> especially here, as it ultimately does not find anything.
> 
> 
> Yes, "find" does denote reaching whatever the search sought, but the
> imperfect sense in general denotes non-completion, hence does not
> denote success.  Eg, we say that the accident occurred while Alice was
> crossing the street and Bob was driving to the store, even though neither
> action completed.  That said, using a term that specifically denotes
> an in-progress search probably would be clearer.  The verb for that is
> "seek", but I get the feeling that "seeking" is decreasing in modern
> usage as redundant with "searching".

I'll install this.  Thanks!

commit 290d3d09d440a7fc699e9553b0357ec900726809
Author: Akim Demaille 
Date:   Sat Sep 10 14:53:39 2022 +0200

cex: clarify message

Reported by Frank Heckenbach.
https://lists.gnu.org/r/bug-bison/2022-07/msg7.html

src/counterexample.c (unifying_example): Prefer "searching for" to
"finding", which is somewhat confusing.

diff --git a/src/counterexample.c b/src/counterexample.c
index 6ce72c9a..71436fd6 100644
--- a/src/counterexample.c
+++ b/src/counterexample.c
@@ -1165,7 +1165,7 @@ unifying_example (state_item_number itm1,
   && stage3result)
 {
   fputs ("Productions leading up to the conflict state found.  
"
- "Still finding a possible unifying counterexample...",
+ "Still searching for a possible unifying 
counterexample...",
  stderr);
   assurance_printed = true;
 }

Re: Minor documentation and message issues

[Akim Demaille]

Hi Frank,

> Le 25 juil. 2022 à 18:18, Frank Heckenbach  a écrit :
> 
> I wrote:
> 
>> The manual states:
>> 
>>  The syntax of the various directives to declare symbols is as follows.
>> 
>>  @example
>>  %token @var{tag}? ( @var{id} @var{number}? @var{string}? )+ ( @var{tag} ( 
>> @var{id} @var{number}? @var{string}? )+ )*
>>  %left  @var{tag}? ( @var{id} @var{number}?)+ ( @var{tag} ( @var{id} 
>> @var{number}? )+ )*
>>  %type  @var{tag}? ( @var{id} | @var{char} | @var{string} )+ ( @var{tag} ( 
>> @var{id} | @var{char} | @var{string} )+ )*
>>  %nterm @var{tag}? @var{id}+ ( @var{tag} @var{id}+ )*
>>  @end example
>> 
>> I think the "@var{char}" option is missing in the "%token" and
>> "%left" cases.
> 
> Also "@var{string}" in "%left".

Thanks!  I had never realized we could write `%token '+'`.  I also didn't know 
we could give a string alias to a char-token.  But I'll leave it as is.

I think this is right now, do you agree?

commit 374cf8bbafea10914b15df8110fb527383095d2f
Author: Akim Demaille 
Date:   Sat Sep 10 14:48:24 2022 +0200

doc: fix the description of the syntax of %token and %left

Reported by Frank Heckenbach.
https://lists.gnu.org/r/bug-bison/2022-07/msg7.html

* doc/bison.texi (Symbol Decls): here.

diff --git a/doc/bison.texi b/doc/bison.texi
index f4ee13e1..a4bc037e 100644
--- a/doc/bison.texi
+++ b/doc/bison.texi
@@ -5318,15 +5318,27 @@ @node Symbol Decls
 The syntax of the various directives to declare symbols is as follows.
 
 @example
-%token @var{tag}? ( @var{id} @var{number}? @var{string}? )+ ( @var{tag} ( 
@var{id} @var{number}? @var{string}? )+ )*
-%left  @var{tag}? ( @var{id} @var{number}?)+ ( @var{tag} ( @var{id} 
@var{number}? )+ )*
-%type  @var{tag}? ( @var{id} | @var{char} | @var{string} )+ ( @var{tag} ( 
@var{id} | @var{char} | @var{string} )+ )*
-%nterm @var{tag}? @var{id}+ ( @var{tag} @var{id}+ )*
+@group
+%token @var{tag}? ( (@var{id}|@var{char}) @var{number}? @var{string}? )+ \
+ ( @var{tag}  ( (@var{id}|@var{char}) @var{number}? @var{string}? )+ )*
+@end group
+@group
+%left  @var{tag}? ( (@var{id}|@var{char}|@var{string}) @var{number}? )+ \
+ ( @var{tag}  ( (@var{id}|@var{char}|@var{string}) @var{number}? )+ )*
+@end group
+@group
+%type  @var{tag}? (@var{id}|@var{char}|@var{string})+ \
+ ( @var{tag}  (@var{id}|@var{char}|@var{string})+ )*
+@end group
+@group
+%nterm @var{tag}? @var{id}+ \
+ ( @var{tag}  @var{id}+ )*
+@end group
 @end example
 
 @noindent
 where @var{tag} denotes a type tag such as @samp{}, @var{id} denotes
-an identifier such as @samp{NUM}, @var{number} a decimal or hexadecimal
+an identifier such as @samp{NUM} or @samp{exp}, @var{number} a decimal or 
hexadecimal
 integer such as @samp{300} or @samp{0x12d}, @var{char} a character literal
 such as @samp{'+'}, and @var{string} a string literal such as
 @samp{"number"}.  The postfix quantifiers are @samp{?} (zero or one),

Re: Enhancement Request

[Akim Demaille]

Hi,

Frank and Kaz developed very good points.  Let me add a few more.

> Le 10 sept. 2022 à 07:40, Kaz Kylheku  a écrit :
> 
> When you think about it, a tool like Yacc should have a simple specification: 
> most of the code snippets in the second language should appear in the output 
> in the same order as they appear in the grammar file.

This sounds natural, but stems on a couple of issues.

First, there is not always one single answer.  Since Bison also generates the 
header file, in one way or another the developper must be clear on what is 
public and should go there, or remain private and stay in the .c file.  
Dependencies won't answer the question here.  It's a design decision that has 
to made, and specified, by the developper.

Second, YACC tries to be fairly liberal wrt order.  For instance

%token  INT
%union {
  int ival;
  float fval;
}
%token  FLOAT

the order here is irrelevant and it is not required from the developper to 
stick to an order where definitions always precede uses.  So the order of the 
directives is fairly meaningless.

Finally, a feature that has been asked for several times is a form of 
modularity where you would be allowed to %import grammar bits from elsewhere.  
In this case, of concept of "order" is even less clear.  (This feature is not 
under development, but I still think about it.  As a matter of fact, some 
features (e.g., split %union) were implemented precisely to facilitate the 
implementation of %import once we go for it.)


So summarize:
- it would be immensely difficult to implement (because of the target languages)
- it would be very costly to run
- it would be fragile
- it would make build rules incredibly difficult (e.g., this foo.h file that is 
used in the grammar needs to be generated before)
- it's impossible to do independently of the target environment (i.e., we break 
the idea that Bison generates a unique output, target-independent)
- etc.


Really, learning a few %code tags is way simpler.  Not just for us, Bison 
developers, but also for Bison users, and for users of Bison-generated files.

Cheers!

Re: Enhancement Request

[Akim Demaille]

Hi,

> Le 20 juin 2022 à 22:02, slipbits  a écrit :
> 
> Hi Kaz;
> 
> Sorry about taking so long to respond.

Compared to my speed, you are almost relativistic.


> > By tag, do you want to remove the %code  feature?
> > From where?
> 
> It looks like it is possible to remove all s from all %code blocks.
> This is  because the Bison generated statements are completely in control 0f
> Bison and since Bison has knowledge of the working environment, Bison can
> elect to move the contents of the declarations in %code blocks with the aid
> of a  dependency graph.

IIUC, you are suggesting that Bison should guess where to place the various 
code blocks, depending on their content instead of their name/tag.

There's a number of reasons that go against this:

- first and foremost, code blocks are black boxes to Bison.  It does not parse 
them, and never will.  That would require whole (compiler) frontends for all 
the languages we support, which will never happen.

- the "dependencies" might be hidden.  For instance in C, a #include might be a 
provider of something another would be consumer of.  Sure, if we have a 
complete C frontend, the problem is solved :)

- we actually invented named blocks to depart from a model where Bison 
"guessed" where to issue the user's code based on some heuristics.  It was 
confusing, resulted in many users puzzled by drastic changes in the generated 
parser because of apparently minor reordering in the source file.

- being picky is definitely not very novice-friendly, but predictability is far 
more important.  Dumb but clear rules are often superior to fragile magic.

- many of these code blocks are specific to the target language, and baking 
into bison-the-executable the resolution of these dependencies would ruin the 
fact that today people can have their own skeletons, possibly for languages we 
are unaware of.

I'm sure that are other issues.


> What I am saying is that it looks like it is possible.

I don't believe it is.  I can perfectly understand the desire to have a thiner 
documentation, and more automation, but in this precise case, I'm fairly 
confident the implementation cost would be enormous, for a marginal win.


Cheers!

Re: C++ token move constructors/assignment with LALR

[Akim Demaille]

Anthony,

Thanks for the suggestion, the patience, and for the patch.  I'm installing the 
following edited version of your commit.  It is "trivial" enough to not need 
FSF paperwork.

Frank:

> BTW, when checking this, I noticed there's a "void move" function
> declared (actually 3 of them for different classes) in the skeletons
> which I thought were involved here, but there doesn't seem to be any
> definition of them. So these declarations may be remnants, and you
> might want to remove them, Akim.


These functions are not C++ standard, they participate in Bison's effort to 
have some form of move semantics in C98.

Cheers!

commit eb83487606276fdefd20415176aff9309a62a5ee
Author: Anthony Heading 
Date:   Fri Apr 29 04:08:35 2022 -0400

c++: enable C++11 assignment operators for lalr1.cc




* data/skeletons/c++.m4: Enable C++11 assignment operators for
all C++ skeletons.

diff --git a/NEWS b/NEWS
index aabca967..f480cb95 100644
--- a/NEWS
+++ b/NEWS
@@ -6,6 +6,8 @@ GNU Bison NEWS
 
   Fixed spurious warnings about input containing `m4_` or `b4_`.
 
+  The C++ skeletons now expose copy and move operators for symbols.
+
 * Noteworthy changes in release 3.8.2 (2021-09-25) [stable]
 
   Fixed portability issues of bison on Cygwin.
diff --git a/data/skeletons/c++.m4 b/data/skeletons/c++.m4
index 78f71fce..d63d1548 100644
--- a/data/skeletons/c++.m4
+++ b/data/skeletons/c++.m4
@@ -346,7 +346,7 @@ m4_define([b4_symbol_type_define],
 clear ();
   }
 
-]b4_glr2_cc_if([[
+#if 201103L <= YY_CPLUSPLUS
   /// Copy assignment.
   basic_symbol& operator= (const basic_symbol& that)
   {
@@ -368,7 +368,7 @@ m4_define([b4_symbol_type_define],
 location = std::move (that.location);]])[
 return *this;
   }
-]])[
+#endif
 
   /// Destroy contents, and record that is empty.
   void clear () YY_NOEXCEPT
@@ -454,13 +454,13 @@ m4_define([b4_symbol_type_define],
   /// Constructor from (external) token numbers.
   by_kind (kind_type t) YY_NOEXCEPT;
 
-]b4_glr2_cc_if([[
+#if 201103L <= YY_CPLUSPLUS
   /// Copy assignment.
   by_kind& operator= (const by_kind& that);
 
   /// Move assignment.
   by_kind& operator= (by_kind&& that);
-]])[
+#endif
 
   /// Record that this symbol is empty.
   void clear () YY_NOEXCEPT;
@@ -584,7 +584,7 @@ m4_define([b4_public_types_define],
 : kind_ (yytranslate_ (t))
   {}
 
-]b4_glr2_cc_if([[
+#if 201103L <= YY_CPLUSPLUS
   ]b4_inline([$1])]b4_parser_class[::by_kind&
   b4_parser_class[::by_kind::by_kind::operator= (const by_kind& that)
   {
@@ -599,7 +599,7 @@ m4_define([b4_public_types_define],
 that.clear ();
 return *this;
   }
-]])[
+#endif
 
   ]b4_inline([$1])[void
   ]b4_parser_class[::by_kind::clear () YY_NOEXCEPT

Re: ./test-suite.log

[Akim Demaille]

Hi,

> Le 4 août 2022 à 19:02, giorgos pap  a écrit :
> 
> =
>   GNU Bison 3.8: ./test-suite.log
> =
> 
> # TOTAL: 14
> # PASS:  12
> # SKIP:  0
> # XFAIL: 0
> # FAIL:  2
> # 

Thanks for the report.  I'm installing this.  Cheers!

commit ec9c5556d9b4c18dbd3917b44a3a3e5c94670b5e
Author: Akim Demaille 
Date:   Sat Sep 3 09:11:20 2022 +0200

tests: ignore messages from the JVM

Reported by Giorgos Pap.
<https://lists.gnu.org/r/bug-bison/2022-08/msg2.html>

* examples/test: Discard "Picked up _JAVA_OPTIONS:" messages.

diff --git a/THANKS b/THANKS
index 52092368..d9159fa5 100644
--- a/THANKS
+++ b/THANKS
@@ -82,6 +82,7 @@ Gavin Smith   gavinsmith0...@gmail.com
 Georg Sauthoffgsaut...@techfak.uni-bielefeld.de
 George Neuner gneun...@comcast.net
 Gilles Espinasse  g@free.fr
+Giorgos Pap   giorgos551...@gmail.com
 Goran Uddeborggoe...@uddeborg.se
 Guido Trentalanciatrent...@aston.ac.uk
 H. Merijn Brand   h.m.br...@hccnet.nl
diff --git a/examples/test b/examples/test
index 45a5d454..738dcae4 100755
--- a/examples/test
+++ b/examples/test
@@ -137,10 +137,13 @@ run ()
 else
   cat out_eff
 fi
+# Ignore informational messages from the JVM.
 if ! $noerr; then
-   sed -e 's/^/err: /g' \
-   -e 's/Reducing stack by rule .* (line .*):/Reducing stack by rule 
XX (line XXX):/g' \
-   err_eff
+  sed \
+-e '/^Picked up _JAVA_OPTIONS: /d' \
+-e 's/Reducing stack by rule .* (line .*):/Reducing stack by rule XX 
(line XXX):/g' \
+-e 's/^/err: /g' \
+err_eff
 fi
   } >eff
 

Re: Bison submit patches

[Akim Demaille]

Hi Bruno,

Thanks a lot for the analysis, and suggestion.  Thanks to Andrei for the 
report, and to Kaz and Paul for routing them to gnulib.

> Le 23 août 2022 à 22:53, Bruno Haible  a écrit :
> 
> The second patch is against Bison proper, and can be simplified like this:
> 
> diff --git a/src/location.c b/src/location.c
> index 5edce82c..86e7c39d 100644
> --- a/src/location.c
> +++ b/src/location.c
> @@ -268,7 +268,7 @@ caret_set_file (const char *file)
>   if (!caret_info.pos.file)
> {
>   caret_info.pos.file = file;
> -  if ((caret_info.file = fopen (caret_info.pos.file, "r")))
> +  if ((caret_info.file = fopen (caret_info.pos.file, "rb")))
> {
>   /* If the file is not regular (imagine #line 1 "/dev/stdin"
>  in the input file for instance), don't try to quote the
> 
> It rationale is that caret_getc_internal already handles the CR/LF
> newlines from Windows, and therefore opening the file in binary mode
> avoids the horrible kludges of the Microsoft stdio runtime for O_TEXT
> files.

I'm installing this:

commit cfef21f5b0a5c4291dcaa019e287210064371edb
Author: Akim Demaille 
Date:   Sat Sep 3 08:51:17 2022 +0200

diagnostics: Windows compatibility issues

Suggested by Bruno Haible
<https://lists.gnu.org/r/bug-bison/2022-08/msg6.html>
following a report from Andrei Malashkin
<https://lists.gnu.org/r/bug-bison/2022-08/msg3.html>

* src/location.c (caret_set_file): Read the file in binary.
We already deal with CRLF in caret_getc_internal.

diff --git a/THANKS b/THANKS
index 391b847e..52092368 100644
--- a/THANKS
+++ b/THANKS
@@ -14,6 +14,7 @@ Alexandre Duret-Lutz  a...@lrde.epita.fr
 Andre da Costa Barros andre.cbar...@yahoo.com
 Andreas Damm  ad...@onica.com
 Andreas Schwabsch...@suse.de
+Andrei Malashkin  malashkin.and...@gmail.com
 Andrew Suffield   asuffi...@users.sourceforge.net
 Angelo Borsotti   angelo.borso...@gmail.com
 Anthony Heading   a...@ajrh.net
diff --git a/src/location.c b/src/location.c
index 94c77ef6..0f56bd39 100644
--- a/src/location.c
+++ b/src/location.c
@@ -268,13 +268,13 @@ caret_set_file (const char *file)
   if (!caret_info.pos.file)
 {
   caret_info.pos.file = file;
-  if ((caret_info.file = fopen (caret_info.pos.file, "r")))
+  if ((caret_info.file = fopen (caret_info.pos.file, "rb")))
 {
   /* If the file is not regular (imagine #line 1 "/dev/stdin"
  in the input file for instance), don't try to quote the
- file.  Keep caret_info.file set so that we don't try to
- open it again, but leave caret_info.file NULL so that we
- don't try to quote it. */
+ file.  Keep caret_info.pos.file set so that we don't try
+ to open it again, but leave caret_info.file NULL so that
+ we don't try to quote it. */
   struct stat buf;
   if (fstat (fileno (caret_info.file), &buf) == 0
   && buf.st_mode & S_IFREG)

Re: PATCH: make sub-includes easier with custom skeletons

[Akim Demaille]

Hi Antony,

> Le 3 juil. 2022 à 23:15, Anthony Heading  a écrit :
> 
> Another patch I've had lying around for a while...   It's easy to use a 
> custom skeleton, just by using the '-S ' flag.   But it's harder to 
> customize their sub-includes because they have fixed paths, for example in 
> lalr1.cc:
> 
> m4_include(b4_skeletonsdir/[c++.m4])
> 
> Attached patch sets the directory of any custom skeleton, and then the 
> b4_skeletonsdir,  as '-I' search-path flags to m4, and makes the c++.m4 
> include path unqualified.
> 
> If it makes sense for adoption,  I can change all the various other system 
> skeleton files similarly and submit a larger patch as github PR,  but 
> hopefully attached (which is all I personally use) is enough to explain the 
> idea.

Could you elaborate a bit your use case?

Are you using a regular lalr1.cc but a tuned c++.m4?

I'm not sure I want to support that: Bison's lalr1.cc wants its own c++.m4, etc.

However I can understand that when you add -Smyskeletons/lalr1.cc, then you 
want lalr1.cc to be free to depend on other files in myskeletons/.  However, 
instead of hard-coding this automatic -I on the base name of -S, I would rather 
truly introduce -I and let the user decide what to use.

With your -S, myskeletons/lalr1.cc would have to include `c++.m4`.  I think it 
should include `myskeletons/lalr1.cc`.  That's why I would prefer -I: the user 
decides what the root is.

Re: Gnulib-related problems building Bison from git

[Akim Demaille]

Hi Bruno,

> Le 13 juil. 2022 à 08:41, Akim Demaille  a écrit :
> 
> What do you think about this?  Can I install it?

I pushed it.  Bison does not build successfully without libtextstyle without 
that.

I've installed the following patch in Bison.

Cheers!

commit c70b68c60d821f200bd3640b236437eae7085b76
Author: Akim Demaille 
Date:   Sun Jul 31 11:05:57 2022 +0200

gnulib: update

We now build cleanly even if libtextstyle is not available.
See https://lists.gnu.org/r/bug-bison/2022-07/msg2.html.

diff --git a/bootstrap.conf b/bootstrap.conf
index 6e9dd1cd..d33e9c8c 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -79,7 +79,7 @@ XGETTEXT_OPTIONS_RUNTIME=$XGETTEXT_OPTIONS'\\\
  --keyword=YY_ \\\
 '
 
-gnulib_tool_option_extras='--symlink --conditional-dependencies 
--makefile-name=gnulib.mk --automake-subdir --po-base=gnulib-po 
--po-domain=bison'
+gnulib_tool_option_extras='--symlink --conditional-dependencies 
--makefile-name=gnulib.mk --automake-subdir --automake-subdir-tests 
--po-base=gnulib-po --po-domain=bison'
 
 bootstrap_post_import_hook()
 {
diff --git a/gnulib b/gnulib
index 1803e3eb..76c7703c 16
--- a/gnulib
+++ b/gnulib
@@ -1 +1 @@
-Subproject commit 1803e3ebe5e956dda2d3483bc16b88f57f66a4df
+Subproject commit 76c7703cb2e9e0e803d1296618d8ab9e86e13d6c

Re: Gnulib-related problems building Bison from git

[Akim Demaille]

Hi Bruno,

> Le 6 juil. 2022 à 07:57, Akim Demaille  a écrit :
> 
> commit 813e5a1787ed156c70bd6d4bba39a8b2db4916db
> Author: Akim Demaille 
> Date:   Mon Jul 4 07:18:07 2022 +0200
> 
>gnulib-tool: add support for --automake-subdir-tests
> 
><https://lists.gnu.org/r/bug-gnulib/2022-01/msg00111.html>
> 
>* gnulib-tool (main): Handle --automake-subdir-tests.
>(func_emit_shellvars_init, func_emit_lib_Makefile_am): Use
>$sourcebase when handling tests and --automake-subdir-tests is
>given.
>(func_append_actionarg): Support --automake-subdir-tests.
>(func_create_testdir): Add missing argument for func_emit_initmacro_end.

What do you think about this?  Can I install it?

Re: Gnulib-related problems building Bison from git

[Akim Demaille]

> Le 6 juil. 2022 à 07:53, Akim Demaille  a écrit :
> 
> Here's my updated proposal.

I had forgotten to s/test/tests/ in the commit message, sorry.


commit 813e5a1787ed156c70bd6d4bba39a8b2db4916db
Author: Akim Demaille 
Date:   Mon Jul 4 07:18:07 2022 +0200

gnulib-tool: add support for --automake-subdir-tests

<https://lists.gnu.org/r/bug-gnulib/2022-01/msg00111.html>

* gnulib-tool (main): Handle --automake-subdir-tests.
(func_emit_shellvars_init, func_emit_lib_Makefile_am): Use
$sourcebase when handling tests and --automake-subdir-tests is
given.
(func_append_actionarg): Support --automake-subdir-tests.
(func_create_testdir): Add missing argument for func_emit_initmacro_end.

diff --git a/ChangeLog b/ChangeLog
index 8694db8900..fb7340e5ea 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,14 @@
+2022-07-06  Akim Demaille  
+
+   gnulib-tool: add support for --automake-subdir-tests
+   <https://lists.gnu.org/r/bug-gnulib/2022-01/msg00111.html>
+   * gnulib-tool (main): Handle --automake-subdir-tests.
+   (func_emit_shellvars_init, func_emit_lib_Makefile_am): Use
+   $sourcebase when handling tests and --automake-subdir-tests is
+   given.
+   (func_append_actionarg): Support --automake-subdir-tests.
+   (func_create_testdir): Add missing argument for func_emit_initmacro_end.
+
 2022-07-03  Bruno Haible  
 
lib-symbol-visibility: Improve documentation.
diff --git a/gnulib-tool b/gnulib-tool
index 5993143f3c..21aa6c4fab 100755
--- a/gnulib-tool
+++ b/gnulib-tool
@@ -311,6 +311,8 @@ Options for --import, --add/remove-import:
   --automake-subdir Specify that the makefile in the source-base
 directory be generated in such a way that it can
 be 'include'd from the toplevel Makefile.am.
+  --automake-subdir-tests
+Likewise, but for the tests directory.
   --macro-prefix=PREFIX  Specify the prefix of the macros 'gl_EARLY' and
 'gl_INIT'. Default is 'gl'.
   --po-domain=NAME  Specify the prefix of the i18n domain. Usually use
@@ -1118,7 +1120,8 @@ func_determine_path_separator
 # - gnu_maketrue if --gnu-make was given, false otherwise
 # - makefile_name   from --makefile-name
 # - tests_makefile_name  from --tests-makefile-name
-# - automake_subdir  true if --automake-subdir was given, false otherwise
+# - automake_subdirtrue if --automake-subdir was given, false otherwise
+# - automake_subdir_tests  true if --automake-subdir-tests was given, false 
otherwise
 # - libtool true if --libtool was given, false if --no-libtool was
 #   given, blank otherwise
 # - macro_prefixfrom --macro-prefix
@@ -1167,6 +1170,7 @@ func_determine_path_separator
   makefile_name=
   tests_makefile_name=
   automake_subdir=false
+  automake_subdir_tests=false
   libtool=
   macro_prefix=
   po_domain=
@@ -1414,6 +1418,9 @@ func_determine_path_separator
   --automake-subdir )
 automake_subdir=true
 shift ;;
+  --automake-subdir-tests )
+automake_subdir_tests=true
+shift ;;
   --libtool )
 libtool=true
 shift ;;
@@ -1527,6 +1534,7 @@ func_determine_path_separator
|| test -n "$excl_privileged_tests" || test -n "$excl_unportable_tests" 
\
|| test -n "$avoidlist" || test -n "$lgpl" || test -n "$makefile_name" \
|| test -n "$tests_makefile_name" || test "$automake_subdir" != false \
+   || test "$automake_subdir_tests" != false \
|| test -n "$macro_prefix" || test -n "$po_domain" \
|| test -n "$witness_c_macro" || test -n "$vc_files"; then
   echo "gnulib-tool: invalid options for 'update' mode" 1>&2
@@ -1617,8 +1625,8 @@ func_determine_path_separator
   func_fatal_error "minimum supported autoconf version is 2.64. Try adding 
AC_PREREQ([$DEFAULT_AUTOCONF_MINVERSION]) to your configure.ac." ;;
   esac
 
-  # Determine whether --automake-subdir is supported.
-  if $automake_subdir; then
+  # Determine whether --automake-subdir/--automake-subdir-tests are supported.
+  if $automake_subdir || $automake_subdir_tests; then
 found_subdir_objects=false
 if test -n "$configure_ac"; then
   my_sed_traces='
@@ -1644,7 +1652,7 @@ func_determine_path_separator
   done
 fi
 if ! $found_subdir_objects; then
-  func_fatal_error "Option --automake-subdir is only supported if the 
definition of AUTOMAKE_OPTIONS in Makefile.am contains 'subdir-objects'."
+  func_fatal_error "Option --automake-subdir/--automake-subdir-tests are 
only supported if the definition of AUTOMAKE_OPTIONS in Ma

Re: Gnulib-related problems building Bison from git

[Akim Demaille]

Hi Bruno,

Thanks for the feedback!

> Le 4 juil. 2022 à 23:31, Bruno Haible  a écrit :
> 
>> @@ -4518,7 +4526,9 @@ func_emit_initmacro_end ()
>>   echo "  sed_dirname4='s,\\(.\\)/[^/]*\$,\\1,'"
>>   echo "  sed_basename1='s,.*/,,'"
>>   echo "changequote([, ])dnl"
>> -  if $automake_subdir && ! "$2" && test -n "$sourcebase" && test 
>> "$sourcebase" != '.'; then
>> +  if (($2 && $automake_subdir_test) || (! $2 && $automake_subdir)) \
>> + && test -n "$sourcebase" \
>> + && test "$sourcebase" != '.'; then
>> subdir="$sourcebase/"
>>   else
>> subdir=
> 
> Three things are wrong here:
>  - If $2, subdir should be set to "$testsbase/", not "$sourcebase/".

That was unclear to me, I expected to put everything in lib/, but
it does work with testsbase, thanks!

>  - Parentheses introduce a subshell, which is more expensive to execute
>than a compound shell command. IOW,
>  { command1 && command2; } || { command3 && command4; }
>is more efficient than
>  (command1 && command2) || (command3 && command4)

Sure...  I'm getting rusty.

>  - Some bash versions interpret '((' as the start of an evaluation
>command. To please these shells, you would have to add a space,
>i.e. write   ( (   instead of   ((

I was unaware of this.  Actually I should not have used (...), so
I shouldn't have needed to know :)

> I don't believe that the unconditional reference to $sourcebase is
> correct here. Look at the various invocations of func_emit_shellvars_init;
> you need to accommodate all different cases.

You are right, I had overlooked other calls.

I do not understand all the details.  Here's my updated proposal.

Cheers.

commit 18d46e1a0ef90bd8425db984d8cf811e7d690394
Author: Akim Demaille 
Date:   Mon Jul 4 07:18:07 2022 +0200

gnulib-tool: add support for --automake-subdir-tests

<https://lists.gnu.org/r/bug-gnulib/2022-01/msg00111.html>

* gnulib-tool (main): Handle --automake-subdir-test.
(func_emit_shellvars_init, func_emit_lib_Makefile_am): Use
$sourcebase when handling tests and --automake-subdir-test is
given.
(func_append_actionarg): Support --automake-subdir-test.
(func_create_testdir): Add missing argument for func_emit_initmacro_end.

diff --git a/ChangeLog b/ChangeLog
index 8694db8900..1130ffd6cb 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,14 @@
+2022-07-06  Akim Demaille  
+
+   gnulib-tool: add support for --automake-subdir-test
+   <https://lists.gnu.org/r/bug-gnulib/2022-01/msg00111.html>
+   * gnulib-tool (main): Handle --automake-subdir-test.
+   (func_emit_shellvars_init, func_emit_lib_Makefile_am): Use
+   $sourcebase when handling tests and --automake-subdir-test is
+   given.
+   (func_append_actionarg): Support --automake-subdir-test.
+   (func_create_testdir): Add missing argument for func_emit_initmacro_end.
+
 2022-07-03  Bruno Haible  
 
lib-symbol-visibility: Improve documentation.
diff --git a/gnulib-tool b/gnulib-tool
index 5993143f3c..21aa6c4fab 100755
--- a/gnulib-tool
+++ b/gnulib-tool
@@ -311,6 +311,8 @@ Options for --import, --add/remove-import:
   --automake-subdir Specify that the makefile in the source-base
 directory be generated in such a way that it can
 be 'include'd from the toplevel Makefile.am.
+  --automake-subdir-tests
+Likewise, but for the tests directory.
   --macro-prefix=PREFIX  Specify the prefix of the macros 'gl_EARLY' and
 'gl_INIT'. Default is 'gl'.
   --po-domain=NAME  Specify the prefix of the i18n domain. Usually use
@@ -1118,7 +1120,8 @@ func_determine_path_separator
 # - gnu_maketrue if --gnu-make was given, false otherwise
 # - makefile_name   from --makefile-name
 # - tests_makefile_name  from --tests-makefile-name
-# - automake_subdir  true if --automake-subdir was given, false otherwise
+# - automake_subdirtrue if --automake-subdir was given, false otherwise
+# - automake_subdir_tests  true if --automake-subdir-tests was given, false 
otherwise
 # - libtool true if --libtool was given, false if --no-libtool was
 #   given, blank otherwise
 # - macro_prefixfrom --macro-prefix
@@ -1167,6 +1170,7 @@ func_determine_path_separator
   makefile_name=
   tests_makefile_name=
   automake_subdir=false
+  automake_subdir_tests=false
   libtool=
   macro_prefix=
   po_domain=
@@ -1414,6 +1418,9 @@ func_determine_path_sep

Re: Gnulib-related problems building Bison from git

[Akim Demaille]

Hi,

Since Bison was moved to use gnulib's support for --automake-subdir, the 
configuration fails for people who don't have libtextstyle installed.  Or 
distcheck fails when libtextstyle is not installed in a default include path.

This is because --automake-subdir is built (apparently) for projects that use a 
single Makefile for the sources, but another for the tests.  This is not the 
case of Bison, where we have a single Makefile for the whole project.

In <https://lists.gnu.org/r/bug-gnulib/2022-01/msg00156.html> I had asked what 
was the preferred fix.  Below, I chose to add another option, 
--automake-subdir-test to signal cases such as Bison's.

If this is validated, I'll complete it with the doc parts.

Cheers!

commit b781a18296d4d5df0325f6618d296b8bac6e6b17
Author: Akim Demaille 
Date:   Mon Jul 4 07:18:07 2022 +0200

gnulib-tool: add support for --automake-subdir-test

<https://lists.gnu.org/r/bug-gnulib/2022-01/msg00111.html>
* gnulib-tool (main): Handle --automake-subdir-test.
(func_emit_shellvars_init, func_emit_lib_Makefile_am): Use
$sourcebase when handling tests and --automake-subdir-test is
given.
(func_append_actionarg): Support --automake-subdir-test.

diff --git a/ChangeLog b/ChangeLog
index 2daa6d8c81..ddb8c8523e 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,13 @@
+2022-07-04  Akim Demaille  
+
+   gnulib-tool: add support for --automake-subdir-test
+   <https://lists.gnu.org/r/bug-gnulib/2022-01/msg00111.html>
+   * gnulib-tool (main): Handle --automake-subdir-test.
+   (func_emit_shellvars_init, func_emit_lib_Makefile_am): Use
+   $sourcebase when handling tests and --automake-subdir-test is
+   given.
+   (func_append_actionarg): Support --automake-subdir-test.
+
 2022-06-12  Paul Eggert  
 
fchmodat: port better to MS-Windows etc.
diff --git a/gnulib-tool b/gnulib-tool
index 5993143f3c..0ba6951cd7 100755
--- a/gnulib-tool
+++ b/gnulib-tool
@@ -311,6 +311,8 @@ Options for --import, --add/remove-import:
   --automake-subdir Specify that the makefile in the source-base
 directory be generated in such a way that it can
 be 'include'd from the toplevel Makefile.am.
+  --automake-subdir-test
+Likewise, but for the test directory.
   --macro-prefix=PREFIX  Specify the prefix of the macros 'gl_EARLY' and
 'gl_INIT'. Default is 'gl'.
   --po-domain=NAME  Specify the prefix of the i18n domain. Usually use
@@ -1118,7 +1120,8 @@ func_determine_path_separator
 # - gnu_maketrue if --gnu-make was given, false otherwise
 # - makefile_name   from --makefile-name
 # - tests_makefile_name  from --tests-makefile-name
-# - automake_subdir  true if --automake-subdir was given, false otherwise
+# - automake_subdir   true if --automake-subdir was given, false otherwise
+# - automake_subdir_test  true if --automake-subdir-test was given, false 
otherwise
 # - libtool true if --libtool was given, false if --no-libtool was
 #   given, blank otherwise
 # - macro_prefixfrom --macro-prefix
@@ -1167,6 +1170,7 @@ func_determine_path_separator
   makefile_name=
   tests_makefile_name=
   automake_subdir=false
+  automake_subdir_test=false
   libtool=
   macro_prefix=
   po_domain=
@@ -1414,6 +1418,9 @@ func_determine_path_separator
   --automake-subdir )
 automake_subdir=true
 shift ;;
+  --automake-subdir-test )
+automake_subdir_test=true
+shift ;;
   --libtool )
 libtool=true
 shift ;;
@@ -1527,6 +1534,7 @@ func_determine_path_separator
|| test -n "$excl_privileged_tests" || test -n "$excl_unportable_tests" 
\
|| test -n "$avoidlist" || test -n "$lgpl" || test -n "$makefile_name" \
|| test -n "$tests_makefile_name" || test "$automake_subdir" != false \
+   || test "$automake_subdir_test" != false \
|| test -n "$macro_prefix" || test -n "$po_domain" \
|| test -n "$witness_c_macro" || test -n "$vc_files"; then
   echo "gnulib-tool: invalid options for 'update' mode" 1>&2
@@ -1617,8 +1625,8 @@ func_determine_path_separator
   func_fatal_error "minimum supported autoconf version is 2.64. Try adding 
AC_PREREQ([$DEFAULT_AUTOCONF_MINVERSION]) to your configure.ac." ;;
   esac
 
-  # Determine whether --automake-subdir is supported.
-  if $automake_subdir; then
+  # Determine whether --automake-subdir/--automake-subdir-test are supported.
+  if $automake_subdir || $automake_subdir_test; then
 found_subdir_objects=false
 if test -n "$configure_ac"; then
   my_sed_traces='
@@ -1644,7 +1652,7 @@ func_determ

Re: Gnulib-related problems building Bison from git

[Akim Demaille]

Hi Paul,

> Le 15 janv. 2022 à 20:51, Paul Eggert  a écrit :
> 
> I can't build Bison from Savannah git. See attached transcript, which ends:
> 
>  GEN  lib/inttypes.h
> make: *** No rule to make target 'textstyle.h', needed by 'all'.  Stop.
> 
> The Makefile (also attached) has a rule to build lib/textstyle.h. However, it 
> says "TEXTSTYLE_H = textstyle.h" and "BUILT_SOURCES = ... $(TEXTSTYLE_H) ..." 
> and since $(BUILT_SOURCES) is a prerequisite for "all" the build fails.
> 
> I am building on Ubuntu 21.10, except with bleeding edge Autoconf 
> (2.72a.22-6027-dirty) because Bison requires at least Autoconf 2.71.

I can reproduce this, once I uninstalled libtextstyle.

The problem is my call to

> # We want ostream_printf and hyperlink support.
> gl_LIBTEXTSTYLE_OPTIONAL([0.20.5])

in configure.ac.  First, it was too early (right after gl_EARLY) and it 
generated

> TEXTSTYLE_H="${gl_source_base_prefix}textstyle.h"

before the definition of gl_source_base_prefix.


If I move it after gl_INIT, I get:

> $ grep gl_source_base_prefix configure
>   gl_source_base_prefix='$(top_build_prefix)lib/'
> ALLOCA_H="${gl_source_base_prefix}alloca.h"
> ERRNO_H="${gl_source_base_prefix}errno.h"
> FLOAT_H="${gl_source_base_prefix}float.h"
> GETOPT_H="${gl_source_base_prefix}getopt.h"
> GETOPT_CDEFS_H="${gl_source_base_prefix}getopt-cdefs.h"
> ICONV_H="${gl_source_base_prefix}iconv.h"
> ICONV_H="${gl_source_base_prefix}iconv.h"
> TEXTSTYLE_H="${gl_source_base_prefix}textstyle.h"
> LIMITS_H="${gl_source_base_prefix}limits.h"
> STDALIGN_H="${gl_source_base_prefix}stdalign.h"
> STDBOOL_H="${gl_source_base_prefix}stdbool.h"
> STDDEF_H="${gl_source_base_prefix}stddef.h"
> STDINT_H="${gl_source_base_prefix}stdint.h"
> LIMITS_H="${gl_source_base_prefix}limits.h"
>   LIBUNISTRING_UNISTR_H="${gl_source_base_prefix}unistr.h"
>   LIBUNISTRING_UNITYPES_H="${gl_source_base_prefix}unitypes.h"
>   LIBUNISTRING_UNIWIDTH_H="${gl_source_base_prefix}uniwidth.h"
>   gl_source_base_prefix=
> TEXTSTYLE_H="${gl_source_base_prefix}textstyle.h"

Both assignments to gl_source_base_prefix are baked in gl_INIT.  This second 
assignment is meant for the tests, it is from

>   gl_COMMON
>   gl_source_base='tests'
>   gl_source_base_prefix=

This is probably affecting all the modules for which the user is expected to 
call the macro by hand: the variables will not be set as expected.

I don't know what is the preferred fix here.  Bruno?

Re: implicit declaration of function 'yylex' is invalid in C99

[Akim Demaille]

Ryan,

> Le 15 janv. 2022 à 11:04, Ryan Schmidt  a écrit :
> 
> On Jan 15, 2022, at 03:57, Akim Demaille wrote:
> 
>> I personally feel that generating these prototypes automatically is actually 
>> more complex (for the user, not for Bison) than merely specifying them.
> 
> Ok. Can you provide any guidance for how to specify them?
> 
> I am a maintainer of packages within the MacPorts package management system.

And I thank you a lot for the effort you put into the MacPorts.

> I've encountered many of our ports that fail to build for this reason, and I 
> haven't known how to fix them. For other implicit declaration of function 
> problems it's usually been a simple matter of including a missing header or 
> writing a function prototype, but with bison/flex, there isn't a .c or .h 
> file into which such a prototype could be inserted, and even if I knew what 
> file to insert it into, I wouldn't know what prototype to write.

It very much depends on the *.l and the *.y files.  But most of the time it 
suffices to include something like

%code provides {
  int yylex (void);
  void yyerror (const char *);
}

in the first part of the *.y file (before %%).

If you provide me with pointers to these packages, I can help finding the 
appropriate fixes.

Re: implicit declaration of function 'yylex' is invalid in C99

[Akim Demaille]

Hi Ryan,

> Le 14 janv. 2022 à 10:52, Ryan Schmidt  a écrit :
> 
> Hi,
> 
> On macOS 10.15 or later with bison 3.8.2, flex 2.6.4, and Xcode 12 or later, 
> when software is built that uses bison/flex, I often see this type of failure:
> 
> 
> error: implicit declaration of function 'yylex' is invalid in C99 
> [-Werror,-Wimplicit-function-declaration]
>  yychar = YYLEX;
>   ^
> note: expanded from macro 'YYLEX'
> # define YYLEX yylex ()
>   ^
> 
> 
> As of Xcode 12, implicit declaration of function is an error. You must 
> provide a function prototype before you call a function.
> 
> A quick search shows many people encountering this problem:
> 
> https://stackoverflow.com/questions/69986502/i-get-some-error-when-i-try-to-complie-yacc-lex-file
> https://stackoverflow.com/questions/23717039/generating-a-compiler-from-lex-and-yacc-grammar
> https://trac.macports.org/ticket/62015
> https://trac.macports.org/ticket/62632
> 
> It sounds very much like the problem reported to this mailing list in 2005:
> 
> https://lists.gnu.org/archive/html/bug-bison/2005-05/msg00055.html
> 
> A fix was proposed there as well. Does anyone know if that proposal was ever 
> accepted? It's difficult for me to determine because I'm not familiar with 
> the bison source code and it has probably evolved significantly since 2005.

I've explained why this is a bad idea (IMHO) in 
https://austingroupbugs.net/view.php?id=1388#c5206.

There are several reasons.  Let's focus on yylex.

First of all, why the heck would the *consumer* (Bison generated parser) have 
the specify that instead of the *provider* (for instance Flex generated 
scanner, or whatever the other tools)?  It feels not right.

Second of all, there are several possible prototypes, depending the user's 
requirements.  What return type are you using to code tokens (int, enum)?  Is 
yylex static?  Are its arguments const?  etc.  As a matter of fact, to get 
compliant with a recent change in POSIX Yacc, we (reluctantly) started 
generating these prototypes, which immediately broke existing code that had 
slightly different prototypes 
(https://lists.gnu.org/r/bug-bison/2021-09/msg5.html).  See the NEWS entry 
on this regard:

* Noteworthy changes in release 3.8 (2021-09-07) [stable]

** Backward incompatible changes

  To comply with the latest POSIX standard, in Yacc compatibility mode
  (options `-y`/`--yacc`) Bison now generates prototypes for yyerror and
  yylex.  In some situations, this is breaking compatibility: if the user
  has already declared these functions but with some differences (e.g., to
  declare them as static, or to use specific attributes), the generated
  parser will fail to compile.  To disable these prototypes, #define yyerror
  (to `yyerror`), and likewise for yylex.

* Noteworthy changes in release 3.8.1 (2021-09-11) [stable]

  The generation of prototypes for yylex and yyerror in Yacc mode is
  breaking existing grammar files.  To avoid breaking too many grammars, the
  prototypes are now generated when `-y/--yacc` is used *and* the
  `POSIXLY_CORRECT` environment variable is defined.

  Avoid using `-y`/`--yacc` simply to comply with Yacc's file name
  conventions, rather, use `-o y.tab.c`.  Autoconf's AC_PROG_YACC macro uses
  `-y`.  Avoid it if possible, for instance by using gnulib's gl_PROG_BISON.

I personally feel that generating these prototypes automatically is actually 
more complex (for the user, not for Bison) than merely specifying them.

Cheers!

Re: Bison good on Ubuntu 8.04

[Akim Demaille]

Hi Jeffrey,

> Le 11 janv. 2022 à 06:09, Jeffrey Walton  a écrit :
> 
> Hi Everyone,
> 
> An FYI... I'm in a thorny position where I need to work on some
> antique hardware. I needed to update some tools.
> 
> My build script was a bit stale, and Bison 3.7.6 had some problems. I
> needed to bump Bison versions.
> 
> I'm happy to report Bison 3.8.2 built and passed its self tests on Ubuntu 
> 8.04.

Excellent news :)  Thanks for reporting it!

Re: RFC: "count_chars" function for locations

[Akim Demaille]

Hi Frank!

> Le 11 janv. 2022 à 01:51, Frank Heckenbach  a écrit :
> 
> Akim Demaille wrote:
> 
>>>   void count_chars (const char *s, size_t n)
>> 
>> I understand the need, and agree with the proposal.  However I'm
>> concerned by the fact that this gives a meaning to 'columns'.  Currently,
>> it is unspecified: it could be just bytes, or actual characters,
> 
> ... or character width (as Bison uses now in its own scanner).

Indeed.

> Then again, the simple form as given in calc++/scanner.ll (also in
> the manual) has the same issues:
> 
>  # define YY_USER_ACTION  loc.columns (yyleng);
> 
> Actually, I used something like this in my own code until I had
> multi-line tokens, when I extended it to support newlines, unaware
> of these other issues.
> 
> You might want to add at least a comment about these issues in this
> and other examples, in the manual, and in location.cc.

I'm not sure what and where to write in location.cc.  But how about
the appended patch?


> BTW, when I added tab handling to my function, I noticed a possible
> issue with the position interface. I had to do something like this:
> 
>  case '\t':
>end.column += tab_width - end.column % tab_width;
>break;
> 
> since the columns() function can only add columns, not query the
> current column as needed to compute tabs. It's not a real problem
> since column is a public field, but it feels a bit like breaking the
> abstraction. You may or may not care about this (just please don't
> make column non-public in the future :).

Nah, it's public on purpose.  I don't plan in over-engineering
this class :)


>> Maybe something like %code location and %code position.  WDYT?  One
>> issue would then be that if someone introduced something new in position
>> or location, and we later add the same name, we would break the
>> user's code :(
> 
> If I understand you right, this would only enable one to put one's
> own implementation of such a function in the parser, rather than the
> scanner file. Of course, one can't add a function to yy::location in
> the scanner file, but one can use a free function and pass the
> location object to it, or subclass yy::location with this function
> added easily. And the scanner file is where this code belongs
> logically more than the parser file (if not the location class in
> the first place). So I don't think it's worth the effort to add this
> to Bison.

Agreed.  The user can easily plug her own definition of locations,
that's good enough as it is.

Cheers!



commit 5e2e1f03fa4e6bae59ad72a8ee10b4344457b8d8
Author: Akim Demaille 
Date:   Sat Jan 15 10:28:16 2022 +0100

doc: explain why location's "column" are defined vaguely

Suuggested by Frank Heckenbach.
<https://lists.gnu.org/r/bug-bison/2022-01/msg0.html>

* doc/bison.texi (Location Type): Explain why location's "column" are
defined vaguely.
Show tab handling in ltcalc and calc++.
* examples/c/bistromathic/parse.y: Show tab handling.
* examples/c++/calc++/calc++.test,
* examples/c/bistromathic/bistromathic.test:
Check tab handling.

diff --git a/doc/bison.texi b/doc/bison.texi
index 2dab6c49..de6021d8 100644
--- a/doc/bison.texi
+++ b/doc/bison.texi
@@ -2365,6 +2365,8 @@ @node Location Tracking Calc
 * Ltcalc Lexer::   The lexical analyzer.
 @end menu
 
+See @ref{Tracking Locations} for details about locations.
+
 @node Ltcalc Declarations
 @subsection Declarations for @code{ltcalc}
 
@@ -2488,7 +2490,7 @@ @node Ltcalc Lexer
 @group
   /* Skip white space. */
   while ((c = getchar ()) == ' ' || c == '\t')
-++yylloc.last_column;
+yylloc.last_column += c == '\t' ? 8 - ((yylloc.last_column - 1) & 7) : 1;
 @end group
 
 @group
@@ -4751,6 +4753,33 @@ @node Location Type
 initialization), use the @code{%initial-action} directive.  @xref{Initial
 Action Decl}.
 
+@sp 1
+
+@cindex column
+The meaning of ``column'' is deliberately left vague since there are several
+options, depending on the use cases.
+
+With multibyte input (say UTF-8), simply counting the number of bytes does
+not match character positions on the screen.  One needs advanced functions
+mapping multibyte characters to their visual width (see for instance
+Gnulib's @code{mbswidth} and @code{mbsnwidth} functions).  Tabulation
+characters probably need a dedicated implementation, to match the ``go to
+next multiple of 8'' behavior.
+
+However to quote input in error messages, as @command{bison} does:
+
+@example
+@group
+1.10-12: @derror{error}: invalid identifier: ‘3.8’
+1 | %require @derror{3.8}
+  |  @derror{^~~}
+@end group
+@end example
+
+@noindent
+then by

Re: Bug in glr2.cc

[Akim Demaille]

Hi Jot,

Sorry for not having picked up this issue sooner.

> Le 24 déc. 2021 à 23:38, Jot Dot  a écrit :
> I have spent a couple of hours slowly reducing my grammar to the point
> where it is about as small as I can make it.

This is excellent work, thanks a lot for this!

However the test case you provided does not compile, it still has dependencies. 
 I can probably get it to work, but without the actual input that crashed your 
parser, it would be useless.

Is it possible to get that input?  As a stream of tokens.  If passing the code 
here, publicly, is a problem, you can send it privately.


> Oh, btw: Seasons Greetings? Happy Holidays? Merry Christmas? (pick one or add 
> one)
> Sorry to find it today (Dec 24) as this is a busy time for many people.

I'm late, but not yet too late, but...  Have a nice year Jot!


> I could remove this mess if we were able to make the %merge routines
> as members of the Parser class. hint hint - just a gentle nudge :)
> 

Will do.  Thanks for the reminder :)

Re: Broken code generated when enabling locations using D

[Akim Demaille]

Hi Cristian,

Adela, could you have a look at this please?

> Le 17 nov. 2021 à 19:32, cristiansimon  a écrit 
> :
> 
> Hello,
> 
> When generating a parser for the following file with location support using 
> GNU Bison 3.8.2:
> 
> %language "D"
> %define api.parser.class {Parser}
> %locations
> 
> %union {
> AST   tast;
> Declaration[] tdecls;
> Declaration   tdecl;
> Statement[]   tstmts;
> Statement tstmt;
> Type  ttype;
> stringstr;
> char  chr;
> long  lng;
> doubledbl;
> }
> 
> %token SEMICOLON";"
>PERIOD   "."
>OPENPAREN"("
>CLOSEPAREN   ")"
>OPENBRACE"{"
>CLOSEBRACE   "}"
>OPENBRACKET  "["
>CLOSEBRACKET "]"
>PLUS "+"
>MINUS"-"
>ASTERISK "*"
>SLASH"/"
> 
> %token RETURN "return"
>VOID   "void"
>INT"int"
>FLOAT  "float"
>CHAR   "char"
>STRING "string"
> 
> %token  IDENTIFIER STRINGVALUE
> %token  CHARACTER
> %token  INTEGER
> %token  FLOATVALUE
> 
> %%
> ast : /* Empty*/
> | declarations
> ;
> 
> declarations : declaration
>  | declaration declarations
>  ;
> 
> declaration : type IDENTIFIER "(" ")" "{" statements "}"
> ;
> 
> statements : statement
>| statement statements
>;
> 
> statement : "return" ";"
>   ;
> 
> type : "void"
>  ;
> %%
> 
> struct AST {}
> class Declaration {}
> class Statement   {}
> final class Function : Declaration {}
> final class Return : Statement {}
> enum Type {}
> 
> Compiling the generated code yields the following error, which does not 
> happen when not enabling locations:
> 
> cdc.p/parser.d(764): Error: function `parser.Parser.YYStack.push(int state, 
> YYSemanticType value, ref YYLocation loc)` is not callable using argument 
> types `(int, immutable(YYSemanticType), YYLocation)`
> cdc.p/parser.d(764):cannot pass argument `yy_semantic_null` of type 
> `immutable(YYSemanticType)` to parameter `YYSemanticType value`
> cdc.p/parser.d(765): Error: function `parser.Parser.YYStack.push(int state, 
> YYSemanticType value, ref YYLocation loc)` is not callable using argument 
> types `(int, immutable(YYSemanticType), YYLocation)`
> cdc.p/parser.d(765):cannot pass argument `yy_semantic_null` of type 
> `immutable(YYSemanticType)` to parameter `YYSemanticType value`
> 
> using ldc2.
> 
> Removing the user-defined classes from the union seems to fix it, but this 
> cannot be done in the original code as they serve a purpose there.
> 
> What is there at fault here? What could be a fix for this issue? Thanks in 
> advance.
> 

Re: Mismatch between HTML report and y.output

[Akim Demaille]

Domingo,

> Le 18 nov. 2021 à 13:50, Domingo Alvarez Duarte  a écrit :
> 
> Hello !
> 
> Trying to figure out the differences in detected conflicts between 
> byacc/bison/lemon here https://github.com/mingodad/lalr-parser-test, I 
> noticed that the generated states on "HTML report" doesn't seem to match with 
> the ones on "y.output" for example for 
> https://github.com/mingodad/cfront-3/blob/master/src/gram.y with bison 3.8.2 
> and it seems to obfuscate rather than clarify the problem.
> 
> y.output:
> 
> 
> 
> State 130
> 
>   174 decl: decl arg_lp • elist RP
>   373 arg_list: arg_lp • arg_type_list ellipsis_opt RP fct_attributes

This is the so-called "kernel" (or "core") of the item set of the state.  Bison 
needs to "complete" this core with the "consequences".  For instance because of 
rule 174 you can be at the beginning of all the RHS of rules whose LHS is 
elist.  And you do do the closure of this process.

You are comparing the text report with the complete html report (which, btw, 
also includes the lookaheads).  See --report to generate more complete text 
reports.

It is true that it's a pity not to highlight the core.  It would be clearer.

Cheers.

> 
> DELETE   shift, and go to state 156
> ENUM shift, and go to state 3
> NEW  shift, and go to state 157
> OPERATOR shift, and go to state 158
> SIZEOF   shift, and go to state 159
> THIS shift, and go to state 160
> LP   shift, and go to state 161
> NOT  shift, and go to state 162
> 
> 
> 
> But in the HTML report:
> 
> 
> 
> State 130
>48 tname → • qualified_tname
>49   | • qualified_tname lt temp_inst_parms gt
>50   | • NAME lt temp_inst_parms gt
>51 tp → • TYPE
>52| • LINKAGE
>53| • tname
>54| • tn_list DECL_MARKER
>55| • class_dcl
>56| • enum_dcl
>57| • DECL_MARKER
>58 type → • tp
>59  | • type TYPE
>60  | • type tname
>61  | • type class_dcl
>62  | • type enum_dcl
>63  | • type DECL_MARKER
>78 enum_dcl → • ENUM LC moe_list RC
>79  | • ENUM enumtag LC moe_list RC
>80  | • ENUM enumtag
>96 class_dcl → • class_head cl_mem_list RC inline_mem_defs
>97   | • AGGR tag
>98   | • AGGR qualified_tname lt temp_inst_parms gt
>99   | • AGGR DECL_MARKER
>   111 class_head → • AGGR LC
>   112| • AGGR tag base_list LC
>   113| • AGGR qualified_tname lt temp_inst_parms gt base_list LC
>   161 scope_qualifiers → • tn_list
>   162 tn_list → • tscope
>   163 | • tn_list tscope
>   164 qualified_tname → • tn_list TNAME
>   165 | • TNAME
>   174 decl → decl arg_lp • elist RP
>   244 elist → • ex_list
>   245 ex_list → • initializer
>   246 | • ex_list CM initializer
>   247 initializer → • e
>   248 | • LC elist RC
> 
> ...
> 
> 
> 
> Cheers !
> 
> 

Re: Different conflicts between byacc and bison

[Akim Demaille]

Hi Domingo,

> Le 15 nov. 2021 à 12:44, Domingo Alvarez Duarte  a écrit :
> 
> Hello Akim !
> 
> Thanks for reply !
> 
> I did a comparison between lemon (from sqlite) byacc (from 
> https://invisible-island.net/byacc/byacc.html) and bison with a custom 
> command line option to ignore all precedence settings  then tested with 
> sqlite3/cfront3 grammar (see here 
> https://github.com/mingodad/lalr-parser-test).

That's great.  Can you make available the grammars themselves in the repo?  It 
might be troublesome wrt licenses, I don't know.

> For sqlite3 grammar:
> 
> lemon : zero conflicts
> byacc/bison: 53 reduce/reduce conflicts

This is very surprising.  I venture that lemon has some system to "solve" RR 
conflicts?  Something like precedence directives applied to RR conflicts.  This 
is not something one can do in Yacc (and so far I have resisted to proposals 
because their scope is too narrow).

We should study the simplest of these conflicts in Yacc (say the "first" one, 
the easiest to reach from the axiom), and follow that path into lemon's 
automaton.


> Again for sqlite3 grammar ignoring all precedences:
> 
> lemon/byacc: 827 parsing conflicts, 788 shift/reduce, 39 reduce/reduce
> 
> bison: 827 parsing conflicts, 734 shift/reduce, 93 reduce/reduce

That's funny.  But then again, there is no unambiguous definition of "conflict" 
that I aware of, and I know if will eventually revise Bison's way to count (as 
mentioned in a previous message of mine).



> Now for cfront3 grammar:
> 
> lemon: 19 parsing conflicts, 19 shift/reduce, 0 reduce/reduce
> 
> byacc: 24 parsing conflicts, 21 shift/reduce, 3 reduce/reduce
> 
> bison: 24 parsing conflicts, 20 shift/reduce, 4 reduce/reduce

I don't understand the 0 rr-conflicts.  As above, we need to compare the 
automata.  Actually, if you can't include the grammars in your repo, you 
probably can include at least the generated artifacts (grammars in various 
syntaxes, and reports).

Have your tried the toy grammar of my previous message with lemon?  Maybe that 
would teach us something.

Cheers!

Re: Bogus Bison 3.8.2 warning: suspicious sequence in the output

[Akim Demaille]

> Le 6 nov. 2021 à 20:09, Akim Demaille  a écrit :
> 
> Hum...  Actually "escaping" the user's input to avoid the warning would be a 
> better option.  I'll think about it.

I'm installing this on top of the previous commit.

commit c559b05fc9336d70eee6ef2bcabf949ed5d92a39
Author: Akim Demaille 
Date:   Sun Nov 7 09:27:52 2021 +0100

warnings: don't complain about m4_foo and b4_foo when from the user

Currently, occurrences of these identifiers in the user's input yield
spurious warnings.

To tell the difference between a legitimate m4_foo from the user, and
a bad m4_foo coming from a non-evaluated macro of a skeleton, escape
the user's identifiers as m4@'_foo.  We already use @' as a special
sequence to be stripped from the skeleton's output.

See <https://lists.gnu.org/r/bug-bison/2021-10/msg00026.html> and
previous commit ("warnings: be less picky about occurrences of m4_/b4_
in the output").

* src/flex-scanner.h (OBSTACK_SGROW): New.
* src/output.c (output_escaped): Escape m4_ and b4_.
* src/scan-code.l: Likewise.
* src/system.h (obstack_escape): Likewise.
And rewrite as a function.
* tests/skeletons.at (Suspicious sequences): Make sure the user can
use m4_foo/b4_foo without spurious warnings.

diff --git a/NEWS b/NEWS
index 1fa344cf..1b7ea390 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,9 @@ GNU Bison NEWS
 
 * Noteworthy changes in release ?.? (-??-??) [?]
 
+  Fixed portability issues of the test suite on Solaris.
+
+  Fixed spurious warnings about input containing `m4_` or `b4_`.
 
 * Noteworthy changes in release 3.8.2 (2021-09-25) [stable]
 
diff --git a/src/flex-scanner.h b/src/flex-scanner.h
index 238ca881..08c0e62c 100644
--- a/src/flex-scanner.h
+++ b/src/flex-scanner.h
@@ -106,6 +106,9 @@ static struct obstack obstack_for_string;
 # define STRING_GROW()  \
   obstack_grow (&obstack_for_string, yytext, yyleng)
 
+# define STRING_SGROW(String)   \
+  obstack_sgrow (&obstack_for_string, String)
+
 # define STRING_FINISH()\
   (last_string = obstack_finish0 (&obstack_for_string))
 
diff --git a/src/output.c b/src/output.c
index 4b1bb0d8..07234f2a 100644
--- a/src/output.c
+++ b/src/output.c
@@ -104,20 +104,45 @@ GENERATE_MUSCLE_INSERT_TABLE 
(muscle_insert_symbol_number_table, symbol_number)
 GENERATE_MUSCLE_INSERT_TABLE (muscle_insert_item_number_table, item_number)
 GENERATE_MUSCLE_INSERT_TABLE (muscle_insert_state_number_table, state_number)
 
+
 /*.
 | Print to OUT a representation of CP quoted and escaped for M4.  |
+| |
+| Keep sync'ed with obstack_escape.   |
 `*/
 
 static void
 output_escaped (FILE *out, const char *cp)
 {
-  for (; *cp; cp++)
+  for (; *cp; ++cp)
 switch (*cp)
   {
   case '$': fputs ("$][", out); break;
   case '@': fputs ("@@",  out); break;
   case '[': fputs ("@{",  out); break;
   case ']': fputs ("@}",  out); break;
+
+  case 'b':
+if (STRPREFIX_LIT ("b4_", cp))
+  {
+fputs ("b4@'_", out);
+cp += strlen ("b4_") - 1;
+break;
+  }
+else
+  goto append;
+
+  case 'm':
+if (STRPREFIX_LIT ("m4_", cp))
+  {
+fputs ("m4@'_", out);
+cp += strlen ("m4_") - 1;
+break;
+  }
+else
+  goto append;
+
+  append:
   default:  fputc (*cp,   out); break;
   }
 }
diff --git a/src/scan-code.l b/src/scan-code.l
index fced37d2..e3e765c8 100644
--- a/src/scan-code.l
+++ b/src/scan-code.l
@@ -197,6 +197,9 @@ ref  -?[0-9]+|{id}|"["{id}"]"|"$"
 
 <*>
 {
+  "b4_"   STRING_SGROW ("b4@'_");
+  "m4_"   STRING_SGROW ("m4@'_");
+
   /* Escape M4 quoting characters in C code.  */
   [$@\[\]]obstack_escape (&obstack_for_string, yytext);
 
diff --git a/src/system.h b/src/system.h
index bbdf9fd0..630ef646 100644
--- a/src/system.h
+++ b/src/system.h
@@ -204,24 +204,53 @@ max_int (int a, int b)
   } while (0)
 
 
-/* Output Str escaped for our postprocessing (i.e., escape M4 special
+/* Output CP escaped for our postprocessing (i.e., escape M4 special
characters).
 
-   For instance "[foo]" -> "@{foo@}", "$$" -> "$][$][". */
+   For instance "[foo]" -> "@{foo@}", "$$" -&

Re: Testsuite summary for GNU Bison 3.8.2 : FAIL 2

[Akim Demaille]

> Le 7 nov. 2021 à 01:44, Dennis Clarke  a écrit :
> 
> OKay I think we are in very very good shape now :
> 
> ## - ##
> ## Test results. ##
> ## - ##
> 
> ERROR: 488 tests were run,
> 1 failed unexpectedly.
> 288 tests were skipped.

Excellent.  Pushed, thanks Dennis!

Re: Bogus Bison 3.8.2 warning: suspicious sequence in the output

[Akim Demaille]

Hi Marko,

> Le 27 oct. 2021 à 15:04, Marko Mäkelä  a écrit :
> 
> Hi,
> 
> Bison 3.8.2 is issuing the following warning for a grammar whose semantic 
> action includes the variable name my_charset_utf8mb4_bin:
> 
> yy_mariadb.cc:35834: warning: suspicious sequence in the output: b4_bin 
> [-Wother]

Bummer...

> It seems to me that the regular expression in 
> http://git.savannah.gnu.org/cgit/bison.git/tree/src/scan-skel.l is too 
> greedy, or the check should be suppressed if the input contains such 
> identifiers.

Let's start with "too greedy".  Collecting all the identifiers in the input is 
too much effort for this warning.

> A quick search of the Bison Texinfo for b4_ or m4_ did not find anything. 
> Because those are not documented as reserved words or sequences, I think that 
> identifiers like that should be allowed in semantic actions without Bison 
> reporting any warnings.

The pattern was stupid and caught occurrences of b4_/m4_ anywhere.  The point 
was to rather to catch \bb4_ and \bm4_.  This is what the attached commit does. 
 Yes, it does mean that if someone uses b4_foo or m4_foo, she'll get a spurious 
warning.  If that's really a problem, I'll have to introduce a dedicated -W.  
But that would lessen the interest of the warning, if most people don't enable 
it.

Hum...  Actually "escaping" the user's input to avoid the warning would be a 
better option.  I'll think about it.

Cheers!


commit c95d0dd5f54b98825e4cfe00a807db78a073e376
Author: Akim Demaille 
Date:   Sat Nov 6 19:24:04 2021 +0100

warnings: be less picky about occurrences of m4_/b4_ in the output

Reported by Marko Mäkelä.
<https://lists.gnu.org/r/bug-bison/2021-10/msg00026.html>

* src/scan-skel.l: It is ok to have foob4_ or foom4_.
* tests/skeletons.at (Suspicious sequences): New.

diff --git a/THANKS b/THANKS
index be743a23..14e678c0 100644
--- a/THANKS
+++ b/THANKS
@@ -117,6 +117,7 @@ Marc Autret   autre...@epita.fr
 Marc Mendiola mmend...@usc.edu
 Marc Schönefeld   marc.schoenef...@gmx.org
 Mark Boyall   wolfeinst...@gmail.com
+Marko Mäkelä  marko.mak...@iki.fi
 Martin Blais  bl...@furius.ca
 Martin Jacobs martin.jac...@arcor.de
 Martin Mokrejsmmokr...@natur.cuni.cz
diff --git a/src/scan-skel.l b/src/scan-skel.l
index 60232a2e..5a93ce03 100644
--- a/src/scan-skel.l
+++ b/src/scan-skel.l
@@ -54,7 +54,9 @@ static void output_mapped_file (char const *name);
 %}
 
 /* Identifiers of our M4 macros.  */
-macro[bm]4_[a-zA-Z_0-9]*
+macro  [bm]4_[a-zA-Z_0-9]*
+/* Safe sequence of word-constituent characters.  */
+identifier [A-Za-z_0-9]+
 
 %x SC_AT_DIRECTIVE_ARGS
 %x SC_AT_DIRECTIVE_SKIP_WS
@@ -90,7 +92,7 @@ macro[bm]4_[a-zA-Z_0-9]*
   /* This pattern must not match more than the previous @ patterns. */
 @[^@{}''(\n]*   fail_for_invalid_at (yytext);
 \n  out_lineno++; ECHO;
-[^bm@\n]+   ECHO;
+[^a-z@\n]+  ECHO;
 
   /* If there are still identifiers that look like macros, such as
  b4_synbol, this probably an error, say a typo in M4, or
@@ -104,6 +106,7 @@ macro[bm]4_[a-zA-Z_0-9]*
 "suspicious sequence in the output: %s", yytext);
   ECHO;
 }
+{identifier}ECHO;
 .   ECHO;
 
 <> {
diff --git a/tests/skeletons.at b/tests/skeletons.at
index 0ed11027..30f25a61 100644
--- a/tests/skeletons.at
+++ b/tests/skeletons.at
@@ -324,3 +324,44 @@ AT_SETUP([[Fatal errors but M4 continues producing 
output]])
 ]])
 
 AT_CLEANUP
+
+
+## -- ##
+## Suspicious sequences.  ##
+## -- ##
+
+AT_SETUP([[Suspicious sequences]])
+
+AT_DATA([[skel.c]],
+[[m4@&t@_include(b4_skeletonsdir/[c.m4])
+m4@&t@_divert_push(0)d@&t@nl
+@output(b4_parser_file_name@)d@&t@nl
+]b4_user_pre_prologue[
+]b4_user_post_prologue[
+b4_unevaluated
+m4@&t@_unevaluated
+]b4_epilogue[
+
+m4@&t@_divert_pop(0)
+]])
+
+AT_DATA([[input1.y]],
+[[%skeleton "./skel.c"
+%{
+  myb4_unevaluated
+  mym4_unevaluated
+%}
+%%
+start: ;
+%%
+myb4_unevaluated
+mym4_unevaluated
+]])
+
+AT_BISON_CHECK([[input1.y]], [], [],
+[[input1.tab.c:10: warning: suspicious sequence in the output: b4_unevaluated 
[-Wother]
+input1.tab.c:11: warning: suspicious sequence in the output: 
m4@&t@_unevaluated [-Wother]
+]])
+
+
+AT_CLEANUP

Re: Different conflicts between byacc and bison

[Akim Demaille]

Hi Domingo,

> Le 3 nov. 2021 à 19:27, Domingo Alvarez Duarte  a écrit :
> 
> Hello !
> 
> I'm trying to build the CFRONT parser

Wow...  I was unaware cfront still existed.

> with bison instead of byacc and I've got bison to accept the grammar 
> https://github.com/mingodad/cfront-3/blob/master/src/gram.y but the generated 
> parser by bison has a small difference in the conflicts found in the grammar 
> (see bellow)

That's an interesting finding, thanks for reporting it!

FWIW, I believe we will change the way conflicts are counted in Bison.  I don't 
know when, and backward compatibility will have to be taken into account, but 
it was found that Bison disagrees with Bison.  Indeed, when looking for 
counterexamples and when merely counting the number of conflicts, we don't get 
the same results, and the latter should comply with the former.  See 
https://github.com/akimd/bison/issues/73#issuecomment-792282225.

> cfront-3/src$ bison -y gram.y
> gram.y: warning: 20 shift/reduce conflicts [-Wconflicts-sr]
> gram.y: warning: 4 reduce/reduce conflicts [-Wconflicts-rr]
> 
> cfront-3/srv$ byacc gram.y
> byacc: 21 shift/reduce conflicts, 3 reduce/reduce conflicts.

I have not tried to compare the SR conflicts (the automata are very similar, 
but the state numbers are different and I don't feel like tracking the 
correspondance right now).  But the RR conflicted state where Bison and Byacc 
differ is:

In Bison (very verbose mode, including counterexamples)

> State 101
> 
>57 tname: . qualified_tname
>58  | . qualified_tname lt temp_inst_parms gt
>59  | . NAME lt temp_inst_parms gt
>   174 scope_qualifiers: . tn_list
>   175 tn_list: . tscope
>   176| . tn_list tscope
>   177 qualified_tname: . tn_list TNAME
>   178| . TNAME
>   188 decl: tname LP . MUL ID RP arg_list
>   189 | tname LP . AND ID RP arg_list
>   190 | tname LP . elist RP
>   191 | tname LP . RP fct_attributes
>   192 | tname LP . MEMPTR decl RP arg_list
>   266 elist: . ex_list
>   267 ex_list: . initializer
>   268| . ex_list CM initializer
>   269 initializer: . e
>   270| . LC elist RC
>   296 e: . e ASSIGN e
>   297  | . e PLUS e
>   298  | . e MINUS e
>   299  | . e MUL e
>   300  | . e AND e
>   301  | . e OR e
>   302  | . e ER e
>   303  | . e SHIFTOP e
>   304  | . e EQUOP e
>   305  | . e DIVOP e
>   306  | . e RELOP e
>   307  | . e LT e
>   308  | . e GT e
>   309  | . e ANDAND e
>   310  | . e OROR e
>   311  | . e ASOP e
>   312  | . e CM e
>   313  | . e QUEST e COLON e
>   314  | . e REFMUL e
>   315  | . DELETE term
>   316  | . DELETE LB e RB term
>   317  | . MEM DELETE term
>   318  | . MEM DELETE LB e RB term
>   319  | . term
>   320  | . THROW term
>   321  | %empty .  [RP, LT, GT, ER, OR, ANDAND, OROR, QUEST, ASSIGN, CM, 
> ASOP, RELOP, EQUOP, DIVOP, SHIFTOP, REFMUL]
>   322 term: . NEW cast_type
>   323 | . NEW new_type
>   324 | . MEM NEW cast_type
>   325 | . MEM NEW new_type
>   326 | . term ICOP
>   327 | . cast_type term
>   328 | . MUL term
>   329 | . AND term
>   330 | . MINUS term
>   331 | . PLUS term
>   332 | . NOT term
>   333 | . COMPL term
>   334 | . ICOP term
>   335 | . SIZEOF term
>   336 | . SIZEOF cast_type
>   337 | . term LB e RB
>   338 | . term REF prim
>   339 | . term REF MEMQ prim
>   340 | . term REF MEMQ TNAME
>   341 | . term REF dtorspec
>   342 | . term REF scope_qualifiers prim
>   343 | . term REF scope_qualifiers TNAME
>   344 | . term DOT prim
>   345 | . term DOT MEMQ prim
>   346 | . term DOT MEMQ TNAME
>   347 | . term DOT dtorspec
>   348 | . term DOT scope_qualifiers prim
>   349 | . term DOT scope_qualifiers TNAME
>   350 | . prim
>   351 | . scope_qualifiers prim
>   352 | . tn_list COMPL tag
>   353 | . tn_list COMPL TYPE
>   354 | . term_elist
>   355 | . term_lp e RP
>   356 | . ZERO
>   357 | . ICON
>   358 | . FCON
>   359 | . STRING
>   360 | . CCON
>   361 | . THIS
>   370 term_elist: . TYPE LP elist RP
>   371   | . tname LP elist RP
>   372   | . NEW term_lp elist RP cast_type
>   373   | . NEW term_lp elist RP new_type
>   374   | . MEM NEW term_lp elist RP cast_type
>   375   | . MEM NEW term_lp elist RP new_type
>   376   | . term LP elist RP
>   377 ptname: . PTNAME lt temp_inst_parms gt
>   378 tscope: . TSCOPE
>   379   | . MEM
>   380   | . ptname TSCOPE
>   381 prim: . ID
>   382 | . OPERATOR oper
>   383 | . OPERATOR c_type
>   384 cast_type: . term_lp type cast_decl RP
>   385 term_lp: . LP
>   395 arg_lp: LP .  [ENUM, RP, CM, NAME, TYPE, TNAME, ELLIPSIS, AGGR, MEM, 
> TSCOPE, DECL_MARKER, LINKAGE, PTNAME]
> 
> DELETEshift, and go to state 154
> NEW   shift, and go to state 155
> OPERATOR  shift, and go to state 156
> SIZEOFshift, and

Re: Minor issue(?) - win_flex generating wrong #line values

[Akim Demaille]

Hi Jot,

Long time no see.

> Le 2 nov. 2021 à 22:53, Jot Dot  a écrit :
> 
> win_flex causes all my generated code to have #line values that are 1 more 
> than actual.
> This is not bad when fixing syntax errors in the lex file, but horrible when 
> "speed debugging" :)

I know.  I have the same problem on occasion on macOS.

> I've had this issue since I've started using win_flex so I imagine this issue 
> may be a low priority.
> But it would be real nice if this issue can be addressed.

Tell the Flex team, not to us.

Cheers.

Re: Testsuite summary for GNU Bison 3.8.2 : FAIL 2

[Akim Demaille]

Hi Dennis,

> Le 4 nov. 2021 à 01:51, Dennis Clarke  a écrit :
> 
> I have for you a pile of data.
> 
> I don't want to bore you with a ton of details here but I will attach
> a log file that explains everything.

On the example side, we have:

> "examples/c/bistromathic/parse.c", line 2023: warning: implicit function 
> declaration: strdup

IOW, same problems as before.  I have updated the commit to also apply
to bistromathic, see below.

On the test suite side:

> 613. torture.at:385: 613. Many lookahead tokens (torture.at:385): FAILED 
> (torture.at:394)

> ./torture.at:394: $CC $CFLAGS $CPPFLAGS  $LDFLAGS -o input input.c $LIBS
> stderr:
> c99: Warning: Option -64 passed to ld, if ld is invoked, ignored otherwise
> "input.c", line 17804: internal compiler error: Out of memory
> c99: acomp failed for input.c

I won't fight this.


> 705. cxx-type.at:406: testing GLR: Resolve ambiguity, impure, no locations ...
> c99: Warning: Option -64 passed to ld, if ld is invoked, ignored otherwise
> "types.y", line 188: warning: implicit function declaration: strdup

Likewise 706, 707, 708, 709, 710, 711, 712, 713.

So I will install the following commit to, imho, address all the failures,
except test 613 from the test suite.

The following tarball contains that commit.  While at it, I have also updated
gnulib.  Could you please check it too?

Thanks Dennis!

https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.2.7-76301.tar.gz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.2.7-76301.tar.lz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.2.7-76301.tar.xz




commit c83425ef4e1c66c81d328338e6bf98f5e3c33b81
Author: Akim Demaille 
Date:   Sat Oct 23 06:01:44 2021 +0200

tests: address portability issues about strdup

Reported by Dennis Clarke
<https://lists.gnu.org/r/bug-bison/2021-10/msg5.html>.
In particular
<https://lists.gnu.org/r/bug-bison/2021-10/msg00023.html>.

* doc/bison.texi, examples/c/glr/c++-types.y,
* examples/c/bistromathic/parse.y tests/testsuite.h:
Define _XOPEN_SOURCE to 600, to get a strdup that works on Solaris.

* tests/glr-regression.at: Use strdup freely.

diff --git a/doc/bison.texi b/doc/bison.texi
index a559649c..77357813 100644
--- a/doc/bison.texi
+++ b/doc/bison.texi
@@ -2644,6 +2644,13 @@ @node Mfcalc Declarations
 
You should have received a copy of the GNU General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.  */
+
+%code top {
+  /* Portability issues for strdup. */
+#ifndef _XOPEN_SOURCE
+# define _XOPEN_SOURCE 600
+#endif
+}
 @end example
 @end ignore
 
@@ -2875,7 +2882,7 @@ @node Mfcalc Symbol Table
not make these assumptions.  */
 #include 
 #include  /* malloc, realloc. */
-#include  /* strlen. */
+#include  /* strdup, strlen. */
 @end group
 
 @group
diff --git a/examples/c/bistromathic/parse.y b/examples/c/bistromathic/parse.y
index fbdeb7fc..5c1e6055 100644
--- a/examples/c/bistromathic/parse.y
+++ b/examples/c/bistromathic/parse.y
@@ -21,6 +21,11 @@
 
 // Emitted on top of the implementation file.
 %code top {
+  /* Portability issues for strdup. */
+  #ifndef _XOPEN_SOURCE
+  # define _XOPEN_SOURCE 600
+  #endif
+
   #include   // isdigit
   #include  // LC_ALL
   #include// cos, sin, etc.
diff --git a/examples/c/glr/c++-types.y b/examples/c/glr/c++-types.y
index 4c978386..ff1b62fb 100644
--- a/examples/c/glr/c++-types.y
+++ b/examples/c/glr/c++-types.y
@@ -53,6 +53,10 @@
 
 %code
 {
+  /* Portability issues for strdup. */
+#ifndef _XOPEN_SOURCE
+# define _XOPEN_SOURCE 600
+#endif
 
 #include 
 #include 
diff --git a/tests/glr-regression.at b/tests/glr-regression.at
index a10f69ba..3a226f65 100644
--- a/tests/glr-regression.at
+++ b/tests/glr-regression.at
@@ -295,9 +295,7 @@ m4_pushdef([AT_TEST],
 else
   {
 assert (strlen (buf) < sizeof buf - 1);
-char *s = YY_CAST (char *, malloc (strlen (buf) + 1));
-strcpy (s, buf);
-]AT_VAL[ = s;
+]AT_VAL[ = strdup (buf);
 return 'V';
   }
 break;
diff --git a/tests/testsuite.h b/tests/testsuite.h
index 46e99dfe..d2be5e62 100644
--- a/tests/testsuite.h
+++ b/tests/testsuite.h
@@ -10,3 +10,8 @@
 #  pragma GCC diagnostic ignored "-Wzero-as-null-pointer-constant"
 # endif
 #endif
+
+/* We use strdup, make it available.  */
+#ifndef _XOPEN_SOURCE
+# define _XOPEN_SOURCE 600
+#endif

Re: Testsuite summary for GNU Bison 3.8.2 : FAIL 2

[Akim Demaille]

> Le 24 oct. 2021 à 09:15, Akim Demaille  a écrit :
> 
> commit ff9771d26e5e387cde51393d97d4f8d42837a25a
> Author: Akim Demaille 
> Date:   Sat Oct 23 06:01:44 2021 +0200
> 
>examples: address portability issues about strdup
> 
>Reported by Dennis Clarke
><https://lists.gnu.org/r/bug-bison/2021-10/msg5.html>.
>In particular
><https://lists.gnu.org/r/bug-bison/2021-10/msg00023.html>.
> 
>* doc/bison.texi, examples/c/glr/c++-types.y (_XOPEN_SOURCE): Define
>to 600, to get a strdup that works on Solaris.

On top of this patch, I will also install these.

commit ab2d834429c2a15b5214079b7b7be68c60e35bd8
Author: Akim Demaille 
Date:   Sat Oct 23 06:09:31 2021 +0200

examples: fix coding style

* examples/c/glr/c++-types.y: Use snake_case identifiers.
Prefer strdup to malloc+strcpy.

diff --git a/examples/c/bistromathic/parse.y b/examples/c/bistromathic/parse.y
index 8b1591e6..fbdeb7fc 100644
--- a/examples/c/bistromathic/parse.y
+++ b/examples/c/bistromathic/parse.y
@@ -244,7 +244,7 @@ init_table (void)
 symrec *
 putsym (char const *name, int sym_type)
 {
-  symrec *res = (symrec *) malloc (sizeof (symrec));
+  symrec *res = malloc (sizeof *res);
   res->name = strdup (name);
   res->type = sym_type;
   res->value.var = 0; // Set value to 0 even if fun.
diff --git a/examples/c/glr/c++-types.y b/examples/c/glr/c++-types.y
index 232d0112..ff1b62fb 100644
--- a/examples/c/glr/c++-types.y
+++ b/examples/c/glr/c++-types.y
@@ -29,24 +29,24 @@
 
 %code requires
 {
-  union Node {
+  union node {
 struct {
-  int isNterm;
+  int is_nterm;
   int parents;
-} nodeInfo;
+} node_info;
 struct {
-  int isNterm; /* 1 */
+  int is_nterm; /* 1 */
   int parents;
   char const *form;
-  union Node *children[3];
+  union node *children[3];
 } nterm;
 struct {
-  int isNterm; /* 0 */
+  int is_nterm; /* 0 */
   int parents;
   char *text;
 } term;
   };
-  typedef union Node Node;
+  typedef union node node_t;
 }
 
 %define api.value.type union
@@ -65,12 +65,12 @@
 #include 
 #include 
 
-  static Node *new_nterm (char const *, Node *, Node *, Node *);
-  static Node *new_term (char *);
-  static void free_node (Node *);
-  static char *node_to_string (const Node *);
-  static void node_print (FILE *, const Node *);
-  static Node *stmt_merge (YYSTYPE x0, YYSTYPE x1);
+  static node_t *new_nterm (char const *, node_t *, node_t *, node_t *);
+  static node_t *new_term (char *);
+  static void free_node (node_t *);
+  static char *node_to_string (const node_t *);
+  static void node_print (FILE *, const node_t *);
+  static node_t *stmt_merge (YYSTYPE x0, YYSTYPE x1);
 
   static void yyerror (YYLTYPE const * const loc, const char *msg);
   static yytoken_kind_t yylex (YYSTYPE *lval, YYLTYPE *lloc);
@@ -87,9 +87,9 @@
 
 %glr-parser
 
-%type  stmt expr decl declarator TYPENAME ID
-%destructor { free_node ($$); } 
-%printer { node_print (yyo, $$); } 
+%type  stmt expr decl declarator TYPENAME ID
+%destructor { free_node ($$); } 
+%printer { node_print (yyo, $$); } 
 
 %%
 
@@ -190,12 +190,12 @@ yylex (YYSTYPE *lval, YYLTYPE *lloc)
 if (isupper ((unsigned char) buffer[0]))
   {
 tok = TYPENAME;
-lval->TYPENAME = new_term (strcpy (malloc (i), buffer));
+lval->TYPENAME = new_term (strdup (buffer));
   }
 else
   {
 tok = ID;
-lval->ID = new_term (strcpy (malloc (i), buffer));
+lval->ID = new_term (strdup (buffer));
   }
   }
 else
@@ -210,45 +210,45 @@ yylex (YYSTYPE *lval, YYLTYPE *lloc)
 }
 }
 
-static Node *
-new_nterm (char const *form, Node *child0, Node *child1, Node *child2)
+static node_t *
+new_nterm (char const *form, node_t *child0, node_t *child1, node_t *child2)
 {
-  Node *res = malloc (sizeof *res);
-  res->nterm.isNterm = 1;
+  node_t *res = malloc (sizeof *res);
+  res->nterm.is_nterm = 1;
   res->nterm.parents = 0;
   res->nterm.form = form;
   res->nterm.children[0] = child0;
   if (child0)
-child0->nodeInfo.parents += 1;
+child0->node_info.parents += 1;
   res->nterm.children[1] = child1;
   if (child1)
-child1->nodeInfo.parents += 1;
+child1->node_info.parents += 1;
   res->nterm.children[2] = child2;
   if (child2)
-child2->nodeInfo.parents += 1;
+child2->node_info.parents += 1;
   return res;
 }
 
-static Node *
+static node_t *
 new_term (char *text)
 {
-  Node *res = malloc (sizeof *res);
-  res->term.isNterm = 0;
+  node_t *res = malloc (sizeof *res);
+  res->term.is_nterm = 0;
   res->term.parents = 0;
   res->term.text = text;
   return res;
 }
 
 static void
-free_node (Node *node)
+free_n

Re: Testsuite summary for GNU Bison 3.8.2 : FAIL 2

[Akim Demaille]

Hi Dennis,

> Le 21 oct. 2021 à 03:58, Dennis Clarke  a écrit :
> 
>>> #define _XOPEN_SOURCE 600
>> 
>> Would that be the difference?  If you define this early enough in
>> the original test case, does it work properly?
>> 
>> I have attached "without-my-strdup.y", that must be installed as
>> mfcalc.y, for convenience.

> PASS: examples/c/mfcalc/mfcalc.test
> PASS: examples/c/rpcalc/rpcalc.test
> SKIP: examples/c/bistromathic/bistromathic.test
> PASS: examples/c/calc/calc.test
> FAIL: examples/c/glr/c++-types.test
> PASS: examples/c/lexcalc/lexcalc.test
> PASS: examples/c/pushcalc/calc.test
> SKIP: examples/c/reccalc/reccalc.test

Good.  I have installed the following patch.  Amusingly enough strdup is the 
example chosen to explain _XOPEN_SOURCE on this answer on SO: 
https://stackoverflow.com/a/5724485.

Could you please try this tarball?

https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.2.6-59bb.tar.gz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.2.6-59bb.tar.lz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.2.6-59bb.tar.xz

Thanks!

commit ff9771d26e5e387cde51393d97d4f8d42837a25a
Author: Akim Demaille 
Date:   Sat Oct 23 06:01:44 2021 +0200

examples: address portability issues about strdup

Reported by Dennis Clarke
<https://lists.gnu.org/r/bug-bison/2021-10/msg5.html>.
In particular
<https://lists.gnu.org/r/bug-bison/2021-10/msg00023.html>.

* doc/bison.texi, examples/c/glr/c++-types.y (_XOPEN_SOURCE): Define
to 600, to get a strdup that works on Solaris.

diff --git a/doc/bison.texi b/doc/bison.texi
index a559649c..77357813 100644
--- a/doc/bison.texi
+++ b/doc/bison.texi
@@ -2644,6 +2644,13 @@ @node Mfcalc Declarations
 
You should have received a copy of the GNU General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.  */
+
+%code top {
+  /* Portability issues for strdup. */
+#ifndef _XOPEN_SOURCE
+# define _XOPEN_SOURCE 600
+#endif
+}
 @end example
 @end ignore
 
@@ -2875,7 +2882,7 @@ @node Mfcalc Symbol Table
not make these assumptions.  */
 #include 
 #include  /* malloc, realloc. */
-#include  /* strlen. */
+#include  /* strdup, strlen. */
 @end group
 
 @group
diff --git a/examples/c/glr/c++-types.y b/examples/c/glr/c++-types.y
index 3a14d9bf..232d0112 100644
--- a/examples/c/glr/c++-types.y
+++ b/examples/c/glr/c++-types.y
@@ -53,6 +53,10 @@
 
 %code
 {
+  /* Portability issues for strdup. */
+#ifndef _XOPEN_SOURCE
+# define _XOPEN_SOURCE 600
+#endif
 
 #include 
 #include 

Re: Testsuite summary for GNU Bison 3.8.2 : FAIL 2

[Akim Demaille]

Hi Dennis,

> Le 15 oct. 2021 à 01:01, Dennis Clarke  a écrit :
> 
> On 10/14/21 01:28, Akim Demaille wrote:
>> Hi Dennis,
>> 
>>> Le 14 oct. 2021 à 03:40, Dennis Clarke  a écrit :
>>> 
>>> *res = {
>>>   name  = 0x105c30 ""
>>>   type  = 0
>>>   value = {
>>>   var = 0.0
>>>   fun = (nil)
>>>   }
>>>   next  = (nil)
>>> }
>> 
>> Really???  So it seems that strdup on this machine is dead broken.
> 
> This will be really silly long but strdup seems to work just fine :
> 
> beta $ cat /tmp/foo.c
...
> #define _XOPEN_SOURCE 600

Would that be the difference?  If you define this early enough in
the original test case, does it work properly?

I have attached "without-my-strdup.y", that must be installed as
mfcalc.y, for convenience.


> That actually looks pretty good given that the mfcalc output seems
> correct and we have a bucket of debug printout stuff in there as noise.
> However the results look correct.

Yes, this time the test was passing.  So yes, we do have a problem
with strdup.

Please check with-my-strdup.y.


> Funny thing, the exact same code blows up in a spectacular fashion on
> Debian Linux sid on ppc64.

What do you mean?  my_strdup is expected to be portable.  It passes
on this MacOS.

> Yet another thing slightly off topic looking at the core dump from
> c++-types I see :
> 
> ...
> 
> So that is recursive fun and strlen(child2) should be zero? Given that
> it is NULL we end up at the res = malloc (1) with no checks anywhere
> that the malloc even works but anyways maybe it is best to just look at
> the mfcalc stuff for now.

The code looks right to me.  And passed asan and ubsan.  I'm pretty
sure it's strdup that does not work again.

Cheers!




without-my-strdup.y
Description: Binary data


with-my-strdup.y
Description: Binary data

Re: Flex size_t sizes

[Akim Demaille]

Hi Uxio,

> Le 16 oct. 2021 à 22:38, uxio prego  a écrit :
> 
> Hi Hans,
> 
>> On 14 Oct 2021, at 15:23, Hans Åberg  wrote:
>> 
>> Hi Akim,
>> 
>> Saw you have edited Flex, so I take it up here, even though not strictly a 
>> Bison topic:
> 
> what do you mean? Isn't westes/flex at GitHub the
> "official Flex with regards to contemporary Bison" anymore?

Bison per se has no special bound with flex.  It can use any parser generator.  
People are used to the "Bison + Flex" tandem, because of Yacc and Lex, that's 
all.

And Flex is not particularly actively maintained.

Re: Flex size_t sizes

[Akim Demaille]

Hi Hans,

> Le 14 oct. 2021 à 15:23, Hans Åberg  a écrit :
> 
> Hi Akim,
> 
> Saw you have edited Flex, so I take it up here, even though not strictly a 
> Bison topic:
> 
> The Apple flex version has been edited to admit size_t sizes, 64-bit on the 
> platform, and perhaps it might be good idea for regular flex, which uses int, 
> only 32-bit there. If using '%option c++' and mixing the versions, then the 
> FlexLexer.h header is incompatible with the C++ source code, generating a 
> compile error.
> 
> On MacOS with MacPorts and Xcode, the files are in
> /opt/local/include/FlexLexer.h
> /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/FlexLexer.h

I have already reported how I dealt with that.  I used a wrapper.

https://github.com/akimd/vcsn/blob/master/build-aux/bin/flex%2B%2B.in

I'm no longer working on this now.

And I don't plan to spend time on flex.

Cheers.

Re: Testsuite summary for GNU Bison 3.8.2 : FAIL 2

[Akim Demaille]

Hi Dennis,

> Le 14 oct. 2021 à 03:40, Dennis Clarke  a écrit :
> 
> *res = {
>name  = 0x105c30 ""
>type  = 0
>value = {
>var = 0.0
>fun = (nil)
>}
>next  = (nil)
> }

Really???  So it seems that strdup on this machine is dead broken.
And... strdup is also used in examples/c/glr/c++-types...

Please try the attached version.

Cheers!



mfcalc.y
Description: Binary data

Re: Testsuite summary for GNU Bison 3.8.2 : FAIL 2

[Akim Demaille]

Hi Dennis,

> Le 13 oct. 2021 à 11:23, Dennis Clarke  a écrit :
> 
> Sorry, no joy.

:(

Could you replace examples/c/mfcalc/mfcalc.y with the attached one, run "make 
examples/c/mfcalc/mfcalc" (or "make check-examples") and then run this please?

echo 'a = 256' | ./examples/c/mfcalc/mfcalc -p

I don't need the logs of make check, just this command.

Thanks!



mfcalc.y
Description: Binary data

Re: Testsuite summary for GNU Bison 3.8.2 : FAIL 2

[Akim Demaille]

> Le 13 oct. 2021 à 18:59, Paul Eggert  a écrit :
> 
> In C, a top-level variable has static storage duration even if its not 
> declared with the 'static' keyword

Ah, ok.  Thanks a lot Paul!

Re: Testsuite summary for GNU Bison 3.8.2 : FAIL 2

[Akim Demaille]

Hi Dennis,

> Le 13 oct. 2021 à 00:19, Dennis Clarke  a écrit :
> 
> On 10/9/21 04:35, Akim Demaille wrote:
>> Hi Dennis,
>> 
>> Thanks for this bug report.
> 
> The least I can do.

:)

> You will love this :
> 
> p = 0x100104410
> (dbx) print *p
> *p = {
>name  = 0x105bd0 ""
>type  = 260
>value = {
>var = 2.1225218246347e-314
>fun = 0x100103f00
>}
>next  = 0x1001043e0
> }
> (dbx) quit
> beta $

Well, maybe the problem is actually very simple: the sym_table pointer is not 
initialized to NULL by the compiler.  And as a matter of fact, it is probably 
right, I don't believe the C standard requires it to do it.  It is mandatory 
for static variables, but that's not the case here.

Could you look for the following lines in examples/c/mfcalc/mfcalc.y

/* The symbol table: a chain of 'struct symrec'. */
symrec *sym_table;

and turn it into this?

/* The symbol table: a chain of 'struct symrec'. */
symrec *sym_table = NULL;

That should do it.  Thanks!

Re: Testsuite summary for GNU Bison 3.8.2 : FAIL 2

[Akim Demaille]

Hi Dennis,

Thanks for this bug report.

> Le 5 oct. 2021 à 21:39, Dennis Clarke  a écrit :
> 
> Basic C99 compile and then test suite run with result :
> 
> beta $ cat test-suite.log
> FAIL: examples/c/mfcalc/mfcalc
> ==
> 
> checking for diff --strip-trailing-cr...
> diff: illegal option -- strip-trailing-cr
> usage: diff [-bitw] [-c | -e | -f | -h | -n | -u] file1 file2
>  diff [-bitw] [-C number | -U number] file1 file2
>  diff [-bitw] [-D string] file1 file2
>  diff [-bitw] [-c | -e | -f | -h | -n | -u] [-l] [-r] [-s] [-S
> name] directory1 directory2
> checking for diff --strip-trailing-cr...

> Looks like ye old diff gets upset at an option there.

That's not a problem.  The test is actually checking whether it can use it or 
not.  Hence the "checking for diff --strip-trailing-cr...".


> mfcalc: PASS: 1
> mfcalc: PASS: 2
> mfcalc: PASS: 3
> mfcalc: FAIL: 4 (expected status: 0, effective: 139)
> ./examples/test: line 50:  7737 Segmentation Fault  (core dumped)
> "$abs_medir/$me" "$@"
> mfcalc: PASS: 5
> FAIL examples/c/mfcalc/mfcalc.test (exit status: 1)

However, this is bad.

Actually I guess that only the tests from the examples were run, and not the 
main test suite.  So I guest if you run `make check-tests`, you'll get many 
many failures.

If I'm not mistaken the fourth test of mfcalc is:

a = 256
sqrt (a)

I doubt that the problem is in the parser itself, I venture it's sqrt that does 
not work properly.  Let's try these inputs please:

echo 'a = 256' | ./examples/c/mfcalc/mfcalc

{ echo 'a = 256'; echo 'a+a'; } | ./examples/c/mfcalc/mfcalc

echo 'sqrt(256)' | ./examples/c/mfcalc/mfcalc

I expect the last one to fail.  If it does, please compile (with the same flags 
as the one used by Bison) and run this C program:

#include 
#include 
#include 

int main (void)
{
 printf ("%f\n", sqrt (256.0));
}

If it does crash, then please try to see what needs to be done for it to work 
properly.



> FAIL: examples/c/glr/c++-types
> ==
> 
> c++-types: FAIL: 1 (expected status: 0, effective: 139)
> ./examples/test: line 50:  7949 Segmentation Fault  (core dumped)
> "$abs_medir/$me" "$@"
> FAIL examples/c/glr/c++-types.test (exit status: 1)


Gee...

Could you please replace examples/c/glr/c++-types/c++-types.test with the 
attached version, and run `make check-examples` again?  It should help us see 
which test case failed, and start tracking the failure.

Cheers!



c++-types.test
Description: Binary data

Re: SEGV in bison at src/lssi:297

[Akim Demaille]

Hi,

> Le 4 oct. 2021 à 16:57, Irfan Ariq  a écrit :
> 
> Hello,
> 
> We are currently working on fuzz testing feature, and we found a *SEGV* error
> on `bison`.
> 
> The stack traces are as follow:
> 
> ==29725==ERROR: AddressSanitizer: SEGV on unknown address 0x
>> (pc 0x5577cbd45d4a bp 0x7ffda59e9300 sp 0x7ffda59e7260 T0)
>> ==29725==The signal is caused by a READ memory access.
>> ==29725==Hint: address points to the zero page.
>>   #0 0x5577cbd45d49 in intersect src/lssi.c:297
>>   #1 0x5577cbd46333 in lssi_reverse_production src/lssi.c:361
>>   #2 0x5577cbd8ab93 in simulate_reduction src/parse-simulation.c:536
>>   #3 0x5577cbd166f2 in reduction_step src/counterexample.c:834
>>   #4 0x5577cbd1849e in generate_next_states src/counterexample.c:1047
>>   #5 0x5577cbd19417 in unifying_example src/counterexample.c:1182
>>   #6 0x5577cbd19e04 in counterexample_report src/counterexample.c:1277
>>   #7 0x5577cbd1ae65 in counterexample_report_reduce_reduce
>> src/counterexample.c:1350
>>   #8 0x5577cbd1b863 in counterexample_report_state
>> src/counterexample.c:1394
>>   #9 0x5577cbd952e0 in print_state src/print.c:366
>>   #10 0x5577cbd9617a in print_results src/print.c:473
>>   #11 0x5577cbd46ee4 in main src/main.c:188

Thanks for this bug report.  Usually bugs found by fuzzers are really 
uninteresting corner cases.  This one is different and is truly a bug somewhere 
in the counterexamples generation.

Once cleaned up from the remains of the exploration of the fuzzer, your example 
boils down to:

   %%
   input:
 line
   | input line
   :

   line:
 '\n'
   | exp '\n'
   ;

   exp: "num"
   a1:
   exp: exp '=' exp a1

   a2:
   exp: exp a2

and sadly `bison -Wcex` crashes on this.

Actually, there's something misleading in the above grammar, and I was very 
happy that -Wempty drew my attention to it.  Let's rewrite it this way:

%%
input:
 line
| input

line:
 %empty
| '\n'
| exp '\n'
;

exp: "num"
a1: %empty
exp: exp '=' exp a1

a2: %empty
exp: exp a2


I have started trying to understand the bug and how to fix it, but to no avail 
so far.  Maybe at some point I'll have to invoke the name of Vincent...

FWIW, I'm fairly confident there's only one bug, and your other reports are 
just other faces of the same coin.

Cheers!

Bison 3.8.2

[Akim Demaille]

Bison 3.8.2 fixes portability issues of bison on Cygwin, and brings
improvements to glr2.cc: support for custom error messages (`%define
parse.error custom`), and allow linking several parsers together.

==

Here are the compressed sources:
 https://ftp.gnu.org/gnu/bison/bison-3.8.2.tar.gz   (6.3MB)
 https://ftp.gnu.org/gnu/bison/bison-3.8.2.tar.lz   (3.1MB)
 https://ftp.gnu.org/gnu/bison/bison-3.8.2.tar.xz   (3.1MB)

Here are the GPG detached signatures[*]:
 https://ftp.gnu.org/gnu/bison/bison-3.8.2.tar.gz.sig
 https://ftp.gnu.org/gnu/bison/bison-3.8.2.tar.lz.sig
 https://ftp.gnu.org/gnu/bison/bison-3.8.2.tar.xz.sig

Use a mirror for higher download bandwidth:
 https://www.gnu.org/order/ftp.html

Here are the SHA1 and SHA256 checksums:

8e4b861eb765a7797ea0a6257ec600b3ff5ee37d  bison-3.8.2.tar.gz
BsnhO99+sk1M62tZIFpPZ8LH5yExGWREMP6C+9FKCrs  bison-3.8.2.tar.gz
36d4ade17c554a27551bdcf9e939ae438ad077c9  bison-3.8.2.tar.lz
/fmL/oKrsEo01DVnU/d0jbvS7xIhsfIChSorXOD3hTQ  bison-3.8.2.tar.lz
da1522a00f8c23c1abf69dbd2e99078d2a484b96  bison-3.8.2.tar.xz
m7oCFMz38QecXVkhAEUie89hlRmEDr+oDNOEnP9aW/I  bison-3.8.2.tar.xz

The SHA256 checksum is base64 encoded, instead of the
hexadecimal encoding that most checksum tools default to.

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

 gpg --verify bison-3.8.2.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

 gpg --keyserver keys.gnupg.net --recv-keys 0DDCAA3278D5264E

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
 Autoconf 2.71
 Automake 1.16b
 Flex 2.6.4
 Gettext 0.20.1.153-6c39c
 Gnulib v0.1-4916-g781845562

==

GNU Bison is a general-purpose parser generator that converts an annotated
context-free grammar into a deterministic LR or generalized LR (GLR) parser
employing LALR(1) parser tables.  Bison can also generate IELR(1) or
canonical LR(1) parser tables.  Once you are proficient with Bison, you can
use it to develop a wide range of language parsers, from those used in
simple desk calculators to complex programming languages.

Bison is upward compatible with Yacc: all properly-written Yacc grammars
work with Bison with no change.  Anyone familiar with Yacc should be able to
use Bison with little trouble.  You need to be fluent in C, C++, D or Java
programming in order to use Bison.

Bison and the parsers it generates are portable, they do not require any
specific compilers.

GNU Bison's home page is https://gnu.org/software/bison/.

==

NEWS

* Noteworthy changes in release 3.8.2 (2021-09-25) [stable]

 Fixed portability issues of bison on Cygwin.

 Improvements in glr2.cc: add support for custom error messages (`%define
 parse.error custom`), allow linking several parsers together.


* Noteworthy changes in release 3.8.1 (2021-09-11) [stable]

 The generation of prototypes for yylex and yyerror in Yacc mode is
 breaking existing grammar files.  To avoid breaking too many grammars, the
 prototypes are now generated when `-y/--yacc` is used *and* the
 `POSIXLY_CORRECT` environment variable is defined.

 Avoid using `-y`/`--yacc` simply to comply with Yacc's file name
 conventions, rather, use `-o y.tab.c`.  Autoconf's AC_PROG_YACC macro uses
 `-y`.  Avoid it if possible, for instance by using gnulib's gl_PROG_BISON.


* Noteworthy changes in release 3.8 (2021-09-07) [stable]

** Backward incompatible changes

 In conformance with the recommendations of the Graphviz team
 (https://marc.info/?l=graphviz-devel&m=129418103126092), `-g`/`--graph`
 now generates a *.gv file by default, instead of *.dot.  A transition
 started in Bison 3.4.

 To comply with the latest POSIX standard, in Yacc compatibility mode
 (options `-y`/`--yacc`) Bison now generates prototypes for yyerror and
 yylex.  In some situations, this is breaking compatibility: if the user
 has already declared these functions but with some differences (e.g., to
 declare them as static, or to use specific attributes), the generated
 parser will fail to compile.  To disable these prototypes, #define yyerror
 (to `yyerror`), and likewise for yylex.

** Deprecated features

 Support for the YYPRINT macro is removed. It worked only with yacc.c and
 only for tokens.  It was obsoleted by %printer, introduced in Bison 1.50
 (November 2002).

 It has always been recommended to prefer `%define api.value.type foo` to
 `#define YYSTYPE foo`.  The latter is supported in C for compatibility
 with Yacc, but not in C++.  Warnings are now issued if `#define YYSTYPE`
 is used in C++, and eventually support will be removed.

 In C++ code, prefer value_type to semantic_type 

Re: bison segv under Cygwin 64 at fatal-signal.c:318

[Akim Demaille]

Hi Brian,

> Le 25 sept. 2021 à 04:30, Brian Inglis  a 
> écrit :
> 
> On 2021-09-24 00:06, Brian Inglis wrote:
> 
> Rebuilt both latest and prerelease versions again, and all tests passed, 
> except 58 tests skipped as always, mainly D and Java.

Thanks a lot for checking.

> Santé!


:) :) :)

Re: libreoffice-7.2.0.4 and bison-3.8.1

[Akim Demaille]

Hi Ken,

> Le 25 sept. 2021 à 00:01, Ken Moffat  a écrit :
> 
> The errors start at
> /scratch/working/libreoffice-7.2.0.4/workdir/YaccTarget/connectivity/source/parse/sqlbison.cxx:
>  In function 'YYRESULTTAG yyuserAction(yyRuleNum, int, yyGLRStackItem*, 
> yyGLRStack*, long int, YYSTYPE*)':
> /scratch/working/libreoffice-7.2.0.4/workdir/YaccTarget/connectivity/source/parse/sqlbison.cxx:138:77:
>  error: 'yyn' was not declared in this scope; did you mean 'yyk'?
> 138 | #define SQL_NEW_RULEnewNode("", SQLNodeType::Rule, 
> yyr1[yyn])
> | 
> ^~~
> /scratch/working/libreoffice-7.2.0.4/workdir/YaccTarget/connectivity/source/parse/sqlbison.cxx:5484:42:
>  note: in expansion of macro 'SQL_NEW_RULE'
> 5484 | ((*yyvalp).pParseNode) = SQL_NEW_RULE;
> |  ^~~~
> 
> and of course there is much more of the same.  Searching hasn't
> found any similar error messages people have encountered.

Well, to know what's happening, I need more details, such as the input file.

I have found it here:

https://cgit.freedesktop.org/libreoffice/core/tree/connectivity/source/parse/sqlbison.y

First important thing: this is using glr.c (via %glr-parser).

Second important thing: someone already took care of this issue (but they 
didn't fix the comment):

https://cgit.freedesktop.org/libreoffice/core/tree/connectivity/source/parse/sqlbison.y#n77:

// yyi is the internal number of the rule that is currently being reduced
// This can be mapped to external rule number via the yyrmap.
#if defined YYBISON && YYBISON >= 30800
#define SQL_NEW_RULEnewNode("", SQLNodeType::Rule, 
yyr1[yyrule])
#define SQL_NEW_LISTRULEnewNode("", SQLNodeType::ListRule, 
yyr1[yyrule])
#define SQL_NEW_COMMALISTRULE   newNode("", SQLNodeType::CommaListRule, 
yyr1[yyrule])
#else
#define SQL_NEW_RULEnewNode("", SQLNodeType::Rule, 
yyr1[yyn])
#define SQL_NEW_LISTRULEnewNode("", SQLNodeType::ListRule, 
yyr1[yyn])
#define SQL_NEW_COMMALISTRULE   newNode("", SQLNodeType::CommaListRule, 
yyr1[yyn])
#endif

So case closed,

Cheers.

Re: bison segv under Cygwin 64 at fatal-signal.c:318

[Akim Demaille]

Hi Brian, [dropping bug-gnulib]

> Le 23 sept. 2021 à 19:54, Brian Inglis  a 
> écrit :
> 
> Sorry Akim,
> 
> I thought it was understood as confirmed in my last message.

You might very well have, but I didn't see it.  I saw a confirmation of success 
of a tarball that I interpreted as referring to Bruno's testdir-thread, but 
maybe you meant bison.


> This month I am busy dealing with a number of maintained package upgrade 
> build and test failures, and at least one other appears to also be due to an 
> obscure gnulib upgrade, after years of upgrades with at most only the 
> occasional package tweak required, requiring only basic knowledge of 
> autotools and none of gnulib!
> With each build requiring one or more hours for the package manager to 
> configure && make && make VPATH install && make check locally, for each arch, 
> then repeat for confirmation on the CI, it takes up a lot of free time.

I sure understand this.

> Your interim release includes the gnulib threadlib.m4 patch, so I had to 
> disable applying that, which is great.

Great.  Then we have Bison 3.8.2.  I'll do that this weekend.

> Cygwin only needs the revert-autoconf-upgrade patch, until an experienced 
> autotools user adopts the autotools as maintainer and provides upgrades.
> 
> The builds and tests run completely, and only the Doxygen tests fail, 
> although the doc builds work: not a significant concern.

I could have a look at that.  Please send the logs.

Cheers!

Re: 'make clean' not working

[Akim Demaille]

> Le 23 sept. 2021 à 09:31, Hans Åberg  a écrit :
> 
>> 
>> On 23 Sep 2021, at 07:31, Akim Demaille  wrote:
>> 
>> Hi Andrea,
>> 
>>> Le 21 sept. 2021 à 16:45, Andrea Monaco  a 
>>> écrit :
>>> 
>>> Hello,
>>> 
>>> I'm trying to rebuild a bison-3.8 release tree, but "make clean" aborts
>>> with this error:
>>> 
>>> CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/bash 
>>> '/root/bison-3.8/build-aux/missing' autoconf
>>> /root/bison-3.8/build-aux/missing: line 81: autoconf: command not found
>>> WARNING: 'autoconf' is missing on your system.
>>>   You should only need it if you modified 'configure.ac',
>>>   or m4 files included by it.
>>>   The 'autoconf' program is part of the GNU Autoconf package:
>>>   <https://www.gnu.org/software/autoconf/>
>>>   It also requires GNU m4 and Perl in order to run:
>>>   <https://www.gnu.org/software/m4/>
>>>   <https://www.perl.org/>
>>> Makefile:3894: recipe for target 'configure' failed
>>> make: *** [configure] Error 127
>>> 
>>> Is it expected?
> …
>> I can't reproduce your problem.  I have run configure, make, make clean,
>> make, without any problem (autoconf was not called).  Maybe something was
>> touched in your tree, I don't know.
> 
> I can reproduce the problem by modifying a file by touching it after the 
> build (tried with an m4 file) and making sure autoconf is not in the PATH.

That's expected: if you touch something that requires Autoconf, of course 
autoconf is fired.

Andrea said he did nothing like that, yet autoconf was called.

Re: 'make clean' not working

[Akim Demaille]

Hi Andraea,

> Le 21 sept. 2021 à 16:45, Andrea Monaco  a écrit 
> :
> 
> 
> Hello,
> 
> 
> I'm trying to rebuild a bison-3.8 release tree, but "make clean" aborts
> with this error:
> 
> 
> CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/bash 
> '/root/bison-3.8/build-aux/missing' autoconf
> /root/bison-3.8/build-aux/missing: line 81: autoconf: command not found
> WARNING: 'autoconf' is missing on your system.
> You should only need it if you modified 'configure.ac',
> or m4 files included by it.
> The 'autoconf' program is part of the GNU Autoconf package:
> 
> It also requires GNU m4 and Perl in order to run:
> 
> 
> Makefile:3894: recipe for target 'configure' failed
> make: *** [configure] Error 127
> 
> 
> Is it expected?

Nope.

> I don't think I modified configure.ac or any m4 file, so
> autoconf shouldn't be needed. Also, I think it's unusual in any case to
> require autotools for a basic operation such as make clean.

Yes, it is.

> I'm on a recent GNU/Hurd release, if it's important.

It shouldn't be.


I can't reproduce your problem.  I have run configure, make, make clean,
make, without any problem (autoconf was not called).  Maybe something was
touched in your tree, I don't know.

Cheers!

Re: bison segv under Cygwin 64 at fatal-signal.c:318

[Akim Demaille]

Hi Brian,

> Le 18 sept. 2021 à 19:04, Brian Inglis  a 
> écrit :
> 
> Thanks very much for your help Bruno, in diagnosing the issue correctly, and 
> providing the patch: I will ensure your patch comment gets prefixed into the 
> respun bison gnulib patch.
> 
> And thanks Akim for getting Bruno involved to address the right area.
> 
> Cygwin bison users will be happy having the latest build available.

I did not see the confirmation that the last tarball I made did work on your 
system.

https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.1.29-5c106.tar.gz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.1.29-5c106.tar.lz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.1.29-5c106.tar.xz

I might have missed it though.

Thanks Bruno, thanks Brian.

Re: bison segv under Cygwin 64 at fatal-signal.c:318

[Akim Demaille]

> Le 18 sept. 2021 à 15:22, Brian Inglis  a 
> écrit :
> 
> Patch would not apply as included gnulib and bison threadlib.m4 appear to be 
> from July and August.

FWIW, the last tarball I provided you with had a very recent gnulib.  The 
tarball below has

commit 7818455627c5e54813ac89924b8b67d0bc869146 (HEAD -> master, origin/master, 
origin/HEAD)
Author: Bruno Haible 
Date:   Fri Sep 17 22:22:50 2021 +0200

threadlib: Avoid crashes in thread-related functions on Cygwin 3.2.0.

Reported by Brian Inglis via Akim Demaille in
<https://lists.gnu.org/archive/html/bug-gnulib/2021-09/msg00063.html>.

* m4/threadlib.m4 (gl_WEAK_SYMBOLS): Force a "guessing no" result on
Cygwin.

at its top.

https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.1.29-5c106.tar.gz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.1.29-5c106.tar.lz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.1.29-5c106.tar.xz


Cheers!

Re: bison segv under Cygwin 64 at fatal-signal.c:318

[Akim Demaille]

Hi Brian,

Sorry for the slow response...  I'm still hoping we could reproduce this 
failure outside of Bison, so it took me quite some time to wrap a tarball of 
gnulib with its test suite.  My first attempt ran almost all day long (C-c to 
rescue), and generated something like a 1GB directory...  My second attempt, 
with 

./gnulib-tool --create-testdir --dir=./gnulib-with-tests --single-configure 
--with-c++-tests --without-privileged-tests 

was much faster: "just" about 2hrs :)

then, ./configure && make dist, and voilà, the following tarball:

https://www.lrde.epita.fr/~akim/private/bison/dummy-0.tar.gz

Could you please try it?  With some luck, some tests will fail, which we'll be 
able to report to gnulib.

Cheers!

Re: bison segv under Cygwin 64 at fatal-signal.c:318

[Akim Demaille]

Hi Brian,

Thanks for the very detailed bug report!  Before trying to go further, could 
you please try this tarball, which includes an updated gnulib?

https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.1.27-dd6e.tar.gz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.1.27-dd6e.tar.lz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.1.27-dd6e.tar.xz

If it does not work any better, we'll probably have to dive into gnulib.

Cheers!

Bison 3.8.1 released

[Akim Demaille]

Hi,

I'm very pleased to announce the release of Bison 3.8(.1), whose main
novelty is the D backend for deterministic parsers, contributed by
Adela Vais.  It supports all the bells and whistles of Bison's other
deterministic parsers, which include: pull/push interfaces, verbose
and custom error messages, lookahead correction, LALR(1), IELR(1),
canonical LR(1), token constructors, internationalization, locations,
printers, token and symbol prefixes, and more.

There are several other notable changes.  Please see the detailed NEWS
below for more details.

Cheers!

==

Here are the compressed sources:
  https://ftp.gnu.org/gnu/bison/bison-3.8.1.tar.gz   (6.1MB)
  https://ftp.gnu.org/gnu/bison/bison-3.8.1.tar.lz   (3.1MB)
  https://ftp.gnu.org/gnu/bison/bison-3.8.1.tar.xz   (3.1MB)

Here are the GPG detached signatures[*]:
  https://ftp.gnu.org/gnu/bison/bison-3.8.1.tar.gz.sig
  https://ftp.gnu.org/gnu/bison/bison-3.8.1.tar.lz.sig
  https://ftp.gnu.org/gnu/bison/bison-3.8.1.tar.xz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

Here are the SHA1 and SHA256 checksums:

79e97c868475c0e20286d62021f2a7cfd20610f7  bison-3.8.1.tar.gz
zjGKRxlhVft8JpErUTEC89DhR1fC5JXjRgh1e2EznFw  bison-3.8.1.tar.gz
e7fe4142c22ac5353ec4416652a56e9da951ffa5  bison-3.8.1.tar.lz
AJ2nWoBj4aO9IVRrN+UkISBWiR/CySr6EanzlphoIbg  bison-3.8.1.tar.lz
9772ea3130d6cbddaefe29a659698775a5701394  bison-3.8.1.tar.xz
MfxgJIiq1r3s8MzFVuD8cvxXzcWVz5I5jwIODPSYDxU  bison-3.8.1.tar.xz

The SHA256 checksum is base64 encoded, instead of the
hexadecimal encoding that most checksum tools default to.

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify bison-3.8.1.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver keys.gnupg.net --recv-keys 0DDCAA3278D5264E

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.71
  Automake 1.16b
  Flex 2.6.4
  Gettext 0.20.1.153-6c39c
  Gnulib v0.1-4853-g964ce0a92

==

GNU Bison is a general-purpose parser generator that converts an annotated
context-free grammar into a deterministic LR or generalized LR (GLR) parser
employing LALR(1) parser tables.  Bison can also generate IELR(1) or
canonical LR(1) parser tables.  Once you are proficient with Bison, you can
use it to develop a wide range of language parsers, from those used in
simple desk calculators to complex programming languages.

Bison is upward compatible with Yacc: all properly-written Yacc grammars
work with Bison with no change.  Anyone familiar with Yacc should be able to
use Bison with little trouble.  You need to be fluent in C, C++, D or Java
programming in order to use Bison.

Bison and the parsers it generates are portable, they do not require any
specific compilers.

GNU Bison's home page is https://gnu.org/software/bison/.

==

NEWS

* Noteworthy changes in release 3.8.1 (2021-09-11) [stable]

  The generation of prototypes for yylex and yyerror in Yacc mode is
  breaking existing grammar files.  To avoid breaking too many grammars, the
  prototypes are now generated when `-y/--yacc` is used *and* the
  `POSIXLY_CORRECT` environment variable is defined.

  Avoid using `-y`/`--yacc` simply to comply with Yacc's file name
  conventions, rather, use `-o y.tab.c`.  Autoconf's AC_PROG_YACC macro uses
  `-y`.  Avoid it if possible, for instance by using gnulib's gl_PROG_BISON.


* Noteworthy changes in release 3.8 (2021-09-07) [stable]

** Backward incompatible changes

  In conformance with the recommendations of the Graphviz team
  (https://marc.info/?l=graphviz-devel&m=129418103126092), `-g`/`--graph`
  now generates a *.gv file by default, instead of *.dot.  A transition
  started in Bison 3.4.

  To comply with the latest POSIX standard, in Yacc compatibility mode
  (options `-y`/`--yacc`) Bison now generates prototypes for yyerror and
  yylex.  In some situations, this is breaking compatibility: if the user
  has already declared these functions but with some differences (e.g., to
  declare them as static, or to use specific attributes), the generated
  parser will fail to compile.  To disable these prototypes, #define yyerror
  (to `yyerror`), and likewise for yylex.

** Deprecated features

  Support for the YYPRINT macro is removed. It worked only with yacc.c and
  only for tokens.  It was obsoleted by %printer, introduced in Bison 1.50
  (November 2002).

  It has always been recommended to prefer `%define api.value.type foo` to
  `#define YYSTYPE foo`.  The latter is supported in C for compatibility
  with Yacc, but not in C+

Re: bug#50469: [bison-3.8] bug or side effect to flex & automake

[Akim Demaille]

I propose the following patch to fix Automake's prototype of yyerror.

Cheers!

commit 38242845a146d6438e3f884100aa3e670142e393
Author: Akim Demaille 
Date:   Sat Sep 11 09:39:00 2021 +0200

tests: let yacc's yyerror take its argument as a const string

Some of yacc error messages are const strings, it makes no sense for yyerror
to take a `char*`, it should be a `const char*`.

Fixes all the failures reported by Kiyoshi Kanazawa.
<https://lists.gnu.org/r/bug-bison/2021-09/msg5.html>
See bug#50469.

* t/cond35.sh, t/silent-many-languages.sh, t/silent-yacc-headers.sh,
* t/silent-yacc.sh, t/subpkg-yacc.sh, t/suffix10.tap, t/yacc-basic.sh,
* t/yacc-clean.sh, t/yacc-d-basic.sh, t/yacc-d-vpath.sh,
* t/yacc-deleted-headers.sh, t/yacc-depend.sh, t/yacc-depend2.sh,
* t/yacc-dist-nobuild-subdir.sh, t/yacc-dist-nobuild.sh, t/yacc-dry.sh,
* t/yacc-line.sh, t/yacc-misc.sh, t/yacc-mix-c-cxx.sh, t/yacc-nodist.sh,
* t/yacc-pr204.sh, t/yacc-subdir.sh, t/yacc-vpath.sh,
* t/yflags-cmdline-override.sh, t/yflags-force-override.sh
(yyerror): From `char*` to `const char*`.
Enforce consistency: prefer `{}` to `{ return; }`.

diff --git a/t/cond35.sh b/t/cond35.sh
index 786bb3cf5..37a25eae3 100644
--- a/t/cond35.sh
+++ b/t/cond35.sh
@@ -73,7 +73,7 @@ END
 
 cat > tparse.y << 'END'
 %{
-void yyerror (char *s) {}
+void yyerror (const char *s) {}
 %}
 %token EOF
 %%
diff --git a/t/silent-many-languages.sh b/t/silent-many-languages.sh
index edb59e7c9..9361b2bf8 100644
--- a/t/silent-many-languages.sh
+++ b/t/silent-many-languages.sh
@@ -188,7 +188,7 @@ EOF
 cat > foo6.y <<'EOF'
 %{
 extern int yylex (void);
-void yyerror (char *s) {}
+void yyerror (const char *s) {}
 %}
 %token EOF
 %%
diff --git a/t/silent-yacc-headers.sh b/t/silent-yacc-headers.sh
index c10fc5cc4..0582973f5 100644
--- a/t/silent-yacc-headers.sh
+++ b/t/silent-yacc-headers.sh
@@ -39,7 +39,7 @@ EOF
 
 cat > parse.y <<'EOF'
 %{
-void yyerror (char *s) { return; }
+void yyerror (const char *s) {}
 int yylex (void) { return 0; }
 int main (void) { return 0; }
 %}
diff --git a/t/silent-yacc.sh b/t/silent-yacc.sh
index d11fc2ab2..3254d4014 100644
--- a/t/silent-yacc.sh
+++ b/t/silent-yacc.sh
@@ -50,7 +50,7 @@ EOF
 
 cat > foo.y <<'EOF'
 %{
-void yyerror (char *s) { return; }
+void yyerror (const char *s) {}
 int yylex (void) { return 0; }
 int main (void) { return 0; }
 %}
diff --git a/t/subpkg-yacc.sh b/t/subpkg-yacc.sh
index eac56cb1f..9e31c5151 100644
--- a/t/subpkg-yacc.sh
+++ b/t/subpkg-yacc.sh
@@ -88,7 +88,7 @@ EOF
 cat > lib/foo.y << 'END'
 %{
 int yylex (void) { return 0; }
-void yyerror (char *s) {}
+void yyerror (const char *s) {}
 %}
 %%
 foobar : 'f' 'o' 'o' 'b' 'a' 'r' {};
diff --git a/t/suffix10.tap b/t/suffix10.tap
index 2258120e7..d233c6c21 100644
--- a/t/suffix10.tap
+++ b/t/suffix10.tap
@@ -39,7 +39,7 @@ libfoo_la_SOURCES = foo.x_
:; { echo '/* autogenerated */' \
  && echo '%{' \
  && echo 'int yylex () {return 0;}' \
- && echo 'void yyerror (char *s) {}' \
+ && echo 'void yyerror (const char *s) {}' \
  && echo '%}' \
  && echo '%%' \
  && echo "foobar : 'f' 'o' 'o' 'b' 'a' 'r' {};" \
diff --git a/t/yacc-basic.sh b/t/yacc-basic.sh
index cfbaa1957..d50157099 100644
--- a/t/yacc-basic.sh
+++ b/t/yacc-basic.sh
@@ -42,7 +42,7 @@ cat > parse.y << 'END'
 #include 
 #include 
 int yylex () { return getchar (); }
-void yyerror (char *s) {}
+void yyerror (const char *s) {}
 %}
 %%
 a : 'a' { exit(0); };
diff --git a/t/yacc-bison-skeleton.sh b/t/yacc-bison-skeleton.sh
index 60e85dd85..6069489da 100644
--- a/t/yacc-bison-skeleton.sh
+++ b/t/yacc-bison-skeleton.sh
@@ -44,7 +44,7 @@ void yyerror (const char *s);
 foobar : 'f' 'o' 'o' 'b' 'a' 'r' {};
 %%
 int yylex () { return 0; }
-void yyerror (const char *s) { return; }
+void yyerror (const char *s) {}
 END
 
 cat > foo.c << 'END'
diff --git a/t/yacc-clean-cxx.sh b/t/yacc-clean-cxx.sh
index fe2795894..811d46939 100644
--- a/t/yacc-clean-cxx.sh
+++ b/t/yacc-clean-cxx.sh
@@ -70,7 +70,7 @@ cat > sub1/parsefoo.yxx << 'END'
 #include 
 // "std::" qualification required by Sun C++ 5.9.
 int yylex (void) { return std::getchar (); }
-void yyerror (const char *s) { return; }
+void yyerror (const char *s) {}
 %}
 %%
 x : 'x' { };
diff --git a/t/yacc-clean.sh b/t/yacc-clean.sh
index 9ea13a704..c248e5c89 100644
--- a/t/yacc-clean.sh
+++ b/t/yacc-clean.sh
@@ -66,7 +66,7 @@ EN

Re: bug#50469: [bison-3.8] bug or side effect to flex & automake

[Akim Demaille]

Hi Sam,

> Le 10 sept. 2021 à 18:51, Sam James  a écrit :
> 
> Thanks your work on this! Brief comments on version changes:
> 
>> diff --git a/src/parse-gram.c b/src/parse-gram.c
>> index 95fe43e0..3bc44dbd 100644
>> --- a/src/parse-gram.c
>> +++ b/src/parse-gram.c
>> @@ -1,4 +1,4 @@
>> -/* A Bison parser, made by GNU Bison 3.7.5.297-b46a.  */
>> +/* A Bison parser, made by GNU Bison 3.8.  */
> 
> 3.8 -> 3.8.1, but I guess this will be automatic :)

This is indeed automatic, but refers to the version of Bison that was used to 
generate bison's parser.  So there's nothing wrong here.

Cheers!

Re: bug#50469: [bison-3.8] bug or side effect to flex & automake

[Akim Demaille]

Hi all,

> Le 9 sept. 2021 à 07:10, Akim Demaille  a écrit :
> 
> Hi!
> 
>> Le 9 sept. 2021 à 00:32, Paul Eggert  a écrit :
>> 
>> On 9/8/21 2:18 PM, Karl Berry wrote:
>>> Just an idea that I don't expect you to adopt, but just to mention --
>>> you could only institute the breaking change if POSIXLY_CORRECT.  That's
>>> why POSIXLY_CORRECT exists. -k
>> 
>> I like this idea. It insulates us against POSIX decisions and/or indecisions 
>> in this area.
> 
> Good with me.  I'll implement that, thanks Karl!

I will soon push the following commit, and wrap a 3.8.1.  Comments most welcome!

Cheers!

(a tarball is available here:

  https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.3-84b7.tar.gz
  https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.3-84b7.tar.lz
  https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.3-84b7.tar.xz)

commit 5623009a76dd8f04b5c19bb5f6ae8e2bf258a51e
Author: Akim Demaille 
Date:   Fri Sep 10 06:48:02 2021 +0200

yacc: declare yyerror/yylex only when POSIXLY_CORRECT

The recent changes to comply with POSIX are breaking Automake's test
suite.
Reported by Kiyoshi Kanazawa.
<https://lists.gnu.org/r/bug-bison/2021-09/msg5.html>

To limit the impact of POSIX changes, bind them to $POSIXLY_CORRECT.
Suggested by Karl Berry.
<https://lists.gnu.org/r/bug-bison/2021-09/msg9.html>

The existing `maintainer-check-posix` Make target checks these
changes.

* src/getargs.h, src/getargs.c (set_yacc): New.
Use it.
* data/skeletons/bison.m4 (b4_posix_if): New.
* data/skeletons/yacc.c (b4_declare_yyerror_and_yylex): Use it.
* doc/bison.texi, tests/local.at: Adjust.

diff --git a/NEWS b/NEWS
index 71c4ae7c..8300548a 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,15 @@ GNU Bison NEWS
 
 * Noteworthy changes in release ?.? (-??-??) [?]
 
+  The generation of prototypes for yylex and yyerror in Yacc mode is
+  breaking existing grammar files.
+
+  **Please don't use `-y`/`--yacc` simply to comply with Yacc's file name
+conventions, rather, use `-o y.tab.c`.**
+
+  To avoid breaking too many grammars, the prototypes are now generated when
+  `-y/--yacc` is used *and* the `POSIXLY_CORRECT` environment variable is
+  defined.
 
 * Noteworthy changes in release 3.8 (2021-09-07) [stable]
 
@@ -4706,7 +4715,7 @@ LocalWords:  autocompletion bistromathic submessages 
Cayuela lexcalc hoc
 LocalWords:  yytoken YYUNDEF YYerror basename Automake's UTF ifdef ffile
 LocalWords:  gotos readline Imbimbo Wcounterexamples Wcex Nonunifying rcex
 LocalWords:  Vais xsltproc YYNOMEM YYLOCATION signedness YYBISON MITRE's
-LocalWords:  libreadline YYMALLOC YYFREE MSVC redefinitions
+LocalWords:  libreadline YYMALLOC YYFREE MSVC redefinitions POSIXLY
 
 Local Variables:
 ispell-dictionary: "american"
diff --git a/data/skeletons/bison.m4 b/data/skeletons/bison.m4
index 488ad610..037dadcf 100644
--- a/data/skeletons/bison.m4
+++ b/data/skeletons/bison.m4
@@ -1110,6 +1110,7 @@ b4_percent_define_if_define([token_ctor], 
[api.token.constructor])
 b4_percent_define_if_define([locations]) # Whether locations are tracked.
 b4_percent_define_if_define([parse.assert])
 b4_percent_define_if_define([parse.trace])
+b4_percent_define_if_define([posix])
 
 
 # b4_bison_locations_if([IF-TRUE])
diff --git a/data/skeletons/yacc.c b/data/skeletons/yacc.c
index d50f3aaf..64b9ac62 100644
--- a/data/skeletons/yacc.c
+++ b/data/skeletons/yacc.c
@@ -367,7 +367,7 @@ m4_define([b4_declare_yyparse],
 # Comply with POSIX Yacc.
 # <https://austingroupbugs.net/view.php?id=1388#c5220>
 m4_define([b4_declare_yyerror_and_yylex],
-[b4_yacc_if([[#if !defined ]b4_prefix[error && !defined 
]b4_api_PREFIX[ERROR_IS_DECLARED
+[b4_posix_if([[#if !defined ]b4_prefix[error && !defined 
]b4_api_PREFIX[ERROR_IS_DECLARED
 ]b4_function_declare([b4_prefix[error]], void, b4_yyerror_formals)[
 #endif
 #if !defined ]b4_prefix[lex && !defined ]b4_api_PREFIX[LEX_IS_DECLARED
diff --git a/doc/bison.texi b/doc/bison.texi
index 9a02fb95..a559649c 100644
--- a/doc/bison.texi
+++ b/doc/bison.texi
@@ -11877,7 +11877,10 @@ @node Tuning the Parser
 Generate @code{#define} statements in addition to an @code{enum} to
 associate token codes with token kind names.
 @item
-Generate prototypes for @code{yyerror} and @code{yylex} (since Bison 3.8):
+If the @code{POSIXLY_CORRECT} environment variable is defined, generate
+prototypes for @code{yyerror} and @code{yylex}@footnote{See
+@url{https://austingroupbugs.net/view.php?id=1388#c5220}.} (since Bison
+3.8):
 @example
 int yylex (void);
 void yyerror (const char *);
diff --git a/src/getargs.c b/src/getargs.c
index 697ad7eb..ef559296 100644
--- a/src/getargs.c
+++ b/src/getargs.c
@@ -71,6 +71,18 @@ struct bison_language const *language = &valid_languages[0];
 
 typedef int* (

Re: [bison-3.8] Link error on HP-UX 11i

[Akim Demaille]

Hi Larkin,

> Le 10 sept. 2021 à 03:45, Larkin Nickle  a écrit :
> 
> Tested with GNU Bison 3.8. This happens with GCC or HP aCC. HP-UX 11i does 
> not have strtol defined and this results in a link error.
> 
>  CCLD src/bison
> /usr/ccs/bin/ld: Unsatisfied symbols:
>   strtof (first referenced in src/bison-counterexample.o) (code)
> make[2]: *** [Makefile:4831: src/bison] Error 1

Doh...  Thanks for the report!  I'm installing the patch below.  Could you 
please confirm that this tarball works?  Thanks in advance!

https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.3-84b7.tar.gz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.3-84b7.tar.lz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.8.3-84b7.tar.xz


commit 84b78cef9bd913013da0e8995a1b1763b3399d01
Author: Akim Demaille 
Date:   Fri Sep 10 11:08:23 2021 +0200

portability: don't use strtof

It is not available on HP-UX 11i.
Reported by Larkin Nickle .
<https://lists.gnu.org/r/bug-bison/2021-09/msg00012.html>

Gnulib provides no replacement, but anyway we should be using doubles,
since difftime uses doubles.

* bootstrap.conf: We want portability on stdtod.
* src/counterexample.c: Use double, not float, for time measurements.

diff --git a/bootstrap.conf b/bootstrap.conf
index 6f1a3fc6..1d8614e7 100644
--- a/bootstrap.conf
+++ b/bootstrap.conf
@@ -45,7 +45,7 @@ gnulib_modules='
   realloc-posix
   relocatable-prog relocatable-script
   rename
-  spawn-pipe stdbool stpcpy stpncpy strdup-posix strerror strverscmp
+  spawn-pipe stdbool stpcpy stpncpy strdup-posix strerror strtod strverscmp
   sys_ioctl
   termios
   timevar
diff --git a/lib/.gitignore b/lib/.gitignore
index c132149f..590b7942 100644
--- a/lib/.gitignore
+++ b/lib/.gitignore
@@ -391,6 +391,7 @@
 /stripslash.c
 /strndup.c
 /strnlen.c
+/strtod.c
 /strverscmp.c
 /sys
 /sys_ioctl.in.h
diff --git a/m4/.gitignore b/m4/.gitignore
index a7b5095c..f1554d4d 100644
--- a/m4/.gitignore
+++ b/m4/.gitignore
@@ -216,6 +216,7 @@
 /strings_h.m4
 /strndup.m4
 /strnlen.m4
+/strtod.m4
 /strverscmp.m4
 /sys_ioctl_h.m4
 /sys_resource_h.m4
diff --git a/src/counterexample.c b/src/counterexample.c
index f6f85fb1..ec427c1e 100644
--- a/src/counterexample.c
+++ b/src/counterexample.c
@@ -57,12 +57,12 @@
 
 /** The time limit before printing an assurance message to the user to
  *  indicate that the search is still running. */
-#define ASSURANCE_LIMIT 2.0f
+#define ASSURANCE_LIMIT 2.0
 
 /* The time limit before giving up looking for unifying counterexample. */
-static float time_limit = 5.0f;
+static double time_limit = 5.0;
 
-#define CUMULATIVE_TIME_LIMIT 120.0f
+#define CUMULATIVE_TIME_LIMIT 120.0
 
 // This is the fastest way to get the tail node from the gl_list API.
 static gl_list_node_t
@@ -1164,7 +1164,7 @@ unifying_example (state_item_number itm1,
 }
   if (TIME_LIMIT_ENFORCED)
 {
-  float time_passed = difftime (time (NULL), start);
+  double time_passed = difftime (time (NULL), start);
   if (!assurance_printed && time_passed > ASSURANCE_LIMIT
   && stage3result)
 {
@@ -1218,7 +1218,7 @@ counterexample_init (void)
 if (cp)
   {
 char *end = NULL;
-float v = strtof (cp, &end);
+double v = strtod (cp, &end);
 if (*end == '\0' && errno == 0)
   time_limit = v;
   }

Re: bug#50469: [bison-3.8] bug or side effect to flex & automake

[Akim Demaille]

Hi!

> Le 9 sept. 2021 à 00:32, Paul Eggert  a écrit :
> 
> On 9/8/21 2:18 PM, Karl Berry wrote:
>> Just an idea that I don't expect you to adopt, but just to mention --
>> you could only institute the breaking change if POSIXLY_CORRECT.  That's
>> why POSIXLY_CORRECT exists. -k
> 
> I like this idea. It insulates us against POSIX decisions and/or indecisions 
> in this area.

Good with me.  I'll implement that, thanks Karl!


>> P.S. Is this related to Automake? If it is, as with Autoconf, there
>> needs to be new volunteer effort to do what's needed. I can't take it on
>> and unfortunately no one else has come forward.

IMHO the prototypes in Automake's test suite are wrong: I see no reason for
yyerror to take a char *, and we do pass constant strings to it, so I'm
quite surprised that there are no compiling errors about it.  We should
fix that.  I can do that.

On the Autoconf side, we dearly need some AC_CHECK_BISON.  It is really a
pity that everybody uses AC_CHECK_YACC, which seriously degrades the point
of using bison.

Cheers!

Re: [bison-3.8] bug or side effect to flex & automake

[Akim Demaille]

Hi Paul,

Thanks for the quick answer.

> Le 8 sept. 2021 à 08:33, Paul Eggert  a écrit :
> 
> On 9/7/21 10:31 PM, Akim Demaille wrote:
>> However, I don't see a published version of the POSIX Yacc "specs" that 
>> includes these changes.
> 
> The next POSIX revision is targeted for 2022, according to 
> <http://www.opengroup.org/austin/papers/posix_faq.html>.

Thanks.  Last time we discussed about this, the conclusion was that Bison 
should implement this now.  Should we rather backtrack and postpone until the 
revision is published?  I have published 3.8, but not announced it yet, so it 
probably doesn't spread too fast.  I could revert that, and publish 3.8.1 very 
soon.

> I suppose there is still opportunity to fix POSIX before the next revision 
> comes out, but someone would have to create a bug report and submit it to the 
> Austin Group.

I don't see why this time they would agree not to change anything.  I believe 
we already clearly stated that this is a breaking change.

Re: [bison-3.8] bug or side effect to flex & automake

[Akim Demaille]

Hi Kiyoshi,

> Le 8 sept. 2021 à 04:11, Kiyoshi KANAZAWA  a 
> écrit :
> 
> Hello,
> 
> Installed bison-3.8, but I'm afraid it is has bug or side effect to 
> flex-2.6.4 & automake-1.16.4.
> 
> $ uname -a
> SunOS hidden 5.11 11.3 i86pc i386 i86pc
> 
> $ gcc --version
> gcc (GCC) 10.3.0
> 
> After installing bison-3.8, detected the following make check error, which 
> did not with bison-3.7.6.
> 
> (1) flex-2.6.4
> mv -f $depbase.Tpo $depbase.Po
> depbase=`echo bison_nr_parser.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
> gcc -m32 -DHAVE_CONFIG_H -I. -I../src  -I../src -I../src-MT 
> bison_nr_parser.o -MD -MP -MF $depbase.Tpo -c -o bison_nr_parser.o 
> bison_nr_parser.c &&\
> mv -f $depbase.Tpo $depbase.Po
> bison_nr_parser.c:70:25: error: conflicting types for 'testerror'
>70 | #define yyerror testerror
>   | ^
> bison_nr_parser.y:40:5: note: in expansion of macro 'yyerror'
>40 | int yyerror(const char* msg);
>   | ^~~
> In file included from bison_nr_parser.y:33:
> bison_nr_parser.h:110:6: note: previous declaration of 'testerror' was here
>   110 | void testerror (const YYLTYPE *yyllocp, const char *msg);
>   |  ^
> bison_nr_parser.c:70:25: error: conflicting types for 'testerror'
>70 | #define yyerror testerror
>   | ^
> bison_nr_parser.y:93:5: note: in expansion of macro 'yyerror'
>93 | int yyerror(const char* msg) {
>   | ^~~
> In file included from bison_nr_parser.y:33:
> bison_nr_parser.h:110:6: note: previous declaration of 'testerror' was here
>   110 | void testerror (const YYLTYPE *yyllocp, const char *msg);
>   |  ^
> make[2]: *** [Makefile:2076: bison_nr_parser.o] Error 1
> 
> And so on.

Well, I'm afraid this change was made on purpose, to conform with the 
(unfortunate IMHO) latest specs of POSIX Yacc.  See the NEWS file:

  To comply with the latest POSIX standard, in Yacc compatibility mode
  (options `-y`/`--yacc`) Bison now generates prototypes for yyerror and
  yylex.  In some situations, this is breaking compatibility: if the user
  has already declared these functions but with some differences (e.g., to
  declare them as static, or to use specific attributes), the generated
  parser will fail to compile.  To disable these prototypes, #define yyerror
  (to `yyerror`), and likewise for yylex.

This change in POSIX was discussed here: 
https://austingroupbugs.net/view.php?id=1388#c5220.  However, I don't see a 
published version of the POSIX Yacc "specs" that includes these changes.  The 
latest edition appears to be from 2018 
(https://pubs.opengroup.org/onlinepubs/9699919799/).


That being said, several tests can be easily adjusted.  For instance 
t/silent-yacc fails because it provides this implementation of yyerror:

void yyerror (char *s) { return; }

while the expected signature is

void yyerror (const char *s);

I have not checked all the tests, but I suspect that many of them can be easily 
fixed.

I don't see you, Kiyoshi, in the logs of Automake, so I suppose you are not 
maintaining/contributing to Automake.  Yet we need to fix these tests (I'd be 
happy to get feedback from Automake's team on this).


This is definitely a change that may break compatibility in some cases, but 
it's out of our control: POSIX decided, we just comply.  One big problem with 
the Autotools as of today is that they promote the use of macros/build rules 
for Yacc, not for Bison.  Because of this, bison is almost always called with 
`-y/--yacc`, which not only forces bison to generate y.tab.c instead of 
foo.tab.c, but also forces it to enter the POSIX Yacc mode (and also to spit 
tons of warnings for each GNU Bison feature which is not part of POSIX Yacc).


Cheers!

Bison 3.7.91 (beta)

[Akim Demaille]

Hi,

I'm very pleased to announce the forthcoming (next week?) release of
Bison 3.8, whose main novelty is the D backend for deterministic
parsers, contributed by Adela Vais.  It supports all the bells and
whistles of Bison's other deterministic parsers, which include:
pull/push interfaces, verbose and custom error messages, lookahead
correction, LALR(1), IELR(1), canonical LR(1), token constructors,
internationalization, locations, printers, token and symbol prefixes,
and more.

There are several other notable changes.  Please see the detailed NEWS
below for more details.

To make sure Bison 3.8 is a successful release, we need feedback on
this beta.  *Please test it!*

Cheers!

==

Here are the compressed sources:
  https://alpha.gnu.org/gnu/bison/bison-3.7.91.tar.gz   (6.1MB)
  https://alpha.gnu.org/gnu/bison/bison-3.7.91.tar.lz   (3.1MB)
  https://alpha.gnu.org/gnu/bison/bison-3.7.91.tar.xz   (3.1MB)

Here are the GPG detached signatures[*]:
  https://alpha.gnu.org/gnu/bison/bison-3.7.91.tar.gz.sig
  https://alpha.gnu.org/gnu/bison/bison-3.7.91.tar.lz.sig
  https://alpha.gnu.org/gnu/bison/bison-3.7.91.tar.xz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

Here are the SHA1 and SHA256 checksums:

aa743c5a4d931108bdd419a62a906d8df2c7f6e3  bison-3.7.91.tar.gz
2q4fi2S8shDkuS0r7IOSKLpYx+leEJ3+WF9PQM9dWuw  bison-3.7.91.tar.gz
573ee882470cc1404c3511691b93ad476ac0fe5e  bison-3.7.91.tar.lz
SLtogiFh1hBldiExE3o7D7yMy9/o4meB4HICGa0XVa8  bison-3.7.91.tar.lz
7faa822a1a4c3564aa9d73c03c7277dcb847d4de  bison-3.7.91.tar.xz
NvESKtwyWp6rVBN6+fT9HjOdCfLFajlRkTF6Y5UzYWU  bison-3.7.91.tar.xz

The SHA256 checksum is base64 encoded, instead of the
hexadecimal encoding that most checksum tools default to.

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify bison-3.7.91.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver keys.gnupg.net --recv-keys 0DDCAA3278D5264E

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.71
  Automake 1.16b
  Flex 2.6.4
  Gettext 0.20.1.153-6c39c
  Gnulib v0.1-4853-g964ce0a92

NEWS

* Noteworthy changes in release 3.7.91 (2021-09-02) [beta]

  Portability issues in the test suite and in C++ skeletons.

* Noteworthy changes in release 3.7.90 (2021-08-13) [beta]

** Backward incompatible changes

  In conformance with the recommendations of the Graphviz team
  (https://marc.info/?l=graphviz-devel&m=129418103126092), `-g`/`--graph`
  now generates a *.gv file by default, instead of *.dot.  A transition
  started in Bison 3.4.

  To comply with the latest POSIX standard, in Yacc compatibility mode
  (options `-y`/`--yacc`) Bison now generates prototypes for yyerror and
  yylex.  In some situations, this is breaking compatibility: if the user
  has already declared these functions but with some differences (e.g., to
  declare them as static, or to use specific attributes), the generated
  parser will fail to compile.  To disable these prototypes, #define yyerror
  (to `yyerror`), and likewise for yylex.

** Deprecated features

  Support for the YYPRINT macro is removed. It worked only with yacc.c and
  only for tokens.  It was obsoleted by %printer, introduced in Bison 1.50
  (November 2002).

  It has always been recommended to prefer `%define api.value.type foo` to
  `#define YYSTYPE foo`.  The latter is supported in C for compatibility
  with Yacc, but not in C++.  Warnings are now issued if `#define YYSTYPE`
  is used in C++, and eventually support will be removed.

  In C++ code, prefer value_type to semantic_type to denote the semantic
  value type, which is specified by the `api.value.type` %define variable.

** New features

*** A skeleton for the D programming language

  The "lalr1.d" skeleton is now officially part of Bison.

  It was originally contributed by Oliver Mangold, based on Paolo Bonzini's
  lalr1.java, and was improved by H. S. Teoh.  Adela Vais then took over
  maintenance and invested a lot of efforts to complete, test and document
  it.

  It now supports all the bells and whistles of the other deterministic
  parsers, which include: pull/push interfaces, verbose and custom error
  messages, lookahead correction, token constructors, internationalization,
  locations, printers, token and symbol prefixes, etc.

  Two examples demonstrate the D parsers: a basic one (examples/d/simple),
  and an advanced one (examples/d/calc).

*** Option -H, --header and directive %header

  The option `-H`/`--header` supersedes the option `--defines`, and the
  directive %header supersedes %defines.  Both `--defines` and `%defines`
  are, of course, maintained for backward compatibility.

**

Re: Bison 3.7.90 (beta): test 4

[Akim Demaille]

> Le 2 sept. 2021 à 07:20, Akim Demaille  a écrit :
> 
> That being said, it is again a test about a feature I don't want to make 
> public in 3.8, so I will disable this test

Done:

commit da0f993fb950df1030725ce51ac766dd7425ff80
Author: Akim Demaille 
Date:   Thu Sep 2 07:24:01 2021 +0200

tests: disable "Multistart reports"

For some reason this test fails on Solaris/x86.  But multistart is not
part of 3.8, so we can postpone the debugging of this issue.
Reported by Dagobert Michelsen.
<https://lists.gnu.org/r/bug-bison/2021-08/msg00027.html>

* tests/report.at (Multistart reports): Comment out.

diff --git a/TODO b/TODO
index a4a91bb0..d89894c7 100644
--- a/TODO
+++ b/TODO
@@ -518,6 +518,7 @@ https://lists.gnu.org/r/bison-patches/2013-09/msg0.html 
and following
 
 ** Multiple start symbols
 Revert a70e75b8a41755ab96ab211a0ea111ac68a4aadd.
+Revert tests: disable "Multistart reports".
 
 Would be very useful when parsing closely related languages.  The idea is to
 declare several start symbols, for instance
diff --git a/tests/report.at b/tests/report.at
index 77b8af4e..83c418f6 100644
--- a/tests/report.at
+++ b/tests/report.at
@@ -1516,1601 +1516,1604 @@ AT_SETUP([Reports])
 AT_CLEANUP
 
 
-##  ##
-## Multistart reports.  ##
-##  ##
-
-AT_SETUP([Multistart reports])
-
-AT_KEYWORDS([multistart report])
-
-AT_BISON_OPTION_PUSHDEFS
-AT_DATA([input.yy],
-[[%token
-  END  0  "end of file"
-  ASSIGN  ":="
-  INCR"incr"
-;
-
-%token  IDENTIFIER "identifier"
-%type  id
-%token  NUMBER "number"
-%type   exp
-
-%%
-%start unit assignments exp;
-unit: assignments exp  { driver.result = $2; };
-
-assignments:
-  %empty {}
-| assignments assignment {};
-
-assignment:
-  id ":=" exp { driver.variables[$id] = $exp; };
-
-id:
-  "identifier";
-
-exp:
-  "incr" exp { $$ = 1; } { $$ = 10; } exp   { $$ = $2 + $3 + $4 + 
$5; }
-| "(" exp ")"   { std::swap ($$, $2); }
-| "identifier"  { $$ = driver.variables[$1]; }
-| "number"  { std::swap ($$, $1); };
-]])
-
-AT_BISON_CHECK([-o input.cc -rall --graph=input.gv --xml input.yy])
-
-# Check the contents of the report.
-AT_CHECK([cat input.output], [],
-[[Grammar
-
-0 $accept: YY_PARSE_unit unit "end of file"
-1| YY_PARSE_assignments assignments "end of file"
-2| YY_PARSE_exp exp "end of file"
-
-3 unit: assignments exp
-
-4 assignments: %empty
-5| assignments assignment
-
-6 assignment: id ":=" exp
-
-7 id: "identifier"
-
-8 @1: %empty
-
-9 @2: %empty
-
-   10 exp: "incr" exp @1 @2 exp
-   11| "(" exp ")"
-   12| "identifier"
-   13| "number"
-
-
-Terminals, with rules where they appear
-
-"end of file" (0) 0 1 2
-error (256)
-":=" (258) 6
-"incr" (259) 10
-"identifier"  (260) 7 12
-"number"  (261) 13
-YY_PARSE_unit (262) 0
-YY_PARSE_assignments (263) 1
-YY_PARSE_exp (264) 2
-"(" (265) 11
-")" (266) 11
-
-
-Nonterminals, with rules where they appear
-
-$accept (12)
-on left: 0 1 2
-unit (13)
-on left: 3
-on right: 0
-assignments (14)
-on left: 4 5
-on right: 1 3 5
-assignment (15)
-on left: 6
-on right: 5
-id  (16)
-on left: 7
-on right: 6
-exp  (17)
-on left: 10 11 12 13
-on right: 2 3 6 10 11
-@1  (18)
-on left: 8
-on right: 10
-@2  (19)
-on left: 9
-on right: 10
-
-
-State 0
-
-0 $accept: . YY_PARSE_unit unit "end of file"
-1| . YY_PARSE_assignments assignments "end of file"
-2| . YY_PARSE_exp exp "end of file"
-
-YY_PARSE_unit shift, and go to state 1
-YY_PARSE_assignments  shift, and go to state 2
-YY_PARSE_exp  shift, and go to state 3
-
-
-State 1
-
-0 $accept: YY_PARSE_unit . unit "end of file"
-3 unit: . assignments exp
-4 assignments: %empty .
-5| . assignments assignment
-
-$default  reduce using rule 4 (assignments)
-
-unit go to state 4
-assignments  go to state 5
-
-
-State 2
-
-1 $accept: YY_PARSE_assignments . assignments "end of file"
-4 assignments: %empty .
-5| . assignments assignment
-
-$default  reduce using rule 4 (assignments)
-
-assignments  go to state 6
-
-
-State 3
-
-2 $accept: YY_PARSE_exp . exp "end of file"
-   10 exp: . "incr" exp @1 @2 exp
-   11| . "(" exp ")"
-   12| . "identifier"
-   13 

Re: Bison 3.7.90 (beta): test 4

[Akim Demaille]

Hi Dagobert,

> Le 29 août 2021 à 21:49, Dagobert Michelsen  a écrit :
> 
> Hi Akim,
> 
> Am 29.08.2021 um 10:08 schrieb Akim Demaille :
>> So it is indeed a problem with sed: neither sed.1.log nor sed.2.log have the 
>> expected NUL byte.
>> 
>> So I'm installing the patch below.  Could you please check this tarball:
>> 
>> https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.90.8-8556.tar.gz
>> https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.90.8-8556.tar.lz
>> https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.90.8-8556.tar.xz
> 
> Sparc now compiles and tests cleanly, but x86 shows one error in the 
> testsuite:
> 
>   Subject: [GNU Bison 3.7.90.8-8556] testsuite: 203 failed

Wow...  This is very very weird.

> 203. report.at:1523: testing Multistart reports ...
> ./report.at:1561: COLUMNS=1000; export COLUMNS; NO_TERM_HYPERLINKS=1; export 
> NO_TERM_HYPERLINKS;  bison --color=no -fno-caret -o input.cc -rall 
> --graph=input.gv --xml input.yy
> --- /dev/null 2021-08-29 20:48:39.0 +0200
> +++ 
> /home/dam/mgar/pkg/bison/trunk/work/solaris10-i386/build-isa-pentium_pro/bison-3.7.90.8-8556/tests/testsuite.dir/at-groups/203/stderr
>  2021-08-29 20:48:44.601359080 +0200
> @@ -0,0 +1,3 @@
> +Assertion failed: !symbols_sorted, file src/symtab.c, line 903, function 
> symbol_from_uniqstr
> +gm4:stdin:14: ERROR: end of file in string
> +/home/dam/mgar/pkg/bison/trunk/work/solaris10-i386/build-isa-pentium_pro/bison-3.7.90.8-8556/tests/bison:
>  line 33:  8409 Abort   (core dumped) $PREBISON 
> "$abs_top_builddir/src/bison" ${1+"$@"}
> ./report.at:1561: exit code was 134, expected 0
> 203. report.at:1523: 203. Multistart reports (report.at:1523): FAILED 
> (report.at:1561)

I do not understand how there could be portability issues in the area.  
Besides, none of the sanitizers I use found anything in this test.

That being said, it is again a test about a feature I don't want to make public 
in 3.8, so I will disable this test, and I'll have to solve that issue 
afterwards.

Many thanks!

Re: Bison 3.7.90 (beta): test 4

[Akim Demaille]

Hi Dagobert,

> Le 23 août 2021 à 15:12, Dagobert Michelsen  a écrit :
> 
> Hi Akim,
> 
> sorry for the delay, got distracted!

No problem.  I was unavailable for a while :)


> Am 20.08.2021 um 09:01 schrieb Akim Demaille :
>> Wow...  So that would be a problem with sed?
>> 
>> sed -e 's/foo/bar/g' dir/sed.1.log
>> LC_ALL=C sed -e 's/foo/bar/g' dir/sed.2.log

So it is indeed a problem with sed: neither sed.1.log nor sed.2.log have the 
expected NUL byte.

So I'm installing the patch below.  Could you please check this tarball:

https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.90.8-8556.tar.gz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.90.8-8556.tar.lz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.90.8-8556.tar.xz

Cheers!

commit bc5aa8e2743e7b56818d54719bf4095a6dc09912
Author: Akim Demaille 
Date:   Sun Aug 29 09:38:07 2021 +0200

tests: don't postprocess stderr with sed

On Solaris, sed throws away the NUL bytes from the stream, even in C
locale.  So instead of postprocessing bison's stderr to neutralize
changes in value of `argv[0]`, use an envvar to actually neutralize
variations of `argv[0]` during tests.

Reported by Dagobert Michelsen.
<https://lists.gnu.org/r/bug-bison/2021-08/msg00025.html>

* src/main.c (main): Change `argv[0]` if BISON_PROGRAM_NAME is
defined.
* tests/bison.in: No longer mess with stderr, just pass
the expected BISON_PROGRAM_NAME value.

diff --git a/src/main.c b/src/main.c
index 22291d23..1185bf24 100644
--- a/src/main.c
+++ b/src/main.c
@@ -63,6 +63,12 @@
 int
 main (int argc, char *argv[])
 {
+  {
+char *cp = getenv ("BISON_PROGRAM_NAME");
+if (cp)
+  argv[0] = cp;
+  }
+
 #define DEPENDS_ON_LIBINTL 1
   set_program_name (argv[0]);
   setlocale (LC_ALL, "");
diff --git a/tests/bison.in b/tests/bison.in
index 80003aca..7732edd4 100644
--- a/tests/bison.in
+++ b/tests/bison.in
@@ -25,27 +25,13 @@ abs_top_builddir='@abs_top_builddir@'
 BISON_PKGDATADIR=$abs_top_srcdir/data
 export BISON_PKGDATADIR
 
-stderr=tmp-bison.$$
-
-# If stderr is a tty, force --color=yes to simulate --color=auto
-# although we save and modify stderr.
-if test -t 2; then
-set x --color=yes ${1+"$@"}
-shift
-fi
-
-# We redirect stderr, which breaks the computation of the terminal
-# screen width.  So export COLUMNS to Bison, hoping for the shell to
-# have defined it.
-: ${COLUMNS=`(tput cols) 2>/dev/null || echo 132`}
-export COLUMNS
-$PREBISON "$abs_top_builddir/src/bison" ${1+"$@"} 2>"$stderr"
-status=$?
-
 # Neutralize path differences in error messages so that check and
 # installcheck behave the same way.
-sed -e "s,$abs_top_builddir/src/,,g" <"$stderr" >&2
-rm -f "$stderr"
+BISON_PROGRAM_NAME=bison
+export BISON_PROGRAM_NAME
+
+$PREBISON "$abs_top_builddir/src/bison" ${1+"$@"}
+status=$?
 
 # As a special dark magic, if we are actually using this wrapper to
 # compile Bison's src/parse-gram.y, post-process the synclines to

Re: Bison 3.7.90 (beta): test 4

[Akim Demaille]

Hi!

> Le 19 août 2021 à 11:29, Dagobert Michelsen  a écrit :
> 
> Hi Akim,
> 
>> There remains one last check to make sure it's bison the binary that is 
>> wrong:
>> 
>> LC_ALL=C ./src/bison ./tests/testsuite.dir/004/input.y 2>/tmp/log
>> 
>> (src/bison, not tests/bison)
>> 
>> But I'm pretty sure the \x00 is still not there, and it's going to painful 
>> to diagnose and fix :(  Yet this rings some bell, I'll dive into the history 
>> to find something similar.
> 
> Surprise! That one does contain \x00 and the above perl line works
> like charm:

Wow...  So that would be a problem with sed?

Please use the attached input.y file for the following commands:



input.y.gz
Description: GNU Zip compressed data


mkdir dir
LC_ALL=C tests/bison input.y 2>dir/test.log
LC_ALL=C src/bison input.y 2>dir/src.log
LC_ALL=C perl -p -e 's{([\0\200\210\360\377])}  {sprintf "\\x%02x", ord($1)}ge' 
dir/src.1.log
LC_ALL=C perl -p -e 's{([\000\200\210\360\377])}{sprintf "\\x%02x", ord($1)}ge' 
dir/src.2.log
LC_ALL=C perl -p -e 's{(\0|[\200\210\360\377])} {sprintf "\\x%02x", ord($1)}ge' 
dir/src.3.log
sed -e 's/foo/bar/g' dir/sed.1.log
LC_ALL=C sed -e 's/foo/bar/g' dir/sed.2.log
tar cf - dir | gzip >dir.tgz

Please send the zipped tarball (to make sure we don't lose the NUL byte because 
of the mailers).

Thanks Dagobert!

Re: Bison 3.7.90 (beta): test 4

[Akim Demaille]

> Le 19 août 2021 à 10:21, Dagobert Michelsen  a écrit :
> 
>> Thanks.  Neither work as I expect.  It seems that \0 does not work in char 
>> classes.  Could you please try this also?
>> 
>> perl -p -e 's{(\0|[\200\210\360\377])}{sprintf "\\x%02x", ord($1)}ge' > >out.3
>> 
>> Currently your output looks like this:
>> 
>> tests/testsuite.dir/004/input.y:1.11: error: invalid null character
>>   1 | %header "\xf0\x80\x88"
>> |   ^
>> 
>> the expected output should also show \x00:
>> 
>>   1 | %header "\xf0\x00\x80\x88"
>> |   ^
> 
> So you are assuming that a NULL byte actually in log, which is not the case:

You are right, I somehow confused things, and log is incorrect.  So it's not 
the test suite which is wrong, but bison itself.

There remains one last check to make sure it's bison the binary that is wrong:

LC_ALL=C ./src/bison ./tests/testsuite.dir/004/input.y 2>/tmp/log

(src/bison, not tests/bison)

But I'm pretty sure the \x00 is still not there, and it's going to painful to 
diagnose and fix :(  Yet this rings some bell, I'll dive into the history to 
find something similar.

Re: Bison 3.7.90 (beta): test 4

[Akim Demaille]

> Le 19 août 2021 à 10:07, Dagobert Michelsen  a écrit :
> 
> Hi Akim,
> 
> Am 19.08.2021 um 10:01 schrieb Akim Demaille :
>> This is good: the input and the output are correct.  So it's the test 
>> suite's post processing which is wrong.  I guess Perl does not behave the 
>> expected way.  Could you please give me the results (and out.1 and out.2) of:
>> 
>> perl --version
>> perl -p -e 's{([\0\200\210\360\377])}{sprintf "\\x%02x", ord($1)}ge' > >out.1
>> perl -p -e 's{([\000\200\210\360\377])}{sprintf "\\x%02x", ord($1)}ge' > >out.2
>> 
>> where log is the previous log file (from LC_ALL=C tests/bison  
>> tests/testsuite.dir/004/input.y 2>log).
> 
> Sure, I also added the respective output from the system shipped perl:

Thanks.  Neither work as I expect.  It seems that \0 does not work in char 
classes.  Could you please try this also?

perl -p -e 's{(\0|[\200\210\360\377])}{sprintf "\\x%02x", ord($1)}ge' out.3

Currently your output looks like this:

tests/testsuite.dir/004/input.y:1.11: error: invalid null character
1 | %header "\xf0\x80\x88"
  |   ^

the expected output should also show \x00:

1 | %header "\xf0\x00\x80\x88"
  |   ^

Re: Bison 3.7.90 (beta): test 4

[Akim Demaille]

> Le 19 août 2021 à 09:28, Dagobert Michelsen  a écrit :
> 
> Hi Akim,
> 
> Am 19.08.2021 um 09:25 schrieb Akim Demaille :
>> Le 19 août 2021 à 09:13, Dagobert Michelsen  a écrit :
>>> Am 19.08.2021 um 09:03 schrieb Akim Demaille :
>>>> So there is something fishy going on with the nul bytes.  I don't know if 
>>>> it's in the input generated by the test suite, the processing by Bison, or 
>>>> the pricessing of the diagnostics by the test suite.
>>>> 
>>>> Could you please send tests/testsuite.dir/004/input.y as an attachment?  
>>>> The results of
>>>> 
>>>> - od -x _build/g11d/tests/testsuite.dir/004/input.y
>>>> - LC_ALL=C tests/bison  tests/testsuite.dir/004/input.y 2>log
>>>> 
>>>> could be useful too.  Thanks!
>> 
>> Actually I still need input.y as an attachment, and the 'log' file of the 
>> previous command.
> 
> Ah sorry, here they are:
> 
> 

This is good: the input and the output are correct.  So it's the test suite's 
post processing which is wrong.  I guess Perl does not behave the expected way. 
 Could you please give me the results (and out.1 and out.2) of:

perl --version
perl -p -e 's{([\0\200\210\360\377])}{sprintf "\\x%02x", ord($1)}ge' out.1
perl -p -e 's{([\000\200\210\360\377])}{sprintf "\\x%02x", ord($1)}ge' out.2

where log is the previous log file (from LC_ALL=C tests/bison  
tests/testsuite.dir/004/input.y 2>log).

Thanks!

Re: Bison 3.7.90 (beta): test 4

[Akim Demaille]

Thanks for the quick response!

> Le 19 août 2021 à 09:13, Dagobert Michelsen  a écrit :
> 
> Hi Akim,
> 
> Am 19.08.2021 um 09:03 schrieb Akim Demaille :
>> So there is something fishy going on with the nul bytes.  I don't know if 
>> it's in the input generated by the test suite, the processing by Bison, or 
>> the pricessing of the diagnostics by the test suite.
>> 
>> Could you please send tests/testsuite.dir/004/input.y as an attachment?  The 
>> results of
>> 
>> - od -x _build/g11d/tests/testsuite.dir/004/input.y
>> - LC_ALL=C tests/bison  tests/testsuite.dir/004/input.y 2>log
>> 
>> could be useful too.  Thanks!

Actually I still need input.y as an attachment, and the 'log' file of the 
previous command.

Thanks in advance!

Re: Bison 3.7.90 (beta): example/c/lexcalc

[Akim Demaille]

> Le 18 août 2021 à 16:28, Dagobert Michelsen  a écrit :
> 
> Hi Akim,
> 
> Am 17.08.2021 um 08:59 schrieb Akim Demaille :
>>> Le 13 août 2021 à 09:50, Dagobert Michelsen  a écrit :
>>> and there are also errors on Solaris 10 Sparc:
>>> YACC examples/c/lexcalc/parse.c
>>> /home/dam/mgar/pkg/bison/trunk/work/solaris10-sparc/build-isa-sparcv8plus/bison-3.7.90/examples/c/lexcalc/parse.y:
>>>  error: 1 nonterminal useless in grammar [-Werror=other]
>>> /home/dam/mgar/pkg/bison/trunk/work/solaris10-sparc/build-isa-sparcv8plus/bison-3.7.90/examples/c/lexcalc/parse.y:
>>>  error: 1 rule useless in grammar [-Werror=other]
>>> /home/dam/mgar/pkg/bison/trunk/work/solaris10-sparc/build-isa-sparcv8plus/bison-3.7.90/examples/c/lexcalc/parse.y:79.25-27:
>>>  error: start symbol NUM does not derive any sentence  
>>> 79 | %start input expression NUM
>>>| ^~~  
>>> gmake[3]: *** [Makefile:9538: examples/c/lexcalc/parse.c] Error 1
>> 
>> That makes no sense to me: bison is emitting an incorrect
>> diagnostic.  I absolutely do not understand how this is possible.
>> Except if we actually are not running the shipped bison, but another
>> one.
>> 
>> Could you please run `make check-examples V=1`?
> 
> Sure:
> 
> dam@unstable10s 
> [unstable10s]:/home/dam/mgar/pkg/bison/trunk/work/solaris10-sparc/build-isa-sparcv8plus/bison-3.7.90
>  > gmake check-examples V=1 
> /bin/bash ./build-aux/ylwrap examples/c/lexcalc/parse.y y.tab.c 
> examples/c/lexcalc/parse.c y.tab.h `echo examples/c/lexcalc/parse.c | sed -e 
> s/cc$/hh/ -e s/cpp$/hpp/ -e s/cxx$/hxx/ -e s/c++$/h++/ -e s/c$/h/` y.output 
> examples/c/lexcalc/parse.output -- ./tests/bison -o y.tab.c --defines -Werror 
> -Wall,dangling-alias --report=all --no-lines 
> /home/dam/mgar/pkg/bison/trunk/work/solaris10-sparc/build-isa-sparcv8plus/bison-3.7.90/examples/c/lexcalc/parse.y:
>  error: 1 nonterminal useless in grammar [-Werror=other]
> /home/dam/mgar/pkg/bison/trunk/work/solaris10-sparc/build-isa-sparcv8plus/bison-3.7.90/examples/c/lexcalc/parse.y:
>  error: 1 rule useless in grammar [-Werror=other]
> /home/dam/mgar/pkg/bison/trunk/work/solaris10-sparc/build-isa-sparcv8plus/bison-3.7.90/examples/c/lexcalc/parse.y:79.25-27:
>  error: start symbol NUM does not derive any sentence
>   79 | %start input expression NUM
>  | ^~~
> gmake: *** [Makefile:9538: examples/c/lexcalc/parse.c] Error 1

This still does not make sense to me.  But let's postpone this issue, since 
it's about an unfinished feature that I don't want to demonstrate in 3.8.  So I 
have installed the following "fix".  But in the future we will have to dig 
deeper.

Thanks a lot for the feedback!

commit a70e75b8a41755ab96ab211a0ea111ac68a4aadd
Author: Akim Demaille 
Date:   Thu Aug 19 09:16:00 2021 +0200

examples: don't demonstrate multistart, which is not part of 3.8

Besides, for mysterious reasons, this fails on some environment.
Reported by Dagobert Michelsen.
<https://lists.gnu.org/archive/html/bug-bison/2021-08/msg8.html>

* examples/c/lexcalc/lexcalc.test, examples/c/lexcalc/parse.y,
* examples/c/lexcalc/scan.l: Revert to a single-start example.

diff --git a/TODO b/TODO
index 2e6e7d5a..4ef16d8d 100644
--- a/TODO
+++ b/TODO
@@ -1,6 +1,3 @@
-* 3.8
-Don't showcase multi start.
-
 * Soon
 ** scan-code
 The default case is scanning char-per-char.
diff --git a/examples/c/lexcalc/lexcalc.test b/examples/c/lexcalc/lexcalc.test
index df8da33b..7559a21e 100644
--- a/examples/c/lexcalc/lexcalc.test
+++ b/examples/c/lexcalc/lexcalc.test
@@ -38,26 +38,9 @@ run -noerr 0 9 -p
 cat >input <input <input <input < NUM "number"
-%type  exp expression line
+%type  exp
 %printer { fprintf (yyo, "%d", $$); } 
 
-%start input expression NUM
-
 // Precedence (from lowest to highest) and associativity.
 %left "+" "-"
 %left "*" "/"
@@ -90,12 +91,8 @@ input:
 ;
 
 line:
-  exp EOL   { $$ = $exp; printf ("%d\n", $$); }
-| error EOL { $$ = 0; yyerrok; }
-;
-
-expression:
-  exp EOL  { $$ = $exp; }
+  exp EOL   { printf ("%d\n", $exp); }
+| error EOL { yyerrok; }
 ;
 
 exp:
@@ -106,7 +103,7 @@ exp:
   {
 if ($3 == 0)
   {
-yyerror (&@$, "error: division by zero");
+yyerror (&@$, nerrs, "error: division by zero");
 YYERROR;
   }
 else
@@ -118,41 +115,24 @@ exp:
 %%
 // Epilogue (C code).
 
-void yyerror (const YYLTYPE *loc, const char *msg)
+void yyerror (const YYLTYPE *loc, int *nerrs, const char *msg)
 {
   YYLOCATION_PRINT (stderr, loc);
   fprintf (stderr, ": %s\n", msg);
+  ++*nerrs;
 }

Re: Bison 3.7.90 (beta): test 4

[Akim Demaille]

Dagobert,

> Le 18 août 2021 à 16:28, Dagobert Michelsen  a écrit :
> 
>> BTW, please also run `make check-tests`.  Because you had errors in the 
>> tests of the examples, the main test suite was not run.
> 
> There are quite a lot of failures:
>  testsuite: 4 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 
> 487 48 500 failed
> 
> The log is attached:

The test case 4 fails this way:

./input.at:104: "$PERL" -p -e 's{([\0\200\210\360\377])}{sprintf "\\x%02x", 
ord($1)}ge' stderr
2c2
< 1 | %header "\xf0\x00\x80\x88"
---
> 1 | %header "\xf0\x80\x88"
5c5
< 2 | \x00
\xff?
---
> 2 | 
\xff?
8c8
< 3 | "\x00"
---
> 3 | ""

So there is something fishy going on with the nul bytes.  I don't know if it's 
in the input generated by the test suite, the processing by Bison, or the 
pricessing of the diagnostics by the test suite.

Could you please send tests/testsuite.dir/004/input.y as an attachment?  The 
results of

- od -x _build/g11d/tests/testsuite.dir/004/input.y
- LC_ALL=C tests/bison  tests/testsuite.dir/004/input.y 2>log

could be useful too.  Thanks!

Re: Bison 3.7.90 (beta)

[Akim Demaille]

Dagobert,

> Le 18 août 2021 à 16:28, Dagobert Michelsen  a écrit :
> 
>>> The compiler in Oracle Studio 12.6 raised some time ago as discussed in
>>> https://lists.gnu.org/archive/html/bug-bison/2019-10/msg00045.html
>> 
>> But did that cause `make` to fail?  Or is it just warnings?
> 
> Well, versions below 12.6 did fail and 12.6 was required with the earlier 
> version
> of bison.

Yes, but I'm confused.  What do you mean by "The compiler in Oracle Studio 12.6 
raised"?  That was just for memories just in case it could be related, or did 
you mean it raised an error?

Re: Bison 3.7.90 (beta)

[Akim Demaille]

Dagobert,

> Le 18 août 2021 à 16:28, Dagobert Michelsen  a écrit :
> 
>> BTW, please also run `make check-tests`.  Because you had errors in the 
>> tests of the examples, the main test suite was not run.
> 
> There are quite a lot of failures:
>  testsuite: 4 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 
> 487 48 500 failed
> 
> The log is attached:
> 
> 

There are two issues here: that of 4, and that of all the others (the message 
looks wrong, it says "487 48 500", but it should be "487 488 489 500").

For the second part, I, again, used grep -E.  So I'm installing this.

commit 3afa9757da056e2dc2abf56b0abb7b4276c030d7
Author: Akim Demaille 
Date:   Thu Aug 19 08:21:23 2021 +0200

tests: grep -E is not portable

Reported by Dagobert Michelsen.
<https://lists.gnu.org/r/bug-bison/2021-08/msg8.html>

* tests/calc.at: Use $EGREP.

diff --git a/tests/calc.at b/tests/calc.at
index 831a2b08..11801543 100644
--- a/tests/calc.at
+++ b/tests/calc.at
@@ -970,7 +970,7 @@ m4_define([_AT_CHECK_CALC],
   [AT_PARSER_CHECK([calc $1 input],0, [m4_ifvaln(m4_quote($3), [$3])], 
[stderr])])
 AT_LANG_MATCH([c\|c++\|java],
   [AT_GLR_IF([],
-[AT_CHECK([grep -c -v -E 'Return for a new token:|LAC:' stderr],
+[AT_CHECK([$EGREP -c -v 'Return for a new token:|LAC:' stderr],
   [ignore],
   [m4_n([AT_DEBUG_IF([$4], [0])])])])])
 ])

Re: Bison 3.7.90 (beta)

[Akim Demaille]

Hi Dagobert,

> Le 13 août 2021 à 09:50, Dagobert Michelsen  a écrit :
> 
> I see errors on Solaris 10 x86:
> 
>  CXX  examples/c++/glr/examples_c___glr_c___types-c++-types.o
> "examples/c++/glr/c++-types.cc", line 734: Warning:  The template qualifier 
> can only be used within a template.
> "/opt/developerstudio12.6/lib/compilers/CC-gcc/include/c++/5.4.0/bits/algorithmfwd.h",
>  line 582: Error: Could not find a match for std::swap<_Tp>(_Tp, _Tp) needed 
> in glr_stack_item::operator=(glr_stack_item).
> "examples/c++/glr/c++-types.cc", line 1360: Where: While instantiating 
> "void std::swap(char(&)[60], char(&)[60])".
> "examples/c++/glr/c++-types.cc", line 1360: Where: Instantiated from 
> non-template code.
> "/opt/developerstudio12.6/lib/compilers/CC-gcc/include/c++/5.4.0/bits/algorithmfwd.h",
>  line 582: Error: A constant expression is required here.
> "examples/c++/glr/c++-types.cc", line 1360: Where: While instantiating 
> "void std::swap(char(&)[60], char(&)[60])".
> "examples/c++/glr/c++-types.cc", line 1360: Where: Instantiated from 
> non-template code.
> "examples/c++/glr/c++-types.cc", line 2124: Warning:  The template qualifier 
> can only be used within a template.
> "examples/c++/glr/c++-types.cc", line 2220: Warning:  The template qualifier 
> can only be used within a template.
> "examples/c++/glr/c++-types.cc", line 2462: Warning:  The template qualifier 
> can only be used within a template.
> "examples/c++/glr/c++-types.cc", line 2858: Warning:  The template qualifier 
> can only be used within a template.
> "examples/c++/glr/c++-types.cc", line 3098: Warning:  The template qualifier 
> can only be used within a template.
> "examples/c++/glr/c++-types.cc", line 3147: Warning:  The template qualifier 
> can only be used within a template.
> 2 Error(s) and 7 Warning(s) detected.
> gmake[3]: *** [Makefile:9470: 
> examples/c++/glr/examples_c___glr_c___types-c++-types.o] Error 2

For this part of the failures, I'm installing this:

commit 36f37568bac3ac9efb2917c5d68dea50a4a619c3
Author: Akim Demaille 
Date:   Tue Aug 17 12:53:09 2021 +0200

glr2.cc: require C++11

Reported by Dagobert Michelsen.
https://lists.gnu.org/r/bug-bison/2021-08/msg6.html

* m4/bison-cxx-std.m4 (_BISON_CXXSTD_98_snippet): We don't need
vector::data, it was only for glr2.cc, which is C++11 anyway.
(_BISON_CXXSTD_11_snippet): We need vector::data and std::swap on
arrays.
* m4/cxx.m4 (BISON_TEST_FOR_WORKING_CXX_COMPILER): We don't need
vector::data.
* tests/local.at (AT_COMPILE_CXX): Skip when glr2.cc and no support
for C++11.

diff --git a/TODO b/TODO
index de240dda..2e6e7d5a 100644
--- a/TODO
+++ b/TODO
@@ -1,3 +1,6 @@
+* 3.8
+Don't showcase multi start.
+
 * Soon
 ** scan-code
 The default case is scanning char-per-char.
@@ -183,7 +186,7 @@ Les catégories d'avertissements incluent :
 
 Line -1 and -3 should mention CATEGORIE, not CATEGORY.
 
-* Bison 3.8
+* Bison 3.9
 ** Rewrite glr.cc (currently glr2.cc)
 *** custom error messages
 
diff --git a/data/skeletons/glr2.cc b/data/skeletons/glr2.cc
index edd05d89..d682a078 100644
--- a/data/skeletons/glr2.cc
+++ b/data/skeletons/glr2.cc
@@ -1370,7 +1370,6 @@ public:
 check_ ();
 other.check_ ();]])[
 std::swap (is_state_, other.is_state_);
-// NB: swap on arrays is C++11.
 std::swap (raw_, other.raw_);
 return *this;
   }
diff --git a/m4/bison-cxx-std.m4 b/m4/bison-cxx-std.m4
index 4a71d20f..dd8a5345 100644
--- a/m4/bison-cxx-std.m4
+++ b/m4/bison-cxx-std.m4
@@ -7,36 +7,22 @@
 # with or without modifications, as long as this notice is preserved.
 
 m4_define([_BISON_CXXSTD_98_snippet],
-[[#include 
-#include 
-
-void cxx98_vector ()
-{
-  typedef std::vector ints;
-
-  // Check support for std::vector::data.
-  // GCC 4.2 on macOS claims to support C++98, but does not feature it.
-  //
-  // input.cc: In member function 'void state_stack::yycompressStack()':
-  // input.cc:1774: error: 'class std::vector >' has no member named 'data'
-  //
-  // 
<https://trac.macports.org/raw-attachment/ticket/59927/bison-test-results-20210811-95b72.log.xz>.
-  ints my_ints;
-  assert (my_ints.data () == &my_ints[0]);
-}
-]])
+[[]])
 
 m4_define([_BISON_CXXSTD_03_snippet],
-[])
+[[]])
 
 m4_define([_BISON_CXXSTD_11_snippet],
-[[#include 
+[[  // C++11
+#include 
+#include 
 #include 
 #include 
 #include 
 #include 
+#include  // std::swap
+#include 
 
-  // C++11
   template 
   struct check
   {
@@ -80,6 +66,22 @@ m4_define([_BISON_CXXSTD_11_snippet],
 
   // GCC 4.8.2 on Solaris 11.3 does not support to_string.
   auto e = std::t

Re: Bison 3.7.90 (beta)

[Akim Demaille]

Hi Dagobert,

Thanks a lot for the feedback!

> Le 13 août 2021 à 09:50, Dagobert Michelsen  a écrit :
> 
> Hi Akim,
> 
> I see errors on Solaris 10 x86:
> 
>  CXX  examples/c++/glr/examples_c___glr_c___types-c++-types.o
> "examples/c++/glr/c++-types.cc", line 734: Warning:  The template qualifier 
> can only be used within a template.
> "/opt/developerstudio12.6/lib/compilers/CC-gcc/include/c++/5.4.0/bits/algorithmfwd.h",
>  line 582: Error: Could not find a match for std::swap<_Tp>(_Tp, _Tp) needed 
> in glr_stack_item::operator=(glr_stack_item).
> "examples/c++/glr/c++-types.cc", line 1360: Where: While instantiating 
> "void std::swap(char(&)[60], char(&)[60])".

Ok, I know that issue: glr2.cc requires C++11.  I'll address that,
thanks!


> and there are also errors on Solaris 10 Sparc:
>  YACC examples/c/lexcalc/parse.c
> /home/dam/mgar/pkg/bison/trunk/work/solaris10-sparc/build-isa-sparcv8plus/bison-3.7.90/examples/c/lexcalc/parse.y:
>  error: 1 nonterminal useless in grammar [-Werror=other]
> /home/dam/mgar/pkg/bison/trunk/work/solaris10-sparc/build-isa-sparcv8plus/bison-3.7.90/examples/c/lexcalc/parse.y:
>  error: 1 rule useless in grammar [-Werror=other]
> /home/dam/mgar/pkg/bison/trunk/work/solaris10-sparc/build-isa-sparcv8plus/bison-3.7.90/examples/c/lexcalc/parse.y:79.25-27:
>  error: start symbol NUM does not derive any sentence  
>   79 | %start input expression NUM
>  | ^~~  
> gmake[3]: *** [Makefile:9538: examples/c/lexcalc/parse.c] Error 1

That makes no sense to me: bison is emitting an incorrect
diagnostic.  I absolutely do not understand how this is possible.
Except if we actually are not running the shipped bison, but another
one.

Could you please run `make check-examples V=1`?

BTW, please also run `make check-tests`.  Because you had errors in the tests 
of the examples, the main test suite was not run.

> The compiler in Oracle Studio 12.6 raised some time ago as discussed in
>  https://lists.gnu.org/archive/html/bug-bison/2019-10/msg00045.html

But did that cause `make` to fail?  Or is it just warnings?

Bison 3.7.90 (beta)

[Akim Demaille]

Hi,

I'm very pleased to announce the forthcoming release of Bison 3.8,
whose main novelty is the D backend for deterministic parsers,
contributed by Adela Vais.  It supports all the bells and whistles of
Bison's other deterministic parsers, which include: pull/push
interfaces, verbose and custom error messages, lookahead correction,
LALR(1), IELR(1), canonical LR(1), token constructors,
internationalization, locations, printers, token and symbol prefixes,
and more.

There are several other notable changes.  Please see the detailed NEWS
below for more details.

To make sure Bison 3.8 will be a successful release, we need feedback
on this beta.  *Please test it!*

==

Here are the compressed sources:
  https://alpha.gnu.org/gnu/bison/bison-3.7.90.tar.gz   (6.1MB)
  https://alpha.gnu.org/gnu/bison/bison-3.7.90.tar.lz   (3.1MB)
  https://alpha.gnu.org/gnu/bison/bison-3.7.90.tar.xz   (3.1MB)

Here are the GPG detached signatures[*]:
  https://alpha.gnu.org/gnu/bison/bison-3.7.90.tar.gz.sig
  https://alpha.gnu.org/gnu/bison/bison-3.7.90.tar.lz.sig
  https://alpha.gnu.org/gnu/bison/bison-3.7.90.tar.xz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

Here are the SHA1 and SHA256 checksums:

8097add41cc3d33066f99032d70be593a50203cb  bison-3.7.90.tar.gz
V3+zEseMeYSM0YQTtEDomqetlinbF5+xJKsD0xQtyFM  bison-3.7.90.tar.gz
4db57df5ea8228d44da785325fa096293c5ea827  bison-3.7.90.tar.lz
DmyJekE8+IW9LJ0QSyb+GgDophOOy0oX8m5Mhpl8o4Q  bison-3.7.90.tar.lz
a20a5922c9bbea5261094c93ef711c6d04cf728e  bison-3.7.90.tar.xz
YZbfET62jSCMx28CymRCuTvbGp6a6f8GULEiaJRa1H4  bison-3.7.90.tar.xz

The SHA256 checksum is base64 encoded, instead of the
hexadecimal encoding that most checksum tools default to.

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify bison-3.7.90.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver keys.gnupg.net --recv-keys 0DDCAA3278D5264E

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.71
  Automake 1.16b
  Flex 2.6.4
  Gettext 0.20.1.153-6c39c
  Gnulib v0.1-4853-g964ce0a92

==

GNU Bison is a general-purpose parser generator that converts an annotated
context-free grammar into a deterministic LR or generalized LR (GLR) parser
employing LALR(1) parser tables.  Bison can also generate IELR(1) or
canonical LR(1) parser tables.  Once you are proficient with Bison, you can
use it to develop a wide range of language parsers, from those used in
simple desk calculators to complex programming languages.

Bison is upward compatible with Yacc: all properly-written Yacc grammars
work with Bison with no change.  Anyone familiar with Yacc should be able to
use Bison with little trouble.  You need to be fluent in C, C++, D or Java
programming in order to use Bison.

Bison and the parsers it generates are portable, they do not require any
specific compilers.

GNU Bison's home page is https://gnu.org/software/bison/.

==

NEWS

* Noteworthy changes in release 3.7.90 (2021-08-13) [beta]

** Backward incompatible changes

  In conformance with the recommendations of the Graphviz team
  (https://marc.info/?l=graphviz-devel&m=129418103126092), `-g`/`--graph`
  now generates a *.gv file by default, instead of *.dot.  A transition
  started in Bison 3.4.

  To comply with the latest POSIX standard, in Yacc compatibility mode
  (options `-y`/`--yacc`) Bison now generates prototypes for yyerror and
  yylex.  In some situations, this is breaking compatibility: if the user
  has already declared these functions but with some differences (e.g., to
  declare them as static, or to use specific attributes), the generated
  parser will fail to compile.  To disable these prototypes, #define yyerror
  (to `yyerror`), and likewise for yylex.

** Deprecated features

  Support for the YYPRINT macro is removed. It worked only with yacc.c and
  only for tokens.  It was obsoleted by %printer, introduced in Bison 1.50
  (November 2002).

  It has always been recommended to prefer `%define api.value.type foo` to
  `#define YYSTYPE foo`.  The latter is supported in C for compatibility
  with Yacc, but not in C++.  Warnings are now issued if `#define YYSTYPE`
  is used in C++, and eventually support will be removed.

  In C++ code, prefer value_type to semantic_type to denote the semantic
  value type, which is specified by the `api.value.type` %define variable.

** New features

*** A skeleton for the D programming language

  The "lalr1.d" skeleton is now officially part of Bison.

  It was originally contributed by Oliv

Re: Error in bison GLR C++ skeleton when using "%locations" & "%define parse.error custom" directives

[Akim Demaille]

Hi Tom,

> Le 11 août 2021 à 03:17, Tom Shields  a 
> écrit :
> 
> The parser generated when using the “%locations” & “%define parse.error 
> custom” directives gets a warning using the bison C skeleton, but gets an 
> error using the C++ skeleton.
> 
> Offending context:
> 
> /* The location of the lookahead of this context.  */
> static YYLTYPE *
> yypcontext_location (const yyGLRStack *yystackp) YY_ATTRIBUTE_UNUSED;
> 
> static YYLTYPE *
> yypcontext_location (const yyGLRStack *yystackp)
> {
>  YY_USE (yystackp);
>  return &yylloc;  /* <-- the error location: "yylloc” is a macro defined 
> as “(yystackp->yyval)” */
> }
> 
> Using the C skeleton:
> macOS gcc: warning: returning 'const YYLTYPE *' (aka 'const struct YYLTYPE 
> *') from a function with result type 'YYLTYPE *' (aka 'struct YYLTYPE *') 
> discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers]
> GNU gcc: warning: return discards 'const' qualifier from pointer target type 
> [-Wdiscarded-qualifiers]

Thanks for the report.  I'm installing the following fix, which will show in 
the forthcoming 3.8 release.

commit c2ba260487927a4cdf061fae10fdd1b387ca3d95
Author: Akim Demaille 
Date:   Wed Aug 11 17:41:27 2021 +0200

glr.c: fix signature when using custom error messages

Reported by Tom Shields .

* data/skeletons/glr.c (yypcontext_location): Fix return type.
* tests/calc.at: Check the case pure, location, custom error messages.

diff --git a/TODO b/TODO
index d9277255..de240dda 100644
--- a/TODO
+++ b/TODO
@@ -185,6 +185,8 @@ Line -1 and -3 should mention CATEGORIE, not CATEGORY.
 
 * Bison 3.8
 ** Rewrite glr.cc (currently glr2.cc)
+*** custom error messages
+
 *** Remove jumps
 We can probably replace setjmp/longjmp with exceptions.  That would help
 tremendously other languages such as D and Java that probably have no
diff --git a/data/skeletons/glr.c b/data/skeletons/glr.c
index 117c996b..fab3733f 100644
--- a/data/skeletons/glr.c
+++ b/data/skeletons/glr.c
@@ -2174,10 +2174,10 @@ yypcontext_token (const yyGLRStack *yystackp)
 }
 
 ]b4_locations_if([[/* The location of the lookahead of this context.  */
-static YYLTYPE *
+static const YYLTYPE *
 yypcontext_location (const yyGLRStack *yystackp) YY_ATTRIBUTE_UNUSED;
 
-static YYLTYPE *
+static const YYLTYPE *
 yypcontext_location (const yyGLRStack *yystackp)
 {
   YY_USE (yystackp);
diff --git a/tests/calc.at b/tests/calc.at
index cd4b50a1..47ef720c 100644
--- a/tests/calc.at
+++ b/tests/calc.at
@@ -1469,6 +1469,8 @@ m4_define([AT_CHECK_CALC_GLR_CC],
 AT_CHECK_CALC_GLR_CC([%define parse.error verbose %debug %name-prefix "calc" 
%verbose])
 AT_CHECK_CALC_GLR_CC([%define parse.error verbose %debug %name-prefix "calc" 
%define api.token.prefix {TOK_} %verbose])
 
+AT_CHECK_CALC_GLR_CC([%define parse.error custom %locations %header 
%name-prefix "calc" %verbose])
+
 AT_CHECK_CALC_GLR_CC([%locations %header %define parse.error verbose %debug 
%name-prefix "calc" %verbose %parse-param {semantic_value *result}{int 
*count}{int *nerrs}])
 AT_CHECK_CALC_GLR_CC([%locations %header %define parse.error verbose %debug 
%define api.prefix {calc} %verbose %parse-param {semantic_value *result}{int 
*count}{int *nerrs}])
 


> Using the C++ skeleton:
> macOS g++: error: cannot initialize return object of type 
> 'yy::parser::location_type *' (aka ‘yy::location *') with an rvalue of type 
> ‘const yy::parser::location_type *' (aka 'const yy::location *’)
> GNU g++: error: invalid conversion from 'const location_type*' {aka 'const 
> yy::location*'} to 'yy::parser::location_type*' {aka 'yy::location*'} 
> [-fpermissive]
> 
> 
> The error in the bison skeleton has been around for years; I vaguely recall 
> it showing up in a version of bison predating the addition of “%define 
> parse.error custom”. It isn’t a problem using the C skeleton, as I just turn 
> off the warning message, but I want to switch to using the C++ skeleton and 
> the C++ compiler error is, obviously, a show stopper.

This is going to be tricky: glr.cc does not support custom error messages.  We 
need someone to port these changes, if you are interested.  Note that we are 
slowly phasing glr.cc out, to be replaced by glr2.cc, which is fully in C++, 
but for lack of time, it does not support custom error messages either.  But it 
will be higher priority than glr.cc.

Still, good wills will always be welcome!

Cheers.

Re: Heap Corruption using MSVC (found the issue)

[Akim Demaille]

Hi Jot,

> Le 12 déc. 2020 à 07:41, Akim Demaille  a écrit :
> 
>> My take on that web page was that YYDEBUG is the minimum that should be 
>> enabled
>> and that the Bison "extension" 'parse.trace' gives more info than the 
>> "standard" YYDEBUG.
>> I can see now that I was wrong.
> 
> And I can see that my wording could use some improvement in this area.
> Will do.  Sorry about that.

It took me quite a while to address this point, but here it is.

Cheers!

commit a83202a43698c2ba3990180ad85d6547ea2e9f17
Author: Akim Demaille 
Date:   Sat Aug 7 07:53:50 2021 +0200

doc: avoid #define YYDEBUG in C++

* doc/bison.texi (Enabling Traces): here.

diff --git a/TODO b/TODO
index 2eab7097..0ae9ef89 100644
--- a/TODO
+++ b/TODO
@@ -35,9 +35,6 @@ Clarify that rule numbers in the skeletons are 1-based.
 There are many macros that should obey api.prefix: YY_CPLUSPLUS, YY_MOVE,
 etc.
 
-** YYDEBUG etc. in C++
-Discourage the use of YYDEBUG in C++ (see thread with Jot).
-
 ** yyerrok in Java
 And add tests in calc.at, to prepare work for D.
 
diff --git a/doc/bison.texi b/doc/bison.texi
index beee6f92..b3b69843 100644
--- a/doc/bison.texi
+++ b/doc/bison.texi
@@ -7000,8 +7000,8 @@ @node %define Summary
 
 In C/C++, define the macro @code{YYDEBUG} (or @code{@var{prefix}DEBUG} with
 @samp{%define api.prefix @{@var{prefix}@}}), see @ref{Multiple Parsers}) to
-1 in the parser implementation file if it is not already defined, so that
-the debugging facilities are compiled.
+1 (if it is not already defined) so that the debugging facilities are
+compiled.
 
 @item Accepted Values: Boolean
 
@@ -11010,8 +11010,8 @@ @node Enabling Traces
 @item the directive @samp{%debug} (deprecated)
 @findex %debug
 Add the @code{%debug} directive (@pxref{Decl Summary}).  This Bison
-extension is maintained for backward compatibility with previous versions of
-Bison; use @code{%define parse.trace} instead.
+extension is maintained for backward compatibility; use @code{%define
+parse.trace} instead.
 
 @item the macro @code{YYDEBUG} (C/C++ only)
 @findex YYDEBUG
@@ -11025,6 +11025,12 @@ @node Enabling Traces
 api.prefix @{c@}}, then if @code{CDEBUG} is defined, its value controls the
 tracing feature (enabled if and only if nonzero); otherwise tracing is
 enabled if and only if @code{YYDEBUG} is nonzero.
+
+In C++, where POSIX compliance makes no sense, avoid this option, and prefer
+@samp{%define parse.trace}.  If you @code{#define} the @code{YYDEBUG} macro
+at the wrong place (e.g., in @samp{%code top} instead of @samp{%code
+require}), the parser class will have two different definitions, thus
+leading to ODR violations and happy debugging times.
 @end table
 
 We suggest that you always enable the trace option so that debugging is

Re: quirks in parse-gram.y

[Akim Demaille]

> Le 22 avr. 2021 à 07:06, Akim Demaille  a écrit :
> 
> Hi!
> 
>> Le 20 avr. 2021 à 04:18, 叶雨飞  a écrit :
>> 2. The rules grammar indicate an ending “;” is actually optional. In fact,
>> the parse-gram.y missed two “;” in separate places.  However the doc is
>> pretty clear an “;” is expected.  Not sure which is intended .
> 
> Good catch.  The grammar would be definitely much simpler if the semicolon
> were mandatory, but YACC compliance forbids this.  So it's the documentation
> which is wrong.  I'll fix that, thanks!

FTR, I have looked for "semicolon" in the documentation, and couldn't
find a place where it was falsely declared as mandated.  So I don't
believe that the documentation was actually wrong.

Cheers!

Re: Assertion fail report: src/parse-gram.c:3268

[Akim Demaille]

Hi,

> Le 29 avr. 2021 à 17:03, Ahcheong Lee  a écrit :
> 
> Hello,
> I report an assertion failure test input generated during a fuzzing test.
> 
> src/parse-gram.c:3268: char *unquote(const char *): Assertion `*cp == '"''
> failed.
> 
> I test on the latest revision of bison uploaded on git.
> You can reproduce the failure by
> ./bison 

Thanks for the bug report.  Its cause is visibly a copy-paste error.  I 
probably copied a \ coming from the terminal where the line was wrapped.

Cheers!

commit 4ee711d4f4f692dbd5133e867ea3364baecf52ff
Author: Akim Demaille 
Date:   Tue Aug 3 10:19:37 2021 +0200

scan: fix typo in UTF-8 escape

We had:

```
-mbchar...|\xF0[\x\90-\xBF]([\x80-\xBF]{2})|...
+mbchar...|\xF0[\x90-\xBF]([\x80-\xBF]{2})|...
```

so a precise sequence that matches the incorrect regex can let NUL
bytes pass through, which triggers an assertion violation downstream.
It is a pity that Flex does not report an error for such input.

Reported by Ahcheong Lee .
<https://lists.gnu.org/r/bug-bison/2021-04/msg3.html>

* src/scan-gram.l (mbchar): Fix the bad regex.
* tests/input.at (Invalid inputs): Check that case.

diff --git a/src/scan-gram.l b/src/scan-gram.l
index 160bda622..f55429ede 100644
--- a/src/scan-gram.l
+++ b/src/scan-gram.l
@@ -160,7 +160,7 @@ xint  0[xX][0-9abcdefABCDEF]+
 eol   \n|\r\n
 
  /* UTF-8 Encoded Unicode Code Point, from Flex's documentation. */
-mbchar
[\x09\x0A\x0D\x20-\x7E]|[\xC2-\xDF][\x80-\xBF]|\xE0[\xA0-\xBF][\x80-\xBF]|[\xE1-\xEC\xEE\xEF]([\x80-\xBF]{2})|\xED[\x80-\x9F][\x80-\xBF]|\xF0[\x\90-\xBF]([\x80-\xBF]{2})|[\xF1-\xF3]([\x80-\xBF]{3})|\xF4[\x80-\x8F]([\x80-\xBF]{2})
+mbchar
[\x09\x0A\x0D\x20-\x7E]|[\xC2-\xDF][\x80-\xBF]|\xE0[\xA0-\xBF][\x80-\xBF]|[\xE1-\xEC\xEE\xEF]([\x80-\xBF]{2})|\xED[\x80-\x9F][\x80-\xBF]|\xF0[\x90-\xBF]([\x80-\xBF]{2})|[\xF1-\xF3]([\x80-\xBF]{3})|\xF4[\x80-\x8F]([\x80-\xBF]{2})
 
 /* Zero or more instances of backslash-newline.  Following GCC, allow
white space between the backslash and the newline.  */
diff --git a/tests/input.at b/tests/input.at
index c639436f7..5f298bc9a 100644
--- a/tests/input.at
+++ b/tests/input.at
@@ -83,7 +83,8 @@ AT_CLEANUP
 AT_SETUP([Invalid inputs])
 
 AT_DATA([input.y],
-[[\000\001\002\377?
+[[%header "\360\000\200\210"
+\000\001\002\377?
 "\000"
 %%
 ?
@@ -98,37 +99,41 @@ AT_PERL_REQUIRE([[-pi -e 's/\\(\d{3})/chr(oct($1))/ge' 
input.y]])
 AT_BISON_CHECK([-fcaret input.y], [1], [], [stderr])
 
 # Autotest's diffing, when there are NUL bytes, just reports "binary
-# files differ".  So don't leave NUL bytes.
-AT_PERL_CHECK([[-p -e 's{([\0\377])}{sprintf "\\x%02x", ord($1)}ge' stderr]], 
[],
-[[input.y:1.1-2: error: invalid characters: '\0\001\002\377?'
-1 | \x00
\xff?
+# files differ".  So don't leave NUL bytes.  And don't leave invalid
+# mbchars either: escape raw binary.
+AT_PERL_CHECK([[-p -e 's{([\0\200\210\360\377])}{sprintf "\\x%02x", 
ord($1)}ge' stderr]], [],
+[[input.y:1.11: error: invalid null character
+1 | %header "\xf0\x00\x80\x88"
+  |   ^
+input.y:2.1-2: error: invalid characters: '\0\001\002\377?'
+2 | \x00
\xff?
   | ^~
-input.y:2.2: error: invalid null character
-2 | "\x00"
+input.y:3.2: error: invalid null character
+3 | "\x00"
   |  ^
-input.y:4.1: error: invalid character: '?'
-4 | ?
+input.y:5.1: error: invalid character: '?'
+5 | ?
   | ^
-input.y:5.14: error: invalid character: '}'
-5 | default: 'a' }
+input.y:6.14: error: invalid character: '}'
+6 | default: 'a' }
   |  ^
-input.y:6.1: error: invalid character: '%'
-6 | %&
+input.y:7.1: error: invalid character: '%'
+7 | %&
   | ^
-input.y:6.2: error: invalid character: '&'
-6 | %&
+input.y:7.2: error: invalid character: '&'
+7 | %&
   |  ^
-input.y:7.1-17: error: invalid directive: '%a-does-not-exist'
-7 | %a-does-not-exist
+input.y:8.1-17: error: invalid directive: '%a-does-not-exist'
+8 | %a-does-not-exist
   | ^
-input.y:8.1: error: invalid character: '%'
-8 | %-
+input.y:9.1: error: invalid character: '%'
+9 | %-
   | ^
-input.y:8.2: error: invalid character: '-'
-8 | %-
+input.y:9.2: error: invalid character: '-'
+9 | %-
   |  ^
-input.y:9.1-10.0: error: missing '%}' at end of file
-9 | %{
+input.y:10.1-11.0: error: missing '%}' at end of file
+   10 | %{
   | ^~
 ]])
 

Re: I found program for AST generating

[Akim Demaille]

Hi Askar!

> Le 11 juil. 2021 à 01:28, Askar Safin  a écrit :
> 
> Hi. I long time ago I was interested in generating input files for bison (i. 
> e. generating .y files) for producing AST. For example, I asked about it 
> here: https://lists.gnu.org/archive/html/bug-bison/2018-12/msg8.html . 
> Now I found solution to this problem: BNFC. https://bnfc.digitalgrammars.com/ 
> . You write single concise input file and BNFC generates bison's .y file 
> (which parses to AST) from it together with pretty-printer.

That looks very nice, thanks for sharing!

Cheers!

Re: [GNU Bison 3.7.6] testsuite: 328 399 497 502 515 596 597 603 607 failed

[Akim Demaille]

Hi!

> Le 1 juin 2021 à 19:25, Evgeniy  a écrit :
> 
> Hello.
> 
> I'm using bison in my build farm to build wine. OS: SLES 11, GCC 4.8.
> 
> It is strange but 32-bit bison built fine hovewer 64-bit bison failed to 
> build (in the "make check").
> 
> I attach the build log.

We actually need tests/test-suite.log to understand the failure.

Thanks in advance.

Re: quirks in parse-gram.y

[Akim Demaille]

Hi!

> Le 20 avr. 2021 à 04:18, 叶雨飞  a écrit :
> 
> Hello!
> 
> I spent weekend hack together an IDEA plugin to support bison grammar files
> (
> https://github.com/thefallentree/idea-bison
> ) , and discovered some quirks , not sure if worth discussing here:

Nice job!

> 1. In “rules” , the bison grammar file shows that it accept ID_COLON
> namedref_opt ‘:’ ,  ID_COLON is of course “Id” immediately follows by an
> “:”,  my understanding is that this make namedref_opt useless because
> otherwise means you can write “id:[name]:” which doesn’t looks right to
> me.

You're referring to this rule:

rules:
  id_colon named_ref.opt ":" rhses.1

Actually id_colon is more complex than "ID:": it actually means "an
identifier followed by COLON" but does not include the colon, as you
can see in the rule above.  And in fact it means "an identifier
followed by COLON possibly preceded by whitespace, comments, and
name reference".

> 2. The rules grammar indicate an ending “;” is actually optional. In fact,
> the parse-gram.y missed two “;” in separate places.  However the doc is
> pretty clear an “;” is expected.  Not sure which is intended .

Good catch.  The grammar would be definitely much simpler if the semicolon
were mandatory, but YACC compliance forbids this.  So it's the documentation
which is wrong.  I'll fix that, thanks!

Cheers!

Re: error: the default argument for parameter 0 of ... has not yet been parsed

[Akim Demaille]

Hi Jeffrey,

> Le 22 mars 2021 à 09:32, Jeffrey Walton  a écrit :
> 
> On Mon, Mar 22, 2021 at 1:53 AM Jeffrey Walton  wrote:
>> 
>> Hi Everyone,
>> 
>> An updated compiler is out of the question because libcxx has been
>> broken for some time. We can't build a compiler.
>> 
>> In this case, I suggest removal of examples. They add no value for
>> users like me. They only increase the risk of a failed build. Users
>> who want examples can configure with --enable-examples or similar.
>> (Our policies and procedures only require 'make && make check'
>> succeed. We don't care about documentation, examples or other
>> non-essential fodder).

The examples carry part of the test suite.  So I don't want to disable
them.

The actual problem here is that the configure tests did not catch
that your compiler is not trustworthy.  The real fix is to strengthen
these checks.

I'll need your help to make sure the new tests do catch this problem.

> If anyone knows how to disable the C++ examples, it would be greatly
> appreciated. I tried both --disable-examples and --disable-c++, but
> neither stopped the building of the c++ projects.
> 
> I also tried building Bison back to 3.0 but everything fails to build.

Running `./configure CXX=false` will address your concern.

Cheers!

Bison 3.7.6 released

[Akim Demaille]

This release fixes two issues found in Bison 3.7.5.

Bison 3.7 introduced the generation of counterexamples for conflicts,
contributed by Vincent Imbimbo.  For instance on a grammar featuring the
infamous "dangling else" problem, "bison -Wcounterexamples" now gives:

$ bison -Wcounterexamples else.y
else.y: warning: 1 shift/reduce conflict [-Wconflicts-sr]
else.y: warning: shift/reduce conflict on token "else" [-Wcounterexamples]
  Example: "if" exp "then" "if" exp "then" exp • "else" exp
  Shift derivation
exp
↳ "if" exp "then" exp
  ↳ "if" exp "then" exp • "else" exp
  Reduce derivation
exp
↳ "if" exp "then" exp "else" exp
  ↳ "if" exp "then" exp •

which actually proves that the grammar is ambiguous by exhibiting a text
sample with two parse trees.

Cheers!

==

Here are the compressed sources:
  https://ftp.gnu.org/gnu/bison/bison-3.7.6.tar.gz   (5.1MB)
  https://ftp.gnu.org/gnu/bison/bison-3.7.6.tar.lz   (3.1MB)
  https://ftp.gnu.org/gnu/bison/bison-3.7.6.tar.xz   (3.1MB)

Here are the GPG detached signatures[*]:
  https://ftp.gnu.org/gnu/bison/bison-3.7.6.tar.gz.sig
  https://ftp.gnu.org/gnu/bison/bison-3.7.6.tar.lz.sig
  https://ftp.gnu.org/gnu/bison/bison-3.7.6.tar.xz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify bison-3.7.6.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver keys.gnupg.net --recv-keys 0DDCAA3278D5264E

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.71
  Automake 1.16b
  Flex 2.6.4
  Gettext 0.20.1.153-6c39c
  Gnulib v0.1-3995-g839ed059f

==

GNU Bison is a general-purpose parser generator that converts an annotated
context-free grammar into a deterministic LR or generalized LR (GLR) parser
employing LALR(1) parser tables.  Bison can also generate IELR(1) or
canonical LR(1) parser tables.  Once you are proficient with Bison, you can
use it to develop a wide range of language parsers, from those used in
simple desk calculators to complex programming languages.

Bison is upward compatible with Yacc: all properly-written Yacc grammars
work with Bison with no change.  Anyone familiar with Yacc should be able to
use Bison with little trouble.  You need to be fluent in C, C++ or Java
programming in order to use Bison.

Bison and the parsers it generates are portable, they do not require any
specific compilers.

GNU Bison's home page is https://gnu.org/software/bison/.

==

NEWS

* Noteworthy changes in release 3.7.6 (2021-03-07) [stable]

** Bug fixes

*** Reused Push Parsers

  When a push-parser state structure is used for multiple parses, it was
  possible for some state to leak from one run into the following one.

*** Fix Table Generation

  In some very rare conditions, when there are many useless tokens, it was
  possible to generate incorrect parsers.


* Noteworthy changes in release 3.7.5 (2021-01-24) [stable]

** Bug fixes

*** Counterexample Generation

  In some cases counterexample generation could crash.  This is fixed.

*** Fix Table Generation

  In some very rare conditions, when there are many useless tokens, it was
  possible to generate incorrect parsers.

*** GLR parsers now support %merge together with api.value.type=union.

*** C++ parsers use noexcept in more places.

*** Generated parsers avoid some warnings about signedness issues.

*** C-language parsers now avoid warnings from pedantic clang.

*** C-language parsers now work around quirks of HP-UX 11.23 (2003).


* Noteworthy changes in release 3.7.4 (2020-11-14) [stable]

** Bug fixes

*** Bug fixes in yacc.c

  In Yacc mode, all the tokens are defined twice: once as an enum, and then
  as a macro.  YYEMPTY was missing its macro.

*** Bug fixes in lalr1.cc

  The lalr1.cc skeleton used to emit internal assertions (using YY_ASSERT)
  even when the `parse.assert` %define variable is not enabled.  It no
  longer does.

  The private internal macro YY_ASSERT now obeys the `api.prefix` %define
  variable.

  When there is a very large number of tokens, some assertions could be long
  enough to hit arbitrary limits in Visual C++.  They have been rewritten to
  work around this limitation.

** Changes

  The YYBISON macro in generated "regular C parsers" (from the "yacc.c"
  skeleton) used to be defined to 1.  It is now defined to the version of
  Bison as an integer (e.g., 30704 for version 3.7.4).


* N

Re: double-reads with push parser

[Akim Demaille]

Hi all,

> Le 7 mars 2021 à 11:38, Akim Demaille  a écrit :
> 
> https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.5.320-7fb5c.tar.gz
> https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.5.320-7fb5c.tar.lz
> https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.5.320-7fb5c.tar.xz

I have wrapped Bison 3.7.6 and made it available here:

https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.6.tar.gz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.6.tar.lz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.6.tar.xz

I plan to release it on GNU servers today, unless someone catches an issue.

Cheers!

Re: double-reads with push parser

[Akim Demaille]

Hu Ryan,

> Le 7 mars 2021 à 01:33, Ryan  a écrit :
> 
> I've noticed a strange behavior with my push parser on bison 3.7+, where
> when I multiple identical lines through it, the first one parses
> correctly, but all future lines parse the first token of the line twice.
> 
> I'm not sure if this is a bug in bison or if there's something wrong
> with my push loop, but it definitely worked as I expect it to as of
> 3.6.4.
> 
> Code to reproduce and the output of that code is below.

Thanks a lot for the analysis.  I was able to spot when the regression was 
introduced, which is, unsurprisingly 
(https://git.savannah.gnu.org/cgit/bison.git/commit/?id=330552ea499ca474f65967160e9d4e50265f9631):

commit 330552ea499ca474f65967160e9d4e50265f9631
Author: Akim Demaille 
Date:   Sun Jun 28 16:09:13 2020 +0200

yacc.c: push: don't clear the parser state when accepting/rejecting

Currently when a push parser finishes its parsing (i.e., it did not
return YYPUSH_MORE), it also clears its state.  It is therefore
impossible to see if it had parse errors.

In the context of autocompletion, because error recovery might have
fired, the parser is actually already in a different state.  For
instance on `(1 + + ` in the bistromathic, because there's a
`exp: "(" error ")"` recovery rule, `1 + +` tokens have already been
popped, replaced by `error`, and autocompletions think we are ready
for the closing ")".  So here, we would like to see if there was a
syntax error, yet `yynerrs` was cleared.

In the case of a successful parse, we still have a problem: if error
recovery succeeded, we won't know it, since, again, `yynerrs` is
clearer.

It seems much more natural to leave the parser state available for
analysis when there is a failure.

To reuse the parser, we should either:

1. provide an explicit means to reinitialize a parser state for future
   parses.

2. automatically reset the parser state when it is used in a new
   parse.

Option 2 requires to check whether we need to reinitialize the parser
each time we call `yypush_parse`, i.e., each time we give a new token.
This seems expensive compared to Option 1, but benchmarks revealed no
difference.  Option 1 is incompatible with the documentation
("After `yypush_parse` returns a status other than `YYPUSH_MORE`, the
parser instance `yyps` may be reused for a new parse.").

So Option 2 wins, reusing the private `yynew` member to record that a
parse was finished, and therefore that the state must reset in the
next call to `yypull_parse`.

While at it, this implementation now reuses the previously enlarged
stacks from one parse to another.

* data/skeletons/yacc.c (yypstate_new): Set up the stacks in their
initial configurations (setting their bottom to the stack array), and
use yypstate_clear to reset them (moving their top to their bottom).
(yypstate_delete): Adjust.
(yypush_parse): At the beginning, clear yypstate if needed, and at the
end, record when yypstate needs to be clearer.

* examples/c/bistromathic/parse.y (expected_tokens): Do not propose
autocompletion when there are parse errors.
* examples/c/bistromathic/bistromathic.test: Check that case.



Reading this commit again, I don't understand why I had changed the handling of 
yynew in yypush_parse this way.  The following patch does it better, IMHO.

I will push this into the maint(enance) branch, and will very soon release 
3.7.6 with this fix.  Could you please confirm that the Bison in this tarball 
does address your issue?  It is not 3.7.6, it is the current development branch 
(to become 3.8), but it should still be relevant to check your issue.

https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.5.320-7fb5c.tar.gz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.5.320-7fb5c.tar.lz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.5.320-7fb5c.tar.xz

Thanks a lot!

Akim


commit 7fb5cf6790fb0e58917892867742cc29c4263384
Author: Akim Demaille 
Date:   Sun Mar 7 10:01:53 2021 +0100

yacc: fix push parser

When a pstate is used for multiple successive parses, some state may
leak from a previous run into the following one.  That was introduced
in 330552ea499ca474f65967160e9d4e50265f9631 "yacc.c: push: don't clear
the parser state when accepting/rejecting".

Reported by Ryan 
https://lists.gnu.org/r/bug-bison/2021-03/msg0.html

* data/skeletons/yacc.c (yypush_parse): We reusing a pstate from a
previous run, do behave as if it were the first run.
* tests/push.at (Pstate reuse): Check this.

diff --git a/data/skeletons/yacc.c b/data/skeletons/yacc.c
index a59e6aef3..98671322d 100644
--- a/data/skeletons/yacc.c
++

Re: One failed self test on Alpine Linux

[Akim Demaille]

Hi Jeffrey,

Sorry for the slow motion on this issue, but it is quite
delicate to try to understand what is going on with such
huge logs.

> Le 16 janv. 2021 à 22:10, Jeffrey Walton  a écrit :
> 
> On Sat, Jan 16, 2021 at 9:33 AM Akim Demaille  wrote:
>> 
>> ...
>> So please, let's try again.  I have again prepared another tarball.
>> 
>> https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.4.273-d85e.tar.gz
>> https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.4.273-d85e.tar.lz
>> https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.4.273-d85e.tar.xz
>> 
>> make check-local TESTSUITEFLAGS='271 -d'
>> TIME_LIMIT=1 ./src/bison -Tcex -Wcex ./tests/testsuite.dir/271/input.y 
>> >/tmp/271.log 2>&1
> 
> The 271 test on Alpine 3.10:
>  * https://www.cryptopp.com/271-3.log.zip
> 
> The 271 test on Alpine 3.12:
>  * https://www.cryptopp.com/271-4.log.zip
> 
> Maybe you can diff them to quickly determine the difference.

To summarize, you said the test passes in 3.12 but fails in 3.10.
The failure is as follows:

>> input.y: warning: reduce/reduce conflict on token $end [-Wcounterexamples]
>> -  Example: b d .
>> +  First example: b d . $end
>>  First reduce derivation
>> -a
>> -`-> b d .
>> +$accept
>> +`-> a $end
>> +`-> b d .
>> +  Second example: b d . $end
>>  Second reduce derivation
>> -a
>> -`-> b d
>> -  `-> d .
>> +$accept
>> +`-> a $end
>> +`-> b d
>> +  `-> d .

i.e., we have "First example: b d . $end" in the output, although we
expected "Example: b d .".  But in the two logs you sent we appear
to have what is actually expected.

$ grep -E '(First example|Example): b' /tmp/3.10.short.log
4 c:  First example: b • c A A $end
  First example: b c • A
  First example: b c • c A A $end
  First example: b c • A
  Example: b d •
$ grep -E '(First example|Example): b' /tmp/3.12.short.log
4 c:  First example: b • c A A $end
  First example: b c • A
  First example: b c • c A A $end
  First example: b c • A
  Example: b d •

This is very very weird.  Did the test pass?

Meanwhile we have released Bison 3.7.5 which includes a fix for
counterexamples.  I'm convinced the fix is completely unrelated
to your issue.  Could you however try yet again with the following
tarball, which includes the fix of 3.7.5?

https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.5.299-6a6b.tar.gz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.5.299-6a6b.tar.lz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.5.299-6a6b.tar.xz

FTR, the procedure is:

make check-local TESTSUITEFLAGS='271 -d'
TIME_LIMIT=1 ./src/bison -Tcex -Wcex ./tests/testsuite.dir/271/input.y 
>/tmp/271.log 2>&1

Please, be sure to use exactly that version of bison on
both architectures.  Cheers!

Re: [Bison-Announce] Bison 3.7.5 released

[Akim Demaille]

> Le 29 janv. 2021 à 22:51, Paul Eggert  a écrit :
> 
> On 1/29/21 9:19 AM, Frank Heckenbach wrote:
>> BTW, you might want to change this to https (at least in the
>> generated output); the server does a redirect to https now, anyway:
> 
> Thanks for pointing this out. I went through the bison sources and changed 
> http: to https: where warranted, shortening a few when I could. I also fixed 
> a few no-longer-working URLs while I was at it.

Big job!  Thanks a lot Paul!

Re: Bison 3.7.5 released

[Akim Demaille]

Hi Hans,

> Le 25 janv. 2021 à 10:59, Hans Åberg  a écrit :
> 
>> On 24 Jan 2021, at 09:04, Akim Demaille  wrote:
>> 
>> This release fixes several issues…
> 
> When I compiled using clang-11 of MacPorts, I got some warnings.
> ../bison-3.7.5/configure CXX=/opt/local/bin/clang++-mp-11 
> CC=/opt/local/bin/clang-mp-11 CPPFLAGS="-g -I /usr/local/include" LDFLAGS="-L 
> /usr/local/lib" 

Thanks for the report.  I'll see if there is interest in fixing it
in gnulib.  As a matter of fact, they might have fixed it already,
but I couldn't update gnulib in the maint branch, because it prevents
Bison from being compiled with old versions of clang.

Re: Bug (segmentation fault) in the generation of counterexamples

[Akim Demaille]

> Le 24 janv. 2021 à 18:36, Bartkowiak, Michal (Nokia - PL/Wroclaw) 
>  a écrit :
> 
> Hello Akim,
> 
> The bugfix works also with our full grammar

Excellent news!

> (although counterexamples generation is unable to find one in reasonable 
> time).

Ouch.  I very much doubt it can make a difference, but you can play with the 
TIME_LIMIT envvar to give more time (default is 5, meaning 5s).

> Nevertheless, thank you for the fix! Counterexamples generation is indeed a 
> very useful feature.

All the credit should go to Chinawat Isradisaikul and Andrew Myers to have 
designed the feature, to Andrew Myers to have recruited Vincent Imbimbo to 
implement it, and... to Vincent Imbimbo to really have implemented it (and 
maintaining it!).

Cheers!

Bison 3.7.5 released

[Akim Demaille]

This release fixes several issues found in Bison 3.7.4.  Many thanks to the
reporters (Albert Chin, Balázs Scheidler, Joe Nelson, Jot Dot, Martin Rehak,
Michal Bartkowiak, Zartaj Majeed) and to Vincent Imbimbo for addressing the
issues in counterexample generation.

Bison 3.7 introduced the generation of counterexamples for conflicts,
contributed by Vincent Imbimbo.  For instance on a grammar featuring the
infamous "dangling else" problem, "bison -Wcounterexamples" now gives:

$ bison -Wcounterexamples else.y
else.y: warning: 1 shift/reduce conflict [-Wconflicts-sr]
else.y: warning: shift/reduce conflict on token "else" [-Wcounterexamples]
  Example: "if" exp "then" "if" exp "then" exp • "else" exp
  Shift derivation
exp
↳ "if" exp "then" exp
  ↳ "if" exp "then" exp • "else" exp
  Reduce derivation
exp
↳ "if" exp "then" exp "else" exp
  ↳ "if" exp "then" exp •

which actually proves that the grammar is ambiguous by exhibiting a text
sample with two parse trees.

Cheers!

==

Here are the compressed sources:
  https://ftp.gnu.org/gnu/bison/bison-3.7.5.tar.gz   (5.1MB)
  https://ftp.gnu.org/gnu/bison/bison-3.7.5.tar.lz   (3.1MB)
  https://ftp.gnu.org/gnu/bison/bison-3.7.5.tar.xz   (3.1MB)

Here are the GPG detached signatures[*]:
  https://ftp.gnu.org/gnu/bison/bison-3.7.5.tar.gz.sig
  https://ftp.gnu.org/gnu/bison/bison-3.7.5.tar.lz.sig
  https://ftp.gnu.org/gnu/bison/bison-3.7.5.tar.xz.sig

Use a mirror for higher download bandwidth:
  https://www.gnu.org/order/ftp.html

[*] Use a .sig file to verify that the corresponding file (without the
.sig suffix) is intact.  First, be sure to download both the .sig file
and the corresponding tarball.  Then, run a command like this:

  gpg --verify bison-3.7.5.tar.gz.sig

If that command fails because you don't have the required public key,
then run this command to import it:

  gpg --keyserver keys.gnupg.net --recv-keys 0DDCAA3278D5264E

and rerun the 'gpg --verify' command.

This release was bootstrapped with the following tools:
  Autoconf 2.69b.24-e9bfc
  Automake 1.16b
  Flex 2.6.4
  Gettext 0.20.1.153-6c39c
  Gnulib v0.1-3995-g839ed059f

==

GNU Bison is a general-purpose parser generator that converts an annotated
context-free grammar into a deterministic LR or generalized LR (GLR) parser
employing LALR(1) parser tables.  Bison can also generate IELR(1) or
canonical LR(1) parser tables.  Once you are proficient with Bison, you can
use it to develop a wide range of language parsers, from those used in
simple desk calculators to complex programming languages.

Bison is upward compatible with Yacc: all properly-written Yacc grammars
work with Bison with no change.  Anyone familiar with Yacc should be able to
use Bison with little trouble.  You need to be fluent in C, C++ or Java
programming in order to use Bison.

Bison and the parsers it generates are portable, they do not require any
specific compilers.

GNU Bison's home page is https://gnu.org/software/bison/.

==

NEWS

* Noteworthy changes in release 3.7.5 (2021-01-24) [stable]

** Bug fixes

*** Counterexample Generation

  In some cases counterexample generation could crash.  This is fixed.

*** Fix Table Generation

  In some very rare conditions, when there are many useless tokens, it was
  possible to generate incorrect parsers.

*** GLR parsers now support %merge together with api.value.type=union.

*** C++ parsers use noexcept in more places.

*** Generated parsers avoid some warnings about signedness issues.

*** C-language parsers now avoid warnings from pedantic clang.

*** C-language parsers now work around quirks of HP-UX 11.23 (2003).


* Noteworthy changes in release 3.7.4 (2020-11-14) [stable]

** Bug fixes

*** Bug fixes in yacc.c

  In Yacc mode, all the tokens are defined twice: once as an enum, and then
  as a macro.  YYEMPTY was missing its macro.

*** Bug fixes in lalr1.cc

  The lalr1.cc skeleton used to emit internal assertions (using YY_ASSERT)
  even when the `parse.assert` %define variable is not enabled.  It no
  longer does.

  The private internal macro YY_ASSERT now obeys the `api.prefix` %define
  variable.

  When there is a very large number of tokens, some assertions could be long
  enough to hit arbitrary limits in Visual C++.  They have been rewritten to
  work around this limitation.

** Changes

  The YYBISON macro in generated "regular C parsers" (from the "yacc.c"
  skeleton) used to be defined to 1.  It is now defined to the version of
  Bison as an integer (e.g., 30704 for version 3.7.4).


* Noteworthy changes in release 3.7.3 (2020-10-13) [stable]

** Bug fixes

  Fix concurrent build issues.

  The bison executable is no longer linked uselessly against l

Re: Bug (segmentation fault) in the generation of counterexamples

[Akim Demaille]

Michal,

Vincent Imbimbo, the author of cex in Bison, fixed your issue in 
https://lists.gnu.org/r/bison-patches/2021-01/msg00106.html.  Bison 3.7.5 
should be released soon with the fix.

Thanks again for the report!

Re: __UINT_LEAST8_MAX__ and HP-UX 11.23/IA

[Akim Demaille]

> Le 21 janv. 2021 à 20:46, Paul Eggert  a écrit :
> 
> On 1/21/21 10:41 AM, Akim Demaille wrote:
>> I'll install the patch (if Paul didn't) and will release a bug fix
>> asap.
> 
> I installed it to the 'maint' branch on Savannah, which is where I assume you 
> want it. If not, please let me know.

That's perfect, thanks Paul!

Re: __UINT_LEAST8_MAX__ and HP-UX 11.23/IA

[Akim Demaille]

Hi Paul, hi Albert,

> Le 21 janv. 2021 à 03:52, Paul Eggert  a écrit :
> 
> [Resending as the first message had the incorrect "bus-bison" as the return 
> address.]
> 
> On 1/20/21 3:50 PM, Albert Chin wrote:
> 
>>> Unfortunately, on HP-UX 11.23/IA, UINT_LEAST8_MAX evaluates to
>>> ((unsigned char)(255)) causing a problem with:
>>>   #elif (!defined __UINT_LEAST8_MAX__ && defined YY_STDINT_H \
>>>  && UINT_LEAST8_MAX <= INT_MAX)
>> This is also an issue for UINT_LEAST16_MAX.
> That is a known bug in HP-UX 11.23 that Gnulib has had a workaround for since 
> 2009.

Wow :(  This is really bad...

> I suppose Bison cannot assume that Gnulib is in use, unfortunately.

:)

> Does the attached Bison patch fix the problem for you?
> <0001-c-port-to-HP-UX-11.23.patch>

> Le 21 janv. 2021 à 19:09, Albert Chin  a 
> écrit :
> 
> Yes, it would fix it. We're currently doing something like the
> attached.

Ok, I will read this "Yes, it would fix it" as "Yes, it does fix it".
I'll install the patch (if Paul didn't) and will release a bug fix
asap.  Does that sound good for you?

Re: One failed self test on Alpine Linux

[Akim Demaille]

Jeffrey,

> Le 14 janv. 2021 à 20:41, Jeffrey Walton  a écrit :
> 
> On Thu, Jan 14, 2021 at 12:44 AM Akim Demaille  wrote:
> 
> The new run is available at https://www.cryptopp.com/271-2.log.zip. It
> also used the directory bison-3.7.4.254-a0905 (and not
> bison-3.7.4.254-82133), so don't be surprised.

I believe these traces were not issued by the tarball I last gave you.
I made a mistake in the generation of the traces, and your logs do
not include that error.

So please, let's try again.  I have again prepared another tarball.

https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.4.273-d85e.tar.gz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.4.273-d85e.tar.lz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.4.273-d85e.tar.xz

make check-local TESTSUITEFLAGS='271 -d'
TIME_LIMIT=1 ./src/bison -Tcex -Wcex ./tests/testsuite.dir/271/input.y 
>/tmp/271.log 2>&1


> I also installed an Alpine Linux 3.12 system. It tests OK.

That's good news.  I wouldn't be surprised that the problem is actually
coming from Alpine 3.10, but...

> The issue is probably related to Alpine Linux 3.10. Probably something
> to do with BusyBox.

I'm not sure I understand what you mean.  To me, busybox is sort of
a shell with many standard Unix tools baked it, but since bison is
independent of such tools, I do see what kind of problem you think
about.

Cheers!

Re: One failed self test on Alpine Linux

[Akim Demaille]

Hi Jeffrey,

> Le 13 janv. 2021 à 07:24, Jeffrey Walton  a écrit :
> 
> On Wed, Jan 13, 2021 at 12:34 AM Jeffrey Walton  wrote:
> 
> OK, so I am seeing the issue on Alpine in different configurations.
> The additional configurations include:
> 
>  1. machine with no changes to /usr/local
>  2. machine with properly built GetText
> 
> For (1), I deleted /usr/local and rebooted. That should remove any
> contamination I provided.
> 
> For (2), I built GetText tarball, and did not use --with-included-XXX.
> --with-included-XXX means to use the system gear, not the gear from
> the tarball.

Thanks for exploring these scenarios.  But I think it is really
something that goes wrong in bison's own behavior.  It might be
related to an Alpine bug, or in bison itself, I have no idea.

The logs you sent are inconvenient to use because bison was
mixing stdout and stderr output so the result is out of order,
and very hard to compare between two different machines.  I
have fixed this, and also added some support to be able to
change the timeout that is fired in that test (the default,
5s, is way too much here).

Could you please try with this new tarball?

https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.4.258-82133.tar.gz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.4.258-82133.tar.lz
https://www.lrde.epita.fr/~akim/private/bison/bison-3.7.4.258-82133.tar.xz

The quick way to run just the one test we need is:

make check-local TESTSUITEFLAGS='271 -d'
TIME_LIMIT=1 ./src/bison -Tcex -Wcex 
./_build/g9d/tests/testsuite.dir/271/input.y >/tmp/271.log 2>&1

Thanks in advance!

Re: Autoconf options and dependencies

[Akim Demaille]

Jeffrey,

> Le 14 janv. 2021 à 02:21, Jeffrey Walton  a écrit :
> 
> Hi Everyone/Akim,
> 
> I was looking at Bison 3.7.4 autoconf options versus link libraries.
> 
> $ ldd /usr/local/bin/bison
>linux-vdso.so.1
>libiconv.so.2 => /usr/local/bin/../lib/libiconv.so.2
>libtextstyle.so.0 => /usr/local/bin/../lib/libtextstyle.so.0
>libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
>libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
>libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
>libtinfow.so.6 => /usr/local/bin/../lib/libtinfow.so.6
>/lib64/ld-linux-x86-64.so.2
> 
> The libtinfow is from Ncurses, but adding --with-libncurses-prefix=DIR
> to configure results in:
> 
> WARNING: unrecognized options: --with-libncurses-prefix

Well, we really don't use libncurses here.  libtinfow might indeed
come from there, but it is not it, so I personally don't think there
is a problem.

That being said, bison was probably not linked against it directly.
Grep did not find occurrences of tinfow in Bison's tarball.
It was most probably inherited from libtextstyle.so.0, so that's
where you should look for such a feature.