Re: [sqlite] (Lemon) (Patch) adding a -f option to Lemon to emit function prototypes

[Tiago Rodrigues]

I see; well, that's unfortunate.  At any rate, parsing lempar.c will
probably be much more complex than the code I have now, which solves a
simple (but common) case.  Maybe I'll look into parsing lempar.c, but since
this solves my use case, I probably won't be in a hurry to do it.  I
apologise to everyone this doesn't help, then.

Cheers,

  -Tiago


On Mon, Jan 7, 2013 at 9:13 PM, Richard Hipp  wrote:

> On Mon, Jan 7, 2013 at 4:18 PM, Tiago Rodrigues  wrote:
>
> > Oops, Dominique alerted me to the fact that the patch I included was
> > stripped by the listserver...  I'm including it inline, then.
> >
>
>
> This patch does not work in the general case.  I don't doubt that it works
> for your use case, but in general it does not.  The reason:  The function
> prototypes depend on the "lempar.c" template file.  Some applications (ex:
> SQLite) change the lempar.c parser template in ways that could change the
> function signatures.  (The function signatures for SQLite are unchanged,
> but that doesn't exclude the possibility that they might in the future.)
>
> For this to really work in general, I suppose you would have to somehow
> extract the function signatures from the lempar.c template file.  Or maybe
> have a separate lempar.h template that contains the header.  Or something.
>
>
>
> >
> > (Begin patch)
> >
> > --- lemon.c2013-01-04 20:39:20 +
> > +++ lemon-new.c2013-01-04 23:09:59 +
> > @@ -109,7 +109,7 @@
> >  void Reprint(struct lemon *);
> >  void ReportOutput(struct lemon *);
> >  void ReportTable(struct lemon *, int);
> > -void ReportHeader(struct lemon *);
> > +void ReportHeader(struct lemon *, int);
> >  void CompressTables(struct lemon *);
> >  void ResortStates(struct lemon *);
> >
> > @@ -1393,11 +1393,13 @@
> >static int mhflag = 0;
> >static int nolinenosflag = 0;
> >static int noResort = 0;
> > +  static int fpflag = 0;
> >static struct s_options options[] = {
> >  {OPT_FLAG, "b", (char*), "Print only the basis in
> report."},
> >  {OPT_FLAG, "c", (char*), "Don't compress the action
> table."},
> >  {OPT_FSTR, "D", (char*)handle_D_option, "Define an %ifdef macro."},
> >  {OPT_FSTR, "T", (char*)handle_T_option, "Specify a template file."},
> > +{OPT_FLAG, "f", (char*), "Generate function prototypes in
> > header."},
> >  {OPT_FLAG, "g", (char*), "Print grammar without actions."},
> >  {OPT_FLAG, "m", (char*), "Output a makeheaders compatible
> > file."},
> >  {OPT_FLAG, "l", (char*), "Do not print #line
> > statements."},
> > @@ -1502,7 +1504,7 @@
> >  /* Produce a header file for use by the scanner.  (This step is
> >  ** omitted if the "-m" option is used because makeheaders will
> >  ** generate the file for us.) */
> > -if( !mhflag ) ReportHeader();
> > +if( !mhflag ) ReportHeader(, fpflag);
> >}
> >if( statistics ){
> >  printf("Parser statistics: %d terminals, %d nonterminals, %d
> rules\n",
> > @@ -4009,16 +4011,20 @@
> >  }
> >
> >  /* Generate a header file for the parser */
> > -void ReportHeader(struct lemon *lemp)
> > +void ReportHeader(struct lemon *lemp, int fpflag)
> >  {
> >FILE *out, *in;
> > +  const char *name;
> >const char *prefix;
> >char line[LINESIZE];
> >char pattern[LINESIZE];
> >int i;
> > +  int protok = 1;
> >
> >if( lemp->tokenprefix ) prefix = lemp->tokenprefix;
> >elseprefix = "";
> > +  if( lemp->name ) name = lemp->name;
> > +  else name = "Parse";
> >in = file_open(lemp,".h","rb");
> >if( in ){
> >  int nextChar;
> > @@ -4026,9 +4032,24 @@
> >sprintf(pattern,"#define %s%-30s
> > %2d\n",prefix,lemp->symbols[i]->name,i);
> >if( strcmp(line,pattern) ) break;
> >  }
> > +if( fpflag ){
> > +sprintf(pattern,"void *%sAlloc(void *(*)(size_t));\n",name);
> > +if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
> > 0; goto after; }
> > +
> > +if( lemp->arg ){
> > +sprintf(pattern,"void %s(void
> > *,int,%s,%s);\n",name,lemp->tokentype,lemp->arg);
> > +}else{
> > +sprintf(pattern,"void %s(void
> > *,int,%s);\n",name,lemp->tokentype);
> > +}
> > +if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
> > 0; goto after; }
> > +
> > +sprintf(pattern,"void *%sFree(void *,void (*)(void
> *));\n",name);
> > +if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
> > 0; goto after; }
> > +}
> > +after:
> >  nextChar = fgetc(in);
> >  fclose(in);
> > -if( i==lemp->nterminal && nextChar==EOF ){
> > +if( i==lemp->nterminal && protok && nextChar==EOF ){
> >/* No change in the file.  Don't rewrite it. */
> >return;
> >  }
> > @@ -4038,6 +4059,16 @@
> >  for(i=1; interminal; i++){
> >fprintf(out,"#define %s%-30s
> > %2d\n",prefix,lemp->symbols[i]->name,i);
> >  }
> > +

Re: [sqlite] (Lemon) (Patch) adding a -f option to Lemon to emit function prototypes

[Richard Hipp]

On Mon, Jan 7, 2013 at 4:18 PM, Tiago Rodrigues  wrote:

> Oops, Dominique alerted me to the fact that the patch I included was
> stripped by the listserver...  I'm including it inline, then.
>


This patch does not work in the general case.  I don't doubt that it works
for your use case, but in general it does not.  The reason:  The function
prototypes depend on the "lempar.c" template file.  Some applications (ex:
SQLite) change the lempar.c parser template in ways that could change the
function signatures.  (The function signatures for SQLite are unchanged,
but that doesn't exclude the possibility that they might in the future.)

For this to really work in general, I suppose you would have to somehow
extract the function signatures from the lempar.c template file.  Or maybe
have a separate lempar.h template that contains the header.  Or something.



>
> (Begin patch)
>
> --- lemon.c2013-01-04 20:39:20 +
> +++ lemon-new.c2013-01-04 23:09:59 +
> @@ -109,7 +109,7 @@
>  void Reprint(struct lemon *);
>  void ReportOutput(struct lemon *);
>  void ReportTable(struct lemon *, int);
> -void ReportHeader(struct lemon *);
> +void ReportHeader(struct lemon *, int);
>  void CompressTables(struct lemon *);
>  void ResortStates(struct lemon *);
>
> @@ -1393,11 +1393,13 @@
>static int mhflag = 0;
>static int nolinenosflag = 0;
>static int noResort = 0;
> +  static int fpflag = 0;
>static struct s_options options[] = {
>  {OPT_FLAG, "b", (char*), "Print only the basis in report."},
>  {OPT_FLAG, "c", (char*), "Don't compress the action table."},
>  {OPT_FSTR, "D", (char*)handle_D_option, "Define an %ifdef macro."},
>  {OPT_FSTR, "T", (char*)handle_T_option, "Specify a template file."},
> +{OPT_FLAG, "f", (char*), "Generate function prototypes in
> header."},
>  {OPT_FLAG, "g", (char*), "Print grammar without actions."},
>  {OPT_FLAG, "m", (char*), "Output a makeheaders compatible
> file."},
>  {OPT_FLAG, "l", (char*), "Do not print #line
> statements."},
> @@ -1502,7 +1504,7 @@
>  /* Produce a header file for use by the scanner.  (This step is
>  ** omitted if the "-m" option is used because makeheaders will
>  ** generate the file for us.) */
> -if( !mhflag ) ReportHeader();
> +if( !mhflag ) ReportHeader(, fpflag);
>}
>if( statistics ){
>  printf("Parser statistics: %d terminals, %d nonterminals, %d rules\n",
> @@ -4009,16 +4011,20 @@
>  }
>
>  /* Generate a header file for the parser */
> -void ReportHeader(struct lemon *lemp)
> +void ReportHeader(struct lemon *lemp, int fpflag)
>  {
>FILE *out, *in;
> +  const char *name;
>const char *prefix;
>char line[LINESIZE];
>char pattern[LINESIZE];
>int i;
> +  int protok = 1;
>
>if( lemp->tokenprefix ) prefix = lemp->tokenprefix;
>elseprefix = "";
> +  if( lemp->name ) name = lemp->name;
> +  else name = "Parse";
>in = file_open(lemp,".h","rb");
>if( in ){
>  int nextChar;
> @@ -4026,9 +4032,24 @@
>sprintf(pattern,"#define %s%-30s
> %2d\n",prefix,lemp->symbols[i]->name,i);
>if( strcmp(line,pattern) ) break;
>  }
> +if( fpflag ){
> +sprintf(pattern,"void *%sAlloc(void *(*)(size_t));\n",name);
> +if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
> 0; goto after; }
> +
> +if( lemp->arg ){
> +sprintf(pattern,"void %s(void
> *,int,%s,%s);\n",name,lemp->tokentype,lemp->arg);
> +}else{
> +sprintf(pattern,"void %s(void
> *,int,%s);\n",name,lemp->tokentype);
> +}
> +if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
> 0; goto after; }
> +
> +sprintf(pattern,"void *%sFree(void *,void (*)(void *));\n",name);
> +if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
> 0; goto after; }
> +}
> +after:
>  nextChar = fgetc(in);
>  fclose(in);
> -if( i==lemp->nterminal && nextChar==EOF ){
> +if( i==lemp->nterminal && protok && nextChar==EOF ){
>/* No change in the file.  Don't rewrite it. */
>return;
>  }
> @@ -4038,6 +4059,16 @@
>  for(i=1; interminal; i++){
>fprintf(out,"#define %s%-30s
> %2d\n",prefix,lemp->symbols[i]->name,i);
>  }
> +if( fpflag ){
> +/* emit function prototypes */
> +fprintf(out,"void *%sAlloc(void *(*)(size_t));\n",name);
> +if( lemp->arg ){
> +fprintf(out,"void %s(void
> *,int,%s,%s);\n",name,lemp->tokentype,lemp->arg);
> +}else{
> +fprintf(out,"void %s(void *,int,%s);\n",name,lemp->tokentype);
> +}
> +fprintf(out,"void *%sFree(void *,void (*)(void *));\n",name);
> +}
>  fclose(out);
>}
>return;
>
> (End patch)
>
> Thanks,
>
>   -Tiago
>
>
> On Mon, Jan 7, 2013 at 6:15 AM, Dominique Devienne  >wrote:
>
> > On Sat, Jan 5, 2013 at 6:22 PM, Tiago Rodrigues 

Re: [sqlite] (Lemon) (Patch) adding a -f option to Lemon to emit function prototypes

[Tiago Rodrigues]

Oops, Dominique alerted me to the fact that the patch I included was
stripped by the listserver...  I'm including it inline, then.

(Begin patch)

--- lemon.c2013-01-04 20:39:20 +
+++ lemon-new.c2013-01-04 23:09:59 +
@@ -109,7 +109,7 @@
 void Reprint(struct lemon *);
 void ReportOutput(struct lemon *);
 void ReportTable(struct lemon *, int);
-void ReportHeader(struct lemon *);
+void ReportHeader(struct lemon *, int);
 void CompressTables(struct lemon *);
 void ResortStates(struct lemon *);

@@ -1393,11 +1393,13 @@
   static int mhflag = 0;
   static int nolinenosflag = 0;
   static int noResort = 0;
+  static int fpflag = 0;
   static struct s_options options[] = {
 {OPT_FLAG, "b", (char*), "Print only the basis in report."},
 {OPT_FLAG, "c", (char*), "Don't compress the action table."},
 {OPT_FSTR, "D", (char*)handle_D_option, "Define an %ifdef macro."},
 {OPT_FSTR, "T", (char*)handle_T_option, "Specify a template file."},
+{OPT_FLAG, "f", (char*), "Generate function prototypes in
header."},
 {OPT_FLAG, "g", (char*), "Print grammar without actions."},
 {OPT_FLAG, "m", (char*), "Output a makeheaders compatible
file."},
 {OPT_FLAG, "l", (char*), "Do not print #line
statements."},
@@ -1502,7 +1504,7 @@
 /* Produce a header file for use by the scanner.  (This step is
 ** omitted if the "-m" option is used because makeheaders will
 ** generate the file for us.) */
-if( !mhflag ) ReportHeader();
+if( !mhflag ) ReportHeader(, fpflag);
   }
   if( statistics ){
 printf("Parser statistics: %d terminals, %d nonterminals, %d rules\n",
@@ -4009,16 +4011,20 @@
 }

 /* Generate a header file for the parser */
-void ReportHeader(struct lemon *lemp)
+void ReportHeader(struct lemon *lemp, int fpflag)
 {
   FILE *out, *in;
+  const char *name;
   const char *prefix;
   char line[LINESIZE];
   char pattern[LINESIZE];
   int i;
+  int protok = 1;

   if( lemp->tokenprefix ) prefix = lemp->tokenprefix;
   elseprefix = "";
+  if( lemp->name ) name = lemp->name;
+  else name = "Parse";
   in = file_open(lemp,".h","rb");
   if( in ){
 int nextChar;
@@ -4026,9 +4032,24 @@
   sprintf(pattern,"#define %s%-30s
%2d\n",prefix,lemp->symbols[i]->name,i);
   if( strcmp(line,pattern) ) break;
 }
+if( fpflag ){
+sprintf(pattern,"void *%sAlloc(void *(*)(size_t));\n",name);
+if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
0; goto after; }
+
+if( lemp->arg ){
+sprintf(pattern,"void %s(void
*,int,%s,%s);\n",name,lemp->tokentype,lemp->arg);
+}else{
+sprintf(pattern,"void %s(void
*,int,%s);\n",name,lemp->tokentype);
+}
+if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
0; goto after; }
+
+sprintf(pattern,"void *%sFree(void *,void (*)(void *));\n",name);
+if( !fgets(line,LINESIZE,in) || strcmp(line,pattern) ){ protok =
0; goto after; }
+}
+after:
 nextChar = fgetc(in);
 fclose(in);
-if( i==lemp->nterminal && nextChar==EOF ){
+if( i==lemp->nterminal && protok && nextChar==EOF ){
   /* No change in the file.  Don't rewrite it. */
   return;
 }
@@ -4038,6 +4059,16 @@
 for(i=1; interminal; i++){
   fprintf(out,"#define %s%-30s %2d\n",prefix,lemp->symbols[i]->name,i);
 }
+if( fpflag ){
+/* emit function prototypes */
+fprintf(out,"void *%sAlloc(void *(*)(size_t));\n",name);
+if( lemp->arg ){
+fprintf(out,"void %s(void
*,int,%s,%s);\n",name,lemp->tokentype,lemp->arg);
+}else{
+fprintf(out,"void %s(void *,int,%s);\n",name,lemp->tokentype);
+}
+fprintf(out,"void *%sFree(void *,void (*)(void *));\n",name);
+}
 fclose(out);
   }
   return;

(End patch)

Thanks,

  -Tiago


On Mon, Jan 7, 2013 at 6:15 AM, Dominique Devienne wrote:

> On Sat, Jan 5, 2013 at 6:22 PM, Tiago Rodrigues  wrote:
> > [...]
> > This patch works against the current version of lemon.c, as linked to by
> > the Lemon page.  This is probably not correct, as the file is amalgamated
> > from other sources, but still might be useful to somebody.
>
> Hi Tiago. Looks like the patch was stripped by the mailing list.
> Perhaps you should include it inline to your message. Thanks, --DD
>



-- 
In those days, in those distant days, in those nights, in those remote
nights, in those years, in those distant years...
  - Gilgamesh, Enkidu and the Underworld
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] (Lemon) (Patch) adding a -f option to Lemon to emit function prototypes

[Tiago Rodrigues]

Hello, all,

First, forgive me if this is the wrong medium for submitting this, but I'm
rather new at this patch submission business.  Anyway, I was using Lemon to
generate a few parsers for a program I'm writing (which uses SQLite, too,
for storage), and I noticed that the files I used to call the parser were
giving me a strange "Pointer made from integer without a cast" on
ParseAlloc(), which is strange, since ParseAlloc() returns a void *.  It
took me a while to figure out that ParseAlloc(), ParseFree() and Parse()
weren't actually declared anywhere, so the compiler was assuming they
received no parameters and returned int.  Since I have more than one parser
to work with, I decided to have lemon emit the function prototypes on the
header file upon a new (-f) flag.  Then I decided that I might as well
submit this back to the community and let it decide whether the inclusion
is useful or not.

This patch works against the current version of lemon.c, as linked to by
the Lemon page.  This is probably not correct, as the file is amalgamated
from other sources, but still might be useful to somebody.

Thanks for reading,

  -Tiago

-- 
In those days, in those distant days, in those nights, in those remote
nights, in those years, in those distant years...
  - Gilgamesh, Enkidu and the Underworld
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users