bug#54390: YACC rules don't check DESTDIR existence for VPATH builds

2023-01-12 Thread Sam James 


> On 13 Jan 2023, at 07:01, Mike Frysinger  wrote:
> 
> On 13 Jan 2023 06:29, Sam James wrote:
>>> On 13 Jan 2023, at 06:13, Mike Frysinger  wrote:
>>> On 14 Mar 2022 17:21, Sam James wrote:
 It appears that YACC rules don't check for whether the destination 
 directory exists before executing ylwrap.
 
 When trying to package libaacs 
 (https://code.videolan.org/videolan/libaacs) with an out-of-source build, 
 I hit an unexpected build failure:
 ```
 /var/tmp/portage/media-libs/libaacs-0.11.1/work/libaacs-0.11.1/src/file/keydbcfg-parser.y:
  warning: fix-its can be applied.  Rerun with option '--update'. [-Wother]
 /var/tmp/portage/media-libs/libaacs-0.11.1/work/libaacs-0.11.1/build-aux/ylwrap:
  206: cannot create ../src/file/keydbcfg-parser.c: Directory nonexistent
 updating src/file/keydbcfg-parser.h
 mv: cannot move 'tmp-keydbcfg-parser.h' to 
 '../src/file/keydbcfg-parser.h': No such file or directory
 make: *** [Makefile:1150: src/file/keydbcfg-parser.c] Error 2
 ```
 
 I can workaround this by running `mkdir -p ${BUILD_DIR}/src/file` to 
 ensure that the necessary directory exists within the build directory 
 beforehand, but
 it feels like I shouldn't have to.
 
 Their Makefile.am can be found here: 
 https://code.videolan.org/videolan/libaacs/-/blob/master/Makefile.am. 
 Snippet:
 ```
 libaacs_la_SOURCES=\
 src/libaacs/aacs.h \
 [...]
 src/file/dirs.h \
 src/file/file.h \
 src/file/file.c \
 src/file/filesystem.h \
 src/file/filesystem.c \
 src/file/keydbcfg.c \
 src/file/keydbcfg.h \
 src/file/keydb.h \
 src/file/keydbcfg-parser.y \
 src/file/keydbcfg-lexer.l \
 src/file/mmc_device.h \
 [...]
 ```
 
 While src/libaacs exists within the build dir, src/file/ doesn't exist at 
 all, hence the failure.
 
 automake yacc rules should mkdir -p the needed directories within the 
 build dir for VPATH builds before running ylwrap/yacc.
>>> 
>>> i think there's more to it.  if you're using a release tarball for this 
>>> project
>>> created by `make dist`, then you shouldn't be running yacc in the first 
>>> place.
>>> 
>>> https://www.gnu.org/software/automake/manual/html_node/Yacc-and-Lex.html
>> 
>> Curiosity got the better of me (see previous reply) so:
>> ```
>> $ cd /tmp
>> $ git clone https://code.videolan.org/videolan/libaacs/
>> $(cd libaacs && ./bootstrap.sh)
>> $mkdir libaacs-oos && cd libaacs
> 
> guessing you meant `cd libaacs-oos`
> 
>> $ /tmp/libaacs/configure YACC=bison LEX=flex
>> $ make
>> $ make
>>  YACC src/file/keydbcfg-parser.c
>> keydbcfg-parser.tab.c is unchanged
>> keydbcfg-parser.tab.h is unchanged
>> make[1]: Entering directory '/tmp/libaacs-oos'
>>  YACC src/file/keydbcfg-parser.c
>> keydbcfg-parser.tab.c is unchanged
>> keydbcfg-parser.tab.h is unchanged
>> make[1]: Leaving directory '/tmp/libaacs-oos'
>> make  all-am
>> make[1]: Entering directory '/tmp/libaacs-oos'
>>  YACC src/file/keydbcfg-parser.c
>> keydbcfg-parser.tab.c is unchanged
>> keydbcfg-parser.tab.h is unchanged
>>  CC   src/file/keydbcfg-parser.lo
>> cc1: fatal error: src/file/keydbcfg-parser.c: No such file or directory
>> compilation terminated.
>> make[1]: *** [Makefile:1009: src/file/keydbcfg-parser.lo] Error 1
>> make[1]: Leaving directory '/tmp/libaacs-oos'
>> make: *** [Makefile:638: all] Error 2
>> ```
>> 
>> so it's still looking in the source rather than the build directory for the 
>> generated file?
> 
> i'm not sure what you're trying to show here.  that running make twice in a
> row produces weird/inconsistent results ?  if that's the case, that's not
> the behavior i'm seeing over here.
> 
No, that was just an error in copying stuff into email. It happens with running 
make once,
so it should be:
```
$ cd /tmp
$ git clone https://code.videolan.org/videolan/libaacs/
$ (cd libaacs && ./bootstrap)
$ mkdir libaacs-oos && cd libaacs-oos
$ /tmp/libaacs/configure YACC=bison LEX=flex
$ make
  YACC src/file/keydbcfg-parser.c
updating keydbcfg-parser.tab.c
updating keydbcfg-parser.tab.h
make[1]: Entering directory '/tmp/libaacs-oos'
  YACC src/file/keydbcfg-parser.c
keydbcfg-parser.tab.c is unchanged
keydbcfg-parser.tab.h is unchanged
make[1]: Leaving directory '/tmp/libaacs-oos'
  LEX  src/file/keydbcfg-lexer.c
make  all-am
make[1]: Entering directory '/tmp/libaacs-oos'
  CC   src/examples/aacs_info-aacs_info.o
  CC   src/libaacs/aacs.lo
  CC   src/libaacs/cci.lo
  CC   src/libaacs/content_cert.lo
  CC   src/libaacs/crypto.lo
  CC   src/libaacs/mkb.lo
  CC   src/libaacs/mmc.lo
  CC   src/libaacs/unit_key.lo
  CC   src/file/file.lo
  CC   src/file/filesystem.lo
  CC   src/file/keydbcfg.lo
  YACC src/file/keydbcfg-parser.c
keydbcfg-parser.tab.c is unchanged
keydbcfg-parser.tab.h is unchanged
  CC   src/file/keydbcfg-parser.lo
cc1: fatal

bug#54390: YACC rules don't check DESTDIR existence for VPATH builds

2023-01-12 Thread Mike Frysinger 
On 13 Jan 2023 06:29, Sam James wrote:
> > On 13 Jan 2023, at 06:13, Mike Frysinger  wrote:
> > On 14 Mar 2022 17:21, Sam James wrote:
> >> It appears that YACC rules don't check for whether the destination 
> >> directory exists before executing ylwrap.
> >> 
> >> When trying to package libaacs 
> >> (https://code.videolan.org/videolan/libaacs) with an out-of-source build, 
> >> I hit an unexpected build failure:
> >> ```
> >> /var/tmp/portage/media-libs/libaacs-0.11.1/work/libaacs-0.11.1/src/file/keydbcfg-parser.y:
> >>  warning: fix-its can be applied.  Rerun with option '--update'. [-Wother]
> >> /var/tmp/portage/media-libs/libaacs-0.11.1/work/libaacs-0.11.1/build-aux/ylwrap:
> >>  206: cannot create ../src/file/keydbcfg-parser.c: Directory nonexistent
> >> updating src/file/keydbcfg-parser.h
> >> mv: cannot move 'tmp-keydbcfg-parser.h' to 
> >> '../src/file/keydbcfg-parser.h': No such file or directory
> >> make: *** [Makefile:1150: src/file/keydbcfg-parser.c] Error 2
> >> ```
> >> 
> >> I can workaround this by running `mkdir -p ${BUILD_DIR}/src/file` to 
> >> ensure that the necessary directory exists within the build directory 
> >> beforehand, but
> >> it feels like I shouldn't have to.
> >> 
> >> Their Makefile.am can be found here: 
> >> https://code.videolan.org/videolan/libaacs/-/blob/master/Makefile.am. 
> >> Snippet:
> >> ```
> >> libaacs_la_SOURCES=\
> >> src/libaacs/aacs.h \
> >> [...]
> >> src/file/dirs.h \
> >> src/file/file.h \
> >> src/file/file.c \
> >> src/file/filesystem.h \
> >> src/file/filesystem.c \
> >> src/file/keydbcfg.c \
> >> src/file/keydbcfg.h \
> >> src/file/keydb.h \
> >> src/file/keydbcfg-parser.y \
> >> src/file/keydbcfg-lexer.l \
> >> src/file/mmc_device.h \
> >> [...]
> >> ```
> >> 
> >> While src/libaacs exists within the build dir, src/file/ doesn't exist at 
> >> all, hence the failure.
> >> 
> >> automake yacc rules should mkdir -p the needed directories within the 
> >> build dir for VPATH builds before running ylwrap/yacc.
> > 
> > i think there's more to it.  if you're using a release tarball for this 
> > project
> > created by `make dist`, then you shouldn't be running yacc in the first 
> > place.
> > 
> > https://www.gnu.org/software/automake/manual/html_node/Yacc-and-Lex.html
> 
> Curiosity got the better of me (see previous reply) so:
> ```
> $ cd /tmp
> $ git clone https://code.videolan.org/videolan/libaacs/
> $(cd libaacs && ./bootstrap.sh)
> $mkdir libaacs-oos && cd libaacs

guessing you meant `cd libaacs-oos`

> $ /tmp/libaacs/configure YACC=bison LEX=flex
> $ make
> $ make
>   YACC src/file/keydbcfg-parser.c
> keydbcfg-parser.tab.c is unchanged
> keydbcfg-parser.tab.h is unchanged
> make[1]: Entering directory '/tmp/libaacs-oos'
>   YACC src/file/keydbcfg-parser.c
> keydbcfg-parser.tab.c is unchanged
> keydbcfg-parser.tab.h is unchanged
> make[1]: Leaving directory '/tmp/libaacs-oos'
> make  all-am
> make[1]: Entering directory '/tmp/libaacs-oos'
>   YACC src/file/keydbcfg-parser.c
> keydbcfg-parser.tab.c is unchanged
> keydbcfg-parser.tab.h is unchanged
>   CC   src/file/keydbcfg-parser.lo
> cc1: fatal error: src/file/keydbcfg-parser.c: No such file or directory
> compilation terminated.
> make[1]: *** [Makefile:1009: src/file/keydbcfg-parser.lo] Error 1
> make[1]: Leaving directory '/tmp/libaacs-oos'
> make: *** [Makefile:638: all] Error 2
> ```
> 
> so it's still looking in the source rather than the build directory for the 
> generated file?

i'm not sure what you're trying to show here.  that running make twice in a
row produces weird/inconsistent results ?  if that's the case, that's not
the behavior i'm seeing over here.

i'll note that the default configure+make works because dependency tracking
is enabled which prepopulates the directory tree.  it's failing for you in
Gentoo because econf automatically adds --disable-dependency-tracking which
turns all that logic off, and the directory tree doesn't exist.

Automake is VPATH-ing the source file.  we prob should change that to use an
explicit $(srcdir) prefix when nodist is not utilized.  then it would always
write the generated .c/.h to the source tree.
-mike


signature.asc
Description: PGP signature

bug#54390: YACC rules don't check DESTDIR existence for VPATH builds

2023-01-12 Thread Sam James 


> On 13 Jan 2023, at 06:13, Mike Frysinger  wrote:
> 
> On 14 Mar 2022 17:21, Sam James wrote:
>> It appears that YACC rules don't check for whether the destination directory 
>> exists before executing ylwrap.
>> 
>> When trying to package libaacs (https://code.videolan.org/videolan/libaacs) 
>> with an out-of-source build, I hit an unexpected build failure:
>> ```
>> /var/tmp/portage/media-libs/libaacs-0.11.1/work/libaacs-0.11.1/src/file/keydbcfg-parser.y:
>>  warning: fix-its can be applied.  Rerun with option '--update'. [-Wother]
>> /var/tmp/portage/media-libs/libaacs-0.11.1/work/libaacs-0.11.1/build-aux/ylwrap:
>>  206: cannot create ../src/file/keydbcfg-parser.c: Directory nonexistent
>> updating src/file/keydbcfg-parser.h
>> mv: cannot move 'tmp-keydbcfg-parser.h' to '../src/file/keydbcfg-parser.h': 
>> No such file or directory
>> make: *** [Makefile:1150: src/file/keydbcfg-parser.c] Error 2
>> ```
>> 
>> I can workaround this by running `mkdir -p ${BUILD_DIR}/src/file` to ensure 
>> that the necessary directory exists within the build directory beforehand, 
>> but
>> it feels like I shouldn't have to.
>> 
>> Their Makefile.am can be found here: 
>> https://code.videolan.org/videolan/libaacs/-/blob/master/Makefile.am. 
>> Snippet:
>> ```
>> libaacs_la_SOURCES=\
>> src/libaacs/aacs.h \
>> [...]
>> src/file/dirs.h \
>> src/file/file.h \
>> src/file/file.c \
>> src/file/filesystem.h \
>> src/file/filesystem.c \
>> src/file/keydbcfg.c \
>> src/file/keydbcfg.h \
>> src/file/keydb.h \
>> src/file/keydbcfg-parser.y \
>> src/file/keydbcfg-lexer.l \
>> src/file/mmc_device.h \
>> [...]
>> ```
>> 
>> While src/libaacs exists within the build dir, src/file/ doesn't exist at 
>> all, hence the failure.
>> 
>> automake yacc rules should mkdir -p the needed directories within the build 
>> dir for VPATH builds before running ylwrap/yacc.
> 
> i think there's more to it.  if you're using a release tarball for this 
> project
> created by `make dist`, then you shouldn't be running yacc in the first place.
> 
> https://www.gnu.org/software/automake/manual/html_node/Yacc-and-Lex.html

Curiosity got the better of me (see previous reply) so:
```
$ cd /tmp
$ git clone https://code.videolan.org/videolan/libaacs/
$(cd libaacs && ./bootstrap.sh)
$mkdir libaacs-oos && cd libaacs
$ /tmp/libaacs/configure YACC=bison LEX=flex
$ make
$ make
  YACC src/file/keydbcfg-parser.c
keydbcfg-parser.tab.c is unchanged
keydbcfg-parser.tab.h is unchanged
make[1]: Entering directory '/tmp/libaacs-oos'
  YACC src/file/keydbcfg-parser.c
keydbcfg-parser.tab.c is unchanged
keydbcfg-parser.tab.h is unchanged
make[1]: Leaving directory '/tmp/libaacs-oos'
make  all-am
make[1]: Entering directory '/tmp/libaacs-oos'
  YACC src/file/keydbcfg-parser.c
keydbcfg-parser.tab.c is unchanged
keydbcfg-parser.tab.h is unchanged
  CC   src/file/keydbcfg-parser.lo
cc1: fatal error: src/file/keydbcfg-parser.c: No such file or directory
compilation terminated.
make[1]: *** [Makefile:1009: src/file/keydbcfg-parser.lo] Error 1
make[1]: Leaving directory '/tmp/libaacs-oos'
make: *** [Makefile:638: all] Error 2
```

so it's still looking in the source rather than the build directory for the 
generated file?


signature.asc
Description: Message signed with OpenPGP

bug#54390: YACC rules don't check DESTDIR existence for VPATH builds

2023-01-12 Thread Sam James 


> On 13 Jan 2023, at 06:13, Mike Frysinger  wrote:
> 
> On 14 Mar 2022 17:21, Sam James wrote:
>> It appears that YACC rules don't check for whether the destination directory 
>> exists before executing ylwrap.
>> 
>> When trying to package libaacs (https://code.videolan.org/videolan/libaacs) 
>> with an out-of-source build, I hit an unexpected build failure:
>> ```
>> /var/tmp/portage/media-libs/libaacs-0.11.1/work/libaacs-0.11.1/src/file/keydbcfg-parser.y:
>>  warning: fix-its can be applied.  Rerun with option '--update'. [-Wother]
>> /var/tmp/portage/media-libs/libaacs-0.11.1/work/libaacs-0.11.1/build-aux/ylwrap:
>>  206: cannot create ../src/file/keydbcfg-parser.c: Directory nonexistent
>> updating src/file/keydbcfg-parser.h
>> mv: cannot move 'tmp-keydbcfg-parser.h' to '../src/file/keydbcfg-parser.h': 
>> No such file or directory
>> make: *** [Makefile:1150: src/file/keydbcfg-parser.c] Error 2
>> ```
>> 
>> I can workaround this by running `mkdir -p ${BUILD_DIR}/src/file` to ensure 
>> that the necessary directory exists within the build directory beforehand, 
>> but
>> it feels like I shouldn't have to.
>> 
>> Their Makefile.am can be found here: 
>> https://code.videolan.org/videolan/libaacs/-/blob/master/Makefile.am. 
>> Snippet:
>> ```
>> libaacs_la_SOURCES=\
>> src/libaacs/aacs.h \
>> [...]
>> src/file/dirs.h \
>> src/file/file.h \
>> src/file/file.c \
>> src/file/filesystem.h \
>> src/file/filesystem.c \
>> src/file/keydbcfg.c \
>> src/file/keydbcfg.h \
>> src/file/keydb.h \
>> src/file/keydbcfg-parser.y \
>> src/file/keydbcfg-lexer.l \
>> src/file/mmc_device.h \
>> [...]
>> ```
>> 
>> While src/libaacs exists within the build dir, src/file/ doesn't exist at 
>> all, hence the failure.
>> 
>> automake yacc rules should mkdir -p the needed directories within the build 
>> dir for VPATH builds before running ylwrap/yacc.
> 
> i think there's more to it.  if you're using a release tarball for this 
> project
> created by `make dist`, then you shouldn't be running yacc in the first place.
> 
> https://www.gnu.org/software/automake/manual/html_node/Yacc-and-Lex.html
>> The intermediate files generated by yacc (or lex) will be included in any
>> distribution that is made. That way the user doesn’t need to have yacc or 
>> lex.
> 
> cloning that repo and running `make dist` shows the file is generated, and 
> it's
> listed in am__DIST_COMMON, but it still doesn't make it into the tarball.  oh,
> it looks like this is self-inflicted:
> https://code.videolan.org/videolan/libaacs/-/blob/0.11.1/Makefile.am#L88
> https://code.videolan.org/videolan/libaacs/-/commit/f60f46da1dc5e87f70b6edc965a8909d3f21c247
> 
> that makes no sense.  file a bug with them.

Thanks - IIRC I hit this when running from git too, which is why I filed the 
bug, but that's definitely
wrong on their side anyway, and I can't say I want to bother trying to build it 
again from git.

I'll report it over there.



signature.asc
Description: Message signed with OpenPGP

bug#55025: Automake should allow one to enable POSIX make behavior

2023-01-12 Thread Mike Frysinger 
On 19 Apr 2022 17:33, Vincent Lefevre wrote:
> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/make.html
> says about the target rules:
> 
>   .POSIX
> The application shall ensure that this special target is specified
> without prerequisites or commands. If it appears as the first
> non-comment line in the makefile, /make/ shall process the makefile
> as specified by this section; otherwise, the behavior of /make/ is
> unspecified.
> 
> But even though one may add a .POSIX target as the first non-comment
> line in one's Makefile.am file, Automake will add various non-comment
> lines before this target in the generated Makefile. I received a
> remark about that for GNU MPFR. Though GNU make does not require
> this target to be the first non-comment line, this may matter with
> other make implementations.
> 
> This could be done either by detecting a .POSIX target in Makefile.am
> or with some AM_* macro in the configure.ac file.

any reason we don't just define it ourselves unconditionally ?  seems
like the whole point of Automake is for devs to not worry about these
kind of nitty details.
-mike

--- a/lib/am/header-vars.am
+++ b/lib/am/header-vars.am
@@ -14,6 +14,8 @@
 ## You should have received a copy of the GNU General Public License
 ## along with this program.  If not, see .
 
+.POSIX:
+
 VPATH = @srcdir@
 
 @SET_MAKE@


signature.asc
Description: PGP signature

bug#54390: YACC rules don't check DESTDIR existence for VPATH builds

2023-01-12 Thread Mike Frysinger 
On 14 Mar 2022 17:21, Sam James wrote:
> It appears that YACC rules don't check for whether the destination directory 
> exists before executing ylwrap.
> 
> When trying to package libaacs (https://code.videolan.org/videolan/libaacs) 
> with an out-of-source build, I hit an unexpected build failure:
> ```
> /var/tmp/portage/media-libs/libaacs-0.11.1/work/libaacs-0.11.1/src/file/keydbcfg-parser.y:
>  warning: fix-its can be applied.  Rerun with option '--update'. [-Wother]
> /var/tmp/portage/media-libs/libaacs-0.11.1/work/libaacs-0.11.1/build-aux/ylwrap:
>  206: cannot create ../src/file/keydbcfg-parser.c: Directory nonexistent
> updating src/file/keydbcfg-parser.h
> mv: cannot move 'tmp-keydbcfg-parser.h' to '../src/file/keydbcfg-parser.h': 
> No such file or directory
> make: *** [Makefile:1150: src/file/keydbcfg-parser.c] Error 2
> ```
> 
> I can workaround this by running `mkdir -p ${BUILD_DIR}/src/file` to ensure 
> that the necessary directory exists within the build directory beforehand, but
> it feels like I shouldn't have to.
> 
> Their Makefile.am can be found here: 
> https://code.videolan.org/videolan/libaacs/-/blob/master/Makefile.am. Snippet:
> ```
> libaacs_la_SOURCES=\
>   src/libaacs/aacs.h \
> [...]
>   src/file/dirs.h \
>   src/file/file.h \
>   src/file/file.c \
>   src/file/filesystem.h \
>   src/file/filesystem.c \
>   src/file/keydbcfg.c \
>   src/file/keydbcfg.h \
>   src/file/keydb.h \
>   src/file/keydbcfg-parser.y \
>   src/file/keydbcfg-lexer.l \
>   src/file/mmc_device.h \
> [...]
> ```
> 
> While src/libaacs exists within the build dir, src/file/ doesn't exist at 
> all, hence the failure.
> 
> automake yacc rules should mkdir -p the needed directories within the build 
> dir for VPATH builds before running ylwrap/yacc.

i think there's more to it.  if you're using a release tarball for this project
created by `make dist`, then you shouldn't be running yacc in the first place.

https://www.gnu.org/software/automake/manual/html_node/Yacc-and-Lex.html
> The intermediate files generated by yacc (or lex) will be included in any
> distribution that is made. That way the user doesn’t need to have yacc or lex.

cloning that repo and running `make dist` shows the file is generated, and it's
listed in am__DIST_COMMON, but it still doesn't make it into the tarball.  oh,
it looks like this is self-inflicted:
https://code.videolan.org/videolan/libaacs/-/blob/0.11.1/Makefile.am#L88
https://code.videolan.org/videolan/libaacs/-/commit/f60f46da1dc5e87f70b6edc965a8909d3f21c247

that makes no sense.  file a bug with them.
-mike


signature.asc
Description: PGP signature

bug#33779: Wrong lib-list in install-%DIR%LTLIBRARIES

2023-01-12 Thread Mike Frysinger 
On Mon, 17 Dec 2018 20:11:48 +0100, Bert Wesarg wrote:
> looking at lib/am/ltlib.am, the rule for installing libraries looks
> weird for the !LIBTOOL case. Though I have the impression that this
> file should not be used at all, if libtool is not used. Nevertheless,
> the install command in this cases uses the $list variable, not the
> sanitized $list2 variable. A simple patch looks like:

i agree it looks funky.  do you have an example of where this fails ?
how did you discover the bug ?  we'd want to write a test for it so we
can be sure it doesn't fail again.
-mike

bug#60773: AM_PROG_AR and LT_INIT don't work together when using lib.exe

2023-01-12 Thread Mike Frysinger 
On Wed, 16 Mar 2022 15:24:37 +0100, Tim Ruffing wrote:
> Assume environment variable AR="lib.exe" and configure.ac has
>
> ```
> AM_PROG_AR
> LT_INIT
> ```

i think the expectation is that, if you're using libtool, then you
use libtool.  it can create static convenience libraries too:
https://www.gnu.org/software/libtool/manual/html_node/Static-libraries.html
-mike