Re: cygport may not create debug info if top directory contains a symlink
On 18/09/2023 18:24, Brian Inglis via Cygwin-apps wrote:
On 2023-09-18 04:41, Christian Franke via Cygwin-apps wrote:
Brian Inglis wrote:
On 2023-09-17 08:01, Jon Turney via Cygwin-apps wrote:
On 16/09/2023 15:17, Christian Franke via Cygwin wrote:
Found during tests of busybox package:
If the path of the top build directory contains a symlink and the
project's build scripts normalize pathnames, no debug info is
created by cygport.
This is because options like
-fdebug-prefix-map=${B}=/usr/src/debug/${PF}
have no effect because ${B} contains a symlink but the compiler is
run with the real source path.
[...]
Sidenote: we should probably also be using file-prefix-map, now
we're on a gcc which supports it.
... also macro-prefix-map, although it looks like changing to
-ffile-prefix-map is equivalent to -f*-prefix-map which future proofs
the options!
So I updated to using -ffile-prefix-map in cygport 0.36.8, since that
seems like the "Right Thing(TM)"
I discovered today that, amazingly, this breaks compiling ruby, since in
one place it does:
#include __FILE__
(yeah, that's pretty jaw dropping...)
Re: cygport may not create debug info if top directory contains a symlink
On 30/10/2023 16:37, Christian Franke via Cygwin-apps wrote:
Jon Turney wrote:
On 20/09/2023 11:58, Christian Franke via Cygwin-apps wrote:
Brian Inglis wrote:
On 2023-09-18 04:41, Christian Franke via Cygwin-apps wrote:
Brian Inglis wrote:
On 2023-09-17 08:01, Jon Turney via Cygwin-apps wrote:
On 16/09/2023 15:17, Christian Franke via Cygwin wrote:
Found during tests of busybox package:
If the path of the top build directory contains a symlink and
the project's build scripts normalize pathnames, no debug info
is created by cygport.
This is because options like
-fdebug-prefix-map=${B}=/usr/src/debug/${PF}
have no effect because ${B} contains a symlink but the compiler
is run with the real source path.
I think that there was some historical bug with gcc where a
relative path for the old path in this mapping wasn't correctly
handled, which is why were using an absolute path here at all.
So changing it to something like [1] (if that works), might be
better.
[1]
https://github.com/jon-turney/cygport/commit/4175d456a9184c5cdebd8bfb4b5ba30583cedd66
Should bin/cygport.in:534: not have $B between the == as in line 531:
declare ${flags}+=" -fdebug-prefix-map=${B}=/usr/src/debug/${PF}"
...
declare ${flags}+=" -fdebug-prefix-map==/usr/src/debug/${PF}"
or be hoist above the condition if identical, unless that is some
undocumented default for cwd?
I guess the == without ${B} is intentional because it makes the debug
source path relative to ${B} as lines 535-536 also do.
[...]>
(It's unclear to me how gcc compares paths to apply this mapping. If
it's a literal string prefix, rather than on some (semi-)canonicalized
path, then we're maybe going to lose here sometimes, depending on the
vagaries of the build-system, unless we list all of relative,
absolute, and canonical absolute paths?)
(But then maybe we can push dealing with or indicating which of those
is correct off onto the individual cygport?)
Adding this if "$(cd ${S} && pwd -P)" != "${S}" should IMO be safe:
-fdebug-prefix-map=$(cd ${B} && pwd -P)=/usr/src/debug/${PF}
-fdebug-prefix-map=$(cd ${S} && pwd -P)=/usr/src/debug/${PF}
(or use realpath)
So it seems there's a new flag '-fcanon-prefix-map' in GCC 13, which
looks like it maybe solves this problem.
Re: cygport may not create debug info if top directory contains a symlink
Jon Turney wrote:
On 20/09/2023 11:58, Christian Franke via Cygwin-apps wrote:
Brian Inglis wrote:
On 2023-09-18 04:41, Christian Franke via Cygwin-apps wrote:
Brian Inglis wrote:
On 2023-09-17 08:01, Jon Turney via Cygwin-apps wrote:
On 16/09/2023 15:17, Christian Franke via Cygwin wrote:
Found during tests of busybox package:
If the path of the top build directory contains a symlink and
the project's build scripts normalize pathnames, no debug info
is created by cygport.
This is because options like
-fdebug-prefix-map=${B}=/usr/src/debug/${PF}
have no effect because ${B} contains a symlink but the compiler
is run with the real source path.
I think that there was some historical bug with gcc where a
relative path for the old path in this mapping wasn't correctly
handled, which is why were using an absolute path here at all.
So changing it to something like [1] (if that works), might be
better.
[1]
https://github.com/jon-turney/cygport/commit/4175d456a9184c5cdebd8bfb4b5ba30583cedd66
Should bin/cygport.in:534: not have $B between the == as in line 531:
declare ${flags}+=" -fdebug-prefix-map=${B}=/usr/src/debug/${PF}"
...
declare ${flags}+=" -fdebug-prefix-map==/usr/src/debug/${PF}"
or be hoist above the condition if identical, unless that is some
undocumented default for cwd?
I guess the == without ${B} is intentional because it makes the debug
source path relative to ${B} as lines 535-536 also do.
Yeah, I think that "==" is shorthand for "=.=", i.e. relative to cwd.
Further tests show that "==" behaves strange and is not useful at all
(no problem as it doesn't actually appear in cygport.in).
The "=.=" only does a simple replacement of first "." in source name.
Testcases with file.c in /var/tmp/test:
$ gcc -g file.c && objdump -dl a.exe | grep -F file.c
/var/tmp/test/file.c:1
$ gcc -g ./file.c && objdump -dl a.exe | grep -F file.c
/var/tmp/test/./file.c:1
$ gcc -g ../test/file.c && objdump -dl a.exe | grep -F file.c
/var/tmp/test/../test/file.c:1
$ gcc -g -fdebug-prefix-map==/foo/bar file.c && ...
/foo/bar/foo/barfile.c:1
$ gcc -g -fdebug-prefix-map=.=/foo/bar file.c && ...
/var/tmp/test/file.c:1
$ gcc -g -fdebug-prefix-map=.=/foo/bar ./file.c && ...
/foo/bar/file.c:1
$ gcc -g -fdebug-prefix-map=/var/tmp/test=/foo/bar file.c && ...
/foo/bar/file.c:1
Source directories with symlinks are preserved in line number info like
with 'pwd -L'.
Does using relative paths in the prefix-map resolve the original issue
seen with busybox?
No, relative paths do not work. Adding this to top of src_compile() works:
CFLAGS+=" -fdebug-prefix-map=$(cd ${S} && pwd -P)=/usr/src/debug/${PF}"
Same for ${B} is not required for this project. This also works (B and S
are not set yet):
DEBUG_PREFIX_MAPS=( "$(pwd -P)/${PF}.${ARCH}/src/${PN}-${PV}" )
(It's unclear to me how gcc compares paths to apply this mapping. If
it's a literal string prefix, rather than on some (semi-)canonicalized
path, then we're maybe going to lose here sometimes, depending on the
vagaries of the build-system, unless we list all of relative,
absolute, and canonical absolute paths?)
(But then maybe we can push dealing with or indicating which of those
is correct off onto the individual cygport?)
Adding this if "$(cd ${S} && pwd -P)" != "${S}" should IMO be safe:
-fdebug-prefix-map=$(cd ${B} && pwd -P)=/usr/src/debug/${PF}
-fdebug-prefix-map=$(cd ${S} && pwd -P)=/usr/src/debug/${PF}
(or use realpath)
Re: cygport may not create debug info if top directory contains a symlink
On 20/09/2023 11:58, Christian Franke via Cygwin-apps wrote:
Brian Inglis wrote:
On 2023-09-18 04:41, Christian Franke via Cygwin-apps wrote:
Brian Inglis wrote:
On 2023-09-17 08:01, Jon Turney via Cygwin-apps wrote:
On 16/09/2023 15:17, Christian Franke via Cygwin wrote:
Found during tests of busybox package:
If the path of the top build directory contains a symlink and the
project's build scripts normalize pathnames, no debug info is
created by cygport.
This is because options like
-fdebug-prefix-map=${B}=/usr/src/debug/${PF}
have no effect because ${B} contains a symlink but the compiler is
run with the real source path.
I think that there was some historical bug with gcc where a
relative path for the old path in this mapping wasn't correctly
handled, which is why were using an absolute path here at all.
So changing it to something like [1] (if that works), might be better.
[1]
https://github.com/jon-turney/cygport/commit/4175d456a9184c5cdebd8bfb4b5ba30583cedd66
Should bin/cygport.in:534: not have $B between the == as in line 531:
declare ${flags}+=" -fdebug-prefix-map=${B}=/usr/src/debug/${PF}"
...
declare ${flags}+=" -fdebug-prefix-map==/usr/src/debug/${PF}"
or be hoist above the condition if identical, unless that is some
undocumented default for cwd?
I guess the == without ${B} is intentional because it makes the debug
source path relative to ${B} as lines 535-536 also do.
Yeah, I think that "==" is shorthand for "=.=", i.e. relative to cwd.
Does using relative paths in the prefix-map resolve the original issue
seen with busybox?
(It's unclear to me how gcc compares paths to apply this mapping. If
it's a literal string prefix, rather than on some (semi-)canonicalized
path, then we're maybe going to lose here sometimes, depending on the
vagaries of the build-system, unless we list all of relative, absolute,
and canonical absolute paths?)
(But then maybe we can push dealing with or indicating which of those is
correct off onto the individual cygport?)
Re: cygport may not create debug info if top directory contains a symlink
Brian Inglis wrote:
On 2023-09-18 04:41, Christian Franke via Cygwin-apps wrote:
Brian Inglis wrote:
On 2023-09-17 08:01, Jon Turney via Cygwin-apps wrote:
On 16/09/2023 15:17, Christian Franke via Cygwin wrote:
Found during tests of busybox package:
If the path of the top build directory contains a symlink and the
project's build scripts normalize pathnames, no debug info is
created by cygport.
This is because options like
-fdebug-prefix-map=${B}=/usr/src/debug/${PF}
have no effect because ${B} contains a symlink but the compiler is
run with the real source path.
I think that there was some historical bug with gcc where a
relative path for the old path in this mapping wasn't correctly
handled, which is why were using an absolute path here at all.
So changing it to something like [1] (if that works), might be better.
[1]
https://github.com/jon-turney/cygport/commit/4175d456a9184c5cdebd8bfb4b5ba30583cedd66
Should bin/cygport.in:534: not have $B between the == as in line 531:
declare ${flags}+=" -fdebug-prefix-map=${B}=/usr/src/debug/${PF}"
...
declare ${flags}+=" -fdebug-prefix-map==/usr/src/debug/${PF}"
or be hoist above the condition if identical, unless that is some
undocumented default for cwd?
I guess the == without ${B} is intentional because it makes the debug
source path relative to ${B} as lines 535-536 also do.
...
An STC script which creates test dirs to demonstrate the issue and
show the alternative outputs would be nice so anyone can see.
$ ln -s /usr/src /tmp/source
$ cd /tmp/source
$ pwd
/tmp/source
$ /bin/pwd
/usr/src
$ pwd -P
/usr/src
$ /bin/pwd -L
/tmp/source
Thanks, looks good - care to submit a patch, including above
suggestions, to cover all bases?
Users may have a good reason to use a symlinked directory, e.g. fake the
original build path during a rebuild. So I'm still not sure how to
handle this.
- Simply warn the user:
declare -r top=$(cd ${_topdir}; pwd);
+if [ "${top}" != "$(cd ${_topdir}; pwd -P)" ]
+then
+ warning "symlinks in ${top} do not work with some build systems."
+fi
unset _topdir;
- or enforce the real path:
-declare -r top=$(cd ${_topdir}; pwd);
+declare -r top=$(cd ${_topdir}; pwd -P);
+if [ "${top}" != "$(cd ${_topdir}; pwd -L)" ]
+then
+ inform "using real path ${top} as top level directory."
+fi
unset _topdir;
Projects using GNU autotools and cyg{conf,make,install} in cygport are
usually not affected by symlinks in ${top}.
--
Regards,
Christian
Re: cygport may not create debug info if top directory contains a symlink
On 2023-09-18 04:41, Christian Franke via Cygwin-apps wrote:
Brian Inglis wrote:
On 2023-09-17 08:01, Jon Turney via Cygwin-apps wrote:
On 16/09/2023 15:17, Christian Franke via Cygwin wrote:
Found during tests of busybox package:
If the path of the top build directory contains a symlink and the project's
build scripts normalize pathnames, no debug info is created by cygport.
This is because options like
-fdebug-prefix-map=${B}=/usr/src/debug/${PF}
have no effect because ${B} contains a symlink but the compiler is run with
the real source path.
I think that there was some historical bug with gcc where a relative path for
the old path in this mapping wasn't correctly handled, which is why were
using an absolute path here at all.
So changing it to something like [1] (if that works), might be better.
[1]
https://github.com/jon-turney/cygport/commit/4175d456a9184c5cdebd8bfb4b5ba30583cedd66
Should bin/cygport.in:534: not have $B between the == as in line 531:
declare ${flags}+=" -fdebug-prefix-map=${B}=/usr/src/debug/${PF}"
...
declare ${flags}+=" -fdebug-prefix-map==/usr/src/debug/${PF}"
or be hoist above the condition if identical, unless that is some undocumented
default for cwd?
Sidenote: we should probably also be using file-prefix-map, now we're on a
gcc which supports it.
... also macro-prefix-map, although it looks like changing to -ffile-prefix-map
is equivalent to -f*-prefix-map which future proofs the options!
The postinstall code then does not find any line number info with source
path /usr/src/debug/${PF}/...
Could be fixed easily in line 414 of /bin/cygport:
-declare -r top=$(cd ${_topdir}; pwd);
+declare -r top=$(cd ${_topdir}; /bin/pwd);
Can you explain why this makes a difference?
In cygport, pwd is a bash builtin defaulting to -L; /bin/pwd defaults to -P.
Both commands support both options and we might expect the same output.
It would be better to use builtin `pwd -P` if that produces the correct result.
It does.
An STC script which creates test dirs to demonstrate the issue and show the
alternative outputs would be nice so anyone can see.
$ ln -s /usr/src /tmp/source
$ cd /tmp/source
$ pwd
/tmp/source
$ /bin/pwd
/usr/src
$ pwd -P
/usr/src
$ /bin/pwd -L
/tmp/source
Thanks, looks good - care to submit a patch, including above suggestions, to
cover all bases?
--
Take care. Thanks, Brian Inglis Calgary, Alberta, Canada
La perfection est atteinte Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut
-- Antoine de Saint-Exupéry
Re: cygport may not create debug info if top directory contains a symlink
Am 18.09.2023 um 12:41 schrieb Christian Franke via Cygwin-apps: $ pwd -P /usr/src $ /bin/pwd -L /tmp/source Generally speaking, if you really need to mess with /usr/src (which is a packaged directory, so any Cygwin application can assume it's existence as a directory) you should do that via a (bind) mount. -- Achim. (on the road :-)
Re: cygport may not create debug info if top directory contains a symlink
Brian Inglis wrote:
On 2023-09-17 08:01, Jon Turney via Cygwin-apps wrote:
On 16/09/2023 15:17, Christian Franke via Cygwin wrote:
Found during tests of busybox package:
If the path of the top build directory contains a symlink and the
project's build scripts normalize pathnames, no debug info is
created by cygport.
This is because options like
-fdebug-prefix-map=${B}=/usr/src/debug/${PF}
have no effect because ${B} contains a symlink but the compiler is
run with the real source path.
I think that there was some historical bug with gcc where a relative
path for the old path in this mapping wasn't correctly handled, which
is why were using an absolute path here at all.
So changing it to something like [1] (if that works), might be better.
[1]
https://github.com/jon-turney/cygport/commit/4175d456a9184c5cdebd8bfb4b5ba30583cedd66
Sidenote: we should probably also be using file-prefix-map, now we're
on a gcc which supports it.
Definitely. in particular useful in conjunction with reproducible builds
and this cygport patch:
https://sourceware.org/pipermail/cygwin-apps/2023-August/043108.html
The related newlib-cygwin patch has been pushed already:
https://cygwin.com/git/?p=newlib-cygwin.git;a=commit;h=f5e37b93
The postinstall code then does not find any line number info with
source path /usr/src/debug/${PF}/...
Could be fixed easily in line 414 of /bin/cygport:
-declare -r top=$(cd ${_topdir}; pwd);
+declare -r top=$(cd ${_topdir}; /bin/pwd);
Can you explain why this makes a difference?
In cygport, pwd is a bash builtin defaulting to -L; /bin/pwd defaults
to -P.
Both commands support both options and we might expect the same output.
It would be better to use builtin `pwd -P` if that produces the
correct result.
It does.
An STC script which creates test dirs to demonstrate the issue and
show the alternative outputs would be nice so anyone can see.
$ ln -s /usr/src /tmp/source
$ cd /tmp/source
$ pwd
/tmp/source
$ /bin/pwd
/usr/src
$ pwd -P
/usr/src
$ /bin/pwd -L
/tmp/source
--
Regards,
Christian
Re: cygport may not create debug info if top directory contains a symlink
On 2023-09-17 08:01, Jon Turney via Cygwin-apps wrote:
On 16/09/2023 15:17, Christian Franke via Cygwin wrote:
Found during tests of busybox package:
If the path of the top build directory contains a symlink and the project's
build scripts normalize pathnames, no debug info is created by cygport.
This is because options like
-fdebug-prefix-map=${B}=/usr/src/debug/${PF}
have no effect because ${B} contains a symlink but the compiler is run with
the real source path.
I think that there was some historical bug with gcc where a relative path for
the old path in this mapping wasn't correctly handled, which is why were using
an absolute path here at all.
So changing it to something like [1] (if that works), might be better.
[1]
https://github.com/jon-turney/cygport/commit/4175d456a9184c5cdebd8bfb4b5ba30583cedd66
Sidenote: we should probably also be using file-prefix-map, now we're on a gcc
which supports it.
The postinstall code then does not find any line number info with source path
/usr/src/debug/${PF}/...
Could be fixed easily in line 414 of /bin/cygport:
-declare -r top=$(cd ${_topdir}; pwd);
+declare -r top=$(cd ${_topdir}; /bin/pwd);
Can you explain why this makes a difference?
In cygport, pwd is a bash builtin defaulting to -L; /bin/pwd defaults to -P.
Both commands support both options and we might expect the same output.
It would be better to use builtin `pwd -P` if that produces the correct result.
An STC script which creates test dirs to demonstrate the issue and show the
alternative outputs would be nice so anyone can see.
No patch provided because I'm not sure whether this has other negative side
effects.
If this is the case, it possibly makes sense to print a warning if "$(pwd)" !=
"$(/bin/pwd)".
This is not unreasonable, and I would take a patch doing this, as there have
been places in cygport where there are bugs handling that in the past (and
probably still are some, since it's not something that gets tested often).
--
Take care. Thanks, Brian Inglis Calgary, Alberta, Canada
La perfection est atteinte Perfection is achieved
non pas lorsqu'il n'y a plus rien à ajouter not when there is no more to add
mais lorsqu'il n'y a plus rien à retirer but when there is no more to cut
-- Antoine de Saint-Exupéry
Re: cygport may not create debug info if top directory contains a symlink
On 16/09/2023 15:17, Christian Franke via Cygwin wrote:
Found during tests of busybox package:
If the path of the top build directory contains a symlink and the
project's build scripts normalize pathnames, no debug info is created by
cygport.
This is because options like
-fdebug-prefix-map=${B}=/usr/src/debug/${PF}
have no effect because ${B} contains a symlink but the compiler is run
with the real source path.
I think that there was some historical bug with gcc where a relative
path for the old path in this mapping wasn't correctly handled, which is
why were using an absolute path here at all.
So changing it to something like [1] (if that works), might be better.
[1]
https://github.com/jon-turney/cygport/commit/4175d456a9184c5cdebd8bfb4b5ba30583cedd66
Sidenote: we should probably also be using file-prefix-map, now we're on
a gcc which supports it.
The postinstall code then does not find any line number info with source
path /usr/src/debug/${PF}/...
Could be fixed easily in line 414 of /bin/cygport:
-declare -r top=$(cd ${_topdir}; pwd);
+declare -r top=$(cd ${_topdir}; /bin/pwd);
Can you explain why this makes a difference?
No patch provided because I'm not sure whether this has other negative
side effects.
If this is the case, it possibly makes sense to print a warning if
"$(pwd)" != "$(/bin/pwd)".
This is not unreasonable, and I would take a patch doing this, as there
have been places in cygport where there are bugs handling that in the
past (and probably still are some, since it's not something that gets
tested often).
