Koji build failure with coreutils-7.5
I tried to build a git update into dist-f12-openssl earlier and had it die in %doc with an error from cp¹: cp: preserving times for `/builddir/build/BUILDROOT/git-1.6.4.1-1.fc12.i386/usr/share/doc/git-1.6.4.1/contrib/hooks': Function not implemented I _think_ this is due to contrib/hooks being a symlink. A nearly identical spec file worked when Tomas build git-1.6.4 in dist-f12-openssl just a few days ago, but that was with coreutils-7.4-6 in the buildroot. It works for me in mock. Perhaps the problem has something to do with the filesystem or mount options on the build system. Has anyone noticed similar problems? ¹ https://koji.fedoraproject.org/koji/getfile?taskID=1627434&name=build.log -- ToddOpenPGP -> KeyID: 0xBEAF0CE3 | URL: www.pobox.com/~tmz/pgp ~~ I could never recommend the use of drugs or alcohol to anyone, but they've always worked for me. -- Hunter S. Thompson pgpFlQPZXtQAu.pgp Description: PGP signature -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: Koji build failure with coreutils-7.5
Todd Zullinger wrote:
> I tried to build a git update into dist-f12-openssl earlier and had it
> die in %doc with an error from cp¹:
>
> cp: preserving times for
> `/builddir/build/BUILDROOT/git-1.6.4.1-1.fc12.i386/usr/share/doc/git-1.6.4.1/contrib/hooks':
> Function not implemented
Hi Todd,
This is because that latest version of coreutils tries to preserve
permissions on symlinks when it thinks that is possible.
It determines whether to try by testing at configure time for the
existence of the utimensat function. If it can compile and link
against that function, then the resulting executable will call it
and report any failure. The trouble is when you configure on a system
with recent libraries and headers, yet *run* with a kernel
that is old enough as to lack the syscall.
Normally in coreutils, I try not to pollute the tools with run-time
work-around code that will be obsolete in a few years, but this
time, it appears to be required, due to the distance between koji's
build and run-time environments.
The solution is probably something like this:
>From 57d640722e04352a468cc595b0b94dbceaec4871 Mon Sep 17 00:00:00 2001
From: Jim Meyering
Date: Mon, 24 Aug 2009 08:21:47 +0200
Subject: [PATCH] cp: ignore obscure failure to preserve symlink time stamps,
when run on a kernel older than what was implied by headers and
libraries tested at configure time.
* src/copy.c (utimens_symlink): Ignore failure when errno == ENOSYS.
* NEWS (Bug fixes): Mention it.
Reported by Todd Zullinger and Kamil Dudka.
---
NEWS |6 ++
src/copy.c |8 +++-
2 files changed, 13 insertions(+), 1 deletions(-)
diff --git a/NEWS b/NEWS
index 2c744b1..c125b31 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,12 @@ GNU coreutils NEWS-*-
outline -*-
* Noteworthy changes in release ?.? (-??-??) [?]
+** Bug fixes
+
+ cp, mv now ignore failure to preserve a symlink time stamp, when it is
+ due to their running on a kernel older than what was implied by headers
+ and libraries tested at configure time.
+
* Noteworthy changes in release 7.5 (2009-08-20) [stable]
diff --git a/src/copy.c b/src/copy.c
index bf9230b..8fc4b68 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -124,7 +124,13 @@ static inline int
utimens_symlink (char const *file, struct timespec const *timespec)
{
#if HAVE_UTIMENSAT
- return utimensat (AT_FDCWD, file, timespec, AT_SYMLINK_NOFOLLOW);
+ int err = utimensat (AT_FDCWD, file, timespec, AT_SYMLINK_NOFOLLOW);
+ /* When configuring on a system with new headers and libraries, and
+ running on one with a kernel that is old enough to lack the syscall,
+ utimensat fails with ENOTSUP. Ignore that. */
+ if (err && errno == ENOSYS)
+err = 0;
+ return err;
#else
/* Don't set errno=ENOTSUP here as we don't want
to output an error message for this case. */
--
1.6.4.378.g88f2f
--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: Koji build failure with coreutils-7.5
On 08/24/2009 02:26 AM, Jim Meyering wrote:
+** Bug fixes
+
+ cp, mv now ignore failure to preserve a symlink time stamp, when it is
+ due to their running on a kernel older than what was implied by headers
+ and libraries tested at configure time.
+
Yeah, this will be easy to trigger for a while, unfortunately...
--- a/src/copy.c
+++ b/src/copy.c
@@ -124,7 +124,13 @@ static inline int
utimens_symlink (char const *file, struct timespec const *timespec)
{
#if HAVE_UTIMENSAT
- return utimensat (AT_FDCWD, file, timespec, AT_SYMLINK_NOFOLLOW);
+ int err = utimensat (AT_FDCWD, file, timespec, AT_SYMLINK_NOFOLLOW);
+ /* When configuring on a system with new headers and libraries, and
+ running on one with a kernel that is old enough to lack the syscall,
+ utimensat fails with ENOTSUP. Ignore that. */
+ if (err&& errno == ENOSYS)
+err = 0;
+ return err;
Seems like the comment (ENOTSUP) needs to be changed to match the code
(ENOSYS)?
Jeff
--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: Koji build failure with coreutils-7.5
Jim Meyering wrote: > Todd Zullinger wrote: >> I tried to build a git update into dist-f12-openssl earlier and had it >> die in %doc with an error from cp¹: >> >> cp: preserving times for >> `/builddir/build/BUILDROOT/git-1.6.4.1-1.fc12.i386/usr/share/doc/git-1.6.4.1/contrib/hooks': >> Function not implemented > > Hi Todd, > > This is because that latest version of coreutils tries to preserve > permissions on symlinks when it thinks that is possible. > It determines whether to try by testing at configure time for the > existence of the utimensat function. If it can compile and link > against that function, then the resulting executable will call it > and report any failure. The trouble is when you configure on a system > with recent libraries and headers, yet *run* with a kernel > that is old enough as to lack the syscall. By the way, to those who maintain koji, It is subtly dangerous to configure a package against headers and libraries that are not well-matched with the kernel. In this case, new headers/libraries suggest a function is available, as detected by a standard autoconf function-existence check. Yet only at run time do we detect (via surprising failure with ENOSYS) that the kernel is too old to support the function that we were led to believe would be available. Here, it wasn't that big a deal, but I can easily imagine this sort of mismatch leading to a more serious problem. It is fine to have a kernel *newer* than would be suggested by headers/libraries. Now you've seen why is risky to use one that is older. -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: Koji build failure with coreutils-7.5
On 08/24/2009 03:11 AM, Jim Meyering wrote: Jim Meyering wrote: Todd Zullinger wrote: I tried to build a git update into dist-f12-openssl earlier and had it die in %doc with an error from cp¹: cp: preserving times for `/builddir/build/BUILDROOT/git-1.6.4.1-1.fc12.i386/usr/share/doc/git-1.6.4.1/contrib/hooks': Function not implemented Hi Todd, This is because that latest version of coreutils tries to preserve permissions on symlinks when it thinks that is possible. It determines whether to try by testing at configure time for the existence of the utimensat function. If it can compile and link against that function, then the resulting executable will call it and report any failure. The trouble is when you configure on a system with recent libraries and headers, yet *run* with a kernel that is old enough as to lack the syscall. By the way, to those who maintain koji, It is subtly dangerous to configure a package against headers and libraries that are not well-matched with the kernel. In this case, new headers/libraries suggest a function is available, as detected by a standard autoconf function-existence check. Yet only at run time do we detect (via surprising failure with ENOSYS) that the kernel is too old to support the function that we were led to believe would be available. Here, it wasn't that big a deal, but I can easily imagine this sort of mismatch leading to a more serious problem. It is fine to have a kernel *newer* than would be suggested by headers/libraries. Now you've seen why is risky to use one that is older. Unfortunately this is quite common for build machines... as it is easy to build any number of buildroots for any number of OS's. But since one cannot chroot into a new kernel, to build with new libraries/headers, the kernel remains inevitably older than that which the machine builds. The only other alternative I can think of is booting a virtual machine for each package build, which I imagine is probably too costly/painful to consider for koji... Outside of koji, speaking as a kernel developer, people DO boot older kernels on newer userlands -- particularly if they are having a problem with their hardware. So it is entirely possible that a run-time check for a newly-added syscall is the only way to make things work. :( That's what people had to do with sendfile(2) for a long time. I imagine most other newly-added Linux syscalls don't find their way into core daemons and utilities, so most people don't notice. Jeff -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: Koji build failure with coreutils-7.5
Jeff Garzik wrote:
> On 08/24/2009 02:26 AM, Jim Meyering wrote:
>> +** Bug fixes
>> +
>> + cp, mv now ignore failure to preserve a symlink time stamp, when it is
>> + due to their running on a kernel older than what was implied by headers
>> + and libraries tested at configure time.
>> +
>
> Yeah, this will be easy to trigger for a while, unfortunately...
>
>
>> --- a/src/copy.c
>> +++ b/src/copy.c
>> @@ -124,7 +124,13 @@ static inline int
>> utimens_symlink (char const *file, struct timespec const *timespec)
>> {
>> #if HAVE_UTIMENSAT
>> - return utimensat (AT_FDCWD, file, timespec, AT_SYMLINK_NOFOLLOW);
>> + int err = utimensat (AT_FDCWD, file, timespec, AT_SYMLINK_NOFOLLOW);
>> + /* When configuring on a system with new headers and libraries, and
>> + running on one with a kernel that is old enough to lack the syscall,
>> + utimensat fails with ENOTSUP. Ignore that. */
>> + if (err&& errno == ENOSYS)
>> +err = 0;
>> + return err;
>
> Seems like the comment (ENOTSUP) needs to be changed to match the code
> (ENOSYS)?
Yes, indeed. Thanks, Jeff.
I've adjusted that patch to remove the comment mentioning
ENOTSUP in the #else block, too (removed the #else block, actually).
There are too many E{not-supported/not-available} codes.
ENOTSUP is what is "returned" by coreutils/gnulib's lgetfilecon
wrapper, yet some versions fail with errno==ENODATA, hence
the errno_unsupported function, which happens to work also with
xattr stuff. But not here, since this function fails with ENOSYS.
And I don't want to relax errno_unsupported to accept ENOSYS,
in case some legit utimensat fails with errno==ENODATA.
>From 3f71bc0a318857d43c419b1fa2df28a7de610c1c Mon Sep 17 00:00:00 2001
From: Jim Meyering
Date: Mon, 24 Aug 2009 08:21:47 +0200
Subject: [PATCH] cp: ignore obscure failure to preserve symlink time stamps,
when run on a kernel older than what was implied by headers and
libraries tested at configure time.
* src/copy.c (utimens_symlink): Ignore failure when errno == ENOSYS.
* NEWS (Bug fixes): Mention it.
Reported by Todd Zullinger and Kamil Dudka.
---
NEWS |6 ++
src/copy.c | 15 ++-
2 files changed, 16 insertions(+), 5 deletions(-)
diff --git a/NEWS b/NEWS
index 2c744b1..c125b31 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,12 @@ GNU coreutils NEWS-*-
outline -*-
* Noteworthy changes in release ?.? (-??-??) [?]
+** Bug fixes
+
+ cp, mv now ignore failure to preserve a symlink time stamp, when it is
+ due to their running on a kernel older than what was implied by headers
+ and libraries tested at configure time.
+
* Noteworthy changes in release 7.5 (2009-08-20) [stable]
diff --git a/src/copy.c b/src/copy.c
index bf9230b..b5cf64c 100644
--- a/src/copy.c
+++ b/src/copy.c
@@ -123,13 +123,18 @@ static char const *top_level_dst_name;
static inline int
utimens_symlink (char const *file, struct timespec const *timespec)
{
+ int err = 0;
+
#if HAVE_UTIMENSAT
- return utimensat (AT_FDCWD, file, timespec, AT_SYMLINK_NOFOLLOW);
-#else
- /* Don't set errno=ENOTSUP here as we don't want
- to output an error message for this case. */
- return 0;
+ err = utimensat (AT_FDCWD, file, timespec, AT_SYMLINK_NOFOLLOW);
+ /* When configuring on a system with new headers and libraries, and
+ running on one with a kernel that is old enough to lack the syscall,
+ utimensat fails with ENOSYS. Ignore that. */
+ if (err && errno == ENOSYS)
+err = 0;
#endif
+
+ return err;
}
/* Perform the O(1) btrfs clone operation, if possible.
--
1.6.4.378.g88f2f
--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: Koji build failure with coreutils-7.5
I agree with Jeff here. Sorry, Jim. I know life is too complicated to keep track of, but that's just how it is. A robustly-written application really needs to distinguish the build-time vs runtime dependencies it has. If you want to make an assumption at run time, then you really have to make sure that they way you were compiled encodes versions requirement that demand a platform where your assumptions are true. The libc stub/not-stub checks at build time do not give you this. When you link against a libc that provides a real definition for the foobar syscall, the foobar symbol has the same symbol version as the older libc that defined an ENOSYS stub for foobar. In fact, libc itself may very well be built so that it works on kernel vintages both with or without foobar, so that you cannot tell at link time whether foobar will return ENOSYS at runtime even when using the very same libc. The bottom line is that a properly portable program has to check for ENOSYS at runtime now if the function in question has ever existed in a libc capable of running on a kernel that does not support that system call. Thanks, Roland -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: Koji build failure with coreutils-7.5
Jeff Garzik wrote: > On 08/24/2009 03:11 AM, Jim Meyering wrote: >> Jim Meyering wrote: >>> Todd Zullinger wrote: I tried to build a git update into dist-f12-openssl earlier and had it die in %doc with an error from cp¹: cp: preserving times for `/builddir/build/BUILDROOT/git-1.6.4.1-1.fc12.i386/usr/share/doc/git-1.6.4.1/contrib/hooks': Function not implemented >>> >>> Hi Todd, >>> >>> This is because that latest version of coreutils tries to preserve >>> permissions on symlinks when it thinks that is possible. >>> It determines whether to try by testing at configure time for the >>> existence of the utimensat function. If it can compile and link >>> against that function, then the resulting executable will call it >>> and report any failure. The trouble is when you configure on a system >>> with recent libraries and headers, yet *run* with a kernel >>> that is old enough as to lack the syscall. >> >> By the way, to those who maintain koji, >> >> It is subtly dangerous to configure a package against headers and >> libraries that are not well-matched with the kernel. >> In this case, new headers/libraries suggest a function is available, >> as detected by a standard autoconf function-existence check. >> Yet only at run time do we detect (via surprising failure with ENOSYS) >> that the kernel is too old to support the function that we were led >> to believe would be available. Here, it wasn't that big a deal, >> but I can easily imagine this sort of mismatch leading to a more >> serious problem. >> >> It is fine to have a kernel *newer* than would be suggested by >> headers/libraries. Now you've seen why is risky to use one that is older. > > Unfortunately this is quite common for build machines... as it is > easy to build any number of buildroots for any number of OS's. > > But since one cannot chroot into a new kernel, to build with new > libraries/headers, the kernel remains inevitably older than that which > the machine builds. > > The only other alternative I can think of is booting a virtual machine > for each package build, which I imagine is probably too costly/painful > to consider for koji... Actually, that sounds like the real solution: build packages for F11 in a VM running F11's base kernel, build packages for F10 in a VM running F10's base kernel, etc. coreutils/Fedora had similar hassles with the various *at functions, too (openat, fstatat, etc.), and I suspect we haven't seen the last of this sort of problem. But maybe it is infrequent enough not to deserve a "real" solution. > Outside of koji, speaking as a kernel developer, people DO boot older > kernels on newer userlands -- particularly if they are having a > problem with their hardware. So it is entirely possible that a > run-time check for a newly-added syscall is the only way to make > things work. :( I'm afraid you're right. > That's what people had to do with sendfile(2) for a > long time. I imagine most other newly-added Linux syscalls don't find > their way into core daemons and utilities, so most people don't > notice. -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: Koji build failure with coreutils-7.5
Roland McGrath wrote: > I agree with Jeff here. Sorry, Jim. I know life is too complicated to Heh, no need to feel sorry, Roland, unless its for the code pollution. It's fixed properly, now. And it's not even that ugly. > keep track of, but that's just how it is. A robustly-written application > really needs to distinguish the build-time vs runtime dependencies it has. ... > The bottom line is that a properly portable program has to check for ENOSYS > at runtime now if the function in question has ever existed in a libc > capable of running on a kernel that does not support that system call. You should know well that we have to strike a balance here. There is a limit. Work-arounds like this are worthwhile only when the older kernels are in frequent-enough use that not applying the work-around would cause significant risk or discomfort. For example, if I were to make coreutils programs run flawlessly on 2.4.* or earlier kernels even when configured/built against modern headers and libraries, I suspect there would be significant performance degradation in a few key tools. If the degradation didn't impact maintainability, and were negligible when running on modern systems, then it might be ok. Unless someone can point out a flaw that causes real trouble, I'll continue making pragmatic compromises. -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: Koji build failure with coreutils-7.5
Jim Meyering píše v Po 24. 08. 2009 v 11:14 +0200: > Roland McGrath wrote: > > I agree with Jeff here. Sorry, Jim. I know life is too complicated to > > Heh, no need to feel sorry, Roland, unless its for the code pollution. > It's fixed properly, now. And it's not even that ugly. > > > keep track of, but that's just how it is. A robustly-written application > > really needs to distinguish the build-time vs runtime dependencies it has. > ... > > The bottom line is that a properly portable program has to check for ENOSYS > > at runtime now if the function in question has ever existed in a libc > > capable of running on a kernel that does not support that system call. > > You should know well that we have to strike a balance here. > There is a limit. Work-arounds like this are worthwhile only > when the older kernels are in frequent-enough use that not applying > the work-around would cause significant risk or discomfort. > > For example, if I were to make coreutils programs run flawlessly on 2.4.* > or earlier kernels even when configured/built against modern headers and > libraries, I suspect there would be significant performance degradation > in a few key tools. If the degradation didn't impact maintainability, > and were negligible when running on modern systems, then it might be ok. AFAIK glibc has its minimum kernel requirement and it should be 2.6.18 (RHEL 5) now. In my opinion the rest of user space should expect this as the minimal version too. > Unless someone can point out a flaw that causes real trouble, > I'll continue making pragmatic compromises. Dan -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: Koji build failure with coreutils-7.5
On Mon, 24 Aug 2009 01:12:49 -0400, Todd wrote: > I tried to build a git update into dist-f12-openssl earlier and had it > die in %doc with an error from cp¹: > > cp: preserving times for > `/builddir/build/BUILDROOT/git-1.6.4.1-1.fc12.i386/usr/share/doc/git-1.6.4.1/contrib/hooks': > Function not implemented > > I _think_ this is due to contrib/hooks being a symlink. A nearly > identical spec file worked when Tomas build git-1.6.4 in > dist-f12-openssl just a few days ago, but that was with > coreutils-7.4-6 in the buildroot. > > It works for me in mock. Perhaps the problem has something to do with > the filesystem or mount options on the build system. > > Has anyone noticed similar problems? > > ¹ https://koji.fedoraproject.org/koji/getfile?taskID=1627434&name=build.log Hits every package that includes a symlink via %doc. -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: Koji build failure with coreutils-7.5
Michael Schwendt wrote: > On Mon, 24 Aug 2009 01:12:49 -0400, Todd wrote: > > > I tried to build a git update into dist-f12-openssl earlier and had it > > die in %doc with an error from cp¹: > > > > cp: preserving times for > > `/builddir/build/BUILDROOT/git-1.6.4.1-1.fc12.i386/usr/share/doc/git-1.6.4.1/contrib/hooks': > > Function not implemented. > > > > Has anyone noticed similar problems? > > > > ¹ https://koji.fedoraproject.org/koji/getfile?taskID=1627434&name=build.log > > Hits every package that includes a symlink via %doc. > Sorry for that, already solved by coreutils-7.5-2.fc12, now built in koji (and it should be already propagated in build repos as it is part of latest root.log in tasks) Greetings, Ondřej Vašík signature.asc Description: Toto je digitálně podepsaná část zprávy -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: Koji build failure with coreutils-7.5
Jim, Jim Meyering wrote: > The solution is probably something like this: Excellent, thanks for the quick reply and patch. (And thanks for getting the speedy build Ondřej.) -- ToddOpenPGP -> KeyID: 0xBEAF0CE3 | URL: www.pobox.com/~tmz/pgp ~~ Until you have the courage to lose sight of the shore, you will not know the terror of being forever lost at sea. -- Demotivators (www.despair.com) pgpXyGCwBjmgs.pgp Description: PGP signature -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
