Re: On time64 and Large File Support

[Sam James]

> On 12 Nov 2022, at 21:31, Paul Eggert  wrote:
> 
> On 2022-11-12 12:23, Wookey wrote:
>> we can't just have everyone who enabled LFS sometime in the
>> last 20 years suddenly being forced into the time_t change (can we?)
> 
> We've done it in the past.
> 
> AC_SYS_LARGEFILE originally was in Gnulib, before it migrated to Autoconf. 
> Originally it affected only off_t; its effect on other types like ino_t came 
> later. Hence people who initially used AC_SYS_LARGEFILE were later "suddenly 
> forced" into an ino_t width change. Similarly for other non-off_t types that 
> AC_SYS_LARGEFILE affects.
> 
> So there's longstanding precedent for AC_SYS_LARGEFILE changing the width of 
> system types that were formerly unaffected. The difference is that time_t is 
> more widely used than ino_t etc., and people are understandably more 
> concerned about time_t changes.
> 

Thanks, that's a helpful bit of history I wasn't aware of there.

> 
> Because of the concerns raised in this thread it's become clear that what's 
> in Autoconf now is too drastic, and I've proposed (though not yet 
> implemented) a change that will cause AC_SYS_LARGEFILE to continue to not 
> affect time_t unless there's an affirmative request like "./configure 
> --enable-year2038". I am waiting for feedback on that from Zack and others, 
> and am hoping for quick turnaround on this because we do need a new Autoconf 
> release.
> 
> 

Sorry, I missed this until now.

That would work well for me too. It's fine for me at least if the same macro 
just haas additional compatibilities, even if a bit confusing
at first.

As for further changes (as the original post wasn't exclusive to autoconf),
I await comment from other glibc maintainers as I still feel like we're
pretty stuck about how to handle this overall. But your suggestion
(or zw's patch) puts out or dampens the autoconf fire at least.

Thanks for helping come to a compromise. I know this isn't
a simple or easy topic at all.


signature.asc
Description: Message signed with OpenPGP

Re: On time64 and Large File Support

[Russ Allbery]

Wookey  writes:

> Now, I'm not yet sure if just having autoconf 2.72 will actually break
> things. AIUI, these changes only apply where LFS
> (-D_FILE_OFFSET_BITS=64) is turned on, so in Debian at least, where that
> is not the default on 32bit arches, maybe this is OK. But probably quite
> a lot of packages already enable LFS so they are suddenly going to get a
> new ABI if they expose timet anywhere?
> https://codesearch.debian.net/search?q=AC_SYS_LARGEFILE=1 shows
> 163 pages of hits, and a quick peruse suggsts that AC_SYS_LARGEFILE is
> used by a lot of packages (as you might expect - this transition has
> been going on for many years). And just having that macro in
> configure.(in|ac) will turn 64-bit timet on if you autoreconf with
> 2.72. Right?

If indeed pre-existing use of AC_SYS_LARGEFILES would suddenly enable
64-bit time_t on autoreconf, I can name two packages just off the top of
my head that this change to Autoconf will immediately break if their
Debian packages are rebuilt with a newer version of Autoconf, creating
severe bugs.

libremctl will have its ABI changed without any coordination or versioning
(which I will be doing, moving forward, but have not started tackling yet
in part because I was waiting to see what the plan would be and whether
there will be some coordinated change to SONAMEs, a new architecture, or
what).  And INN, which admittedly is a disaster about things like this for
lots of historical reasons, will have its *on-disk file format* changed
without notice in a way that will cause serious failure and possibly data
corruption on upgrades.

This is just wildly backward-incompatible and seems like an awful idea.
If we're going to throw a big switch and rebuild everything, it needs to
be done at a distro-wide level.  I believe the only safe thing for
Autoconf to do is to provide an opt-in facility, similar to what was done
for AC_SYS_LARGEFILE, and then leave deciding whether to opt in to
higher-level machinery.

> However my limited understanding as of right now says that autoconf 2.72
> tying 64bit time_t to use of AC_SYS_LARGEFILE means that 2.72 can't be
> used in debian yet. So I currently favour not tying them together in
> this release.

That's also my understanding from the thread so far, although I'm not sure
that I'm following all of the subtleties.

> People have been using AC_SYS_LARGEFILE without 64bit time_t for many
> years now so it's not yet clear to me why that cannot continue.

And these are conceptually not at all the same thing.  I saw Paul's
explanation for why he views them as fundamentally the same because of
their effect on system calls like stat, but I certainly don't think of
them that way and I am quite dubious many other people will either.  The
set of things that I have to check to ensure that time_t is handled
correctly is totally different than the set of things I thought about when
enabling AC_SYS_LARGEFILE many years in the past.

I recognize that there will be overlap once file timestamps are past 2038
and that will happen sooner than anyone plans for, but it's still true
that this has *not* happened right now and this therefore is not currently
creating many bugs, whereas this switch in this way will create many, very
serious bugs immediately.

-- 
Russ Allbery (ea...@eyrie.org) 

Re: On time64 and Large File Support

[Paul Eggert]

On 2022-11-12 12:23, Wookey wrote:

we can't just have everyone who enabled LFS sometime in the
last 20 years suddenly being forced into the time_t change (can we?)


We've done it in the past.

AC_SYS_LARGEFILE originally was in Gnulib, before it migrated to 
Autoconf. Originally it affected only off_t; its effect on other types 
like ino_t came later. Hence people who initially used AC_SYS_LARGEFILE 
were later "suddenly forced" into an ino_t width change. Similarly for 
other non-off_t types that AC_SYS_LARGEFILE affects.


So there's longstanding precedent for AC_SYS_LARGEFILE changing the 
width of system types that were formerly unaffected. The difference is 
that time_t is more widely used than ino_t etc., and people are 
understandably more concerned about time_t changes.



Because of the concerns raised in this thread it's become clear that 
what's in Autoconf now is too drastic, and I've proposed (though not yet 
implemented) a change that will cause AC_SYS_LARGEFILE to continue to 
not affect time_t unless there's an affirmative request like 
"./configure --enable-year2038". I am waiting for feedback on that from 
Zack and others, and am hoping for quick turnaround on this because we 
do need a new Autoconf release.

Re: On time64 and Large File Support

[Wookey]

On 2022-11-12 11:15 -0800, Paul Eggert wrote:
> On 2022-11-12 10:50, Bruno Haible wrote:
> > I'm saying
> > "tiny" because we are still 15 years away, and new releases of the (not
> > many) affected packages are likely to come in the next 1-2 years.
> 
> Not so "tiny", I'm afraid. My department is still running a server with
> libraries and executables that are over 17 years old.

Nobody disagrees with the idea that 2038 fixes are now fairly urgent
if we want to avoid real-world stuff breaking in 15 years time. That
has been increasingly clear for the last few years.

But the point here is that we need a bit of co-ordination and a plan,
particularly for binary distros. This isn't a minor change that should
just happen to people because there was an autoconf upgrade. Or at
least I don't think it is. My assumption is that if you just turned it
on in random packages as they were uploaded, there would be very
serious (unacceptably bad) breakage - we need to co-ordinate sets of
matching-ABI packages to upgrade together, and the worry is that the
matching set is 'most of the distro'.

Now, I'm not yet sure if just having autoconf 2.72 will actually break
things. AIUI, these changes only apply where LFS
(-D_FILE_OFFSET_BITS=64) is turned on, so in Debian at least, where
that is not the default on 32bit arches, maybe this is OK. But
probably quite a lot of packages already enable LFS so they are
suddenly going to get a new ABI if they expose timet anywhere?
https://codesearch.debian.net/search?q=AC_SYS_LARGEFILE=1 shows
163 pages of hits, and a quick peruse suggsts that AC_SYS_LARGEFILE is
used by a lot of packages (as you might expect - this transition has
been going on for many years). And just having that macro in
configure.(in|ac) will turn 64-bit timet on if you autoreconf with
2.72. Right?

We can't embark on that until we decide whether this transition will
be done within the existing arch/triplet or with a new one. I agree we
should decide that pronto. And I think that 'we' is bigger than just Debian.

If new autoconf (and gnulib) just turns this on within the existing
arch/triplet then we, the distro, can't use those new versions unless
we back out those changes, or until we decide that we are going to
attempt this ABI transition within the existing arch/triplet.

I'm OK with this being done either way (complicated transition within
arch/triplet, or whole-world rebuild with new arch/triplet) once the
size of the problem (and thus the amount of work) is reasonably
understood and there is some consensus amongst distros. TTBOMK we need
some more research before we can make that choice (although I realise
some others are further down this road than I am, and yes I did plan
to get to this point about a year ago, so apologies for
slowness). Hence my suggestion of a full discussion on the
cross-distro list. It is overdue.

However my limited understanding as of right now says that autoconf
2.72 tying 64bit time_t to use of AC_SYS_LARGEFILE means that 2.72
can't be used in debian yet. So I currently favour not tying them
together in this release.

People have been using AC_SYS_LARGEFILE without 64bit time_t for many
years now so it's not yet clear to me why that cannot continue. And
even if it is a good idea to complete both transitions in the same
upheaval we can't just have everyone who enabled LFS sometime in the
last 20 years suddenly being forced into the time_t change (can we?).

Wookey
-- 
Principal hats:  Debian, Wookware, ARM
http://wookware.org/


signature.asc
Description: PGP signature

Re: On time64 and Large File Support

[Paul Eggert]

On 2022-11-12 10:50, Bruno Haible wrote:

I'm saying
"tiny" because we are still 15 years away, and new releases of the (not
many) affected packages are likely to come in the next 1-2 years.


Not so "tiny", I'm afraid. My department is still running a server with 
libraries and executables that are over 17 years old. I have asked for 
it to be decommissioned, but there are backward compatibility concerns. 
(The OS is still supported by its supplier, and we install security 
patches, most recently the day before yesterday.)


Admittedly my situation differs from embedded environments where 
_TIME_BITS=64 is likely to make a difference 15 years from now. 
Unfortunately the situation in embedded environments is often worse.




My assessment is based on the understanding that Zack's proposed change
is essentially [a small change to the code]


Yes, that's my understanding too. Unfortunately the Autoconf change 
would have to be more complicated than that, since documentation and 
comments would have to change accordingly. And the change to Gnulib code 
would be considerably more complicated; this inevitably follows from any 
significant change we make in this area to Autoconf on Savannah. I would 
rather we spent our limited resources moving forward not backward.

Re: On time64 and Large File Support

[Bruno Haible]

Paul Eggert wrote:
> On 2022-11-12 06:16, Zack Weinberg wrote:
> > I am going to go ahead and do this if nobody raises a concrete objection
> > within the next 24 hours.
> 
> I object to a simple reversion, as this will cause problems downstream 
> with Gnulib-using applications, several of which have already been 
> released assuming the current Autoconf git will go into the next 
> release, and which will stop working if built from Git with an Autoconf 
> 2.72 where the AC_SYS_LARGEFILE changes have been reverted.

Paul, my assessment of Zack's proposed change is different. Things will
not "stop working". But rather, for packages
  * that have been released in the last three months, with the then-newest
Gnulib,
  * only when this package is reconfigured with Autoconf 2.72
then
  (1) a portability problem to HP-UX/ia64 in 32-bit mode will appear,
  (2) year 2038 compliance measures will not be enabled.

(1) is not something we need to worry about, since HP-UX/ia64 has so few
users, and why should them run 'autoreconf'.

(2) is a tiny setback on our journey to year 2038 compliance. I'm saying
"tiny" because we are still 15 years away, and new releases of the (not
many) affected packages are likely to come in the next 1-2 years.

My assessment is based on the understanding that Zack's proposed change
is essentially

diff --git a/lib/autoconf/specific.m4 b/lib/autoconf/specific.m4
index 0a9adba5..649d2d17 100644
--- a/lib/autoconf/specific.m4
+++ b/lib/autoconf/specific.m4
@@ -278,7 +278,7 @@ AS_IF([test "$enable_largefile" != no],
  [Define for large files, on AIX-style hosts.],
  [_AC_SYS_LARGEFILE_TEST_INCLUDES])],
 [64],
-  [_AC_SYS_YEAR2038()])])
+  [])])
 
   AH_VERBATIM([__MINGW_USE_VC2005_COMPAT],
 [#if !defined __MINGW_USE_VC2005_COMPAT && defined __MINGW32__

and upon inspection of the largefile.m4 part of this Gnulib commit:
https://git.savannah.gnu.org/gitweb/?p=gnulib.git;a=commitdiff;h=b9bf95fd0a6ab666b484b1e224321664f051f7fa

Bruno

Re: On time64 and Large File Support

[Paul Eggert]

On 2022-11-11 18:20, Zack Weinberg wrote:


I don’t think Paul considered the new behavior of AC_SYS_LARGEFILE to be
overriding the glibc maintainers. Rather, I think he was only thinking
about applications, not libraries, and only about source distribution.


No, I was thinking about libraries as well. Although there are problems 
with libraries and time_t, for decades we've had the same problems with 
AC_SYS_LARGEFILE and off_t, rlim_t, ino_t, etc. Why should time_t be 
treated differently from the other types?




(It’s unfortunate, IMNSHO, that glibc chose not to provide a time_t
equivalent of _LARGEFILE64_SOURCE, as this means it’s much more
difficult for any shared library other than glibc to offer *both* time_t
and time64_t binary interfaces.)


I can easily understand why glibc didn't take that approach. The LFS API 
designed in the mid-1990s to support both off_t and off64_t in 
user-level code was supposed to be a "transitional API", but we're still 
stuck with it nearly 30 years later, even though almost nobody uses it 
(and the few who do often don't use it correctly). Even if we weren't 
dissuaded by the problems of the LFS API, we won't have a luxury of a 
30-year transition for time_t, as we're only 15 years away from 2038.


More generally, the LFS API approach doesn't scale, as the complexity of 
the implementation would grow exponentially with the number of features 
involved. Nobody wants to implement or support that sort of thing. This 
is why glibc put rlimt_t, ino_t, etc. under the off_t umbrella.




I am honestly not sure what to do about this in the long term, but for
the proposed “this weekend, just bugfixes” Autoconf 2.72, I do think it
makes sense to back out change #2, only — that is, AC_SYS_YEAR2038 will
exist, but AC_SYS_LARGEFILE will *not* imply AC_SYS_YEAR2038. 


AC_SYS_LARGEFILE doesn't imply AC_SYS_YEAR2038 now, in Autoconf git. 
Things are a bit more complicated, unfortunately.

Re: On time64 and Large File Support

[Paul Eggert]

On 2022-11-12 06:16, Zack Weinberg wrote:

I am going to go ahead and do this if nobody raises a concrete objection
within the next 24 hours.


I object to a simple reversion, as this will cause problems downstream 
with Gnulib-using applications, several of which have already been 
released assuming the current Autoconf git will go into the next 
release, and which will stop working if built from Git with an Autoconf 
2.72 where the AC_SYS_LARGEFILE changes have been reverted. Current 
Gnulib assumes that AC_SYS_LARGEFILE will behave in Autoconf 2.72 like 
what's in Git now. If we change AC_SYS_LARGFILE back, we will need to 
change Gnulib too, and update downstream apps accordingly.


Instead, I suggest a partial reversion, in which AC_SYS_LARGEFILE goes 
back to its previous meaning by default, but there is a well-documented 
way to get the current in-Git meaning, a way that Gnulib can recognize 
and use. For example, you'd get the new meaning if you configure with 
--enable-year2038, or if configure.ac uses a new macro 
AC_SYS_LARGER_FILE that supersedes AC_SYS_LARGEFILE. This will also 
require some Gnulib changes, but they'll be more reliable and stable 
than whipsaw changes required by reverting then whatever future changes 
Autoconf makes.


This proposal would resolve the backward-compatibility concerns for 
people who want to delay worrying about year-2038 issues, while also 
resolving the compatibility concerns of Gnulib. It would also provide a 
better-documented way for distros that want to switch to 64-bit time_t.


Of course this sort of thing cannot be written and tested in an hour. 
But reverting is not that simple either, and would be a less efficient 
use of everybody's time.

Re: On time64 and Large File Support

[Zack Weinberg]

Sam James  writes:

>> On 12 Nov 2022, at 02:20, Zack Weinberg via Libc-alpha 
>>  wrote:
>> I am honestly not sure what to do about this in the long term, but for
>> the proposed “this weekend, just bugfixes” Autoconf 2.72, I do think it
>> makes sense to back out change #2, only — that is, AC_SYS_YEAR2038 will
>> exist, but AC_SYS_LARGEFILE will *not* imply AC_SYS_YEAR2038.  That will
>> limit the impact of AC_SYS_YEAR2038 to packages that have explicitly
>> added it, and should make it safe for Fedora and Gentoo to drop in 2.72
>> in order to unblock C23 testing — am I correct?  It doesn’t resolve the
>> larger issue, but it gives us more time to think about what the
>> resolution ought to be.
>> 
>> What do you think?
>
> This is really I think the best option while allowing us time & space
> to complete the larger discussion.
[…]

I am going to go ahead and do this if nobody raises a concrete objection
within the next 24 hours.

zw

Re: On time64 and Large File Support

[Sam James]

> On 12 Nov 2022, at 02:20, Zack Weinberg via Libc-alpha 
>  wrote:
> 
> Sam James  writes:
>>> On 11 Nov 2022, at 09:19, Florian Weimer  wrote:
>>> We need to support legacy binaries on i386.  Few libraries are
>>> explicitly dual-ABI.  Whether it's safe to switch libraries above glibc
>>> to LFS or time64 needs to be evaluated on a per-library basis.  For most
>>> distributions, no one is going to do that work, and we have to stick to
>>> whathever we are building today.
>> 
>> While I agree, I don't think it's as well-known that it should be that
>> these are ABI breaking and require inspection. It's being done ad-hoc
>> or in many cases, not at all.
> …
 We're now (possibly) on the eve of an autoconf 2.72 release which
 contains two changes of note [2][3]
 1. addition of a new AC_SYS_YEAR2038 macro;
 2. making AC_SYS_LARGEFILE change behaviour to imply AC_SYS_YEAR2038.
 
 Indeed, the gnulib version of change #2 is exactly how we ended up with
 wget/gnutls breaking [1]. I feel this shows that the only approach
 "supported" by glibc right now is untenable.
>>> 
>>> AC_SYS_LARGEFILE defaulting to AC_SYS_YEAR2038 is extremely destructive
>>> for Fedora unfortunately.
>>> 
>>> I thought the gnulib change has been reverted?
>>> 
>>> I really wish the rest of GNU would talk to glibc maintainers before
>>> overriding glibc maintainer decisions.
> 
> There seems to be a disconnect between Paul Eggert’s position on these
> changes and everyone else on this thread’s position.
> 
> I don’t think Paul considered the new behavior of AC_SYS_LARGEFILE to be
> overriding the glibc maintainers. Rather, I think he was only thinking
> about applications, not libraries, and only about source distribution.
> If an application is being built on the machine where it’ll be used, and
> both it and the system support building it with 64-bit time_t and off_t,
> *why wouldn’t you*?  It has no impact on anything else to do that, and
> it future-proofs your installataion.
> 
> But everyone else is worrying about cases where, either, an application
> is built with 64-bit time_t and/or off_t and then copied to a different
> system where the underlying libraries *don’t* support that, or a *shared
> library* is rebuilt with 64-bit time_t and/or off_t and that silently
> changes its ABI out from under programs that use it.
> 

Precisely.

> (It’s unfortunate, IMNSHO, that glibc chose not to provide a time_t
> equivalent of _LARGEFILE64_SOURCE, as this means it’s much more
> difficult for any shared library other than glibc to offer *both* time_t
> and time64_t binary interfaces.)
> 
> (It’s also unfortunate that, 20+ years after the invention of ELF symbol
> versioning, it’s still ridiculously difficult.  So many macros and
> assembly hacks and fighting with libtool.  But I digress.)
> 
> I am honestly not sure what to do about this in the long term, but for
> the proposed “this weekend, just bugfixes” Autoconf 2.72, I do think it
> makes sense to back out change #2, only — that is, AC_SYS_YEAR2038 will
> exist, but AC_SYS_LARGEFILE will *not* imply AC_SYS_YEAR2038.  That will
> limit the impact of AC_SYS_YEAR2038 to packages that have explicitly
> added it, and should make it safe for Fedora and Gentoo to drop in 2.72
> in order to unblock C23 testing — am I correct?  It doesn’t resolve the
> larger issue, but it gives us more time to think about what the
> resolution ought to be.
> 
> What do you think?

This is really I think the best option while allowing us time & space
to complete the larger discussion.

We keep AC_SYS_YEAR2038 which lets upstreams opt in,
but it avoids the risk of sudden LFS-into-time64 rollover.

Year 2038 work is really important (which is why I brought
up the topic) but I don't want confusion around it to make
people avoid adopting 2.72 or leave us without a 2.72
at all.

I do sympathise with Paul's position, but I'm not
a believer in piecemeal anyway now and it's fair to say there's no
consensus on this yet.

Paul's point Is fair that distros who wish can set the cache var to
ff, so *if* you and Paul want to keep it in.

I could live with a bigger "please check this!" in NEWS. I don't think it's a
total disaster as long as we brief distros first, as we're already
doing it for gnuilb & upstreams may well end up doing themselves,
so the train is on its way out of the station anyway.

(I think we need to keep the discussion going on that front,
but I think it'd be wrong to let it block this release, as it's
not a simple problem and I'm still unsure how I fully feel
about the issues at its core.)

I don't think it's conservative to make a macro suddenly
do a new thing which might require testing on the part
of projects using autoconf. Even if I get the aim.

And if we want to change it later, we still can. If we all
magically reach consensus in a month, there's nothing
stopping that being pursued.

So let's go for keep AC_SYS_YEAR2038 only?

Best,
sam


signature.asc
Description: 

Re: On time64 and Large File Support

[Zack Weinberg]

Sam James  writes:
>> On 11 Nov 2022, at 09:19, Florian Weimer  wrote:
>> We need to support legacy binaries on i386.  Few libraries are
>> explicitly dual-ABI.  Whether it's safe to switch libraries above glibc
>> to LFS or time64 needs to be evaluated on a per-library basis.  For most
>> distributions, no one is going to do that work, and we have to stick to
>> whathever we are building today.
>
> While I agree, I don't think it's as well-known that it should be that
> these are ABI breaking and require inspection. It's being done ad-hoc
> or in many cases, not at all.
…
>>> We're now (possibly) on the eve of an autoconf 2.72 release which
>>> contains two changes of note [2][3]
>>> 1. addition of a new AC_SYS_YEAR2038 macro;
>>> 2. making AC_SYS_LARGEFILE change behaviour to imply AC_SYS_YEAR2038.
>>> 
>>> Indeed, the gnulib version of change #2 is exactly how we ended up with
>>> wget/gnutls breaking [1]. I feel this shows that the only approach
>>> "supported" by glibc right now is untenable.
>> 
>> AC_SYS_LARGEFILE defaulting to AC_SYS_YEAR2038 is extremely destructive
>> for Fedora unfortunately.
>> 
>> I thought the gnulib change has been reverted?
>> 
>> I really wish the rest of GNU would talk to glibc maintainers before
>> overriding glibc maintainer decisions.

There seems to be a disconnect between Paul Eggert’s position on these
changes and everyone else on this thread’s position.

I don’t think Paul considered the new behavior of AC_SYS_LARGEFILE to be
overriding the glibc maintainers. Rather, I think he was only thinking
about applications, not libraries, and only about source distribution.
If an application is being built on the machine where it’ll be used, and
both it and the system support building it with 64-bit time_t and off_t,
*why wouldn’t you*?  It has no impact on anything else to do that, and
it future-proofs your installataion.

But everyone else is worrying about cases where, either, an application
is built with 64-bit time_t and/or off_t and then copied to a different
system where the underlying libraries *don’t* support that, or a *shared
library* is rebuilt with 64-bit time_t and/or off_t and that silently
changes its ABI out from under programs that use it.

(It’s unfortunate, IMNSHO, that glibc chose not to provide a time_t
equivalent of _LARGEFILE64_SOURCE, as this means it’s much more
difficult for any shared library other than glibc to offer *both* time_t
and time64_t binary interfaces.)

(It’s also unfortunate that, 20+ years after the invention of ELF symbol
versioning, it’s still ridiculously difficult.  So many macros and
assembly hacks and fighting with libtool.  But I digress.)

I am honestly not sure what to do about this in the long term, but for
the proposed “this weekend, just bugfixes” Autoconf 2.72, I do think it
makes sense to back out change #2, only — that is, AC_SYS_YEAR2038 will
exist, but AC_SYS_LARGEFILE will *not* imply AC_SYS_YEAR2038.  That will
limit the impact of AC_SYS_YEAR2038 to packages that have explicitly
added it, and should make it safe for Fedora and Gentoo to drop in 2.72
in order to unblock C23 testing — am I correct?  It doesn’t resolve the
larger issue, but it gives us more time to think about what the
resolution ought to be.

What do you think?
zw

Re: On time64 and Large File Support

[Paul Eggert]

On 2022-11-11 03:38, Florian Weimer wrote:

But that said, these binaries are broken anyway in 2038?

No, I expect users to run them in time-shifted VMs or containers.


That's reasonable for systems where accurate timestamps are not 
important and where time_t width mismatches would just get in the way.


However, if this is the expected approach, I suggest having a standard 
and well-documented way to timeshift VMs and containers, and unless you 
want to cede the field entirely to other platforms this documentation 
effort and testing should be done now rather than in the year 2037. 
(Another thing to add to your bulging to-do list)

Re: On time64 and Large File Support

[Florian Weimer]

* Sam James:

>> On 11 Nov 2022, at 09:19, Florian Weimer  wrote:
>> 
>> * Sam James:
>> 
>>> In Gentoo, we've been planning out what we should do for time64 on
>>> glibc [0] and concluded that we need some support in glibc for a newer
>>> option. I'll outline why below.
>>> 
>>> Proposal: glibc gains two new build-time configure options:
>>> * --enable-hard-time64
>>> * --enable-hard-lfs
>> 
>> We should define new target triplets for this if it's really required.
>
> I hadn't considered that option. I'll reflect on it. Please let me know
> if you have further thoughts on this.
>
> But that said, these binaries are broken anyway in 2038?

No, I expect users to run them in time-shifted VMs or containers.
Wrong dates are better than no ability to run anything at all.

And whoever can recompile to switch to time64 can just recompile for a
64-bit target.  There are some embedded users that stick to 32-bit for
cost savings, but I think the cost allocation is quite wrong: They save
a bit on per-unit costs, but they do not really contribute back to GNU
(and most don't even use glibc, although there is some use out there).

>> We need to support legacy binaries on i386.  Few libraries are
>> explicitly dual-ABI.  Whether it's safe to switch libraries above glibc
>> to LFS or time64 needs to be evaluated on a per-library basis.  For most
>> distributions, no one is going to do that work, and we have to stick to
>> whathever we are building today.
>
> While I agree, I don't think it's as well-known that it should be that
> these are ABI breaking and require inspection. It's being done ad-hoc
> or in many cases, not at all.
>
> Part of the use of this thread is, if nothing else, we can show upstreams
> and other distros It if they're confused.
>
> It's very easy to miss that a package has started enabling LFS
> and then your opportunity to catch the ABI breakage is gone.
>
> It doesn't help that I (and I suspect most distribution maintainers)
> do all development on x86_64 and hence even ABI diffing isn't
> going to notice. You have to specifically diff the build system, which I
> do, but it's not easy if it's buried deep within gnulib or something.

I really assumed that setting the default in glibc would solve this for
everyone: binary distributions keep using time32, and source-based
embedded distributions can switch to time64 if they want to. *sigh*

I mean, we have things like more compact stack usage through certain
ABI-breaking GCC options.  The kernel can use those safely, but few
people attempt to apply them to userspace.  There, having the right
default in the toolchain is sufficient.  I didn't expect time64 to be
different.

Thanks,
Florian

Re: On time64 and Large File Support

[Sam James]

> On 11 Nov 2022, at 09:19, Florian Weimer  wrote:
> 
> * Sam James:
> 
>> In Gentoo, we've been planning out what we should do for time64 on
>> glibc [0] and concluded that we need some support in glibc for a newer
>> option. I'll outline why below.
>> 
>> Proposal: glibc gains two new build-time configure options:
>> * --enable-hard-time64
>> * --enable-hard-lfs
> 
> We should define new target triplets for this if it's really required.

I hadn't considered that option. I'll reflect on it. Please let me know
if you have further thoughts on this.

But that said, these binaries are broken anyway in 2038?

> We need to support legacy binaries on i386.  Few libraries are
> explicitly dual-ABI.  Whether it's safe to switch libraries above glibc
> to LFS or time64 needs to be evaluated on a per-library basis.  For most
> distributions, no one is going to do that work, and we have to stick to
> whathever we are building today.

While I agree, I don't think it's as well-known that it should be that
these are ABI breaking and require inspection. It's being done ad-hoc
or in many cases, not at all.

Part of the use of this thread is, if nothing else, we can show upstreams
and other distros It if they're confused.

It's very easy to miss that a package has started enabling LFS
and then your opportunity to catch the ABI breakage is gone.

It doesn't help that I (and I suspect most distribution maintainers)
do all development on x86_64 and hence even ABI diffing isn't
going to notice. You have to specifically diff the build system, which I
do, but it's not easy if it's buried deep within gnulib or something.

> 
>> These would hard-enable the relevant #defines within glibc's headers
>> and ensure that any binaries built with such a glibc have both Large
>> File Support (LFS) and time64 support.
>> 
>> I've come to the conclusion it's infeasible to try handle the
>> migration piecemeal. Various mismatches can and will occur (and while
>> it's more likely with time64, it's possible with LFS too) [1].
>> 
>> We're now (possibly) on the eve of an autoconf 2.72 release which contains 
>> two changes
>> of note [2][3]
>> 1. addition of a new AC_SYS_YEAR2038 macro;
>> 2. making AC_SYS_LARGEFILE change behaviour to imply AC_SYS_YEAR2038.
>> 
>> Indeed, the gnulib version of change #2 is exactly how we ended up with
>> wget/gnutls breaking [1]. I feel this shows that the only approach
>> "supported" by glibc right now is untenable.
> 
> AC_SYS_LARGEFILE defaulting to AC_SYS_YEAR2038 is extremely destructive
> for Fedora unfortunately.
> 
> I thought the gnulib change has been reverted?
> 
> I really wish the rest of GNU would talk to glibc maintainers before
> overriding glibc maintainer decisions.  If we cannot revert this in
> autoconf (and gnulib), this will very much endanger the Fedora i386
> port.  Debian will probably be impacted in the same way.
> 

Right, we need to be unified on this, because it's confusing enough
without unilateralism.



signature.asc
Description: Message signed with OpenPGP