Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Arnd Bergmann]

On Monday 19 May 2014 18:12:18 Joseph S. Myers wrote:
> On Mon, 19 May 2014, Arnd Bergmann wrote:
> 
> > A related question would be how you plan to support future CPU architectures
> > that never had the 32-bit time_t in the kernel ABI. Would you also want
> > to provide both 32 and 64 bit time_t to user space on those?
> 
> I'd expect those just to have 64-bit time_t in userspace (like x32) - 
> choosing a different type for time_t from the start is a lot simpler than 
> setting up a second set of interfaces with associated symbol versioning 
> for an existing architecture.  This whole discussion started with the 
> question of whether Nios II should be such an architecture

Ok
 
> (Other variants may arise as well, e.g. architectures with existing kernel 
> support that only get glibc support later.)

Good point.

Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Joseph S. Myers]

On Mon, 19 May 2014, Arnd Bergmann wrote:

> A related question would be how you plan to support future CPU architectures
> that never had the 32-bit time_t in the kernel ABI. Would you also want
> to provide both 32 and 64 bit time_t to user space on those?

I'd expect those just to have 64-bit time_t in userspace (like x32) - 
choosing a different type for time_t from the start is a lot simpler than 
setting up a second set of interfaces with associated symbol versioning 
for an existing architecture.  This whole discussion started with the 
question of whether Nios II should be such an architecture

(Other variants may arise as well, e.g. architectures with existing kernel 
support that only get glibc support later.)

-- 
Joseph S. Myers
jos...@codesourcery.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Arnd Bergmann]

On Monday 19 May 2014 14:46:59 Joseph S. Myers wrote:
> On Mon, 19 May 2014, Arnd Bergmann wrote:
>
> Now, when the motivation is not space reduction but ensuring nothing is 
> there that will break in 2038 (unless it's doing dubious things like 
> storing a time_t value in an int or long in the application - something a 
> reasonably stupid static analysis should be able to detect), you could 
> always identify the (symbol, version) pairs for a given architecture that 
> use 32-bit time_t and develop a tool that checks for binaries using 
> problem symbols.  (Of course, that won't help with ioctls.)  Or put a 
> small patch in the headers to force _TIME_BITS=64 and not allow an 
> override with _TIME_BITS=32.  And if the kernel had a config option to 
> disable all the old interfaces, that would make problem binaries break now 
> rather than in 2038.

I had already thought of the kernel option, that's probably one thing
we will do here, but it won't help if glibc then emulates the interfaces
we remove ;-)

A related question would be how you plan to support future CPU architectures
that never had the 32-bit time_t in the kernel ABI. Would you also want
to provide both 32 and 64 bit time_t to user space on those?

Then again I'm not sure how relevant glibc is to the deeply embedded
distros that would want to run without the backwards compatibility.
I suspect that uClibc, musl and bionic would just offer a compile time
switch without any emulation.

> > > > I don't know why timespec on x32 uses 'long tv_nsec', it does seem
> > > > problematic.
> > > 
> > > Yes, we have a glibc bug 
> > >  about the glibc 
> > > definition (64-bit tv_nsec in x32 userspace to match the kernel's 64-bit 
> > > "long") being nonconforming, but without the kernel treating upper bits 
> > > as 
> > > padding, fixing glibc requires wrappers that copy the value and clear the 
> > > upper bits.
> > 
> > Ok. There is also work going on to have an x32-like ABI for ARM64, and
> > that will likely have to support big-endian as well, so then it's not
> > just about clearing the padding bits, but also having them in the right
> > place.
> 
> Yes, I see no problem with putting the padding bits in the right place so 
> that when the kernel is passing struct timespec values *to* userspace, it 
> can just fill in the native 64-bit structure and have it automatically be 
> correct when interpreted as a 32-bit structure with padding (which the 
> kernel will have zeroed implicitly) on tv_nsec.

Ok.

Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Joseph S. Myers]

On Mon, 19 May 2014, Arnd Bergmann wrote:

> I'm mostly worried about the embedded use case, which at least for time_t
> would have to build everything the new way as soon as possible. For
> desktop and server systems, doing a slow migration to catch the bugs
> and never introduce regressions in the field makes a lot of sense,
> but when someone today wants to build an embedded system, they already
> build everything from source, and it's more important to find all
> the bugs now (ideally at compile time) than to have a chance of hitting
> a subtle issue 24 years ago on today's build.
> 
> For that, it would be nice to even build a glibc that has no backwards
> compatibility interfaces for 32-bit time_t at all.

Building glibc without compatibility interfaces is --enable-oldest-abi (to 
disable ABIs obsoleted before a given point, so that if the 32-bit time_t 
interfaces are obsoleted and no longer available for new builds then it 
would disable them) - but that's been bitrotten for years 
 and there is 
consensus to remove it if anyone sends a patch.  (Similarly, 
--disable-versioning support was removed.  The sort of really tiny 
embedded systems for which that sort of thing is relevant as space 
reduction no longer seem so relevant to glibc; there has been no interest 
in merging option group support from EGLIBC to glibc now EGLIBC 
development has been discontinued.)

Now, when the motivation is not space reduction but ensuring nothing is 
there that will break in 2038 (unless it's doing dubious things like 
storing a time_t value in an int or long in the application - something a 
reasonably stupid static analysis should be able to detect), you could 
always identify the (symbol, version) pairs for a given architecture that 
use 32-bit time_t and develop a tool that checks for binaries using 
problem symbols.  (Of course, that won't help with ioctls.)  Or put a 
small patch in the headers to force _TIME_BITS=64 and not allow an 
override with _TIME_BITS=32.  And if the kernel had a config option to 
disable all the old interfaces, that would make problem binaries break now 
rather than in 2038.

> > > I don't know why timespec on x32 uses 'long tv_nsec', it does seem
> > > problematic.
> > 
> > Yes, we have a glibc bug 
> >  about the glibc 
> > definition (64-bit tv_nsec in x32 userspace to match the kernel's 64-bit 
> > "long") being nonconforming, but without the kernel treating upper bits as 
> > padding, fixing glibc requires wrappers that copy the value and clear the 
> > upper bits.
> 
> Ok. There is also work going on to have an x32-like ABI for ARM64, and
> that will likely have to support big-endian as well, so then it's not
> just about clearing the padding bits, but also having them in the right
> place.

Yes, I see no problem with putting the padding bits in the right place so 
that when the kernel is passing struct timespec values *to* userspace, it 
can just fill in the native 64-bit structure and have it automatically be 
correct when interpreted as a 32-bit structure with padding (which the 
kernel will have zeroed implicitly) on tv_nsec.

-- 
Joseph S. Myers
jos...@codesourcery.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Arnd Bergmann]

On Thursday 15 May 2014 21:05:03 Joseph S. Myers wrote:
> On Thu, 15 May 2014, Arnd Bergmann wrote:
> 
> > > The distribution side of the change for _FILE_OFFSET_BITS (i.e., moving 
> > > to 
> > > building libraries that way so a glibc change to the default wouldn't 
> > > cause issues for other libraries' ABIs) has gradually been done.  The 
> > > discussion in March on libc-alpha about changing the default tailed off.  
> > > This is something that needs someone to take the lead with a *careful and 
> > > detailed analysis of the information from the previous discussion* in 
> > > order to present a properly reasoned proposal for a change to the default 
> > > - not scattergun patches, not patches with brief or no analysis of the 
> > > environment in which glibc is used, not dismissing concerns, but a 
> > > properly reasoned argument for why the change should be made, along with 
> > > details of how distributions can determine whether ABI issues would arise 
> > > from rebuilding a particular library against newer glibc.
> > 
> > Ok, I see. I wasn't aware that distributions actually set _FILE_OFFSET_BITS
> > globally for building packages. I guess the effect (from the distro point
> > of view) of that is similar to having a configure option when building glibc
> > as I expected to be the normal way to do it.
> 
> I'm not sure they set it globally (modulo the suggestion in that recent 
> thread that Gentoo might try changing the default and seeing what breaks) 
> - but it at least seems to be set for most libraries for which it affects 
> the ABI (whether upstream or by the distributions), based on the sample 
> Paul Eggert looked at 
>  (which is an 
> important piece of evidence that a change to the default in glibc wouldn't 
> be dumping a large problem on distributors regarding the ABIs of many 
> widely-used libraries).

I'm mostly worried about the embedded use case, which at least for time_t
would have to build everything the new way as soon as possible. For
desktop and server systems, doing a slow migration to catch the bugs
and never introduce regressions in the field makes a lot of sense,
but when someone today wants to build an embedded system, they already
build everything from source, and it's more important to find all
the bugs now (ideally at compile time) than to have a chance of hitting
a subtle issue 24 years ago on today's build.

For that, it would be nice to even build a glibc that has no backwards
compatibility interfaces for 32-bit time_t at all.

> > I don't know why timespec on x32 uses 'long tv_nsec', it does seem
> > problematic.
> 
> Yes, we have a glibc bug 
>  about the glibc 
> definition (64-bit tv_nsec in x32 userspace to match the kernel's 64-bit 
> "long") being nonconforming, but without the kernel treating upper bits as 
> padding, fixing glibc requires wrappers that copy the value and clear the 
> upper bits.

Ok. There is also work going on to have an x32-like ABI for ARM64, and
that will likely have to support big-endian as well, so then it's not
just about clearing the padding bits, but also having them in the right
place.

I don't know what happens with powerpc64le, but I assume they will also
at some point want to support 32-bit user space if they don't already.

> > For timeval, I think we don't care about the padding, because we wouldn't
> > use it on new interfaces when the kernel uses nanosecond resolution
> > internally.
> 
> Indeed - I haven't checked, but hopefully none of the asm-generic 
> interfaces use timeval (and I was suggesting that only the interfaces 
> present in asm-generic should be duplicated for 64-bit time_t).
> 
> (You also mentioned ioctls.  All affected ioctls would need new variants 
> unless you have a new variant of the ioctl syscall itself.  Logically I 
> suppose _TIME_BITS=64 should then map the generic version of an ioctl to 
> the 64-bit time_t version so applications don't need to change beyond 
> defining _TIME_BITS=64, though I don't know if there's any precedent for 
> doing that sort of thing for definitions in kernel headers rather than 
> glibc's headers.)

For ioctls, it's not that uncommon to change the definition in the kernel
headers to fix bugs, implying we break compatibility between a user space
tool built against new kernel headers but running on older kernels. That
is already the case for any user space relying on newly introduced ioctls,
so we can e.g. introduce a new ioctl command using a
struct __kernel_timespec64 in one of its structs.

Unfortunately, not every driver installs the header files into the user
visible location, some of them have a copy in their (only) user space
tool. This is still fine if we can catch it and if the driver uses
the correct macro to create the ioctl number, which changes the command
code because of the new argument size. The same would be true if we get
a glibc wi

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Joseph S. Myers]

On Sun, 18 May 2014, H. Peter Anvin wrote:

> This potentially requires the kernel to maintain three separate ABIs if
> there is a legacy 32-bit ABI, which is quite frankly idiotic.  You seem

Three ABIs seems reasonable enough to me where appropriate.

> to be thinking of POSIX as something handed down by gods, which it
> isn't... quite frankly they did something stupid here and should fix
> their own mess.

In the real world it's useful to support lots of different standard 
versions, not just some hypothetical future version (I see no evidence of 
anyone thinking there is a POSIX defect actually having raised the issue 
with the Austin Group to see if anyone there agrees a change would be 
useful).  We've only just removed support for _BSD_SOURCE / _SVID_SOURCE 
as defined API levels from glibc; I'd guess it will be about 20 years 
before removing support for POSIX.1-2008/2013 is appropriate (even if the 
next major edition relaxes this requirement).

> As far as glibc is concerned, this would require glibc to intercept each
> ioctl, which is completely infeasible... struct timespec/timeval is
> embedded far too deep everywhere.

Well, maybe the subset of applications passing timespec to ioctls should 
carry the cost of converting from the userspace timespec to the kernel 
timespec, rather than POSIX applications carrying the cost of unnecessary 
incompatibility.  (But an enumeration of the relevant ioctls is needed 
anyway simply to provide new versions of them all for 64-bit time_t.)

-- 
Joseph S. Myers
jos...@codesourcery.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[H. Peter Anvin]

On 05/18/2014 06:46 AM, Joseph S. Myers wrote:
> 
> It's advising against breaking compatibility with existing applications 
> (in that case, C90 applications that assume sizes can be cast to unsigned 
> long for printing, etc.) unless necessary.
> 

Yes, the values in question will not be a problem for that in this case,
so it doesn't apply.

> Rich Felker suggests 
>  there are similar 
> security risks from various types in the x32 ABI being wider than size_t.  
> That seems entirely plausible, if unconfirmed; I don't think there's been 
> much effort to look for such issues where applications expect types to be 
> no wider than size_t when there's no use to them being wider than size_t.  
> Most of those choices are valid but may be risky to existing software much 
> as IL32LLP64 can be risky to existing software.

Everything can be risky if done poorly, unfortunately.

> The userspace ABI for struct timespec with 64-bit time_t and 32-bit long 
> should use long tv_nsec in any future ports, with appropriate conversion 
> code in glibc if the kernel interface is different (for ports that started 
> out with 32-bit time_t, conversion code will be needed anyway to convert 
> to the 32-bit structure if the new syscalls are unavailable at runtime and 
> so the old syscalls need to be used, but if the kernel's structure with 
> 64-bit time_t is different from that in userspace then both old and new 
> kernels need conversion code).  Given Rich Felker's bug reports, it seems 
> reasonable to suppose that musl will wish to do this as well as glibc.

This potentially requires the kernel to maintain three separate ABIs if
there is a legacy 32-bit ABI, which is quite frankly idiotic.  You seem
to be thinking of POSIX as something handed down by gods, which it
isn't... quite frankly they did something stupid here and should fix
their own mess.

As far as glibc is concerned, this would require glibc to intercept each
ioctl, which is completely infeasible... struct timespec/timeval is
embedded far too deep everywhere.

-hpa

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Joseph S. Myers]

On Sat, 17 May 2014, H. Peter Anvin wrote:

> > Adding snseconds_t to POSIX seems pointless when there is no need for this 
> > field to store values that can't fit in "long".  Even if it were added, 
> > good practice would say that implementations should keep using "long" 
> > whenever possible for compatibility with existing applications (just like 
> > the ISO C recommendation "The types used for size_t and ptrdiff_t should 
> > not have an integer conversion rank greater than that of signed long int 
> > unless the implementation supports objects large enough to make this 
> > necessary.").
> 
> That is a very different thing, though.

It's advising against breaking compatibility with existing applications 
(in that case, C90 applications that assume sizes can be cast to unsigned 
long for printing, etc.) unless necessary.

Rich Felker suggests 
 there are similar 
security risks from various types in the x32 ABI being wider than size_t.  
That seems entirely plausible, if unconfirmed; I don't think there's been 
much effort to look for such issues where applications expect types to be 
no wider than size_t when there's no use to them being wider than size_t.  
Most of those choices are valid but may be risky to existing software much 
as IL32LLP64 can be risky to existing software.

> > If you were designing from scratch, no doubt a typedef such as snseconds_t 
> > would be there, but with real-world APIs that have accumulated over time, 
> > deviating from "long" now is a bad idea.
> 
> Given that you already have a long long member of the same structure, it
> seems unlikely that adding another long long to this is a problem.
> 
> Anyway, this was discussed back in 2011:
> 
> https://lkml.org/lkml/2011/8/31/244

Just because one buggy port (I consider this quite clearly a bug at the 
glibc level, whatever the kernel does, agreeing with Rich Felker in 
) managed to get 
into glibc does not mean any more should be added.  glibc's function is to 
provide POSIX (and other) interfaces on top of the underlying kernel, 
which includes fixing any mismatch between the kernel and POSIX interfaces 
(such as converting from userspace POSIX struct timespec to the form in 
which times are passed to the kernel, by copying and zeroing padding, if 
necessary).  That's just like implementing POSIX threads on top of clone - 
or, if future POSIX disallows newlines in filenames (something that's been 
discussed for some time but doesn't seem to have seen much action lately) 
but the Linux kernel doesn't follow, making such checks on filenames 
before passing them to the kernel.  Of course, one would hope that the 
underlying kernel interfaces are designed to make it straightforward to 
implement POSIX on top of them (as well as other things not envisaged by 
POSIX).

The userspace ABI for struct timespec with 64-bit time_t and 32-bit long 
should use long tv_nsec in any future ports, with appropriate conversion 
code in glibc if the kernel interface is different (for ports that started 
out with 32-bit time_t, conversion code will be needed anyway to convert 
to the 32-bit structure if the new syscalls are unavailable at runtime and 
so the old syscalls need to be used, but if the kernel's structure with 
64-bit time_t is different from that in userspace then both old and new 
kernels need conversion code).  Given Rich Felker's bug reports, it seems 
reasonable to suppose that musl will wish to do this as well as glibc.

-- 
Joseph S. Myers
jos...@codesourcery.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[H. Peter Anvin]

On 05/16/2014 04:47 PM, Joseph S. Myers wrote:
>>
>> struct timespec is specified in POSIX as having type "long" for tv_nsec.
>>  This, as Linus pointed out, is totally braindamaged.
> 
> long is a perfectly reasonable type for this purpose - it's guaranteed to 
> have enough precision.  ISO C11 follows POSIX in the definition of struct 
> timespec.
> 

That is not the issue.  The issue is that by fixing the type, the POSIX
committee foolishly fixed the ABI unnecessarily.  There is nothing wrong
with storing this value in a long, of course, but:

a) there is no reason it needs to be as wide as long, on an LP64 ABI an
   int would serve just fine;
b) ABI compatibilility issue, especially since on virtually all ABIs a
   96-bit struct timespec would end up padded to 128 bits anyway;
c) gratuitously different from struct timeval.

>> x32 does not follow POSIX (Linus pretty much dictated that), and instead
>> does the __kernel_suseconds_t to match the native kernel type.  The
>> proposal at some point was to try to push a snseconds_t into POSIX.
> 
> If the x32 kernel won't be fixed to treat the high part as padding then 
> glibc will need fixing to copy all struct timespec values passed to the 
> kernel and clear the padding when copying (really, fixing that bug would 
> need such copying anyway unless --enable-kernel is used to allow a fixed 
> kernel to be assumed).  In any case, we should not put any more ports into 
> glibc with that x32 bug; "long" and an explicit padding field is the way 
> to go (and copying / zeroing if the kernel doesn't treat that padding 
> field as padding).
> 
> Adding snseconds_t to POSIX seems pointless when there is no need for this 
> field to store values that can't fit in "long".  Even if it were added, 
> good practice would say that implementations should keep using "long" 
> whenever possible for compatibility with existing applications (just like 
> the ISO C recommendation "The types used for size_t and ptrdiff_t should 
> not have an integer conversion rank greater than that of signed long int 
> unless the implementation supports objects large enough to make this 
> necessary.").

That is a very different thing, though.

> If you were designing from scratch, no doubt a typedef such as snseconds_t 
> would be there, but with real-world APIs that have accumulated over time, 
> deviating from "long" now is a bad idea.

Given that you already have a long long member of the same structure, it
seems unlikely that adding another long long to this is a problem.

Anyway, this was discussed back in 2011:

https://lkml.org/lkml/2011/8/31/244

-hpa


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Joseph S. Myers]

On Sat, 17 May 2014, H. Peter Anvin wrote:

> On 05/15/2014 01:38 PM, Arnd Bergmann wrote:
> > 
> > For practical purposes in the kernel, we may still want to use 64-bit
> > nanoseconds: if we use a 96 bit struct timespec, that would be incompatible
> > with the native type on 64-bit kernels, thus complicating the syscall
> > emulation layer.
> > 
> > I don't know why timespec on x32 uses 'long tv_nsec', it does seem
> > problematic.
> 
> struct timespec is specified in POSIX as having type "long" for tv_nsec.
>  This, as Linus pointed out, is totally braindamaged.

long is a perfectly reasonable type for this purpose - it's guaranteed to 
have enough precision.  ISO C11 follows POSIX in the definition of struct 
timespec.

> x32 does not follow POSIX (Linus pretty much dictated that), and instead
> does the __kernel_suseconds_t to match the native kernel type.  The
> proposal at some point was to try to push a snseconds_t into POSIX.

If the x32 kernel won't be fixed to treat the high part as padding then 
glibc will need fixing to copy all struct timespec values passed to the 
kernel and clear the padding when copying (really, fixing that bug would 
need such copying anyway unless --enable-kernel is used to allow a fixed 
kernel to be assumed).  In any case, we should not put any more ports into 
glibc with that x32 bug; "long" and an explicit padding field is the way 
to go (and copying / zeroing if the kernel doesn't treat that padding 
field as padding).

Adding snseconds_t to POSIX seems pointless when there is no need for this 
field to store values that can't fit in "long".  Even if it were added, 
good practice would say that implementations should keep using "long" 
whenever possible for compatibility with existing applications (just like 
the ISO C recommendation "The types used for size_t and ptrdiff_t should 
not have an integer conversion rank greater than that of signed long int 
unless the implementation supports objects large enough to make this 
necessary.").

If you were designing from scratch, no doubt a typedef such as snseconds_t 
would be there, but with real-world APIs that have accumulated over time, 
deviating from "long" now is a bad idea.

-- 
Joseph S. Myers
jos...@codesourcery.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[H. Peter Anvin]

On 05/15/2014 01:38 PM, Arnd Bergmann wrote:
> 
> For practical purposes in the kernel, we may still want to use 64-bit
> nanoseconds: if we use a 96 bit struct timespec, that would be incompatible
> with the native type on 64-bit kernels, thus complicating the syscall
> emulation layer.
> 
> I don't know why timespec on x32 uses 'long tv_nsec', it does seem
> problematic.

struct timespec is specified in POSIX as having type "long" for tv_nsec.
 This, as Linus pointed out, is totally braindamaged.

x32 does not follow POSIX (Linus pretty much dictated that), and instead
does the __kernel_suseconds_t to match the native kernel type.  The
proposal at some point was to try to push a snseconds_t into POSIX.

-hpa

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[H. Peter Anvin]

On 05/16/2014 12:48 AM, Christoph Hellwig wrote:
> 
> The stat(64) syscall already has stopped time_t for a long time:
> 
> struct stat64 {
>   ...
>   int st_atime;   /* Time of last access.  */
>   unsigned intst_atime_nsec;
>   int st_mtime;   /* Time of last modification.  */
>   unsigned intst_mtime_nsec;
>   int st_ctime;   /* Time of last status change. */
>   unsigned intst_ctime_nsec;
> };
> 

... except that that is really a struct timespec just written weirdly,
and userspace uses it as a struct timespec.

-hpa


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Christoph Hellwig]

On Thu, May 15, 2014 at 10:07:17PM +0100, One Thousand Gnomes wrote:
> True sort of for the kernel (except you've got problems with file system
> compatibility and I believe stuff like NFS and that fact some file
> systems just break).

modern filesystems including NFS don't use anything that corresponds to
a time_t on disk or on the wire, so we'd just need to adjust the few
places the conert time representations.

> > The main advantage that I see with 64-bit time_t is that a lot of user
> > space already works with it, because NetBSD and OpenBSD use this, and
> > all 64-bit Linux systems already work with this without changing the ABI.
> > If we want a POSIX extension to cover a new ABI, this would be the most
> > likely candidate.
> 
> The main advantage IMHO of just using 64bit on 32bit boxes is getting it
> right. I'd go so far as to consider changing userspace time_t to be a
> struct. Yes it'll break lots of code but at least you'll know about it.

I think we're attacking that from the wrong angle.  We'll need to work
with the glibc and other libc folks to what can be implemented as the
library API/ABI and then ensure the kernel provides proper support for
it.

Chaning the size of time_t in the library seems like the only feasible
approach that keeps existing programs compatible.  If we support this
with a time64_, a shifted epoch or structures at the kernel ABI is a
question of balaning what libc needs and what we can use efficiently
internally.

> > As mentioned earlier, between kernel and user space it's probably best
> > to avoid time_t and timeval completely and just use timespec64 or some
> > other safe type, but there has to be a way to port user space that relies
> > on time_t or timespec.
> 
> This I think is bang on. We've already got lots of cases where we can't
> shoehorn useful info into syscalls due to time_t, including stuff like
> exposing accurate stamps in stat(). On an 8MHz 68000 not much happened in
> the same second, on a 32 core x86 it's a bit different.

The stat(64) syscall already has stopped time_t for a long time:

struct stat64 {
...
int st_atime;   /* Time of last access.  */
unsigned intst_atime_nsec;
int st_mtime;   /* Time of last modification.  */
unsigned intst_mtime_nsec;
int st_ctime;   /* Time of last status change. */
unsigned intst_ctime_nsec;
};

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[One Thousand Gnomes]

> > could do intelligent interpolation about what the epoch is, so in 2039
> > if we get a low 32 bit time_t value, we assume epoch == 1, conversely if
> > we see a high one, we assume epoch == 0.  We could add epoch on the end
> > of the syscalls and detect if it's not present and fill in an
> > interpolated value.  Interfaces which truly want 64 bit time_t would get
> > it from epoch + 32 bit time_t.

True sort of for the kernel (except you've got problems with file system
compatibility and I believe stuff like NFS and that fact some file
systems just break).

> The main advantage that I see with 64-bit time_t is that a lot of user
> space already works with it, because NetBSD and OpenBSD use this, and
> all 64-bit Linux systems already work with this without changing the ABI.
> If we want a POSIX extension to cover a new ABI, this would be the most
> likely candidate.

The main advantage IMHO of just using 64bit on 32bit boxes is getting it
right. I'd go so far as to consider changing userspace time_t to be a
struct. Yes it'll break lots of code but at least you'll know about it.

(and for some other systems like 2BSD it's probably the least of evils
because there isn't a 64bit type in the compiler!)

> As mentioned earlier, between kernel and user space it's probably best
> to avoid time_t and timeval completely and just use timespec64 or some
> other safe type, but there has to be a way to port user space that relies
> on time_t or timespec.

This I think is bang on. We've already got lots of cases where we can't
shoehorn useful info into syscalls due to time_t, including stuff like
exposing accurate stamps in stat(). On an 8MHz 68000 not much happened in
the same second, on a 32 core x86 it's a bit different.

Alan
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Joseph S. Myers]

On Thu, 15 May 2014, Arnd Bergmann wrote:

> > The distribution side of the change for _FILE_OFFSET_BITS (i.e., moving to 
> > building libraries that way so a glibc change to the default wouldn't 
> > cause issues for other libraries' ABIs) has gradually been done.  The 
> > discussion in March on libc-alpha about changing the default tailed off.  
> > This is something that needs someone to take the lead with a *careful and 
> > detailed analysis of the information from the previous discussion* in 
> > order to present a properly reasoned proposal for a change to the default 
> > - not scattergun patches, not patches with brief or no analysis of the 
> > environment in which glibc is used, not dismissing concerns, but a 
> > properly reasoned argument for why the change should be made, along with 
> > details of how distributions can determine whether ABI issues would arise 
> > from rebuilding a particular library against newer glibc.
> 
> Ok, I see. I wasn't aware that distributions actually set _FILE_OFFSET_BITS
> globally for building packages. I guess the effect (from the distro point
> of view) of that is similar to having a configure option when building glibc
> as I expected to be the normal way to do it.

I'm not sure they set it globally (modulo the suggestion in that recent 
thread that Gentoo might try changing the default and seeing what breaks) 
- but it at least seems to be set for most libraries for which it affects 
the ABI (whether upstream or by the distributions), based on the sample 
Paul Eggert looked at 
 (which is an 
important piece of evidence that a change to the default in glibc wouldn't 
be dumping a large problem on distributors regarding the ABIs of many 
widely-used libraries).

> I don't know why timespec on x32 uses 'long tv_nsec', it does seem
> problematic.

Yes, we have a glibc bug 
 about the glibc 
definition (64-bit tv_nsec in x32 userspace to match the kernel's 64-bit 
"long") being nonconforming, but without the kernel treating upper bits as 
padding, fixing glibc requires wrappers that copy the value and clear the 
upper bits.

> What could work is a type that has explicit padding:
> 
> struct timespec {
>   __s64 tv_sec;
> #ifdef BIG_ENDIAN_32BIT
>   u32 __pad;
> #endif
>   long tv_nsec;
> #ifdef LITTLE_ENDIAN_32BIT
>   u32 __pad;
> #endif
> };

Yes.  That's fine for POSIX (and I think x32 should move to such a 
definition, though for glibc there would still be the issue of 
compatibility with existing kernels).

> For timeval, I think we don't care about the padding, because we wouldn't
> use it on new interfaces when the kernel uses nanosecond resolution
> internally.

Indeed - I haven't checked, but hopefully none of the asm-generic 
interfaces use timeval (and I was suggesting that only the interfaces 
present in asm-generic should be duplicated for 64-bit time_t).

(You also mentioned ioctls.  All affected ioctls would need new variants 
unless you have a new variant of the ioctl syscall itself.  Logically I 
suppose _TIME_BITS=64 should then map the generic version of an ioctl to 
the 64-bit time_t version so applications don't need to change beyond 
defining _TIME_BITS=64, though I don't know if there's any precedent for 
doing that sort of thing for definitions in kernel headers rather than 
glibc's headers.)

-- 
Joseph S. Myers
jos...@codesourcery.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Arnd Bergmann]

On Thursday 15 May 2014 20:10:05 Joseph S. Myers wrote:
> On Thu, 15 May 2014, Arnd Bergmann wrote:
> 
> > Earlier in the thread there seemed to be a rough consensus that
> > _TIME_BITS=64 wouldn't be a good idea because we wouldn't get everything
> > changed to use it. For _FILE_OFFSET_BITS=64 that's ok because most
> > user space doesn't ever want to deal with large files.
> 
> Well, I'm coming into this in the middle since it isn't on linux-api and 
> noone has tried to work out on libc-alpha what things should look like 
> from the glibc side.  _TIME_BITS seemed to make sense when I thought about 
> this previously, however.
> 
> > Can you elaborate on how the switch to the new default would work?
> 
> At some appropriate release (probably after _TIME_BITS=64 is widely used 
> in distributions), the glibc headers would change so that _TIME_BITS=64 is 
> the default and _TIME_BITS=32 can be set to get the old interfaces.  At 
> some later point _TIME_BITS=32 API support might be removed, leaving the 
> old symbols as compat symbols for existing binaries.

Ok.

> > If it's easy, why hasn't it been done for _FILE_OFFSET_BITS already
> > and what's stopping us from changing the default as soon as the interfaces
> > are there? If it's hard, what would need to happen before the default
> > time_t can be set?
> 
> The distribution side of the change for _FILE_OFFSET_BITS (i.e., moving to 
> building libraries that way so a glibc change to the default wouldn't 
> cause issues for other libraries' ABIs) has gradually been done.  The 
> discussion in March on libc-alpha about changing the default tailed off.  
> This is something that needs someone to take the lead with a *careful and 
> detailed analysis of the information from the previous discussion* in 
> order to present a properly reasoned proposal for a change to the default 
> - not scattergun patches, not patches with brief or no analysis of the 
> environment in which glibc is used, not dismissing concerns, but a 
> properly reasoned argument for why the change should be made, along with 
> details of how distributions can determine whether ABI issues would arise 
> from rebuilding a particular library against newer glibc.

Ok, I see. I wasn't aware that distributions actually set _FILE_OFFSET_BITS
globally for building packages. I guess the effect (from the distro point
of view) of that is similar to having a configure option when building glibc
as I expected to be the normal way to do it.

> > > Obviously 64-bit time_t syscalls would be an appropriately narrow set of 
> > > syscalls like those in the generic ABI (so glibc would implement stat for 
> > > _TIME_BITS=64 using fstatat64_time64 or whatever the syscall is called, 
> > > just as the stat functions for generic ABI architectures are implemented 
> > > with newfstatat / fstatat64 rather than lots of separate syscalls.
> > 
> > This assumes that we'd leave the kernel time_t/timespec/timeval using 'long'
> > and introduce a new timespec64 using a signed 64-bit type, rather than
> > changing the kernel headers to the new syscalls and data structures with
> > new names for the existing ones, right?
> 
> Yes.  I consider it simply common sense that new kernel headers should 
> continue to work with much older glibc, meaning that the API (syscall 
> names etc.) presented by the headers from headers_install should not 
> change incompatibly.

Right. we have done it both ways in the past, but it seems that renaming
syscalls hasn't been done in some time. I can only find definitions for
oldfstat, oldlstat, oldolduname, olduname, oldumount, vm86old and oldwait4.
It's possible they all predate libc6.

> (64-bit type only for time_t, of course.  There's no need for a 64-bit 
> type for nanoseconds and tv_nsec is explicitly "long" in POSIX, meaning 
> that if the kernel uses a 64-bit type for nanoseconds on systems where 
> "long" is 32-bit in userspace, either it needs to treat the high word as 
> padding or glibc needs to wrap all interfaces passing a struct timespec 
> into the kernel so they clear the padding field.  There's even less need 
> for a 64-bit type for microseconds.)

For practical purposes in the kernel, we may still want to use 64-bit
nanoseconds: if we use a 96 bit struct timespec, that would be incompatible
with the native type on 64-bit kernels, thus complicating the syscall
emulation layer.

I don't know why timespec on x32 uses 'long tv_nsec', it does seem
problematic.

What could work is a type that has explicit padding:

struct timespec {
__s64 tv_sec;
#ifdef BIG_ENDIAN_32BIT
u32 __pad;
#endif
long tv_nsec;
#ifdef LITTLE_ENDIAN_32BIT
u32 __pad;
#endif
};

For timeval, I think we don't care about the padding, because we wouldn't
use it on new interfaces when the kernel uses nanosecond resolution
internally.

Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
Mor

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Joseph S. Myers]

On Thu, 15 May 2014, Arnd Bergmann wrote:

> Earlier in the thread there seemed to be a rough consensus that
> _TIME_BITS=64 wouldn't be a good idea because we wouldn't get everything
> changed to use it. For _FILE_OFFSET_BITS=64 that's ok because most
> user space doesn't ever want to deal with large files.

Well, I'm coming into this in the middle since it isn't on linux-api and 
noone has tried to work out on libc-alpha what things should look like 
from the glibc side.  _TIME_BITS seemed to make sense when I thought about 
this previously, however.

> Can you elaborate on how the switch to the new default would work?

At some appropriate release (probably after _TIME_BITS=64 is widely used 
in distributions), the glibc headers would change so that _TIME_BITS=64 is 
the default and _TIME_BITS=32 can be set to get the old interfaces.  At 
some later point _TIME_BITS=32 API support might be removed, leaving the 
old symbols as compat symbols for existing binaries.

> If it's easy, why hasn't it been done for _FILE_OFFSET_BITS already
> and what's stopping us from changing the default as soon as the interfaces
> are there? If it's hard, what would need to happen before the default
> time_t can be set?

The distribution side of the change for _FILE_OFFSET_BITS (i.e., moving to 
building libraries that way so a glibc change to the default wouldn't 
cause issues for other libraries' ABIs) has gradually been done.  The 
discussion in March on libc-alpha about changing the default tailed off.  
This is something that needs someone to take the lead with a *careful and 
detailed analysis of the information from the previous discussion* in 
order to present a properly reasoned proposal for a change to the default 
- not scattergun patches, not patches with brief or no analysis of the 
environment in which glibc is used, not dismissing concerns, but a 
properly reasoned argument for why the change should be made, along with 
details of how distributions can determine whether ABI issues would arise 
from rebuilding a particular library against newer glibc.

> > Obviously 64-bit time_t syscalls would be an appropriately narrow set of 
> > syscalls like those in the generic ABI (so glibc would implement stat for 
> > _TIME_BITS=64 using fstatat64_time64 or whatever the syscall is called, 
> > just as the stat functions for generic ABI architectures are implemented 
> > with newfstatat / fstatat64 rather than lots of separate syscalls.
> 
> This assumes that we'd leave the kernel time_t/timespec/timeval using 'long'
> and introduce a new timespec64 using a signed 64-bit type, rather than
> changing the kernel headers to the new syscalls and data structures with
> new names for the existing ones, right?

Yes.  I consider it simply common sense that new kernel headers should 
continue to work with much older glibc, meaning that the API (syscall 
names etc.) presented by the headers from headers_install should not 
change incompatibly.

(64-bit type only for time_t, of course.  There's no need for a 64-bit 
type for nanoseconds and tv_nsec is explicitly "long" in POSIX, meaning 
that if the kernel uses a 64-bit type for nanoseconds on systems where 
"long" is 32-bit in userspace, either it needs to treat the high word as 
padding or glibc needs to wrap all interfaces passing a struct timespec 
into the kernel so they clear the padding field.  There's even less need 
for a 64-bit type for microseconds.)

-- 
Joseph S. Myers
jos...@codesourcery.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Arnd Bergmann]

On Thursday 15 May 2014 18:01:37 Joseph S. Myers wrote:
> On Thu, 15 May 2014, Chung-Lin Tang wrote:
> 
> > > c) glibc may or may not provide a way for applications to use
> > >the extended interfaces without a user space ABI break. My
> > >impression so far is that this is going to be too hard and
> > >it won't be done, but this is for the glibc developers to
> > >determine.
> > 
> > glibc does version its exported symbols, so provided new/old syscalls
> > are both provided, haveing a new version of a routine (using 64-bit
> > time_t and new syscall interfaces) and the old compat routine co-exist
> > should be possible. Of course, old binaries may still not be saved when
> > 2038 arrives.

How would that work for ioctl? There are a lot of device drivers that
have data structure with time_t or struct timespec in them.

> You could do it with symbol versioning, though my guess would be that it 
> would be natural to provide _TIME_BITS like _FILE_OFFSET_BITS (with the 
> combination _TIME_BITS=64, _FILE_OFFSET_BITS=32 not being supported at 
> all) and then later transition the default.  (Discussion of the 
> possibility of moving the default to _FILE_OFFSET_BITS=64 suggested that 
> in practice most libraries in distributions where this affects the ABI are 
> already built that way, though noone has followed up on that discussion to 
> try to reach a consensus that we have enough evidence to change the 
> default now.  There are a couple of issues with the way 
> _FILE_OFFSET_BITS=64 is implemented that should be avoided in any 
> implementation of _TIME_BITS=64: (a) _FILE_OFFSET_BITS=64 maps functions 
> to corresponding *64 functions in the user's namespace, when it should use 
> reserved-namespace names; (b) it's not a no-op for 64-bit platforms - it 
> still remaps function names and types.  So _TIME_BITS=64 should only map 
> to reserved-namespace names, and not do any remapping at all on platforms 
> where time_t is already 64-bit.)

Earlier in the thread there seemed to be a rough consensus that
_TIME_BITS=64 wouldn't be a good idea because we wouldn't get everything
changed to use it. For _FILE_OFFSET_BITS=64 that's ok because most
user space doesn't ever want to deal with large files.

Can you elaborate on how the switch to the new default would work?
If it's easy, why hasn't it been done for _FILE_OFFSET_BITS already
and what's stopping us from changing the default as soon as the interfaces
are there? If it's hard, what would need to happen before the default
time_t can be set?

> Obviously 64-bit time_t syscalls would be an appropriately narrow set of 
> syscalls like those in the generic ABI (so glibc would implement stat for 
> _TIME_BITS=64 using fstatat64_time64 or whatever the syscall is called, 
> just as the stat functions for generic ABI architectures are implemented 
> with newfstatat / fstatat64 rather than lots of separate syscalls.

This assumes that we'd leave the kernel time_t/timespec/timeval using 'long'
and introduce a new timespec64 using a signed 64-bit type, rather than
changing the kernel headers to the new syscalls and data structures with
new names for the existing ones, right?

Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Joseph S. Myers]

On Thu, 15 May 2014, Chung-Lin Tang wrote:

> > c) glibc may or may not provide a way for applications to use
> >the extended interfaces without a user space ABI break. My
> >impression so far is that this is going to be too hard and
> >it won't be done, but this is for the glibc developers to
> >determine.
> 
> glibc does version its exported symbols, so provided new/old syscalls
> are both provided, haveing a new version of a routine (using 64-bit
> time_t and new syscall interfaces) and the old compat routine co-exist
> should be possible. Of course, old binaries may still not be saved when
> 2038 arrives.

You could do it with symbol versioning, though my guess would be that it 
would be natural to provide _TIME_BITS like _FILE_OFFSET_BITS (with the 
combination _TIME_BITS=64, _FILE_OFFSET_BITS=32 not being supported at 
all) and then later transition the default.  (Discussion of the 
possibility of moving the default to _FILE_OFFSET_BITS=64 suggested that 
in practice most libraries in distributions where this affects the ABI are 
already built that way, though noone has followed up on that discussion to 
try to reach a consensus that we have enough evidence to change the 
default now.  There are a couple of issues with the way 
_FILE_OFFSET_BITS=64 is implemented that should be avoided in any 
implementation of _TIME_BITS=64: (a) _FILE_OFFSET_BITS=64 maps functions 
to corresponding *64 functions in the user's namespace, when it should use 
reserved-namespace names; (b) it's not a no-op for 64-bit platforms - it 
still remaps function names and types.  So _TIME_BITS=64 should only map 
to reserved-namespace names, and not do any remapping at all on platforms 
where time_t is already 64-bit.)

Obviously 64-bit time_t syscalls would be an appropriately narrow set of 
syscalls like those in the generic ABI (so glibc would implement stat for 
_TIME_BITS=64 using fstatat64_time64 or whatever the syscall is called, 
just as the stat functions for generic ABI architectures are implemented 
with newfstatat / fstatat64 rather than lots of separate syscalls.

-- 
Joseph S. Myers
jos...@codesourcery.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Chung-Lin Tang]

On 2014/5/15 07:08 PM, Arnd Bergmann wrote:
> On Wednesday 14 May 2014 14:33:18 John Stultz wrote:
>> On Tue, May 13, 2014 at 12:32 PM, Arnd Bergmann  wrote:
>>> On Tuesday 13 May 2014 20:24:59 Geert Uytterhoeven wrote:
 On Tue, May 13, 2014 at 8:10 PM, Arnd Bergmann  wrote:
> Using 64-bit time_t on x32 is fine, because it's fast to operate
> in user space with 64-bit registers, and the kernel is 64-bit
> anyway. Inside of the kernel, we may get into trouble using
> a 64-bit time_t on 32-bit architectures because of the overhead
> in 64-bit math, e.g. all the timekeeping code that is based on
> timespec or some code paths in file systems and network code where
> we actually require division of time_t values.

 While going over time_t uses, have you found a pattern for use cases
 involving division of time_t values in filesystem and networking code?
>>>
>>> In ipv4, we have multiple places doing this:
>>>
>>> icmp_param.data.times[1] = htonl((tv.tv_sec % 86400) * MSEC_PER_SEC 
>>> +
>>>  tv.tv_nsec / NSEC_PER_MSEC);
>>>
>>> to calculate the miliseconds since midnight. For file systems, I
>>> found that FAT uses seconds/minutes/hours/days/month/year representation,
>>> which is a lot of divides, but that can probably be optimized and
>>> we need to handle years beyond 2038 anyway.
>>
>> We can do some tricks for internal optimizations here if these are
>> critical.  I'd be more concerned about userland divisions where moving
>> to a 64bit time_t would cause performance issues that we cannot help
>> optimize.
> 
> Good point.
> 
> We clearly have to change that code in some for to deal with y2038,
> but 64-bit time_t may not be the best option. A lot of the
> in-kernel code can probably use ktime_t, which we can change
> to a different representation (e.g. 34 bit seconds) if needed,
> and all the code that is only interested in relative time
> (e.g. nanosleep) doesn't have to change at all.

 Yeah. 32-bit uptimes should be good enough for everyone (don't quote
 me on that), so adding a 64-bit offset when there's a need for absolute
 time should be OK.
>>>
>>> I think we have three categories:
>>>
>>> a) interfaces that uses relative time_t/timespec/timeval:
>>>- nanosleep
>>>- select/pselect/poll/ppoll/epoll
>>>- getrusage
>>>- sched_rr_get_interval
>>>- sigtimedwait
>>>- clock_nanosleep
>>>- alarm
>>>- siginfo (rusage)
>>>
>>>   These can stay compatible, but we'd have to use a different
>>>   type if we change time_t.
>>
>>
>> So as a correction, at least clock_nanosleep can specify sleep times
>> using absolute time.
> 
> Thanks.
> 
>>> b) interfaces that don't make sense for times in the past:
>>>- getitimer/setitimer
>>>- timer_settime/timer_gettime
>>>- gettimeofday/settimeofday
>>>- adjtimex
>>>- clock_gettime/clock_settime/clock_adjtime
>>>- time/stime
>>>- socket time stamps
>>>- audio time stamps
>>>- v4l time stamps
>>>- input event time stamps
>>>- sysv ipc (msg, sem, shm)
>>>
>>>Here, we are relatively free to change the start of the
>>>epoch in the kernel but convert to something else on the
>>>user space boundary. One possibility is to scale them to
>>>boot time and use ktime_t in the kernel.
>>
>> I'm not sure I'm totally following this... Are you suggesting we keep
>> 32bit time internally w/ some different offset but then pass to
>> userland a 64bit time_t?  Or are you suggesting we change the abi to
>> move the epoch?
> 
> What I meant is that regardless of what we decide for the ABI,
> we can change the in-kernel representation in any way we like as
> long as we can represent all dates that can occur during the runtime
> of the kernel, i.e. we don't have to represent times between 1970 and
> 2014. This could mean one of many representations:
> 
> - time_t scaled forward by 44 years and/or made unsigned
> - ktime_t scaled to boot time
> - 64-bit nanoseconds starting at the epoch
> - timespec64
> 
>> I think I'm with hpa in his recent mail in that the internal
>> representation is an optimization detail, and the bigger question is
>> do we use a  64bit time_t for future systems (possibly w/ a major ABI
>> break - with compat interface for existing 32bit applications), or do
>> we try to rev interfaces function by function to provide 2038 safe
>> versions which applications will have to be modified to use?
>>
>> Me, I'm a fan of moving time_t to 64bits, since it makes "porting"
>> applications to a 2038 safe ABI easier.
> 
> I think there are two or three distinct problems:
> 
> a) We absolutely have to find a way to build a user space that
>can survive 2038. This probably involves moving at least
>time_t, timeval and timespec to use 64-bit representation.
> b) We have to keep compatibility with existing user space running
>on future kernels, which means at lea

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Chung-Lin Tang]

On 2014/5/15 07:08 PM, Arnd Bergmann wrote:
> On Wednesday 14 May 2014 14:33:18 John Stultz wrote:
>> On Tue, May 13, 2014 at 12:32 PM, Arnd Bergmann  wrote:
>>> On Tuesday 13 May 2014 20:24:59 Geert Uytterhoeven wrote:
 On Tue, May 13, 2014 at 8:10 PM, Arnd Bergmann  wrote:
> Using 64-bit time_t on x32 is fine, because it's fast to operate
> in user space with 64-bit registers, and the kernel is 64-bit
> anyway. Inside of the kernel, we may get into trouble using
> a 64-bit time_t on 32-bit architectures because of the overhead
> in 64-bit math, e.g. all the timekeeping code that is based on
> timespec or some code paths in file systems and network code where
> we actually require division of time_t values.

 While going over time_t uses, have you found a pattern for use cases
 involving division of time_t values in filesystem and networking code?
>>>
>>> In ipv4, we have multiple places doing this:
>>>
>>> icmp_param.data.times[1] = htonl((tv.tv_sec % 86400) * MSEC_PER_SEC 
>>> +
>>>  tv.tv_nsec / NSEC_PER_MSEC);
>>>
>>> to calculate the miliseconds since midnight. For file systems, I
>>> found that FAT uses seconds/minutes/hours/days/month/year representation,
>>> which is a lot of divides, but that can probably be optimized and
>>> we need to handle years beyond 2038 anyway.
>>
>> We can do some tricks for internal optimizations here if these are
>> critical.  I'd be more concerned about userland divisions where moving
>> to a 64bit time_t would cause performance issues that we cannot help
>> optimize.
> 
> Good point.
> 
> We clearly have to change that code in some for to deal with y2038,
> but 64-bit time_t may not be the best option. A lot of the
> in-kernel code can probably use ktime_t, which we can change
> to a different representation (e.g. 34 bit seconds) if needed,
> and all the code that is only interested in relative time
> (e.g. nanosleep) doesn't have to change at all.

 Yeah. 32-bit uptimes should be good enough for everyone (don't quote
 me on that), so adding a 64-bit offset when there's a need for absolute
 time should be OK.
>>>
>>> I think we have three categories:
>>>
>>> a) interfaces that uses relative time_t/timespec/timeval:
>>>- nanosleep
>>>- select/pselect/poll/ppoll/epoll
>>>- getrusage
>>>- sched_rr_get_interval
>>>- sigtimedwait
>>>- clock_nanosleep
>>>- alarm
>>>- siginfo (rusage)
>>>
>>>   These can stay compatible, but we'd have to use a different
>>>   type if we change time_t.
>>
>>
>> So as a correction, at least clock_nanosleep can specify sleep times
>> using absolute time.
> 
> Thanks.
> 
>>> b) interfaces that don't make sense for times in the past:
>>>- getitimer/setitimer
>>>- timer_settime/timer_gettime
>>>- gettimeofday/settimeofday
>>>- adjtimex
>>>- clock_gettime/clock_settime/clock_adjtime
>>>- time/stime
>>>- socket time stamps
>>>- audio time stamps
>>>- v4l time stamps
>>>- input event time stamps
>>>- sysv ipc (msg, sem, shm)
>>>
>>>Here, we are relatively free to change the start of the
>>>epoch in the kernel but convert to something else on the
>>>user space boundary. One possibility is to scale them to
>>>boot time and use ktime_t in the kernel.
>>
>> I'm not sure I'm totally following this... Are you suggesting we keep
>> 32bit time internally w/ some different offset but then pass to
>> userland a 64bit time_t?  Or are you suggesting we change the abi to
>> move the epoch?
> 
> What I meant is that regardless of what we decide for the ABI,
> we can change the in-kernel representation in any way we like as
> long as we can represent all dates that can occur during the runtime
> of the kernel, i.e. we don't have to represent times between 1970 and
> 2014. This could mean one of many representations:
> 
> - time_t scaled forward by 44 years and/or made unsigned
> - ktime_t scaled to boot time
> - 64-bit nanoseconds starting at the epoch
> - timespec64
> 
>> I think I'm with hpa in his recent mail in that the internal
>> representation is an optimization detail, and the bigger question is
>> do we use a  64bit time_t for future systems (possibly w/ a major ABI
>> break - with compat interface for existing 32bit applications), or do
>> we try to rev interfaces function by function to provide 2038 safe
>> versions which applications will have to be modified to use?
>>
>> Me, I'm a fan of moving time_t to 64bits, since it makes "porting"
>> applications to a 2038 safe ABI easier.
> 
> I think there are two or three distinct problems:
> 
> a) We absolutely have to find a way to build a user space that
>can survive 2038. This probably involves moving at least
>time_t, timeval and timespec to use 64-bit representation.
> b) We have to keep compatibility with existing user space running
>on future kernels, which means at lea

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Arnd Bergmann]

On Thursday 15 May 2014 11:47:03 James Bottomley wrote:
> On Wed, 2014-05-14 at 13:00 +0200, Arnd Bergmann wrote:
> > On Tuesday 13 May 2014 22:35:08 Geert Uytterhoeven wrote:
> > > 
> > > On Tue, May 13, 2014 at 9:32 PM, Arnd Bergmann  wrote:
> > > > I think we have three categories:
> > > 
> > > Thanks for the list!
> > > 
> > > > a) interfaces that uses relative time_t/timespec/timeval:
> > > > b) interfaces that don't make sense for times in the past:
> > > 
> > > > c) interfaces that require absolute times:
> > > >- stat/lstat/fstatat/
> > > >- utime/utimes/futimesat
> > > >
> > > > These absolutely have to use something better than time_t
> > > > both in user space and in the kernel so we can deal with
> > > > old files. A lot of file systems need to be fixed as well so
> > > > we can actually store the times, regardless of whether we
> > > > are running a 32 or 64 bit kernel.
> > > 
> > > So these are the ones we have to worry about.
> > > It looks like they all involve I/O? Apart from the case of using block 
> > > data
> > > from the buffer cache, the 64-bit operations should disappear in the
> > > actual I/O noise, right?
> > 
> > Right. Also there have been proposals for a better 'stat' replacement
> > for years, which would solve half of the interface problem for the
> > file system interfaces.
> > 
> > However, we also need to find a solution for category b), I only put
> > them into a different category above because we can treat them
> > differently in the kernel. For instance, we could use ktime_t for
> > the kernel code in category b) and a new struct timespec64 for
> > the times in struct inode.
> > On the user interface side, using timespec64 would be a reasonable
> > choice for both categories, because we already have two implementations
> > of all those syscalls in order to handle 32-on-64 compat tasks,
> > and we could use the same set of syscall implementations for time64-on-32.
> 
> To step back a bit, is a 64 bit time_t actually a good solution on 32
> bits?  Paying a 64 bit penalty on every time operation does seem a bit
> overkill.  Most eventual uses are either monotonic counting or relative
> addition/subtraction.  If we added an additional 32 bit quantity called
> epoch, this would increase once every 68 years.  Within the kernel, we
> could do intelligent interpolation about what the epoch is, so in 2039
> if we get a low 32 bit time_t value, we assume epoch == 1, conversely if
> we see a high one, we assume epoch == 0.  We could add epoch on the end
> of the syscalls and detect if it's not present and fill in an
> interpolated value.  Interfaces which truly want 64 bit time_t would get
> it from epoch + 32 bit time_t.

The main advantage that I see with 64-bit time_t is that a lot of user
space already works with it, because NetBSD and OpenBSD use this, and
all 64-bit Linux systems already work with this without changing the ABI.
If we want a POSIX extension to cover a new ABI, this would be the most
likely candidate.

For Linux-only interfaces, the 64-bit nanosecond timestamps seem like
a good alternative, and they would also speed up the 64-bit architectures
because we can skip the normalization.

As mentioned earlier, between kernel and user space it's probably best
to avoid time_t and timeval completely and just use timespec64 or some
other safe type, but there has to be a way to port user space that relies
on time_t or timespec.

Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[James Bottomley]

On Wed, 2014-05-14 at 13:00 +0200, Arnd Bergmann wrote:
> On Tuesday 13 May 2014 22:35:08 Geert Uytterhoeven wrote:
> > 
> > On Tue, May 13, 2014 at 9:32 PM, Arnd Bergmann  wrote:
> > > I think we have three categories:
> > 
> > Thanks for the list!
> > 
> > > a) interfaces that uses relative time_t/timespec/timeval:
> > > b) interfaces that don't make sense for times in the past:
> > 
> > > c) interfaces that require absolute times:
> > >- stat/lstat/fstatat/
> > >- utime/utimes/futimesat
> > >
> > > These absolutely have to use something better than time_t
> > > both in user space and in the kernel so we can deal with
> > > old files. A lot of file systems need to be fixed as well so
> > > we can actually store the times, regardless of whether we
> > > are running a 32 or 64 bit kernel.
> > 
> > So these are the ones we have to worry about.
> > It looks like they all involve I/O? Apart from the case of using block data
> > from the buffer cache, the 64-bit operations should disappear in the
> > actual I/O noise, right?
> 
> Right. Also there have been proposals for a better 'stat' replacement
> for years, which would solve half of the interface problem for the
> file system interfaces.
> 
> However, we also need to find a solution for category b), I only put
> them into a different category above because we can treat them
> differently in the kernel. For instance, we could use ktime_t for
> the kernel code in category b) and a new struct timespec64 for
> the times in struct inode.
> On the user interface side, using timespec64 would be a reasonable
> choice for both categories, because we already have two implementations
> of all those syscalls in order to handle 32-on-64 compat tasks,
> and we could use the same set of syscall implementations for time64-on-32.

To step back a bit, is a 64 bit time_t actually a good solution on 32
bits?  Paying a 64 bit penalty on every time operation does seem a bit
overkill.  Most eventual uses are either monotonic counting or relative
addition/subtraction.  If we added an additional 32 bit quantity called
epoch, this would increase once every 68 years.  Within the kernel, we
could do intelligent interpolation about what the epoch is, so in 2039
if we get a low 32 bit time_t value, we assume epoch == 1, conversely if
we see a high one, we assume epoch == 0.  We could add epoch on the end
of the syscalls and detect if it's not present and fill in an
interpolated value.  Interfaces which truly want 64 bit time_t would get
it from epoch + 32 bit time_t.

James


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Christoph Hellwig]

On Thu, May 15, 2014 at 01:08:02PM +0200, Arnd Bergmann wrote:
> c) glibc may or may not provide a way for applications to use
>the extended interfaces without a user space ABI break. My
>impression so far is that this is going to be too hard and
>it won't be done, but this is for the glibc developers to
>determine.
>
> The important distinction here is between user space time_t
> (timeval, timespec) and __kernel_time_t. We probably need
> to make the user space time_t a build-time conditional,
> at least for the foreseeable future. New architectures or
> new C libraries can start out using 64-bit time_t unconditionally.

I don't think there is any point in introducing new kernel interfaces
without having a talk to the glibc folks and people that represent the
Linux community in the standard body (e.g. Eric Blake).

How about you start sending a first high-level proposal with libc-alpha,
linux-kernel, linux-api and Eric in the Cc list and we'll see how we
can come up with something that is actually usable to solve our users
Y2038 problem.  Without coordination we're just handing on the hot
potato.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Arnd Bergmann]

On Wednesday 14 May 2014 14:33:18 John Stultz wrote:
> On Tue, May 13, 2014 at 12:32 PM, Arnd Bergmann  wrote:
> > On Tuesday 13 May 2014 20:24:59 Geert Uytterhoeven wrote:
> >> On Tue, May 13, 2014 at 8:10 PM, Arnd Bergmann  wrote:
> >> > Using 64-bit time_t on x32 is fine, because it's fast to operate
> >> > in user space with 64-bit registers, and the kernel is 64-bit
> >> > anyway. Inside of the kernel, we may get into trouble using
> >> > a 64-bit time_t on 32-bit architectures because of the overhead
> >> > in 64-bit math, e.g. all the timekeeping code that is based on
> >> > timespec or some code paths in file systems and network code where
> >> > we actually require division of time_t values.
> >>
> >> While going over time_t uses, have you found a pattern for use cases
> >> involving division of time_t values in filesystem and networking code?
> >
> > In ipv4, we have multiple places doing this:
> >
> > icmp_param.data.times[1] = htonl((tv.tv_sec % 86400) * MSEC_PER_SEC 
> > +
> >  tv.tv_nsec / NSEC_PER_MSEC);
> >
> > to calculate the miliseconds since midnight. For file systems, I
> > found that FAT uses seconds/minutes/hours/days/month/year representation,
> > which is a lot of divides, but that can probably be optimized and
> > we need to handle years beyond 2038 anyway.
> 
> We can do some tricks for internal optimizations here if these are
> critical.  I'd be more concerned about userland divisions where moving
> to a 64bit time_t would cause performance issues that we cannot help
> optimize.

Good point.

> >> > We clearly have to change that code in some for to deal with y2038,
> >> > but 64-bit time_t may not be the best option. A lot of the
> >> > in-kernel code can probably use ktime_t, which we can change
> >> > to a different representation (e.g. 34 bit seconds) if needed,
> >> > and all the code that is only interested in relative time
> >> > (e.g. nanosleep) doesn't have to change at all.
> >>
> >> Yeah. 32-bit uptimes should be good enough for everyone (don't quote
> >> me on that), so adding a 64-bit offset when there's a need for absolute
> >> time should be OK.
> >
> > I think we have three categories:
> >
> > a) interfaces that uses relative time_t/timespec/timeval:
> >- nanosleep
> >- select/pselect/poll/ppoll/epoll
> >- getrusage
> >- sched_rr_get_interval
> >- sigtimedwait
> >- clock_nanosleep
> >- alarm
> >- siginfo (rusage)
> >
> >   These can stay compatible, but we'd have to use a different
> >   type if we change time_t.
> 
> 
> So as a correction, at least clock_nanosleep can specify sleep times
> using absolute time.

Thanks.

> > b) interfaces that don't make sense for times in the past:
> >- getitimer/setitimer
> >- timer_settime/timer_gettime
> >- gettimeofday/settimeofday
> >- adjtimex
> >- clock_gettime/clock_settime/clock_adjtime
> >- time/stime
> >- socket time stamps
> >- audio time stamps
> >- v4l time stamps
> >- input event time stamps
> >- sysv ipc (msg, sem, shm)
> >
> >Here, we are relatively free to change the start of the
> >epoch in the kernel but convert to something else on the
> >user space boundary. One possibility is to scale them to
> >boot time and use ktime_t in the kernel.
> 
> I'm not sure I'm totally following this... Are you suggesting we keep
> 32bit time internally w/ some different offset but then pass to
> userland a 64bit time_t?  Or are you suggesting we change the abi to
> move the epoch?

What I meant is that regardless of what we decide for the ABI,
we can change the in-kernel representation in any way we like as
long as we can represent all dates that can occur during the runtime
of the kernel, i.e. we don't have to represent times between 1970 and
2014. This could mean one of many representations:

- time_t scaled forward by 44 years and/or made unsigned
- ktime_t scaled to boot time
- 64-bit nanoseconds starting at the epoch
- timespec64

> I think I'm with hpa in his recent mail in that the internal
> representation is an optimization detail, and the bigger question is
> do we use a  64bit time_t for future systems (possibly w/ a major ABI
> break - with compat interface for existing 32bit applications), or do
> we try to rev interfaces function by function to provide 2038 safe
> versions which applications will have to be modified to use?
> 
> Me, I'm a fan of moving time_t to 64bits, since it makes "porting"
> applications to a 2038 safe ABI easier.

I think there are two or three distinct problems:

a) We absolutely have to find a way to build a user space that
   can survive 2038. This probably involves moving at least
   time_t, timeval and timespec to use 64-bit representation.
b) We have to keep compatibility with existing user space running
   on future kernels, which means at least x86, arm and a few
   other 32-bit architectures (we can ignore some of the obsolete
   ones if that 

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Nicolas Pitre]

On Wed, 14 May 2014, H. Peter Anvin wrote:

> On 05/14/2014 02:33 PM, John Stultz wrote:
> > 
> > We can do some tricks for internal optimizations here if these are
> > critical.  I'd be more concerned about userland divisions where moving
> > to a 64bit time_t would cause performance issues that we cannot help
> > optimize.
> > 
> 
> Honestly, the cost of not addressing the Y2038 problem in plenty of time
> is going to be way, way, way, way more expensive than some additional
> arithmetic.
> 
> As far as converting to calendar notation (e.g. the Gregorian calendar),
> only the first division needs to be done with full time_t width.
> Furthermore, since the divisor is known a priori, the compiler *should*
> be able to do it as an inverse multiply.[1]

The compiler is apparently not that smart.  It does it already for 32 by 
32 bit divisions, but not for 64 by 32 divisions. Fortunately, with 
gcc-4 at least, constant propagation on long longs works well enough to 
do it by hand.  That's how I implemented do_div() for ARM years ago.  
See arch/arm/include/asm/div64.h from line 65.  The same could be 
implemented for x86 and others.


Nicolas
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[H. Peter Anvin]

On 05/14/2014 02:33 PM, John Stultz wrote:
> 
> We can do some tricks for internal optimizations here if these are
> critical.  I'd be more concerned about userland divisions where moving
> to a 64bit time_t would cause performance issues that we cannot help
> optimize.
> 

Honestly, the cost of not addressing the Y2038 problem in plenty of time
is going to be way, way, way, way more expensive than some additional
arithmetic.

As far as converting to calendar notation (e.g. the Gregorian calendar),
only the first division needs to be done with full time_t width.
Furthermore, since the divisor is known a priori, the compiler *should*
be able to do it as an inverse multiply.[1]

The remaining divisions are bounded, and a smart compiler should even
know that.

> I'm not sure I'm totally following this... Are you suggesting we keep
> 32bit time internally w/ some different offset but then pass to
> userland a 64bit time_t?  Or are you suggesting we change the abi to
> move the epoch?

I think this makes sense as a way of emergency rescue of legacy ABIs --
similar to the windowing solution used to Y2K-rescue legacy systems.
Nonlegacy ABIs should just be designed correctly from the start.

> I think I'm with hpa in his recent mail in that the internal
> representation is an optimization detail, and the bigger question is146
> do we use a  64bit time_t for future systems (possibly w/ a major ABI
> break - with compat interface for existing 32bit applications), or do
> we try to rev interfaces function by function to provide 2038 safe
> versions which applications will have to be modified to use?
> 
> Me, I'm a fan of moving time_t to 64bits, since it makes "porting"
> applications to a 2038 safe ABI easier.

I obviously couldn't agree more.

-hpa


[1] Not entirely true: the highest possible divisor necessary when doing
Gregorian calendar calculations is 12_622_780_800 (0x2_f060_5980), the
number of seconds in each Gregorian 400-year cycle.  Similarly, dividing
with 86400 first to separate time and date requires the date part of the
result to remain 64 bits until the next division.  Thus, depending on
exactly how the calculations are partitioned it may be necessary to do
two 64-bit wide divisions; this can, however, be avoided with some care.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[John Stultz]

On Tue, May 13, 2014 at 12:32 PM, Arnd Bergmann  wrote:
> On Tuesday 13 May 2014 20:24:59 Geert Uytterhoeven wrote:
>> On Tue, May 13, 2014 at 8:10 PM, Arnd Bergmann  wrote:
>> > Using 64-bit time_t on x32 is fine, because it's fast to operate
>> > in user space with 64-bit registers, and the kernel is 64-bit
>> > anyway. Inside of the kernel, we may get into trouble using
>> > a 64-bit time_t on 32-bit architectures because of the overhead
>> > in 64-bit math, e.g. all the timekeeping code that is based on
>> > timespec or some code paths in file systems and network code where
>> > we actually require division of time_t values.
>>
>> While going over time_t uses, have you found a pattern for use cases
>> involving division of time_t values in filesystem and networking code?
>
> In ipv4, we have multiple places doing this:
>
> icmp_param.data.times[1] = htonl((tv.tv_sec % 86400) * MSEC_PER_SEC +
>  tv.tv_nsec / NSEC_PER_MSEC);
>
> to calculate the miliseconds since midnight. For file systems, I
> found that FAT uses seconds/minutes/hours/days/month/year representation,
> which is a lot of divides, but that can probably be optimized and
> we need to handle years beyond 2038 anyway.

We can do some tricks for internal optimizations here if these are
critical.  I'd be more concerned about userland divisions where moving
to a 64bit time_t would cause performance issues that we cannot help
optimize.


>> > We clearly have to change that code in some for to deal with y2038,
>> > but 64-bit time_t may not be the best option. A lot of the
>> > in-kernel code can probably use ktime_t, which we can change
>> > to a different representation (e.g. 34 bit seconds) if needed,
>> > and all the code that is only interested in relative time
>> > (e.g. nanosleep) doesn't have to change at all.
>>
>> Yeah. 32-bit uptimes should be good enough for everyone (don't quote
>> me on that), so adding a 64-bit offset when there's a need for absolute
>> time should be OK.
>
> I think we have three categories:
>
> a) interfaces that uses relative time_t/timespec/timeval:
>- nanosleep
>- select/pselect/poll/ppoll/epoll
>- getrusage
>- sched_rr_get_interval
>- sigtimedwait
>- clock_nanosleep
>- alarm
>- siginfo (rusage)
>
>   These can stay compatible, but we'd have to use a different
>   type if we change time_t.


So as a correction, at least clock_nanosleep can specify sleep times
using absolute time.


> b) interfaces that don't make sense for times in the past:
>- getitimer/setitimer
>- timer_settime/timer_gettime
>- gettimeofday/settimeofday
>- adjtimex
>- clock_gettime/clock_settime/clock_adjtime
>- time/stime
>- socket time stamps
>- audio time stamps
>- v4l time stamps
>- input event time stamps
>- sysv ipc (msg, sem, shm)
>
>Here, we are relatively free to change the start of the
>epoch in the kernel but convert to something else on the
>user space boundary. One possibility is to scale them to
>boot time and use ktime_t in the kernel.

I'm not sure I'm totally following this... Are you suggesting we keep
32bit time internally w/ some different offset but then pass to
userland a 64bit time_t?  Or are you suggesting we change the abi to
move the epoch?

I think I'm with hpa in his recent mail in that the internal
representation is an optimization detail, and the bigger question is
do we use a  64bit time_t for future systems (possibly w/ a major ABI
break - with compat interface for existing 32bit applications), or do
we try to rev interfaces function by function to provide 2038 safe
versions which applications will have to be modified to use?

Me, I'm a fan of moving time_t to 64bits, since it makes "porting"
applications to a 2038 safe ABI easier.

thanks
-john
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[H. Peter Anvin]

On 05/14/2014 03:33 AM, Arnd Bergmann wrote:
>>
>> Hi Arnd,
>>
>> From your comment above, can I assume we don't need this patchset any more?
> 
> I won't require it, but it's not just my decision to make.
> 

Linus has pushed extremely hard for it, and it *is* his decision to
make.  If you want an authoritative answer I guess we need to wait until
Linus comes back.

> Let's see what Peter Anvin and Thomas Gleixner think about it, as they
> have argued strongly in favor of using 64-bit time_t for new architectures
> in the past.

I think it is downright looney to introduce a new ABI without a 64-bit
time_t at this point.

You are talking about a ktime_t; that is a kernel internal
implementation detail which can be changed later once the appropriate
kernel infrastructure is there, but we should make the change now to get
the external ABIs right, then we can microoptimize the kernel internal
implementation later.

-hpa


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Arnd Bergmann]

On Wednesday 14 May 2014 16:46:29 Thomas Gleixner wrote:
> On Wed, 14 May 2014, Arnd Bergmann wrote:
> 
> > On Wednesday 14 May 2014 14:21:48 Thomas Gleixner wrote:
> > > 
> > > So in the 32-on-64 case we'll have two compat variants:
> > > 
> > > SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val,
> > > struct timespec __user *, utime, u32 __user *, uaddr2,
> > > u32, val3)
> > > 
> > > COMPAT_SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val,
> > > struct compat_timespec __user *, utime, u32 __user *, 
> > > uaddr2,
> > > u32, val3)
> > > 
> > > COMPAT_SYSCALL_DEFINE6(futex64, u32 __user *, uaddr, int, op, u32, val,
> > > struct timespec64 __user *, utime, u32 __user *, uaddr2,
> > > u32, val3)
> > > 
> > > The native 64bit futex64 syscall is mapped to futex.
> > 
> > I was actually hoping that we could map the compat futex64 to futex
> > as well here, since 64-bit timespec and compat timespec64 would be
> > the same structure.
> 
> Right, that might work with this one, but not for anything which has a
> pointer to a timespec and some other argument based on long.

We'd have to look at each one individually of course, but out of this
list, I think it will work for most:

   - getitimer/setitimer
   - timer_settime/timer_gettime
   - gettimeofday/settimeofday
   - adjtimex
   - clock_gettime/clock_settime/clock_adjtime
   - time/stime
   - sysv ipc (msg, sem, shm)
   - fstatat (or the statx replacement once we get to that)
   - futimesat

but not these ioctls:

   - socket time stamps
   - audio time stamps
   - v4l time stamps
   - input event time stamps

> > > I'm curious, whether quite some code, like high frequency timestamps
> > > wouldn't be better of with a strict 64 bit nanosecond granular time
> > > represenation.
> > 
> > At least in the kernel, I think ktime_t is already the right type
> > to use on both 64-bit and 32-bit architectures as it can be slow to
> > extract the seconds portion of 64-bit nanoseconds on a 32-bit machine.
> 
> On some of them yes. On i386 the u64 nsec ktime_t variant is way more
> efficient.

Ah, interesting. Do we have actual data for other architectures? If
it's much better on x86 and never much worse anywhere else, that
alone may be a reason to migrate the timestamp interfaces.

Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Thomas Gleixner]

On Wed, 14 May 2014, Arnd Bergmann wrote:

> On Wednesday 14 May 2014 14:21:48 Thomas Gleixner wrote:
> > 
> > So in the 32-on-64 case we'll have two compat variants:
> > 
> > SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val,
> > struct timespec __user *, utime, u32 __user *, uaddr2,
> > u32, val3)
> > 
> > COMPAT_SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val,
> > struct compat_timespec __user *, utime, u32 __user *, 
> > uaddr2,
> > u32, val3)
> > 
> > COMPAT_SYSCALL_DEFINE6(futex64, u32 __user *, uaddr, int, op, u32, val,
> > struct timespec64 __user *, utime, u32 __user *, uaddr2,
> > u32, val3)
> > 
> > The native 64bit futex64 syscall is mapped to futex.
> 
> I was actually hoping that we could map the compat futex64 to futex
> as well here, since 64-bit timespec and compat timespec64 would be
> the same structure.

Right, that might work with this one, but not for anything which has a
pointer to a timespec and some other argument based on long.

> > I'm curious, whether quite some code, like high frequency timestamps
> > wouldn't be better of with a strict 64 bit nanosecond granular time
> > represenation.
> 
> At least in the kernel, I think ktime_t is already the right type
> to use on both 64-bit and 32-bit architectures as it can be slow to
> extract the seconds portion of 64-bit nanoseconds on a 32-bit machine.

On some of them yes. On i386 the u64 nsec ktime_t variant is way more
efficient.

> FWIW, 64-bit ns gives us 584 years worth of nanoseconds, which
> means none of us or the people we know will be around before this
> becomes a problem ;-)

Indeed.
 
> For the user interface, we can decide which representation to use
> for each syscall individually depending on the needs. We should just
> not have to many different variants. I was going for timespec64
> just because that would allow us to keep the 64-bit kernel ABI
> unchanged.

Right. It's the way of least resistance.

If there are desires from user space to have a new format instead of
blindly timespec64 for certain syscalls, we should really think
discuss that on a case by case basis.

Thanks,

tglx

 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Arnd Bergmann]

On Wednesday 14 May 2014 14:21:48 Thomas Gleixner wrote:
> 
> So in the 32-on-64 case we'll have two compat variants:
> 
> SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val,
> struct timespec __user *, utime, u32 __user *, uaddr2,
> u32, val3)
> 
> COMPAT_SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val,
> struct compat_timespec __user *, utime, u32 __user *, uaddr2,
> u32, val3)
> 
> COMPAT_SYSCALL_DEFINE6(futex64, u32 __user *, uaddr, int, op, u32, val,
> struct timespec64 __user *, utime, u32 __user *, uaddr2,
> u32, val3)
> 
> The native 64bit futex64 syscall is mapped to futex.

I was actually hoping that we could map the compat futex64 to futex
as well here, since 64-bit timespec and compat timespec64 would be
the same structure.

> And for a 32bit kernel you have two
> 
> SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val,
> struct timespec __user *, utime, u32 __user *, uaddr2,
> u32, val3)
> 
> SYSCALL_DEFINE6(futex64, u32 __user *, uaddr, int, op, u32, val,
> struct timespec64 __user *, utime, u32 __user *, uaddr2,
> u32, val3)

And here, we could think about renaming the kernel internal symbols
so that we use the same two definitions as well, one for compat_timespec
and one for timespec64. Those are details we don't need to debate now.

> Fine with me, but we really need to discuss the timespec64 with user
> space folks.

Definitely.

> I'm curious, whether quite some code, like high frequency timestamps
> wouldn't be better of with a strict 64 bit nanosecond granular time
> represenation.

At least in the kernel, I think ktime_t is already the right type
to use on both 64-bit and 32-bit architectures as it can be slow to
extract the seconds portion of 64-bit nanoseconds on a 32-bit machine.
FWIW, 64-bit ns gives us 584 years worth of nanoseconds, which
means none of us or the people we know will be around before this
becomes a problem ;-)

For the user interface, we can decide which representation to use
for each syscall individually depending on the needs. We should just
not have to many different variants. I was going for timespec64
just because that would allow us to keep the 64-bit kernel ABI
unchanged.

> I often enough cursed timespec for clock_nanosleep on an absolute
> timeline. I need to go through all that normalizing stuff instead of
> just doing next_event += 50;

And all user space needs to do it too.

Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Thomas Gleixner]

On Wed, 14 May 2014, Arnd Bergmann wrote:
> On Tuesday 13 May 2014 22:35:08 Geert Uytterhoeven wrote:
> > 
> > On Tue, May 13, 2014 at 9:32 PM, Arnd Bergmann  wrote:
> > > I think we have three categories:
> > 
> > Thanks for the list!
> > 
> > > a) interfaces that uses relative time_t/timespec/timeval:
> > > b) interfaces that don't make sense for times in the past:
> > 
> > > c) interfaces that require absolute times:
> > >- stat/lstat/fstatat/
> > >- utime/utimes/futimesat
> > >
> > > These absolutely have to use something better than time_t
> > > both in user space and in the kernel so we can deal with
> > > old files. A lot of file systems need to be fixed as well so
> > > we can actually store the times, regardless of whether we
> > > are running a 32 or 64 bit kernel.
> > 
> > So these are the ones we have to worry about.
> > It looks like they all involve I/O? Apart from the case of using block data
> > from the buffer cache, the 64-bit operations should disappear in the
> > actual I/O noise, right?
> 
> Right. Also there have been proposals for a better 'stat' replacement
> for years, which would solve half of the interface problem for the
> file system interfaces.
> 
> However, we also need to find a solution for category b), I only put
> them into a different category above because we can treat them
> differently in the kernel. For instance, we could use ktime_t for
> the kernel code in category b) and a new struct timespec64 for
> the times in struct inode.
> On the user interface side, using timespec64 would be a reasonable
> choice for both categories, because we already have two implementations
> of all those syscalls in order to handle 32-on-64 compat tasks,
> and we could use the same set of syscall implementations for time64-on-32.

So in the 32-on-64 case we'll have two compat variants:

SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val,
struct timespec __user *, utime, u32 __user *, uaddr2,
u32, val3)

COMPAT_SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val,
struct compat_timespec __user *, utime, u32 __user *, uaddr2,
u32, val3)

COMPAT_SYSCALL_DEFINE6(futex64, u32 __user *, uaddr, int, op, u32, val,
struct timespec64 __user *, utime, u32 __user *, uaddr2,
u32, val3)

The native 64bit futex64 syscall is mapped to futex.

And for a 32bit kernel you have two

SYSCALL_DEFINE6(futex, u32 __user *, uaddr, int, op, u32, val,
struct timespec __user *, utime, u32 __user *, uaddr2,
u32, val3)

SYSCALL_DEFINE6(futex64, u32 __user *, uaddr, int, op, u32, val,
struct timespec64 __user *, utime, u32 __user *, uaddr2,
u32, val3)

Fine with me, but we really need to discuss the timespec64 with user
space folks.

I'm curious, whether quite some code, like high frequency timestamps
wouldn't be better of with a strict 64 bit nanosecond granular time
represenation.

I often enough cursed timespec for clock_nanosleep on an absolute
timeline. I need to go through all that normalizing stuff instead of
just doing next_event += 50;

Thanks,

tglx
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Arnd Bergmann]

On Tuesday 13 May 2014 22:35:08 Geert Uytterhoeven wrote:
> 
> On Tue, May 13, 2014 at 9:32 PM, Arnd Bergmann  wrote:
> > I think we have three categories:
> 
> Thanks for the list!
> 
> > a) interfaces that uses relative time_t/timespec/timeval:
> > b) interfaces that don't make sense for times in the past:
> 
> > c) interfaces that require absolute times:
> >- stat/lstat/fstatat/
> >- utime/utimes/futimesat
> >
> > These absolutely have to use something better than time_t
> > both in user space and in the kernel so we can deal with
> > old files. A lot of file systems need to be fixed as well so
> > we can actually store the times, regardless of whether we
> > are running a 32 or 64 bit kernel.
> 
> So these are the ones we have to worry about.
> It looks like they all involve I/O? Apart from the case of using block data
> from the buffer cache, the 64-bit operations should disappear in the
> actual I/O noise, right?

Right. Also there have been proposals for a better 'stat' replacement
for years, which would solve half of the interface problem for the
file system interfaces.

However, we also need to find a solution for category b), I only put
them into a different category above because we can treat them
differently in the kernel. For instance, we could use ktime_t for
the kernel code in category b) and a new struct timespec64 for
the times in struct inode.
On the user interface side, using timespec64 would be a reasonable
choice for both categories, because we already have two implementations
of all those syscalls in order to handle 32-on-64 compat tasks,
and we could use the same set of syscall implementations for time64-on-32.

Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Arnd Bergmann]

On Wednesday 14 May 2014 18:13:41 Ley Foon Tan wrote:
> On Wed, May 14, 2014 at 2:10 AM, Arnd Bergmann  wrote:
> > I've just spent two days looking at stuff that uses time_t inside
> > of the kernel, to get a better idea of what we actually need to
> > do to get provide new user interfaces for the existing architectures.
> >
> > My impression so far is that we're better off fixing it for the
> > existing architectures first and then using the new interfaces
> > exclusively on new ones, rather than changing over the ABI for
> > all new architectures at this point, which would likely create
> > yet another variant to maintain in the long run.
> >
> > Using 64-bit time_t on x32 is fine, because it's fast to operate
> > in user space with 64-bit registers, and the kernel is 64-bit
> > anyway. Inside of the kernel, we may get into trouble using
> > a 64-bit time_t on 32-bit architectures because of the overhead
> > in 64-bit math, e.g. all the timekeeping code that is based on
> > timespec or some code paths in file systems and network code where
> > we actually require division of time_t values.
> > We clearly have to change that code in some for to deal with y2038,
> > but 64-bit time_t may not be the best option. A lot of the
> > in-kernel code can probably use ktime_t, which we can change
> > to a different representation (e.g. 34 bit seconds) if needed,
> > and all the code that is only interested in relative time
> > (e.g. nanosleep) doesn't have to change at all.
> 
> Hi Arnd,
> 
> From your comment above, can I assume we don't need this patchset any more?

I won't require it, but it's not just my decision to make.

Let's see what Peter Anvin and Thomas Gleixner think about it, as they
have argued strongly in favor of using 64-bit time_t for new architectures
in the past.

Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Ley Foon Tan]

On Wed, May 14, 2014 at 2:10 AM, Arnd Bergmann  wrote:
> I've just spent two days looking at stuff that uses time_t inside
> of the kernel, to get a better idea of what we actually need to
> do to get provide new user interfaces for the existing architectures.
>
> My impression so far is that we're better off fixing it for the
> existing architectures first and then using the new interfaces
> exclusively on new ones, rather than changing over the ABI for
> all new architectures at this point, which would likely create
> yet another variant to maintain in the long run.
>
> Using 64-bit time_t on x32 is fine, because it's fast to operate
> in user space with 64-bit registers, and the kernel is 64-bit
> anyway. Inside of the kernel, we may get into trouble using
> a 64-bit time_t on 32-bit architectures because of the overhead
> in 64-bit math, e.g. all the timekeeping code that is based on
> timespec or some code paths in file systems and network code where
> we actually require division of time_t values.
> We clearly have to change that code in some for to deal with y2038,
> but 64-bit time_t may not be the best option. A lot of the
> in-kernel code can probably use ktime_t, which we can change
> to a different representation (e.g. 34 bit seconds) if needed,
> and all the code that is only interested in relative time
> (e.g. nanosleep) doesn't have to change at all.

Hi Arnd,

>From your comment above, can I assume we don't need this patchset any more?

Regards
Ley Foon
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Geert Uytterhoeven]

Hi Arnd,

On Tue, May 13, 2014 at 9:32 PM, Arnd Bergmann  wrote:
> I think we have three categories:

Thanks for the list!

> a) interfaces that uses relative time_t/timespec/timeval:
> b) interfaces that don't make sense for times in the past:

> c) interfaces that require absolute times:
>- stat/lstat/fstatat/
>- utime/utimes/futimesat
>
> These absolutely have to use something better than time_t
> both in user space and in the kernel so we can deal with
> old files. A lot of file systems need to be fixed as well so
> we can actually store the times, regardless of whether we
> are running a 32 or 64 bit kernel.

So these are the ones we have to worry about.
It looks like they all involve I/O? Apart from the case of using block data
from the buffer cache, the 64-bit operations should disappear in the
actual I/O noise, right?

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Arnd Bergmann]

On Tuesday 13 May 2014 20:24:59 Geert Uytterhoeven wrote:
> On Tue, May 13, 2014 at 8:10 PM, Arnd Bergmann  wrote:
> > Using 64-bit time_t on x32 is fine, because it's fast to operate
> > in user space with 64-bit registers, and the kernel is 64-bit
> > anyway. Inside of the kernel, we may get into trouble using
> > a 64-bit time_t on 32-bit architectures because of the overhead
> > in 64-bit math, e.g. all the timekeeping code that is based on
> > timespec or some code paths in file systems and network code where
> > we actually require division of time_t values.
> 
> While going over time_t uses, have you found a pattern for use cases
> involving division of time_t values in filesystem and networking code?

In ipv4, we have multiple places doing this:

icmp_param.data.times[1] = htonl((tv.tv_sec % 86400) * MSEC_PER_SEC +
 tv.tv_nsec / NSEC_PER_MSEC);

to calculate the miliseconds since midnight. For file systems, I
found that FAT uses seconds/minutes/hours/days/month/year representation,
which is a lot of divides, but that can probably be optimized and
we need to handle years beyond 2038 anyway.

> > We clearly have to change that code in some for to deal with y2038,
> > but 64-bit time_t may not be the best option. A lot of the
> > in-kernel code can probably use ktime_t, which we can change
> > to a different representation (e.g. 34 bit seconds) if needed,
> > and all the code that is only interested in relative time
> > (e.g. nanosleep) doesn't have to change at all.
> 
> Yeah. 32-bit uptimes should be good enough for everyone (don't quote
> me on that), so adding a 64-bit offset when there's a need for absolute
> time should be OK.

I think we have three categories:

a) interfaces that uses relative time_t/timespec/timeval:
   - nanosleep
   - select/pselect/poll/ppoll/epoll
   - getrusage
   - sched_rr_get_interval
   - sigtimedwait
   - clock_nanosleep
   - alarm
   - siginfo (rusage)

  These can stay compatible, but we'd have to use a different
  type if we change time_t.

b) interfaces that don't make sense for times in the past:
   - getitimer/setitimer
   - timer_settime/timer_gettime
   - gettimeofday/settimeofday
   - adjtimex
   - clock_gettime/clock_settime/clock_adjtime
   - time/stime
   - socket time stamps
   - audio time stamps
   - v4l time stamps
   - input event time stamps
   - sysv ipc (msg, sem, shm)

   Here, we are relatively free to change the start of the
   epoch in the kernel but convert to something else on the
   user space boundary. One possibility is to scale them to
   boot time and use ktime_t in the kernel.

c) interfaces that require absolute times:
   - stat/lstat/fstatat/
   - utime/utimes/futimesat

These absolutely have to use something better than time_t
both in user space and in the kernel so we can deal with
old files. A lot of file systems need to be fixed as well so
we can actually store the times, regardless of whether we
are running a 32 or 64 bit kernel.

Arnd 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Thomas Gleixner]

On Tue, 13 May 2014, Christoph Hellwig wrote:

> On Tue, May 13, 2014 at 05:33:01PM +0200, Thomas Gleixner wrote:
> > Err, we CANNOT do that. We cannot move ANY 32 bit architecture to
> > 64bit time_t without breaking the world and some more.
> 
> Of course we can, we just need to still provide the existing 32-bit
> time_t ABIs for backwards compatiblity.  That's how we handled
> extensions of other types like off_t in the past.

Sorry, misunderstood you there.

Sure, I agree that we need to fix it anyway plus all timespec/timeval
usage sites. It's on my and John Stultz radar as well. Just the sheer
amount of syscalls, ioctls etc. makes one shudder.

Thanks,

tglx
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Geert Uytterhoeven]

On Tue, May 13, 2014 at 8:10 PM, Arnd Bergmann  wrote:
> On Tuesday 13 May 2014 10:46:40 Christoph Hellwig wrote:
>> On Tue, May 13, 2014 at 05:33:01PM +0200, Thomas Gleixner wrote:
>> > and it
>> > would be dumb as hell to have new archs use time_t 32bit when we are
>> > currently twisting our brain around how to solve the y2038
>> > problem. Simply because we can not do the BSD flag day approach and
>> > change it.
>>
>> I don't think it's a good idea to have minor new architectures
>> pointlessly different than the major ones.  Especially given that we'll
>> absolutely have to fir the y2038 problem for 32-bit arm and probably
>> x86 anyway.
>
> I've just spent two days looking at stuff that uses time_t inside
> of the kernel, to get a better idea of what we actually need to
> do to get provide new user interfaces for the existing architectures.
>
> My impression so far is that we're better off fixing it for the
> existing architectures first and then using the new interfaces
> exclusively on new ones, rather than changing over the ABI for
> all new architectures at this point, which would likely create
> yet another variant to maintain in the long run.

Makes sense.

> Using 64-bit time_t on x32 is fine, because it's fast to operate
> in user space with 64-bit registers, and the kernel is 64-bit
> anyway. Inside of the kernel, we may get into trouble using
> a 64-bit time_t on 32-bit architectures because of the overhead
> in 64-bit math, e.g. all the timekeeping code that is based on
> timespec or some code paths in file systems and network code where
> we actually require division of time_t values.

While going over time_t uses, have you found a pattern for use cases
involving division of time_t values in filesystem and networking code?

> We clearly have to change that code in some for to deal with y2038,
> but 64-bit time_t may not be the best option. A lot of the
> in-kernel code can probably use ktime_t, which we can change
> to a different representation (e.g. 34 bit seconds) if needed,
> and all the code that is only interested in relative time
> (e.g. nanosleep) doesn't have to change at all.

Yeah. 32-bit uptimes should be good enough for everyone (don't quote
me on that), so adding a 64-bit offset when there's a need for absolute
time should be OK.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Geert Uytterhoeven]

On Tue, May 13, 2014 at 7:46 PM, Christoph Hellwig  wrote:
>> and it
>> would be dumb as hell to have new archs use time_t 32bit when we are
>> currently twisting our brain around how to solve the y2038
>> problem. Simply because we can not do the BSD flag day approach and
>> change it.
>
> I don't think it's a good idea to have minor new architectures
> pointlessly different than the major ones.  Especially given that we'll
> absolutely have to fir the y2038 problem for 32-bit arm and probably
> x86 anyway.

Considering someone just told me he plans to keep on using PPC
based MVME boards for 8 more years, you can expect some of today's
32-bit hardware to be around in 24 years...

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Arnd Bergmann]

On Tuesday 13 May 2014 10:46:40 Christoph Hellwig wrote:
> On Tue, May 13, 2014 at 05:33:01PM +0200, Thomas Gleixner wrote:
> > and it
> > would be dumb as hell to have new archs use time_t 32bit when we are
> > currently twisting our brain around how to solve the y2038
> > problem. Simply because we can not do the BSD flag day approach and
> > change it.
> 
> I don't think it's a good idea to have minor new architectures
> pointlessly different than the major ones.  Especially given that we'll
> absolutely have to fir the y2038 problem for 32-bit arm and probably
> x86 anyway.

I've just spent two days looking at stuff that uses time_t inside
of the kernel, to get a better idea of what we actually need to
do to get provide new user interfaces for the existing architectures.

My impression so far is that we're better off fixing it for the
existing architectures first and then using the new interfaces
exclusively on new ones, rather than changing over the ABI for
all new architectures at this point, which would likely create
yet another variant to maintain in the long run.

Using 64-bit time_t on x32 is fine, because it's fast to operate
in user space with 64-bit registers, and the kernel is 64-bit
anyway. Inside of the kernel, we may get into trouble using
a 64-bit time_t on 32-bit architectures because of the overhead
in 64-bit math, e.g. all the timekeeping code that is based on
timespec or some code paths in file systems and network code where
we actually require division of time_t values.
We clearly have to change that code in some for to deal with y2038,
but 64-bit time_t may not be the best option. A lot of the
in-kernel code can probably use ktime_t, which we can change
to a different representation (e.g. 34 bit seconds) if needed,
and all the code that is only interested in relative time
(e.g. nanosleep) doesn't have to change at all.

Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Christoph Hellwig]

On Tue, May 13, 2014 at 05:33:01PM +0200, Thomas Gleixner wrote:
> Err, we CANNOT do that. We cannot move ANY 32 bit architecture to
> 64bit time_t without breaking the world and some more.

Of course we can, we just need to still provide the existing 32-bit
time_t ABIs for backwards compatiblity.  That's how we handled
extensions of other types like off_t in the past.

> All 64bit architectures use 64bit time_t already plus the x32 ABI of x86.

64-bit is not relevant for the pain as there time_t is simply a long,
while for 32-bit types we move to a longer than pointer sized type,
in direct contradiction to older standards the explicitly prohibit this.

> So why would user space explode? It did not explode with x32

Who knows?  It's not like x32 actually has any significant user base.
In fact just weeks ago we fixed fundamental type system bugs for it.

> and it
> would be dumb as hell to have new archs use time_t 32bit when we are
> currently twisting our brain around how to solve the y2038
> problem. Simply because we can not do the BSD flag day approach and
> change it.

I don't think it's a good idea to have minor new architectures
pointlessly different than the major ones.  Especially given that we'll
absolutely have to fir the y2038 problem for 32-bit arm and probably
x86 anyway.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Christoph Hellwig]

On Tue, May 13, 2014 at 04:57:36PM +0800, Ley Foon Tan wrote:
> This patchset change default time_t and clock_t to 64 bit in
> include/uapi/asm-generic/posix_types.h. The existing 32 bit architectures 
> override
> these define to 32 bit in arch posix_types.h.
> 
> There is request to support 64 bit time_t for new architecture [1]. According 
> to the

I think this is an utterly wrong, and very dangerous approach.  A 64-bit
time_t is something that will need non-trivial porting effort in
userland, and introducing it only for new fringe architectures is a
guaranteed way to create silent breakage.

If you do care about it make sure the architectures that are heavily
used support it and userland can properly deal with it first, and only
then default new architectures to it.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Thomas Gleixner]

On Tue, 13 May 2014, Christoph Hellwig wrote:
> On Tue, May 13, 2014 at 04:57:36PM +0800, Ley Foon Tan wrote:
> > This patchset change default time_t and clock_t to 64 bit in
> > include/uapi/asm-generic/posix_types.h. The existing 32 bit architectures 
> > override
> > these define to 32 bit in arch posix_types.h.
> > 
> > There is request to support 64 bit time_t for new architecture [1]. 
> > According to the
> 
> I think this is an utterly wrong, and very dangerous approach.  A 64-bit
> time_t is something that will need non-trivial porting effort in
> userland, and introducing it only for new fringe architectures is a
> guaranteed way to create silent breakage.
> 
> If you do care about it make sure the architectures that are heavily
> used support it and userland can properly deal with it first, and only
> then default new architectures to it.

Err, we CANNOT do that. We cannot move ANY 32 bit architecture to
64bit time_t without breaking the world and some more.

All 64bit architectures use 64bit time_t already plus the x32 ABI of x86.

So why would user space explode? It did not explode with x32 and it
would be dumb as hell to have new archs use time_t 32bit when we are
currently twisting our brain around how to solve the y2038
problem. Simply because we can not do the BSD flag day approach and
change it.

User space needs to be audited anyway so there is nothing wrong if we
have a few more architectures aside of x32 using a 64bit time_t on a
32 bit machine.

Thanks,

tglx




--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Ley Foon Tan]

On Tue, May 13, 2014 at 5:10 PM, Geert Uytterhoeven
 wrote:
> Hi,
>
> On Tue, May 13, 2014 at 10:57 AM, Ley Foon Tan  wrote:
>> This patchset change default time_t and clock_t to 64 bit in
>> include/uapi/asm-generic/posix_types.h. The existing 32 bit architectures 
>> override
>> these define to 32 bit in arch posix_types.h.
>
> Thanks for your series!
>
>> Ley Foon Tan (25):
>>   asm-generic: Change time_t and clock_t to 64 bit
>
> For bisectability, this should be the last change, not the first.
Noted, will put this as last patch in next revision.
Thanks.

Regards
Ley Foon
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH 00/25] Change time_t and clock_t to 64 bit

[Geert Uytterhoeven]

Hi,

On Tue, May 13, 2014 at 10:57 AM, Ley Foon Tan  wrote:
> This patchset change default time_t and clock_t to 64 bit in
> include/uapi/asm-generic/posix_types.h. The existing 32 bit architectures 
> override
> these define to 32 bit in arch posix_types.h.

Thanks for your series!

> Ley Foon Tan (25):
>   asm-generic: Change time_t and clock_t to 64 bit

For bisectability, this should be the last change, not the first.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[PATCH 00/25] Change time_t and clock_t to 64 bit

[Ley Foon Tan]

This patchset change default time_t and clock_t to 64 bit in
include/uapi/asm-generic/posix_types.h. The existing 32 bit architectures 
override
these define to 32 bit in arch posix_types.h.

There is request to support 64 bit time_t for new architecture [1]. According 
to the
previous discussion we had (in [2]), defining suseconds_t as 64-bit, while 
probably
reasonable for LP64 or LP64+ILP32 architectures, but can makes things 
troublesome
for ILP32 architectures, not to mention the additional overhead.

This series is compiled tested on ARM, mips, powerpc32 and x86_64, but only boot
tested on ARM.

These patches are also available on this branch:
git://git.rocketboards.org/linux-socfpga-next.git  64bit_time

[1] https://lkml.org/lkml/2012/11/14/358
[2] https://lkml.org/lkml/2014/4/18/216

Ley Foon Tan (25):
  asm-generic: Change time_t and clock_t to 64 bit
  arc: Add 32 bit time_t and clock_t
  arm: Add 32 bit time_t and clock_t
  avr32: Add 32 bit time_t and clock_t
  blackfin: Add 32 bit time_t and clock_t
  c6x: Add 32 bit time_t and clock_t
  cris: Add 32 bit time_t and clock_t
  frv: Add 32 bit time_t and clock_t
  hexagon: Add 32 bit time_t and clock_t
  m32r: Add 32 bit time_t and clock_t
  m68k: Add 32 bit time_t and clock_t
  metag: Add 32 bit time_t and clock_t
  microblaze: Add 32 bit time_t and clock_t
  mips: Add 32 bit time_t and clock_t
  mn10300: Add 32 bit time_t and clock_t
  openrisc: Add 32 bit time_t and clock_t
  parisc: Add 32 bit time_t and clock_t
  powerpc: Add 32 bit time_t and clock_t
  s390: Add 32 bit time_t and clock_t
  score: Add 32 bit time_t and clock_t
  sh: Add 32 bit time_t and clock_t
  sparc: Add 32 bit time_t and clock_t
  unicore32: Add 32 bit time_t and clock_t
  x86: Add 32 bit time_t and clock_t
  xtense: Add 32 bit time_t and clock_t

 arch/arc/include/asm/Kbuild|  1 -
 arch/arc/include/uapi/asm/Kbuild   |  1 +
 arch/arc/include/uapi/asm/posix_types.h| 17 +
 arch/arm/include/uapi/asm/posix_types.h|  6 ++
 arch/avr32/include/uapi/asm/posix_types.h  |  6 ++
 arch/blackfin/include/uapi/asm/posix_types.h   |  6 ++
 arch/c6x/include/asm/Kbuild|  1 -
 arch/c6x/include/uapi/asm/Kbuild   |  1 +
 arch/c6x/include/uapi/asm/posix_types.h| 17 +
 arch/cris/include/uapi/asm/posix_types.h   |  6 ++
 arch/frv/include/uapi/asm/posix_types.h|  6 ++
 arch/hexagon/include/asm/Kbuild|  1 -
 arch/hexagon/include/uapi/asm/Kbuild   |  1 +
 arch/hexagon/include/uapi/asm/posix_types.h| 17 +
 arch/m32r/include/uapi/asm/posix_types.h   |  6 ++
 arch/m68k/include/uapi/asm/posix_types.h   |  6 ++
 arch/metag/include/asm/Kbuild  |  1 -
 arch/metag/include/uapi/asm/Kbuild |  1 +
 arch/metag/include/uapi/asm/posix_types.h  | 17 +
 arch/microblaze/include/uapi/asm/posix_types.h |  6 ++
 arch/mips/include/uapi/asm/posix_types.h   |  6 ++
 arch/mn10300/include/uapi/asm/posix_types.h|  6 ++
 arch/openrisc/include/asm/Kbuild   |  1 -
 arch/openrisc/include/uapi/asm/Kbuild  |  1 +
 arch/openrisc/include/uapi/asm/posix_types.h   | 17 +
 arch/parisc/include/uapi/asm/posix_types.h |  8 
 arch/powerpc/include/uapi/asm/posix_types.h|  6 ++
 arch/s390/include/uapi/asm/posix_types.h   |  6 ++
 arch/score/include/uapi/asm/posix_types.h  |  6 ++
 arch/sh/include/uapi/asm/posix_types_32.h  |  6 ++
 arch/sparc/include/uapi/asm/posix_types.h  |  6 ++
 arch/unicore32/include/asm/Kbuild  |  1 -
 arch/unicore32/include/uapi/asm/Kbuild |  1 +
 arch/unicore32/include/uapi/asm/posix_types.h  | 17 +
 arch/x86/include/uapi/asm/posix_types_32.h |  6 ++
 arch/xtensa/include/uapi/asm/posix_types.h |  6 ++
 include/uapi/asm-generic/posix_types.h | 14 --
 37 files changed, 230 insertions(+), 8 deletions(-)
 create mode 100644 arch/arc/include/uapi/asm/posix_types.h
 create mode 100644 arch/c6x/include/uapi/asm/posix_types.h
 create mode 100644 arch/hexagon/include/uapi/asm/posix_types.h
 create mode 100644 arch/metag/include/uapi/asm/posix_types.h
 create mode 100644 arch/openrisc/include/uapi/asm/posix_types.h
 create mode 100644 arch/unicore32/include/uapi/asm/posix_types.h

-- 
1.8.2.1

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/