Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
On Monday 16 February 2015 12:51:35 Rich Felker wrote:
> On Mon, Feb 16, 2015 at 06:20:18PM +0100, Arnd Bergmann wrote:
> > > Would it really be that hard to do:
> > >
> > > if (ILP32_on_64_process) tv_nsec = (int)tv_nsec;
> > >
> > > or similar? That's all that's needed.
> > >
> > > > In some cases, there may also be a measurable performance penalty
> > > > in interpreting a user space data structure manually over copying
> > > > it (including the timespec values) in one chunk.
> > >
> > > I don't think the above would be measurable.
> >
> > It depends: Copying the structure first and then doing the conversion
> > in kernel space on the specific members as you do in the example
> > should indeed have a trivial performance impact. However, it is also
> > the hardest for driver writers to get right, and it's better not to
> > trust them with corner cases like this.
> >
> > To make it more readable, we would probably introduce a helper function
> > that copies the timespec from user space memory to kernel space and
> > then does all the checks and conversions as required. However, doing
> > separate copies can (depending on the architecture) have a noticeable
> > impact. An example for this would be architectures that require setting
> > up a page table entry for the user space page in order to access the
> > data and then destroy it again afterwards, with the correct TLB flushes.
> >
> > We can do something like this for the old-style compat handlers that
> > use 32-bit time_t, but I'd prefer not to have it in the fast path for
> > the native 64-bit time_t on 64-bit architectures.
>
> I know this isn't the place to discuss large architectural kernel
> changes, but it would be really nice if the kernel had proper abstract
> knowledge, at syscall entry time, what regions of memory from
> userspace the syscall is going to need and a way of marshalling them
> all together as prep for enterring the code that implements the
> syscalls, and if conversion between different ABIs could take place
> mostly automatically at this layer. Perhaps this kind of thing is an
> idea that could be kept open for the future. I suspect the
> combinatorics of different legacy interfaces are going to continue
> getting worse, and it would be much nicer to have the support factored
> out of the actual syscall implementations.
Some subsystems (e.g. v4l) do this in their ioctls, and it sounds
like a nice idea, but as I count 17457 instances of copy_{to,from}_user
or {get,put}_user in the kernel, I believe we are basically stuck
with the current way.
> > > Generally I would think the kernel knows the model the process is
> > > using, but if not, all you need is separate ioctl numbers for
> > > userspace to use depending on which definition it's using.
> >
> > I've checked now, and indeed the kernel knows for ilp32 x86 and arm, since
> > it uses a different ELF interpreter. I thought it might be running the
> > ilp32 binaries as ELF64, but it does not.
>
> This would result in lots of problems like argv[], auxv[], envp[],
> etc. being in the wrong format.
Right. It depends a bit on the scope though: My impression is that
a lot of people who ask for ilp32 mode on arm64 are just interested
in getting to work one or two applications. If that was the only
goal, they could work around the problem (mostly) in user space, but
as it turns out, the kernel patch is doing the entire job of
implementing the new ABI at syscall level to the point where you
can (mostly) just recompile all of user space.
Arnd
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
On Mon, Feb 16, 2015 at 06:20:18PM +0100, Arnd Bergmann wrote: > > Would it really be that hard to do: > > > > if (ILP32_on_64_process) tv_nsec = (int)tv_nsec; > > > > or similar? That's all that's needed. > > > > > In some cases, there may also be a measurable performance penalty > > > in interpreting a user space data structure manually over copying > > > it (including the timespec values) in one chunk. > > > > I don't think the above would be measurable. > > It depends: Copying the structure first and then doing the conversion > in kernel space on the specific members as you do in the example > should indeed have a trivial performance impact. However, it is also > the hardest for driver writers to get right, and it's better not to > trust them with corner cases like this. > > To make it more readable, we would probably introduce a helper function > that copies the timespec from user space memory to kernel space and > then does all the checks and conversions as required. However, doing > separate copies can (depending on the architecture) have a noticeable > impact. An example for this would be architectures that require setting > up a page table entry for the user space page in order to access the > data and then destroy it again afterwards, with the correct TLB flushes. > > We can do something like this for the old-style compat handlers that > use 32-bit time_t, but I'd prefer not to have it in the fast path for > the native 64-bit time_t on 64-bit architectures. I know this isn't the place to discuss large architectural kernel changes, but it would be really nice if the kernel had proper abstract knowledge, at syscall entry time, what regions of memory from userspace the syscall is going to need and a way of marshalling them all together as prep for enterring the code that implements the syscalls, and if conversion between different ABIs could take place mostly automatically at this layer. Perhaps this kind of thing is an idea that could be kept open for the future. I suspect the combinatorics of different legacy interfaces are going to continue getting worse, and it would be much nicer to have the support factored out of the actual syscall implementations. > > Generally I would think the kernel knows the model the process is > > using, but if not, all you need is separate ioctl numbers for > > userspace to use depending on which definition it's using. > > I've checked now, and indeed the kernel knows for ilp32 x86 and arm, since > it uses a different ELF interpreter. I thought it might be running the > ilp32 binaries as ELF64, but it does not. This would result in lots of problems like argv[], auxv[], envp[], etc. being in the wrong format. > I would like to avoid separate ioctl command numbers, but we have to > do it for 64-bit time_t on the original 32-bit architectures in the > cases where the size is not already encoded in the command number. Indeed, I don't see any way around that. Rich -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wednesday 11 February 2015 16:37:58 Rich Felker wrote:
> On Wed, Feb 11, 2015 at 10:02:55PM +0100, a...@arndb.de wrote:
> > Rich Felker hat am 11. Februar 2015 um 21:12 geschrieben:
> > > On Wed, Feb 11, 2015 at 08:50:06PM +0100, a...@arndb.de wrote:
> > > > > > At least for AArch64 ILP32 we are still free to change the
> > > > > > user/kernel
> > > > > > ABI, so we could add wrappers for the affected syscalls to fix this
> > > > > > up.
> > > > > yes, afaik on x32 the 64bit kernel expects 64bit layout,
> > > > > arm64 can fix this
> > > >
> > > > We have to fix it on all 32-bit architectures when we move to 64-bit
> > > > time_t.
> > > >
> > > > I think ideally you'd want a user space definition like
> > > >
> > > > typedef long long time_t;
> > > > struct timespec {
> > > > time_t tv_sec;
> > > > long long tv_nsec;
> > > > };
> > > >
> > > > which is the only way to avoid passing uninitialized tv_nsec into the
> > > > kernel
> > > > from arbitrary user space doing ioctl. This is of course against POSIX
> > > > and
> > > > C99. Changing POSIX to allow it is probably easier than the C standard,
> > > > but we have a couple of years before we need to make this the default.
> > >
> > > I don't see why you want it to be long long. There is no harm in
> > > passing uninitialized padding to the kernel; the kernel just needs to
> > > do the right thing and ignore it (or avoid reading it to begin with).
> >
> > This would however mean having three different implementations
> > in the kernel rather than just two: Every driver that can pass a timespec
> > with this model needs to handle the native 64-bit case (64/64), the legacy
> > 32-bit case (32/32) and the y2038-safe case (64/32). Most code can
> > already handle the first two, and none today handles the third. If you
> > want to make the handling explicitly incompatible with native 64-bit
> > mode, you get a lot of untested code in obscure places that are never
> > tested properly, while using the normal behavior in the kernel at least
> > gives us the same bugs that we already have on native 64-bit systems.
>
> Would it really be that hard to do:
>
> if (ILP32_on_64_process) tv_nsec = (int)tv_nsec;
>
> or similar? That's all that's needed.
>
> > In some cases, there may also be a measurable performance penalty
> > in interpreting a user space data structure manually over copying
> > it (including the timespec values) in one chunk.
>
> I don't think the above would be measurable.
It depends: Copying the structure first and then doing the conversion
in kernel space on the specific members as you do in the example
should indeed have a trivial performance impact. However, it is also
the hardest for driver writers to get right, and it's better not to
trust them with corner cases like this.
To make it more readable, we would probably introduce a helper function
that copies the timespec from user space memory to kernel space and
then does all the checks and conversions as required. However, doing
separate copies can (depending on the architecture) have a noticeable
impact. An example for this would be architectures that require setting
up a page table entry for the user space page in order to access the
data and then destroy it again afterwards, with the correct TLB flushes.
We can do something like this for the old-style compat handlers that
use 32-bit time_t, but I'd prefer not to have it in the fast path for
the native 64-bit time_t on 64-bit architectures.
> > An alternative would be to change the native 64-bit case to ignore the upper
> > half of tv_nsec and always just copy the low bits. This should work
> > fine almost all of the time, but I fear that there might be corner cases
> > where existing 64-bit user space depends on passing large or negative
> > tv_nsec values into the kernel.
>
> Most functions using caller-provided timespecs are required to
> diagnose invalid forms with EINVAL when tv_nsec>=10 or <0, so
> if the kernel examines only the low 32 bits on ABIs where long is
> 64-bit, userspace would need to be responsible for doing this
> checking.
Right, that would not be good, in particular because we should not
change that for existing architectures.
> > > > In the kernel headers, the current plan is to provide interfaces taking
> > > > structures
> > > >
> > > > typedef long long __kernel_time64_t;
> > > > struct __kernel_timespec64_t {
> > > > __kernel_time64_t tv_sec;
> > > > long long tv_nsec;
> > > > };
> > > >
> > > > at least for ioctls, to avoid the ambiguity with libc headers specifying
> > > > something else.
> > >
> > > This seems hideous from an application standpoint. Application
> > > programmers don't want to know, and shouldn't need to know, these
> > > silly implementation details that make no sense except as historical
> > > baggage. They should just be able to use "struct timespec" everywhere
> > > and have it work.
> >
> > The kernel does not even know how timespec is defined by libc, and
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Monday 16 February 2015 10:38:18 Rich Felker wrote: > On Mon, Feb 16, 2015 at 03:40:54PM +0100, Arnd Bergmann wrote: > > On Friday 13 February 2015 13:37:07 Rich Felker wrote: > > > On Fri, Feb 13, 2015 at 05:33:46PM +, Catalin Marinas wrote: > > > > I think there is another problem with sign-extending tv_nsec in libc. > > > > The prototype for functions like clock_settime(2) take a const struct > > > > timespec *. There isn't anything to prevent such structure being in a > > > > read-only section, even though it is unlikely. So libc would have to > > > > duplicate the structure rather than just sign-extending tv_nsec in > > > > place. > > > > Do we actually need sign-extend, or does zero-extend have the exact > > same effect? For all I can tell, all invalid nanoseconds values > > remain invalid, and the accepted values are unchanged regardless > > of which type extension gets used. > > I think it matters for futimensat which has some special negative > codes you can store in tv_nsec, but perhaps there's an easy trick to > distinguish them even with zero extending. Ah, good point. > > > Yes, we already have to do this for x32 in musl. I'd rather not have > > > to do the same for aarch64-ILP32. > > > > This would of course be solved by using a 64-bit __kernel_snseconds_t > > or snseconds_t, and I suspect other libc implementations would just do > > that, when they are less strict about posix/c11 compliance compared > > to musl. > > I think they would be more strict if this were for a target that > actually sees use and they were getting bug reports from C programmers > annoyed that their code was not working correctly or not even > compiling. AFAIK there are no distros based on x32 now and it's > something of an alternate model on x86_64 distros that some people are > playing around with. I would expect to see much more build breakage and runtime problems from going to 64-bit time_t than from anything accessing the tv_nsec field. I'd also like to hear opinions from other libc maintainers on this. > > If you don't mind the (slight) distraction, can you describe what your > > plans are for handling 64-bit time_t on the existing 32-bit ABIs? > > I'm involved in both the efforts to do that and the ilp32 code on > > ARM, so it would be good for me to understand your plans for musl to > > get the bigger picture. Specifically, which of these do you plan > > to support (if you know already): > > It largely depends on if there's demand. If we have users who want to > run 32-bit systems with an ABI that will survive Y2038, it will be > supported, but as a new ABI for these targets. This will likely allow > fixing other ABI issues at the same time -- for example, on i386 I > would probably switch to mandating SSE2 for floating point, and > possibly using regparm everywhere. I see. Note that in case of i386, the main use case would be embedded systems, so while using regparm works, mandating anything that is not part of the quark soc (mmx, sse, cmov, ...) might be counterproductive. Regarding ARM, you can probably do it more modern though and require armv7-hardfloat, if you don't do that already. > There are a couple of different ways it could be done though: > > 1. On a per-arch basis, defining a new ABI variant for the arch. > > 2. With a new abstraction at the syscall boundary to get rid of all > kernel-arch-specific structures in userspace and redefine all types to > have plenty of room for growth. We currently plan to change the kernel for all 32-bit architectures to support the new ABI everywhere, in order to avoid special casing rarely used architectures, which would in turn be a potential source for bugs. We will only add system calls with 64-bit time_t that don't have a replacement already. So e.g. according to the current plan, there won't be a time(2) or gettimeofday(2) system call with 64-bit time_t in the kernel, but we expect the C library to implement these through clock_gettime(2). > In regards to your specific questions about ways it could be done: > > > - using 64-bit time_t on future arm32/i386/... kernels > > - using 64-bit time_t on existing arm32/i386/... kernels with native > > 32-bit time_t > > If the former is supported, I would think we'd want to support the > latter too. An ABI that only works on very-new kernels is very > restrictive in who can use it. Kernel support hardly matters (until > Y2038 actually arrives); the point of 64-bit time_t is to have an ABI > that's _ready_ for it so existing binaries can keep working. Ok. > > - using 32-bit time_t on future architectures that only support 64-bit > > time_t in the kernel > > Definitely will not be supported. Introducing a new ABI with 32-bit > time_t is a huge mistake, and the only reason it's been done for some > of the new targets musl supports is because the kernel does it, and > working around a mismatch between kernel and user time_t is a huge > problem -- all sorts of things, including for example
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Mon, Feb 16, 2015 at 03:40:54PM +0100, Arnd Bergmann wrote: > On Friday 13 February 2015 13:37:07 Rich Felker wrote: > > On Fri, Feb 13, 2015 at 05:33:46PM +, Catalin Marinas wrote: > > > > > > The data structure definition is a little bit fragile, as it > > > > > > depends on > > > > > > user space not using the __BIT_ENDIAN symbol in a conflicting way. > > > > > > So > > > > > > far we have managed to keep that outside of general purpose > > > > > > headers, but > > > > > > it should at least blow up in an obvious way if it does, rather than > > > > > > breaking silently. > > > > > > > > > > > > I still think it's more practical to keep the zeroing in user space > > > > > > though. > > > > > > In that case, we keep defining __kernel_timespec64 with a 'typedef > > > > > > long > > > > > > long __kernel_snseconds_t', and it's up to the libc to either use > > > > > > __kernel_timespec64 as its timespec, or to define a C11-compliant > > > > > > timespec itself and zero out the bits before passing the data to > > > > > > the kernel. > > > > > > > > > > The problem with doing this in user space is syscall(2). If we don't > > > > > allow it, then it's fine to do the padding in libc. > > > > > > > > It's already the case that callers have to tiptoe around syscall(2) > > > > usage on a per-arch basis for silly things like the convention for > > > > passing 64-bit arguments on 32-bit archs, different arg orders to work > > > > around 64-bit alignment and issues with too many args, and various > > > > legacy issues. > > Right. If one wants to use syscall(), they have to know exactly what the > kernel's calling conventions are, including knowing what the timespec > definition looks like, which could have a different size and padding > compared to the user space one. > > > > I think there is another problem with sign-extending tv_nsec in libc. > > > The prototype for functions like clock_settime(2) take a const struct > > > timespec *. There isn't anything to prevent such structure being in a > > > read-only section, even though it is unlikely. So libc would have to > > > duplicate the structure rather than just sign-extending tv_nsec in > > > place. > > Do we actually need sign-extend, or does zero-extend have the exact > same effect? For all I can tell, all invalid nanoseconds values > remain invalid, and the accepted values are unchanged regardless > of which type extension gets used. I think it matters for futimensat which has some special negative codes you can store in tv_nsec, but perhaps there's an easy trick to distinguish them even with zero extending. > > Yes, we already have to do this for x32 in musl. I'd rather not have > > to do the same for aarch64-ILP32. > > This would of course be solved by using a 64-bit __kernel_snseconds_t > or snseconds_t, and I suspect other libc implementations would just do > that, when they are less strict about posix/c11 compliance compared > to musl. I think they would be more strict if this were for a target that actually sees use and they were getting bug reports from C programmers annoyed that their code was not working correctly or not even compiling. AFAIK there are no distros based on x32 now and it's something of an alternate model on x86_64 distros that some people are playing around with. > If you don't mind the (slight) distraction, can you describe what your > plans are for handling 64-bit time_t on the existing 32-bit ABIs? > I'm involved in both the efforts to do that and the ilp32 code on > ARM, so it would be good for me to understand your plans for musl to > get the bigger picture. Specifically, which of these do you plan > to support (if you know already): It largely depends on if there's demand. If we have users who want to run 32-bit systems with an ABI that will survive Y2038, it will be supported, but as a new ABI for these targets. This will likely allow fixing other ABI issues at the same time -- for example, on i386 I would probably switch to mandating SSE2 for floating point, and possibly using regparm everywhere. There are a couple of different ways it could be done though: 1. On a per-arch basis, defining a new ABI variant for the arch. 2. With a new abstraction at the syscall boundary to get rid of all kernel-arch-specific structures in userspace and redefine all types to have plenty of room for growth. In regards to your specific questions about ways it could be done: > - using 64-bit time_t on future arm32/i386/... kernels > - using 64-bit time_t on existing arm32/i386/... kernels with native > 32-bit time_t If the former is supported, I would think we'd want to support the latter too. An ABI that only works on very-new kernels is very restrictive in who can use it. Kernel support hardly matters (until Y2038 actually arrives); the point of 64-bit time_t is to have an ABI that's _ready_ for it so existing binaries can keep working. > - using 32-bit time_t on future architectures that only support 64-bit >
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Friday 13 February 2015 13:37:07 Rich Felker wrote: > On Fri, Feb 13, 2015 at 05:33:46PM +, Catalin Marinas wrote: > > > > > The data structure definition is a little bit fragile, as it depends > > > > > on > > > > > user space not using the __BIT_ENDIAN symbol in a conflicting way. So > > > > > far we have managed to keep that outside of general purpose headers, > > > > > but > > > > > it should at least blow up in an obvious way if it does, rather than > > > > > breaking silently. > > > > > > > > > > I still think it's more practical to keep the zeroing in user space > > > > > though. > > > > > In that case, we keep defining __kernel_timespec64 with a 'typedef > > > > > long > > > > > long __kernel_snseconds_t', and it's up to the libc to either use > > > > > __kernel_timespec64 as its timespec, or to define a C11-compliant > > > > > timespec itself and zero out the bits before passing the data to the > > > > > kernel. > > > > > > > > The problem with doing this in user space is syscall(2). If we don't > > > > allow it, then it's fine to do the padding in libc. > > > > > > It's already the case that callers have to tiptoe around syscall(2) > > > usage on a per-arch basis for silly things like the convention for > > > passing 64-bit arguments on 32-bit archs, different arg orders to work > > > around 64-bit alignment and issues with too many args, and various > > > legacy issues. Right. If one wants to use syscall(), they have to know exactly what the kernel's calling conventions are, including knowing what the timespec definition looks like, which could have a different size and padding compared to the user space one. > > I think there is another problem with sign-extending tv_nsec in libc. > > The prototype for functions like clock_settime(2) take a const struct > > timespec *. There isn't anything to prevent such structure being in a > > read-only section, even though it is unlikely. So libc would have to > > duplicate the structure rather than just sign-extending tv_nsec in > > place. Do we actually need sign-extend, or does zero-extend have the exact same effect? For all I can tell, all invalid nanoseconds values remain invalid, and the accepted values are unchanged regardless of which type extension gets used. > Yes, we already have to do this for x32 in musl. I'd rather not have > to do the same for aarch64-ILP32. This would of course be solved by using a 64-bit __kernel_snseconds_t or snseconds_t, and I suspect other libc implementations would just do that, when they are less strict about posix/c11 compliance compared to musl. If you don't mind the (slight) distraction, can you describe what your plans are for handling 64-bit time_t on the existing 32-bit ABIs? I'm involved in both the efforts to do that and the ilp32 code on ARM, so it would be good for me to understand your plans for musl to get the bigger picture. Specifically, which of these do you plan to support (if you know already): - using 64-bit time_t on future arm32/i386/... kernels - using 64-bit time_t on existing arm32/i386/... kernels with native 32-bit time_t - using 32-bit time_t on future architectures that only support 64-bit time_t in the kernel - running existing binaries with 32-bit time_t on a library with 64-bit time_t support, using symbol versioning - compiling new code with 32-bit time_t against a library that supports both 32-bit and 64-bit time_t at runtime. - building a libc for existing architectures but without support for running existing 32-bit time_t applications. Arnd -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
On Mon, Feb 16, 2015 at 06:20:18PM +0100, Arnd Bergmann wrote: Would it really be that hard to do: if (ILP32_on_64_process) tv_nsec = (int)tv_nsec; or similar? That's all that's needed. In some cases, there may also be a measurable performance penalty in interpreting a user space data structure manually over copying it (including the timespec values) in one chunk. I don't think the above would be measurable. It depends: Copying the structure first and then doing the conversion in kernel space on the specific members as you do in the example should indeed have a trivial performance impact. However, it is also the hardest for driver writers to get right, and it's better not to trust them with corner cases like this. To make it more readable, we would probably introduce a helper function that copies the timespec from user space memory to kernel space and then does all the checks and conversions as required. However, doing separate copies can (depending on the architecture) have a noticeable impact. An example for this would be architectures that require setting up a page table entry for the user space page in order to access the data and then destroy it again afterwards, with the correct TLB flushes. We can do something like this for the old-style compat handlers that use 32-bit time_t, but I'd prefer not to have it in the fast path for the native 64-bit time_t on 64-bit architectures. I know this isn't the place to discuss large architectural kernel changes, but it would be really nice if the kernel had proper abstract knowledge, at syscall entry time, what regions of memory from userspace the syscall is going to need and a way of marshalling them all together as prep for enterring the code that implements the syscalls, and if conversion between different ABIs could take place mostly automatically at this layer. Perhaps this kind of thing is an idea that could be kept open for the future. I suspect the combinatorics of different legacy interfaces are going to continue getting worse, and it would be much nicer to have the support factored out of the actual syscall implementations. Generally I would think the kernel knows the model the process is using, but if not, all you need is separate ioctl numbers for userspace to use depending on which definition it's using. I've checked now, and indeed the kernel knows for ilp32 x86 and arm, since it uses a different ELF interpreter. I thought it might be running the ilp32 binaries as ELF64, but it does not. This would result in lots of problems like argv[], auxv[], envp[], etc. being in the wrong format. I would like to avoid separate ioctl command numbers, but we have to do it for 64-bit time_t on the original 32-bit architectures in the cases where the size is not already encoded in the command number. Indeed, I don't see any way around that. Rich -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wednesday 11 February 2015 16:37:58 Rich Felker wrote:
On Wed, Feb 11, 2015 at 10:02:55PM +0100, a...@arndb.de wrote:
Rich Felker dal...@libc.org hat am 11. Februar 2015 um 21:12 geschrieben:
On Wed, Feb 11, 2015 at 08:50:06PM +0100, a...@arndb.de wrote:
At least for AArch64 ILP32 we are still free to change the
user/kernel
ABI, so we could add wrappers for the affected syscalls to fix this
up.
yes, afaik on x32 the 64bit kernel expects 64bit layout,
arm64 can fix this
We have to fix it on all 32-bit architectures when we move to 64-bit
time_t.
I think ideally you'd want a user space definition like
typedef long long time_t;
struct timespec {
time_t tv_sec;
long long tv_nsec;
};
which is the only way to avoid passing uninitialized tv_nsec into the
kernel
from arbitrary user space doing ioctl. This is of course against POSIX
and
C99. Changing POSIX to allow it is probably easier than the C standard,
but we have a couple of years before we need to make this the default.
I don't see why you want it to be long long. There is no harm in
passing uninitialized padding to the kernel; the kernel just needs to
do the right thing and ignore it (or avoid reading it to begin with).
This would however mean having three different implementations
in the kernel rather than just two: Every driver that can pass a timespec
with this model needs to handle the native 64-bit case (64/64), the legacy
32-bit case (32/32) and the y2038-safe case (64/32). Most code can
already handle the first two, and none today handles the third. If you
want to make the handling explicitly incompatible with native 64-bit
mode, you get a lot of untested code in obscure places that are never
tested properly, while using the normal behavior in the kernel at least
gives us the same bugs that we already have on native 64-bit systems.
Would it really be that hard to do:
if (ILP32_on_64_process) tv_nsec = (int)tv_nsec;
or similar? That's all that's needed.
In some cases, there may also be a measurable performance penalty
in interpreting a user space data structure manually over copying
it (including the timespec values) in one chunk.
I don't think the above would be measurable.
It depends: Copying the structure first and then doing the conversion
in kernel space on the specific members as you do in the example
should indeed have a trivial performance impact. However, it is also
the hardest for driver writers to get right, and it's better not to
trust them with corner cases like this.
To make it more readable, we would probably introduce a helper function
that copies the timespec from user space memory to kernel space and
then does all the checks and conversions as required. However, doing
separate copies can (depending on the architecture) have a noticeable
impact. An example for this would be architectures that require setting
up a page table entry for the user space page in order to access the
data and then destroy it again afterwards, with the correct TLB flushes.
We can do something like this for the old-style compat handlers that
use 32-bit time_t, but I'd prefer not to have it in the fast path for
the native 64-bit time_t on 64-bit architectures.
An alternative would be to change the native 64-bit case to ignore the upper
half of tv_nsec and always just copy the low bits. This should work
fine almost all of the time, but I fear that there might be corner cases
where existing 64-bit user space depends on passing large or negative
tv_nsec values into the kernel.
Most functions using caller-provided timespecs are required to
diagnose invalid forms with EINVAL when tv_nsec=10 or 0, so
if the kernel examines only the low 32 bits on ABIs where long is
64-bit, userspace would need to be responsible for doing this
checking.
Right, that would not be good, in particular because we should not
change that for existing architectures.
In the kernel headers, the current plan is to provide interfaces taking
structures
typedef long long __kernel_time64_t;
struct __kernel_timespec64_t {
__kernel_time64_t tv_sec;
long long tv_nsec;
};
at least for ioctls, to avoid the ambiguity with libc headers specifying
something else.
This seems hideous from an application standpoint. Application
programmers don't want to know, and shouldn't need to know, these
silly implementation details that make no sense except as historical
baggage. They should just be able to use struct timespec everywhere
and have it work.
The kernel does not even know how timespec is defined by libc, and we have
to at least be able to handle the common cases of timespec being 32/32
and 64/64 (or 64/32 plus explicit padding). For system calls, we can rely
on libc calling the syscalls that match the definition (or convert the
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Monday 16 February 2015 10:38:18 Rich Felker wrote: On Mon, Feb 16, 2015 at 03:40:54PM +0100, Arnd Bergmann wrote: On Friday 13 February 2015 13:37:07 Rich Felker wrote: On Fri, Feb 13, 2015 at 05:33:46PM +, Catalin Marinas wrote: I think there is another problem with sign-extending tv_nsec in libc. The prototype for functions like clock_settime(2) take a const struct timespec *. There isn't anything to prevent such structure being in a read-only section, even though it is unlikely. So libc would have to duplicate the structure rather than just sign-extending tv_nsec in place. Do we actually need sign-extend, or does zero-extend have the exact same effect? For all I can tell, all invalid nanoseconds values remain invalid, and the accepted values are unchanged regardless of which type extension gets used. I think it matters for futimensat which has some special negative codes you can store in tv_nsec, but perhaps there's an easy trick to distinguish them even with zero extending. Ah, good point. Yes, we already have to do this for x32 in musl. I'd rather not have to do the same for aarch64-ILP32. This would of course be solved by using a 64-bit __kernel_snseconds_t or snseconds_t, and I suspect other libc implementations would just do that, when they are less strict about posix/c11 compliance compared to musl. I think they would be more strict if this were for a target that actually sees use and they were getting bug reports from C programmers annoyed that their code was not working correctly or not even compiling. AFAIK there are no distros based on x32 now and it's something of an alternate model on x86_64 distros that some people are playing around with. I would expect to see much more build breakage and runtime problems from going to 64-bit time_t than from anything accessing the tv_nsec field. I'd also like to hear opinions from other libc maintainers on this. If you don't mind the (slight) distraction, can you describe what your plans are for handling 64-bit time_t on the existing 32-bit ABIs? I'm involved in both the efforts to do that and the ilp32 code on ARM, so it would be good for me to understand your plans for musl to get the bigger picture. Specifically, which of these do you plan to support (if you know already): It largely depends on if there's demand. If we have users who want to run 32-bit systems with an ABI that will survive Y2038, it will be supported, but as a new ABI for these targets. This will likely allow fixing other ABI issues at the same time -- for example, on i386 I would probably switch to mandating SSE2 for floating point, and possibly using regparm everywhere. I see. Note that in case of i386, the main use case would be embedded systems, so while using regparm works, mandating anything that is not part of the quark soc (mmx, sse, cmov, ...) might be counterproductive. Regarding ARM, you can probably do it more modern though and require armv7-hardfloat, if you don't do that already. There are a couple of different ways it could be done though: 1. On a per-arch basis, defining a new ABI variant for the arch. 2. With a new abstraction at the syscall boundary to get rid of all kernel-arch-specific structures in userspace and redefine all types to have plenty of room for growth. We currently plan to change the kernel for all 32-bit architectures to support the new ABI everywhere, in order to avoid special casing rarely used architectures, which would in turn be a potential source for bugs. We will only add system calls with 64-bit time_t that don't have a replacement already. So e.g. according to the current plan, there won't be a time(2) or gettimeofday(2) system call with 64-bit time_t in the kernel, but we expect the C library to implement these through clock_gettime(2). In regards to your specific questions about ways it could be done: - using 64-bit time_t on future arm32/i386/... kernels - using 64-bit time_t on existing arm32/i386/... kernels with native 32-bit time_t If the former is supported, I would think we'd want to support the latter too. An ABI that only works on very-new kernels is very restrictive in who can use it. Kernel support hardly matters (until Y2038 actually arrives); the point of 64-bit time_t is to have an ABI that's _ready_ for it so existing binaries can keep working. Ok. - using 32-bit time_t on future architectures that only support 64-bit time_t in the kernel Definitely will not be supported. Introducing a new ABI with 32-bit time_t is a huge mistake, and the only reason it's been done for some of the new targets musl supports is because the kernel does it, and working around a mismatch between kernel and user time_t is a huge problem -- all sorts of things, including for example struct stat, depend on the time_t definition, and if you're going to allow mismatch with kernel you might as well go
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
On Monday 16 February 2015 12:51:35 Rich Felker wrote:
On Mon, Feb 16, 2015 at 06:20:18PM +0100, Arnd Bergmann wrote:
Would it really be that hard to do:
if (ILP32_on_64_process) tv_nsec = (int)tv_nsec;
or similar? That's all that's needed.
In some cases, there may also be a measurable performance penalty
in interpreting a user space data structure manually over copying
it (including the timespec values) in one chunk.
I don't think the above would be measurable.
It depends: Copying the structure first and then doing the conversion
in kernel space on the specific members as you do in the example
should indeed have a trivial performance impact. However, it is also
the hardest for driver writers to get right, and it's better not to
trust them with corner cases like this.
To make it more readable, we would probably introduce a helper function
that copies the timespec from user space memory to kernel space and
then does all the checks and conversions as required. However, doing
separate copies can (depending on the architecture) have a noticeable
impact. An example for this would be architectures that require setting
up a page table entry for the user space page in order to access the
data and then destroy it again afterwards, with the correct TLB flushes.
We can do something like this for the old-style compat handlers that
use 32-bit time_t, but I'd prefer not to have it in the fast path for
the native 64-bit time_t on 64-bit architectures.
I know this isn't the place to discuss large architectural kernel
changes, but it would be really nice if the kernel had proper abstract
knowledge, at syscall entry time, what regions of memory from
userspace the syscall is going to need and a way of marshalling them
all together as prep for enterring the code that implements the
syscalls, and if conversion between different ABIs could take place
mostly automatically at this layer. Perhaps this kind of thing is an
idea that could be kept open for the future. I suspect the
combinatorics of different legacy interfaces are going to continue
getting worse, and it would be much nicer to have the support factored
out of the actual syscall implementations.
Some subsystems (e.g. v4l) do this in their ioctls, and it sounds
like a nice idea, but as I count 17457 instances of copy_{to,from}_user
or {get,put}_user in the kernel, I believe we are basically stuck
with the current way.
Generally I would think the kernel knows the model the process is
using, but if not, all you need is separate ioctl numbers for
userspace to use depending on which definition it's using.
I've checked now, and indeed the kernel knows for ilp32 x86 and arm, since
it uses a different ELF interpreter. I thought it might be running the
ilp32 binaries as ELF64, but it does not.
This would result in lots of problems like argv[], auxv[], envp[],
etc. being in the wrong format.
Right. It depends a bit on the scope though: My impression is that
a lot of people who ask for ilp32 mode on arm64 are just interested
in getting to work one or two applications. If that was the only
goal, they could work around the problem (mostly) in user space, but
as it turns out, the kernel patch is doing the entire job of
implementing the new ABI at syscall level to the point where you
can (mostly) just recompile all of user space.
Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Friday 13 February 2015 13:37:07 Rich Felker wrote: On Fri, Feb 13, 2015 at 05:33:46PM +, Catalin Marinas wrote: The data structure definition is a little bit fragile, as it depends on user space not using the __BIT_ENDIAN symbol in a conflicting way. So far we have managed to keep that outside of general purpose headers, but it should at least blow up in an obvious way if it does, rather than breaking silently. I still think it's more practical to keep the zeroing in user space though. In that case, we keep defining __kernel_timespec64 with a 'typedef long long __kernel_snseconds_t', and it's up to the libc to either use __kernel_timespec64 as its timespec, or to define a C11-compliant timespec itself and zero out the bits before passing the data to the kernel. The problem with doing this in user space is syscall(2). If we don't allow it, then it's fine to do the padding in libc. It's already the case that callers have to tiptoe around syscall(2) usage on a per-arch basis for silly things like the convention for passing 64-bit arguments on 32-bit archs, different arg orders to work around 64-bit alignment and issues with too many args, and various legacy issues. Right. If one wants to use syscall(), they have to know exactly what the kernel's calling conventions are, including knowing what the timespec definition looks like, which could have a different size and padding compared to the user space one. I think there is another problem with sign-extending tv_nsec in libc. The prototype for functions like clock_settime(2) take a const struct timespec *. There isn't anything to prevent such structure being in a read-only section, even though it is unlikely. So libc would have to duplicate the structure rather than just sign-extending tv_nsec in place. Do we actually need sign-extend, or does zero-extend have the exact same effect? For all I can tell, all invalid nanoseconds values remain invalid, and the accepted values are unchanged regardless of which type extension gets used. Yes, we already have to do this for x32 in musl. I'd rather not have to do the same for aarch64-ILP32. This would of course be solved by using a 64-bit __kernel_snseconds_t or snseconds_t, and I suspect other libc implementations would just do that, when they are less strict about posix/c11 compliance compared to musl. If you don't mind the (slight) distraction, can you describe what your plans are for handling 64-bit time_t on the existing 32-bit ABIs? I'm involved in both the efforts to do that and the ilp32 code on ARM, so it would be good for me to understand your plans for musl to get the bigger picture. Specifically, which of these do you plan to support (if you know already): - using 64-bit time_t on future arm32/i386/... kernels - using 64-bit time_t on existing arm32/i386/... kernels with native 32-bit time_t - using 32-bit time_t on future architectures that only support 64-bit time_t in the kernel - running existing binaries with 32-bit time_t on a library with 64-bit time_t support, using symbol versioning - compiling new code with 32-bit time_t against a library that supports both 32-bit and 64-bit time_t at runtime. - building a libc for existing architectures but without support for running existing 32-bit time_t applications. Arnd -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Mon, Feb 16, 2015 at 03:40:54PM +0100, Arnd Bergmann wrote: On Friday 13 February 2015 13:37:07 Rich Felker wrote: On Fri, Feb 13, 2015 at 05:33:46PM +, Catalin Marinas wrote: The data structure definition is a little bit fragile, as it depends on user space not using the __BIT_ENDIAN symbol in a conflicting way. So far we have managed to keep that outside of general purpose headers, but it should at least blow up in an obvious way if it does, rather than breaking silently. I still think it's more practical to keep the zeroing in user space though. In that case, we keep defining __kernel_timespec64 with a 'typedef long long __kernel_snseconds_t', and it's up to the libc to either use __kernel_timespec64 as its timespec, or to define a C11-compliant timespec itself and zero out the bits before passing the data to the kernel. The problem with doing this in user space is syscall(2). If we don't allow it, then it's fine to do the padding in libc. It's already the case that callers have to tiptoe around syscall(2) usage on a per-arch basis for silly things like the convention for passing 64-bit arguments on 32-bit archs, different arg orders to work around 64-bit alignment and issues with too many args, and various legacy issues. Right. If one wants to use syscall(), they have to know exactly what the kernel's calling conventions are, including knowing what the timespec definition looks like, which could have a different size and padding compared to the user space one. I think there is another problem with sign-extending tv_nsec in libc. The prototype for functions like clock_settime(2) take a const struct timespec *. There isn't anything to prevent such structure being in a read-only section, even though it is unlikely. So libc would have to duplicate the structure rather than just sign-extending tv_nsec in place. Do we actually need sign-extend, or does zero-extend have the exact same effect? For all I can tell, all invalid nanoseconds values remain invalid, and the accepted values are unchanged regardless of which type extension gets used. I think it matters for futimensat which has some special negative codes you can store in tv_nsec, but perhaps there's an easy trick to distinguish them even with zero extending. Yes, we already have to do this for x32 in musl. I'd rather not have to do the same for aarch64-ILP32. This would of course be solved by using a 64-bit __kernel_snseconds_t or snseconds_t, and I suspect other libc implementations would just do that, when they are less strict about posix/c11 compliance compared to musl. I think they would be more strict if this were for a target that actually sees use and they were getting bug reports from C programmers annoyed that their code was not working correctly or not even compiling. AFAIK there are no distros based on x32 now and it's something of an alternate model on x86_64 distros that some people are playing around with. If you don't mind the (slight) distraction, can you describe what your plans are for handling 64-bit time_t on the existing 32-bit ABIs? I'm involved in both the efforts to do that and the ilp32 code on ARM, so it would be good for me to understand your plans for musl to get the bigger picture. Specifically, which of these do you plan to support (if you know already): It largely depends on if there's demand. If we have users who want to run 32-bit systems with an ABI that will survive Y2038, it will be supported, but as a new ABI for these targets. This will likely allow fixing other ABI issues at the same time -- for example, on i386 I would probably switch to mandating SSE2 for floating point, and possibly using regparm everywhere. There are a couple of different ways it could be done though: 1. On a per-arch basis, defining a new ABI variant for the arch. 2. With a new abstraction at the syscall boundary to get rid of all kernel-arch-specific structures in userspace and redefine all types to have plenty of room for growth. In regards to your specific questions about ways it could be done: - using 64-bit time_t on future arm32/i386/... kernels - using 64-bit time_t on existing arm32/i386/... kernels with native 32-bit time_t If the former is supported, I would think we'd want to support the latter too. An ABI that only works on very-new kernels is very restrictive in who can use it. Kernel support hardly matters (until Y2038 actually arrives); the point of 64-bit time_t is to have an ABI that's _ready_ for it so existing binaries can keep working. - using 32-bit time_t on future architectures that only support 64-bit time_t in the kernel Definitely will not be supported. Introducing a new ABI with 32-bit time_t is a huge mistake, and the only reason it's been done for some of the new targets musl supports is
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Fri, Feb 13, 2015 at 05:33:46PM +, Catalin Marinas wrote: > > > > The data structure definition is a little bit fragile, as it depends on > > > > user space not using the __BIT_ENDIAN symbol in a conflicting way. So > > > > far we have managed to keep that outside of general purpose headers, but > > > > it should at least blow up in an obvious way if it does, rather than > > > > breaking silently. > > > > > > > > I still think it's more practical to keep the zeroing in user space > > > > though. > > > > In that case, we keep defining __kernel_timespec64 with a 'typedef long > > > > long __kernel_snseconds_t', and it's up to the libc to either use > > > > __kernel_timespec64 as its timespec, or to define a C11-compliant > > > > timespec itself and zero out the bits before passing the data to the > > > > kernel. > > > > > > The problem with doing this in user space is syscall(2). If we don't > > > allow it, then it's fine to do the padding in libc. > > > > It's already the case that callers have to tiptoe around syscall(2) > > usage on a per-arch basis for silly things like the convention for > > passing 64-bit arguments on 32-bit archs, different arg orders to work > > around 64-bit alignment and issues with too many args, and various > > legacy issues. So I think manual use of syscall(2) is a less-critical > > issue, though of course from a libc perspective I would very much like > > for the kernel to handle it right. > > I think there is another problem with sign-extending tv_nsec in libc. > The prototype for functions like clock_settime(2) take a const struct > timespec *. There isn't anything to prevent such structure being in a > read-only section, even though it is unlikely. So libc would have to > duplicate the structure rather than just sign-extending tv_nsec in > place. Yes, we already have to do this for x32 in musl. I'd rather not have to do the same for aarch64-ILP32. Rich -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Fri, Feb 13, 2015 at 11:30:13AM -0500, Rich Felker wrote:
> On Fri, Feb 13, 2015 at 01:33:56PM +, Catalin Marinas wrote:
> > On Thu, Feb 12, 2015 at 07:59:24PM +0100, Arnd Bergmann wrote:
> > > Catalin Marinas hat am 12. Februar 2015 um 19:17
> > > geschrieben:
> > > > The solution (for new ports) could be similar to the other such
> > > > solutions in the compat layer. A kernel internal structure which is
> > > > binary-compatible with the ILP32 user one (as exported by the kernel):
> > > >
> > > > struct ilp32_timespec_kernel_internal_only {
> > > > __kernel_time_t tv_sec; /* seconds */
> > > > int tv_nsec; /* nanoseconds */
> > > > };
> > > >
> > > > and a syscall wrapper which converts between ilp32_timespec and timespec
> > > > (take compat_sys_clock_settime as an example).
> > >
> > > We then have to to this on all architectures, and not call it
> > > ilp32_timespec,
> > > but call it something else.
> > >
> > > I would much prefer to only have two versions of each syscall that takes a
> > > timespec rather than three versions, or having a version that behaves
> > > differently based on the type of program calling it. On native 32-bit
> > > systems, we should have the native syscall taking the 16-byte structure
> > > (using long long __kernel_time64_t)
> >
> > Can this also be 12 bytes in general if tv_nsec stays as 32-bit? The
> > size of such structure would be 16 bytes on ARM but I guess this depends
> > on long long the alignment requirements on specific architectures.
>
> The only archs with modern relevance I'm aware of where 64-bit types
> are not aligned are i386 and, by a regretable but hard-to-fix mistake,
> or1k. I don't have much opinion on whether the 64-bit-time_t timespec
> should be 12 bytes or 16 bytes on such archs. From my perspective it's
> a new ABI anyway so I'd like to be able to fix the 64-bit alignment
> issue at the same time, in which case the question would go away, but
> I'm sure others (glibc) will prefer a more transitional approach with
> symbol versioning or feature test macros or something.
The good thing about 16-byte timespec64 with appropriate (endianness
aware) struct padding is that the kernel can write tv_nsec to user as a
64-bit value (long on a 64-bit kernel). It's only the reading from user
that the 32-bit needs to be sign-extended into the kernel structure.
> > > In the kernel, it comes down to a function like
> > >
> > > int get_user_timespec64(struct timespec64 *ts, struct __kernel_timespec64
> > > __user
> > > *uts, bool task_32bit)
> > > {
> > >struct __kernel_timespec64 input;
> > >
> > >if (copy_from_user(, uts, sizeof(input))
> > > return -EFAULT;
> > >
> > >ts->tv_sec = input.tv_sec;
> > >if (task_32bit)
> > >ts->tv_nsec = (int)input.tv_nsec;
> > >else
> > >ts->tv_nsec = input.tv_nsec;
> > >
> > >return 0;
> > > }
> >
> > The only drawback is that native 64-bit and new 32-bit have the same
> > handling path, potentially slowing down the former (it may not be
> > noticeable).
>
> Offhand, I would not consider a single predictable branch on syscall
> entry or return to be noticable relative to general syscall overhead.
It's not just a check+branch but accessing some TIF flag which requires
reading the current_thread_info()->flags and testing it. It is probably
lost in the noise, unless you do such calls in loop where you may notice
a slight variation (it depends on the branch predictor as well; on some
architecture we may be able to make use of unlikely(task_32bit)).
> > > The data structure definition is a little bit fragile, as it depends on
> > > user space not using the __BIT_ENDIAN symbol in a conflicting way. So
> > > far we have managed to keep that outside of general purpose headers, but
> > > it should at least blow up in an obvious way if it does, rather than
> > > breaking silently.
> > >
> > > I still think it's more practical to keep the zeroing in user space
> > > though.
> > > In that case, we keep defining __kernel_timespec64 with a 'typedef long
> > > long __kernel_snseconds_t', and it's up to the libc to either use
> > > __kernel_timespec64 as its timespec, or to define a C11-compliant
> > > timespec itself and zero out the bits before passing the data to the
> > > kernel.
> >
> > The problem with doing this in user space is syscall(2). If we don't
> > allow it, then it's fine to do the padding in libc.
>
> It's already the case that callers have to tiptoe around syscall(2)
> usage on a per-arch basis for silly things like the convention for
> passing 64-bit arguments on 32-bit archs, different arg orders to work
> around 64-bit alignment and issues with too many args, and various
> legacy issues. So I think manual use of syscall(2) is a less-critical
> issue, though of course from a libc perspective I would very much like
> for the kernel to handle it right.
I think there is another problem with sign-extending
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Fri, Feb 13, 2015 at 01:33:56PM +, Catalin Marinas wrote:
> On Thu, Feb 12, 2015 at 07:59:24PM +0100, Arnd Bergmann wrote:
> > > Catalin Marinas hat am 12. Februar 2015 um 19:17
> > > geschrieben:
> > > On Wed, Feb 11, 2015 at 02:21:18PM -0500, Rich Felker wrote:
> > > > On Wed, Feb 11, 2015 at 05:39:19PM +, Catalin Marinas wrote:
> > > > > On Tue, Feb 10, 2015 at 06:13:02PM +, Rich Felker wrote:
> > > > > > I don't know if this has been discussed on libc-alpha yet or not,
> > > > > > but
> > > > > > I think we need to open a discussion of how it relates to open glibc
> > > > > > bug #16437, which presently applies only to x32 (ILP32 ABI on
> > > > > > x86_64):
> > > > > >
> > > > > > https://sourceware.org/bugzilla/show_bug.cgi?id=16437
> > > > >
> > > > > I'm trying to understand the problem first. Quoting from the bug above
> > > > > (which I guess is quoted form C11):
> > > > >
> > > > > "The range and precision of times representable in clock_t and time_t
> > > > > are implementation-defined. The timespec structure shall contain at
> > > > > least the following members, in any order.
> > > > >
> > > > > time_t tv_sec; // whole seconds -- >= 0
> > > > > long tv_nsec; // nanoseconds -- [0, 9]"
> > > > >
> > > > > So changing time_t to 64-bit is fine on x32. The timespec struct
> > > > > exported by the kernel always uses a long for tv_nsec. However, glibc
> > > > > uses
> > > > > __syscall_slong_t which ends up as 64-bit for x32 (I guess it mirrors
> > > > > the __kernel_long_t definition).
> > > > >
> > > > > So w.r.t. C11, the exported kernel timespec looks fine. But I think
> > > > > the
> > > > > x32 kernel support (and the current ILP32 patches) assume a native
> > > > > struct timespec with tv_nsec as 64-bit.
> > > >
> > > > The exported kernel timespec is not fine if long is defined as a
> > > > 32-bit type, which it is for x32 and the proposed aarch64-ILP32 ABIs.
> > >
> > > The exported kernel headers comply with POSIX as they use long for
> > > tv_nsec. The exported headers can be used in user space and with an
> > > ILP32 ABI, long is 32-bit. The problem is the syscall handler which uses
> > > the same structure in kernel where long is 64-bit. But this doesn't
> > > change the fact that the exported header was still correct from a user
> > > perspective.
> >
> > This is not ILP32 specific really, we need to add the same set of syscalls
> > for all 32-bit systems, in addition to the existing ones that take
> > a 32-bit time_t.
>
> We can look at this as two scenarios:
>
> 1. existing 32-bit user space with a 32-bit time_t
> 2. new 32-bit user space, potentially with 64-bit time_t
>
> For (1), we need an additional set of syscalls in parallel with the
> old ones and most likely a different structure, let's say timespec64.
>
> For (2), we could go for a 64-bit time_t in timespec directly, without
> any timespec64 and additional set of syscalls (though internally the
> kernel may handle them as timespec64).
>
> For compat support on a 64-bit kernel, we may need to support both
> 32-bit time_t via compat_timespec and a 64-bit time_t via a new
> compat_timespec64. In case of AArch64 ILP32, any timespec syscall should
> be routed directly to the corresponding compat_timespec64 handlers as we
> define a 64-bit time_t.
>
> For new 32-bit native architectures (no compat layer), we may want to
> enforce a 64-bit time_t from the beginning.
>
> Anyway, since AArch64 ILP32 does not have a legacy ABI with 32-bit
> time_t, we can start implementing it independently of the additional
> syscalls for 32-bit timespec64. Eventually, the same code path will be
> used for legacy 32-bit with the new 64-bit time_t syscalls.
This sounds right.
> > > The solution (for new ports) could be similar to the other such
> > > solutions in the compat layer. A kernel internal structure which is
> > > binary-compatible with the ILP32 user one (as exported by the kernel):
> > >
> > > struct ilp32_timespec_kernel_internal_only {
> > > __kernel_time_t tv_sec; /* seconds */
> > > int tv_nsec; /* nanoseconds */
> > > };
> > >
> > > and a syscall wrapper which converts between ilp32_timespec and timespec
> > > (take compat_sys_clock_settime as an example).
> >
> > We then have to to this on all architectures, and not call it
> > ilp32_timespec,
> > but call it something else.
> >
> > I would much prefer to only have two versions of each syscall that takes a
> > timespec rather than three versions, or having a version that behaves
> > differently based on the type of program calling it. On native 32-bit
> > systems, we should have the native syscall taking the 16-byte structure
> > (using long long __kernel_time64_t)
>
> Can this also be 12 bytes in general if tv_nsec stays as 32-bit? The
> size of such structure would be 16 bytes on ARM but I guess this depends
> on long long the alignment requirements on specific architectures.
The only archs with modern relevance I'm aware of where 64-bit
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Thu, Feb 12, 2015 at 07:59:24PM +0100, Arnd Bergmann wrote:
> > Catalin Marinas hat am 12. Februar 2015 um 19:17
> > geschrieben:
> > On Wed, Feb 11, 2015 at 02:21:18PM -0500, Rich Felker wrote:
> > > On Wed, Feb 11, 2015 at 05:39:19PM +, Catalin Marinas wrote:
> > > > On Tue, Feb 10, 2015 at 06:13:02PM +, Rich Felker wrote:
> > > > > I don't know if this has been discussed on libc-alpha yet or not, but
> > > > > I think we need to open a discussion of how it relates to open glibc
> > > > > bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64):
> > > > >
> > > > > https://sourceware.org/bugzilla/show_bug.cgi?id=16437
> > > >
> > > > I'm trying to understand the problem first. Quoting from the bug above
> > > > (which I guess is quoted form C11):
> > > >
> > > > "The range and precision of times representable in clock_t and time_t
> > > > are implementation-defined. The timespec structure shall contain at
> > > > least the following members, in any order.
> > > >
> > > > time_t tv_sec; // whole seconds -- >= 0
> > > > long tv_nsec; // nanoseconds -- [0, 9]"
> > > >
> > > > So changing time_t to 64-bit is fine on x32. The timespec struct
> > > > exported by the kernel always uses a long for tv_nsec. However, glibc
> > > > uses
> > > > __syscall_slong_t which ends up as 64-bit for x32 (I guess it mirrors
> > > > the __kernel_long_t definition).
> > > >
> > > > So w.r.t. C11, the exported kernel timespec looks fine. But I think the
> > > > x32 kernel support (and the current ILP32 patches) assume a native
> > > > struct timespec with tv_nsec as 64-bit.
> > >
> > > The exported kernel timespec is not fine if long is defined as a
> > > 32-bit type, which it is for x32 and the proposed aarch64-ILP32 ABIs.
> >
> > The exported kernel headers comply with POSIX as they use long for
> > tv_nsec. The exported headers can be used in user space and with an
> > ILP32 ABI, long is 32-bit. The problem is the syscall handler which uses
> > the same structure in kernel where long is 64-bit. But this doesn't
> > change the fact that the exported header was still correct from a user
> > perspective.
>
> This is not ILP32 specific really, we need to add the same set of syscalls
> for all 32-bit systems, in addition to the existing ones that take
> a 32-bit time_t.
We can look at this as two scenarios:
1. existing 32-bit user space with a 32-bit time_t
2. new 32-bit user space, potentially with 64-bit time_t
For (1), we need an additional set of syscalls in parallel with the
old ones and most likely a different structure, let's say timespec64.
For (2), we could go for a 64-bit time_t in timespec directly, without
any timespec64 and additional set of syscalls (though internally the
kernel may handle them as timespec64).
For compat support on a 64-bit kernel, we may need to support both
32-bit time_t via compat_timespec and a 64-bit time_t via a new
compat_timespec64. In case of AArch64 ILP32, any timespec syscall should
be routed directly to the corresponding compat_timespec64 handlers as we
define a 64-bit time_t.
For new 32-bit native architectures (no compat layer), we may want to
enforce a 64-bit time_t from the beginning.
Anyway, since AArch64 ILP32 does not have a legacy ABI with 32-bit
time_t, we can start implementing it independently of the additional
syscalls for 32-bit timespec64. Eventually, the same code path will be
used for legacy 32-bit with the new 64-bit time_t syscalls.
> > The solution (for new ports) could be similar to the other such
> > solutions in the compat layer. A kernel internal structure which is
> > binary-compatible with the ILP32 user one (as exported by the kernel):
> >
> > struct ilp32_timespec_kernel_internal_only {
> > __kernel_time_t tv_sec; /* seconds */
> > int tv_nsec; /* nanoseconds */
> > };
> >
> > and a syscall wrapper which converts between ilp32_timespec and timespec
> > (take compat_sys_clock_settime as an example).
>
> We then have to to this on all architectures, and not call it ilp32_timespec,
> but call it something else.
>
> I would much prefer to only have two versions of each syscall that takes a
> timespec rather than three versions, or having a version that behaves
> differently based on the type of program calling it. On native 32-bit
> systems, we should have the native syscall taking the 16-byte structure
> (using long long __kernel_time64_t)
Can this also be 12 bytes in general if tv_nsec stays as 32-bit? The
size of such structure would be 16 bytes on ARM but I guess this depends
on long long the alignment requirements on specific architectures.
> along with the compatibility syscall with a 8-byte structure for
> existing applications.
>
> On 64-bit systems, the same syscall source can be used for the normal 16-byte
> structure on native 64-bit tasks, ilp32 tasks (x32, aarch64-32), and future
> compat32 (i386, aarch32, ...) tasks, while the syscall for the 8-byte
> structure
> deals with legacy compat32 tasks that do
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Thu, Feb 12, 2015 at 07:59:24PM +0100, Arnd Bergmann wrote:
Catalin Marinas catalin.mari...@arm.com hat am 12. Februar 2015 um 19:17
geschrieben:
On Wed, Feb 11, 2015 at 02:21:18PM -0500, Rich Felker wrote:
On Wed, Feb 11, 2015 at 05:39:19PM +, Catalin Marinas wrote:
On Tue, Feb 10, 2015 at 06:13:02PM +, Rich Felker wrote:
I don't know if this has been discussed on libc-alpha yet or not, but
I think we need to open a discussion of how it relates to open glibc
bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64):
https://sourceware.org/bugzilla/show_bug.cgi?id=16437
I'm trying to understand the problem first. Quoting from the bug above
(which I guess is quoted form C11):
The range and precision of times representable in clock_t and time_t
are implementation-defined. The timespec structure shall contain at
least the following members, in any order.
time_t tv_sec; // whole seconds -- = 0
long tv_nsec; // nanoseconds -- [0, 9]
So changing time_t to 64-bit is fine on x32. The timespec struct
exported by the kernel always uses a long for tv_nsec. However, glibc
uses
__syscall_slong_t which ends up as 64-bit for x32 (I guess it mirrors
the __kernel_long_t definition).
So w.r.t. C11, the exported kernel timespec looks fine. But I think the
x32 kernel support (and the current ILP32 patches) assume a native
struct timespec with tv_nsec as 64-bit.
The exported kernel timespec is not fine if long is defined as a
32-bit type, which it is for x32 and the proposed aarch64-ILP32 ABIs.
The exported kernel headers comply with POSIX as they use long for
tv_nsec. The exported headers can be used in user space and with an
ILP32 ABI, long is 32-bit. The problem is the syscall handler which uses
the same structure in kernel where long is 64-bit. But this doesn't
change the fact that the exported header was still correct from a user
perspective.
This is not ILP32 specific really, we need to add the same set of syscalls
for all 32-bit systems, in addition to the existing ones that take
a 32-bit time_t.
We can look at this as two scenarios:
1. existing 32-bit user space with a 32-bit time_t
2. new 32-bit user space, potentially with 64-bit time_t
For (1), we need an additional set of syscalls in parallel with the
old ones and most likely a different structure, let's say timespec64.
For (2), we could go for a 64-bit time_t in timespec directly, without
any timespec64 and additional set of syscalls (though internally the
kernel may handle them as timespec64).
For compat support on a 64-bit kernel, we may need to support both
32-bit time_t via compat_timespec and a 64-bit time_t via a new
compat_timespec64. In case of AArch64 ILP32, any timespec syscall should
be routed directly to the corresponding compat_timespec64 handlers as we
define a 64-bit time_t.
For new 32-bit native architectures (no compat layer), we may want to
enforce a 64-bit time_t from the beginning.
Anyway, since AArch64 ILP32 does not have a legacy ABI with 32-bit
time_t, we can start implementing it independently of the additional
syscalls for 32-bit timespec64. Eventually, the same code path will be
used for legacy 32-bit with the new 64-bit time_t syscalls.
The solution (for new ports) could be similar to the other such
solutions in the compat layer. A kernel internal structure which is
binary-compatible with the ILP32 user one (as exported by the kernel):
struct ilp32_timespec_kernel_internal_only {
__kernel_time_t tv_sec; /* seconds */
int tv_nsec; /* nanoseconds */
};
and a syscall wrapper which converts between ilp32_timespec and timespec
(take compat_sys_clock_settime as an example).
We then have to to this on all architectures, and not call it ilp32_timespec,
but call it something else.
I would much prefer to only have two versions of each syscall that takes a
timespec rather than three versions, or having a version that behaves
differently based on the type of program calling it. On native 32-bit
systems, we should have the native syscall taking the 16-byte structure
(using long long __kernel_time64_t)
Can this also be 12 bytes in general if tv_nsec stays as 32-bit? The
size of such structure would be 16 bytes on ARM but I guess this depends
on long long the alignment requirements on specific architectures.
along with the compatibility syscall with a 8-byte structure for
existing applications.
On 64-bit systems, the same syscall source can be used for the normal 16-byte
structure on native 64-bit tasks, ilp32 tasks (x32, aarch64-32), and future
compat32 (i386, aarch32, ...) tasks, while the syscall for the 8-byte
structure
deals with legacy compat32 tasks that do not yet use __kernel_time64_t.
We could do with two syscalls but, as you said, we need some padding and
zeroing when the sizeof(time_t) != sizeof(long).
If the
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Fri, Feb 13, 2015 at 01:33:56PM +, Catalin Marinas wrote:
On Thu, Feb 12, 2015 at 07:59:24PM +0100, Arnd Bergmann wrote:
Catalin Marinas catalin.mari...@arm.com hat am 12. Februar 2015 um 19:17
geschrieben:
On Wed, Feb 11, 2015 at 02:21:18PM -0500, Rich Felker wrote:
On Wed, Feb 11, 2015 at 05:39:19PM +, Catalin Marinas wrote:
On Tue, Feb 10, 2015 at 06:13:02PM +, Rich Felker wrote:
I don't know if this has been discussed on libc-alpha yet or not,
but
I think we need to open a discussion of how it relates to open glibc
bug #16437, which presently applies only to x32 (ILP32 ABI on
x86_64):
https://sourceware.org/bugzilla/show_bug.cgi?id=16437
I'm trying to understand the problem first. Quoting from the bug above
(which I guess is quoted form C11):
The range and precision of times representable in clock_t and time_t
are implementation-defined. The timespec structure shall contain at
least the following members, in any order.
time_t tv_sec; // whole seconds -- = 0
long tv_nsec; // nanoseconds -- [0, 9]
So changing time_t to 64-bit is fine on x32. The timespec struct
exported by the kernel always uses a long for tv_nsec. However, glibc
uses
__syscall_slong_t which ends up as 64-bit for x32 (I guess it mirrors
the __kernel_long_t definition).
So w.r.t. C11, the exported kernel timespec looks fine. But I think
the
x32 kernel support (and the current ILP32 patches) assume a native
struct timespec with tv_nsec as 64-bit.
The exported kernel timespec is not fine if long is defined as a
32-bit type, which it is for x32 and the proposed aarch64-ILP32 ABIs.
The exported kernel headers comply with POSIX as they use long for
tv_nsec. The exported headers can be used in user space and with an
ILP32 ABI, long is 32-bit. The problem is the syscall handler which uses
the same structure in kernel where long is 64-bit. But this doesn't
change the fact that the exported header was still correct from a user
perspective.
This is not ILP32 specific really, we need to add the same set of syscalls
for all 32-bit systems, in addition to the existing ones that take
a 32-bit time_t.
We can look at this as two scenarios:
1. existing 32-bit user space with a 32-bit time_t
2. new 32-bit user space, potentially with 64-bit time_t
For (1), we need an additional set of syscalls in parallel with the
old ones and most likely a different structure, let's say timespec64.
For (2), we could go for a 64-bit time_t in timespec directly, without
any timespec64 and additional set of syscalls (though internally the
kernel may handle them as timespec64).
For compat support on a 64-bit kernel, we may need to support both
32-bit time_t via compat_timespec and a 64-bit time_t via a new
compat_timespec64. In case of AArch64 ILP32, any timespec syscall should
be routed directly to the corresponding compat_timespec64 handlers as we
define a 64-bit time_t.
For new 32-bit native architectures (no compat layer), we may want to
enforce a 64-bit time_t from the beginning.
Anyway, since AArch64 ILP32 does not have a legacy ABI with 32-bit
time_t, we can start implementing it independently of the additional
syscalls for 32-bit timespec64. Eventually, the same code path will be
used for legacy 32-bit with the new 64-bit time_t syscalls.
This sounds right.
The solution (for new ports) could be similar to the other such
solutions in the compat layer. A kernel internal structure which is
binary-compatible with the ILP32 user one (as exported by the kernel):
struct ilp32_timespec_kernel_internal_only {
__kernel_time_t tv_sec; /* seconds */
int tv_nsec; /* nanoseconds */
};
and a syscall wrapper which converts between ilp32_timespec and timespec
(take compat_sys_clock_settime as an example).
We then have to to this on all architectures, and not call it
ilp32_timespec,
but call it something else.
I would much prefer to only have two versions of each syscall that takes a
timespec rather than three versions, or having a version that behaves
differently based on the type of program calling it. On native 32-bit
systems, we should have the native syscall taking the 16-byte structure
(using long long __kernel_time64_t)
Can this also be 12 bytes in general if tv_nsec stays as 32-bit? The
size of such structure would be 16 bytes on ARM but I guess this depends
on long long the alignment requirements on specific architectures.
The only archs with modern relevance I'm aware of where 64-bit types
are not aligned are i386 and, by a regretable but hard-to-fix mistake,
or1k. I don't have much opinion on whether the 64-bit-time_t timespec
should be 12 bytes or 16 bytes on such archs. From my perspective it's
a new ABI anyway so I'd like to be able to fix the
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Fri, Feb 13, 2015 at 05:33:46PM +, Catalin Marinas wrote: The data structure definition is a little bit fragile, as it depends on user space not using the __BIT_ENDIAN symbol in a conflicting way. So far we have managed to keep that outside of general purpose headers, but it should at least blow up in an obvious way if it does, rather than breaking silently. I still think it's more practical to keep the zeroing in user space though. In that case, we keep defining __kernel_timespec64 with a 'typedef long long __kernel_snseconds_t', and it's up to the libc to either use __kernel_timespec64 as its timespec, or to define a C11-compliant timespec itself and zero out the bits before passing the data to the kernel. The problem with doing this in user space is syscall(2). If we don't allow it, then it's fine to do the padding in libc. It's already the case that callers have to tiptoe around syscall(2) usage on a per-arch basis for silly things like the convention for passing 64-bit arguments on 32-bit archs, different arg orders to work around 64-bit alignment and issues with too many args, and various legacy issues. So I think manual use of syscall(2) is a less-critical issue, though of course from a libc perspective I would very much like for the kernel to handle it right. I think there is another problem with sign-extending tv_nsec in libc. The prototype for functions like clock_settime(2) take a const struct timespec *. There isn't anything to prevent such structure being in a read-only section, even though it is unlikely. So libc would have to duplicate the structure rather than just sign-extending tv_nsec in place. Yes, we already have to do this for x32 in musl. I'd rather not have to do the same for aarch64-ILP32. Rich -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Fri, Feb 13, 2015 at 11:30:13AM -0500, Rich Felker wrote:
On Fri, Feb 13, 2015 at 01:33:56PM +, Catalin Marinas wrote:
On Thu, Feb 12, 2015 at 07:59:24PM +0100, Arnd Bergmann wrote:
Catalin Marinas catalin.mari...@arm.com hat am 12. Februar 2015 um 19:17
geschrieben:
The solution (for new ports) could be similar to the other such
solutions in the compat layer. A kernel internal structure which is
binary-compatible with the ILP32 user one (as exported by the kernel):
struct ilp32_timespec_kernel_internal_only {
__kernel_time_t tv_sec; /* seconds */
int tv_nsec; /* nanoseconds */
};
and a syscall wrapper which converts between ilp32_timespec and timespec
(take compat_sys_clock_settime as an example).
We then have to to this on all architectures, and not call it
ilp32_timespec,
but call it something else.
I would much prefer to only have two versions of each syscall that takes a
timespec rather than three versions, or having a version that behaves
differently based on the type of program calling it. On native 32-bit
systems, we should have the native syscall taking the 16-byte structure
(using long long __kernel_time64_t)
Can this also be 12 bytes in general if tv_nsec stays as 32-bit? The
size of such structure would be 16 bytes on ARM but I guess this depends
on long long the alignment requirements on specific architectures.
The only archs with modern relevance I'm aware of where 64-bit types
are not aligned are i386 and, by a regretable but hard-to-fix mistake,
or1k. I don't have much opinion on whether the 64-bit-time_t timespec
should be 12 bytes or 16 bytes on such archs. From my perspective it's
a new ABI anyway so I'd like to be able to fix the 64-bit alignment
issue at the same time, in which case the question would go away, but
I'm sure others (glibc) will prefer a more transitional approach with
symbol versioning or feature test macros or something.
The good thing about 16-byte timespec64 with appropriate (endianness
aware) struct padding is that the kernel can write tv_nsec to user as a
64-bit value (long on a 64-bit kernel). It's only the reading from user
that the 32-bit needs to be sign-extended into the kernel structure.
In the kernel, it comes down to a function like
int get_user_timespec64(struct timespec64 *ts, struct __kernel_timespec64
__user
*uts, bool task_32bit)
{
struct __kernel_timespec64 input;
if (copy_from_user(input, uts, sizeof(input))
return -EFAULT;
ts-tv_sec = input.tv_sec;
if (task_32bit)
ts-tv_nsec = (int)input.tv_nsec;
else
ts-tv_nsec = input.tv_nsec;
return 0;
}
The only drawback is that native 64-bit and new 32-bit have the same
handling path, potentially slowing down the former (it may not be
noticeable).
Offhand, I would not consider a single predictable branch on syscall
entry or return to be noticable relative to general syscall overhead.
It's not just a check+branch but accessing some TIF flag which requires
reading the current_thread_info()-flags and testing it. It is probably
lost in the noise, unless you do such calls in loop where you may notice
a slight variation (it depends on the branch predictor as well; on some
architecture we may be able to make use of unlikely(task_32bit)).
The data structure definition is a little bit fragile, as it depends on
user space not using the __BIT_ENDIAN symbol in a conflicting way. So
far we have managed to keep that outside of general purpose headers, but
it should at least blow up in an obvious way if it does, rather than
breaking silently.
I still think it's more practical to keep the zeroing in user space
though.
In that case, we keep defining __kernel_timespec64 with a 'typedef long
long __kernel_snseconds_t', and it's up to the libc to either use
__kernel_timespec64 as its timespec, or to define a C11-compliant
timespec itself and zero out the bits before passing the data to the
kernel.
The problem with doing this in user space is syscall(2). If we don't
allow it, then it's fine to do the padding in libc.
It's already the case that callers have to tiptoe around syscall(2)
usage on a per-arch basis for silly things like the convention for
passing 64-bit arguments on 32-bit archs, different arg orders to work
around 64-bit alignment and issues with too many args, and various
legacy issues. So I think manual use of syscall(2) is a less-critical
issue, though of course from a libc perspective I would very much like
for the kernel to handle it right.
I think there is another problem with sign-extending tv_nsec in libc.
The prototype for functions like clock_settime(2) take a const struct
timespec *. There isn't anything to prevent such structure being in a
read-only section, even though it is
Re: [PATCHv3 00/24] ILP32 support in ARM64
> Catalin Marinas hat am 12. Februar 2015 um 19:17
> geschrieben:
> On Wed, Feb 11, 2015 at 02:21:18PM -0500, Rich Felker wrote:
> > On Wed, Feb 11, 2015 at 05:39:19PM +, Catalin Marinas wrote:
> > > On Tue, Feb 10, 2015 at 06:13:02PM +, Rich Felker wrote:
> > > > I don't know if this has been discussed on libc-alpha yet or not, but
> > > > I think we need to open a discussion of how it relates to open glibc
> > > > bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64):
> > > >
> > > > https://sourceware.org/bugzilla/show_bug.cgi?id=16437
> > >
> > > I'm trying to understand the problem first. Quoting from the bug above
> > > (which I guess is quoted form C11):
> > >
> > > "The range and precision of times representable in clock_t and time_t
> > > are implementation-defined. The timespec structure shall contain at
> > > least the following members, in any order.
> > >
> > > time_t tv_sec; // whole seconds -- >= 0
> > > long tv_nsec; // nanoseconds -- [0, 9]"
> > >
> > > So changing time_t to 64-bit is fine on x32. The timespec struct
> > > exported by the kernel always uses a long for tv_nsec. However, glibc uses
> > > __syscall_slong_t which ends up as 64-bit for x32 (I guess it mirrors
> > > the __kernel_long_t definition).
> > >
> > > So w.r.t. C11, the exported kernel timespec looks fine. But I think the
> > > x32 kernel support (and the current ILP32 patches) assume a native
> > > struct timespec with tv_nsec as 64-bit.
> >
> > The exported kernel timespec is not fine if long is defined as a
> > 32-bit type, which it is for x32 and the proposed aarch64-ILP32 ABIs.
>
> The exported kernel headers comply with POSIX as they use long for
> tv_nsec. The exported headers can be used in user space and with an
> ILP32 ABI, long is 32-bit. The problem is the syscall handler which uses
> the same structure in kernel where long is 64-bit. But this doesn't
> change the fact that the exported header was still correct from a user
> perspective.
This is not ILP32 specific really, we need to add the same set of syscalls
for all 32-bit systems, in addition to the existing ones that take
a 32-bit time_t.
> The solution (for new ports) could be similar to the other such
> solutions in the compat layer. A kernel internal structure which is
> binary-compatible with the ILP32 user one (as exported by the kernel):
>
> struct ilp32_timespec_kernel_internal_only {
> __kernel_time_t tv_sec; /* seconds */
> int tv_nsec; /* nanoseconds */
> };
>
> and a syscall wrapper which converts between ilp32_timespec and timespec
> (take compat_sys_clock_settime as an example).
We then have to to this on all architectures, and not call it ilp32_timespec,
but call it something else.
I would much prefer to only have two versions of each syscall that takes a
timespec rather than three versions, or having a version that behaves
differently based on the type of program calling it. On native 32-bit
systems, we should have the native syscall taking the 16-byte structure
(using long long __kernel_time64_t) along with the compatibility syscall
with a 8-byte structure for existing applications.
On 64-bit systems, the same syscall source can be used for the normal 16-byte
structure on native 64-bit tasks, ilp32 tasks (x32, aarch64-32), and future
compat32 (i386, aarch32, ...) tasks, while the syscall for the 8-byte structure
deals with legacy compat32 tasks that do not yet use __kernel_time64_t.
> If the user structure has some padding (and as I've read in this thread
> it is allowed), it could be even easier for the kernel. The padding
> could be 32-bit before or after tv_nsec, depending on endianness.
The problem as pointed out before is that if you do this, 32-bit tasks
need to have the padding word zeroed at some stage for data passed into
the kernel, while 64-bit tasks need to return an error if the upper half
of the tv_nsec word is nonzero, at least for interfaces that are documented
to do this.
This can be done either in the kernel or in the libc. In the kernel, it
comes down to a function like
int get_user_timespec64(struct timespec64 *ts, struct __kernel_timespec64 __user
*uts, bool task_32bit)
{
struct __kernel_timespec64 input;
if (copy_from_user(, uts, sizeof(input))
return -EFAULT;
ts->tv_sec = input.tv_sec;
if (task_32bit)
ts->tv_nsec = (int)input.tv_nsec;
else
ts->tv_nsec = input.tv_nsec;
return 0;
}
with data types of
struct timespec64 {
time64_t tv_sec;
long tv_nsec;
};
struct __kernel_timespec64 {
__kernel_time64_t tv_nsec;
#if (__BYTE_ORDER == __BIG_ENDIAN) && (__BITS_PER_LONG == 32)
u32 __pad;
#endif
long tv_nsec;
#if (__BYTE_ORDER == __LITTLE_ENDIAN) && (__BITS_PER_LONG == 32)
u32 __pad;
#endif
};
The data structure definition is a little bit fragile, as it depends on
user space not using the __BIT_ENDIAN symbol in a conflicting way. So
far we have
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 02:21:18PM -0500, Rich Felker wrote:
> On Wed, Feb 11, 2015 at 05:39:19PM +, Catalin Marinas wrote:
> > On Tue, Feb 10, 2015 at 06:13:02PM +, Rich Felker wrote:
> > > I don't know if this has been discussed on libc-alpha yet or not, but
> > > I think we need to open a discussion of how it relates to open glibc
> > > bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64):
> > >
> > > https://sourceware.org/bugzilla/show_bug.cgi?id=16437
> >
> > I'm trying to understand the problem first. Quoting from the bug above
> > (which I guess is quoted form C11):
> >
> > "The range and precision of times representable in clock_t and time_t
> > are implementation-defined. The timespec structure shall contain at
> > least the following members, in any order.
> >
> > time_t tv_sec; // whole seconds -- >= 0
> > long tv_nsec; // nanoseconds -- [0, 9]"
> >
> > So changing time_t to 64-bit is fine on x32. The timespec struct
> > exported by the kernel always uses a long for tv_nsec. However, glibc uses
> > __syscall_slong_t which ends up as 64-bit for x32 (I guess it mirrors
> > the __kernel_long_t definition).
> >
> > So w.r.t. C11, the exported kernel timespec looks fine. But I think the
> > x32 kernel support (and the current ILP32 patches) assume a native
> > struct timespec with tv_nsec as 64-bit.
>
> The exported kernel timespec is not fine if long is defined as a
> 32-bit type, which it is for x32 and the proposed aarch64-ILP32 ABIs.
The exported kernel headers comply with POSIX as they use long for
tv_nsec. The exported headers can be used in user space and with an
ILP32 ABI, long is 32-bit. The problem is the syscall handler which uses
the same structure in kernel where long is 64-bit. But this doesn't
change the fact that the exported header was still correct from a user
perspective.
The solution (for new ports) could be similar to the other such
solutions in the compat layer. A kernel internal structure which is
binary-compatible with the ILP32 user one (as exported by the kernel):
struct ilp32_timespec_kernel_internal_only {
__kernel_time_t tv_sec; /* seconds */
int tv_nsec;/* nanoseconds */
};
and a syscall wrapper which converts between ilp32_timespec and timespec
(take compat_sys_clock_settime as an example).
If the user structure has some padding (and as I've read in this thread
it is allowed), it could be even easier for the kernel. The padding
could be 32-bit before or after tv_nsec, depending on endianness.
--
Catalin
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
On Thu, Feb 12, 2015 at 09:12:34AM +0100, Szabolcs Nagy wrote: > * Szabolcs Nagy [2015-02-11 20:05:37 +0100]: > > (i think this is also a problem if userspace code uses syscall(2) directly, > > libc cannot possibly know where to signextend and the kernel side does not > > do the fixup right now) > > nobody picked up this issue, is this resolved? > > ie. if userspace calls syscall(SYS_foo,...) directly with 32bit > longs does it always work out correctly on the kernel side? I think the only way to solve this is to have syscall wrappers in the kernel rather than glibc. > the sign extension is a problem for signed long arguments, > i only found these in the kernel: > > fs/buffer.c:SYSCALL_DEFINE2(bdflush, int, func, long, data) This is part of the deprecated syscalls, it is not used on new user ABIs. > fs/open.c:SYSCALL_DEFINE2(truncate, const char __user *, path, long, length) The kernel uses a long (64-bit) here and the user ABI defines this as an off_t. With x32, this should be a long long (__kernel_long_t), so not a problem. > fs/aio.c:SYSCALL_DEFINE3(io_submit, aio_context_t, ctx_id, long, nr, > fs/aio.c- struct iocb __user * __user *, iocbpp) > > fs/aio.c:SYSCALL_DEFINE5(io_getevents, aio_context_t, ctx_id, > fs/aio.c- long, min_nr, > fs/aio.c- long, nr, These would need some int->long conversion for nr, min_nr (it may be done in x32 glibc already but as you said it would not work via syscall() directly). > kernel/ptrace.c:SYSCALL_DEFINE4(ptrace, long, request, long, pid, unsigned > long, addr, > kernel/ptrace.c-unsigned long, data) The pid in user space would be pid_t which is 32-bit. The kernel seems to use it as pid_t afterwards, so looks safe. For addr and data, I guess it needs wrappers to zero the top part. > ipc/syscall.c:SYSCALL_DEFINE6(ipc, unsigned int, call, int, first, unsigned > long, second, > ipc/syscall.c- unsigned long, third, void __user *, ptr, long, fifth) ipc(2) shows the first, second, third as ints. I guess some kernel wrapper is needed here as well. -- Catalin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Thu, Feb 12, 2015 at 08:30:10AM -0800, H.J. Lu wrote: > On Thu, Feb 12, 2015 at 7:50 AM, Catalin Marinas > wrote: > > On Wed, Feb 11, 2015 at 12:15:56PM -0800, Andy Lutomirski wrote: > >> On 02/11/2015 11:57 AM, H.J. Lu wrote: > >> trivially satisfied if you consider x32 and x86_64 separate > >> compilation environments, but it's not related to the core issue: that > >> the definition of timespec violates core (not obscure) requirements of > >> both POSIX and C11. At the time you were probably unaware of the C11 > >> requirement. Note that it's a LOT harder to effect change in the C > >> standard, so even if the Austin Group would be amenable to changing > >> the requirement for timespec to allow something like nseconds_t, > >> getting WG14 to make this change to work around a Linux/glibc mistake > >> does not sound practical. > >> >>> > >> >>>That is very unfortunate. I consider it is too late for x32 to change. > >> >> > >> >>Why? It's hardly an incompatible ABI change, as long as the > >> >>kernel/libc fills the upper bits (for old programs that read them > >> >>based on the old headers) when structs are read from the kernel to the > >> >>application, and ignores the upper bits (potentially set or left > >> >>uninitialized by the application) when strings are passed from > >> >>userspace to the kernel. Newly built apps using the struct definition > >> >>with 32-bit tv_nsec would need new libc to ensure that the high bits > >> >>aren't interpreted, but this could be handled by symbol versioning. > >> >> > >> > > >> >We have considered this option. But since kernel wouldn't change > >> >tv_nsec/tv_usec handling just for x32, it wasn't selected. > >> > >> Did anyone *ask* the kernel people (e.g. hpa)? > > > > It seems so: > > > > https://lkml.org/lkml/2011/8/31/244 > > > > Couple of more replies from hpa: > > > > https://lkml.org/lkml/2011/8/31/261 > > https://lkml.org/lkml/2012/2/8/408 > > > > It looks like hpa was going to talk the POSIX committee but I don't know > > what the conclusion was and didn't follow the thread (at the time I > > wasn't interested in ARM ILP32). > > Just for the record, tv_nsec/tv_usec can be changed to long > as long as kernel always read them as 32 bits and write them > as 64 bits for both LP64 and ILP32 in 64-bit imespec amd timeval. No; currently userspace relies on the kernel to produce EINVAL when tv_nsec is not in the range [0,9]. If the kernel just reads it as 32-bit unconditionally, tv_nsec=0x1 would fail to produce EINVAL in LP64 models where tv_nsec is a 64-bit object in userspace. > In glibc, they can be changed to long without breaking existing binaries. This is true only if glibc or the kernel ignores the upper bits. Otherwise, programs could end up passing junk that glibc and/or the kernel interprets. > For x86-32, 64-bit __time_t must be 64-bit aligned. Otherwise, there will > be no padding in 64-bit timespec nor timeval. Just adding an explicit padding member when long is 32-bit would be cleaner. This makes it possible to manually set/clear/inspect the bits without memset. I don't see any reason to require actual alignment of the struct on x86-32 unless you're going with a whole new ABI where 64-bit types are aligned. Of course if we're thinking about making 64-bit time_t on 32-bit archs, that's an incompatible ABI already and would be a great time to make lots of other ABI fixes... But I wonder if anyone is going to care about actual x86-32 hardware as Y2038 approaches. Rich -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Thu, Feb 12, 2015 at 7:50 AM, Catalin Marinas wrote: > On Wed, Feb 11, 2015 at 12:15:56PM -0800, Andy Lutomirski wrote: >> On 02/11/2015 11:57 AM, H.J. Lu wrote: >> trivially satisfied if you consider x32 and x86_64 separate >> compilation environments, but it's not related to the core issue: that >> the definition of timespec violates core (not obscure) requirements of >> both POSIX and C11. At the time you were probably unaware of the C11 >> requirement. Note that it's a LOT harder to effect change in the C >> standard, so even if the Austin Group would be amenable to changing >> the requirement for timespec to allow something like nseconds_t, >> getting WG14 to make this change to work around a Linux/glibc mistake >> does not sound practical. >> >>> >> >>>That is very unfortunate. I consider it is too late for x32 to change. >> >> >> >>Why? It's hardly an incompatible ABI change, as long as the >> >>kernel/libc fills the upper bits (for old programs that read them >> >>based on the old headers) when structs are read from the kernel to the >> >>application, and ignores the upper bits (potentially set or left >> >>uninitialized by the application) when strings are passed from >> >>userspace to the kernel. Newly built apps using the struct definition >> >>with 32-bit tv_nsec would need new libc to ensure that the high bits >> >>aren't interpreted, but this could be handled by symbol versioning. >> >> >> > >> >We have considered this option. But since kernel wouldn't change >> >tv_nsec/tv_usec handling just for x32, it wasn't selected. >> >> Did anyone *ask* the kernel people (e.g. hpa)? > > It seems so: > > https://lkml.org/lkml/2011/8/31/244 > > Couple of more replies from hpa: > > https://lkml.org/lkml/2011/8/31/261 > https://lkml.org/lkml/2012/2/8/408 > > It looks like hpa was going to talk the POSIX committee but I don't know > what the conclusion was and didn't follow the thread (at the time I > wasn't interested in ARM ILP32). Just for the record, tv_nsec/tv_usec can be changed to long as long as kernel always read them as 32 bits and write them as 64 bits for both LP64 and ILP32 in 64-bit imespec amd timeval. In glibc, they can be changed to long without breaking existing binaries. For x86-32, 64-bit __time_t must be 64-bit aligned. Otherwise, there will be no padding in 64-bit timespec nor timeval. -- H.J. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Thu, Feb 12, 2015 at 03:50:24PM +, Catalin Marinas wrote: > On Wed, Feb 11, 2015 at 12:15:56PM -0800, Andy Lutomirski wrote: > > On 02/11/2015 11:57 AM, H.J. Lu wrote: > > trivially satisfied if you consider x32 and x86_64 separate > > compilation environments, but it's not related to the core issue: that > > the definition of timespec violates core (not obscure) requirements of > > both POSIX and C11. At the time you were probably unaware of the C11 > > requirement. Note that it's a LOT harder to effect change in the C > > standard, so even if the Austin Group would be amenable to changing > > the requirement for timespec to allow something like nseconds_t, > > getting WG14 to make this change to work around a Linux/glibc mistake > > does not sound practical. > > >>> > > >>>That is very unfortunate. I consider it is too late for x32 to change. > > >> > > >>Why? It's hardly an incompatible ABI change, as long as the > > >>kernel/libc fills the upper bits (for old programs that read them > > >>based on the old headers) when structs are read from the kernel to the > > >>application, and ignores the upper bits (potentially set or left > > >>uninitialized by the application) when strings are passed from > > >>userspace to the kernel. Newly built apps using the struct definition > > >>with 32-bit tv_nsec would need new libc to ensure that the high bits > > >>aren't interpreted, but this could be handled by symbol versioning. > > >> > > > > > >We have considered this option. But since kernel wouldn't change > > >tv_nsec/tv_usec handling just for x32, it wasn't selected. > > > > Did anyone *ask* the kernel people (e.g. hpa)? > > It seems so: > > https://lkml.org/lkml/2011/8/31/244 > > Couple of more replies from hpa: > > https://lkml.org/lkml/2011/8/31/261 > https://lkml.org/lkml/2012/2/8/408 > > It looks like hpa was going to talk the POSIX committee but I don't know > what the conclusion was and didn't follow the thread (at the time I > wasn't interested in ARM ILP32). At this point POSIX committee is not sufficient. ISO C specifies timespec now, and as Jens Gustedt mentioned (I don't think his reply made it to the whole CC list; see the musl list archive here: http://www.openwall.com/lists/musl/2015/02/11/21 ), it seems unlikely that one could pose a convincing argument for this requirement to be changed in the C language. Rich -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 12:15:56PM -0800, Andy Lutomirski wrote: > On 02/11/2015 11:57 AM, H.J. Lu wrote: > trivially satisfied if you consider x32 and x86_64 separate > compilation environments, but it's not related to the core issue: that > the definition of timespec violates core (not obscure) requirements of > both POSIX and C11. At the time you were probably unaware of the C11 > requirement. Note that it's a LOT harder to effect change in the C > standard, so even if the Austin Group would be amenable to changing > the requirement for timespec to allow something like nseconds_t, > getting WG14 to make this change to work around a Linux/glibc mistake > does not sound practical. > >>> > >>>That is very unfortunate. I consider it is too late for x32 to change. > >> > >>Why? It's hardly an incompatible ABI change, as long as the > >>kernel/libc fills the upper bits (for old programs that read them > >>based on the old headers) when structs are read from the kernel to the > >>application, and ignores the upper bits (potentially set or left > >>uninitialized by the application) when strings are passed from > >>userspace to the kernel. Newly built apps using the struct definition > >>with 32-bit tv_nsec would need new libc to ensure that the high bits > >>aren't interpreted, but this could be handled by symbol versioning. > >> > > > >We have considered this option. But since kernel wouldn't change > >tv_nsec/tv_usec handling just for x32, it wasn't selected. > > Did anyone *ask* the kernel people (e.g. hpa)? It seems so: https://lkml.org/lkml/2011/8/31/244 Couple of more replies from hpa: https://lkml.org/lkml/2011/8/31/261 https://lkml.org/lkml/2012/2/8/408 It looks like hpa was going to talk the POSIX committee but I don't know what the conclusion was and didn't follow the thread (at the time I wasn't interested in ARM ILP32). -- Catalin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
* Szabolcs Nagy [2015-02-11 20:05:37 +0100]: > > (i think this is also a problem if userspace code uses syscall(2) directly, > libc cannot possibly know where to signextend and the kernel side does not > do the fixup right now) > nobody picked up this issue, is this resolved? ie. if userspace calls syscall(SYS_foo,...) directly with 32bit longs does it always work out correctly on the kernel side? the sign extension is a problem for signed long arguments, i only found these in the kernel: fs/buffer.c:SYSCALL_DEFINE2(bdflush, int, func, long, data) fs/open.c:SYSCALL_DEFINE2(truncate, const char __user *, path, long, length) fs/aio.c:SYSCALL_DEFINE3(io_submit, aio_context_t, ctx_id, long, nr, fs/aio.c- struct iocb __user * __user *, iocbpp) fs/aio.c:SYSCALL_DEFINE5(io_getevents, aio_context_t, ctx_id, fs/aio.c- long, min_nr, fs/aio.c- long, nr, kernel/ptrace.c:SYSCALL_DEFINE4(ptrace, long, request, long, pid, unsigned long, addr, kernel/ptrace.c-unsigned long, data) ipc/syscall.c:SYSCALL_DEFINE6(ipc, unsigned int, call, int, first, unsigned long, second, ipc/syscall.c- unsigned long, third, void __user *, ptr, long, fifth) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 02:21:18PM -0500, Rich Felker wrote:
On Wed, Feb 11, 2015 at 05:39:19PM +, Catalin Marinas wrote:
On Tue, Feb 10, 2015 at 06:13:02PM +, Rich Felker wrote:
I don't know if this has been discussed on libc-alpha yet or not, but
I think we need to open a discussion of how it relates to open glibc
bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64):
https://sourceware.org/bugzilla/show_bug.cgi?id=16437
I'm trying to understand the problem first. Quoting from the bug above
(which I guess is quoted form C11):
The range and precision of times representable in clock_t and time_t
are implementation-defined. The timespec structure shall contain at
least the following members, in any order.
time_t tv_sec; // whole seconds -- = 0
long tv_nsec; // nanoseconds -- [0, 9]
So changing time_t to 64-bit is fine on x32. The timespec struct
exported by the kernel always uses a long for tv_nsec. However, glibc uses
__syscall_slong_t which ends up as 64-bit for x32 (I guess it mirrors
the __kernel_long_t definition).
So w.r.t. C11, the exported kernel timespec looks fine. But I think the
x32 kernel support (and the current ILP32 patches) assume a native
struct timespec with tv_nsec as 64-bit.
The exported kernel timespec is not fine if long is defined as a
32-bit type, which it is for x32 and the proposed aarch64-ILP32 ABIs.
The exported kernel headers comply with POSIX as they use long for
tv_nsec. The exported headers can be used in user space and with an
ILP32 ABI, long is 32-bit. The problem is the syscall handler which uses
the same structure in kernel where long is 64-bit. But this doesn't
change the fact that the exported header was still correct from a user
perspective.
The solution (for new ports) could be similar to the other such
solutions in the compat layer. A kernel internal structure which is
binary-compatible with the ILP32 user one (as exported by the kernel):
struct ilp32_timespec_kernel_internal_only {
__kernel_time_t tv_sec; /* seconds */
int tv_nsec;/* nanoseconds */
};
and a syscall wrapper which converts between ilp32_timespec and timespec
(take compat_sys_clock_settime as an example).
If the user structure has some padding (and as I've read in this thread
it is allowed), it could be even easier for the kernel. The padding
could be 32-bit before or after tv_nsec, depending on endianness.
--
Catalin
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
Catalin Marinas catalin.mari...@arm.com hat am 12. Februar 2015 um 19:17
geschrieben:
On Wed, Feb 11, 2015 at 02:21:18PM -0500, Rich Felker wrote:
On Wed, Feb 11, 2015 at 05:39:19PM +, Catalin Marinas wrote:
On Tue, Feb 10, 2015 at 06:13:02PM +, Rich Felker wrote:
I don't know if this has been discussed on libc-alpha yet or not, but
I think we need to open a discussion of how it relates to open glibc
bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64):
https://sourceware.org/bugzilla/show_bug.cgi?id=16437
I'm trying to understand the problem first. Quoting from the bug above
(which I guess is quoted form C11):
The range and precision of times representable in clock_t and time_t
are implementation-defined. The timespec structure shall contain at
least the following members, in any order.
time_t tv_sec; // whole seconds -- = 0
long tv_nsec; // nanoseconds -- [0, 9]
So changing time_t to 64-bit is fine on x32. The timespec struct
exported by the kernel always uses a long for tv_nsec. However, glibc uses
__syscall_slong_t which ends up as 64-bit for x32 (I guess it mirrors
the __kernel_long_t definition).
So w.r.t. C11, the exported kernel timespec looks fine. But I think the
x32 kernel support (and the current ILP32 patches) assume a native
struct timespec with tv_nsec as 64-bit.
The exported kernel timespec is not fine if long is defined as a
32-bit type, which it is for x32 and the proposed aarch64-ILP32 ABIs.
The exported kernel headers comply with POSIX as they use long for
tv_nsec. The exported headers can be used in user space and with an
ILP32 ABI, long is 32-bit. The problem is the syscall handler which uses
the same structure in kernel where long is 64-bit. But this doesn't
change the fact that the exported header was still correct from a user
perspective.
This is not ILP32 specific really, we need to add the same set of syscalls
for all 32-bit systems, in addition to the existing ones that take
a 32-bit time_t.
The solution (for new ports) could be similar to the other such
solutions in the compat layer. A kernel internal structure which is
binary-compatible with the ILP32 user one (as exported by the kernel):
struct ilp32_timespec_kernel_internal_only {
__kernel_time_t tv_sec; /* seconds */
int tv_nsec; /* nanoseconds */
};
and a syscall wrapper which converts between ilp32_timespec and timespec
(take compat_sys_clock_settime as an example).
We then have to to this on all architectures, and not call it ilp32_timespec,
but call it something else.
I would much prefer to only have two versions of each syscall that takes a
timespec rather than three versions, or having a version that behaves
differently based on the type of program calling it. On native 32-bit
systems, we should have the native syscall taking the 16-byte structure
(using long long __kernel_time64_t) along with the compatibility syscall
with a 8-byte structure for existing applications.
On 64-bit systems, the same syscall source can be used for the normal 16-byte
structure on native 64-bit tasks, ilp32 tasks (x32, aarch64-32), and future
compat32 (i386, aarch32, ...) tasks, while the syscall for the 8-byte structure
deals with legacy compat32 tasks that do not yet use __kernel_time64_t.
If the user structure has some padding (and as I've read in this thread
it is allowed), it could be even easier for the kernel. The padding
could be 32-bit before or after tv_nsec, depending on endianness.
The problem as pointed out before is that if you do this, 32-bit tasks
need to have the padding word zeroed at some stage for data passed into
the kernel, while 64-bit tasks need to return an error if the upper half
of the tv_nsec word is nonzero, at least for interfaces that are documented
to do this.
This can be done either in the kernel or in the libc. In the kernel, it
comes down to a function like
int get_user_timespec64(struct timespec64 *ts, struct __kernel_timespec64 __user
*uts, bool task_32bit)
{
struct __kernel_timespec64 input;
if (copy_from_user(input, uts, sizeof(input))
return -EFAULT;
ts-tv_sec = input.tv_sec;
if (task_32bit)
ts-tv_nsec = (int)input.tv_nsec;
else
ts-tv_nsec = input.tv_nsec;
return 0;
}
with data types of
struct timespec64 {
time64_t tv_sec;
long tv_nsec;
};
struct __kernel_timespec64 {
__kernel_time64_t tv_nsec;
#if (__BYTE_ORDER == __BIG_ENDIAN) (__BITS_PER_LONG == 32)
u32 __pad;
#endif
long tv_nsec;
#if (__BYTE_ORDER == __LITTLE_ENDIAN) (__BITS_PER_LONG == 32)
u32 __pad;
#endif
};
The data structure definition is a little bit fragile, as it depends on
user space not using the __BIT_ENDIAN symbol in a conflicting way. So
far we have managed to keep that outside of general purpose headers, but
it should at least blow up in an
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
On Thu, Feb 12, 2015 at 09:12:34AM +0100, Szabolcs Nagy wrote: * Szabolcs Nagy n...@port70.net [2015-02-11 20:05:37 +0100]: (i think this is also a problem if userspace code uses syscall(2) directly, libc cannot possibly know where to signextend and the kernel side does not do the fixup right now) nobody picked up this issue, is this resolved? ie. if userspace calls syscall(SYS_foo,...) directly with 32bit longs does it always work out correctly on the kernel side? I think the only way to solve this is to have syscall wrappers in the kernel rather than glibc. the sign extension is a problem for signed long arguments, i only found these in the kernel: fs/buffer.c:SYSCALL_DEFINE2(bdflush, int, func, long, data) This is part of the deprecated syscalls, it is not used on new user ABIs. fs/open.c:SYSCALL_DEFINE2(truncate, const char __user *, path, long, length) The kernel uses a long (64-bit) here and the user ABI defines this as an off_t. With x32, this should be a long long (__kernel_long_t), so not a problem. fs/aio.c:SYSCALL_DEFINE3(io_submit, aio_context_t, ctx_id, long, nr, fs/aio.c- struct iocb __user * __user *, iocbpp) fs/aio.c:SYSCALL_DEFINE5(io_getevents, aio_context_t, ctx_id, fs/aio.c- long, min_nr, fs/aio.c- long, nr, These would need some int-long conversion for nr, min_nr (it may be done in x32 glibc already but as you said it would not work via syscall() directly). kernel/ptrace.c:SYSCALL_DEFINE4(ptrace, long, request, long, pid, unsigned long, addr, kernel/ptrace.c-unsigned long, data) The pid in user space would be pid_t which is 32-bit. The kernel seems to use it as pid_t afterwards, so looks safe. For addr and data, I guess it needs wrappers to zero the top part. ipc/syscall.c:SYSCALL_DEFINE6(ipc, unsigned int, call, int, first, unsigned long, second, ipc/syscall.c- unsigned long, third, void __user *, ptr, long, fifth) ipc(2) shows the first, second, third as ints. I guess some kernel wrapper is needed here as well. -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 12:15:56PM -0800, Andy Lutomirski wrote: On 02/11/2015 11:57 AM, H.J. Lu wrote: trivially satisfied if you consider x32 and x86_64 separate compilation environments, but it's not related to the core issue: that the definition of timespec violates core (not obscure) requirements of both POSIX and C11. At the time you were probably unaware of the C11 requirement. Note that it's a LOT harder to effect change in the C standard, so even if the Austin Group would be amenable to changing the requirement for timespec to allow something like nseconds_t, getting WG14 to make this change to work around a Linux/glibc mistake does not sound practical. That is very unfortunate. I consider it is too late for x32 to change. Why? It's hardly an incompatible ABI change, as long as the kernel/libc fills the upper bits (for old programs that read them based on the old headers) when structs are read from the kernel to the application, and ignores the upper bits (potentially set or left uninitialized by the application) when strings are passed from userspace to the kernel. Newly built apps using the struct definition with 32-bit tv_nsec would need new libc to ensure that the high bits aren't interpreted, but this could be handled by symbol versioning. We have considered this option. But since kernel wouldn't change tv_nsec/tv_usec handling just for x32, it wasn't selected. Did anyone *ask* the kernel people (e.g. hpa)? It seems so: https://lkml.org/lkml/2011/8/31/244 Couple of more replies from hpa: https://lkml.org/lkml/2011/8/31/261 https://lkml.org/lkml/2012/2/8/408 It looks like hpa was going to talk the POSIX committee but I don't know what the conclusion was and didn't follow the thread (at the time I wasn't interested in ARM ILP32). -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Thu, Feb 12, 2015 at 03:50:24PM +, Catalin Marinas wrote: On Wed, Feb 11, 2015 at 12:15:56PM -0800, Andy Lutomirski wrote: On 02/11/2015 11:57 AM, H.J. Lu wrote: trivially satisfied if you consider x32 and x86_64 separate compilation environments, but it's not related to the core issue: that the definition of timespec violates core (not obscure) requirements of both POSIX and C11. At the time you were probably unaware of the C11 requirement. Note that it's a LOT harder to effect change in the C standard, so even if the Austin Group would be amenable to changing the requirement for timespec to allow something like nseconds_t, getting WG14 to make this change to work around a Linux/glibc mistake does not sound practical. That is very unfortunate. I consider it is too late for x32 to change. Why? It's hardly an incompatible ABI change, as long as the kernel/libc fills the upper bits (for old programs that read them based on the old headers) when structs are read from the kernel to the application, and ignores the upper bits (potentially set or left uninitialized by the application) when strings are passed from userspace to the kernel. Newly built apps using the struct definition with 32-bit tv_nsec would need new libc to ensure that the high bits aren't interpreted, but this could be handled by symbol versioning. We have considered this option. But since kernel wouldn't change tv_nsec/tv_usec handling just for x32, it wasn't selected. Did anyone *ask* the kernel people (e.g. hpa)? It seems so: https://lkml.org/lkml/2011/8/31/244 Couple of more replies from hpa: https://lkml.org/lkml/2011/8/31/261 https://lkml.org/lkml/2012/2/8/408 It looks like hpa was going to talk the POSIX committee but I don't know what the conclusion was and didn't follow the thread (at the time I wasn't interested in ARM ILP32). At this point POSIX committee is not sufficient. ISO C specifies timespec now, and as Jens Gustedt mentioned (I don't think his reply made it to the whole CC list; see the musl list archive here: http://www.openwall.com/lists/musl/2015/02/11/21 ), it seems unlikely that one could pose a convincing argument for this requirement to be changed in the C language. Rich -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Thu, Feb 12, 2015 at 7:50 AM, Catalin Marinas catalin.mari...@arm.com wrote: On Wed, Feb 11, 2015 at 12:15:56PM -0800, Andy Lutomirski wrote: On 02/11/2015 11:57 AM, H.J. Lu wrote: trivially satisfied if you consider x32 and x86_64 separate compilation environments, but it's not related to the core issue: that the definition of timespec violates core (not obscure) requirements of both POSIX and C11. At the time you were probably unaware of the C11 requirement. Note that it's a LOT harder to effect change in the C standard, so even if the Austin Group would be amenable to changing the requirement for timespec to allow something like nseconds_t, getting WG14 to make this change to work around a Linux/glibc mistake does not sound practical. That is very unfortunate. I consider it is too late for x32 to change. Why? It's hardly an incompatible ABI change, as long as the kernel/libc fills the upper bits (for old programs that read them based on the old headers) when structs are read from the kernel to the application, and ignores the upper bits (potentially set or left uninitialized by the application) when strings are passed from userspace to the kernel. Newly built apps using the struct definition with 32-bit tv_nsec would need new libc to ensure that the high bits aren't interpreted, but this could be handled by symbol versioning. We have considered this option. But since kernel wouldn't change tv_nsec/tv_usec handling just for x32, it wasn't selected. Did anyone *ask* the kernel people (e.g. hpa)? It seems so: https://lkml.org/lkml/2011/8/31/244 Couple of more replies from hpa: https://lkml.org/lkml/2011/8/31/261 https://lkml.org/lkml/2012/2/8/408 It looks like hpa was going to talk the POSIX committee but I don't know what the conclusion was and didn't follow the thread (at the time I wasn't interested in ARM ILP32). Just for the record, tv_nsec/tv_usec can be changed to long as long as kernel always read them as 32 bits and write them as 64 bits for both LP64 and ILP32 in 64-bit imespec amd timeval. In glibc, they can be changed to long without breaking existing binaries. For x86-32, 64-bit __time_t must be 64-bit aligned. Otherwise, there will be no padding in 64-bit timespec nor timeval. -- H.J. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Thu, Feb 12, 2015 at 08:30:10AM -0800, H.J. Lu wrote: On Thu, Feb 12, 2015 at 7:50 AM, Catalin Marinas catalin.mari...@arm.com wrote: On Wed, Feb 11, 2015 at 12:15:56PM -0800, Andy Lutomirski wrote: On 02/11/2015 11:57 AM, H.J. Lu wrote: trivially satisfied if you consider x32 and x86_64 separate compilation environments, but it's not related to the core issue: that the definition of timespec violates core (not obscure) requirements of both POSIX and C11. At the time you were probably unaware of the C11 requirement. Note that it's a LOT harder to effect change in the C standard, so even if the Austin Group would be amenable to changing the requirement for timespec to allow something like nseconds_t, getting WG14 to make this change to work around a Linux/glibc mistake does not sound practical. That is very unfortunate. I consider it is too late for x32 to change. Why? It's hardly an incompatible ABI change, as long as the kernel/libc fills the upper bits (for old programs that read them based on the old headers) when structs are read from the kernel to the application, and ignores the upper bits (potentially set or left uninitialized by the application) when strings are passed from userspace to the kernel. Newly built apps using the struct definition with 32-bit tv_nsec would need new libc to ensure that the high bits aren't interpreted, but this could be handled by symbol versioning. We have considered this option. But since kernel wouldn't change tv_nsec/tv_usec handling just for x32, it wasn't selected. Did anyone *ask* the kernel people (e.g. hpa)? It seems so: https://lkml.org/lkml/2011/8/31/244 Couple of more replies from hpa: https://lkml.org/lkml/2011/8/31/261 https://lkml.org/lkml/2012/2/8/408 It looks like hpa was going to talk the POSIX committee but I don't know what the conclusion was and didn't follow the thread (at the time I wasn't interested in ARM ILP32). Just for the record, tv_nsec/tv_usec can be changed to long as long as kernel always read them as 32 bits and write them as 64 bits for both LP64 and ILP32 in 64-bit imespec amd timeval. No; currently userspace relies on the kernel to produce EINVAL when tv_nsec is not in the range [0,9]. If the kernel just reads it as 32-bit unconditionally, tv_nsec=0x1 would fail to produce EINVAL in LP64 models where tv_nsec is a 64-bit object in userspace. In glibc, they can be changed to long without breaking existing binaries. This is true only if glibc or the kernel ignores the upper bits. Otherwise, programs could end up passing junk that glibc and/or the kernel interprets. For x86-32, 64-bit __time_t must be 64-bit aligned. Otherwise, there will be no padding in 64-bit timespec nor timeval. Just adding an explicit padding member when long is 32-bit would be cleaner. This makes it possible to manually set/clear/inspect the bits without memset. I don't see any reason to require actual alignment of the struct on x86-32 unless you're going with a whole new ABI where 64-bit types are aligned. Of course if we're thinking about making 64-bit time_t on 32-bit archs, that's an incompatible ABI already and would be a great time to make lots of other ABI fixes... But I wonder if anyone is going to care about actual x86-32 hardware as Y2038 approaches. Rich -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
* Szabolcs Nagy n...@port70.net [2015-02-11 20:05:37 +0100]: (i think this is also a problem if userspace code uses syscall(2) directly, libc cannot possibly know where to signextend and the kernel side does not do the fixup right now) nobody picked up this issue, is this resolved? ie. if userspace calls syscall(SYS_foo,...) directly with 32bit longs does it always work out correctly on the kernel side? the sign extension is a problem for signed long arguments, i only found these in the kernel: fs/buffer.c:SYSCALL_DEFINE2(bdflush, int, func, long, data) fs/open.c:SYSCALL_DEFINE2(truncate, const char __user *, path, long, length) fs/aio.c:SYSCALL_DEFINE3(io_submit, aio_context_t, ctx_id, long, nr, fs/aio.c- struct iocb __user * __user *, iocbpp) fs/aio.c:SYSCALL_DEFINE5(io_getevents, aio_context_t, ctx_id, fs/aio.c- long, min_nr, fs/aio.c- long, nr, kernel/ptrace.c:SYSCALL_DEFINE4(ptrace, long, request, long, pid, unsigned long, addr, kernel/ptrace.c-unsigned long, data) ipc/syscall.c:SYSCALL_DEFINE6(ipc, unsigned int, call, int, first, unsigned long, second, ipc/syscall.c- unsigned long, third, void __user *, ptr, long, fifth) -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, 11 Feb 2015, Rich Felker wrote: > > > https://sourceware.org/bugzilla/show_bug.cgi?id=16437 > > > > Please leave x32 out of this discussion. I have resolved this bug > > as WONTFIX. > > From the glibc side, I thought things went by a consensus process > these days, not the old WONTFIX regime of he who shall not be named. > If this is not fixed for x32, then x32 cannot provide a conforming C > environment and thus it's rather a toy target. But I think we should > discuss this on libc-alpha. In the mean time please leave it REOPENED. Indeed. x86 is handled primarily by community review, and even when we have maintainers for architectures or other subsystems, being maintainer serves as a shortcut to presume consensus in the absence of controversy (in the expectation that the community won't object), not to override community discussion if something is more controversial. I've reopened the bug. I believe I made clear in the discussion of 64-bit time interfaces for 32-bit systems that the x32 ABI mistake was not one to be repeated - that since there is obviously no need for nanoseconds values that cannot fit in 32 bits, nanoseconds (and microseconds) values should remain as long in accordance with POSIX. It's absolutely fine for the userspace structures to have an explicit __glibc_reserved padding field in an endian-dependent place to keep the low part of the nanoseconds value in the same place as it would be for a 64-bit type, but if the kernel doesn't ignore that padding for the 64-bit time interfaces then all places that pass these structures from glibc to the kernel need to copy them and zero the padding in the copy. Whether the high 32 bits can be treated as padding for interfaces where long is 64-bit depends on whether the interfaces in question are required to return an error such as EINVAL for out-of-range nanoseconds values. -- Joseph S. Myers jos...@codesourcery.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 10:02:55PM +0100, a...@arndb.de wrote:
> Rich Felker hat am 11. Februar 2015 um 21:12 geschrieben:
> > On Wed, Feb 11, 2015 at 08:50:06PM +0100, a...@arndb.de wrote:
> > > > > At least for AArch64 ILP32 we are still free to change the user/kernel
> > > > > ABI, so we could add wrappers for the affected syscalls to fix this
> > > > > up.
> > > > yes, afaik on x32 the 64bit kernel expects 64bit layout,
> > > > arm64 can fix this
> > >
> > > We have to fix it on all 32-bit architectures when we move to 64-bit
> > > time_t.
> > >
> > > I think ideally you'd want a user space definition like
> > >
> > > typedef long long time_t;
> > > struct timespec {
> > > time_t tv_sec;
> > > long long tv_nsec;
> > > };
> > >
> > > which is the only way to avoid passing uninitialized tv_nsec into the
> > > kernel
> > > from arbitrary user space doing ioctl. This is of course against POSIX and
> > > C99. Changing POSIX to allow it is probably easier than the C standard,
> > > but we have a couple of years before we need to make this the default.
> >
> > I don't see why you want it to be long long. There is no harm in
> > passing uninitialized padding to the kernel; the kernel just needs to
> > do the right thing and ignore it (or avoid reading it to begin with).
>
> This would however mean having three different implementations
> in the kernel rather than just two: Every driver that can pass a timespec
> with this model needs to handle the native 64-bit case (64/64), the legacy
> 32-bit case (32/32) and the y2038-safe case (64/32). Most code can
> already handle the first two, and none today handles the third. If you
> want to make the handling explicitly incompatible with native 64-bit
> mode, you get a lot of untested code in obscure places that are never
> tested properly, while using the normal behavior in the kernel at least
> gives us the same bugs that we already have on native 64-bit systems.
Would it really be that hard to do:
if (ILP32_on_64_process) tv_nsec = (int)tv_nsec;
or similar? That's all that's needed.
> In some cases, there may also be a measurable performance penalty
> in interpreting a user space data structure manually over copying
> it (including the timespec values) in one chunk.
I don't think the above would be measurable.
> An alternative would be to change the native 64-bit case to ignore the upper
> half of tv_nsec and always just copy the low bits. This should work
> fine almost all of the time, but I fear that there might be corner cases
> where existing 64-bit user space depends on passing large or negative
> tv_nsec values into the kernel.
Most functions using caller-provided timespecs are required to
diagnose invalid forms with EINVAL when tv_nsec>=10 or <0, so
if the kernel examines only the low 32 bits on ABIs where long is
64-bit, userspace would need to be responsible for doing this
checking.
> > The other direction, passing uninitialized data from the kernel to
> > userspace, would be dangerous. But it doesn't happen as long as the
> > userspace padding is positioned (in an endian-dependent manner) where
> > the high bits of the kernel type would lie. It could happen if you
> > used a separate conversion wrapper that ony wrote 32 bits, but if you
> > wanted to take that approach you'd just need the wrapper to also write
> > the padding field manually.
>
> Going from kernel to user space should not be an issue as long as we
> always just write two 64-bit words, and this will zero-fill the upper half.
Agreed.
> > > In the kernel headers, the current plan is to provide interfaces taking
> > > structures
> > >
> > > typedef long long __kernel_time64_t;
> > > struct __kernel_timespec64_t {
> > > __kernel_time64_t tv_sec;
> > > long long tv_nsec;
> > > };
> > >
> > > at least for ioctls, to avoid the ambiguity with libc headers specifying
> > > something else.
> >
> > This seems hideous from an application standpoint. Application
> > programmers don't want to know, and shouldn't need to know, these
> > silly implementation details that make no sense except as historical
> > baggage. They should just be able to use "struct timespec" everywhere
> > and have it work.
>
> The kernel does not even know how timespec is defined by libc, and we have
> to at least be able to handle the common cases of timespec being 32/32
> and 64/64 (or 64/32 plus explicit padding). For system calls, we can rely
> on libc calling the syscalls that match the definition (or convert the
> structure as necessary), while for ioctl the command number is chosen
> by the application and has to match the structure definition provided in
> the same header.
Generally I would think the kernel knows the model the process is
using, but if not, all you need is separate ioctl numbers for
userspace to use depending on which definition it's using.
> In a lot of cases, the ioctl command number is defined (correctly) using the
> _IOR/_IOW macros that take the size of the
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
Sorry about the HTML mail, I'm currently travelling without access to my regular
mail client.
> "a...@arndb.de" hat am 11. Februar 2015 um 22:02 geschrieben:
>
> Rich Felker hat am 11. Februar 2015 um 21:12 geschrieben:
> > On Wed, Feb 11, 2015 at 08:50:06PM +0100, a...@arndb.de wrote:
> > > > > At least for AArch64 ILP32 we are still free to change the
> > > > > user/kernel
> > > > > ABI, so we could add wrappers for the affected syscalls to fix this
> > > > > up.
> > > > yes, afaik on x32 the 64bit kernel expects 64bit layout,
> > > > arm64 can fix this
> > >
> > > We have to fix it on all 32-bit architectures when we move to 64-bit
> > > time_t.
> > >
> > > I think ideally you'd want a user space definition like
> > >
> > > typedef long long time_t;
> > > struct timespec {
> > > time_t tv_sec;
> > > long long tv_nsec;
> > > };
> > >
> > > which is the only way to avoid passing uninitialized tv_nsec into the
> > > kernel
> > > from arbitrary user space doing ioctl. This is of course against POSIX
> > > and
> > > C99. Changing POSIX to allow it is probably easier than the C standard,
> > > but we have a couple of years before we need to make this the default.
> >
> > I don't see why you want it to be long long. There is no harm in
> > passing uninitialized padding to the kernel; the kernel just needs to
> > do the right thing and ignore it (or avoid reading it to begin with).
>
> This would however mean having three different implementations
> in the kernel rather than just two: Every driver that can pass a timespec
> with this model needs to handle the native 64-bit case (64/64), the legacy
> 32-bit case (32/32) and the y2038-safe case (64/32). Most code can
> already handle the first two, and none today handles the third. If you
> want to make the handling explicitly incompatible with native 64-bit
> mode, you get a lot of untested code in obscure places that are never
> tested properly, while using the normal behavior in the kernel at least
> gives us the same bugs that we already have on native 64-bit systems.
>
> In some cases, there may also be a measurable performance penalty
> in interpreting a user space data structure manually over copying
> it (including the timespec values) in one chunk.
>
> An alternative would be to change the native 64-bit case to ignore the upper
> half of tv_nsec and always just copy the low bits. This should work
> fine almost all of the time, but I fear that there might be corner cases
> where existing 64-bit user space depends on passing large or negative
> tv_nsec values into the kernel.
>
> > The other direction, passing uninitialized data from the kernel to
> > userspace, would be dangerous. But it doesn't happen as long as the
> > userspace padding is positioned (in an endian-dependent manner) where
> > the high bits of the kernel type would lie. It could happen if you
> > used a separate conversion wrapper that ony wrote 32 bits, but if you
> > wanted to take that approach you'd just need the wrapper to also write
> > the padding field manually.
>
> Going from kernel to user space should not be an issue as long as we
> always just write two 64-bit words, and this will zero-fill the upper half.
>
> > > In the kernel headers, the current plan is to provide interfaces taking
> > > structures
> > >
> > > typedef long long __kernel_time64_t;
> > > struct __kernel_timespec64_t {
> > > __kernel_time64_t tv_sec;
> > > long long tv_nsec;
> > > };
> > >
> > > at least for ioctls, to avoid the ambiguity with libc headers specifying
> > > something else.
> >
> > This seems hideous from an application standpoint. Application
> > programmers don't want to know, and shouldn't need to know, these
> > silly implementation details that make no sense except as historical
> > baggage. They should just be able to use "struct timespec" everywhere
> > and have it work.
> The kernel does not even know how timespec is defined by libc, and we have
> to at least be able to handle the common cases of timespec being 32/32
> and 64/64 (or 64/32 plus explicit padding). For system calls, we can rely
> on libc calling the syscalls that match the definition (or convert the
> structure as necessary), while for ioctl the command number is chosen
> by the application and has to match the structure definition provided in
> the same header.
>
> In a lot of cases, the ioctl command number is defined (correctly) using the
> _IOR/_IOW macros that take the size of the structure into account, but then
> you also have cases where you get indirect pointers and the size of data
> structure
> passed by the ioctl command is independent of the size of timespec or time_t.
>
> This is not just limited to time_t, we have a lot of data types for which we
> define
> __kernel_*_t types for this purpose, to deal with ioctls that need a specific
> layout independent of what libc uses.
>
>Arnd
>
Re: [PATCHv3 00/24] ILP32 support in ARM64
On 02/11/2015 11:57 AM, H.J. Lu wrote: trivially satisfied if you consider x32 and x86_64 separate compilation environments, but it's not related to the core issue: that the definition of timespec violates core (not obscure) requirements of both POSIX and C11. At the time you were probably unaware of the C11 requirement. Note that it's a LOT harder to effect change in the C standard, so even if the Austin Group would be amenable to changing the requirement for timespec to allow something like nseconds_t, getting WG14 to make this change to work around a Linux/glibc mistake does not sound practical. That is very unfortunate. I consider it is too late for x32 to change. Why? It's hardly an incompatible ABI change, as long as the kernel/libc fills the upper bits (for old programs that read them based on the old headers) when structs are read from the kernel to the application, and ignores the upper bits (potentially set or left uninitialized by the application) when strings are passed from userspace to the kernel. Newly built apps using the struct definition with 32-bit tv_nsec would need new libc to ensure that the high bits aren't interpreted, but this could be handled by symbol versioning. We have considered this option. But since kernel wouldn't change tv_nsec/tv_usec handling just for x32, it wasn't selected. Did anyone *ask* the kernel people (e.g. hpa)? --Andy -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 08:50:06PM +0100, a...@arndb.de wrote:
> > > At least for AArch64 ILP32 we are still free to change the user/kernel
> > > ABI, so we could add wrappers for the affected syscalls to fix this up.
> > >
> >
> > yes, afaik on x32 the 64bit kernel expects 64bit layout,
> > arm64 can fix this
>
> We have to fix it on all 32-bit architectures when we move to 64-bit time_t.
>
> I think ideally you'd want a user space definition like
>
> typedef long long time_t;
> struct timespec {
> time_t tv_sec;
> long long tv_nsec;
> };
>
> which is the only way to avoid passing uninitialized tv_nsec into the kernel
> from arbitrary user space doing ioctl. This is of course against POSIX and
> C99. Changing POSIX to allow it is probably easier than the C standard,
> but we have a couple of years before we need to make this the default.
I don't see why you want it to be long long. There is no harm in
passing uninitialized padding to the kernel; the kernel just needs to
do the right thing and ignore it (or avoid reading it to begin with).
Changing the C standard in an incompatible way that invalidates
existing code is not preferable over fixing an implementation bug in
one implementation. Even if C16 or so changed the requirement, people
will still be looking to C11 (and even C99) for years or decades to
come. Alignment of code to language standards moves slowly.
The other direction, passing uninitialized data from the kernel to
userspace, would be dangerous. But it doesn't happen as long as the
userspace padding is positioned (in an endian-dependent manner) where
the high bits of the kernel type would lie. It could happen if you
used a separate conversion wrapper that ony wrote 32 bits, but if you
wanted to take that approach you'd just need the wrapper to also write
the padding field manually.
> In the kernel headers, the current plan is to provide interfaces taking
> structures
>
> typedef long long __kernel_time64_t;
> struct __kernel_timespec64_t {
> __kernel_time64_t tv_sec;
> long long tv_nsec;
> };
>
> at least for ioctls, to avoid the ambiguity with libc headers specifying
> something else.
This seems hideous from an application standpoint. Application
programmers don't want to know, and shouldn't need to know, these
silly implementation details that make no sense except as historical
baggage. They should just be able to use "struct timespec" everywhere
and have it work.
Rich
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
>> > trivially satisfied if you consider x32 and x86_64 separate >> > compilation environments, but it's not related to the core issue: that >> > the definition of timespec violates core (not obscure) requirements of >> > both POSIX and C11. At the time you were probably unaware of the C11 >> > requirement. Note that it's a LOT harder to effect change in the C >> > standard, so even if the Austin Group would be amenable to changing >> > the requirement for timespec to allow something like nseconds_t, >> > getting WG14 to make this change to work around a Linux/glibc mistake >> > does not sound practical. >> >> That is very unfortunate. I consider it is too late for x32 to change. > > Why? It's hardly an incompatible ABI change, as long as the > kernel/libc fills the upper bits (for old programs that read them > based on the old headers) when structs are read from the kernel to the > application, and ignores the upper bits (potentially set or left > uninitialized by the application) when strings are passed from > userspace to the kernel. Newly built apps using the struct definition > with 32-bit tv_nsec would need new libc to ensure that the high bits > aren't interpreted, but this could be handled by symbol versioning. > We have considered this option. But since kernel wouldn't change tv_nsec/tv_usec handling just for x32, it wasn't selected. -- H.J. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
> Szabolcs Nagy hat am 11. Februar 2015 um 20:05 geschrieben:
> * Catalin Marinas [2015-02-11 17:39:19 +]:
> > (adding Marcus)
> >
> > On Tue, Feb 10, 2015 at 06:13:02PM +, Rich Felker wrote:
> > > I don't know if this has been discussed on libc-alpha yet or not, but
> > > I think we need to open a discussion of how it relates to open glibc
> > > bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64):
> > >
> > > https://sourceware.org/bugzilla/show_bug.cgi?id=16437
> ...
> > So w.r.t. C11, the exported kernel timespec looks fine. But I think the
> > x32 kernel support (and the current ILP32 patches) assume a native
> > struct timespec with tv_nsec as 64-bit.
> >
> > If we are to be C11 conformant, glibc on x32 has a bug as it defines
> > timespec incorrectly. This hid a bug in the kernel handling the
> > corresponding x32 syscalls. What's the best fix for x32 I can't really
> > tell (we need people to agree on where the bugs are).
> >
> > At least for AArch64 ILP32 we are still free to change the user/kernel
> > ABI, so we could add wrappers for the affected syscalls to fix this up.
> >
>
> yes, afaik on x32 the 64bit kernel expects 64bit layout,
> arm64 can fix this
We have to fix it on all 32-bit architectures when we move to 64-bit time_t.
I think ideally you'd want a user space definition like
typedef long long time_t;
struct timespec {
time_t tv_sec;
long long tv_nsec;
};
which is the only way to avoid passing uninitialized tv_nsec into the kernel
from arbitrary user space doing ioctl. This is of course against POSIX and
C99. Changing POSIX to allow it is probably easier than the C standard,
but we have a couple of years before we need to make this the default.
In the kernel headers, the current plan is to provide interfaces taking
structures
typedef long long __kernel_time64_t;
struct __kernel_timespec64_t {
__kernel_time64_t tv_sec;
long long tv_nsec;
};
at least for ioctls, to avoid the ambiguity with libc headers specifying
something else.
A libc could use other definitions with added padding for what it exports
to user space though, or even make this depend on compile-time flags to
determine whether to make tv_nsec as long long, or to insert padding
in the right place based on endianess.
> > > While most of the other type changes proposed (I'm looking at
> > > https://lkml.org/lkml/2014/9/3/719) are permissible and simply
> > > ugly/undesirable,
> >
> > They may be ugly but definitely not undesirable ;).
> >
>
> several types have the same c level definition across all archs..
> except x32 because of
>
> typedef long long __kernel_long_t;
>
> this should not cause posix/c conformance issues (as you noted
> timespec is ok in the uapi header only the kernel side behaviour
> is wrong)
The kernel behavior is right for the syscalls except ioctl, the uapi
header is wrong (not according to C99, but according to common sense)
and needs to be changed in order to work with big-endian. For x32,
I wonder if we can just #define timespec as __kernel_timespec64
once we introduce that.
> > > Working around the discrepencies in userspace IS possible, but ugly.
> > > We do it in musl libc for x32 right now -- see:
> > >
> > > http://git.musl-libc.org/cgit/musl/tree/arch/x32/syscall_arch.h?id=v1.1.6
> > > http://git.musl-libc.org/cgit/musl/tree/arch/x32/src/syscall_cp_fixup.c?id=v1.1.6
> >
> > For AArch64 ILP32 I would rather see the fix-ups in kernel wrappers.
> >
> > Are you aware of other cases like this?
> >
>
> i know at least one android kernel issue: there is an ioctl for the
> alarm device that takes timespec argument
>
> (i think it's not in the mainline kernel and i guess android does
> not care about x32 so it was not an issue so far, but this is something
> that should not be fixed on the libc side)
ioctl is a known problem with x32 and the ARM ILP32 support. This is
not limited to timespec but to any driver that uses an ioctl with
a data structure that includes a __kernel_long_t or __kernel_ulong_t
argument. There are a couple of drivers using these, and we either
need to change the structures to use 'long' instead, or fix the driver
to be aware of the difference between old-style 32-bit compat and
x32-style compat ioctl handling.
The types affected by this are
include/uapi/asm-generic/posix_types.h:typedef __kernel_ulong_t __kernel_ino_t;
include/uapi/asm-generic/posix_types.h:typedef __kernel_ulong_t __kernel_size_t;
include/uapi/asm-generic/posix_types.h:typedef __kernel_long_t
__kernel_suseconds_t;
include/uapi/asm-generic/posix_types.h:typedef __kernel_long_t
__kernel_ssize_t;
include/uapi/asm-generic/posix_types.h:typedef __kernel_long_t
__kernel_ptrdiff_t;
include/uapi/asm-generic/posix_types.h:typedef __kernel_long_t __kernel_off_t;
include/uapi/asm-generic/posix_types.h:typedef __kernel_long_t __kernel_time_t;
include/uapi/asm-generic/posix_types.h:typedef __kernel_long_t
__kernel_clock_t;
and anything derived
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 11:34:23AM -0800, H.J. Lu wrote: > On Wed, Feb 11, 2015 at 11:25 AM, Rich Felker wrote: > > On Wed, Feb 11, 2015 at 11:16:58AM -0800, H.J. Lu wrote: > >> >> > I don't know if this has been discussed on libc-alpha yet or not, but > >> >> > I think we need to open a discussion of how it relates to open glibc > >> >> > bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): > >> >> > > >> >> > https://sourceware.org/bugzilla/show_bug.cgi?id=16437 > >> >> > >> >> Please leave x32 out of this discussion. I have resolved this bug > >> >> as WONTFIX. > >> > > >> > From the glibc side, I thought things went by a consensus process > >> > these days, not the old WONTFIX regime of he who shall not be named. > >> > If this is not fixed for x32, then x32 cannot provide a conforming C > >> > environment and thus it's rather a toy target. But I think we should > >> > discuss this on libc-alpha. In the mean time please leave it REOPENED. > >> > >> As I said in PR, the issue has been raised in Mar, 2012 when the > >> x32 port was submitted. It has been decided that x32 won't conform > >> to tv_nsec, blksize_t, and suseconds_t as long. I don't believe we > >> will change them to conform to POSIX. > > > > I briefly reviewed that discussion and I think the decision made was > > about an obscure POSIX requirement about supporting at least one > > compilation environment where certain types have rank <= long. This is > > The example you gave in PR is similar to > > https://sourceware.org/ml/libc-alpha/2012-03/msg00456.html Yes, but after that the conversation seemed to get derailed into the blksize_t etc. stuff about "compilation environments" that's largely irrelevant. I think this prevented the core tv_nsec issue from getting discussed further, unless I'm missing part of that thread. > > trivially satisfied if you consider x32 and x86_64 separate > > compilation environments, but it's not related to the core issue: that > > the definition of timespec violates core (not obscure) requirements of > > both POSIX and C11. At the time you were probably unaware of the C11 > > requirement. Note that it's a LOT harder to effect change in the C > > standard, so even if the Austin Group would be amenable to changing > > the requirement for timespec to allow something like nseconds_t, > > getting WG14 to make this change to work around a Linux/glibc mistake > > does not sound practical. > > That is very unfortunate. I consider it is too late for x32 to change. Why? It's hardly an incompatible ABI change, as long as the kernel/libc fills the upper bits (for old programs that read them based on the old headers) when structs are read from the kernel to the application, and ignores the upper bits (potentially set or left uninitialized by the application) when strings are passed from userspace to the kernel. Newly built apps using the struct definition with 32-bit tv_nsec would need new libc to ensure that the high bits aren't interpreted, but this could be handled by symbol versioning. Rich -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 11:25 AM, Rich Felker wrote: > On Wed, Feb 11, 2015 at 11:16:58AM -0800, H.J. Lu wrote: >> >> > I don't know if this has been discussed on libc-alpha yet or not, but >> >> > I think we need to open a discussion of how it relates to open glibc >> >> > bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): >> >> > >> >> > https://sourceware.org/bugzilla/show_bug.cgi?id=16437 >> >> >> >> Please leave x32 out of this discussion. I have resolved this bug >> >> as WONTFIX. >> > >> > From the glibc side, I thought things went by a consensus process >> > these days, not the old WONTFIX regime of he who shall not be named. >> > If this is not fixed for x32, then x32 cannot provide a conforming C >> > environment and thus it's rather a toy target. But I think we should >> > discuss this on libc-alpha. In the mean time please leave it REOPENED. >> >> As I said in PR, the issue has been raised in Mar, 2012 when the >> x32 port was submitted. It has been decided that x32 won't conform >> to tv_nsec, blksize_t, and suseconds_t as long. I don't believe we >> will change them to conform to POSIX. > > I briefly reviewed that discussion and I think the decision made was > about an obscure POSIX requirement about supporting at least one > compilation environment where certain types have rank <= long. This is The example you gave in PR is similar to https://sourceware.org/ml/libc-alpha/2012-03/msg00456.html > trivially satisfied if you consider x32 and x86_64 separate > compilation environments, but it's not related to the core issue: that > the definition of timespec violates core (not obscure) requirements of > both POSIX and C11. At the time you were probably unaware of the C11 > requirement. Note that it's a LOT harder to effect change in the C > standard, so even if the Austin Group would be amenable to changing > the requirement for timespec to allow something like nseconds_t, > getting WG14 to make this change to work around a Linux/glibc mistake > does not sound practical. That is very unfortunate. I consider it is too late for x32 to change. -- H.J. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 11:16:58AM -0800, H.J. Lu wrote: > >> > I don't know if this has been discussed on libc-alpha yet or not, but > >> > I think we need to open a discussion of how it relates to open glibc > >> > bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): > >> > > >> > https://sourceware.org/bugzilla/show_bug.cgi?id=16437 > >> > >> Please leave x32 out of this discussion. I have resolved this bug > >> as WONTFIX. > > > > From the glibc side, I thought things went by a consensus process > > these days, not the old WONTFIX regime of he who shall not be named. > > If this is not fixed for x32, then x32 cannot provide a conforming C > > environment and thus it's rather a toy target. But I think we should > > discuss this on libc-alpha. In the mean time please leave it REOPENED. > > As I said in PR, the issue has been raised in Mar, 2012 when the > x32 port was submitted. It has been decided that x32 won't conform > to tv_nsec, blksize_t, and suseconds_t as long. I don't believe we > will change them to conform to POSIX. I briefly reviewed that discussion and I think the decision made was about an obscure POSIX requirement about supporting at least one compilation environment where certain types have rank <= long. This is trivially satisfied if you consider x32 and x86_64 separate compilation environments, but it's not related to the core issue: that the definition of timespec violates core (not obscure) requirements of both POSIX and C11. At the time you were probably unaware of the C11 requirement. Note that it's a LOT harder to effect change in the C standard, so even if the Austin Group would be amenable to changing the requirement for timespec to allow something like nseconds_t, getting WG14 to make this change to work around a Linux/glibc mistake does not sound practical. Rich -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 11:05 AM, Szabolcs Nagy wrote: > i know at least one android kernel issue: there is an ioctl for the > alarm device that takes timespec argument > > (i think it's not in the mainline kernel and i guess android does > not care about x32 so it was not an issue so far, but this is something > that should not be fixed on the libc side) > I have documented some remaining x32 kernel issues at: https://sites.google.com/site/x32abi/ -- H.J. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 05:39:19PM +, Catalin Marinas wrote: > (adding Marcus) > > On Tue, Feb 10, 2015 at 06:13:02PM +, Rich Felker wrote: > > On Thu, 2 Oct 2014 at 16:52:18 +0100, Catalin Marinas wrote: > > > On Wed, Sep 03, 2014 at 10:18:54PM +0100, Andrew Pinski wrote: > > > > New version with all of the requested changes. Updated to the > > > > latest sources. > > > > > > > > Notable changes from the previous versions: > > > > VDSO code has been factored out to be easier to understand and > > > > easier to maintain. > > > > Move the config option to the last thing that gets added. > > > > Added some extra COMPAT_* macros for core dumping for easier usage. > > > > > > Apart from a few comments I've made, I would also like to see non-empty > > > commit logs and long line wrapping (both in commit logs and > > > Documentation/). Otherwise, the patches look fine. > > > > > > So what are the next steps? Are the glibc folk ok with the ILP32 Linux > > > ABI? On the kernel side, what I would like to see: > > > > I don't know if this has been discussed on libc-alpha yet or not, but > > I think we need to open a discussion of how it relates to open glibc > > bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): > > > > https://sourceware.org/bugzilla/show_bug.cgi?id=16437 > > I'm trying to understand the problem first. Quoting from the bug above > (which I guess is quoted form C11): > > "The range and precision of times representable in clock_t and time_t > are implementation-defined. The timespec structure shall contain at > least the following members, in any order. > > time_t tv_sec; // whole seconds -- >= 0 > long tv_nsec; // nanoseconds -- [0, 9]" > > So changing time_t to 64-bit is fine on x32. The timespec struct > exported by the kernel always uses a long for tv_nsec. However, glibc uses > __syscall_slong_t which ends up as 64-bit for x32 (I guess it mirrors > the __kernel_long_t definition). > > So w.r.t. C11, the exported kernel timespec looks fine. But I think the > x32 kernel support (and the current ILP32 patches) assume a native > struct timespec with tv_nsec as 64-bit. The exported kernel timespec is not fine if long is defined as a 32-bit type, which it is for x32 and the proposed aarch64-ILP32 ABIs. The type being long means that code like long *p = >tv_nsec; *p = 42; is valid. With the proposed definition of timespec, this results in a compiler warning or error on valid code, which is better than dangerously wrong code generation, but we can easily hide the warning via: void *p = >tv_nsec; long *q = p; *q = 42; Imagine this happening in places such as with callbacks that take void pointers, for example pthread_create. But even if the breakage could always be diagnosed by the compiler, you're still breaking perfectly valid, conforming C programs. > If we are to be C11 conformant, glibc on x32 has a bug as it defines > timespec incorrectly. This hid a bug in the kernel handling the > corresponding x32 syscalls. What's the best fix for x32 I can't really > tell (we need people to agree on where the bugs are). For the kernel, it should be casting the value to int/int32_t to ignore the junk in the upper bits. But that only works on little endian. On big endian it's a bigger mess. However at this point fixing it on the kernel side is not very useful for x32, at least not right away. Since people will still be running old kernels with the wrong behavior, the fixup has to happen in userspace to support those old kernels until the libc drops support for kernels too old to handle it right. > At least for AArch64 ILP32 we are still free to change the user/kernel > ABI, so we could add wrappers for the affected syscalls to fix this up. Yes. That's what I'd like to see before aarch64-ILP32 is officially launched. It's a lot harder to retroactively fix this. Right now it's fairly easy -- just a small wrapper/fixup layer in the kernel. > > While most of the other type changes proposed (I'm looking at > > https://lkml.org/lkml/2014/9/3/719) are permissible and simply > > ugly/undesirable, > > They may be ugly but definitely not undesirable ;). I can point you to a few other cases that may be undesirable, but less severe than timespec: - https://sourceware.org/bugzilla/show_bug.cgi?id=16438 - http://man7.org/linux/man-pages/man2/sysinfo.2.html In the case of sysinfo, the public API documentation documents the fields as having type unsigned long, but on x32 (and presumably aarch64-ILP32 as proposed) they have type unsigned long long. I can probably find more examples if you're interested. > > defining struct timespec with tv_nsec having any type other than long > > conflicts with the requirements of C11 and POSIX, and WG14 is unlikely > > to be interested in changing the C language because the Linux kernel > > has the wrong type in timespec. > > I agree. The strange thing is that the Linux
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 11:02 AM, Rich Felker wrote: > On Wed, Feb 11, 2015 at 10:33:32AM -0800, H.J. Lu wrote: >> On Tue, Feb 10, 2015 at 10:13 AM, Rich Felker wrote: >> > On Thu, 2 Oct 2014 at 16:52:18 +0100, Catalin Marinas wrote: >> >> On Wed, Sep 03, 2014 at 10:18:54PM +0100, Andrew Pinski wrote: >> >> > New version with all of the requested changes. Updated to the >> >> > latest sources. >> >> > >> >> > Notable changes from the previous versions: >> >> > VDSO code has been factored out to be easier to understand and >> >> > easier to maintain. >> >> > Move the config option to the last thing that gets added. >> >> > Added some extra COMPAT_* macros for core dumping for easier usage. >> >> >> >> Apart from a few comments I've made, I would also like to see non-empty >> >> commit logs and long line wrapping (both in commit logs and >> >> Documentation/). Otherwise, the patches look fine. >> >> >> >> So what are the next steps? Are the glibc folk ok with the ILP32 Linux >> >> ABI? On the kernel side, what I would like to see: >> > >> > I don't know if this has been discussed on libc-alpha yet or not, but >> > I think we need to open a discussion of how it relates to open glibc >> > bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): >> > >> > https://sourceware.org/bugzilla/show_bug.cgi?id=16437 >> >> Please leave x32 out of this discussion. I have resolved this bug >> as WONTFIX. > > From the glibc side, I thought things went by a consensus process > these days, not the old WONTFIX regime of he who shall not be named. > If this is not fixed for x32, then x32 cannot provide a conforming C > environment and thus it's rather a toy target. But I think we should > discuss this on libc-alpha. In the mean time please leave it REOPENED. As I said in PR, the issue has been raised in Mar, 2012 when the x32 port was submitted. It has been decided that x32 won't conform to tv_nsec, blksize_t, and suseconds_t as long. I don't believe we will change them to conform to POSIX. As for if x32 is a toy target or not, it will be decided by whether it delivers what users are looking for, not by if tv_nsec, blksize_t, and suseconds_t conform to POSIX. -- H.J. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
* Catalin Marinas [2015-02-11 17:39:19 +]: > (adding Marcus) > > On Tue, Feb 10, 2015 at 06:13:02PM +, Rich Felker wrote: > > I don't know if this has been discussed on libc-alpha yet or not, but > > I think we need to open a discussion of how it relates to open glibc > > bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): > > > > https://sourceware.org/bugzilla/show_bug.cgi?id=16437 ... > So w.r.t. C11, the exported kernel timespec looks fine. But I think the > x32 kernel support (and the current ILP32 patches) assume a native > struct timespec with tv_nsec as 64-bit. > > If we are to be C11 conformant, glibc on x32 has a bug as it defines > timespec incorrectly. This hid a bug in the kernel handling the > corresponding x32 syscalls. What's the best fix for x32 I can't really > tell (we need people to agree on where the bugs are). > > At least for AArch64 ILP32 we are still free to change the user/kernel > ABI, so we could add wrappers for the affected syscalls to fix this up. > yes, afaik on x32 the 64bit kernel expects 64bit layout, arm64 can fix this > > While most of the other type changes proposed (I'm looking at > > https://lkml.org/lkml/2014/9/3/719) are permissible and simply > > ugly/undesirable, > > They may be ugly but definitely not undesirable ;). > several types have the same c level definition across all archs.. except x32 because of typedef long long __kernel_long_t; this should not cause posix/c conformance issues (as you noted timespec is ok in the uapi header only the kernel side behaviour is wrong) however note that the kernel documentation is explicit about some of the types and now x32 does not match those which you may consider as a documentation issue, but it can easily break existing code so at least some of the type changes are undesirable (now it's not clear what libc should do with these) http://man7.org/linux/man-pages/man2/sysinfo.2.html http://man7.org/linux/man-pages/man2/adjtimex.2.html http://man7.org/linux/man-pages/man2/getrusage.2.html > > Note that on aarch64 ILP32, the consequences of not fixing this right > > away will be much worse than on x32, since aarch64 (at least as I > > understand it) supports big endian where it's not just a matter of > > sign-extending the value from userspace and ignoring the padding, but > > rather changing the offset of the tv_nsec member. > > Indeed. > the ugliest (little endian only) workaround in glibc now is i think http://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/unix/sysv/linux/bits/resource.h;hb=HEAD#l183 > > Working around the discrepencies in userspace IS possible, but ugly. > > We do it in musl libc for x32 right now -- see: > > > > http://git.musl-libc.org/cgit/musl/tree/arch/x32/syscall_arch.h?id=v1.1.6 > > http://git.musl-libc.org/cgit/musl/tree/arch/x32/src/syscall_cp_fixup.c?id=v1.1.6 > > For AArch64 ILP32 I would rather see the fix-ups in kernel wrappers. > > Are you aware of other cases like this? > i know at least one android kernel issue: there is an ioctl for the alarm device that takes timespec argument (i think it's not in the mainline kernel and i guess android does not care about x32 so it was not an issue so far, but this is something that should not be fixed on the libc side) wrt. ioctl/fcntl another issue if there is ever a call that takes signed long or signed int argument which has to be signextended when doing a syscall (i think this is also a problem if userspace code uses syscall(2) directly, libc cannot possibly know where to signextend and the kernel side does not do the fixup right now) > (the rest of the comment below for Marcus' attention) > > > I imagine the workarounds in glibc might need to be considerably more > > widespread and uglier. > > > > Whatever happens on the kernel side, this needs to be coordinated with > > userspace (glibc, etc.) properly so that the type error (glibc bug > > 16437) is not propagated into a new target that we actually want > > people to use. I'd really like it if other undesirable type changes > > could be cleaned up too, but perhaps that's too much to ask from the > > kernel side. > > -- > Catalin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 10:33:32AM -0800, H.J. Lu wrote: > On Tue, Feb 10, 2015 at 10:13 AM, Rich Felker wrote: > > On Thu, 2 Oct 2014 at 16:52:18 +0100, Catalin Marinas wrote: > >> On Wed, Sep 03, 2014 at 10:18:54PM +0100, Andrew Pinski wrote: > >> > New version with all of the requested changes. Updated to the > >> > latest sources. > >> > > >> > Notable changes from the previous versions: > >> > VDSO code has been factored out to be easier to understand and > >> > easier to maintain. > >> > Move the config option to the last thing that gets added. > >> > Added some extra COMPAT_* macros for core dumping for easier usage. > >> > >> Apart from a few comments I've made, I would also like to see non-empty > >> commit logs and long line wrapping (both in commit logs and > >> Documentation/). Otherwise, the patches look fine. > >> > >> So what are the next steps? Are the glibc folk ok with the ILP32 Linux > >> ABI? On the kernel side, what I would like to see: > > > > I don't know if this has been discussed on libc-alpha yet or not, but > > I think we need to open a discussion of how it relates to open glibc > > bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): > > > > https://sourceware.org/bugzilla/show_bug.cgi?id=16437 > > Please leave x32 out of this discussion. I have resolved this bug > as WONTFIX. >From the glibc side, I thought things went by a consensus process these days, not the old WONTFIX regime of he who shall not be named. If this is not fixed for x32, then x32 cannot provide a conforming C environment and thus it's rather a toy target. But I think we should discuss this on libc-alpha. In the mean time please leave it REOPENED. Rich -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Tue, Feb 10, 2015 at 10:13 AM, Rich Felker wrote: > On Thu, 2 Oct 2014 at 16:52:18 +0100, Catalin Marinas wrote: >> On Wed, Sep 03, 2014 at 10:18:54PM +0100, Andrew Pinski wrote: >> > New version with all of the requested changes. Updated to the >> > latest sources. >> > >> > Notable changes from the previous versions: >> > VDSO code has been factored out to be easier to understand and >> > easier to maintain. >> > Move the config option to the last thing that gets added. >> > Added some extra COMPAT_* macros for core dumping for easier usage. >> >> Apart from a few comments I've made, I would also like to see non-empty >> commit logs and long line wrapping (both in commit logs and >> Documentation/). Otherwise, the patches look fine. >> >> So what are the next steps? Are the glibc folk ok with the ILP32 Linux >> ABI? On the kernel side, what I would like to see: > > I don't know if this has been discussed on libc-alpha yet or not, but > I think we need to open a discussion of how it relates to open glibc > bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): > > https://sourceware.org/bugzilla/show_bug.cgi?id=16437 Please leave x32 out of this discussion. I have resolved this bug as WONTFIX. > While most of the other type changes proposed (I'm looking at > https://lkml.org/lkml/2014/9/3/719) are permissible and simply > ugly/undesirable, defining struct timespec with tv_nsec having any > type other than long conflicts with the requirements of C11 and POSIX, > and WG14 is unlikely to be interested in changing the C language > because the Linux kernel has the wrong type in timespec. > > Note that on aarch64 ILP32, the consequences of not fixing this right > away will be much worse than on x32, since aarch64 (at least as I > understand it) supports big endian where it's not just a matter of > sign-extending the value from userspace and ignoring the padding, but > rather changing the offset of the tv_nsec member. > > Working around the discrepencies in userspace IS possible, but ugly. > We do it in musl libc for x32 right now -- see: > > http://git.musl-libc.org/cgit/musl/tree/arch/x32/syscall_arch.h?id=v1.1.6 > http://git.musl-libc.org/cgit/musl/tree/arch/x32/src/syscall_cp_fixup.c?id=v1.1.6 You are free to do what you feel appropriate. I have no plans to change x32 on this in glibc at this moment. > I imagine the workarounds in glibc might need to be considerably more > widespread and uglier. > > Whatever happens on the kernel side, this needs to be coordinated with > userspace (glibc, etc.) properly so that the type error (glibc bug > 16437) is not propagated into a new target that we actually want > people to use. I'd really like it if other undesirable type changes > could be cleaned up too, but perhaps that's too much to ask from the > kernel side. > > Rich -- H.J. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
(adding Marcus) On Tue, Feb 10, 2015 at 06:13:02PM +, Rich Felker wrote: > On Thu, 2 Oct 2014 at 16:52:18 +0100, Catalin Marinas wrote: > > On Wed, Sep 03, 2014 at 10:18:54PM +0100, Andrew Pinski wrote: > > > New version with all of the requested changes. Updated to the > > > latest sources. > > > > > > Notable changes from the previous versions: > > > VDSO code has been factored out to be easier to understand and > > > easier to maintain. > > > Move the config option to the last thing that gets added. > > > Added some extra COMPAT_* macros for core dumping for easier usage. > > > > Apart from a few comments I've made, I would also like to see non-empty > > commit logs and long line wrapping (both in commit logs and > > Documentation/). Otherwise, the patches look fine. > > > > So what are the next steps? Are the glibc folk ok with the ILP32 Linux > > ABI? On the kernel side, what I would like to see: > > I don't know if this has been discussed on libc-alpha yet or not, but > I think we need to open a discussion of how it relates to open glibc > bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): > > https://sourceware.org/bugzilla/show_bug.cgi?id=16437 I'm trying to understand the problem first. Quoting from the bug above (which I guess is quoted form C11): "The range and precision of times representable in clock_t and time_t are implementation-defined. The timespec structure shall contain at least the following members, in any order. time_t tv_sec; // whole seconds -- >= 0 long tv_nsec; // nanoseconds -- [0, 9]" So changing time_t to 64-bit is fine on x32. The timespec struct exported by the kernel always uses a long for tv_nsec. However, glibc uses __syscall_slong_t which ends up as 64-bit for x32 (I guess it mirrors the __kernel_long_t definition). So w.r.t. C11, the exported kernel timespec looks fine. But I think the x32 kernel support (and the current ILP32 patches) assume a native struct timespec with tv_nsec as 64-bit. If we are to be C11 conformant, glibc on x32 has a bug as it defines timespec incorrectly. This hid a bug in the kernel handling the corresponding x32 syscalls. What's the best fix for x32 I can't really tell (we need people to agree on where the bugs are). At least for AArch64 ILP32 we are still free to change the user/kernel ABI, so we could add wrappers for the affected syscalls to fix this up. > While most of the other type changes proposed (I'm looking at > https://lkml.org/lkml/2014/9/3/719) are permissible and simply > ugly/undesirable, They may be ugly but definitely not undesirable ;). > defining struct timespec with tv_nsec having any type other than long > conflicts with the requirements of C11 and POSIX, and WG14 is unlikely > to be interested in changing the C language because the Linux kernel > has the wrong type in timespec. I agree. The strange thing is that the Linux exported headers are fine. > Note that on aarch64 ILP32, the consequences of not fixing this right > away will be much worse than on x32, since aarch64 (at least as I > understand it) supports big endian where it's not just a matter of > sign-extending the value from userspace and ignoring the padding, but > rather changing the offset of the tv_nsec member. Indeed. > Working around the discrepencies in userspace IS possible, but ugly. > We do it in musl libc for x32 right now -- see: > > http://git.musl-libc.org/cgit/musl/tree/arch/x32/syscall_arch.h?id=v1.1.6 > http://git.musl-libc.org/cgit/musl/tree/arch/x32/src/syscall_cp_fixup.c?id=v1.1.6 For AArch64 ILP32 I would rather see the fix-ups in kernel wrappers. Are you aware of other cases like this? (the rest of the comment below for Marcus' attention) > I imagine the workarounds in glibc might need to be considerably more > widespread and uglier. > > Whatever happens on the kernel side, this needs to be coordinated with > userspace (glibc, etc.) properly so that the type error (glibc bug > 16437) is not propagated into a new target that we actually want > people to use. I'd really like it if other undesirable type changes > could be cleaned up too, but perhaps that's too much to ask from the > kernel side. -- Catalin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
* Catalin Marinas catalin.mari...@arm.com [2015-02-11 17:39:19 +]: (adding Marcus) On Tue, Feb 10, 2015 at 06:13:02PM +, Rich Felker wrote: I don't know if this has been discussed on libc-alpha yet or not, but I think we need to open a discussion of how it relates to open glibc bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): https://sourceware.org/bugzilla/show_bug.cgi?id=16437 ... So w.r.t. C11, the exported kernel timespec looks fine. But I think the x32 kernel support (and the current ILP32 patches) assume a native struct timespec with tv_nsec as 64-bit. If we are to be C11 conformant, glibc on x32 has a bug as it defines timespec incorrectly. This hid a bug in the kernel handling the corresponding x32 syscalls. What's the best fix for x32 I can't really tell (we need people to agree on where the bugs are). At least for AArch64 ILP32 we are still free to change the user/kernel ABI, so we could add wrappers for the affected syscalls to fix this up. yes, afaik on x32 the 64bit kernel expects 64bit layout, arm64 can fix this While most of the other type changes proposed (I'm looking at https://lkml.org/lkml/2014/9/3/719) are permissible and simply ugly/undesirable, They may be ugly but definitely not undesirable ;). several types have the same c level definition across all archs.. except x32 because of typedef long long __kernel_long_t; this should not cause posix/c conformance issues (as you noted timespec is ok in the uapi header only the kernel side behaviour is wrong) however note that the kernel documentation is explicit about some of the types and now x32 does not match those which you may consider as a documentation issue, but it can easily break existing code so at least some of the type changes are undesirable (now it's not clear what libc should do with these) http://man7.org/linux/man-pages/man2/sysinfo.2.html http://man7.org/linux/man-pages/man2/adjtimex.2.html http://man7.org/linux/man-pages/man2/getrusage.2.html Note that on aarch64 ILP32, the consequences of not fixing this right away will be much worse than on x32, since aarch64 (at least as I understand it) supports big endian where it's not just a matter of sign-extending the value from userspace and ignoring the padding, but rather changing the offset of the tv_nsec member. Indeed. the ugliest (little endian only) workaround in glibc now is i think http://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/unix/sysv/linux/bits/resource.h;hb=HEAD#l183 Working around the discrepencies in userspace IS possible, but ugly. We do it in musl libc for x32 right now -- see: http://git.musl-libc.org/cgit/musl/tree/arch/x32/syscall_arch.h?id=v1.1.6 http://git.musl-libc.org/cgit/musl/tree/arch/x32/src/syscall_cp_fixup.c?id=v1.1.6 For AArch64 ILP32 I would rather see the fix-ups in kernel wrappers. Are you aware of other cases like this? i know at least one android kernel issue: there is an ioctl for the alarm device that takes timespec argument (i think it's not in the mainline kernel and i guess android does not care about x32 so it was not an issue so far, but this is something that should not be fixed on the libc side) wrt. ioctl/fcntl another issue if there is ever a call that takes signed long or signed int argument which has to be signextended when doing a syscall (i think this is also a problem if userspace code uses syscall(2) directly, libc cannot possibly know where to signextend and the kernel side does not do the fixup right now) (the rest of the comment below for Marcus' attention) I imagine the workarounds in glibc might need to be considerably more widespread and uglier. Whatever happens on the kernel side, this needs to be coordinated with userspace (glibc, etc.) properly so that the type error (glibc bug 16437) is not propagated into a new target that we actually want people to use. I'd really like it if other undesirable type changes could be cleaned up too, but perhaps that's too much to ask from the kernel side. -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 11:16:58AM -0800, H.J. Lu wrote: I don't know if this has been discussed on libc-alpha yet or not, but I think we need to open a discussion of how it relates to open glibc bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): https://sourceware.org/bugzilla/show_bug.cgi?id=16437 Please leave x32 out of this discussion. I have resolved this bug as WONTFIX. From the glibc side, I thought things went by a consensus process these days, not the old WONTFIX regime of he who shall not be named. If this is not fixed for x32, then x32 cannot provide a conforming C environment and thus it's rather a toy target. But I think we should discuss this on libc-alpha. In the mean time please leave it REOPENED. As I said in PR, the issue has been raised in Mar, 2012 when the x32 port was submitted. It has been decided that x32 won't conform to tv_nsec, blksize_t, and suseconds_t as long. I don't believe we will change them to conform to POSIX. I briefly reviewed that discussion and I think the decision made was about an obscure POSIX requirement about supporting at least one compilation environment where certain types have rank = long. This is trivially satisfied if you consider x32 and x86_64 separate compilation environments, but it's not related to the core issue: that the definition of timespec violates core (not obscure) requirements of both POSIX and C11. At the time you were probably unaware of the C11 requirement. Note that it's a LOT harder to effect change in the C standard, so even if the Austin Group would be amenable to changing the requirement for timespec to allow something like nseconds_t, getting WG14 to make this change to work around a Linux/glibc mistake does not sound practical. Rich -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 11:05 AM, Szabolcs Nagy n...@port70.net wrote: i know at least one android kernel issue: there is an ioctl for the alarm device that takes timespec argument (i think it's not in the mainline kernel and i guess android does not care about x32 so it was not an issue so far, but this is something that should not be fixed on the libc side) I have documented some remaining x32 kernel issues at: https://sites.google.com/site/x32abi/ -- H.J. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
(adding Marcus) On Tue, Feb 10, 2015 at 06:13:02PM +, Rich Felker wrote: On Thu, 2 Oct 2014 at 16:52:18 +0100, Catalin Marinas wrote: On Wed, Sep 03, 2014 at 10:18:54PM +0100, Andrew Pinski wrote: New version with all of the requested changes. Updated to the latest sources. Notable changes from the previous versions: VDSO code has been factored out to be easier to understand and easier to maintain. Move the config option to the last thing that gets added. Added some extra COMPAT_* macros for core dumping for easier usage. Apart from a few comments I've made, I would also like to see non-empty commit logs and long line wrapping (both in commit logs and Documentation/). Otherwise, the patches look fine. So what are the next steps? Are the glibc folk ok with the ILP32 Linux ABI? On the kernel side, what I would like to see: I don't know if this has been discussed on libc-alpha yet or not, but I think we need to open a discussion of how it relates to open glibc bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): https://sourceware.org/bugzilla/show_bug.cgi?id=16437 I'm trying to understand the problem first. Quoting from the bug above (which I guess is quoted form C11): The range and precision of times representable in clock_t and time_t are implementation-defined. The timespec structure shall contain at least the following members, in any order. time_t tv_sec; // whole seconds -- = 0 long tv_nsec; // nanoseconds -- [0, 9] So changing time_t to 64-bit is fine on x32. The timespec struct exported by the kernel always uses a long for tv_nsec. However, glibc uses __syscall_slong_t which ends up as 64-bit for x32 (I guess it mirrors the __kernel_long_t definition). So w.r.t. C11, the exported kernel timespec looks fine. But I think the x32 kernel support (and the current ILP32 patches) assume a native struct timespec with tv_nsec as 64-bit. If we are to be C11 conformant, glibc on x32 has a bug as it defines timespec incorrectly. This hid a bug in the kernel handling the corresponding x32 syscalls. What's the best fix for x32 I can't really tell (we need people to agree on where the bugs are). At least for AArch64 ILP32 we are still free to change the user/kernel ABI, so we could add wrappers for the affected syscalls to fix this up. While most of the other type changes proposed (I'm looking at https://lkml.org/lkml/2014/9/3/719) are permissible and simply ugly/undesirable, They may be ugly but definitely not undesirable ;). defining struct timespec with tv_nsec having any type other than long conflicts with the requirements of C11 and POSIX, and WG14 is unlikely to be interested in changing the C language because the Linux kernel has the wrong type in timespec. I agree. The strange thing is that the Linux exported headers are fine. Note that on aarch64 ILP32, the consequences of not fixing this right away will be much worse than on x32, since aarch64 (at least as I understand it) supports big endian where it's not just a matter of sign-extending the value from userspace and ignoring the padding, but rather changing the offset of the tv_nsec member. Indeed. Working around the discrepencies in userspace IS possible, but ugly. We do it in musl libc for x32 right now -- see: http://git.musl-libc.org/cgit/musl/tree/arch/x32/syscall_arch.h?id=v1.1.6 http://git.musl-libc.org/cgit/musl/tree/arch/x32/src/syscall_cp_fixup.c?id=v1.1.6 For AArch64 ILP32 I would rather see the fix-ups in kernel wrappers. Are you aware of other cases like this? (the rest of the comment below for Marcus' attention) I imagine the workarounds in glibc might need to be considerably more widespread and uglier. Whatever happens on the kernel side, this needs to be coordinated with userspace (glibc, etc.) properly so that the type error (glibc bug 16437) is not propagated into a new target that we actually want people to use. I'd really like it if other undesirable type changes could be cleaned up too, but perhaps that's too much to ask from the kernel side. -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 11:25 AM, Rich Felker dal...@libc.org wrote: On Wed, Feb 11, 2015 at 11:16:58AM -0800, H.J. Lu wrote: I don't know if this has been discussed on libc-alpha yet or not, but I think we need to open a discussion of how it relates to open glibc bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): https://sourceware.org/bugzilla/show_bug.cgi?id=16437 Please leave x32 out of this discussion. I have resolved this bug as WONTFIX. From the glibc side, I thought things went by a consensus process these days, not the old WONTFIX regime of he who shall not be named. If this is not fixed for x32, then x32 cannot provide a conforming C environment and thus it's rather a toy target. But I think we should discuss this on libc-alpha. In the mean time please leave it REOPENED. As I said in PR, the issue has been raised in Mar, 2012 when the x32 port was submitted. It has been decided that x32 won't conform to tv_nsec, blksize_t, and suseconds_t as long. I don't believe we will change them to conform to POSIX. I briefly reviewed that discussion and I think the decision made was about an obscure POSIX requirement about supporting at least one compilation environment where certain types have rank = long. This is The example you gave in PR is similar to https://sourceware.org/ml/libc-alpha/2012-03/msg00456.html trivially satisfied if you consider x32 and x86_64 separate compilation environments, but it's not related to the core issue: that the definition of timespec violates core (not obscure) requirements of both POSIX and C11. At the time you were probably unaware of the C11 requirement. Note that it's a LOT harder to effect change in the C standard, so even if the Austin Group would be amenable to changing the requirement for timespec to allow something like nseconds_t, getting WG14 to make this change to work around a Linux/glibc mistake does not sound practical. That is very unfortunate. I consider it is too late for x32 to change. -- H.J. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Tue, Feb 10, 2015 at 10:13 AM, Rich Felker dal...@libc.org wrote: On Thu, 2 Oct 2014 at 16:52:18 +0100, Catalin Marinas wrote: On Wed, Sep 03, 2014 at 10:18:54PM +0100, Andrew Pinski wrote: New version with all of the requested changes. Updated to the latest sources. Notable changes from the previous versions: VDSO code has been factored out to be easier to understand and easier to maintain. Move the config option to the last thing that gets added. Added some extra COMPAT_* macros for core dumping for easier usage. Apart from a few comments I've made, I would also like to see non-empty commit logs and long line wrapping (both in commit logs and Documentation/). Otherwise, the patches look fine. So what are the next steps? Are the glibc folk ok with the ILP32 Linux ABI? On the kernel side, what I would like to see: I don't know if this has been discussed on libc-alpha yet or not, but I think we need to open a discussion of how it relates to open glibc bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): https://sourceware.org/bugzilla/show_bug.cgi?id=16437 Please leave x32 out of this discussion. I have resolved this bug as WONTFIX. While most of the other type changes proposed (I'm looking at https://lkml.org/lkml/2014/9/3/719) are permissible and simply ugly/undesirable, defining struct timespec with tv_nsec having any type other than long conflicts with the requirements of C11 and POSIX, and WG14 is unlikely to be interested in changing the C language because the Linux kernel has the wrong type in timespec. Note that on aarch64 ILP32, the consequences of not fixing this right away will be much worse than on x32, since aarch64 (at least as I understand it) supports big endian where it's not just a matter of sign-extending the value from userspace and ignoring the padding, but rather changing the offset of the tv_nsec member. Working around the discrepencies in userspace IS possible, but ugly. We do it in musl libc for x32 right now -- see: http://git.musl-libc.org/cgit/musl/tree/arch/x32/syscall_arch.h?id=v1.1.6 http://git.musl-libc.org/cgit/musl/tree/arch/x32/src/syscall_cp_fixup.c?id=v1.1.6 You are free to do what you feel appropriate. I have no plans to change x32 on this in glibc at this moment. I imagine the workarounds in glibc might need to be considerably more widespread and uglier. Whatever happens on the kernel side, this needs to be coordinated with userspace (glibc, etc.) properly so that the type error (glibc bug 16437) is not propagated into a new target that we actually want people to use. I'd really like it if other undesirable type changes could be cleaned up too, but perhaps that's too much to ask from the kernel side. Rich -- H.J. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 11:02 AM, Rich Felker dal...@libc.org wrote: On Wed, Feb 11, 2015 at 10:33:32AM -0800, H.J. Lu wrote: On Tue, Feb 10, 2015 at 10:13 AM, Rich Felker dal...@libc.org wrote: On Thu, 2 Oct 2014 at 16:52:18 +0100, Catalin Marinas wrote: On Wed, Sep 03, 2014 at 10:18:54PM +0100, Andrew Pinski wrote: New version with all of the requested changes. Updated to the latest sources. Notable changes from the previous versions: VDSO code has been factored out to be easier to understand and easier to maintain. Move the config option to the last thing that gets added. Added some extra COMPAT_* macros for core dumping for easier usage. Apart from a few comments I've made, I would also like to see non-empty commit logs and long line wrapping (both in commit logs and Documentation/). Otherwise, the patches look fine. So what are the next steps? Are the glibc folk ok with the ILP32 Linux ABI? On the kernel side, what I would like to see: I don't know if this has been discussed on libc-alpha yet or not, but I think we need to open a discussion of how it relates to open glibc bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): https://sourceware.org/bugzilla/show_bug.cgi?id=16437 Please leave x32 out of this discussion. I have resolved this bug as WONTFIX. From the glibc side, I thought things went by a consensus process these days, not the old WONTFIX regime of he who shall not be named. If this is not fixed for x32, then x32 cannot provide a conforming C environment and thus it's rather a toy target. But I think we should discuss this on libc-alpha. In the mean time please leave it REOPENED. As I said in PR, the issue has been raised in Mar, 2012 when the x32 port was submitted. It has been decided that x32 won't conform to tv_nsec, blksize_t, and suseconds_t as long. I don't believe we will change them to conform to POSIX. As for if x32 is a toy target or not, it will be decided by whether it delivers what users are looking for, not by if tv_nsec, blksize_t, and suseconds_t conform to POSIX. -- H.J. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 10:33:32AM -0800, H.J. Lu wrote: On Tue, Feb 10, 2015 at 10:13 AM, Rich Felker dal...@libc.org wrote: On Thu, 2 Oct 2014 at 16:52:18 +0100, Catalin Marinas wrote: On Wed, Sep 03, 2014 at 10:18:54PM +0100, Andrew Pinski wrote: New version with all of the requested changes. Updated to the latest sources. Notable changes from the previous versions: VDSO code has been factored out to be easier to understand and easier to maintain. Move the config option to the last thing that gets added. Added some extra COMPAT_* macros for core dumping for easier usage. Apart from a few comments I've made, I would also like to see non-empty commit logs and long line wrapping (both in commit logs and Documentation/). Otherwise, the patches look fine. So what are the next steps? Are the glibc folk ok with the ILP32 Linux ABI? On the kernel side, what I would like to see: I don't know if this has been discussed on libc-alpha yet or not, but I think we need to open a discussion of how it relates to open glibc bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): https://sourceware.org/bugzilla/show_bug.cgi?id=16437 Please leave x32 out of this discussion. I have resolved this bug as WONTFIX. From the glibc side, I thought things went by a consensus process these days, not the old WONTFIX regime of he who shall not be named. If this is not fixed for x32, then x32 cannot provide a conforming C environment and thus it's rather a toy target. But I think we should discuss this on libc-alpha. In the mean time please leave it REOPENED. Rich -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 05:39:19PM +, Catalin Marinas wrote: (adding Marcus) On Tue, Feb 10, 2015 at 06:13:02PM +, Rich Felker wrote: On Thu, 2 Oct 2014 at 16:52:18 +0100, Catalin Marinas wrote: On Wed, Sep 03, 2014 at 10:18:54PM +0100, Andrew Pinski wrote: New version with all of the requested changes. Updated to the latest sources. Notable changes from the previous versions: VDSO code has been factored out to be easier to understand and easier to maintain. Move the config option to the last thing that gets added. Added some extra COMPAT_* macros for core dumping for easier usage. Apart from a few comments I've made, I would also like to see non-empty commit logs and long line wrapping (both in commit logs and Documentation/). Otherwise, the patches look fine. So what are the next steps? Are the glibc folk ok with the ILP32 Linux ABI? On the kernel side, what I would like to see: I don't know if this has been discussed on libc-alpha yet or not, but I think we need to open a discussion of how it relates to open glibc bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): https://sourceware.org/bugzilla/show_bug.cgi?id=16437 I'm trying to understand the problem first. Quoting from the bug above (which I guess is quoted form C11): The range and precision of times representable in clock_t and time_t are implementation-defined. The timespec structure shall contain at least the following members, in any order. time_t tv_sec; // whole seconds -- = 0 long tv_nsec; // nanoseconds -- [0, 9] So changing time_t to 64-bit is fine on x32. The timespec struct exported by the kernel always uses a long for tv_nsec. However, glibc uses __syscall_slong_t which ends up as 64-bit for x32 (I guess it mirrors the __kernel_long_t definition). So w.r.t. C11, the exported kernel timespec looks fine. But I think the x32 kernel support (and the current ILP32 patches) assume a native struct timespec with tv_nsec as 64-bit. The exported kernel timespec is not fine if long is defined as a 32-bit type, which it is for x32 and the proposed aarch64-ILP32 ABIs. The type being long means that code like long *p = ts-tv_nsec; *p = 42; is valid. With the proposed definition of timespec, this results in a compiler warning or error on valid code, which is better than dangerously wrong code generation, but we can easily hide the warning via: void *p = ts-tv_nsec; long *q = p; *q = 42; Imagine this happening in places such as with callbacks that take void pointers, for example pthread_create. But even if the breakage could always be diagnosed by the compiler, you're still breaking perfectly valid, conforming C programs. If we are to be C11 conformant, glibc on x32 has a bug as it defines timespec incorrectly. This hid a bug in the kernel handling the corresponding x32 syscalls. What's the best fix for x32 I can't really tell (we need people to agree on where the bugs are). For the kernel, it should be casting the value to int/int32_t to ignore the junk in the upper bits. But that only works on little endian. On big endian it's a bigger mess. However at this point fixing it on the kernel side is not very useful for x32, at least not right away. Since people will still be running old kernels with the wrong behavior, the fixup has to happen in userspace to support those old kernels until the libc drops support for kernels too old to handle it right. At least for AArch64 ILP32 we are still free to change the user/kernel ABI, so we could add wrappers for the affected syscalls to fix this up. Yes. That's what I'd like to see before aarch64-ILP32 is officially launched. It's a lot harder to retroactively fix this. Right now it's fairly easy -- just a small wrapper/fixup layer in the kernel. While most of the other type changes proposed (I'm looking at https://lkml.org/lkml/2014/9/3/719) are permissible and simply ugly/undesirable, They may be ugly but definitely not undesirable ;). I can point you to a few other cases that may be undesirable, but less severe than timespec: - https://sourceware.org/bugzilla/show_bug.cgi?id=16438 - http://man7.org/linux/man-pages/man2/sysinfo.2.html In the case of sysinfo, the public API documentation documents the fields as having type unsigned long, but on x32 (and presumably aarch64-ILP32 as proposed) they have type unsigned long long. I can probably find more examples if you're interested. defining struct timespec with tv_nsec having any type other than long conflicts with the requirements of C11 and POSIX, and WG14 is unlikely to be interested in changing the C language because the Linux kernel has the wrong type in timespec. I agree. The strange thing is that the Linux exported headers are fine. I'm confused why you think they're fine. Note that on aarch64 ILP32, the
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 11:34:23AM -0800, H.J. Lu wrote: On Wed, Feb 11, 2015 at 11:25 AM, Rich Felker dal...@libc.org wrote: On Wed, Feb 11, 2015 at 11:16:58AM -0800, H.J. Lu wrote: I don't know if this has been discussed on libc-alpha yet or not, but I think we need to open a discussion of how it relates to open glibc bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): https://sourceware.org/bugzilla/show_bug.cgi?id=16437 Please leave x32 out of this discussion. I have resolved this bug as WONTFIX. From the glibc side, I thought things went by a consensus process these days, not the old WONTFIX regime of he who shall not be named. If this is not fixed for x32, then x32 cannot provide a conforming C environment and thus it's rather a toy target. But I think we should discuss this on libc-alpha. In the mean time please leave it REOPENED. As I said in PR, the issue has been raised in Mar, 2012 when the x32 port was submitted. It has been decided that x32 won't conform to tv_nsec, blksize_t, and suseconds_t as long. I don't believe we will change them to conform to POSIX. I briefly reviewed that discussion and I think the decision made was about an obscure POSIX requirement about supporting at least one compilation environment where certain types have rank = long. This is The example you gave in PR is similar to https://sourceware.org/ml/libc-alpha/2012-03/msg00456.html Yes, but after that the conversation seemed to get derailed into the blksize_t etc. stuff about compilation environments that's largely irrelevant. I think this prevented the core tv_nsec issue from getting discussed further, unless I'm missing part of that thread. trivially satisfied if you consider x32 and x86_64 separate compilation environments, but it's not related to the core issue: that the definition of timespec violates core (not obscure) requirements of both POSIX and C11. At the time you were probably unaware of the C11 requirement. Note that it's a LOT harder to effect change in the C standard, so even if the Austin Group would be amenable to changing the requirement for timespec to allow something like nseconds_t, getting WG14 to make this change to work around a Linux/glibc mistake does not sound practical. That is very unfortunate. I consider it is too late for x32 to change. Why? It's hardly an incompatible ABI change, as long as the kernel/libc fills the upper bits (for old programs that read them based on the old headers) when structs are read from the kernel to the application, and ignores the upper bits (potentially set or left uninitialized by the application) when strings are passed from userspace to the kernel. Newly built apps using the struct definition with 32-bit tv_nsec would need new libc to ensure that the high bits aren't interpreted, but this could be handled by symbol versioning. Rich -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On 02/11/2015 11:57 AM, H.J. Lu wrote: trivially satisfied if you consider x32 and x86_64 separate compilation environments, but it's not related to the core issue: that the definition of timespec violates core (not obscure) requirements of both POSIX and C11. At the time you were probably unaware of the C11 requirement. Note that it's a LOT harder to effect change in the C standard, so even if the Austin Group would be amenable to changing the requirement for timespec to allow something like nseconds_t, getting WG14 to make this change to work around a Linux/glibc mistake does not sound practical. That is very unfortunate. I consider it is too late for x32 to change. Why? It's hardly an incompatible ABI change, as long as the kernel/libc fills the upper bits (for old programs that read them based on the old headers) when structs are read from the kernel to the application, and ignores the upper bits (potentially set or left uninitialized by the application) when strings are passed from userspace to the kernel. Newly built apps using the struct definition with 32-bit tv_nsec would need new libc to ensure that the high bits aren't interpreted, but this could be handled by symbol versioning. We have considered this option. But since kernel wouldn't change tv_nsec/tv_usec handling just for x32, it wasn't selected. Did anyone *ask* the kernel people (e.g. hpa)? --Andy -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
trivially satisfied if you consider x32 and x86_64 separate compilation environments, but it's not related to the core issue: that the definition of timespec violates core (not obscure) requirements of both POSIX and C11. At the time you were probably unaware of the C11 requirement. Note that it's a LOT harder to effect change in the C standard, so even if the Austin Group would be amenable to changing the requirement for timespec to allow something like nseconds_t, getting WG14 to make this change to work around a Linux/glibc mistake does not sound practical. That is very unfortunate. I consider it is too late for x32 to change. Why? It's hardly an incompatible ABI change, as long as the kernel/libc fills the upper bits (for old programs that read them based on the old headers) when structs are read from the kernel to the application, and ignores the upper bits (potentially set or left uninitialized by the application) when strings are passed from userspace to the kernel. Newly built apps using the struct definition with 32-bit tv_nsec would need new libc to ensure that the high bits aren't interpreted, but this could be handled by symbol versioning. We have considered this option. But since kernel wouldn't change tv_nsec/tv_usec handling just for x32, it wasn't selected. -- H.J. -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 08:50:06PM +0100, a...@arndb.de wrote:
At least for AArch64 ILP32 we are still free to change the user/kernel
ABI, so we could add wrappers for the affected syscalls to fix this up.
yes, afaik on x32 the 64bit kernel expects 64bit layout,
arm64 can fix this
We have to fix it on all 32-bit architectures when we move to 64-bit time_t.
I think ideally you'd want a user space definition like
typedef long long time_t;
struct timespec {
time_t tv_sec;
long long tv_nsec;
};
which is the only way to avoid passing uninitialized tv_nsec into the kernel
from arbitrary user space doing ioctl. This is of course against POSIX and
C99. Changing POSIX to allow it is probably easier than the C standard,
but we have a couple of years before we need to make this the default.
I don't see why you want it to be long long. There is no harm in
passing uninitialized padding to the kernel; the kernel just needs to
do the right thing and ignore it (or avoid reading it to begin with).
Changing the C standard in an incompatible way that invalidates
existing code is not preferable over fixing an implementation bug in
one implementation. Even if C16 or so changed the requirement, people
will still be looking to C11 (and even C99) for years or decades to
come. Alignment of code to language standards moves slowly.
The other direction, passing uninitialized data from the kernel to
userspace, would be dangerous. But it doesn't happen as long as the
userspace padding is positioned (in an endian-dependent manner) where
the high bits of the kernel type would lie. It could happen if you
used a separate conversion wrapper that ony wrote 32 bits, but if you
wanted to take that approach you'd just need the wrapper to also write
the padding field manually.
In the kernel headers, the current plan is to provide interfaces taking
structures
typedef long long __kernel_time64_t;
struct __kernel_timespec64_t {
__kernel_time64_t tv_sec;
long long tv_nsec;
};
at least for ioctls, to avoid the ambiguity with libc headers specifying
something else.
This seems hideous from an application standpoint. Application
programmers don't want to know, and shouldn't need to know, these
silly implementation details that make no sense except as historical
baggage. They should just be able to use struct timespec everywhere
and have it work.
Rich
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Feb 11, 2015 at 10:02:55PM +0100, a...@arndb.de wrote:
Rich Felker dal...@libc.org hat am 11. Februar 2015 um 21:12 geschrieben:
On Wed, Feb 11, 2015 at 08:50:06PM +0100, a...@arndb.de wrote:
At least for AArch64 ILP32 we are still free to change the user/kernel
ABI, so we could add wrappers for the affected syscalls to fix this
up.
yes, afaik on x32 the 64bit kernel expects 64bit layout,
arm64 can fix this
We have to fix it on all 32-bit architectures when we move to 64-bit
time_t.
I think ideally you'd want a user space definition like
typedef long long time_t;
struct timespec {
time_t tv_sec;
long long tv_nsec;
};
which is the only way to avoid passing uninitialized tv_nsec into the
kernel
from arbitrary user space doing ioctl. This is of course against POSIX and
C99. Changing POSIX to allow it is probably easier than the C standard,
but we have a couple of years before we need to make this the default.
I don't see why you want it to be long long. There is no harm in
passing uninitialized padding to the kernel; the kernel just needs to
do the right thing and ignore it (or avoid reading it to begin with).
This would however mean having three different implementations
in the kernel rather than just two: Every driver that can pass a timespec
with this model needs to handle the native 64-bit case (64/64), the legacy
32-bit case (32/32) and the y2038-safe case (64/32). Most code can
already handle the first two, and none today handles the third. If you
want to make the handling explicitly incompatible with native 64-bit
mode, you get a lot of untested code in obscure places that are never
tested properly, while using the normal behavior in the kernel at least
gives us the same bugs that we already have on native 64-bit systems.
Would it really be that hard to do:
if (ILP32_on_64_process) tv_nsec = (int)tv_nsec;
or similar? That's all that's needed.
In some cases, there may also be a measurable performance penalty
in interpreting a user space data structure manually over copying
it (including the timespec values) in one chunk.
I don't think the above would be measurable.
An alternative would be to change the native 64-bit case to ignore the upper
half of tv_nsec and always just copy the low bits. This should work
fine almost all of the time, but I fear that there might be corner cases
where existing 64-bit user space depends on passing large or negative
tv_nsec values into the kernel.
Most functions using caller-provided timespecs are required to
diagnose invalid forms with EINVAL when tv_nsec=10 or 0, so
if the kernel examines only the low 32 bits on ABIs where long is
64-bit, userspace would need to be responsible for doing this
checking.
The other direction, passing uninitialized data from the kernel to
userspace, would be dangerous. But it doesn't happen as long as the
userspace padding is positioned (in an endian-dependent manner) where
the high bits of the kernel type would lie. It could happen if you
used a separate conversion wrapper that ony wrote 32 bits, but if you
wanted to take that approach you'd just need the wrapper to also write
the padding field manually.
Going from kernel to user space should not be an issue as long as we
always just write two 64-bit words, and this will zero-fill the upper half.
Agreed.
In the kernel headers, the current plan is to provide interfaces taking
structures
typedef long long __kernel_time64_t;
struct __kernel_timespec64_t {
__kernel_time64_t tv_sec;
long long tv_nsec;
};
at least for ioctls, to avoid the ambiguity with libc headers specifying
something else.
This seems hideous from an application standpoint. Application
programmers don't want to know, and shouldn't need to know, these
silly implementation details that make no sense except as historical
baggage. They should just be able to use struct timespec everywhere
and have it work.
The kernel does not even know how timespec is defined by libc, and we have
to at least be able to handle the common cases of timespec being 32/32
and 64/64 (or 64/32 plus explicit padding). For system calls, we can rely
on libc calling the syscalls that match the definition (or convert the
structure as necessary), while for ioctl the command number is chosen
by the application and has to match the structure definition provided in
the same header.
Generally I would think the kernel knows the model the process is
using, but if not, all you need is separate ioctl numbers for
userspace to use depending on which definition it's using.
In a lot of cases, the ioctl command number is defined (correctly) using the
_IOR/_IOW macros that take the size of the structure into account, but then
you also have cases where you get indirect pointers and the size of data
structure
passed by the ioctl command is independent of the
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, 11 Feb 2015, Rich Felker wrote: https://sourceware.org/bugzilla/show_bug.cgi?id=16437 Please leave x32 out of this discussion. I have resolved this bug as WONTFIX. From the glibc side, I thought things went by a consensus process these days, not the old WONTFIX regime of he who shall not be named. If this is not fixed for x32, then x32 cannot provide a conforming C environment and thus it's rather a toy target. But I think we should discuss this on libc-alpha. In the mean time please leave it REOPENED. Indeed. x86 is handled primarily by community review, and even when we have maintainers for architectures or other subsystems, being maintainer serves as a shortcut to presume consensus in the absence of controversy (in the expectation that the community won't object), not to override community discussion if something is more controversial. I've reopened the bug. I believe I made clear in the discussion of 64-bit time interfaces for 32-bit systems that the x32 ABI mistake was not one to be repeated - that since there is obviously no need for nanoseconds values that cannot fit in 32 bits, nanoseconds (and microseconds) values should remain as long in accordance with POSIX. It's absolutely fine for the userspace structures to have an explicit __glibc_reserved padding field in an endian-dependent place to keep the low part of the nanoseconds value in the same place as it would be for a 64-bit type, but if the kernel doesn't ignore that padding for the 64-bit time interfaces then all places that pass these structures from glibc to the kernel need to copy them and zero the padding in the copy. Whether the high 32 bits can be treated as padding for interfaces where long is 64-bit depends on whether the interfaces in question are required to return an error such as EINVAL for out-of-range nanoseconds values. -- Joseph S. Myers jos...@codesourcery.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
Szabolcs Nagy n...@port70.net hat am 11. Februar 2015 um 20:05 geschrieben:
* Catalin Marinas catalin.mari...@arm.com [2015-02-11 17:39:19 +]:
(adding Marcus)
On Tue, Feb 10, 2015 at 06:13:02PM +, Rich Felker wrote:
I don't know if this has been discussed on libc-alpha yet or not, but
I think we need to open a discussion of how it relates to open glibc
bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64):
https://sourceware.org/bugzilla/show_bug.cgi?id=16437
...
So w.r.t. C11, the exported kernel timespec looks fine. But I think the
x32 kernel support (and the current ILP32 patches) assume a native
struct timespec with tv_nsec as 64-bit.
If we are to be C11 conformant, glibc on x32 has a bug as it defines
timespec incorrectly. This hid a bug in the kernel handling the
corresponding x32 syscalls. What's the best fix for x32 I can't really
tell (we need people to agree on where the bugs are).
At least for AArch64 ILP32 we are still free to change the user/kernel
ABI, so we could add wrappers for the affected syscalls to fix this up.
yes, afaik on x32 the 64bit kernel expects 64bit layout,
arm64 can fix this
We have to fix it on all 32-bit architectures when we move to 64-bit time_t.
I think ideally you'd want a user space definition like
typedef long long time_t;
struct timespec {
time_t tv_sec;
long long tv_nsec;
};
which is the only way to avoid passing uninitialized tv_nsec into the kernel
from arbitrary user space doing ioctl. This is of course against POSIX and
C99. Changing POSIX to allow it is probably easier than the C standard,
but we have a couple of years before we need to make this the default.
In the kernel headers, the current plan is to provide interfaces taking
structures
typedef long long __kernel_time64_t;
struct __kernel_timespec64_t {
__kernel_time64_t tv_sec;
long long tv_nsec;
};
at least for ioctls, to avoid the ambiguity with libc headers specifying
something else.
A libc could use other definitions with added padding for what it exports
to user space though, or even make this depend on compile-time flags to
determine whether to make tv_nsec as long long, or to insert padding
in the right place based on endianess.
While most of the other type changes proposed (I'm looking at
https://lkml.org/lkml/2014/9/3/719) are permissible and simply
ugly/undesirable,
They may be ugly but definitely not undesirable ;).
several types have the same c level definition across all archs..
except x32 because of
typedef long long __kernel_long_t;
this should not cause posix/c conformance issues (as you noted
timespec is ok in the uapi header only the kernel side behaviour
is wrong)
The kernel behavior is right for the syscalls except ioctl, the uapi
header is wrong (not according to C99, but according to common sense)
and needs to be changed in order to work with big-endian. For x32,
I wonder if we can just #define timespec as __kernel_timespec64
once we introduce that.
Working around the discrepencies in userspace IS possible, but ugly.
We do it in musl libc for x32 right now -- see:
http://git.musl-libc.org/cgit/musl/tree/arch/x32/syscall_arch.h?id=v1.1.6
http://git.musl-libc.org/cgit/musl/tree/arch/x32/src/syscall_cp_fixup.c?id=v1.1.6
For AArch64 ILP32 I would rather see the fix-ups in kernel wrappers.
Are you aware of other cases like this?
i know at least one android kernel issue: there is an ioctl for the
alarm device that takes timespec argument
(i think it's not in the mainline kernel and i guess android does
not care about x32 so it was not an issue so far, but this is something
that should not be fixed on the libc side)
ioctl is a known problem with x32 and the ARM ILP32 support. This is
not limited to timespec but to any driver that uses an ioctl with
a data structure that includes a __kernel_long_t or __kernel_ulong_t
argument. There are a couple of drivers using these, and we either
need to change the structures to use 'long' instead, or fix the driver
to be aware of the difference between old-style 32-bit compat and
x32-style compat ioctl handling.
The types affected by this are
include/uapi/asm-generic/posix_types.h:typedef __kernel_ulong_t __kernel_ino_t;
include/uapi/asm-generic/posix_types.h:typedef __kernel_ulong_t __kernel_size_t;
include/uapi/asm-generic/posix_types.h:typedef __kernel_long_t
__kernel_suseconds_t;
include/uapi/asm-generic/posix_types.h:typedef __kernel_long_t
__kernel_ssize_t;
include/uapi/asm-generic/posix_types.h:typedef __kernel_long_t
__kernel_ptrdiff_t;
include/uapi/asm-generic/posix_types.h:typedef __kernel_long_t __kernel_off_t;
include/uapi/asm-generic/posix_types.h:typedef __kernel_long_t __kernel_time_t;
include/uapi/asm-generic/posix_types.h:typedef __kernel_long_t
__kernel_clock_t;
and anything derived from this.
Arnd
--
To unsubscribe from this list: send the
Re: [musl] Re: [PATCHv3 00/24] ILP32 support in ARM64
Sorry about the HTML mail, I'm currently travelling without access to my regular
mail client.
a...@arndb.de a...@arndb.de hat am 11. Februar 2015 um 22:02 geschrieben:
Rich Felker dal...@libc.org hat am 11. Februar 2015 um 21:12 geschrieben:
On Wed, Feb 11, 2015 at 08:50:06PM +0100, a...@arndb.de wrote:
At least for AArch64 ILP32 we are still free to change the
user/kernel
ABI, so we could add wrappers for the affected syscalls to fix this
up.
yes, afaik on x32 the 64bit kernel expects 64bit layout,
arm64 can fix this
We have to fix it on all 32-bit architectures when we move to 64-bit
time_t.
I think ideally you'd want a user space definition like
typedef long long time_t;
struct timespec {
time_t tv_sec;
long long tv_nsec;
};
which is the only way to avoid passing uninitialized tv_nsec into the
kernel
from arbitrary user space doing ioctl. This is of course against POSIX
and
C99. Changing POSIX to allow it is probably easier than the C standard,
but we have a couple of years before we need to make this the default.
I don't see why you want it to be long long. There is no harm in
passing uninitialized padding to the kernel; the kernel just needs to
do the right thing and ignore it (or avoid reading it to begin with).
This would however mean having three different implementations
in the kernel rather than just two: Every driver that can pass a timespec
with this model needs to handle the native 64-bit case (64/64), the legacy
32-bit case (32/32) and the y2038-safe case (64/32). Most code can
already handle the first two, and none today handles the third. If you
want to make the handling explicitly incompatible with native 64-bit
mode, you get a lot of untested code in obscure places that are never
tested properly, while using the normal behavior in the kernel at least
gives us the same bugs that we already have on native 64-bit systems.
In some cases, there may also be a measurable performance penalty
in interpreting a user space data structure manually over copying
it (including the timespec values) in one chunk.
An alternative would be to change the native 64-bit case to ignore the upper
half of tv_nsec and always just copy the low bits. This should work
fine almost all of the time, but I fear that there might be corner cases
where existing 64-bit user space depends on passing large or negative
tv_nsec values into the kernel.
The other direction, passing uninitialized data from the kernel to
userspace, would be dangerous. But it doesn't happen as long as the
userspace padding is positioned (in an endian-dependent manner) where
the high bits of the kernel type would lie. It could happen if you
used a separate conversion wrapper that ony wrote 32 bits, but if you
wanted to take that approach you'd just need the wrapper to also write
the padding field manually.
Going from kernel to user space should not be an issue as long as we
always just write two 64-bit words, and this will zero-fill the upper half.
In the kernel headers, the current plan is to provide interfaces taking
structures
typedef long long __kernel_time64_t;
struct __kernel_timespec64_t {
__kernel_time64_t tv_sec;
long long tv_nsec;
};
at least for ioctls, to avoid the ambiguity with libc headers specifying
something else.
This seems hideous from an application standpoint. Application
programmers don't want to know, and shouldn't need to know, these
silly implementation details that make no sense except as historical
baggage. They should just be able to use struct timespec everywhere
and have it work.
The kernel does not even know how timespec is defined by libc, and we have
to at least be able to handle the common cases of timespec being 32/32
and 64/64 (or 64/32 plus explicit padding). For system calls, we can rely
on libc calling the syscalls that match the definition (or convert the
structure as necessary), while for ioctl the command number is chosen
by the application and has to match the structure definition provided in
the same header.
In a lot of cases, the ioctl command number is defined (correctly) using the
_IOR/_IOW macros that take the size of the structure into account, but then
you also have cases where you get indirect pointers and the size of data
structure
passed by the ioctl command is independent of the size of timespec or time_t.
This is not just limited to time_t, we have a lot of data types for which we
define
__kernel_*_t types for this purpose, to deal with ioctls that need a specific
layout independent of what libc uses.
Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Thu, 2 Oct 2014 at 16:52:18 +0100, Catalin Marinas wrote: > On Wed, Sep 03, 2014 at 10:18:54PM +0100, Andrew Pinski wrote: > > New version with all of the requested changes. Updated to the > > latest sources. > > > > Notable changes from the previous versions: > > VDSO code has been factored out to be easier to understand and > > easier to maintain. > > Move the config option to the last thing that gets added. > > Added some extra COMPAT_* macros for core dumping for easier usage. > > Apart from a few comments I've made, I would also like to see non-empty > commit logs and long line wrapping (both in commit logs and > Documentation/). Otherwise, the patches look fine. > > So what are the next steps? Are the glibc folk ok with the ILP32 Linux > ABI? On the kernel side, what I would like to see: I don't know if this has been discussed on libc-alpha yet or not, but I think we need to open a discussion of how it relates to open glibc bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): https://sourceware.org/bugzilla/show_bug.cgi?id=16437 While most of the other type changes proposed (I'm looking at https://lkml.org/lkml/2014/9/3/719) are permissible and simply ugly/undesirable, defining struct timespec with tv_nsec having any type other than long conflicts with the requirements of C11 and POSIX, and WG14 is unlikely to be interested in changing the C language because the Linux kernel has the wrong type in timespec. Note that on aarch64 ILP32, the consequences of not fixing this right away will be much worse than on x32, since aarch64 (at least as I understand it) supports big endian where it's not just a matter of sign-extending the value from userspace and ignoring the padding, but rather changing the offset of the tv_nsec member. Working around the discrepencies in userspace IS possible, but ugly. We do it in musl libc for x32 right now -- see: http://git.musl-libc.org/cgit/musl/tree/arch/x32/syscall_arch.h?id=v1.1.6 http://git.musl-libc.org/cgit/musl/tree/arch/x32/src/syscall_cp_fixup.c?id=v1.1.6 I imagine the workarounds in glibc might need to be considerably more widespread and uglier. Whatever happens on the kernel side, this needs to be coordinated with userspace (glibc, etc.) properly so that the type error (glibc bug 16437) is not propagated into a new target that we actually want people to use. I'd really like it if other undesirable type changes could be cleaned up too, but perhaps that's too much to ask from the kernel side. Rich -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Thu, 2 Oct 2014 at 16:52:18 +0100, Catalin Marinas wrote: On Wed, Sep 03, 2014 at 10:18:54PM +0100, Andrew Pinski wrote: New version with all of the requested changes. Updated to the latest sources. Notable changes from the previous versions: VDSO code has been factored out to be easier to understand and easier to maintain. Move the config option to the last thing that gets added. Added some extra COMPAT_* macros for core dumping for easier usage. Apart from a few comments I've made, I would also like to see non-empty commit logs and long line wrapping (both in commit logs and Documentation/). Otherwise, the patches look fine. So what are the next steps? Are the glibc folk ok with the ILP32 Linux ABI? On the kernel side, what I would like to see: I don't know if this has been discussed on libc-alpha yet or not, but I think we need to open a discussion of how it relates to open glibc bug #16437, which presently applies only to x32 (ILP32 ABI on x86_64): https://sourceware.org/bugzilla/show_bug.cgi?id=16437 While most of the other type changes proposed (I'm looking at https://lkml.org/lkml/2014/9/3/719) are permissible and simply ugly/undesirable, defining struct timespec with tv_nsec having any type other than long conflicts with the requirements of C11 and POSIX, and WG14 is unlikely to be interested in changing the C language because the Linux kernel has the wrong type in timespec. Note that on aarch64 ILP32, the consequences of not fixing this right away will be much worse than on x32, since aarch64 (at least as I understand it) supports big endian where it's not just a matter of sign-extending the value from userspace and ignoring the padding, but rather changing the offset of the tv_nsec member. Working around the discrepencies in userspace IS possible, but ugly. We do it in musl libc for x32 right now -- see: http://git.musl-libc.org/cgit/musl/tree/arch/x32/syscall_arch.h?id=v1.1.6 http://git.musl-libc.org/cgit/musl/tree/arch/x32/src/syscall_cp_fixup.c?id=v1.1.6 I imagine the workarounds in glibc might need to be considerably more widespread and uglier. Whatever happens on the kernel side, this needs to be coordinated with userspace (glibc, etc.) properly so that the type error (glibc bug 16437) is not propagated into a new target that we actually want people to use. I'd really like it if other undesirable type changes could be cleaned up too, but perhaps that's too much to ask from the kernel side. Rich -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Sep 03, 2014 at 10:18:54PM +0100, Andrew Pinski wrote: > New version with all of the requested changes. Updated to the latest sources. > > Notable changes from the previous versions: > VDSO code has been factored out to be easier to understand and easier to > maintain. > Move the config option to the last thing that gets added. > Added some extra COMPAT_* macros for core dumping for easier usage. Apart from a few comments I've made, I would also like to see non-empty commit logs and long line wrapping (both in commit logs and Documentation/). Otherwise, the patches look fine. So what are the next steps? Are the glibc folk ok with the ILP32 Linux ABI? On the kernel side, what I would like to see: 1. LTP results (in all combinations of AArch32, LP64, ILP32, big and little endian) 2. A way for us to reproduce the ILP32 tests - at which point could we get a toolchain? Ideally a filesystem as well, though maybe we could just use a busybox in the meantime. Thanks. -- Catalin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCHv3 00/24] ILP32 support in ARM64
On Wed, Sep 03, 2014 at 10:18:54PM +0100, Andrew Pinski wrote: New version with all of the requested changes. Updated to the latest sources. Notable changes from the previous versions: VDSO code has been factored out to be easier to understand and easier to maintain. Move the config option to the last thing that gets added. Added some extra COMPAT_* macros for core dumping for easier usage. Apart from a few comments I've made, I would also like to see non-empty commit logs and long line wrapping (both in commit logs and Documentation/). Otherwise, the patches look fine. So what are the next steps? Are the glibc folk ok with the ILP32 Linux ABI? On the kernel side, what I would like to see: 1. LTP results (in all combinations of AArch32, LP64, ILP32, big and little endian) 2. A way for us to reproduce the ILP32 tests - at which point could we get a toolchain? Ideally a filesystem as well, though maybe we could just use a busybox in the meantime. Thanks. -- Catalin -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCHv3 00/24] ILP32 support in ARM64
New version with all of the requested changes. Updated to the latest sources. Notable changes from the previous versions: VDSO code has been factored out to be easier to understand and easier to maintain. Move the config option to the last thing that gets added. Added some extra COMPAT_* macros for core dumping for easier usage. Andrew Pinski (24): ARM64: Force LP64 to compile the kernel ARM64: Rename COMPAT to AARCH32_EL0 in Kconfig ARM64: Change some CONFIG_COMPAT over to use CONFIG_AARCH32_EL0 instead ARM64:ILP32: Set kernel_long to long long so we can reuse most of the same syscalls as LP64 ARM64:UAPI: Set the correct __BITS_PER_LONG for ILP32 Allow for some signal structures to be the same between a 32bit ABI and the 64bit ABI ARM64:ILP32: Use the same size and layout of the signal structures for ILP32 as for LP64 Allow a 32bit ABI to use the naming of the 64bit ABI syscalls to avoid confusion of not splitting the registers ARM64:ILP32: Use the same syscall names as LP64 ARM64: Introduce is_a32_task/is_a32_thread and TIF_AARCH32 and use them in the correct locations ARM64: Add is_ilp32_compat_task and is_ilp32_compat_thread ARM64:ILP32: COMPAT_USE_64BIT_TIME is true for ILP32 tasks ARM64:ILP32: Use the non compat HWCAP for ILP32 ARM64:ILP32 use the standard start_thread for ILP32 so the processor state is not AARCH32 compat_binfmt_elf: coredump: Allow some core dump macros be overridden for compat. ARM64:ILP32: Support core dump for ILP32 ARM64: Add loading of ILP32 binaries ARM64: Add vdso for ILP32 and use it for the signal return ptrace: Allow compat to use the native siginfo ARM64:ILP32: The native siginfo is used instead of the compat siginfo ARM64:ILP32: Use a seperate syscall table as a few syscalls need to be using the compat syscalls ARM64:ILP32: Fix signal return for ILP32 when the user modified the signal stack ARM64: Add ARM64_ILP32 to Kconfig Add documentation about ARM64 ILP32 ABI Documentation/arm64/ilp32.txt | 57 +++ arch/arm64/Kconfig| 15 ++- arch/arm64/Makefile |4 + arch/arm64/include/asm/arch_timer.h |2 +- arch/arm64/include/asm/compat.h | 59 arch/arm64/include/asm/elf.h | 100 - arch/arm64/include/asm/fpsimd.h |2 +- arch/arm64/include/asm/hwcap.h|2 - arch/arm64/include/asm/processor.h| 11 ++ arch/arm64/include/asm/ptrace.h |2 +- arch/arm64/include/asm/signal32.h |4 +- arch/arm64/include/asm/stat.h |4 +- arch/arm64/include/asm/syscalls.h |4 + arch/arm64/include/asm/thread_info.h |1 + arch/arm64/include/asm/unistd.h |6 +- arch/arm64/include/asm/vdso.h |4 + arch/arm64/include/uapi/asm/bitsperlong.h |9 +- arch/arm64/include/uapi/asm/posix_types.h |8 +- arch/arm64/include/uapi/asm/siginfo.h | 33 arch/arm64/include/uapi/asm/signal.h | 34 + arch/arm64/include/uapi/asm/unistd.h |6 + arch/arm64/kernel/Makefile|8 +- arch/arm64/kernel/asm-offsets.c |2 +- arch/arm64/kernel/entry.S | 19 ++- arch/arm64/kernel/head.S |2 +- arch/arm64/kernel/hw_breakpoint.c |7 +- arch/arm64/kernel/process.c |6 +- arch/arm64/kernel/ptrace.c| 52 +-- arch/arm64/kernel/signal.c| 38 +- arch/arm64/kernel/sys_ilp32.c | 195 + arch/arm64/kernel/traps.c |4 +- arch/arm64/kernel/vdso-ilp32/.gitignore |2 + arch/arm64/kernel/vdso-ilp32/Makefile | 72 + arch/arm64/kernel/vdso-ilp32/vdso-ilp32.S | 33 arch/arm64/kernel/vdso-ilp32/vdso-ilp32.lds.S | 98 + arch/arm64/kernel/vdso.c | 74 -- fs/compat_binfmt_elf.c| 17 ++ include/linux/compat.h|4 + include/uapi/asm-generic/siginfo.h| 17 ++- include/uapi/asm-generic/signal.h | 27 +++- include/uapi/asm-generic/unistd.h |6 +- kernel/ptrace.c | 25 +++- 42 files changed, 989 insertions(+), 86 deletions(-) create mode 100644 Documentation/arm64/ilp32.txt create mode 100644 arch/arm64/kernel/sys_ilp32.c create mode 100644 arch/arm64/kernel/vdso-ilp32/.gitignore create mode 100644 arch/arm64/kernel/vdso-ilp32/Makefile create mode 100644 arch/arm64/kernel/vdso-ilp32/vdso-ilp32.S create mode 100644 arch/arm64/kernel/vdso-ilp32/vdso-ilp32.lds.S -- 1.7.2.5 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
[PATCHv3 00/24] ILP32 support in ARM64
New version with all of the requested changes. Updated to the latest sources. Notable changes from the previous versions: VDSO code has been factored out to be easier to understand and easier to maintain. Move the config option to the last thing that gets added. Added some extra COMPAT_* macros for core dumping for easier usage. Andrew Pinski (24): ARM64: Force LP64 to compile the kernel ARM64: Rename COMPAT to AARCH32_EL0 in Kconfig ARM64: Change some CONFIG_COMPAT over to use CONFIG_AARCH32_EL0 instead ARM64:ILP32: Set kernel_long to long long so we can reuse most of the same syscalls as LP64 ARM64:UAPI: Set the correct __BITS_PER_LONG for ILP32 Allow for some signal structures to be the same between a 32bit ABI and the 64bit ABI ARM64:ILP32: Use the same size and layout of the signal structures for ILP32 as for LP64 Allow a 32bit ABI to use the naming of the 64bit ABI syscalls to avoid confusion of not splitting the registers ARM64:ILP32: Use the same syscall names as LP64 ARM64: Introduce is_a32_task/is_a32_thread and TIF_AARCH32 and use them in the correct locations ARM64: Add is_ilp32_compat_task and is_ilp32_compat_thread ARM64:ILP32: COMPAT_USE_64BIT_TIME is true for ILP32 tasks ARM64:ILP32: Use the non compat HWCAP for ILP32 ARM64:ILP32 use the standard start_thread for ILP32 so the processor state is not AARCH32 compat_binfmt_elf: coredump: Allow some core dump macros be overridden for compat. ARM64:ILP32: Support core dump for ILP32 ARM64: Add loading of ILP32 binaries ARM64: Add vdso for ILP32 and use it for the signal return ptrace: Allow compat to use the native siginfo ARM64:ILP32: The native siginfo is used instead of the compat siginfo ARM64:ILP32: Use a seperate syscall table as a few syscalls need to be using the compat syscalls ARM64:ILP32: Fix signal return for ILP32 when the user modified the signal stack ARM64: Add ARM64_ILP32 to Kconfig Add documentation about ARM64 ILP32 ABI Documentation/arm64/ilp32.txt | 57 +++ arch/arm64/Kconfig| 15 ++- arch/arm64/Makefile |4 + arch/arm64/include/asm/arch_timer.h |2 +- arch/arm64/include/asm/compat.h | 59 arch/arm64/include/asm/elf.h | 100 - arch/arm64/include/asm/fpsimd.h |2 +- arch/arm64/include/asm/hwcap.h|2 - arch/arm64/include/asm/processor.h| 11 ++ arch/arm64/include/asm/ptrace.h |2 +- arch/arm64/include/asm/signal32.h |4 +- arch/arm64/include/asm/stat.h |4 +- arch/arm64/include/asm/syscalls.h |4 + arch/arm64/include/asm/thread_info.h |1 + arch/arm64/include/asm/unistd.h |6 +- arch/arm64/include/asm/vdso.h |4 + arch/arm64/include/uapi/asm/bitsperlong.h |9 +- arch/arm64/include/uapi/asm/posix_types.h |8 +- arch/arm64/include/uapi/asm/siginfo.h | 33 arch/arm64/include/uapi/asm/signal.h | 34 + arch/arm64/include/uapi/asm/unistd.h |6 + arch/arm64/kernel/Makefile|8 +- arch/arm64/kernel/asm-offsets.c |2 +- arch/arm64/kernel/entry.S | 19 ++- arch/arm64/kernel/head.S |2 +- arch/arm64/kernel/hw_breakpoint.c |7 +- arch/arm64/kernel/process.c |6 +- arch/arm64/kernel/ptrace.c| 52 +-- arch/arm64/kernel/signal.c| 38 +- arch/arm64/kernel/sys_ilp32.c | 195 + arch/arm64/kernel/traps.c |4 +- arch/arm64/kernel/vdso-ilp32/.gitignore |2 + arch/arm64/kernel/vdso-ilp32/Makefile | 72 + arch/arm64/kernel/vdso-ilp32/vdso-ilp32.S | 33 arch/arm64/kernel/vdso-ilp32/vdso-ilp32.lds.S | 98 + arch/arm64/kernel/vdso.c | 74 -- fs/compat_binfmt_elf.c| 17 ++ include/linux/compat.h|4 + include/uapi/asm-generic/siginfo.h| 17 ++- include/uapi/asm-generic/signal.h | 27 +++- include/uapi/asm-generic/unistd.h |6 +- kernel/ptrace.c | 25 +++- 42 files changed, 989 insertions(+), 86 deletions(-) create mode 100644 Documentation/arm64/ilp32.txt create mode 100644 arch/arm64/kernel/sys_ilp32.c create mode 100644 arch/arm64/kernel/vdso-ilp32/.gitignore create mode 100644 arch/arm64/kernel/vdso-ilp32/Makefile create mode 100644 arch/arm64/kernel/vdso-ilp32/vdso-ilp32.S create mode 100644 arch/arm64/kernel/vdso-ilp32/vdso-ilp32.lds.S -- 1.7.2.5 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in
