Re: RFC: Add 32bit x86-64 support to binutils

2011-01-15 Thread H.J. Lu
On Fri, Jan 14, 2011 at 3:46 PM, H.J. Lu  wrote:
> On Fri, Dec 31, 2010 at 6:50 AM, H.J. Lu  wrote:
>> On Thu, Dec 30, 2010 at 4:48 PM, H.J. Lu  wrote:
>>> On Thu, Dec 30, 2010 at 4:42 PM, H. Peter Anvin  wrote:
 On 12/30/2010 01:08 PM, Robert Millan wrote:
> Hi folks,
>
> I had this unsubmitted patch in my local filesystem.  It makes Linux
> detect ELF32 AMD64 binaries and sets a flag to restrict them to
> 32-bit address space.
>
> It's not rocket science but can save you some work in case you
> haven't implemented this already.
>

 I have pushed my old kernel patches to a git tree at:

 git://git.kernel.org//pub/scm/linux/kernel/git/hpa/linux-2.6-ilp32.git

 They are currently based on 2.6.31 since that was the released version
 when I first did this work; they are not intended to be mergeble but
 rather as a prototype.

 Note that we have no intention of supporting this ABI for the kernel
 itself.  The kernel will be a normal x86-64 kernel.

>>>
>>> Here is the updated ILP32 patch for 2.6.35.
>>>
>>>
>>
>> I put my ILP32 gdb on hjl/ilp32 branch at
>>
>> http://git.kernel.org/?p=devel/gdb/hjl/x86.git;a=summary
>>
>> and my gcc 4.4 ILP32 on hjl/ilp32/gcc-4_4-branch branch at
>>
>> http://git.kernel.org/?p=devel/gcc/hjl/x86.git;a=summary
>>
>
> I checked a bunch ILP32 bug fixes into binutils, gdb and gcc.
> I also renamed the option from n32 to x32.
>
> Binutils and gdb should work correctly now. I tested them on
> a simple C library with static and dynamic binaries.  Gcc only
> works with -O0.

I put a small C library for x32 at

http://git.kernel.org/?p=devel/binutils/hjl/x32.git;a=summary


-- 
H.J.


Re: RFC: Add 32bit x86-64 support to binutils

2011-01-14 Thread H.J. Lu
On Fri, Dec 31, 2010 at 6:50 AM, H.J. Lu  wrote:
> On Thu, Dec 30, 2010 at 4:48 PM, H.J. Lu  wrote:
>> On Thu, Dec 30, 2010 at 4:42 PM, H. Peter Anvin  wrote:
>>> On 12/30/2010 01:08 PM, Robert Millan wrote:
 Hi folks,

 I had this unsubmitted patch in my local filesystem.  It makes Linux
 detect ELF32 AMD64 binaries and sets a flag to restrict them to
 32-bit address space.

 It's not rocket science but can save you some work in case you
 haven't implemented this already.

>>>
>>> I have pushed my old kernel patches to a git tree at:
>>>
>>> git://git.kernel.org//pub/scm/linux/kernel/git/hpa/linux-2.6-ilp32.git
>>>
>>> They are currently based on 2.6.31 since that was the released version
>>> when I first did this work; they are not intended to be mergeble but
>>> rather as a prototype.
>>>
>>> Note that we have no intention of supporting this ABI for the kernel
>>> itself.  The kernel will be a normal x86-64 kernel.
>>>
>>
>> Here is the updated ILP32 patch for 2.6.35.
>>
>>
>
> I put my ILP32 gdb on hjl/ilp32 branch at
>
> http://git.kernel.org/?p=devel/gdb/hjl/x86.git;a=summary
>
> and my gcc 4.4 ILP32 on hjl/ilp32/gcc-4_4-branch branch at
>
> http://git.kernel.org/?p=devel/gcc/hjl/x86.git;a=summary
>

I checked a bunch ILP32 bug fixes into binutils, gdb and gcc.
I also renamed the option from n32 to x32.

Binutils and gdb should work correctly now. I tested them on
a simple C library with static and dynamic binaries.  Gcc only
works with -O0.

-- 
H.J.


Re: RFC: Add 32bit x86-64 support to binutils

2011-01-05 Thread H.J. Lu
On Wed, Jan 5, 2011 at 11:52 AM, Jonathan Wakely  wrote:
> On 30 December 2010 18:23, H.J. Lu wrote:
>>
>> This patch adds 32bit x86-64 support to binutils. Support in compiler,
>> library and OS is required to use it.  It can be used to implement the
>> new 32bit OS for x86-64.  Any comments?
>
> I have a small comment on the changes to the c-i386.texi docs:
>
> diff --git a/gas/doc/c-i386.texi b/gas/doc/c-i386.texi
> index 1c6175b..c3956a8 100644
> --- a/gas/doc/c-i386.texi
> +++ b/gas/doc/c-i386.texi
> @@ -56,11 +56,14 @@ dependent options:
> �...@table @gcctabopt
> �...@cindex @samp{--32} option, i386
> �...@cindex @samp{--32} option, x86-64
> +...@cindex @samp{--n32} option, i386
> +...@cindex @samp{--n32} option, x86-64
> �...@cindex @samp{--64} option, i386
> �...@cindex @samp{--64} option, x86-64
> -...@item --32 | --64
> +...@item --32 | --n32 | --64
>  Select the word size, either 32 bits or 64 bits. Selecting 32-bit
>  implies Intel i386 architecture, while 64-bit implies AMD x86-64
> +architecture. �...@samp{--n32} selects 32bit word size with AMD x86-64
>  architecture.
>
> Simply adding the new sentence at the end is not very clear, because
> the last sentence contradicts the second sentence:  --n32 selects
> 32-bit word size, but does not imply Intel i386 architecture.
>
> Also, "32bit" and "32-bit" should be used consistently.
>
> How about:
>
>  Select the word size, either 32 bits or 64 bits. @samp{--32}
>  implies Intel i386 architecture, while @samp{--n32} and @samp{--64}
>  imply AMD x86-64 architecture with 32-bit or 64-bit word-size
>  respectively.
>

Thanks. I checked it in.


-- 
H.J.
---
Index: ChangeLog
===
RCS file: /cvs/src/src/gas/ChangeLog,v
retrieving revision 1.4376
diff -u -p -r1.4376 ChangeLog
--- ChangeLog   5 Jan 2011 00:16:49 -   1.4376
+++ ChangeLog   5 Jan 2011 21:34:34 -
@@ -1,3 +1,7 @@
+2011-01-05  Jonathan Wakely  
+
+   * doc/c-i386.texi: Clarify --n32.
+
 2011-01-04  H.J. Lu  

* config/tc-i386.c (build_modrm_byte): Allow encoding 32/64bit
Index: doc/c-i386.texi
===
RCS file: /cvs/src/src/gas/doc/c-i386.texi,v
retrieving revision 1.53
diff -u -p -r1.53 c-i386.texi
--- doc/c-i386.texi 31 Dec 2010 00:33:34 -  1.53
+++ doc/c-i386.texi 5 Jan 2011 21:34:34 -
@@ -61,10 +61,10 @@ dependent options:
 @cindex @samp{--64} option, i386
 @cindex @samp{--64} option, x86-64
 @item --32 | --n32 | --64
-Select the word size, either 32 bits or 64 bits. Selecting 32-bit
-implies Intel i386 architecture, while 64-bit implies AMD x86-64
-architecture.  @samp{--n32} selects 32bit word size with AMD x86-64
-architecture.
+Select the word size, either 32 bits or 64 bits.  @samp{--32}
+implies Intel i386 architecture, while @samp{--n32} and @samp{--64}
+imply AMD x86-64 architecture with 32-bit or 64-bit word-size
+respectively.

 These options are only available with the ELF object file format, and
 require that the necessary BFD support has been included (on a 32-bit


Re: RFC: Add 32bit x86-64 support to binutils

2011-01-05 Thread Jonathan Wakely
On 30 December 2010 18:23, H.J. Lu wrote:
>
> This patch adds 32bit x86-64 support to binutils. Support in compiler,
> library and OS is required to use it.  It can be used to implement the
> new 32bit OS for x86-64.  Any comments?

I have a small comment on the changes to the c-i386.texi docs:

diff --git a/gas/doc/c-i386.texi b/gas/doc/c-i386.texi
index 1c6175b..c3956a8 100644
--- a/gas/doc/c-i386.texi
+++ b/gas/doc/c-i386.texi
@@ -56,11 +56,14 @@ dependent options:
 @table @gcctabopt
 @cindex @samp{--32} option, i386
 @cindex @samp{--32} option, x86-64
+...@cindex @samp{--n32} option, i386
+...@cindex @samp{--n32} option, x86-64
 @cindex @samp{--64} option, i386
 @cindex @samp{--64} option, x86-64
-...@item --32 | --64
+...@item --32 | --n32 | --64
 Select the word size, either 32 bits or 64 bits. Selecting 32-bit
 implies Intel i386 architecture, while 64-bit implies AMD x86-64
+architecture.  @samp{--n32} selects 32bit word size with AMD x86-64
 architecture.

Simply adding the new sentence at the end is not very clear, because
the last sentence contradicts the second sentence:  --n32 selects
32-bit word size, but does not imply Intel i386 architecture.

Also, "32bit" and "32-bit" should be used consistently.

How about:

 Select the word size, either 32 bits or 64 bits. @samp{--32}
 implies Intel i386 architecture, while @samp{--n32} and @samp{--64}
 imply AMD x86-64 architecture with 32-bit or 64-bit word-size
 respectively.


Re: RFC: Add 32bit x86-64 support to binutils

2011-01-05 Thread Jan Beulich
>>> On 05.01.11 at 09:01, "H. Peter Anvin"  wrote:
> On 01/04/2011 11:46 PM, Jan Beulich wrote:

 Oh god, please, no.

 I have to say I'm highly questioning to Jan's statement in the first
 place.  Crossing 32- and 64-bit ELF like that sounds like a kernel
 security hole waiting to happen.
>> 
>> A particular OS/kernel has the freedom to not implement support for
>> other than the default format. But having the ABI disallow it
>> altogether certainly isn't the right choice. And yes, we had been
>> allowing cross-bitness ELF in an experimental (long canceled) OS
>> of ours.
>> 
>>> Yeah, and there are other targets where the elf class determines ABI
>>> too (e.g. EM_S390 is used for both 31-bit and 64-bit binaries and
>>> the ELF class determines which).
>> 
>> So the usual thing is going to happen - someone made a mistake (I'm
>> convinced the ELF class was never meant to affect anything but the
>> file format), and this gets taken as an excuse to let the mistake
>> spread.
>> 
> 
> I don't think it's all that unreasonable to say the ELF class affects
> the ABI.  After all, there are lots of things about the ABI that is
> related to the ELF class -- the format of the GOT and PLT, for one thing.

That's in executables and dynamic objects only. I'm not aware of
anything in relocatable objects, and I'd question it for core files.
The ABI, however, has to cover all of them.

Jan



Re: RFC: Add 32bit x86-64 support to binutils

2011-01-05 Thread H. Peter Anvin
On 01/04/2011 11:46 PM, Jan Beulich wrote:
>>>
>>> Oh god, please, no.
>>>
>>> I have to say I'm highly questioning to Jan's statement in the first
>>> place.  Crossing 32- and 64-bit ELF like that sounds like a kernel
>>> security hole waiting to happen.
> 
> A particular OS/kernel has the freedom to not implement support for
> other than the default format. But having the ABI disallow it
> altogether certainly isn't the right choice. And yes, we had been
> allowing cross-bitness ELF in an experimental (long canceled) OS
> of ours.
> 
>> Yeah, and there are other targets where the elf class determines ABI
>> too (e.g. EM_S390 is used for both 31-bit and 64-bit binaries and
>> the ELF class determines which).
> 
> So the usual thing is going to happen - someone made a mistake (I'm
> convinced the ELF class was never meant to affect anything but the
> file format), and this gets taken as an excuse to let the mistake
> spread.
> 

I don't think it's all that unreasonable to say the ELF class affects
the ABI.  After all, there are lots of things about the ABI that is
related to the ELF class -- the format of the GOT and PLT, for one thing.

-hpa


Re: RFC: Add 32bit x86-64 support to binutils

2011-01-04 Thread Jan Beulich
>>> On 04.01.11 at 21:02, Jakub Jelinek  wrote:
> On Tue, Jan 04, 2011 at 10:35:42AM -0800, H. Peter Anvin wrote:
>> On 01/04/2011 09:56 AM, H.J. Lu wrote:
>> >>
>> >> I think it is a gross misconception to tie the ABI to the ELF class of
>> >> an object. Specifying the ABI should imo be done via e_flags or
>> >> one of the unused bytes of e_ident, and in all reality the ELF class
>> >> should *only* affect the file layout (and 64-bit should never have
>> >> forbidden to use 32-bit ELF containers; similarly 64-bit ELF objects
>> >> may have uses for 32-bit architectures/ABIs, e.g. when debug
>> >> information exceeds the 4G boundary).
>> > 
>> > I agree with you in principle. But I think it should be done via
>> > a new attribute section, similar to ARM.
>> > 
>> 
>> Oh god, please, no.
>> 
>> I have to say I'm highly questioning to Jan's statement in the first
>> place.  Crossing 32- and 64-bit ELF like that sounds like a kernel
>> security hole waiting to happen.

A particular OS/kernel has the freedom to not implement support for
other than the default format. But having the ABI disallow it
altogether certainly isn't the right choice. And yes, we had been
allowing cross-bitness ELF in an experimental (long canceled) OS
of ours.

> Yeah, and there are other targets where the elf class determines ABI
> too (e.g. EM_S390 is used for both 31-bit and 64-bit binaries and
> the ELF class determines which).

So the usual thing is going to happen - someone made a mistake (I'm
convinced the ELF class was never meant to affect anything but the
file format), and this gets taken as an excuse to let the mistake
spread.

Jan



Re: RFC: Add 32bit x86-64 support to binutils

2011-01-04 Thread Jakub Jelinek
On Tue, Jan 04, 2011 at 10:35:42AM -0800, H. Peter Anvin wrote:
> On 01/04/2011 09:56 AM, H.J. Lu wrote:
> >>
> >> I think it is a gross misconception to tie the ABI to the ELF class of
> >> an object. Specifying the ABI should imo be done via e_flags or
> >> one of the unused bytes of e_ident, and in all reality the ELF class
> >> should *only* affect the file layout (and 64-bit should never have
> >> forbidden to use 32-bit ELF containers; similarly 64-bit ELF objects
> >> may have uses for 32-bit architectures/ABIs, e.g. when debug
> >> information exceeds the 4G boundary).
> > 
> > I agree with you in principle. But I think it should be done via
> > a new attribute section, similar to ARM.
> > 
> 
> Oh god, please, no.
> 
> I have to say I'm highly questioning to Jan's statement in the first
> place.  Crossing 32- and 64-bit ELF like that sounds like a kernel
> security hole waiting to happen.

Yeah, and there are other targets where the elf class determines ABI
too (e.g. EM_S390 is used for both 31-bit and 64-bit binaries and
the ELF class determines which).

Jakub


Re: RFC: Add 32bit x86-64 support to binutils

2011-01-04 Thread H. Peter Anvin
On 01/04/2011 09:56 AM, H.J. Lu wrote:
>>
>> I think it is a gross misconception to tie the ABI to the ELF class of
>> an object. Specifying the ABI should imo be done via e_flags or
>> one of the unused bytes of e_ident, and in all reality the ELF class
>> should *only* affect the file layout (and 64-bit should never have
>> forbidden to use 32-bit ELF containers; similarly 64-bit ELF objects
>> may have uses for 32-bit architectures/ABIs, e.g. when debug
>> information exceeds the 4G boundary).
> 
> I agree with you in principle. But I think it should be done via
> a new attribute section, similar to ARM.
> 

Oh god, please, no.

I have to say I'm highly questioning to Jan's statement in the first
place.  Crossing 32- and 64-bit ELF like that sounds like a kernel
security hole waiting to happen.

-hpa



Re: RFC: Add 32bit x86-64 support to binutils

2011-01-04 Thread H.J. Lu
On Mon, Jan 3, 2011 at 2:40 AM, Jan Beulich  wrote:
 On 30.12.10 at 21:02, "H.J. Lu"  wrote:
>>
>> Here is the ILP32 psABI:
>>
>> http://www.kernel.org/pub/linux/devel/binutils/ilp32/
>>
>
> I think it is a gross misconception to tie the ABI to the ELF class of
> an object. Specifying the ABI should imo be done via e_flags or
> one of the unused bytes of e_ident, and in all reality the ELF class
> should *only* affect the file layout (and 64-bit should never have
> forbidden to use 32-bit ELF containers; similarly 64-bit ELF objects
> may have uses for 32-bit architectures/ABIs, e.g. when debug
> information exceeds the 4G boundary).
>

I agree with you in principle. But I think it should be done via
a new attribute section, similar to ARM.


-- 
H.J.


Re: RFC: Add 32bit x86-64 support to binutils

2011-01-03 Thread Jan Beulich
>>> On 30.12.10 at 21:02, "H.J. Lu"  wrote:
> 
> Here is the ILP32 psABI:
> 
> http://www.kernel.org/pub/linux/devel/binutils/ilp32/ 
> 

I think it is a gross misconception to tie the ABI to the ELF class of
an object. Specifying the ABI should imo be done via e_flags or
one of the unused bytes of e_ident, and in all reality the ELF class
should *only* affect the file layout (and 64-bit should never have
forbidden to use 32-bit ELF containers; similarly 64-bit ELF objects
may have uses for 32-bit architectures/ABIs, e.g. when debug
information exceeds the 4G boundary).

Jan



Re: RFC: Add 32bit x86-64 support to binutils

2010-12-31 Thread H. Peter Anvin
On 12/31/2010 02:03 AM, Jakub Jelinek wrote:
> On Thu, Dec 30, 2010 at 01:42:05PM -0800, H. Peter Anvin wrote:
>> On 12/30/2010 11:57 AM, Jakub Jelinek wrote:

 Would be nice if LFS would be mandatory on the new ABI, thus
 off_t being 64bits.
>>>
>>> And avoid ambiguous cases that x86-64 ABI has, e.g. whether
>>> caller or callee is responsible for sign/zero extension of arguments, to
>>> avoid the need to sign/zero extend twice, etc.
>>>
>>
>> Ehwhat?  x86-64 is completely unambiguous on that point; the i386 one is
>> not.
> 
> It is not, sadly, see http://gcc.gnu.org/PR46942
> From what I can see the psABI doesn't talk about it, GCC usually sign/zero
> extends on both sides (exception is 32-bit arguments into 64-bit isn't
> apparently sign/zero extended on the caller side when doing tail calls),
> from what I gathered LLVM expects the caller to sign/zero extend (which is
> incompatible with GCC tail calls then), not sure about ICC, and kernel
> probably expects for security reasons that the callee sign/zero extends.
> 

This is weird... we had long discussions about this when the psABI was
originally written, and the decision was that any bits outside the
fundamental type was undefined -- callee extends (caller in the case of
a return value.)  Yet somehow that (and several other discussions) seem
to either never have made it into the document or otherwise have
disappeared somewhere in the process.

There seems to have been problems with closing the loop on a number of
things, and in some cases the compiler writers have gone off and
implemented something completely different from the written document,
yet failed to get the documentation updated to match reality (it took
many years until the definition of _Bool matched what the compilers
actually implemented.)

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.



Re: RFC: Add 32bit x86-64 support to binutils

2010-12-31 Thread H.J. Lu
On Fri, Dec 31, 2010 at 8:50 AM, H.J. Lu  wrote:
> On Fri, Dec 31, 2010 at 2:03 AM, Jakub Jelinek  wrote:
>> On Thu, Dec 30, 2010 at 01:42:05PM -0800, H. Peter Anvin wrote:
>>> On 12/30/2010 11:57 AM, Jakub Jelinek wrote:
>>> >>
>>> >> Would be nice if LFS would be mandatory on the new ABI, thus
>>> >> off_t being 64bits.
>>> >
>>> > And avoid ambiguous cases that x86-64 ABI has, e.g. whether
>>> > caller or callee is responsible for sign/zero extension of arguments, to
>>> > avoid the need to sign/zero extend twice, etc.
>>> >
>>>
>>> Ehwhat?  x86-64 is completely unambiguous on that point; the i386 one is
>>> not.
>>
>> It is not, sadly, see http://gcc.gnu.org/PR46942
>> From what I can see the psABI doesn't talk about it, GCC usually sign/zero
>> extends on both sides (exception is 32-bit arguments into 64-bit isn't
>> apparently sign/zero extended on the caller side when doing tail calls),
>> from what I gathered LLVM expects the caller to sign/zero extend (which is
>> incompatible with GCC tail calls then), not sure about ICC, and kernel
>> probably expects for security reasons that the callee sign/zero extends.
>>
>>        Jakub
>>
>
> I added
>
> ---
> When a value of type signed/unsigned char or short is returned or passed
> in a register or on the stack, it should be sign/zero extended to
> signed/unsigned
> int.
> ---
>
> to hjl/extension branch at
>
> http://git.kernel.org/?p=devel/binutils/hjl/x86-64-psabi.git;a=summary
>

According to

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42324

all compilers clears bits 1-31 when passing _Bool to a function.  Also
GCC doesn't
sign/zero extend on char/short return.  I updated x86-64 psABI to


When a value of type _Bool is returned in a register, bit 0 contains the truth
value and bits 1 to 7 shall be zero. When an argument of type _Bool is passed
in a register or on the stack, bit 0 contains the truth value and bits
1 to 31 shall be
zero.

When a value of type signed/unsigned char or short is returned in a register,
bits 0 to 7 for char and bits 0 to 15 for short contain the value and other
bits are left unspecified. When an argument of signed/unsigned type char or
short is passed in a register or on the stack, it shall be sign/zero extended to
signed/unsigned int.
---

on hjl/extension branch at

http://git.kernel.org/?p=devel/binutils/hjl/x86-64-psabi.git;a=summary


-- 
H.J.


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-31 Thread H.J. Lu
On Fri, Dec 31, 2010 at 2:03 AM, Jakub Jelinek  wrote:
> On Thu, Dec 30, 2010 at 01:42:05PM -0800, H. Peter Anvin wrote:
>> On 12/30/2010 11:57 AM, Jakub Jelinek wrote:
>> >>
>> >> Would be nice if LFS would be mandatory on the new ABI, thus
>> >> off_t being 64bits.
>> >
>> > And avoid ambiguous cases that x86-64 ABI has, e.g. whether
>> > caller or callee is responsible for sign/zero extension of arguments, to
>> > avoid the need to sign/zero extend twice, etc.
>> >
>>
>> Ehwhat?  x86-64 is completely unambiguous on that point; the i386 one is
>> not.
>
> It is not, sadly, see http://gcc.gnu.org/PR46942
> From what I can see the psABI doesn't talk about it, GCC usually sign/zero
> extends on both sides (exception is 32-bit arguments into 64-bit isn't
> apparently sign/zero extended on the caller side when doing tail calls),
> from what I gathered LLVM expects the caller to sign/zero extend (which is
> incompatible with GCC tail calls then), not sure about ICC, and kernel
> probably expects for security reasons that the callee sign/zero extends.
>
>        Jakub
>

I added

---
When a value of type signed/unsigned char or short is returned or passed
in a register or on the stack, it should be sign/zero extended to
signed/unsigned
int.
---

to hjl/extension branch at

http://git.kernel.org/?p=devel/binutils/hjl/x86-64-psabi.git;a=summary


-- 
H.J.


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-31 Thread H.J. Lu
On Thu, Dec 30, 2010 at 4:48 PM, H.J. Lu  wrote:
> On Thu, Dec 30, 2010 at 4:42 PM, H. Peter Anvin  wrote:
>> On 12/30/2010 01:08 PM, Robert Millan wrote:
>>> Hi folks,
>>>
>>> I had this unsubmitted patch in my local filesystem.  It makes Linux
>>> detect ELF32 AMD64 binaries and sets a flag to restrict them to
>>> 32-bit address space.
>>>
>>> It's not rocket science but can save you some work in case you
>>> haven't implemented this already.
>>>
>>
>> I have pushed my old kernel patches to a git tree at:
>>
>> git://git.kernel.org//pub/scm/linux/kernel/git/hpa/linux-2.6-ilp32.git
>>
>> They are currently based on 2.6.31 since that was the released version
>> when I first did this work; they are not intended to be mergeble but
>> rather as a prototype.
>>
>> Note that we have no intention of supporting this ABI for the kernel
>> itself.  The kernel will be a normal x86-64 kernel.
>>
>
> Here is the updated ILP32 patch for 2.6.35.
>
>

I put my ILP32 gdb on hjl/ilp32 branch at

http://git.kernel.org/?p=devel/gdb/hjl/x86.git;a=summary

and my gcc 4.4 ILP32 on hjl/ilp32/gcc-4_4-branch branch at

http://git.kernel.org/?p=devel/gcc/hjl/x86.git;a=summary


-- 
H.J.


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-31 Thread H.J. Lu
On Fri, Dec 31, 2010 at 2:03 AM, Jakub Jelinek  wrote:
> On Thu, Dec 30, 2010 at 01:42:05PM -0800, H. Peter Anvin wrote:
>> On 12/30/2010 11:57 AM, Jakub Jelinek wrote:
>> >>
>> >> Would be nice if LFS would be mandatory on the new ABI, thus
>> >> off_t being 64bits.
>> >
>> > And avoid ambiguous cases that x86-64 ABI has, e.g. whether
>> > caller or callee is responsible for sign/zero extension of arguments, to
>> > avoid the need to sign/zero extend twice, etc.
>> >
>>
>> Ehwhat?  x86-64 is completely unambiguous on that point; the i386 one is
>> not.
>
> It is not, sadly, see http://gcc.gnu.org/PR46942
> From what I can see the psABI doesn't talk about it, GCC usually sign/zero
> extends on both sides (exception is 32-bit arguments into 64-bit isn't
> apparently sign/zero extended on the caller side when doing tail calls),
> from what I gathered LLVM expects the caller to sign/zero extend (which is
> incompatible with GCC tail calls then), not sure about ICC, and kernel
> probably expects for security reasons that the callee sign/zero extends.

I prefer caller to do sign/zero extension so that we don't update partial
register.

-- 
H.J.


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-31 Thread Jakub Jelinek
On Thu, Dec 30, 2010 at 01:42:05PM -0800, H. Peter Anvin wrote:
> On 12/30/2010 11:57 AM, Jakub Jelinek wrote:
> >>
> >> Would be nice if LFS would be mandatory on the new ABI, thus
> >> off_t being 64bits.
> > 
> > And avoid ambiguous cases that x86-64 ABI has, e.g. whether
> > caller or callee is responsible for sign/zero extension of arguments, to
> > avoid the need to sign/zero extend twice, etc.
> > 
> 
> Ehwhat?  x86-64 is completely unambiguous on that point; the i386 one is
> not.

It is not, sadly, see http://gcc.gnu.org/PR46942
>From what I can see the psABI doesn't talk about it, GCC usually sign/zero
extends on both sides (exception is 32-bit arguments into 64-bit isn't
apparently sign/zero extended on the caller side when doing tail calls),
from what I gathered LLVM expects the caller to sign/zero extend (which is
incompatible with GCC tail calls then), not sure about ICC, and kernel
probably expects for security reasons that the callee sign/zero extends.

Jakub


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H.J. Lu
On Thu, Dec 30, 2010 at 4:42 PM, H. Peter Anvin  wrote:
> On 12/30/2010 01:08 PM, Robert Millan wrote:
>> Hi folks,
>>
>> I had this unsubmitted patch in my local filesystem.  It makes Linux
>> detect ELF32 AMD64 binaries and sets a flag to restrict them to
>> 32-bit address space.
>>
>> It's not rocket science but can save you some work in case you
>> haven't implemented this already.
>>
>
> I have pushed my old kernel patches to a git tree at:
>
> git://git.kernel.org//pub/scm/linux/kernel/git/hpa/linux-2.6-ilp32.git
>
> They are currently based on 2.6.31 since that was the released version
> when I first did this work; they are not intended to be mergeble but
> rather as a prototype.
>
> Note that we have no intention of supporting this ABI for the kernel
> itself.  The kernel will be a normal x86-64 kernel.
>

Here is the updated ILP32 patch for 2.6.35.


-- 
H.J.
diff --git a/arch/x86/ia32/ia32_signal.c b/arch/x86/ia32/ia32_signal.c
index 588a7aa..ae915c8 100644
--- a/arch/x86/ia32/ia32_signal.c
+++ b/arch/x86/ia32/ia32_signal.c
@@ -42,6 +42,14 @@
 
 void signal_fault(struct pt_regs *regs, void __user *frame, char *where);
 
+/* We use the standard 64-bit versions of these for the ILP32 variants */
+int
+restore_sigcontext(struct pt_regs *regs, struct sigcontext __user *sc,
+  unsigned long *pax);
+int
+setup_sigcontext(struct sigcontext __user *sc, void __user *fpstate,
+struct pt_regs *regs, unsigned long mask);
+
 int copy_siginfo_to_user32(compat_siginfo_t __user *to, siginfo_t *from)
 {
int err = 0;
@@ -565,3 +573,118 @@ int ia32_setup_rt_frame(int sig, struct k_sigaction *ka, 
siginfo_t *info,
 
return 0;
 }
+
+int ia32_setup_ilp32_frame(int sig, struct k_sigaction *ka, siginfo_t *info,
+  compat_sigset_t *set, struct pt_regs *regs)
+{
+   struct rt_sigframe_ilp32 __user *frame;
+   void __user *restorer;
+   int err = 0;
+   void __user *fpstate = NULL;
+
+   /* __copy_to_user optimizes that into a single 8 byte store */
+   static const struct {
+   u8 movl;
+   u32 val;
+   u16 int80;
+   u8  pad;
+   } __attribute__((packed)) code = {
+   0xb8,
+   __NR_ia32_ilp32_sigreturn,
+   0x80cd,
+   0,
+   };
+
+   frame = get_sigframe(ka, regs, sizeof(*frame), &fpstate);
+
+   if (!access_ok(VERIFY_WRITE, frame, sizeof(*frame)))
+   return -EFAULT;
+
+   put_user_try {
+   put_user_ex(sig, &frame->sig);
+   put_user_ex(ptr_to_compat(&frame->info), &frame->pinfo);
+   put_user_ex(ptr_to_compat(&frame->uc), &frame->puc);
+   err |= copy_siginfo_to_user32(&frame->info, info);
+
+   /* Create the ucontext.  */
+   if (cpu_has_xsave)
+   put_user_ex(UC_FP_XSTATE, &frame->uc.uc_flags);
+   else
+   put_user_ex(0, &frame->uc.uc_flags);
+   put_user_ex(0, &frame->uc.uc_link);
+   put_user_ex(current->sas_ss_sp, &frame->uc.uc_stack.ss_sp);
+   put_user_ex(sas_ss_flags(regs->sp),
+   &frame->uc.uc_stack.ss_flags);
+   put_user_ex(current->sas_ss_size, &frame->uc.uc_stack.ss_size);
+   err |= setup_sigcontext(&frame->uc.uc_mcontext, fpstate,
+   regs, set->sig[0]);
+   err |= __copy_to_user(&frame->uc.uc_sigmask, set, sizeof(*set));
+
+   if (ka->sa.sa_flags & SA_RESTORER)
+   restorer = ka->sa.sa_restorer;
+   else
+   restorer = VDSO32_SYMBOL(current->mm->context.vdso,
+rt_sigreturn);
+   put_user_ex(ptr_to_compat(restorer), &frame->pretcode);
+
+   /*
+* Not actually used anymore, but left because some gdb
+* versions need it.
+*/
+   put_user_ex(*((u64 *)&code), (u64 *)frame->retcode);
+   } put_user_catch(err);
+
+   if (err)
+   return -EFAULT;
+
+   /* Set up registers for signal handler */
+   regs->sp = (unsigned long) frame;
+   regs->ip = (unsigned long) ka->sa.sa_handler;
+
+   /* We use the 64-bit ILP32 calling convention here... */
+   regs->di = sig;
+   regs->si = (unsigned long) &frame->info;
+   regs->dx = (unsigned long) &frame->uc;
+
+   loadsegment(ds, __USER_DS);
+   loadsegment(es, __USER_DS);
+
+   regs->cs = __USER_CS;
+   regs->ss = __USER_DS;
+
+   return 0;
+}
+
+asmlinkage long sys32_ilp32_sigreturn(struct pt_regs *regs)
+{
+   struct rt_sigframe_ilp32 __user *frame;
+   sigset_t set;
+   unsigned long ax;
+   struct pt_regs tregs;
+
+   frame = (struct rt_sigframe_ilp32 __user *)(regs->sp - 4);
+
+   if (!access_ok(VERIFY_READ,

Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H. Peter Anvin
On 12/30/2010 01:08 PM, Robert Millan wrote:
> Hi folks,
> 
> I had this unsubmitted patch in my local filesystem.  It makes Linux
> detect ELF32 AMD64 binaries and sets a flag to restrict them to
> 32-bit address space.
> 
> It's not rocket science but can save you some work in case you
> haven't implemented this already.
> 

I have pushed my old kernel patches to a git tree at:

git://git.kernel.org//pub/scm/linux/kernel/git/hpa/linux-2.6-ilp32.git

They are currently based on 2.6.31 since that was the released version
when I first did this work; they are not intended to be mergeble but
rather as a prototype.

Note that we have no intention of supporting this ABI for the kernel
itself.  The kernel will be a normal x86-64 kernel.

-hpa



Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H.J. Lu
On Thu, Dec 30, 2010 at 4:25 PM, H.J. Lu  wrote:
> On Thu, Dec 30, 2010 at 3:00 PM, Joseph S. Myers
>  wrote:
>> On Thu, 30 Dec 2010, H. Peter Anvin wrote:
>>
>>> On 12/30/2010 02:21 PM, Robert Millan wrote:
>>> > 2010/12/30 Richard Guenther :
>>> >> Would be nice if LFS would be mandatory on the new ABI, thus
>>> >> off_t being 64bits.
>>> >
>>> > Please do also consider time_t.
>>> >
>>>
>>> Changing the kernel-facing time_t might completely wreck the reuse of
>>> the i386 kernel ABI; I'm not sure.
>>
>> Before changing time_t for a new ILP32 ABI, you probably want to work out
>> what is required - on both the libc and kernel sides - to change it for
>> existing 32-bit ABIs (whether providing a new ABI variant like
>> _FILE_OFFSET_BITS does, or changing it unconditionally and using symbol
>> versioning for compatibility with old binaries built for 32-bit time_t).
>> Having done that you then have whatever new syscalls may be needed to work
>> with 64-bit time_t on IA32, and can make the new ILP32 ABI not use the old
>> 32-bit time_t syscalls if desired.
>>
>> Of course making LFS (or 64-bit time_t) mandatory doesn't help with those
>> interfaces that hardcode types such as "int" or "long" - you'll still have
>> code that uses fseek rather than fseeko, for example.  If you follow the
>> GNU principles of avoiding arbitrary (or at least inappropriate) limits,
>> there are quite a lot of libc interfaces that can be problematic in
>> extreme cases (large files, strings over 2GB (e.g. regoff_t - glibc bug
>> 5945 - which is probably one of the easier cases), etc.).  It would be
>> good to fix these things, both on the GNU principles and for general
>> robustness (there are probably various security holes related to these
>> issues - integer overflow issues are always tricky to avoid in C, but bad
>> choice of types in APIs certainly doesn't help), but it's quite tricky
>> (lots of core ISO C interfaces are involved) and really needs to be kept
>> separate from the introduction of new ABIs at the level of x86_64 ILP32.
>>
>
> I am checking in ILP32 binutils so that people can play with it.
>

I checked in this patch to avoid using ELF32 relocations on ELF64
inputs and vice verse.


-- 
H.J.
---
2010-12-30  H.J. Lu  

* elf64-x86-64.c (elf_x86_64_relocs_compatible): New.
(elf_backend_relocs_compatible): Defined to
elf_x86_64_relocs_compatible.

diff --git a/bfd/elf64-x86-64.c b/bfd/elf64-x86-64.c
index a50dccc..3dd16ba 100644
--- a/bfd/elf64-x86-64.c
+++ b/bfd/elf64-x86-64.c
@@ -4496,6 +4496,17 @@ elf_x86_64_hash_symbol (struct elf_link_hash_entry *h)
   return _bfd_elf_hash_symbol (h);
 }

+/* Return TRUE iff relocations for INPUT are compatible with OUTPUT. */
+
+static bfd_boolean
+elf_x86_64_relocs_compatible (const bfd_target *input,
+ const bfd_target *output)
+{
+  return ((xvec_get_elf_backend_data (input)->s->elfclass
+  == xvec_get_elf_backend_data (output)->s->elfclass)
+ && _bfd_elf_relocs_compatible (input, output));
+}
+
 static const struct bfd_elf_special_section
   elf_x86_64_special_sections[]=
 {
@@ -4536,7 +4547,7 @@ static const struct bfd_elf_special_section
   elf_x86_64_reloc_name_lookup

 #define elf_backend_adjust_dynamic_symbol   elf_x86_64_adjust_dynamic_symbol
-#define elf_backend_relocs_compatible  _bfd_elf_relocs_compatible
+#define elf_backend_relocs_compatible  elf_x86_64_relocs_compatible
 #define elf_backend_check_relocs   elf_x86_64_check_relocs
 #define elf_backend_copy_indirect_symbolelf_x86_64_copy_indirect_symbol
 #define elf_backend_create_dynamic_sections elf_x86_64_create_dynamic_sections


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H.J. Lu
On Thu, Dec 30, 2010 at 3:00 PM, Joseph S. Myers
 wrote:
> On Thu, 30 Dec 2010, H. Peter Anvin wrote:
>
>> On 12/30/2010 02:21 PM, Robert Millan wrote:
>> > 2010/12/30 Richard Guenther :
>> >> Would be nice if LFS would be mandatory on the new ABI, thus
>> >> off_t being 64bits.
>> >
>> > Please do also consider time_t.
>> >
>>
>> Changing the kernel-facing time_t might completely wreck the reuse of
>> the i386 kernel ABI; I'm not sure.
>
> Before changing time_t for a new ILP32 ABI, you probably want to work out
> what is required - on both the libc and kernel sides - to change it for
> existing 32-bit ABIs (whether providing a new ABI variant like
> _FILE_OFFSET_BITS does, or changing it unconditionally and using symbol
> versioning for compatibility with old binaries built for 32-bit time_t).
> Having done that you then have whatever new syscalls may be needed to work
> with 64-bit time_t on IA32, and can make the new ILP32 ABI not use the old
> 32-bit time_t syscalls if desired.
>
> Of course making LFS (or 64-bit time_t) mandatory doesn't help with those
> interfaces that hardcode types such as "int" or "long" - you'll still have
> code that uses fseek rather than fseeko, for example.  If you follow the
> GNU principles of avoiding arbitrary (or at least inappropriate) limits,
> there are quite a lot of libc interfaces that can be problematic in
> extreme cases (large files, strings over 2GB (e.g. regoff_t - glibc bug
> 5945 - which is probably one of the easier cases), etc.).  It would be
> good to fix these things, both on the GNU principles and for general
> robustness (there are probably various security holes related to these
> issues - integer overflow issues are always tricky to avoid in C, but bad
> choice of types in APIs certainly doesn't help), but it's quite tricky
> (lots of core ISO C interfaces are involved) and really needs to be kept
> separate from the introduction of new ABIs at the level of x86_64 ILP32.
>

I am checking in ILP32 binutils so that people can play with it.

-- 
H.J.


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread Joseph S. Myers
On Thu, 30 Dec 2010, H. Peter Anvin wrote:

> On 12/30/2010 02:21 PM, Robert Millan wrote:
> > 2010/12/30 Richard Guenther :
> >> Would be nice if LFS would be mandatory on the new ABI, thus
> >> off_t being 64bits.
> > 
> > Please do also consider time_t.
> > 
> 
> Changing the kernel-facing time_t might completely wreck the reuse of
> the i386 kernel ABI; I'm not sure.

Before changing time_t for a new ILP32 ABI, you probably want to work out 
what is required - on both the libc and kernel sides - to change it for 
existing 32-bit ABIs (whether providing a new ABI variant like 
_FILE_OFFSET_BITS does, or changing it unconditionally and using symbol 
versioning for compatibility with old binaries built for 32-bit time_t).  
Having done that you then have whatever new syscalls may be needed to work 
with 64-bit time_t on IA32, and can make the new ILP32 ABI not use the old 
32-bit time_t syscalls if desired.

Of course making LFS (or 64-bit time_t) mandatory doesn't help with those 
interfaces that hardcode types such as "int" or "long" - you'll still have 
code that uses fseek rather than fseeko, for example.  If you follow the 
GNU principles of avoiding arbitrary (or at least inappropriate) limits, 
there are quite a lot of libc interfaces that can be problematic in 
extreme cases (large files, strings over 2GB (e.g. regoff_t - glibc bug 
5945 - which is probably one of the easier cases), etc.).  It would be 
good to fix these things, both on the GNU principles and for general 
robustness (there are probably various security holes related to these 
issues - integer overflow issues are always tricky to avoid in C, but bad 
choice of types in APIs certainly doesn't help), but it's quite tricky 
(lots of core ISO C interfaces are involved) and really needs to be kept 
separate from the introduction of new ABIs at the level of x86_64 ILP32.

-- 
Joseph S. Myers
jos...@codesourcery.com


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H. Peter Anvin
On 12/30/2010 02:21 PM, Robert Millan wrote:
> 2010/12/30 Richard Guenther :
>> Would be nice if LFS would be mandatory on the new ABI, thus
>> off_t being 64bits.
> 
> Please do also consider time_t.
> 

Changing the kernel-facing time_t might completely wreck the reuse of
the i386 kernel ABI; I'm not sure.

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.



Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H. Peter Anvin
On 12/30/2010 02:18 PM, Robert Millan wrote:
> 2010/12/30 H.J. Lu :
>> I also have a patch for gcc 4.4 which works on simple codes.
>>
>> H.J.
>> On Thu, Dec 30, 2010 at 1:31 PM, H. Peter Anvin  wrote:
>>> We do have a slightly more extensive patch already implemented.
> 
> Could you make those patches available somewhere?  It'd be
> interesting to play with them.
> 
> Btw, I recommend against 8-byte longs.  In the tests I did in
> 2009, I recall glibc source was extremely unhappy due to
> sizeof(long)==sizeof(void *) assumptions.
> 

Yes, it's ILP32.

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.



Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H.J. Lu
On Thu, Dec 30, 2010 at 2:18 PM, Robert Millan  wrote:
> 2010/12/30 H.J. Lu :
>> I also have a patch for gcc 4.4 which works on simple codes.
>>
>> H.J.
>> On Thu, Dec 30, 2010 at 1:31 PM, H. Peter Anvin  wrote:
>>> We do have a slightly more extensive patch already implemented.
>
> Could you make those patches available somewhere?  It'd be
> interesting to play with them.
>
> Btw, I recommend against 8-byte longs.  In the tests I did in
> 2009, I recall glibc source was extremely unhappy due to
> sizeof(long)==sizeof(void *) assumptions.
>

ILP32 psABI specifies 4byte for long.

-- 
H.J.


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread Robert Millan
2010/12/30 Richard Guenther :
> Would be nice if LFS would be mandatory on the new ABI, thus
> off_t being 64bits.

Please do also consider time_t.

-- 
Robert Millan


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread Robert Millan
2010/12/30 H.J. Lu :
> I also have a patch for gcc 4.4 which works on simple codes.
>
> H.J.
> On Thu, Dec 30, 2010 at 1:31 PM, H. Peter Anvin  wrote:
>> We do have a slightly more extensive patch already implemented.

Could you make those patches available somewhere?  It'd be
interesting to play with them.

Btw, I recommend against 8-byte longs.  In the tests I did in
2009, I recall glibc source was extremely unhappy due to
sizeof(long)==sizeof(void *) assumptions.

-- 
Robert Millan


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H. Peter Anvin
On 12/30/2010 11:57 AM, Jakub Jelinek wrote:
>>
>> Would be nice if LFS would be mandatory on the new ABI, thus
>> off_t being 64bits.
> 
> And avoid ambiguous cases that x86-64 ABI has, e.g. whether
> caller or callee is responsible for sign/zero extension of arguments, to
> avoid the need to sign/zero extend twice, etc.
> 

Ehwhat?  x86-64 is completely unambiguous on that point; the i386 one is
not.

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.



Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H. Peter Anvin
On 12/30/2010 12:39 PM, David Daney wrote:
> 
> Really I don't care one way or the other.  The necessity of syscall
> wrappers is actually probably beneficial to me.  It will create a
> greater future employment demand for people with the necessary skills to
> write them.
> 

Or perhaps automatic generation will actually get implemented.  I wrote
an automatic syscall wrapper generator for klibc; one of the best design
decisions I made for that project.

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.



Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H.J. Lu
I also have a patch for gcc 4.4 which works on simple codes.


H.J.
On Thu, Dec 30, 2010 at 1:31 PM, H. Peter Anvin  wrote:
> We do have a slightly more extensive patch already implemented.
>
> "Robert Millan"  wrote:
>
>>Hi folks,
>>
>>I had this unsubmitted patch in my local filesystem.  It makes Linux
>>detect ELF32 AMD64 binaries and sets a flag to restrict them to
>>32-bit address space.
>>
>>It's not rocket science but can save you some work in case you
>>haven't implemented this already.
>>
>>Best regards
>>
>>--
>>Robert Millan
>
> --
> Sent from my mobile phone.  Please pardon any lack of formatting.
>



-- 
H.J.


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H. Peter Anvin
We do have a slightly more extensive patch already implemented.

"Robert Millan"  wrote:

>Hi folks,
>
>I had this unsubmitted patch in my local filesystem.  It makes Linux
>detect ELF32 AMD64 binaries and sets a flag to restrict them to
>32-bit address space.
>
>It's not rocket science but can save you some work in case you
>haven't implemented this already.
>
>Best regards
>
>-- 
>Robert Millan

-- 
Sent from my mobile phone.  Please pardon any lack of formatting.


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H. Peter Anvin
I believe it covers all cases *relevant for this particular situation* (unlike, 
say, MIPS) and that any deviation is a bug which can and should be fixed.

"David Daney"  wrote:

>On 12/30/2010 12:12 PM, H. Peter Anvin wrote:
>> On 12/30/2010 11:34 AM, David Daney wrote:
>>>
>>> My suggestion:  Since people already spend a great deal of effort
>>> maintaining the existing i386 compatible Linux syscall
>infrastructure,
>>> make your new 32-bit x86-64 Linux syscall ABI identical to the
>existing
>>> i386 syscall ABI.  This means that the psABI must use the same size
>and
>>> alignment rules for in-memory structures as the i386 does.
>>>
>>
>> No, it doesn't.  It just means it need to do so *for the types used
>by
>> the kernel*.  The kernel uses types like __u64, which would indeed
>have
>> to be declared aligned(4).
>>
>
>Some legacy interfaces don't use fixed width types.  There almost 
>certainly are some ioctls that don't use your fancy __u64.
>
>Then there are things like ppoll() that take a pointer to:
>
>struct timespec {
>longtv_sec; /* seconds */
>longtv_nsec;/* nanoseconds */
>};
>
>There are no fields in there that are controlled by __u64 either. 
>Admittedly this case might not differ between the two 32-bit ABIs, but 
>it shows that __u64/__u32 are not universally used in the Linux syscall
>
>ABIs.
>
>If you are happy with potential memory layout differences between the 
>two 32-bit ABIs, then don't specify that they are the same.  But don't 
>claim that use of __u64/__u32 covers all cases.
>
>David Daney

-- 
Sent from my mobile phone.  Please pardon any lack of formatting.


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread Robert Millan
Hi folks,

I had this unsubmitted patch in my local filesystem.  It makes Linux
detect ELF32 AMD64 binaries and sets a flag to restrict them to
32-bit address space.

It's not rocket science but can save you some work in case you
haven't implemented this already.

Best regards

-- 
Robert Millan
diff -Nur linux-2.6-2.6.26-libre2.old/arch/x86/kernel/sys_x86_64.c 
linux-2.6-2.6.26-libre2/arch/x86/kernel/sys_x86_64.c
--- linux-2.6-2.6.26-libre2.old/arch/x86/kernel/sys_x86_64.c2008-07-13 
23:51:29.0 +0200
+++ linux-2.6-2.6.26-libre2/arch/x86/kernel/sys_x86_64.c2009-05-29 
22:57:41.0 +0200
@@ -48,7 +48,7 @@
 static void find_start_end(unsigned long flags, unsigned long *begin,
   unsigned long *end)
 {
-   if (!test_thread_flag(TIF_IA32) && (flags & MAP_32BIT)) {
+   if ((!test_thread_flag(TIF_IA32) && (flags & MAP_32BIT)) || 
test_thread_flag(TIF_AMD32)) {
unsigned long new_begin;
/* This is usually used needed to map code in small
   model, so it needs to be in the first 31bit. Limit
@@ -94,7 +94,7 @@
(!vma || addr + len <= vma->vm_start))
return addr;
}
-   if (((flags & MAP_32BIT) || test_thread_flag(TIF_IA32))
+   if (((flags & MAP_32BIT) || test_thread_flag(TIF_IA32) || 
test_thread_flag(TIF_AMD32))
&& len <= mm->cached_hole_size) {
mm->cached_hole_size = 0;
mm->free_area_cache = begin;
@@ -150,8 +150,8 @@
if (flags & MAP_FIXED)
return addr;
 
-   /* for MAP_32BIT mappings we force the legact mmap base */
-   if (!test_thread_flag(TIF_IA32) && (flags & MAP_32BIT))
+   /* for MAP_32BIT mappings we force the legacy mmap base */
+   if ((!test_thread_flag(TIF_IA32) && (flags & MAP_32BIT)) || 
test_thread_flag(TIF_AMD32))
goto bottomup;
 
/* requesting a specific address */
@@ -232,5 +232,7 @@
up_read(&uts_sem);
if (personality(current->personality) == PER_LINUX32) 
err |= copy_to_user(&name->machine, "i686", 5); 
+   else if (test_thread_flag(TIF_AMD32))
+   err |= copy_to_user(&name->machine, "amd32", 6);
return err ? -EFAULT : 0;
 }
diff -Nur linux-2.6-2.6.26-libre2.old/arch/x86/mm/mmap.c 
linux-2.6-2.6.26-libre2/arch/x86/mm/mmap.c
--- linux-2.6-2.6.26-libre2.old/arch/x86/mm/mmap.c  2008-07-13 
23:51:29.0 +0200
+++ linux-2.6-2.6.26-libre2/arch/x86/mm/mmap.c  2009-05-26 14:30:53.0 
+0200
@@ -53,6 +53,15 @@
return 0;
 }
 
+static int mmap_is_32bit(void)
+{
+   if (mmap_is_ia32 ())
+   return 1;
+   if (test_thread_flag(TIF_AMD32))
+   return 1;
+   return 0;
+}
+
 static int mmap_is_legacy(void)
 {
if (current->personality & ADDR_COMPAT_LAYOUT)
@@ -73,7 +82,7 @@
* 28 bits of randomness in 64bit mmaps, 40 address space bits
*/
if (current->flags & PF_RANDOMIZE) {
-   if (mmap_is_ia32())
+   if (mmap_is_32bit())
rnd = (long)get_random_int() % (1<<8);
else
rnd = (long)(get_random_int() % (1<<28));
diff -Nur linux-2.6-2.6.26-libre2.old/fs/binfmt_elf_amd32.c 
linux-2.6-2.6.26-libre2/fs/binfmt_elf_amd32.c
--- linux-2.6-2.6.26-libre2.old/fs/binfmt_elf_amd32.c   1970-01-01 
01:00:00.0 +0100
+++ linux-2.6-2.6.26-libre2/fs/binfmt_elf_amd32.c   2009-05-26 
14:26:24.0 +0200
@@ -0,0 +1,46 @@
+/*
+ *  Support for loading AMD32 binaries
+ *  Copyright (C) 2009  Robert Millan
+ *
+ *  This program is free software: you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation, either version 2 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program.  If not, see .
+ */
+
+#include 
+
+#undef ELF_CLASS
+#defineELF_CLASS   ELFCLASS32
+
+#undef elfhdr
+#define elfhdr elf32_hdr
+#undef elf_phdr
+#define elf_phdr   elf32_phdr
+#undef elf_note
+#define elf_note   elf32_note
+#undef elf_addr_t
+#define elf_addr_t Elf32_Addr
+
+#undef ELF_PLATFORM
+#defineELF_PLATFORM("amd32")
+
+#undef elf_check_arch
+#defineelf_check_arch(x)   ((x)->e_machine == EM_X86_64 && 
(x)->e_ident[EI_CLASS] == ELFCLASS32)
+
+#undef SET_PERSONALITY
+#define SET_PERSONALITY(ex, ibcs2) do { set_personality_64bit(); 
set_thread_flag(TIF_AMD32); current->personality |= force_personality32;

Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread David Daney

On 12/30/2010 12:28 PM, H.J. Lu wrote:

On Thu, Dec 30, 2010 at 12:27 PM, David Daney  wrote:

On 12/30/2010 12:12 PM, H. Peter Anvin wrote:


On 12/30/2010 11:34 AM, David Daney wrote:


My suggestion:  Since people already spend a great deal of effort
maintaining the existing i386 compatible Linux syscall infrastructure,
make your new 32-bit x86-64 Linux syscall ABI identical to the existing
i386 syscall ABI.  This means that the psABI must use the same size and
alignment rules for in-memory structures as the i386 does.



No, it doesn't.  It just means it need to do so *for the types used by
the kernel*.  The kernel uses types like __u64, which would indeed have
to be declared aligned(4).



Some legacy interfaces don't use fixed width types.  There almost certainly
are some ioctls that don't use your fancy __u64.

Then there are things like ppoll() that take a pointer to:

   struct timespec {
   longtv_sec; /* seconds */
   longtv_nsec;/* nanoseconds */
   };

There are no fields in there that are controlled by __u64 either. Admittedly
this case might not differ between the two 32-bit ABIs, but it shows that
__u64/__u32 are not universally used in the Linux syscall ABIs.

If you are happy with potential memory layout differences between the two
32-bit ABIs, then don't specify that they are the same.  But don't claim
that use of __u64/__u32 covers all cases.


We can put a syscall wrapper to translate it.



Of course you can.

But you are starting with a blank slate, you should be asking yourself 
why you would want to.


What is your objective here?  Is it:

1) Fastest time to a relatively bug free useful system?

or

2) Purity of ABI design?


What would the performance penalty be for identical structure layout 
between the two 32-bit ABIs?


Really I don't care one way or the other.  The necessity of syscall 
wrappers is actually probably beneficial to me.  It will create a 
greater future employment demand for people with the necessary skills to 
write them.


David Daney


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread Joseph S. Myers
On Thu, 30 Dec 2010, Richard Guenther wrote:

> Would be nice if LFS would be mandatory on the new ABI, thus
> off_t being 64bits.

That's certainly abstractly better (and something BSDs do better than 
GNU/Linux).  I expect you'd run into a few complications actually making a 
32-bit glibc port like that (lots of the generic 32-bit code will want to 
build separate 32-bit and 64-bit versions as functions; maybe it will be 
easy to build things with those just ending up as duplicates, but making 
them aliases, as they generally are for 64-bit, could be harder).

-- 
Joseph S. Myers
jos...@codesourcery.com


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H.J. Lu
On Thu, Dec 30, 2010 at 12:27 PM, David Daney  wrote:
> On 12/30/2010 12:12 PM, H. Peter Anvin wrote:
>>
>> On 12/30/2010 11:34 AM, David Daney wrote:
>>>
>>> My suggestion:  Since people already spend a great deal of effort
>>> maintaining the existing i386 compatible Linux syscall infrastructure,
>>> make your new 32-bit x86-64 Linux syscall ABI identical to the existing
>>> i386 syscall ABI.  This means that the psABI must use the same size and
>>> alignment rules for in-memory structures as the i386 does.
>>>
>>
>> No, it doesn't.  It just means it need to do so *for the types used by
>> the kernel*.  The kernel uses types like __u64, which would indeed have
>> to be declared aligned(4).
>>
>
> Some legacy interfaces don't use fixed width types.  There almost certainly
> are some ioctls that don't use your fancy __u64.
>
> Then there are things like ppoll() that take a pointer to:
>
>           struct timespec {
>               long    tv_sec;         /* seconds */
>               long    tv_nsec;        /* nanoseconds */
>           };
>
> There are no fields in there that are controlled by __u64 either. Admittedly
> this case might not differ between the two 32-bit ABIs, but it shows that
> __u64/__u32 are not universally used in the Linux syscall ABIs.
>
> If you are happy with potential memory layout differences between the two
> 32-bit ABIs, then don't specify that they are the same.  But don't claim
> that use of __u64/__u32 covers all cases.

We can put a syscall wrapper to translate it.


-- 
H.J.


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread David Daney

On 12/30/2010 12:12 PM, H. Peter Anvin wrote:

On 12/30/2010 11:34 AM, David Daney wrote:


My suggestion:  Since people already spend a great deal of effort
maintaining the existing i386 compatible Linux syscall infrastructure,
make your new 32-bit x86-64 Linux syscall ABI identical to the existing
i386 syscall ABI.  This means that the psABI must use the same size and
alignment rules for in-memory structures as the i386 does.



No, it doesn't.  It just means it need to do so *for the types used by
the kernel*.  The kernel uses types like __u64, which would indeed have
to be declared aligned(4).



Some legacy interfaces don't use fixed width types.  There almost 
certainly are some ioctls that don't use your fancy __u64.


Then there are things like ppoll() that take a pointer to:

   struct timespec {
   longtv_sec; /* seconds */
   longtv_nsec;/* nanoseconds */
   };

There are no fields in there that are controlled by __u64 either. 
Admittedly this case might not differ between the two 32-bit ABIs, but 
it shows that __u64/__u32 are not universally used in the Linux syscall 
ABIs.


If you are happy with potential memory layout differences between the 
two 32-bit ABIs, then don't specify that they are the same.  But don't 
claim that use of __u64/__u32 covers all cases.


David Daney


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H.J. Lu
On Thu, Dec 30, 2010 at 12:02 PM, H.J. Lu  wrote:
>
> Here is the ILP32 psABI:
>
> http://www.kernel.org/pub/linux/devel/binutils/ilp32/
>

I put my x86-64 psABI changes at:

http://git.kernel.org/?p=devel/binutils/hjl/x86-64-psabi.git;a=summary

Please send me patches to improve the ILP32 psABI.

Thanks.


-- 
H.J.


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H. Peter Anvin
On 12/30/2010 11:34 AM, David Daney wrote:
> 
> My suggestion:  Since people already spend a great deal of effort
> maintaining the existing i386 compatible Linux syscall infrastructure,
> make your new 32-bit x86-64 Linux syscall ABI identical to the existing
> i386 syscall ABI.  This means that the psABI must use the same size and
> alignment rules for in-memory structures as the i386 does.
> 

No, it doesn't.  It just means it need to do so *for the types used by
the kernel*.  The kernel uses types like __u64, which would indeed have
to be declared aligned(4).

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.



Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H. Peter Anvin
On 12/30/2010 11:53 AM, Richard Guenther wrote:
> 
> Would be nice if LFS would be mandatory on the new ABI, thus
> off_t being 64bits.
> 

Yes, although that's a higher-order thing.

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.



Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H.J. Lu
On Thu, Dec 30, 2010 at 11:40 AM, H.J. Lu  wrote:
> On Thu, Dec 30, 2010 at 11:30 AM, Joseph S. Myers
>  wrote:
>> On Thu, 30 Dec 2010, H.J. Lu wrote:
>>
>>> On Thu, Dec 30, 2010 at 10:42 AM, Joseph S. Myers
>>>  wrote:
>>> > On Thu, 30 Dec 2010, H.J. Lu wrote:
>>> >
>>> >> Hi,
>>> >>
>>> >> This patch adds 32bit x86-64 support to binutils. Support in compiler,
>>> >> library and OS is required to use it.  It can be used to implement the
>>> >> new 32bit OS for x86-64.  Any comments?
>>> >
>>> > Do you have a public psABI document?  I think the psABI at the ELF level
>>> > needs to come before the binutils bits, at the function call level needs
>>> > to come before the GCC bits, etc.
>>>
>>> The psABI is the same as x86-64 psABI, except for 32bit ELF instead of
>>> 64bit.
>>
>> I don't think that's an adequate description.  If the "ILP32" name is
>> accurate then it's certainly wrong at the C level since some type sizes
>> are different, with effects in turn on such things as the description of
>> initial stack layout storing argv.  At the ELF level are you saying that
>> each relocation applies to a relocatable field of the same width as for
>> 64-bit (but with the width of the addend being restricted, of course)?  Do
>> any relocations applying to word64 fields need 32-bit variants applying to
>> word32 for the ILP32 ABI?
>
> ILP32 uses small model in x86-64 psABI.
>

Here is the ILP32 psABI:

http://www.kernel.org/pub/linux/devel/binutils/ilp32/


-- 
H.J.


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread Jakub Jelinek
On Thu, Dec 30, 2010 at 08:53:32PM +0100, Richard Guenther wrote:
> > Syscalls sometimes need three different versions in the kernel; sometimes
> > the wrong version gets put in the n32 syscall table.  Special syscall
> > wrappers are often needed in glibc; although for most purposes the glibc
> > port is a 32-bit one, including having separate functions for 32-bit and
> > 64-bit off_t, syscalls tend to need to be called in the 64-bit way (long
> > long values as single arguments, in particular).
> 
> Would be nice if LFS would be mandatory on the new ABI, thus
> off_t being 64bits.

And avoid ambiguous cases that x86-64 ABI has, e.g. whether
caller or callee is responsible for sign/zero extension of arguments, to
avoid the need to sign/zero extend twice, etc.

Jakub


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread Richard Guenther
On Thu, Dec 30, 2010 at 8:30 PM, Joseph S. Myers
 wrote:
> On Thu, 30 Dec 2010, H.J. Lu wrote:
>
>> On Thu, Dec 30, 2010 at 10:42 AM, Joseph S. Myers
>>  wrote:
>> > On Thu, 30 Dec 2010, H.J. Lu wrote:
>> >
>> >> Hi,
>> >>
>> >> This patch adds 32bit x86-64 support to binutils. Support in compiler,
>> >> library and OS is required to use it.  It can be used to implement the
>> >> new 32bit OS for x86-64.  Any comments?
>> >
>> > Do you have a public psABI document?  I think the psABI at the ELF level
>> > needs to come before the binutils bits, at the function call level needs
>> > to come before the GCC bits, etc.
>>
>> The psABI is the same as x86-64 psABI, except for 32bit ELF instead of
>> 64bit.
>
> I don't think that's an adequate description.  If the "ILP32" name is
> accurate then it's certainly wrong at the C level since some type sizes
> are different, with effects in turn on such things as the description of
> initial stack layout storing argv.  At the ELF level are you saying that
> each relocation applies to a relocatable field of the same width as for
> 64-bit (but with the width of the addend being restricted, of course)?  Do
> any relocations applying to word64 fields need 32-bit variants applying to
> word32 for the ILP32 ABI?
>
> The right thing to do would be to go through the ABI sources and prepare a
> patch adding a description of the ILP32 ABI at least place where any
> change is needed.
>
>> > the problems that have plagued the MIPS n32 syscall ABI, which seems like
>> > a similar case?
>>
>> Can you describe MIPS n32 problems?
>
> Syscalls sometimes need three different versions in the kernel; sometimes
> the wrong version gets put in the n32 syscall table.  Special syscall
> wrappers are often needed in glibc; although for most purposes the glibc
> port is a 32-bit one, including having separate functions for 32-bit and
> 64-bit off_t, syscalls tend to need to be called in the 64-bit way (long
> long values as single arguments, in particular).

Would be nice if LFS would be mandatory on the new ABI, thus
off_t being 64bits.

Richard.


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H.J. Lu
On Thu, Dec 30, 2010 at 11:30 AM, Joseph S. Myers
 wrote:
> On Thu, 30 Dec 2010, H.J. Lu wrote:
>
>> On Thu, Dec 30, 2010 at 10:42 AM, Joseph S. Myers
>>  wrote:
>> > On Thu, 30 Dec 2010, H.J. Lu wrote:
>> >
>> >> Hi,
>> >>
>> >> This patch adds 32bit x86-64 support to binutils. Support in compiler,
>> >> library and OS is required to use it.  It can be used to implement the
>> >> new 32bit OS for x86-64.  Any comments?
>> >
>> > Do you have a public psABI document?  I think the psABI at the ELF level
>> > needs to come before the binutils bits, at the function call level needs
>> > to come before the GCC bits, etc.
>>
>> The psABI is the same as x86-64 psABI, except for 32bit ELF instead of
>> 64bit.
>
> I don't think that's an adequate description.  If the "ILP32" name is
> accurate then it's certainly wrong at the C level since some type sizes
> are different, with effects in turn on such things as the description of
> initial stack layout storing argv.  At the ELF level are you saying that
> each relocation applies to a relocatable field of the same width as for
> 64-bit (but with the width of the addend being restricted, of course)?  Do
> any relocations applying to word64 fields need 32-bit variants applying to
> word32 for the ILP32 ABI?

ILP32 uses small model in x86-64 psABI.

> The right thing to do would be to go through the ABI sources and prepare a
> patch adding a description of the ILP32 ABI at least place where any
> change is needed.

That is on my TODO list. I am planning to publish an ILP32 psABI.

>> > the problems that have plagued the MIPS n32 syscall ABI, which seems like
>> > a similar case?
>>
>> Can you describe MIPS n32 problems?
>
> Syscalls sometimes need three different versions in the kernel; sometimes
> the wrong version gets put in the n32 syscall table.  Special syscall
> wrappers are often needed in glibc; although for most purposes the glibc
> port is a 32-bit one, including having separate functions for 32-bit and
> 64-bit off_t, syscalls tend to need to be called in the 64-bit way (long
> long values as single arguments, in particular).
>
>> You are right.  Add ILP32 support to Linux kernel may be tricky.
>> We did some experiment to use IA32 syscall interface for ILP32:
>
> That seems likely to run into the structure layout issues I mentioned
> (long long only being 4-byte aligned for IA32, but 8-byte aligned for
> x86-64 and I presume for the new ABI).
>
>> That means we pass 64bit integers to kernel the same way as ia32.
>
> That has its own problems.  If the C prototype of a system call, at the
> userspace level, has a 64-bit integer argument, it's desirable that the
> argument is passed to the kernel in the same way as it is passed to the C
> function implementing the syscall.  This allows glibc's automatically
> generated syscall wrappers to work.  (Note how fanotify_mark is defined to
> use i:is for 32-bit, i:s for 64-bit - and i:s for MIPS n32
> because a 64-bit argument is passed in a single register for n32.
> Recently added syscalls have had care taken to position 64-bit arguments
> so that this sort of thing does work for 32-bit architectures, rather than
> leaving an unused register for alignment as can arise with bad positioning
> of the 64-bit arguments.)  MIPS n32 still needs special wrappers in
> various cases where generic Linux needs C wrappers (posix_fadvise,
> fallocate, posix_fallocate, for example) but avoids them for syscalls.list
> cases.
>
> If the C ABI for 64-bit integers passes them in single registers (as is
> the natural adaptation of the 64-bit ABI), then such integers should also
> be passed to the kernel in single registers; otherwise you need custom
> wrappers for each affected syscall.  But then you certainly can't use the
> IA32 syscall ABI
>

We will address those issues if we want to support ILP32 on Linux.

-- 
H.J.


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread David Daney

On 12/30/2010 10:59 AM, H.J. Lu wrote:

On Thu, Dec 30, 2010 at 10:42 AM, Joseph S. Myers
  wrote:

On Thu, 30 Dec 2010, H.J. Lu wrote:


Hi,

This patch adds 32bit x86-64 support to binutils. Support in compiler,
library and OS is required to use it.  It can be used to implement the
new 32bit OS for x86-64.  Any comments?


Do you have a public psABI document?  I think the psABI at the ELF level
needs to come before the binutils bits, at the function call level needs
to come before the GCC bits, etc.


The psABI is the same as x86-64 psABI, except for 32bit ELF instead of
64bit.


You appear (judging by the support for Linux targets in the binutils
patch) to envisage Linux support for this ABI.  How do you plan to avoid


I enabled it for Linux so that I can run ILP32 binutils tests on Linux/x86-64.


the problems that have plagued the MIPS n32 syscall ABI, which seems like
a similar case?


Can you describe MIPS n32 problems?



I can.  As Joseph indicated, any syscall that passes data in memory 
(ioctl, {set,get}sockopt, etc) potentially must have a translation done 
between kernel and user ABIs.


Currently this is done in kernel/compat.c fs/compat_binfmt_elf.c and 
fs/compat_ioctl.c as well as a bunch of architecture specific ad hoc 
code.  Look at the change history for those files to see that there is 
an unending flow of bugs being fixed due to this ABI mismatch.


Even today there are many obscure ioctls that don't work on MIPS n32. 
Most of the code works most of the time, but then someone tries 
something new, and BAM! ABI mismatch hits anew.


My suggestion:  Since people already spend a great deal of effort 
maintaining the existing i386 compatible Linux syscall infrastructure, 
make your new 32-bit x86-64 Linux syscall ABI identical to the existing 
i386 syscall ABI.  This means that the psABI must use the same size and 
alignment rules for in-memory structures as the i386 does.


David Daney


Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread Joseph S. Myers
On Thu, 30 Dec 2010, H.J. Lu wrote:

> On Thu, Dec 30, 2010 at 10:42 AM, Joseph S. Myers
>  wrote:
> > On Thu, 30 Dec 2010, H.J. Lu wrote:
> >
> >> Hi,
> >>
> >> This patch adds 32bit x86-64 support to binutils. Support in compiler,
> >> library and OS is required to use it.  It can be used to implement the
> >> new 32bit OS for x86-64.  Any comments?
> >
> > Do you have a public psABI document?  I think the psABI at the ELF level
> > needs to come before the binutils bits, at the function call level needs
> > to come before the GCC bits, etc.
> 
> The psABI is the same as x86-64 psABI, except for 32bit ELF instead of
> 64bit.

I don't think that's an adequate description.  If the "ILP32" name is 
accurate then it's certainly wrong at the C level since some type sizes 
are different, with effects in turn on such things as the description of 
initial stack layout storing argv.  At the ELF level are you saying that 
each relocation applies to a relocatable field of the same width as for 
64-bit (but with the width of the addend being restricted, of course)?  Do 
any relocations applying to word64 fields need 32-bit variants applying to 
word32 for the ILP32 ABI?

The right thing to do would be to go through the ABI sources and prepare a 
patch adding a description of the ILP32 ABI at least place where any 
change is needed.

> > the problems that have plagued the MIPS n32 syscall ABI, which seems like
> > a similar case?
> 
> Can you describe MIPS n32 problems?

Syscalls sometimes need three different versions in the kernel; sometimes 
the wrong version gets put in the n32 syscall table.  Special syscall 
wrappers are often needed in glibc; although for most purposes the glibc 
port is a 32-bit one, including having separate functions for 32-bit and 
64-bit off_t, syscalls tend to need to be called in the 64-bit way (long 
long values as single arguments, in particular).

> You are right.  Add ILP32 support to Linux kernel may be tricky.
> We did some experiment to use IA32 syscall interface for ILP32:

That seems likely to run into the structure layout issues I mentioned 
(long long only being 4-byte aligned for IA32, but 8-byte aligned for 
x86-64 and I presume for the new ABI).

> That means we pass 64bit integers to kernel the same way as ia32.

That has its own problems.  If the C prototype of a system call, at the 
userspace level, has a 64-bit integer argument, it's desirable that the 
argument is passed to the kernel in the same way as it is passed to the C 
function implementing the syscall.  This allows glibc's automatically 
generated syscall wrappers to work.  (Note how fanotify_mark is defined to 
use i:is for 32-bit, i:s for 64-bit - and i:s for MIPS n32 
because a 64-bit argument is passed in a single register for n32.  
Recently added syscalls have had care taken to position 64-bit arguments 
so that this sort of thing does work for 32-bit architectures, rather than 
leaving an unused register for alignment as can arise with bad positioning 
of the 64-bit arguments.)  MIPS n32 still needs special wrappers in 
various cases where generic Linux needs C wrappers (posix_fadvise, 
fallocate, posix_fallocate, for example) but avoids them for syscalls.list 
cases.

If the C ABI for 64-bit integers passes them in single registers (as is 
the natural adaptation of the 64-bit ABI), then such integers should also 
be passed to the kernel in single registers; otherwise you need custom 
wrappers for each affected syscall.  But then you certainly can't use the 
IA32 syscall ABI

-- 
Joseph S. Myers
jos...@codesourcery.com

Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H. Peter Anvin
On 12/30/2010 10:59 AM, H.J. Lu wrote:
> 
>> (If you could arrange for the syscall ABI always to be the same as the
>> existing 64-bit ABI, rather than needing to handle three different syscall
>> ABIs in the kernel, that might be one solution, but it could have its own
>> complexities in ensuring that none of the types whose layout forms part of
>> the kernel/userspace interface have layout differing between n32 and the
>> existing ABI; without any action, structures would tend to get layout
>> similar to that of the existing 32-bit ABI, though quite possibly not the
>> same depending on alignment peculiarities - I'm guessing that the new ABI
>> will use natural alignment - while long long arguments would tend to be
>> passed in a single register, resulting in the complicated hybrid syscall
>> ABI present on MIPS.  If you do have an all-new syscall ABI rather than
>> sharing the existing 64-bit one, I imagine it would need to follow the
>> cut-down set of syscalls for new ports, so involving the issue of how to
>> build glibc for that set of syscalls discussed three months ago in the
>> Tilera context.)
>>
> 
> You are right.  Add ILP32 support to Linux kernel may be tricky.
> We did some experiment to use IA32 syscall interface for ILP32:
> 

The current plan is to simply use the 32-bit kernel ABI more or less
unmodified, although probably with a different entry point using syscall
rather than int 0x80 for performance.  In order for the ABI to map 1:1,
there needs to be a few concessions:

a) 64-bit arguments will need to be split in user space.
b) The Linux kernel  exported __u64 type will need to be declared
   __attribute__((aligned(4))).  This will only affect a handful of
   structures in practice since implicit padding is frowned upon.

(a) could also be fixed by a different syscall dispatch table, it's not
the hard part of this.  We definitely want to avoid adding a different
memory ABI; that's the part that hurts.

-hpa

-- 
H. Peter Anvin, Intel Open Source Technology Center
I work for Intel.  I don't speak on their behalf.



Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread H.J. Lu
On Thu, Dec 30, 2010 at 10:42 AM, Joseph S. Myers
 wrote:
> On Thu, 30 Dec 2010, H.J. Lu wrote:
>
>> Hi,
>>
>> This patch adds 32bit x86-64 support to binutils. Support in compiler,
>> library and OS is required to use it.  It can be used to implement the
>> new 32bit OS for x86-64.  Any comments?
>
> Do you have a public psABI document?  I think the psABI at the ELF level
> needs to come before the binutils bits, at the function call level needs
> to come before the GCC bits, etc.

The psABI is the same as x86-64 psABI, except for 32bit ELF instead of
64bit.

> You appear (judging by the support for Linux targets in the binutils
> patch) to envisage Linux support for this ABI.  How do you plan to avoid

I enabled it for Linux so that I can run ILP32 binutils tests on Linux/x86-64.

> the problems that have plagued the MIPS n32 syscall ABI, which seems like
> a similar case?

Can you describe MIPS n32 problems?

> (If you could arrange for the syscall ABI always to be the same as the
> existing 64-bit ABI, rather than needing to handle three different syscall
> ABIs in the kernel, that might be one solution, but it could have its own
> complexities in ensuring that none of the types whose layout forms part of
> the kernel/userspace interface have layout differing between n32 and the
> existing ABI; without any action, structures would tend to get layout
> similar to that of the existing 32-bit ABI, though quite possibly not the
> same depending on alignment peculiarities - I'm guessing that the new ABI
> will use natural alignment - while long long arguments would tend to be
> passed in a single register, resulting in the complicated hybrid syscall
> ABI present on MIPS.  If you do have an all-new syscall ABI rather than
> sharing the existing 64-bit one, I imagine it would need to follow the
> cut-down set of syscalls for new ports, so involving the issue of how to
> build glibc for that set of syscalls discussed three months ago in the
> Tilera context.)
>

You are right.  Add ILP32 support to Linux kernel may be tricky.
We did some experiment to use IA32 syscall interface for ILP32:

[...@gnu-18 simple]$ make ilp32
make LDFLAGS="-m elf32_x86_64" CFLAGS="-g -O2 -D__i386__ -mn32"
make[1]: Entering directory `/export/gnu/import/svn/psABI/x86-64/ilp32/simple'
/export/build/gnu/gcc-ilp32/build-x86_64-linux/gcc/xgcc
-B/export/build/gnu/gcc-ilp32/build-x86_64-linux/gcc/ -g -O2
-D__i386__ -mn32 -c -D__ASSEMBLY__ start.S
/export/build/gnu/gcc-ilp32/build-x86_64-linux/gcc/xgcc
-B/export/build/gnu/gcc-ilp32/build-x86_64-linux/gcc/ -g -O2
-D__i386__ -mn32   -c -o simple.o simple.c
/export/build/gnu/gcc-ilp32/build-x86_64-linux/gcc/xgcc
-B/export/build/gnu/gcc-ilp32/build-x86_64-linux/gcc/ -g -O2
-D__i386__ -mn32 -c -D__ASSEMBLY__ syscall.S
ld -m elf32_x86_64 -o simple start.o simple.o syscall.o
./simple This is a test.
This is a test.
Hello world
make[1]: Leaving directory `/export/gnu/import/svn/psABI/x86-64/ilp32/simple'
[...@gnu-18 simple]$ file simple
simple: ELF 32-bit LSB executable, x86-64, version 1 (SYSV),
statically linked, not stripped
[...@gnu-18 simple]$

I also have a patch for GDB:

[...@gnu-18 simple]$ ./gdb simple
GNU gdb (GDB) 7.2.50.20101229-cvs
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from
/export/gnu/import/svn/psABI/x86-64/ilp32/simple/simple...done.
(gdb) b main
Breakpoint 1 at 0x4000f0: file simple.c, line 25.
(gdb) r
Starting program: /export/gnu/import/svn/psABI/x86-64/ilp32/simple/simple

Breakpoint 1, main (argc=1, argv=0xd4f4) at simple.c:25
25  {
(gdb) info reg
rax0x0  0
rbx0x0  0
rcx0x0  0
rdx0x0  0
rsi0xd4f4   4294956276
rdi0x1  1
rbp0x0  0x0
rsp0xd4e8   0xd4e8
r8 0x0  0
r9 0x0  0
r100x0  0
r110x200512
r120x0  0
r130x0  0
r140x0  0
r150x0  0
rip0x4000f0 0x4000f0 
eflags 0x286[ PF SF IF ]
cs 0x33 51
ss 0x2b 43
ds 0x2b 43
es 0x2b 43
fs 0x0  0
---Type  to continue, or q  to quit---
gs 0x0  0
(gdb) p $rsp
$1 = (void *) 0xd4e8
(gdb) p $sp
$2 = (void *) 0xd4e8
(gdb) p $pc
$3 = (void (*)()) 0x4000f0 
(gdb)

int syscall(int number, ...);

is implemented with

pushq %rbp
pushq %rbx
mov %edi, %eax  /* Syscall number -> rax.  */
mov %esi, %ebx

Re: RFC: Add 32bit x86-64 support to binutils

2010-12-30 Thread Joseph S. Myers
On Thu, 30 Dec 2010, H.J. Lu wrote:

> Hi,
> 
> This patch adds 32bit x86-64 support to binutils. Support in compiler,
> library and OS is required to use it.  It can be used to implement the
> new 32bit OS for x86-64.  Any comments?

Do you have a public psABI document?  I think the psABI at the ELF level 
needs to come before the binutils bits, at the function call level needs 
to come before the GCC bits, etc.

You appear (judging by the support for Linux targets in the binutils 
patch) to envisage Linux support for this ABI.  How do you plan to avoid 
the problems that have plagued the MIPS n32 syscall ABI, which seems like 
a similar case?

(If you could arrange for the syscall ABI always to be the same as the 
existing 64-bit ABI, rather than needing to handle three different syscall 
ABIs in the kernel, that might be one solution, but it could have its own 
complexities in ensuring that none of the types whose layout forms part of 
the kernel/userspace interface have layout differing between n32 and the 
existing ABI; without any action, structures would tend to get layout 
similar to that of the existing 32-bit ABI, though quite possibly not the 
same depending on alignment peculiarities - I'm guessing that the new ABI 
will use natural alignment - while long long arguments would tend to be 
passed in a single register, resulting in the complicated hybrid syscall 
ABI present on MIPS.  If you do have an all-new syscall ABI rather than 
sharing the existing 64-bit one, I imagine it would need to follow the 
cut-down set of syscalls for new ports, so involving the issue of how to 
build glibc for that set of syscalls discussed three months ago in the 
Tilera context.)

-- 
Joseph S. Myers
jos...@codesourcery.com